Schedule test runs in TestOps

Requirements

• You have created a Local Test Environment with Agent or you have already created a different Test Environment (Docker, Kubernetes, etc.).

• You have created a Git script repository and linked it to your project on TestOps. For more information, see Upload test scripts from the Git repository to TestOps.

Schedule test runs

Follow these steps on how to schedule Test Runs in TestOps.

1. Go to your Project and select the Test Execution tab.

   The Test Run Calendar page appears as shown below:

   

2. Click Schedule Test Run.

   The Schedule Test Run dialog appears as shown below:

   

3. Enter a name for the test run, then select your repository from the Script Repository drop-down.
4. Choose an option:
   • Test Suite: Run test suites created in TestOps or from Katalon Studio (KS).
     Note:

     • Make sure to download a new local agent to execute test suites in the local environment. For more information, see Create a Local Test Environment with an Agent.
     • Make sure Kobiton integration is disabled because you can only execute test suites on desktop applications (browsers). You can toggle Kobiton off in advanced settings. For more information, see Advanced settings.
   • Test Suite Collection: Select the Test Suite Collection directly from KS. Katalon TestOps automatically fetches your Test Suite Collection. For more details, see Override test environments in a test suite collection.
   • Katalon Command: Execute tests with KS. The Katalon commands can be generated from KS in the Katalon Command dialog). For more details, see Command Syntax.
   • Generic Command: Execute tests with other frameworks outside KS (e.g., Pytest).
   Note:

   • If you choose Test Suite or Test Suite Collection, the Visual Testing section is available. You can specify an existing baseline collection to associate it with the scheduled test runs. If you do not specify a baseline collection, TestOps automatically generates a new baseline collection when the test run schedule is executed.

     For more information on visual testing, see TestOps Visual Testing.

5. In the Environment section, click on the drop-down menu and select from the suggested environments.
   1. If you want to select other environments, click on More options .
      The dialog appears as shown below. Choose among Local Test Environment, TestCloud Test Environment, Kubernetes Test Environment, or CircleCI Test Environment.
   2. When you finish, click the Return arrow.
6. Set the Start date for your test, then:
   • To run tests periodically, turn the Repeat toggle on, set the End date and the interval you want to run the tests, then click Schedule.

     

   • To run tests immediately, turn the Repeat toggle off, then click Run.
7. If you want to save the configuration without running tests, click the drop-down arrow, then select Save Configuration.

   

   You can come back another time to run tests manually. See: Execute test run manually.

Advanced settings

You can optimize your configurations in the Advanced settings section of the Schedule Test Run dialog.

When you open the Schedule Test Run dialog, at the bottom left corner, click Advanced settings.

The dialog displays with the following settings:

• Visual Testing: select a Baseline Collection ID or generate a new Baseline Collection to compare with the captured images.

• Katalon Runtime Engine Version: choose a Katalon Runtime Engine version to run your test.

• Execution Mode: choose between Sequential (run one test after another) or Parallel (run tests at the same time). See: Run multiple Test Suites in parallel with Agents.

  Note:

  • You can only run test suites in a sequential mode. If you choose Test Suite, there is no Parallel option.
• Integration:
  • For Xray Test Plan and Xray Push Mechanism: see Link a Katalon test run with an Xray test plan.
  • For Kobiton: switch the Kobiton toggle on to enable the integration, then enter your Kobiton Device ID to run tests on that device. See: How to configure Kobiton integration with Katalon TestOps.

• Custom Field & Tag: assign custom fields and tags to your test run schedule. See: Custom Fields and Tags.

• Release Version: select the release version you want to link your test runs to. See: Manage Test Runs by Release.

• Timeout in Minutes: define the time after which test execution is canceled. Add more minutes to this limit to avoid test executions getting timed out before completion.

After creating the schedule, Katalon TestOps automatically directs you to the Test Run List page.

The Test Run List page shows you a collection of Test Runs with the same configurations (Test Environment, Script Repository).

View test run types in Katalon Studio

• This feature is available from Katalon Studio version 7.9.1 onwards.
• You have enabled TestOps integration in Katalon Studio. See: Katalon Studio Integration.

The Test Run List in TestOps is equivalent to the Test Run Types in Katalon Studio.

Follows these steps:

1. Open your project in Katalon Studio.
2. In the Tests Explorer sidebar, go to TestOps > Test Run Types.
   You can view a summary of Test Run Types that have been configured in TestOps.

   

3. Click on one of the Test Run names. This opens the Test Runs Summary page in Katalon TestOps.
   

You've successfully viewed the test run types.