Azure DevOps Integration

Katalon TestOps Continurous Integration (CI) is an easier way to execute Katalon Studio tests remotely or schedule remote Katalon Studio execution. See Test Planning Overview for detailed information.

This step-by-step guide shows you how to install the Katalon for Azure DevOps extension and run Katalon projects with Azure DevOps.

Requirements

• An active Katalon Runtime Engine license.

Install the Azure DevOps Extension

To run Katalon projects with Azure DevOps, you first need to install our Katalon for Azure DevOps extension from the Visual Studio Marketplace. This extension helps download, deploy, and execute Katalon Studio on Azure DevOps automatically.

To install the Katalon for Azure DevOps extension, perform the following steps:

1. Navigate to Visual Studio Marketplace and install this plugin: Katalon for Azure DevOps.
2. Click on Get it Free.
   
3. Select your Azure DevOps organization, then click Install.
   
4. Optional: If you are using Azure DevOps Server, click Download and follow instructions as given in the image below.
   

Sample Projects

For Azure DevOps integration, you can refer to our sample project on our GitHub repository: Azure DevOps Extension Samples.

For Linux

The Azure DevOps extension was tested on Ubuntu 16.0.4 and Ubuntu 18.04. You can refer to the sample yml file for Linux on our GitHub repository: azure-pipelines-linux.yml.

For Windows

For Windows, VM resolution is 1024 x 768 by default. If your app can render correctly on this resolution, use this sample script from our GitHub repository: azure-pipelines.yml.

If you need to run your test on a larger resolution, you need to:

• Install the Screen Resolution Utility extension from the Visual Studio Marketplace. To run UI tests on Azure Pipelines, you might need to adjust the screen resolution so that apps render correctly. Learn more about screen resolution on Microsoft document: Setting screen resolution.

• Use this sample script from our GitHub repository: azure-pipelines-windows-srs.yml.

Configure your pipeline

• You have installed the Katalon for Azure DevOps from the Visual Studio Marketplace.

To run your Katalon project on Azure DevOps, you need to configure your pipeline. Do as follows:

1. Open your Katalon project in Azure DevOps.
   If you don't have a project yet, follow this guide on Microsoft documentation to create a new project in Azure DevOps: Create a project in Azure DevOps. Then, upload your Katalon project or clone our demo project from our GitHub repository: Azure DevOps extension samples.
2. In your opened project, go to Repos > Files and click Set up build.
   
   
   
3. In the Configure your pipeline page that displays, select Starter pipeline.
   
   The Review your pipeline YAML section appears. Here, you can refer to our sample script to edit the YAML file. However, with the Katalon for AzureDevOps extension, you can set up your pipeline using assistant in an easier way.
4. Click Show assistant.
   
   The Tasks section appears.

   

5. Click on Search tasks and search for Execute Katalon Studio Tests. Click to choose the Execute Katalon Studio Tests task.
   
6. In the Execute Katalon Studio Tests section, enter the Katalon Studio version you want to run your test with. For example, in the Download Katalon Studio version, enter 8.5.2.
   Optional: If Katalon Studio cannot be downloaded automatically (often due to network conditions), you can input the path to the pre-installed Katalon Studio in the Use pre-installed Katalon Studio field. For example: /var/lib/azureDevops/Katalon_Studio_Linux_64-8.5.2.
7. Enter the command arguments. You can generate the arguments from Katalon Studio by using the command builder. To learn more, see Command Syntax. For example:

   -retry=0 -statusDelay=15 -testSuitePath="Test Suites/New Test Suite" -browserType="Chrome" -apiKey=$(katalon_api_key)
8. Optional: If you are using Linux, you need to take a look at these fields:

   • X11 DISPLAY (For Linux): Leave this field blank.

   • Xvfb-run (For Linux): Configure Xvfb-run following Ubuntu Manuals on Xenial xvfb-run. The function still works if you only change the resolution to 1024x768x24 and leave other options as-is.

9. Click Add to add these inputs to the pipeline script.

The YAML file is updated with a Settings section. You can now save and run your pipeline.

If the script is properly configured, it is successfully executed via Azure DevOps.

For troubleshooting, see Microsoft documentation: Troubleshoot pipeline runs.