[WebUI] Analyze Test Suite Reports and Resolve Errors in Katalon Studio

After Test Suite execution, Katalon Studio generates Test Suite reports that organize the execution logs of the respective Test Cases contained within the suite. You can view Test Suite reports to monitor test coverage and troubleshoot errors quickly.

This tutorial walks you through the steps to analyze a Test Suite report and resolve errors.

In our example, the Test Suite contains a Test Case with the scenario "Order and check out a single product multiple times." This Test Suite uses incorrect test data that result in an execution error. With the Test Suite report generated, we use the interface provided by Katalon Studio to analyze the Test Suite report and resolve the error.

Note:

• You can download the sample project here: Shopping Cart Tests.

Analyze a Test Suite Report

After executing the Test Suite, we view the report directly within Katalon Studio.

Follow these steps:

1. Open the Report History section. Go to Test Explorer > Reports.

   The Report History section organizes Test Suite Reports into separate folders with execution dates in the format YYYYMMDD_HHmmss.

   
   
   

   Here we view the latest execution report by expanding the folder.

   
   
   

2. To open the report, double-click on it.

   In our example, the report is displayed as below.

   
   
   

   Katalon Studio lists the Test Cases in the Test Cases Table with different execution statuses: Passed, Failed, Error, Incomplete, and Skipped.

   Here we have four Test executions; the second Test execution has an Error status.

   
   
   

   The Error status indicates user mistakes in writing Test Cases, such as incorrect syntax, missing test objects, or incorrect test data.

   By contrast, a Failed status would indicate that the application under test (AUT) fails to perform a function expected by the Test Case, such as an unexpected change in the Web user interface.

   Below the Test Cases Table is the Report Summary section.

   The report summary includes information about the Test Suite, such as the Test Suite ID, execution environment, execution time, and execution status.

   
   
   

3. Open the Test Case Log. Select a Test Case in the Test Cases Table and click on the Show Test Case Details button.

   
   
   

   The execution log of the selected Test Case is displayed as follows:

   
   
   

   This log section outlines the steps in the Test Case with their execution statuses.

4. To view the detailed log message, double-click on the step.

   
   
   

   The message is displayed in the Information tab below.

   
   
   

   In our example, step 5 of the Test Case contains an error with the exception com.kms.katalon.core.exception.StepErrorException: org.openqa.selenium.NoSuchElementException. The keyword sample.Checkout.CheckoutShop() in this step failed to locate the element with the XPath normalize-space(text())='Neverland'.

   Since the Test Suite contains only one Test Case, we know that the error in the second Test execution is caused by incorrect test data. Specifically, the second Test execution failed to locate a Web element using the text data 'Neverland'.

   Note:

   • To learn how to troubleshoot common exceptions in Web tests, you can refer to this document: Troubleshoot common exceptions when executing web tests.

Resolve the Error

Because the incorrect test data in our example is associated with an external data file (Excel test data), we navigate to the data file and update it.

Note:

• To learn more about Test Data management, refer to this guide: Manage Test Data.

Follow these steps:

1. Open the Test Suite, then click on the Show Data Binding button.

   
   
   

2. Open the associated data file. In the displayed Test Data section, right-click on the data file in use and select Open Data File.

   In our example, we open the data file "Multiple Checkout."

   
   
   

3. Update the data file.

   In the opened data file, we can see the file contains the incorrect test data.

   
   
   

   We then update the data file with the correct test data.

   
   
   

4. Run the Test Suite and verify the results in the new Test Report.

   
   
   

Note:

• Katalon Studio also supports exporting test reports into different formats, such as HTML, CSV, PDF, and JUnit. For detailed instructions on exporting test reports, refer to this guide: Export reports to other formats .
• For real-time monitoring and better reporting capabilities, consider integrating your project with Katalon TestOps. Learn more about test result reports here: Upload Test Results to Katalon TestOps from Katalon Studio.

See also:

• Test Suite and Test Suite Collection Reports.