StudioAssist in Katalon Studio

StudioAssist is an AI-powered assistant in Katalon Studio that enables testers to write automated tests more effectively and efficiently. StudioAssist is built based on OpenAI GPT API and capable of intelligent code generation and explanation.

In this proof of concept release, StudioAssist is built directly into Katalon Studio, allowing testers to seamlessly interact with the assistant with two capabilities:

• Code generation: StudioAssist can generate automation test code based on user prompt in the form of code comments.

• Code explanation: You can highlight a code block and ask StudioAssist to explain it.

Important:

• This proof of concept (PoC) is not ready for production use. We recommend using this PoC for evaluation purposes only.
• StudioAssist is available with Katalon Studio version 8.6.7.beta at: katalon-studio/katalon-studio/releases/tag/v8.6.7.beta.
• You need to generate an OpenAI API key. To generate the key, go to: OpenAI API Key.
• To learn more about OpenAI API data policies, refer to API data usage policies.

Enable StudioAssist

As StudioAssist is built with OpenAI API, you need to provide your API key and enable the feature in Katalon Studio settings. Follow these steps:

1. In Katalon Studio, click on the OpenAI icon.
   
   1. You can also go to Katalon Studio > Settings > Katalon > StudioAssist.
2. In the StudioAssist settings, input your OpenAI secret key and token limit.

   

   To learn more about the token limits, refer to the OpenAI rate limits documentation: OpenAI Token Limits.

3. Click Apply.

StudioAssist is now enabled.

Generate code

StudioAssist can generate intelligent automation test code suggestion based on a prompt.

To generate code with StudioAssist, follow these steps:

1. Open a test case in Script mode.
2. Provide your code prompt in terms of code comments, single line or block comment.
   For example, the prompt can be:

   
   /* Write a Katalon Studio test case to perform the following steps.
    * 1. Open browser to the URL stored in G_SiteURL
    * 2. Click the make appointment button
    * 3. Fill in the username and password fields based on the variables
    * 4. Click the login button
    * 5. Verify that the appointment div exists
    * 6. Close the browser
    * */
3. Select the prompt text. Right-click and select StudioAssist > Generate Code. You can also use the hotkeys, Ctrl + Shift + C for Windows or ^ + ⌥ + C for macOS.

   

StudioAssist then generates the test script below the prompt text:

Explain code

You can highlight a code snippet and ask StudioAssist the explain the code.

To generate code explanation, follow these steps:

1. Open a test case with existing test steps in Script mode.
2. Select the the desired code snippet. Right-click and select StudioAssist > Explain Code. You can also use the hotkeys, Ctrl + Shift + E for Windows or ^ + ⌥ + E for macOS.
   

StudioAssist then generates the code explanation below the selected script:

Known limitations

As a proof of concept, StudioAssist comes with some limitations:

• Possible AI hallucinations: StudioAssist might generate code with made-up built-in keywords. You need to double-check the generated code and revise with the valid equivalent.
• Lack of project context: StudioAssist is not aware of test object library, global variables, custom keywords, other test cases, test data, etc. To make sure that the code is generated with proper variables and locators, you should specify them in the prompt.
• Requirements for programming and debugging experience: the lack of context awareness leads to incorrect code generation and explanation. Therefore, you might need programming experience and Katalon Studio keyword knowledge for debugging.

Tips to use StudioAssist

To generate desirable results with StudioAssist, you can provide prompt text in certain format and with specific context. The following are some tips to use StudioAssist effectively.

1. When you need to perform multiple actions, write each action in a bullet point.

   For example, instead of writing a prompt in one paragraph:

   /* Write me a test case that open a browser to a URL, log in to a website, click the login button, input the username and password, then close the browser */
   You can provide a list of actions:

   /* Write me a test case that:
   * 1. Open a browser to a URL
   * 2. Log in to the website
   * 3. Click the login button
   * 4. Input username and password
   * 5. Clock the browser
   */

2. Break complex tasks into smaller tasks and use StudioAssist on each small task.

3. To produce output in a specific format, you can provide the context in your comment.

   For example, to generate a custom keyword, you should provide details about the method template, data type, and possible exceptions:

   /*
    * Write me a clickElement method that receives a test object id as a string and does as follows:
    *
    * 1. Find a web element
    * 2. Click the element
    * 3. If cannot click the element, catch WebElementNotFoundException exception and use KeywordUtil.markFailed to say that you cannot find the element
    * 4. If other errors, catch Exception saying that you failed to click the element with the KeywordUtil.markFailed
    * 5. Otherwise, mark the keyword as succeed
    *
    * The clickElement method should have the following format
    *
    * @Keyword
    * def clickElement(<test object string>) {
    * 	<content of the method>
    * }
   */