Officially published by OpenAI on 30th November 2022, ChatGPT recorded a surge of over 1 million users only in one week after launching. This technology enables genuine conversational interactions, allowing it to answer and explain concepts plainly, reply to follow-up questions, and acknowledge errors, just like a human. In fact, its capabilities have been applied in various aspects, such as writing an article, composing song lyrics, or even generating lines of code in different programming languages.
It’s undeniable that ChatGPT is powerful. But do you think this AI technology can transform software testing? Let’s explore the possibility!
What is ChatGPT?
ChatGPT is a natural language processing chatbot that uses the Generative Pre-Trained Transformer 3 (GPT-3) model to generate human-like text for back-and-forth communication. Since it is trained on a massive amount of text data, the response tone from ChatGPT can be varied based on different scenarios.
Here’re some of the inherent capabilities of this AI chatbot:
- Understand up to 95 natural languages and analyse words for context.
- Use a conversation model for a chat-like interaction with any user.
- Create creative responses, translate languages, summarise and paraphrase content, and analyse the sentiment of the text, all along with details.
- Remember previous questions and answers, leverage human feedback to follow directions and develop a more coherent conversation.
How Is ChatGPT Related to Automation Testing?
Among its capabilities, ChatGPT can automatically produce correctly formatted and relevant code in response to a simple natural language request. The advantage is that it can generate code in a variety of languages and frameworks and employ multiple built-in packages across those languages. For example, this AI chatbot can write Selenium code in many languages and can even be further queried if any adjustments are necessary.
So, can ChatGPT create code for use in test automation? The answer at this point is, “Yes, sort of.” Following are some possible use cases of ChatGPT in automation testing:
- Building automation test cases for different scenarios
- Leveraging CI/CD to create a complex test automation pipeline
- Testing different kinds of applications and software using multiple microservices
- Delivering easy-to-understand code and clear instructions on how to use the developed code.
ChatGPT can help improve ROI in automation testing by improving test coverage and increasing testing efficiency while reducing the time and effort required for test case creation and cutting down costs.
How to Improve Test Automation with ChatGPT?
Here are some of the typical testing activities that CahtGPT can perform:
1. Design All Possible Test Cases
It is possible that high-level test cases may miss critical information that can mislead any QA expert. The AI-driven ChatGPT tool can build test scenarios and enhance its results on the repetition of the same question.
ChatGPT can generate test cases for a variety of requests, including:
- Sample data for a website login form
- Testing ideas for an eCommerce transaction
- Test data for a date picker
Prompt: “Prepare all test cases to test a login form”
ChatGPT:
Additionally, ChatGPT can provide recommendations and best practices for designing test cases. Since it lists down all the necessary test cases that you need to perform, your testers can have ideas for testing even though they aren’t familiar with that functionality.
2. Generate UI Automation Script in Different Languages
As mentioned above, ChatGPT’s capability includes generating test scripts in various programming languages. You can request ChatGPT to generate a test case to open a specific website and perform an action on that with defined frameworks.
Cypress is a JavaScript-based test automation framework for web applications, so I’ll use ChatGPT to create a test script in this language.
Prompt: “Write Cypress/ Javascript code for opening the URL of the site https://kms-solutions.asia/ and Click on the Link “Blogs”
ChatGPT:
The answer from ChatGPT is designed to be straightforward, easy to understand, and flexible, allowing testers to quickly and effortlessly generate automated UI tests. By minimising the need for time-consuming manual testing, ChatGPT frees up engineers to work on higher-level projects while saving the company time and money.
3. Prepare Test Plan for Website Automation Testing
Imagine the following scenario: you want to arrange the test automation for a website named kms-solutions.asia, for which you have created several features. In these cases, you may use ChatGPT to produce a test plan by providing the tool with information about the functional components and how they are utilised.
Prompt: “kms-solutions.asia is a website. Prepare a test plan for an automation test to check the functionality of a form on the homepage: Open kms-solutions.asia. Success if the button “Submit” is clicked and there is user information recorded in the system.”
ChatGPT:
4. Format Data
ChatGPT can be used to format any information and get it ready for any other format. For instance, you need to format the test plan to input data and track bug simply like Jira or Github; in this scenario, you may leverage ChatGPT by prompting further in the same thread:
Prompt: “Prepare test case table to enter into Jira tickets.”
ChatGPT
.jpg?width=838&height=671&name=Group%2048095621%20(2).jpg)
Implementing ChatGPT to Low-code Testing
ChatGPT can be perceived as a powerful tool for writing test cases with no code experience. This AI technology accepts natural language as input, allowing users to write in their natural cadence while still being understood, in contrast to template-based models that often depend on certain language structures or key phrases.
In essence, using ChatGPT, QA teams can generate test automation scripts, classes, and functions and define both positive and negative test cases for complicated scenarios in automated testing.
Additionally, this AI tool can:
- Support multiple programming languages, libraries, and frameworks like Cucumber (based on behaviour-driven development or BDD).
- Whenever there is any change in the element locator or application structure, it can use the Page Object model to update the code.
- Separate the code from the test cases, hence streamlining the maintainability of the test script.
As a low-code testing tool, ChatGPT makes it effortless to update and fix code. For instance, QA experts can request ChatGPT to change the previously created code to correct any bugs. Users can leverage ChatGPT to perform precise changes in the code by using its conversational model.
Limitations of ChatGPT for Automated Testing
Despite tons of potential to become a low-code solution for automation testing, ChatGPT also has some downsides, and it cannot replace the main job of an automtion tester.
- Limited domain knowledge: ChatGPT has been trained on a vast amount of text data, but its knowledge and expertise in specific domains may be limited. This means that ChatGPT may not be able to provide accurate responses to certain test scenarios or situations that require deep domain knowledge.
- Lack of control over the test environment: ChatGPT is an external tool that runs independently of the application being tested. This means it has limited control over the test environment and may not be able to simulate conditions or scenarios that are necessary for some tests. There is the possibility of factual mistakes or disinformation in the training data ended up in the model.
- Inability to identify deprecated methods: The tool repeatedly used the “find_element_by_*” method to locate elements on the screen, even though that method is no longer functional. You can request ChatGPT to update the code with the correct method, but it needs the ability to recognise the issue.
- Lack of flexibility: As a pre-trained model, which means it may not be able to adapt to changes in the application being tested or accommodate custom requirements for specific testing scenarios. This can limit the flexibility and customizability of automated tests.
For example, some websites require both the username and password entered on the same page before submitting, but others have you first input the username, then click a button like “Next” and then input the password. ChatGPT cannot flexibly change the test strategies and will need to be trained to perform the test, which can be done simply by the automation tester.
Although ChatGPT has some drawbacks, it is used regularly to reduce stress related to automation testing and create test cases faster.
Conclusion
ChatGPT is undoubtedly a powerful natural language model with huge potential in automation testing. From conversation to code, it helps simplify the automation testing process, revolutionise overall growth and achieve top quality assurance. Several capabilities from ChatGPT, including advanced language processing, vast knowledge, and others, allow efficient and natural testing experience.
However, as ChatGPT is in the beta phase so it has various limitations and cannot perform complicated tasks. Thus, this tool is believed to be “a future of automation testing”, but now, it is mainly the QA experts’ job.
