Software testing has gradually evolved in the past few decades. Starting from manual testing, the features and functionality of software, to Integrated Development Environments (IDEs) that integrate automation tools and frameworks into the development cycle. Those tools and frameworks including the ones based on keywords, are data-driven, and are designed to follow Test-Driven Development (TDD) and Behaviour-Driven Development (BDD) methodologies.
Most of these tools and frameworks have evolved in the past decade by migrating to the cloud in varying degrees. The move to the cloud has fundamentally changed how software testing is done. The cloud has made it possible to execute testing throughout the development lifecycle by allowing test environments to scale when needed. Furthering the evolution of testing – and seen in almost all the latest tools and frameworks – is the usage of Artificial Intelligence (AI) in Test Automation.
However, while AI is often talked about, how mature is the usage? The answer isn’t so simple even though AI has been with us for more than two decades. That is because this isn’t I, Robot-level AI technology in Test Automation – so don’t worry, your test cases won’t be gaining sentience any time soon. The maturity of the AI-technology being used in Test Automation is most often a form of the same AI that has been available (to a degree) for more than 20 years.
Aren’t so sure? Well, you have already seen AI applied within our cell phones, you just may not have noticed it yet. For example, like the way we can type something, and it already tells you that there is a mistake in your spelling. Similarly, Google Search will automatically provide predictive suggestions based on your recent and relevant search patterns. This is because there is so much data around what words people misspell as well as what types of searches they are making.
In fact, a general rule is that higher maturity levels of AI can only be achieved with the help of data…lots of data. AI utilizes a combination of computing power to process data and different algorithms to understand or replicate the patterns, suggestions, and/or actions for future data. If the desired output is not correct, with machine learning (ML) technology, AI can learn to self-correct.
From a high-level view, AI is comprised of a cycle: Data – Algorithm – Compute – Predict
Some of the popular AI based test automation tools available are –
Traditional Scriptless vs AI-Based test automation tools
AI-based test automation tools require similar initial development efforts as the traditional IDEs. Compared to traditional scriptless solutions for automation, the major difference found with tools offering AI and ML is that they use dynamic locators for identifying elements. This tool scans the entire DOM, extracts elements, and scores the element attributes with their algorithm to determine what test cases need maintenance, what that maintenance is, and then are able to make that change automatically.
Pros:
- These tools often have a user-friendly UI
- No or minimal technical knowledge required to use these tools
- The tools can be integrated into or already leverage a cloud environment
- These tools have the ability to use AI for self-maintenance of element locators
- They are better than the traditional ‘record-and-play’ manual-to-automation testing procedure
- These tools also provide a form of Test Data Management
Cons:
- The AI in these tools is used primarily for element identification and a type of alternate locator
- Occasionally the tools reduce failures due to minor DOM change, but not consistently
- The tools do not integrate with the test case repository
- There are restrictions that exist when trying to switch from one AI tool to another
- These tools are not an end-to-end solution. Similar to traditional scriptless solutions, the AI tools may encounter challenges in fulfilling all needs (i.e. logical steps) for enterprise software regression
Can the current automation scripting approach be replaced with these AI-Based test automation tools?
AI-based tools are most suited for teams who are using IDEs for “record and play” process to automate manual test cases. This is where someone performs the test case and the tool records that process and then automates it for future use when that test case needs to be ran. Additionally, AI-based tools can also be considered as replacement of other, traditional scriptless automation solutions. For enterprise projects in which the infrastructure of the entire organization comes into the picture, a scriptless approach is not likely to meet all the automation expectations. Few tools enable you to seize control and allow your scripts at a certain level in such test automation.
Since AI-based tools function best in cloud environments, enterprises often experience a challenge found with any cloud-based solution they look at: they may not be ready to expose their assets on/to the cloud. In order to overcome this, enterprises often use a multiple deployment model in the execution environment. This allows those enterprises to still take advantage of some of the benefits that cloud offers, yet few of the AI-based tools in the market today can support this. To elaborate on the execution environment challenges, users might also find themselves struggling to decide between using their own internal execution lab setup or an external cloud service provider such as SauceLabs or Perfecto. Presently, most of the tools’ execution environment is tied up with the solutions, but in the near future, we may see a shift where those tools may also start supporting different execution environment options.
So how do we move forward in test automation?
AI and ML are without a doubt the two big buzzwords circulating around in the testing industry. But the question that we need to ask ourselves is, “are these tools real, or just hype?”. Some aspects of test automation tools have been able to implement AI to replace traditional scriptless solutions with their cloud-based alternatives. However these tools are still evolving, so only time will tell if they will finally fulfill the promise of reliable, easy to maintain test automation for all. But we can all agree that there is a high likelihood that AI-based solutions- with or without scriptless automation – will be the way going forward for test automation.
While using automation frameworks, teams should consider a framework that supports a mechanism for alternate locators and some sort of intelligence, i.e. QMetry Automation Framework already supports the ability to define alternate locators for element and fallback mechanisms through a listener. Additionally, Appium has introduced new features of element-finding-plugins and Test.ai‘s ML element type classifier plugin. Nice Locators strategy is also in the Selenium-4 project plan. Utilities can be developed to capture locators, and custom element finders can be created to support it natively, at least for mobile/Appium, for now. To reduce scripting efforts, you can use a framework that has a built-in step library and also supports the ability to add your custom steps. For example, Apexon’ open source automation framework – QMetry Automation Framework, provides inbuilt steps for web, web-services, and Perfecto-Quantum provides built-in steps for mobile.
Submit the form below to talk to us about how AI can improve your testing process.
Also read: Cost Optimisation for AWS SageMaker in GenAI Real-Time Inference Endpoints
