While increasing delivery agility and quality
Every major sector had several dozen software horror stories – with Public Service/Healthcare and Consumer Tech/Retail industries making up over 63% of the failures. The impact ranged from “best-case” scenarios of user dissatisfaction, poor usage rates, and customer loss – to worst-case scenarios of serious security breaches, shattered brand reputations, severe revenue loss, expensive legal complications, and crashing stock prices.
Tricentis unearthed the scope of these failures in its annual Software Fail Watch report. The 2017 report identified 606 recorded software failures, impacting half of the world’s population (3.7 billion people), $1.7 trillion in assets, and 314 companies. And that’s just the incidents that made their way into the news.
The ability to get new functionality to market ahead of the competition – like Bluetooth connected devices, Apple Pay, and facial recognition – can often be the difference between winning and losing.
But in the race to meet customer’s digital demands, many companies are unable to assure optimal performance, functionality, security and quality.
Many of these companies are simply not as mature in their software testing and automation processes as they know they should be; for example, only automating a fraction of their regression testing. We surveyed 100+ customers during last year and the most common reasons we found for this lack of evolution were: An inability to put the latest best practices into action, difficulties putting the right team and skill sets together, and a lack of expertise with new technologies and tools such as Qmetry, Perfecto, Headspin, Tricentis Tosca, Applitools, and many more. Even opensource tools in the market like Selenium, Appium which are widely used, haven’t even been explored, much less taken full advantage by many of these companies. And the faster delivery paces being driven by DevOps and Agile initiatives only exposes those shortcomings further.
In a recent Infostretch survey of test/QA professionals and executives in the US and UK, participants reported their teams being very challenged by the increasing complexity of software testing and growing pressure to ensure seamless processes and integrations. This includes different components of Continuous Integration such
as Version Control tools, Distributed Build Systems, and Artifactory; Continuous Testing elements such as Execution Grids, Containers, Test Case Analytics, and Build Analytics; Continuous Delivery aspects like Cloud, On-Premise, and Appstore deployments; and Logging and Monitoring layers like Log Aggregators, Log Indexers, and Log Analyzers. 50% of the respondents felt that this high degree of complexity caused by increased technological combinations was a major challenge. An equal number said that they did not have sufficient time to test new apps and services before pushing them into production.
While there is a strong understanding that automation can help speed test cycle time and is critical to advancing their DevOps and Agile goals, the majority of organizations polled by Infostretch had automated less than 25% of their testing. In the latest World Quality Report, the average level of overall automation was only 16%. Both surveys highlight a huge opportunity for improvement. But as mentioned earlier, delivery paces are becoming faster, and that will mean more test cases. If organizations don’t figure out, and quickly, that they must automate their existing backlog of manual test cases, they will be stuck with more test cases, less time, and no feasible way to determine the correct gap analysis for future releases – leaving them in the digital dust with low quality product and inferior user experience.
If real progress is to be made, and higher levels of automation are to be achieved in a material way, enterprises must find innovative ways to optimize the complete testing cycles.
This can happen in user experience, functionality, quality, security, performance – or a combination of them all. And when it does, it’s inevitable that declines in market share, revenue and customer loyalty will follow. It might be a slow slide, or a sharp decline, but it will happen, especially where a high-profile software error occurs. Remember Equifax? (see the FTC report here)
The point is companies have to continue to innovate in software testing just as they do in software development. To deliver better products, delivery teams need to be able to test better. Most people are entirely oblivious to how many versions and updates of their most-used apps hit the market weekly. Many auto-update daily, integrate with multiple systems and accounts in the digital and physical world, and all have to operate seamlessly without anyone knowing it. This is just one example of how test and QA organizations need to keep pace in this digital age or app economy. They must look at the entire architecture of their testing cycle, testing tools, testing process, resource skills and how they are organized to enable the best business outcome. There has to be a cultural shift and commitment to continuously improve their processes for faster cycle time and greater operational efficiency Successful organizations achieve Continuous Testing by taking this approach, and not simply looking at speeding up and automating regression cycles to get the desired outcome.
While this all may seem concerning and disheartening to those leading the transformation that many organizations are making, in fact it is meant to do the opposite. While there are some organizations thriving at implementing continuous environments, the fact is that those still making the transformation aren’t alone. And most importantly, there are experts who know how to make this transformation as seamless as possible.
A few of the common misconceptions in continuous testing are what it means, what it encompasses, where to start, and how to make it successful.
The ability to get new software functionality to market ahead of the competition can be the difference between winning and losing. But many companies are unable to assure performance, functionality and quality.
If user experience, functionality, and performance suffer, it’s inevitable that declines in market share, revenue and customer loyalty will follow.
Automation by definition is the technique, method, or system of operation or controlling g process by highly automatic means, as by electronic devices, reduction human intervention to a minimum. The main words here are ‘a process’, which is a series of actions or steps.
Continuous Testing on the other hand is a methodology, which is a system of ways of doing something. So within Continuous Testing we must look at how we do everything from a holistic stance, not just a tactical one. From a testing perspective, that includes code quality, test coverage, performance testing, and what components of the entire testing environment can be done in parallel or need to be further optimized. This means Continuous Testing needs to enhance the testing of software on a broader level through an organization’s approach to testing.
Through the traditional waterfall approach in development, it was easier to know exactly what you had to test and when to find the respective bugs and defects. However, that approach put a higher importance on UI testing, but all layers need to be accounted for in today’s digital landscape – API, component, and unit testing all need equal consideration in the adoption process. This is why there is a need to look at the testing activities from a holistic perspective which originally wasn’t there. Additionally, now development isn’t done in waterfall. It is done to increase speed and success within an Agile methodology. Again, this isn’t a direct process, but a way of reprioritizing items based on ability, importance and potential
success. Those fundamentals of agile development are often what lead to the success of Continuous Testing: reprioritizing testing activities to test what you can as soon as you can by testing in the early stages of development.
While this testing early in the stages of development may seem like an easy adoption – this tends to be an issue with how traditional organizations are set up due to the defacto cultural implications that waterfall provided. Gaining the trust of developers in the initial phases of projects is pivotal to understanding not only their development methodology, but also why they are prioritizing certain aspects over others. This in turn gives testers the ability to provision the correct environments based off what could potentially be delivered in the latest sprint. Once this understanding occurs, testing teams will be able to leverage their expertise to enhance the performance and code quality of an application based on that same feature prioritization. The ability for testing to provide actionable insights earlier in the process creates an invaluable benefit to both development time and consumer experience.
That is why we mentioned earlier there needs to be continuous improvement. By accelerating the feedback cycle of this enhanced environment, we can simultaneously limit business risks and increase the business values associated with any given release. Continuous Testing extends beyond traditional test automation strategies to address the faster pace and increased complexity of current application development and delivery.
Essentially, Continuous Testing demands holistic testing optimization for higher rates of effective testing and that can’t be done without automation. This can only be enabled by making some fundamental changes in the test and QA process, including
Many businesses focus on automating just one or two aspects of the testing ecosystem. There are a lot of tools on the market – many good ones like Applitools, Hexawise, and Mabl– that do just that. But these tools can only do so much on their own. Continuous Testing requires tight integration and coordination between every stage of the test/QA process starting with the definition of Requirements & Expectations and optimizing future test with the added Insights, Analytics and User Feedback.
As a result of this broader, more integrated view, Continuous Testing enables organizations to understand where they can and should automate for the best results, as well as where they shouldn’t. This translates into big cycle-time efficiencies and effectiveness to get desired test coverage while also truly enhancing the end product.
This approach has helped countless personas within the testing and development lifecycles, but their adoptions often come with varying degrees of challenges and hurdles at the beginning of the process.
With so many tools readily available in the market to help different testing phase, it is important to align your testing evolution with your business strategy.
For example, running your entire operation as Cloud Native and having automatic triggers to spin up or down testing environments with absolutely no manual intervention. A great example of what wants to be technically achieved from a testing standpoint, but the business initiative may be more geared to minimizing existing operational costs by optimizing testing repositories increasing UAT automation tools.
Companies and organizations often want “the shiniest and newest thing” first, and to make testing and enabler to your company’s success, understanding that
prioritization of what is needed and what is wanted must occur. Yes, there are major advantages that mature organizations can take advantage, but more often than not – addressing the maturity of a testing organization has to be done first. These aren’t always the shiniest tools, but they often are the most pivotal.
In situations where the applications are already in market and departments need to adopt the agile approach, the best approach is to implement complementing tools in parallel. This can be done in multiple phases with multiple tools, or in one phase with multiple tools. For example in phase of test identification and creation, an organization can both optimize the existing test repository, implement model based testing for new application functionalities, and integrate
auto test case generation and changes to existing functionalities.
In other situations, the implementation and optimization of testing fundamentals like service virtualization and microservices designed to leverage today’s cloud technology are applicable. However, this all depends on the complexity and dependencies that exist within an organization’s digital ecosystem. But regardless of the maturity of an organization’s digital ecosystem, infrastructure, or testing department – there are small steps everyone can take to move towards an more efficient and effecting methodology. The benefits from the increase at an as exponential rate due to these tools being able to generate mass amounts of data that create a feedback loop for better decision making.
They all agree, yet follow it up with ‘when can you have it done by?’. This is when we have to reiterate the fundamentals we established earlier: process versus methodology.
Continuous Testing it isn’t a plug-and-play process, it is a methodology that needs to be pruned, tuned, and enhanced along the longevity of an organization. The most effective way to get the point across is by proclaiming “Well, when do you want to stop ensuring the confidence levels of your coverage? When do you want to stop optimizing your resources and become more cost and effort efficient?”
While this is a blunt approach, the bottom line is while we are moving at the speed of light, the lines between testing being accomplished from a syntaxial context versus and a semantical one blur more and more each day. The only way to incorporate the evolving parameters of what is done correctly from a structural execution standpoint (syntaxial) as well as what is done correctly by the desired end result (semantically), is to implement a Continuous Testing methodology that can adapt in parallel to your application development.
The most impactful way to make those decision is by investing in dynamic, real-time dashboards. Just like it was previously mentioned, all of these tools give us data to make better decisions. But there are so many tools today integrated into your organization and environment – not just in testing. They are all spitting out vast amounts of logs which all need to be aggregated to make those best decisions. For example, once a build gets checked in you want to start the unit testing, and follow-up stages with the right quality gates at each level. That way when a build fails, you will be able to and want to know why so you can take the right. Action.
It’s hard to argue with all the benefits of Continuous Testing, but how do you get there?
Artificial intelligence (AI), including machine learning and BOTs, has emerged as a major disruptive force in the current digital landscape. 43% of the respondents to the Infostretch survey mentioned above see the increased use of AI as inevitable in new applications and devices, as well as software testing processes. These AI-powered test automation solutions, also known as intelligent testing, can help accelerate test automation in multiple ways by:
By incorporating AI into their test automation efforts, businesses can increase their ability to leverage Continuous Testing to overcome time-to-market pressures while also meeting the stringent quality standards needed to achieve desired business outcomes.
Many believe AI will be the core enabler for organizations trying to achieve widespread test automation to fuel Continuous Testing. Nearly 60% of respondents to the Infostretch survey see AI and self-learning technologies as a priority in their QA efforts, and there is a high demand for talent with intelligence-driven QA skills.
At Infostretch we have created a Continuous Testing framework to optimize every aspect of the test/QA process. Our approach is designed to enhance the success of software development and its impact on businesses and their end users.
Infostretch provides the systems, processes, engineering, and automation needed to enhance the agility, efficiency and quality at each testing phase of the software development lifecycle – resulting in accelerated development cycles, faster time to market and increased customer satisfaction and loyalty.
We leverage our extensive expertise in Quality Engineering and integrate the latest technologies to incorporate continuous feedback and analytics into testing activities to ensure business objectives and user requirements are being met even as they evolve.
By looking at all phases of the QA process, Infostretch can help organizations identify the key challenges they face in their software testing, pinpoint the best places to start making improvements, and understand how that will impact and improve other areas.
This data can be used to take your Continuous Testing initiative to the next level.
Combining our expertise in quality engineering and test automation with the latest AI technologies, Infostretch helps organizations leverage these analytics to optimize their testing processes even further. These capabilities include:
For companies that do not want to deal with the significant skill and resource requirements of Continuous Testing, Infostretch offers Test Automation Factory. This is a full or partial- outsourced offering that delivers specific and measurable business outcomes.
Continuous Testing enables organizations to put in place the systems, processes, engineering, and automation needed to truly accelerate cycle time and deliver real business outcomes – higher quality products and services, enhanced customer experience, competitive differentiation, and operational efficiency.
Furthermore, it increases the flexibility of their delivery organization to adjust to changes in the business, an increasingly important capability – no matter what business you are in.
We would love to hear more about your project.
Even a short phone call can help us explain how our solutions can accelerate your mobility, jump start your continuous delivery and help reduce costs. And that’s just for starters, understanding more about your project will enable us to build a solution that fits your objectives, infrastructure and aspirations!
Contact us
