Common sense coding Test automation isnt really that automated

Are you looking to reduce testing costs and improve your efficiency throughout the testing process? Automated testing brings people and technology together to simplify and improve application testing but while it may seem straight forward, it isn’t completely automated, at least not in the beginning.

With manual testing taking up as much as one-third of an IT department’s budget, automated testing’s biggest draw is that it can help free up manual testing hours. But it also goes a long way toward improving quality assurance and driving innovation forward.

Consider your adoption capabilities and future goals for testing. Is automation in this space right for you and your business?

What is test automation?

Test automation is the process of using tools to automatically execute pre-defined tests to review and verify software products, such as web applications, and their results.

Using scripted sequences, this method examines the software, reports outcomes, and can compare results with previous tests.

Some main benefits of automating testing include decreasing manual testing and associated costs, increasing accuracy, coverage and reusability as well as improving reporting, bug detection, and efficiency.

What uses can test automation have?

Test automation can be used to perform a wide variety of tasks, including:

• Unit Testing – Isolating a single unit of an application from the rest of the software to test and validate its behaviour.
• Integration Testing – Testing how units are integrated and working as a group to verify how they communicate and behave together.
• Smoke Testing – Quickly verifying whether a new build is stable and meets a base level of functionality to allow for more thorough testing activity or greenlighting a return to normal operations.
• Regression Testing – Thorough validation that a change did not break existing features of functionality of a system.

Limitations to consider

• Lacks the ability to do exploratory testing by responding to unanticipated results or performing unexpected actions.
• It takes considerable time and effort to build. It can often take a comparable amount of time to build the tests as it does to build the code and it will take longer to build a test than it does to run the test manually.
• The test creator can make mistakes and as a result, the test itself may have bugs.
• Tests must be maintained and updated when the code it is testing is modified. This can be inefficient and take time if the code being tested is new or being frequently changed.
• Creating tests for code that doesn’t already exist is challenging and can require significant changes to a team’s process to do test driven development (TDD) or behaviour driven development (BDD) efficiently.

Example of challenges you may run into with test automation

To examine some challenges you may run into when implementing test automation, and how to get ahead of them, let’s look at an example of a team that was using new technology to create a new version of one of their systems to replace an old version that was being discontinued.

As part of the new project, the team wanted to implement as many modern best practices as they could to increase efficiency and quality. This included incorporating automated testing.

In an effort to leverage automation wherever possible, the team chose to allocate their quality assurance (QA) staff to test automation development at the expense of manual verification.

This is where they ran into some problems. Without a strong skillset in TDD/BDD, exclusive reliance on post-development validation test automation was not effective or efficient for quick turnaround on volatile features in active development.

Development would take place over a six-to-eight-week cycle prior to a two-week user acceptance testing (UAT) period and test automation only had access to the completed features for script development at the start of UAT.

Because test automation was inappropriately used as the primary QA, this produced a situation where test automation didn’t have sufficient time to complete scripting work. It also created a conflict between test automation and UAT needs because there was a single QA environment.

Features were then deployed to production where the only QA was performed by the developer who created it and a cursory time-constrained UAT tester. This left test automation spending the next development cycle catching up to develop tests for the previous cycle.

Ultimately, the solution quality did not improve, any savings gained by automating testing were lost from a lack of timely QA on new features and there were no qualified QA staff performing manual testing, increasing the overall risk of introducing new issues with higher severity and impact.

These challenges were caused by a fundamental misunderstanding of where test automation would be most effective, which process changes needed to be implemented to realize the benefits of test automation, and how to reallocate resources to maintain application quality.