Software Testing Automation in an Agile Environment

There is no doubt that software testing automation increases agility. But there is a long process that you have to follow until you can realize the advantages of automation. I will share my experience in tackling software automation processes in our Agile workplace. We learned a lot from our failures with the testing automation work while following Agile processes, and I will discuss the challenges that we encountered. We also improved our practices from the lessons learned.

Our experience with software testing automation

Software testing automation is the process of converting manual test cases into automatically executable procedures. Automation can save a significant amount of time with testing efforts, especially in regression testing. We have been working in a Scrum environment for the past four years, but we never had automation until about three years ago.

The "fun" started when we added the automation task as one of the acceptance criteria for a user story’s Definition of Done (DoD). In the beginning, we all agreed that we would satisfy this criterion, but the efforts were a major failure. Into our sixth month or so, I noticed a pattern emerging: Stories were left in the backlog because automation was not completed. Based on our DoD agreement, we could not "accept" such stories when examined against the exit criteria.

Our challenges.

In reality, when you have the pressure of releasing a feature, automation takes the lowest priority. Our product owner decided to release without automation because we were confident that our manual tests had ferreted out any feature issues or defects. As a result, the automation story was added to the backlog, and the product was released.

The problem with this logic is that automation is not meant for the client. Automation is meant for the internal teams to release software faster. Gradually, the automation backlog was increasing, and we had to address this pile-up somehow.

Despite our software testing automation efforts, we continued to face the following challenges:

  • We did not have a supported platform for automation on which we could continue to add automation tests.
  • Automation testing is not difficult, but many software developers, if they have a choice, prefer not to develop automation for the manual tests. This might not be the case for every work environment, however.
  • There was some confusion about the role of automation. Would QA or the software developer be responsible for doing it?
  • Not all manual tests were candidates for converting to automation tests.

To overcome our challenges, we took one step back and adjusted our DoD. We decided that, until we had the automation framework set up, we would not make automation part of our DoD. Therefore, we invested our time and resources into building the automation team by dedicating the full team for automation. Having one person as the software developer, software tester, and automation developer did not work well. I'm also guessing that it is difficult to find that combination in one person. (If you have found such an individual, you are lucky.)

We tackled the automation backlog. We focused on our pain points and prioritized those tests that were most important to convert to automation. The automation team was solely focused on converting manual tests to automation tests. In parallel, we ran ongoing sessions to learn about automation test development.

We have not achieved the full success of having the person who develops a feature also be the one who completes the testing automation. However, there are cases in which a few developers have begun working with automation. In any case, I feel that we are at least on the right track.

We are working through a few more months to take full advantage of our software testing automation. We are gradually building a solid automation repository and running automated tests for minor releases (e.g., v4.3.2) and main releases (e.g., v4.0). This approach has reduced our testing efforts by at least 25%. Finally I can say that we are taking small steps in the right direction.


Stevbros delivers project management training worldwide, our courses have proven their worldwide acceptance and reputation by being the choice of project management professionals in 168 countries.

Share in

Whoops, looks like something went wrong.