In the wonderful world that we know as IT , new terms arise every day. One that has been catching my attention for a while is TestOps. For professionals, a term that covers the load of their daily activities, for many others this blogpost will be a revelation.

A good implementation of TestOps promises great coverage, finds, ensures and delivers important bugs early, ensures that core functionality keeps on working every release, and delivers high quality software on a regular basis with a frictionless experience for the end-user.

Yep, isn’t that what every well-trained marketeer will posit to his customers? Well, this blog is not necessarily aimed at the average customer, but rather at test automation engineers, development teams, team leads and management. Where we have seen enormous amounts of effort and budget invested in DevOps in recent years, the way for TestOps seems to be gradually opening up.

TestOps and Test Automation: a great combination

Testing and test automation has come a long way to become somewhat established, and we still run into a number of shortcomings. Manual testing is good, but most of the time too slow to keep up with development. Test automation is certainly part of the solution.

DevOps often already integrates API and unit testing, but does not yet provide sufficient key functionally testing.

TestOps does provide this, and will have better coverage, in combination with automated, full functional, testing.

A number of articles also mention the use of Artificial Intelligence and Machine Learning in combination with TestOps, but let's write about that in another blog.

Basic criteria of TestOps Test cases

If development teams already run automated tests that meet the basic criteria, they have come a long way. For TestOps, only the most easy and important scenarios should be automated. Try not to spend too much time and resources on difficult and complex test cases, as they are, undeniably, the enemies of reliability. (This time) At first it is better to invest in automating the end-to-end basics. Maximum reliability is key for TestOps test cases, reason is that they will be executed regularly with the critical eye of everyone in the development and operations team on it.

These are the basic criteria for me for TestOps test cases:

1. automate the most used and most important functionalities

2. perform this as often as possible

3. report and fix any problem as soon as possible

4. it must be important enough to wake up the necessary people to fix it in case of failure.

Additional (automated) test runs can still be scheduled from time to time to test exceptions and edge cases, these do not belong in the TestOps story.

Ok, rewind a bit … Let’s go back to the real objective of TestOps: the intention is to run the tests as much as you can, on any possible moment, by as many people and teams as possible. This way problems and bugs are identified early in the process. The TestOps cycle would, ideally, start on developers' machines. But, should at least be executed before every build or deployment. This way test automation is not something that is run coincidentally or afterwards, and definitely not only for the eyes of the test automation engineers, but for the entire team. The only thing that stakeholders want to know is what has broken and is blocking the next release, preferably in real-time. As TestOps mainly tests key functionality, when these tests fail, the development and operations teams should want to know about it immediately. So again, if the test isn’t important enough to wake up somebody in the middle of the night, it shouldn't be included in TestOps. In this context, as in so many other cases, less is more, meaning that it is better to run fewer test cases that are reliable than the opposite. This way you get the maximum value from your test automation investment.