Over my career, my job titles have been Test Analyst, Software Tester, Senior Tester, Test Engineer, and since joining the Guardian: QA Analyst, and more recently QA Automation Engineer. The first four of these roles were essentially the same; I manually tested software, once it had been built by developers, using a variety of techniques to assess its suitability for public, or internal business consumption. Even when I moved onto automated testing, the idea was essentially the same.
Industry-wide, those of us who work in software development who were traditionally called testers have been going through, not an identity crisis, but certainly a naming crisis for years now. Where our colleagues in software development are generally engineers or developers, we don’t really have a universal identity despite largely doing the same thing; testing software for problems that prevent it from being released to the public, or would hinder it from achieving its objectives.
During this time however, we’ve seen a shift in the role testers play in a team, and, as a discipline, we’re only just beginning to adjust to these changes. Sticking with conventional testing methods of backwards and forwards between developers and testers, followed by regression cycles, is not going to allow us to ship features as frequently as we’d like to, as we’d be looking at huge bottlenecks when it comes to testing. So, instead of letting that happen, our teams have taken a step back and have come to the conclusion that what we really need is for our Quality team to be exactly that, focusing on quality, as opposed to just testing.
That probably sounds purely semantic. The conventional wisdom would be that the quality of a product is inversely proportional to bugs, and the focus of testing has generally been to find bugs in their manifested form by testing as exhaustively as reasonably possible, using risk based approaches etc. All software that is released has bugs in it somewhere, that is a fact of life; so the role of the tester has been to find these bugs and advise on their severity. The potential risk of a bug is then appraised by the team, and decided on whether it needs to be fixed.
As a discipline, testers have always looked beyond the traditional testing phase, with the view to allay potential issues during the planning stages of a feature. That idea is not anything new, it’s even on the ISTQB syllabus, but what we’re seeing from the industry is still a focus on testers mainly testing and developers mainly developing. This is due to the identification of the manifested bug as the biggest risk to the quality of our software. This has resulted in us focusing on providing testing as a service, and aligned all our efforts towards the testing function. In the majority of modern development practices the team’s primary focus is not providing a testing function or a development function, it is to deliver great software. Put more simply, we aren’t aiming to have the best test team, we are aiming to have the best product, and that means quality needs to flow through the team.
The distinction between testing and quality is that testing is a reaction to quality problems; something has occurred further up the chain of development that has allowed a problem to occur. This is something that as an industry we have tried to address since the move to Agile-based development approaches. We are emphasising unit testing, continuous integration build systems and peer reviewing pull requests. But, and this is the real step that we made when we starting thinking of quality, rather than testing; what if the conventional bug is not the biggest quality risk to our software?
I can not stress enough that this is not a blanket approach, teams need to appraise their own risks and draw their own conclusions. However, for example, for a modern website with fast deployment tools, CDNs and the ability to almost instantaneously roll back changes, the risk posed by the average bug is actually very small, as very few people will see it before the offending release can be reverted. The bigger risk is that either we don’t know about that issue, or we aren’t able to revert the last release. Therefore one of the biggest quality risks to the team is its monitoring systems and build and deployment tools.
Of course manual tests and automated checks still make a large part of the quality effort, but why do they need to be performed by a separate function? It could be argued that the separate function of testing exists because it takes too much time for developers to test each other’s work. But if the time consuming issues around testing can be taken away, such as deploying new builds to environments, or in the case of automated testing, switching context to a different codebase, or even language, then the blockers to the rest of the team no longer exist.
For example, for our website, manually testing meant launching the same webpage on multiple devices and browsers, which is a lot of hassle, so the QA team built a device wall that is operated from a single point to forward URLs to a number of devices. On our mobile apps team, all of our automated UI tests were in a separate Scala codebase, using Appium. Android apps are written in Java, iOS apps are in Objective C & Swift; so we’ve started to migrate all of our tests over to the native platform tools so the tests are now in the main repositories. Previously, to write a UI test, an Android developer would have to switch IDE, fire up the new repo and tools, and switch to thinking and writing in Scala. Now they just go to the tests package in the Android repo, in their existing IDE and language and start working.
The overall result of the QA function shifting its focus to enabling the team around testing is that not only does the rest of the team assist in the process of testing, but also quality becomes a team-wide concern, rather than the preserve of the QA function. The QA team stop being just gatekeepers to releases, and become champions for quality across the whole team.