In Defence of Book Burning

On the back of CAST 2014 in New York last week, there is revolution in the air in the testing world. While I unfortunately wasn’t at CAST, it appears that James Christie’s talk on software testing standards really got attendees going and prompted the initiation of both a manifesto that rejects a certification/standard driven culture of professional testing, and a petition to oppose the publication and continued developments of the ISO29119 standards for software testing.

If you are a tester, or involved in the development or consumption of software in any way (that’s you, everyone) I urge you to watch James’ talk, read Karen Johnson’s considered post and consider the value of the certifications and standards that the statements oppose, and to judge for yourself whether you think they do and will contribute to the evolution of the profession of software testing, and ultimately the way we test and deliver software.

• The Professional Tester’s Manifesto is available here.
• The petition to oppose the ISO29119 standards is available here.

For what it is worth, I have proudly signed both the manifesto and the petition. I strongly believe that the future of our craft depends not on the standardisation and codification of our complex, cognitive and challenging profession into a one-size-fits-all set of standards and certifications, but in the continued attempt to diversify and broaden the skills and techniques we use to analyse, investigate and communicate quality related information about software.

However, there has already been some resistance to these movements against the predominant certifications and standards.

The Professional Tester website has labelled those in support of these movements “[b]ook burners”, thus comparing a peaceful but impassioned coming together of a community in support of excellence in testing practices to the subversive repression of freedom practiced in Nazi Germany. All because “[e]ffective, generic, documented systematic testing processes and methods impact [those who support the movement’s] ability to depict testing as a mystic art and themselves as its gurus.”

Indeed, another oppositional response to Karen’s aforementioned article I heard today was that “[t]o go through a certification process shows professional commitment … that someone is engaged in personal professional development, of course there are many ways to show this and I applaud them all. As an interviewer I place value on certification in that it shows ‘professional effort’.”

These two comments both seem to suggest that the movement against the prevalent certifications and standards is somehow simultaneously an attempt to mystify testing and discourage “professional effort” toward studying the craft. This, however, could not be further from the truth, and in fact the opposite is true. The movement against these certifications and standards exist because we wish to encourage the broad development of testing skill, rather than limit it to a standard set of so-called best practices.

I responded directly to the second comment quoted above, but in light of the melodramatic allegations of subversive imperialism and book burning, thought that posting my response and thus my position on what I stand for by signing the manifesto and petition might be useful in helping give some additional context to other testers who may not be otherwise aware of the debate around these standards and certifications.

My response to the above quoted comment follows, with minor alterations to tidy it up and generalise it for the purpose of this post:

I agree, as I’m sure all who have signed the manifesto and petition would agree, that effort on behalf of an individual to develop testing skills and knowledge should always be encouraged.

However, I believe the concern with the certifications and standards prevalent in testing* presently is that:

a) the certifications/standards are not being represented and consumed by the industry as simply a pathway toward self-development in testing, but instead as a mandatory signifier of one’s qualification to fulfil the role of professional software tester, and

b) these certifications/standards are actually detrimental to the development of broad and important testing skills and approaches by presenting a limited (and arguably flawed) paradigm of what testing is as “best practice”, which means that even where people are demonstrating “professional effort” and undertaking/adhering to these certifications/standards innocently and with admirable intent, it may harm them as a committed professional tester, rather than help them.

The petition and manifesto are not intended to prevent “professional effort” (quite the opposite!), but to encourage our industry to be more responsible and aware in terms of where that effort is directed, and the negative consequences of allowing our profession and our consumers to place value in certifications and standards that are actually holding back the evolution of our craft.

Delving deeper into a), these certifications and standards are marketed in such a way that they are perceived as a gatekeeper on the pathway into a testing role. We commonly see people and organisations who don’t understand testing mistakenly operate under this assumption too – believing that someone without a certificate is in some way unable and unqualified to test professionally.

The graduates at Assurity, the company I work for, learned about testing from an in depth four week programme run by experienced and skilled practitioners – and demonstrated their ability through multiple practical testing projects and presentations. And yet some clients will not allow them on site without a certification that can be passed after a few short hours of simple cramming to memorise some definitions.

It is not the intent of those consuming these certifications/standards, but of those peddling them that is the problem. They suggest that testing is something that can be learned from a book in a few days, and this creates a real credibility problem for us as testers – it’s no wonder people think that testing is “easy”, and that they can whip up a few business users, give them a template and let them loose as testers when these certifications and standards are the commonly accepted entry criteria to a job as a tester.

Which leads us to b) – I am not fundamentally opposed to the idea of testing certifications or qualifications (though I think due to the nature of the profession, creating a valid one would be difficult and require at least weeks of interactive, practical study and assessment, and even then we would really only be assessing certain skills not some sort of general “testing ability”), it’s just that the prominent certifications and standards that are the target of this attention are promoting a deliberately limited view of what testing is, and what skills/thinking are required to do the job.

So while we should of course encourage testers in the study and development of their craft (and I and many others who have signed the manifesto and petition are actively engaged in that pursuit), that doesn’t mean that studying any old thing will help them. If these same certifications/standards were represented differently, with an acknowledgement that they represent one approach to testing, or one set of skills, and thus imply to the unwary consumer that there is more to learn – or at least consider learning – then they would be less problematic because they wouldn’t actively seek to close off the minds of those who undertake them. But because they are presented as “best practice”, it means that the well-intentioned student of testing may actually be led astray.

*I say “testing”, but I’m aware that I’m not well placed to comment on the value of certifications/standards in some of the more “technical” testing disciplines. They may well be less problematic, but I wager at least some of them have similar limitations.

I would add to this response that those engaged in opposing the currently predominant certifications and standards of testing are in no way doing so to “depict testing as a mystic art” – in fact, many of these people are as active as any in the global testing community in attempting to deconstruct and communicate the cognitive art of testing in a way that makes it accessible to all testers (here I’m thinking most notably of James Bach & Michael Bolton‘s prolific work).

In addition, many of these testers are similarly active in working with other testers the world over to help coach and mentor them, sharing their skills and their expertise for the betterment of the individuals themselves and inspiring communities of testing to challenge and develop themselves, and to continually re-evaluate what “good testing” means. Indeed, James Bach and Anne-Marie Charret (amongst others) regularly coach testers online via Skype as part of this commitment to developing testers.

So a world without certifications and standards for testing is not a world where testing becomes a secret, dark art practiced by candle light and discussed only in harsh, dramatic tones. A world without certifications and standards is a world where variable, valuable testing practices are allowed to flourish and grow, leading to more diverse and effective ways of doing what we do – delivering quality related information; rather than mindlessly applying frameworks and following instructions.