Monthly Archives: August 2013

A solution to a non-existent problem

Recently, there was a post made to the Swedish community site “TestZonen” with some information about how the work with the new ISO/IEC/IEEE 29119 standard for software testing is progressing. That post sparked a rather intense series of comments from different members of the Swedish testing community.

My personal stance on the whole thing has in the past been that I “don’t care”. I don’t care for it and I don’t care much about it. If you’re not familiar with the standard, here’s what the organization’s website says the goal of it is:

The aim of ISO/IEC/IEEE 29119 Software Testing is to provide one definitive standard for software testing that defines vocabulary, processes, documentation, techniques and a process assessment model for software testing that can be used within any software development life cycle.

That’s a pretty tall order if you ask me. And words like “definitive” definitely set off some alarm bells in my head. I also think it’s a near impossible goal to reach, at least if one by “standard” mean anything near what I interpret the word to mean. So when I say I don’t care, I partly mean that I think it’s a waste of time and won’t help the craft or the industry, so why bother? After having thought a bit more about it though, I’ve come to realize that I do care a little bit. Why? Because not only do I not think it would help our craft, I also think it can actually hurt the software testing industry. Now, of course you can always opt-out and simply not use the standard in your organization, but I don’t think it’s as easy as that for the industry as a whole.

What follows below is a not-quite-but-pretty-close-to literal translation of what my comments to the article were. It’s a bit long and maybe a bit of a rant, sorry about that. Look, even an introvert can go off on rants sometimes. It started off with me quoting a line from the article where the author gave some examples about what he thought the might be the outcome of the release of this new standard.

One possibility is that it will show up as a requirement, in procurement negotiations or when selecting partners, that you need to be certified according to ISO/IEC/IEEE 29119.

Yes, that’s one of the risks I see with this standard. As if it’s not bad enough that some buyers of testing services today are already throwing in must-have requirements that they don’t know the value (or cost) of. Here comes even more crap that a big number of test professionals must spend time and money getting certified on, or risk becoming disqualified when competing for contracts.

Another risk is that this will be yet another silver bullet or best practice that well-meaning but dangerously uninformed stakeholders will decide to implement rather than taking the time to understand what software development is really all about. Especially since this silver bullet has ISO and IEEE written all over it. How much will the time saved picking this “best practice” over developing a context-appropriate approach end up costing their shareholders in the end…?

Ok, so isn’t it then a good thing that we get a standard so that buyers and stakeholders can feel more secure and know what they’re betting their money on? No, not in this case. Software development is much too context-dependent for that. Just because there’s a standard out there that somebody’s cooked up, doesn’t mean that people will all the sudden become more knowledgeable about software testing. Instead, this will become an opportunity for people to keep ignoring to learn anything about all the choices and judgement calls that go into creating good software. Just use this standard and quality will flow forth from your projects, that’s all you need to know. I don’t see how it would ever be possible to automatically equate using this (or any other) standard with delivering valuable testing and useful information. But unfortunately I believe that’s what a lot of uninformed people will think it means.

What I think is needed instead is a continued dialogue with our stakeholders. We need to gain their understanding for our job and have them realize that creating software is a task filled with creativity, checking and exploration, and that we perform this exercise guided by sound principles and past experiences, but very rarely by standards. And we can of course talk about these principles and explain and defend them as well as use them when we’re asked to estimate amounts of work, or report on progress, or give a status report. But to standardize the practical application of a principle is as unthinkable to me as it is to tell a boxer that he or she will win against any opponent as long as a certain combination of punches is used, regardless of what the opponent is doing. But, teach the same boxer a few principles like how it’s generally a good idea to keep the guard up and keep moving, and their chance to win increases.

The principles and heuristics we use have very little to do with the kind of rigid process maps I find on the standard organization’s website though. I sincerely hope those are old drafts that have been discarded a long time ago. Judging by that material though, I think the best case scenario we’re looking at is that we’ll end up with a new alternative to RUP or some other fairly large and prescriptive process. And since this process will have ISO and IEEE in its name, it will likely be welcomed by many industries and once again they will go back to focusing on being “compliant” rather than making sure to deliver something valuable to customers. So again we’ll be back in that rigid grind where documents are being drawn up just for the heck of it, just because the process says so. Again.

Like many other comments to the article said: What’s the use of this standard? Where’s the value? Who is it really benefiting? To me, it seems like a solution to a non-existent problem!

The article asked if the standard will mean new opportunities or if it will mean that our everyday work will be less open to new ideas. I believe it will mean neither to me. Because I don’t see the opportunities, and I also don’t see that it’s even possible to do good work without being open to new ideas and having the freedom to try them out and then look back on the result and evaluate those ideas. What I see instead is yet another meaningless abbreviation that testers will get hit over the head with time and time again and that we will have to spend time and energy arguing against following, if we want to do valuable testing in our projects and organizations.

The standard is scheduled for release soon. You can choose to ignore it or you can choose to fight it. But please don’t adopt it. You’ll only make life harder for your peers and you’re unlikely to gain anything of value.