Category Archives: Columns

A solution to a non-existent problem

Recently, there was a post made to the Swedish community site “TestZonen” with some information about how the work with the new ISO/IEC/IEEE 29119 standard for software testing is progressing. That post sparked a rather intense series of comments from different members of the Swedish testing community.

My personal stance on the whole thing has in the past been that I “don’t care”. I don’t care for it and I don’t care much about it. If you’re not familiar with the standard, here’s what the organization’s website says the goal of it is:

The aim of ISO/IEC/IEEE 29119 Software Testing is to provide one definitive standard for software testing that defines vocabulary, processes, documentation, techniques and a process assessment model for software testing that can be used within any software development life cycle.

That’s a pretty tall order if you ask me. And words like “definitive” definitely set off some alarm bells in my head. I also think it’s a near impossible goal to reach, at least if one by “standard” mean anything near what I interpret the word to mean. So when I say I don’t care, I partly mean that I think it’s a waste of time and won’t help the craft or the industry, so why bother? After having thought a bit more about it though, I’ve come to realize that I do care a little bit. Why? Because not only do I not think it would help our craft, I also think it can actually hurt the software testing industry. Now, of course you can always opt-out and simply not use the standard in your organization, but I don’t think it’s as easy as that for the industry as a whole.

What follows below is a not-quite-but-pretty-close-to literal translation of what my comments to the article were. It’s a bit long and maybe a bit of a rant, sorry about that. Look, even an introvert can go off on rants sometimes. It started off with me quoting a line from the article where the author gave some examples about what he thought the might be the outcome of the release of this new standard.

One possibility is that it will show up as a requirement, in procurement negotiations or when selecting partners, that you need to be certified according to ISO/IEC/IEEE 29119.

Yes, that’s one of the risks I see with this standard. As if it’s not bad enough that some buyers of testing services today are already throwing in must-have requirements that they don’t know the value (or cost) of. Here comes even more crap that a big number of test professionals must spend time and money getting certified on, or risk becoming disqualified when competing for contracts.

Another risk is that this will be yet another silver bullet or best practice that well-meaning but dangerously uninformed stakeholders will decide to implement rather than taking the time to understand what software development is really all about. Especially since this silver bullet has ISO and IEEE written all over it. How much will the time saved picking this “best practice” over developing a context-appropriate approach end up costing their shareholders in the end…?

Ok, so isn’t it then a good thing that we get a standard so that buyers and stakeholders can feel more secure and know what they’re betting their money on? No, not in this case. Software development is much too context-dependent for that. Just because there’s a standard out there that somebody’s cooked up, doesn’t mean that people will all the sudden become more knowledgeable about software testing. Instead, this will become an opportunity for people to keep ignoring to learn anything about all the choices and judgement calls that go into creating good software. Just use this standard and quality will flow forth from your projects, that’s all you need to know. I don’t see how it would ever be possible to automatically equate using this (or any other) standard with delivering valuable testing and useful information. But unfortunately I believe that’s what a lot of uninformed people will think it means.

What I think is needed instead is a continued dialogue with our stakeholders. We need to gain their understanding for our job and have them realize that creating software is a task filled with creativity, checking and exploration, and that we perform this exercise guided by sound principles and past experiences, but very rarely by standards. And we can of course talk about these principles and explain and defend them as well as use them when we’re asked to estimate amounts of work, or report on progress, or give a status report. But to standardize the practical application of a principle is as unthinkable to me as it is to tell a boxer that he or she will win against any opponent as long as a certain combination of punches is used, regardless of what the opponent is doing. But, teach the same boxer a few principles like how it’s generally a good idea to keep the guard up and keep moving, and their chance to win increases.

The principles and heuristics we use have very little to do with the kind of rigid process maps I find on the standard organization’s website though. I sincerely hope those are old drafts that have been discarded a long time ago. Judging by that material though, I think the best case scenario we’re looking at is that we’ll end up with a new alternative to RUP or some other fairly large and prescriptive process. And since this process will have ISO and IEEE in its name, it will likely be welcomed by many industries and once again they will go back to focusing on being “compliant” rather than making sure to deliver something valuable to customers. So again we’ll be back in that rigid grind where documents are being drawn up just for the heck of it, just because the process says so. Again.

Like many other comments to the article said: What’s the use of this standard? Where’s the value? Who is it really benefiting? To me, it seems like a solution to a non-existent problem!

The article asked if the standard will mean new opportunities or if it will mean that our everyday work will be less open to new ideas. I believe it will mean neither to me. Because I don’t see the opportunities, and I also don’t see that it’s even possible to do good work without being open to new ideas and having the freedom to try them out and then look back on the result and evaluate those ideas. What I see instead is yet another meaningless abbreviation that testers will get hit over the head with time and time again and that we will have to spend time and energy arguing against following, if we want to do valuable testing in our projects and organizations.

The standard is scheduled for release soon. You can choose to ignore it or you can choose to fight it. But please don’t adopt it. You’ll only make life harder for your peers and you’re unlikely to gain anything of value.

Do your best with what you get

A while back, I wrote a short response post to one of Huib Schoot’s blog posts on the difference between a mindset tentatively labeled “adaptability” and that of the context-driven tester. I submitted (as did Huib) that the thinking behind the label adaptability was very different from how context-driven testers think.

This post is about addressing another misunderstanding about context-driven testing that I’ve come to fear is worryingly common, namely the notion that for one reason or another, context-driven testers are incompatible with so called “agile testing”. If I were to address all potential sources for such a misunderstanding, this post would turn into a short book, so for now I’ll only focus on the misunderstanding that “context-driven testers work in an exclusively reactive fashion”.

I was reminded of this when I was reading Sigge Birgisson’s blog a couple of days ago. Sigge has recently published a nice series of comparative posts where he compares Agile Testing to a series of other perspectives on testing. The series is not complete yet (I think) and there are hints of a concluding post down the line, but there’s quite a lot of material to go through already. In particular, Sigge’s comparison with context-driven testing has proved thought provoking and has spawned quite a lot of discussion already, which is of course lovely to see. (Way to go, Sigge.)

One of the things discussed refers back to part of the commentary on the context-driven-testing.com website where the the guiding principles of the context-driven school is published in its original form.

At two points in the commentary following the principles, there is a statement that says that: “Ultimately, context-driven testing is about doing the best we can with what we get.” To me, there’s nothing strange or controversial with that commentary, but it can evidently be misinterpreted to include an implicit call to “accept the status quo” or “work only reactively, not proactively”. If that was true, then context-driven testers would indeed not work well in agile context. Fortunately, this is not the case. Of course, I didn’t write the commentary, but let me explain what interpretation I would propose makes more sense.

The “doing the best you can with what you get” commentary is meant to help illustrate one of the most basic context-driven principles: Context is what drives our selection of tools (or approaches, frameworks, process, techniques, etc.). We don’t force the context to comply to our favorite tools. If we enter a context with poorly written requirements, then no competent tester, regardless of label or affiliation, would say: “I can’t do any testing until I get proper requirements”. Instead, we’d do the best we can, given the situation we’re in.  Or put another way, instead of treating every problem as a nails just because you happen to love your hammer, examine the problem and try to find a reasonable way to either (1) solve the problem or (2) find a way to work around the problem until you are able to solve it. Use the WWMD heuristic: What Would MacGyver Do?

Depending on the rest of the context, in our “poor requirements” example that could include executing a couple of survey sessions, having yet another chat with the product owner or customer representatives, talking to other developers, go out hunting for other requirement sources than what we’ve been given (requirements document ≪ requirements!) or performing some claims testing… Basically whatever you need to do in order to deliver valuable information back to your team or your stakeholders. And naturally, we also work in parallel to try to improve the situation, together with our team. After all, what other alternative is there? (Apart from leaving the situation altogether and finding a new job.)

If I were to try to come to any sort of conclusion with this rambling, I would say that identifying yourself as a context-driven tester doesn’t say anything about how active a participant you will be in working with continuous improvement activities in the workplace, or whether you will just accept things as they are or work tirelessly to change them for the better. Neither does it say anything about your feelings about proactive testing activities. In fact, specific practices is not even something that the context-driven principles address. (Because they are principles! The value of any practice depends on its context!) Nevertheless, constantly trying to make a difference in the workplace, trying to get a testing perspective involved as early as possible in a project and striving to make the place we work in more harmonious and effective, is something I think everybody I know advocates, regardless of labels or affiliations.

All in in, while the context-driven community traditionally employs a somewhat different emphasis and/or perspective on testing than the agile community tend to do, they’re not incompatible. And although it has nothing to do with context-driven principles, the last thing a context-driven tester would object to is continuous improvement or proactive thinking about testing and checking.

If you think differently, then please do challenge me: Name a testing practice thought of as traditionally “agile” that a context-driven driven tester would reject out of hand, or even be unjustifiably suspicious against, and tell me why you think that is. If nothing else it should make for a good discussion.

Eulogy for Ola

This past Wednesday, our community was reached by the sad news that our friend Ola Hyltén had passed away. I was out of the country when I first heard the news, and though I’m back home now, I’m still having trouble coming to terms with the fact. I wasn’t planning on writing anything about this at first, and several others have already expressed their feelings better than I could hope to do in blogs, comments and tweets. But now I’m thinking that I might write a few short lines anyway, mostly for my own sake, in hope that it might provide some cathartic effect.

I didn’t know Ola up until a couple of years ago. I had heard of him before that, but it wasn’t until SWET2 that I got to meet him in person. I found him to be a very likable guy who got along well with everybody and who always seemed to be just a second away from saying something funny or burst out laughing himself. After SWET2, I kept running in to Ola every now and then, for instance at a local pub gathering for testers down in Malmö and later at SWET3. However, it was during our time together on the Let’s Test conference committee, leading up to Let’s Test 2012, that I really got to know him. Ola always seemed to be easygoing, even when things were going against him and it was easy, even for an introvert like myself, to slip into long and entertaining conversations with him about everything and nothing.

I considered Ola to be one of the more influential Swedish testers in recent years, and it’s an influence that we as a community will surely miss having around. We’ve lost a friend, a great conversationalist and a valued colleague and I know it will take a lot of time still before that fact truly sinks in for me.

My condolences goes out to Ola’s family and close friends as they go through this difficult time.

More eulogies for Ola can be found here, here, here and here.

I’m a sucker for analogies

I love analogies. I learn a lot from them and I use them a lot myself to teach others about different things. Sure, even a good analogy is not the same as evidence of something, and if  taken too far, analogies can probably do more harm than good (e.g. “The software industry is a lot like the manufacturing industry, because… <insert far fetched similarity of choice>”). However, I find that the main value of analogies is not that they teach us “truths”, but rather that they help us think about problems from different angles, or help illustrate thinking behind new ideas.

I came across such an analogy this morning in a mail list discussion about regression testing. One participant offered a new way of thinking about the perceived problem of keeping old regression tests updated, in this way: “Pause for a moment and ask… why should maintenance of old tests be happening at all? […] To put it another way, why ask old questions again? We don’t give spelling tests to college students […]”

I like that analogy – spelling tests to college students. If our software has matured past a certain point, then why should we go out of our way to keep checking that same old, unchanged functionality in the same way as we’ve done a hundred times before? Still, the point was not “stop asking old questions”, but rather an encouragement to examine our motivations and think about possible alternatives.

A reply in that same thread made a point that their regression tests were more like blood tests than like spelling tests. The analogy there: Just because a patient “passes” a blood test today, doesn’t mean it’s pointless for the physician to draw blood on the patient’s next visit. Even if the process of drawing blood is the same every time, the physician can choose to screen for a single problem, or multiple problems, based on symptoms or claims made by the patient. Sort of like how a tester can follow the same path through a program twice but vary the data.

So what does this teach us about testing? Again, analogies rarely teach us any hard truths, but they serve as useful stimuli and help us think from new angles. I use them as I use any other heuristic methods. So with this spelling test/blood test analogy in mind, I start to  think about the test ideas I have lined up for the coming few days at work. Are most of them going to be like spelling tests and if so, can I still make a good argument for why those would be the best use of my time? Or are there a few ideas in there that could work like blood tests? If so, what qualifies them as such and can I improve their screening capability even further in some way (e.g. vary the data)?

Like I said earlier, I came across this analogy just this morning, which means I’m probably not really done thinking about it myself yet, but I thought it worth sharing nonetheless. Much like cookies, sometimes a half-baked thought is even better than the real thing. Or at least better than no cookie at all. So here it is. And with that analogy, or maybe with this one below, I bid you a good day.

XKCD: Analogies

The little blog that could…

People sometimes start blogs, write a few posts and then they die away. I’ve taken a slightly different route. I started this blog in June of 2007 and has since then not written a single post. Not one… Until now… So now that I’ve decided to resurrect it 5 years later, I’m curious to see if I can keep it alive. I suspect that before to long I’ll start treating it like I treat my Twitter account. That is, long periods of nothing followed by days of super intense posting every now and then (usually when at workshops, conferences or other gatherings).

Some of you finding your way here already know me as a software tester. That’s good, because that’s what I intend to blog about here, so you don’t have to spend time getting to know me a second time around. If you know me as anything other than a software tester then you probably won’t get much out of following me through this forum. But I’ll still be your friend on Facebook, Twitter, LinkedIn, Entaggle or some other social media if you’d like. Or maybe even in *gasp* real life.

That’s it for now. First post done. There will be more to follow. Soon.

Or will there?