Pay to speak at conferences

[tl;dr] I will never actively pay to speak at a for-profit conference. Speakers are the only reason a conference get attendees in the first place. Minimizing the cost and energy it takes for a speaker to present at a conference, and offering profit-sharing or honorariums for content like e.g. workshops or tutorials, is the least we can ask for from the larger conference organizers. It’s a matter of common decency and respect for the people who make their events possible. It’s also the socially responsible thing to do. Anything less deserves to be publicly called out and shunned. That said, there are exceptions and nuance to be found, but you’ll have to keep reading for those…

Preface

I love going to conferences. It’s my primary way of getting inspiration and tap in to experiences from other professionals in my field. Blogs are great too, but nothing beats having the opportunity to have a conversation, a discussion, a debate, and exchange ideas in real-time, face-to-face.

I also love organizing conference. I’ve organized quite a number of them during the past few years. My first conference (not counting in-house, employees only conferences) was when we launched Let’s Test back in 2012. A number of Let’s Tests, peer conferences and meetups later, I still very much enjoy creating spaces for the community to meet and share.

Most of the time when I attend a conference, I’m also a speaker at that conference. Coming from a music background, I enjoy being on stage, and the work that I put in to create the content that I’m presenting is a great learning tool for me. If I count everything from the inception of the idea to the finished presentation or workshop, I usually spend several work days getting the material ready, and it goes through many iterations before I’m happy with it. And that’s fine, because as I work on the material, I learn more about the subject matter. Much more than I knew when I drafted the abstract. But it’s definitely an investment. I don’t create content only for the fun of it.

What I mean by “pay to speak”

A few days ago, I had reason to reexamine both my criteria for bringing my own content to a conference, as well as what the policies I should use for the conferences I help organize. My thinking got triggered by my old friend Ilari Henrik Aegerter, who wrote on Twitter:

https://twitter.com/ilarihenrik/status/1055911801827790848

There are different definitions of pay to speak, but in this particular instance, the conference organizer had asked speakers who are part of a company that provides services and who might benefit from the “networking opportunity” a the conference, to pay for a sponsorship package, should their talk be accepted on the program for next year’s conference. Details can be found at Ilari’s blog.

In short, I find the idea of forced sponsorship appalling, morally bankrupt, and down right offensive and disrespectful to speakers, but also to attendees, who pay to go to a conference to hear interesting ideas, not to hear from whoever is willing to pay to get up on stage. What they deserve are good stories, told by their peers. If they go to a conference where people have payed to secure their spot, then the validity and motivation for the message they are delivering will immediately be called into question. Or at least it should be.

Trying to get speakers to pay extra for the privilege of delivering their content that they’ve already spent time and energy on creating, on a stage at a for-profit event, is just down-right wrong. It doesn’t matter how big a platform you as an organizer are providing. Such policies are an abomination, and conferences who practice it need to be called out and shunned. It’s hurting the quality of the conference scene in general, and it’s hurting the collective reputation of honest conference organizers.

Many people would also include conferences that don’t fully reimburse the costs of their speakers, i.e. travel and accommodation, the the pay to speak definition. I fully understand and respect that definition, it makes sense, but when I extend my criticism to these types of conferences, and if I’m to be able to form a somewhat coherent stance for my own policy moving forward, I need to add some more nuance.

An established conference, with corporate backing, with a clear mission to maximize profits (as corporations understandably want to do), definitely need to make sure to reimburse their speakers fully, and preferably pay an honorarium on top. Anything less will deserve criticism and hopefully the marketplace of both speakers and attendees will start to dry out for those types of conferences rapidly as a result. It’s simply bad form to try to make money off other people’s work in that way.

I wouldn’t personally be as quick to critizise first time conferences, non-profits, or conferences that otherwise don’t have a big money backer, if they said they weren’t able to fully reimburse me though. There are definitely conferences that I’ve both presented at, and organized, that would fit in that category. Putting on a conference that isn’t yet a household name, in a big venue that requires a significant pile of non-refundable money up front, and maybe with social media and word of mouth as your only “reliable” marketing channel… It’s risky. It’s hard.

At those conferences, if there is a community upside to them, I wouldn’t hesitate to “sponsor” by carrying some of my own costs, or agree to share a financial risk and have the overall success of the event be reflected in my reimbursement or pay. I’ve done that before and I would do that again. I would expect to see changes over time though, and that as the conference in question becomes established, the guaranteed reimbursements should go up to fully cover expenses.

One final exception is of course also peer conferences and meetups, where the basic premise is different. Peer conferences work on the assumption that everybody cover their own costs and share fixed costs for the event equally between themselves. Meetups’ goal is never for-profit in my experience and so I don’t have any problem supporting them as long as they are focused on learning and sharing instead of being thinly veiled marketing ploys for some sponsor (which rarely, if ever, have been the case with meetups I’ve visited).

The elusive non-monetary value

Some of the bigger conferences will often come right out and say that they instead of cost reimbursement will offer “exposure” to speakers. Sure, theoretically that exposure might hold some value for certain types of speakers (e.g. business owners, independent consultants, or people who for one reason or another get rewarded by their employer to present at conferences). But most of the time and for most people, offering “exposure” is just a load of bullshit used to justify not having to pay their content creators fairly.

I personally don’t find the pure exposure very alluring, even though I’m in that category of people I just mentioned that might benefit from it. There are, however, other intangible advantages that come from presenting at the bigger stages that I would be lying if I didn’t say I do find valuable.

So the question for me, as an independent consultant, is whether I should take a principled stand and reject speaking at conferences who only or mostly offer non-monetary value back to speakers. And I honestly don’t know the answer. There are many solidarity based reasons to actively push back against the practice, as e.g. Cassandra Leung’s very thoughtful and detailed take on the subject shows. And at the same time I rarely find that any given situation, where a speaking offer is on the table, is completely black and white one where a should or should not accept answer readily presents itself when weighing the pros and cons.

I’ve been heavily engaged in the testing community for over 10 years now, and I love to support its growth and well-being whenever I can, but I can not say that I will never again present at one of those less-than-ideal conferences, while at the same time realizing that I’m privileged to be able to say that. What I can say though, is that I will definitely take a more active role in calling out bad behavior by conference organizers, in terms of them e.g. lacking a fair financial compensation models, downplaying diversity or otherwise fail to be inclusive or fair. And I will also increasingly promote the good and fair examples of conferences who either reimburse speakers fully, or who are operating under a non-profit umbrella for the betterment of the software testing craft. Hopefully the market will shift over time if more people become more picky about what sort of events they attend.

Closing thoughts

My thoughts on this subject are evolving even as I write this post. What I’m convinced of though is that it would be preferable if conferences were maximally inclusive and open to as wide an array of viewpoints as possible. That means that unless your aim as a conference organizer is to maximize profit, you have a responsiblity to actively work for that inclusivity and give equal opportunity so that we as attendees at conferences can be sure that the ideas we see on stage are the best available, based solely on their merit, and not affected or limited by anything else.

The only way we can see which ideas are better and progress as a craft, is if the ideas are available to be heard and challenged. Yes, there are other avenues than conferences to get ideas out there, but nothing beats face-to-face conversation.

Finally, holding to principles and showing solidarity is great. But I don’t think that will change the big picture unfortunately. At best we’ll see a shift where the good speakers gravitate toward the good and fair conferences, and the big money grubbing conferences see a drop in content quality, but will still be able to lure the vast majority of people based solely on their financial muscles and reputation (and marketing) of being big and noteworthy. And that won’t move the needle for the craft. So the answer, on top of people speaking out, is to create more competition for them, in addition to the encouraging number of fair conferences that are already out there today.

If anybody wants to make the conference scene a bit more diversified by putting on a socially responsible new brand of conference, I’m more than willing to lend them my experience as a conference organizer, pro bono. Because let’s face it, that’s how real change happens. By starting something new rather than just boycotting the old.

Peer Conference: How can we convince everyone to prioritize testing?

In the evening the day before DevLin 2018, a small band of merry test specialists in the Linköping area gathered for a short peer conference session on the topic: “How can we convince everyone to prioritize testing?”

“Everyone” in this case primarily means people working in other disciplines than testing, e.g. product owners, managers, programmers, requirements analysts, system engineers, and so on. All testers have probably many times experienced the difficulties involved in getting someone else to understand the importance of correctly weighing the need for thorough testing against the demands for quicker releases, more features and faster time to market. With that background, and armed with a few examples to get us on the right track, this peer conferences was ready to start.

To kick things off, James Bach gave a short presentation of his recent and yet-to-be-published work that him and Michael Bolton have done on their updated version of the Agile Testing Quadrants (from 2014), which both had a couple of new additions, but which was also easier to explain than the previous version.

After the presentation, we initiated a k-card facilitated discussion about a broad collection of thoughts and reactions to that presentation and the general topic for the night. These are some of the threads (definitely not all) that I’m pulling from memory:

What does it mean to do deep testing? Is there an implicit level of coverage associated with claiming that you’re doing deep testing? Opinions differed from everywhere between “deep testing has happened when you can assert that you “know” you’ve found all important/significant bugs in a given area” and “deep testing can occur on a very limited set of variables for a given function or quality aspect in a larger scope of mostly shallow testing”. (These are not meant to be exact quotes. Interpretation and emphasis are mine.)

We also talked about combining multiple testing activities from multiple quadrants, e.g. testing designed to answer the question “Did we build what we think we built” together with deep testing designed to reveal and provide “knowledge of every important bug”. While it is normally the case that these two types of testing activities can be done in parallel quite successfully, we still spent a good while discussing contexts where there two may not be suitable to run in parallel and what to do instead. In a context where a lot of big changes are happening rapidly, or where there is general chaos, deep testing might not be an efficient use of resources at that certain point in time. Testers working in that domain might do well to consciously move into a preparation domain to perform activities that help us test more efficiently: testability advocacy, analysis, specification, test data generation, constructing test environments, etc. This type of movement reminded me of the dynamics of Cynefin and would in my mind be a type of movement that one could make both voluntarily or involuntarily depending on the circumstances surrounding the tester.

Another extremely fascinating discussion was on the concept of Critical distance and its relationship with Social distance. From Bach/Bolton: “Critical Distance refers to the difference between one perspective and another. Testing benefits from diverse perspectives. Testing benefits from diverse perspectives. Shallow testing is tractable at a close critical distance, whereas deeper or naturalistic long‐form testing tends to require or create more distance from the builder’s mindset.”

In other words, you want and need a fair bit of critical distance in order to do deep testing, but in order to work well with others and build rapport with the people who built the thing your testing, you want a close social distance. The problem is that critical distance and social distance go hand in hand. They are more or less bungee chorded to each other, which creates a an interesting trade-off. As your critical distance increases, so does your social distance, and vice versa. Decrease social distance, and you risk decreasing your critical distance. On the other hand, a certain amount of social distance is necessary both to be able to gain information about the thing being built, and to not be seen as a socially inept weirdo who no one listens to anyway. It’s all about finding the sweet-spot. (And there are of course exceptions and things that can be done to increase critical distance without negatively impact social distance in the workplace, though maybe not always easily.)

Getting programmers on board with testing is something that many of us have tried in the past, and have fairly good experience with, and as such it was hardly addressed head on as far as I can remember(?). Pairing, sharing test techniques knowledge and discussing the concept and specifics of testability and its benefits for both disciplines are examples of ways to get programmers more susceptible to “testing talk”.

Finally, we spent some time discussing how to get management to understand the importance of testing. This is sometimes a difficult nut to crack. I myself find that it can be valuable and fruitful to talk to management about various ways to look at quality (e.g. quality criteria) and how much of the risk associated with many quality criteria will never be written down in checkable requirements, but must be discovered through exploration and deep testing. It was also pointed out in the group that some domain specific examples and examples of bugs that have been covered lately in the news can also be a good way to get their attention. And a third way, which is easier said than done, is to achieve high credibility with management, which will make it more likely that they will listen to you when you try to raise awareness of the importance of testing.

Achieving credibility can either be done by doing a good job over time, or by doing an exceptionally thorough and excellent job on a single task that has the potential for high visibility, in which case it’s worth going the extra mile in order to be able to cash in those credibility chips later on.

Like I’ve already stated, there were many more topics covered that for the moment escapes my memory, but all-in-all, this evening was for me an awesome example of how much value can be squeezed out of only a few (~3) hours when a small peer group sit down to discuss big topics with a lot of passion. Good fun and great company too. I will definitely try to help schedule these types of sit-downs more often.

Thank you to all participants for a great evening, and a special thank you to to Agnetha and Erik for co-organizing the evening together with me, and to Rebecca and Morgan for providing the room, and to James Bach for joining us while in town.

Credit for the contents of this post belongs to the contributors of this peer conference:  Johan Jonasson, Morgan Filipsson, Rebecca Källsten, Erik Brickarp, Agnetha Bennstam, Magnus Karlsson, Anders Elm, James Bach & Martin Gladh

Report from QA Fest 2018

On September 21-22 2018, I was invited to speak at QA Fest 2018 in Kiev, Ukraine. I presented a talk titled “The Secret to Successful Testing”, which focused both on the importance of having a multi-tier test strategy with a clear division of responsibilities, as well as on how to begin to implement such a strategy in a team or organization, while at the same time dealing with some of the human challenges and quirks that sometimes get in the way of adopting new ideas and changes.

The Conference

The 2018 edition of QA Fest was the conference’s 5th anniversary. The team responsible also has experience running a wide variety of “Fest” conferences (DevOps Fest, .NET Fest, etc.), which was evident in the level of professionalism and attention to detail that the Fest crew showed us speakers, and I’m sure attendees as well. Both pre-conference arragements as well as stage equipment, sound management, lighting, video recording and the overall venue experience were all great.

I particularly liked the idea of an Expert’s Corner, where selected speakers took turns being available for intimate conversaions and Q&A in small groups. The only critisism I can offer was that the space was a bit small and elongated in a way that made it difficult for more than 5-6 people to engage simultaneously with the Expert. But the idea of encouraging conferring at a conference in that way is commendable, which also I’m sure lowers the bar e.g. for people who are new to conferences to engage with speakers, and who might otherwise hesitate to approach us out on the floor.

On the first night of the conference, the organizers threw a big celebration party, which unfortunately was only open until 9 p.m. (venue restrictions), but which nontheless very much engaged the people there and had something for everyone: Board games, stage performances, karaoke, drinks and conversations. Personally, I took the opportunity to both sit down and talk with my old friend Michael Bolton, who gave two separate talks during the conference, and also to connect with a few new friends in the local testing community as well as visitors from outside of Ukraine (special shout out to Roman, Serge and Erkki, though many more engaged with us throughout the evening).

All in all, a very well-managed conference with I think between 700-800 participants, and an expo area with companies that seemed to want to engage in fun and interactive ways with the participants, rather than just do suit-and-tie marketing. Much appreciated.

The only drawback during the conference was that very few speakers had chosen to give their talks in English, which limited the number of takeaways for people from outside the Ukrainian/Russian language group. Not to say that’s necessarily a bad decision though, considering how many people from the region showed up to consume a for them very accessible program. But having a bit more English material on the program (e.g. one full track) would allow for a more diverse crowd of people from other countries to attend, which would give all attendees access to a wider range of experiences and contexts to draw inspiration from. That said, if you’re a speaker looking to try a new conference, I can highly recommend submitting to QA Fest 2019. You will be supremely well looked after.

The City

I had a very brief stay in Kiev, but I did manage to squeeze out a few hours to go out and experience the city in the evenings. It’s fairly easy to get around Kiev, either by subway or by Über and although I wasn’t really looking for them, I got a glimpse of both the Bohdan Khmelnytsky Monument and the Saint Sophia Cathedral (located in close proximity of each other).

I was struck by the stunning architecture all around the city, as well as the attention to detail placed even in many mundane objects. I could definitely imagine coming back here as a tourist and take my time experiencing the city with more care another time.

And if you happen to find yourself in Kiev, do give the Khinkalnaya Gogi restaurant on Leo Tolstoy Street a try. Fantastic Georgian food in a very nice setting, with friendly staff and good atmosphere. No khinkali – no party! 🙂

Summary

Thank you Dasha, Marina, Kate and Misha, and the rest of the Fest crew for your help with all the arrangements and guidance, and congratulations on a great conference. Looking forward to next time.

I’ll be writing more in depth about some of the topics in my presentation in separate blog posts here shortly. Stay tuned.

Calling on the testing community

Yesterday there was a tweet sent out by Kristoffer Nordström, a fellow tester in our great community of software testing:

Needless to say, Kristoffer and his family are going through hell at the moment and I hope that we as a community can come together and show them our support in a time when it’s very much needed.

Kristoffer’s daughter is now in need of a special treatment in order to have a chance to survive. Those of you already know Kristoffer know that he’s a driving force in our testing community and someone who would readily help others in need.

I urge you to help in any way you can, by spreading the word about the fundraiser that Kristoffer has created at YouCaring.com to collect donations to be able to pay for the treatment needed, and also of course by donating if you are able.

The fundraising goal is €110,000 which will cover the initial operation cost and the cost of some post operation treatments. Should any funds remain after the treatment is completed, they will be donated to DIPG (Diffuse Pons Glioma) research initiatives and thus helping other children in need. If you are willing to donate, any amount helps, even if it’s only a €1.

Let’s help our friend and his family.

Update: Through an amazing display of solidarity, Kristoffer’s family, friends and professional community came together to meet the funding goal within the first 24 or so hours of the campaign! For me personally, this has restored my faith in humanity, and being a father of young children myself I was very moved to see the outpouring of support from all over the world. Please continue to support the campaign though, both since there’s no way of knowing for sure how much money will be needed and also because, as previously stated, any surplus funds will be donated to advancing the science and treatment of this terrible disease.

The business value of testing

The business value of testing is directly correlated to testers’ ability to provide quality related information that either increase the chance of higher revenue or decrease the risk of higher costs for the company.

An important implication of this is that the higher revenue or lower cost must be greater than the cost related to the testing effort that provided the information in the first place. That can be a slightly uncomfortable insight for testers to make, but it’s an essential thing to keep in mind when we try to determine whether we should stop testing or not.

A more positive way to frame it is that we must continuously ask ourselves if there’s something more important that we could be doing at any given time, i.e. we must be aware of the opportunity cost of testing. Ask: Why am I testing? What kind of information am I looking for (a.k.a. information objective)? Is there potentially more valuable information to be found by modifying the way I’m testing or by testing some other aspect of the product?

From time to time we must also realize that our curiosity as testers will never be completely satisfied and we’ll more likely than not be forced to stop testing and ship the product before we’re comfortable with doing so. That feeling is of course a positive thing. It should feel a bit uncomfortable for testers to ship. If you’re completely comfortable with shipping the product then you’ve probably spent too much time testing in relation to the business value you’ve produced, or you just don’t care. Either way, I wouldn’t want to work with a tester who didn’t feel some sort of internal resistance when confronted by the PM calling “time’s up!”.

This is my first attempt to lower my personal bar for how long or thought through a blog post needs to be. It came about in a conversation about business value on the Swedish testing community’s Slack team http://testsverige.se/ and this blog post is essentially an exact copy what I wrote on Slack in that discussion. Anyway, I hope this has some value to somebody. My goal with this experiment is to get better at not preventing myself from publishing half-baked thoughts, which I regularly do (the preventing bit), which results in nothing getting published ever… A lot of you can probably relate to that feeling…? 🙂

Listen up and read

In the technology adoption lifecycle, I usually fall somewhere in between the early adopters and early majority. Every now and then though, I find myself among the last of the laggards. Case in point: audiobooks. I’ve discovered their usefulness just a couple of decades after mostly everyone else. Turns out, audiobooks have proved themselves a highly efficient way of getting through the pile of shame that is my ever increasing “to read”-list.

And grown it has… since the birth of my first child little over three and a half years ago I’ve not read a single book. Not even one. I started reading a few, but I soon ran low on time, and after collecting dust for a couple of weeks, it became apparent over and over again that I would probably have to start over if I wanted to pick them up again. After becoming a father of two, time has become an even more precious commodity.

That’s why this thing with listening to books excites me, and why I want to tell more people about it (even though I’m probably among the last people to the party). I’m sure I’m not the only one out there who struggles to find the time to sit down and do some serious reading.

I got started after listening to one of my favorite podcasts: Dan Carlin’s Hardcore History. He (and many other podcasters) is sponsored by Audible.com, and since I didn’t know anything about audiobooks, and because I could support Dan by signing up through his website, that seemed like as good a place as any to try things out at. I downloaded the Android Audible app, created an account, and found a few books to start building my library.

In one and a half years I’ve now gotten through 37 books, clocking in at roughly 300 hours or the equivalent of somewhere around 11250 pages. That’s more than zero. Some would say a lot more than zero. (I would.)

Since I can “read” these while doing other monotonous things (mowing the lawn, cooking, doing dishes…), I get plenty more opportunities to read compared to if I would be forced to sit down in a quiet space in order to get some book time. This in turn has afforded me the opportunity to go beyond books that I would normally read and dive into a few classic books that I would otherwise never find the time for, like the Thomas Paine books (“Rights of Man”, “Common Sense” and “The Age of Reason”) which I’ve been eyeing for quite some time, or John Stuart Mill’s excellent text “On Liberty” as well things from the opposite end of the liberty/humanist spectrum, e.g. what seemed to be a pretty good translation of “Mein Kampf” which was both fascinating and frightening. I’ve also fulfilled a promise to my wife to read “Brain Rules for Baby” by John Medina, which turned out to be quite insightful.

I also very much enjoyed the much more political and autobiographical books by Maajid Nawaz and the extraordinary Ayaan Hirsi Ali, and of course the insightful books by one of my personal heroes, Sam Harris, who seems to move with ease between a wide range of topics like religion, morality, politics and secular spirituality. Very much looking forward to his upcoming book on artificial intelligence.

For those of you who are working in software testing and/or are entrepreneurs, I especially recommend the fairly short books by Seth Godin in the list below, as well as “The Challenger Sale”, “Your Brain at Work”, “The Power of Habit” and also of course the Malcom Gladwell books. I found I could both digest and retain them all fairly well in this format.

My current audiobook library of books I’ve read:

A. C. Grayling The Good Book: A Humanist Bible
A. Hitler, M. Ford (translator) Mein Kampf: The Ford Translation
Alan Dean Foster Star Wars: The Force Awakens
Ayaan Hirsi Ali Heretic
Benedict de Spinoza Ethics
Brendon Burchard The Charge
Charles Duhigg The Power of Habit: Why We Do What We Do in Life and Business
Christopher Hitchens Thomas Paine’s Rights of Man
Dan Ariely Predictably Irrational
Daniel H. Pink To Sell Is Human
David Rock Your Brain at Work: Strategies for Overcoming Distraction, Regaining Focus, and Working Smarter All Day Long
H. Dreyfus,
S.D Kelly
All Things Shining
Jean-Jacques Rousseau On the Social Contract
John E. Smith Georg Wilhelm Friedrich Hegel
John Medina Brain Rules (Updated and Expanded)
John Medina Brain Rules for Baby (Updated and Expanded)
John Stuart Mill On Liberty
Jonah Berger Invisible Influence
Maajid Nawaz Radical
Malcolm Gladwell Blink
Malcolm Gladwell The Tipping Point
M. Dixon,
B. Adamson
The Challenger Sale
Nicholas Capaldi David Hume
Rene Descartes Meditations on First Philosophy
Richard Dawkins The Selfish Gene
Sam Harris Waking Up
Sam Harris Lying
Sam Harris The Moral Landscape
Sam Harris, Maajid Nawaz Islam and the Future of Tolerance
Seth Godin Free Prize Inside!
Seth Godin The Icarus Deception
Seth Godin Survival Is Not Enough
Seth Godin Purple Cow
Steven Pinker The Better Angels of Our Nature: Why Violence Has Declined
Thomas Paine Rights of Man
Thomas Paine Common Sense
Thomas Paine The Age of Reason

The only drawback to my “method” is that there aren’t many tech books in an audio format. At least not that I’ve been able to find. But hey, no system is perfect, but some are useful. I’ll probably keep chipping away at my now somewhat shorter pile in this way for a good while longer.

What’s your favorite audiobook? I’d love to hear your recommendations.

Peer conference awesomeness at SWETish

I spent this past weekend at a peer conference. This one marks my 5th peer conference, but it’s been a long while since I was at my 4th. In fact, it’s been four years since SWET4. Peer conferences are awesome though, as they let the participants really go deep and have thorough and meaningful conversations over one or more days in a small enough group that makes such discussions possible.

Ever since I moved to Linköping a few years ago, I had promised to do my part in organizing a local peer conference for the testing community in and around Linköping and this weekend we finally got that project off the ground! We decided to call this particular conference SWETish instead of making it another increment of the regular SWET. The reasons being that we wanted to keep participation first and foremost from within the local communities and the regular SWET conferences invite people from all over country, and also because we wanted to keep the theme broad and inclusive whereas SWET has already been through a fair number of iterations (SWET7 being the latest one) and we sort of didn’t want to break their set of more specific topics in exchange for a general one that has already been covered. Maybe next time we’ll put on “SWET8” though, if nobody in the Meetups.com group beats us to it (hint hint, wink wink, nudge nudge).

So, sort of but not quite a SWET conference i.e. SWETish (with its eponymous Twitter hashtag #SWETish).

The whole thing took place at the Vadstena Abbey Hotel, which is made up of a beautiful set of buildings, some dating back to the 12th, 13th and 14th century. From an organizer standpoint, I can certainly recommend this venue. Nice staff, cozy environment and above average food. And a nice historic atmosphere too of course. (Click the link in the tweet below for a couple of snapshots.)

When I sent out the initial invitations to this peer conference, I had my mind set on getting a total of 15 participants, as that seemed to be a good number of people to ensure that all speakers get plentiful of questions and that there would be a good mix of experiences and viewpoints, while at the same time not being too many people so that everybody gets to participate thoroughly and nobody is forced to sit quiet for too long. However, because a few people who had initially signed up couldn’t make it there in the end, we turned into a group of “only” 10 people. Turns out that’s an excellent number! Most if not all of us there agreed that the low number of participants helped create an environment where everybody got relaxed with each other really quickly which in turn helped when discussions and questions got more critical or pointed, without destroying the mood or productivity of those conversations.

Another pleasant surprise was that we only got through (almost) three presentations + Open Season (facilitated Q&A) during the conference (1,5 days). If memory serves, the average at my past peer conferences is four and sometimes we even start a fifth presentation and Q&A before time runs out. What I liked about us only getting through three is that that is a testament to how talkative and inquisitive the group was, even though 5 out of 10 participants were at their first ever peer conference! I facilitated the first presentation myself and so I can tell you that in that session alone we had 11 unique discussion threads (green cards) and 48 follow-up questions (yellow cards), plus quite a few legit red cards. So for those of you familiar with the k-cards facilitation system, you can tell that this wasn’t a quiet group who only wanted to listen to others speak. Which is great, because that’s the very thing that makes peer conferences so fantastically rewarding.

2016-11-19-09-53-39
2016-11-19-11-09-22

15094441_10153914893765741_4654744730839257377_n
2016-11-19-09-22-05

Apart from the facilitated LAWST-style sessions, we also spent 1 hour on Lightning Talks, to make sure that everyone got to have a few minutes of “stage time” to present something of their own choosing.

The evening was spent chatting around the dinner table, in the SPA and in smaller groups throughout the venue until well past midnight. And even though we’d spent a full day talking about testing, most of the conversations were still about testing! How awesome is that?

img_20161119_212822

If you want to read more about what was actually said during the conference, I suggest you check out the Twitter hashtag feed, or read Erik Brickarp’s report that goes more into the content side of things. This blog post is/was more about evangelizing about the concept itself and provide some reflections from an organizer perspective. Maybe I should have mentioned that at the start? Oops.

A peer conference is made possible by the active participation of each and every member of the conference, and as such, credit for all resulting material, including this blog post, goes to the entire group. Namely, and in alphabetical order:

  • Agnetha Bennstam
  • Anders Elm
  • Anna Elmsjö
  • Björn Kinell
  • Erik Brickarp
  • Göran Bakken
  • Johan Jonasson
  • Kristian Randjelovic
  • Morgan Filipsson
  • Tim Jönsson
2016-11-20-13-19-34

Thank you to all the participants, including the few of you who wanted to be there but couldn’t for reasons outside of your control. Next time! And thank you to my partners in crime in the organizing committee: Erik Brickarp, Anna Elmsjö and Björn Kinell.

There! You’ve now reached the end of my triennial blog post. See you in another three years! Actually, hopefully I’ll see you much sooner. The powerful dip in my blogging frequency has partly been due to the continuous deployment of new family members in recent years, which has forced me to cut back on more than one extracurricular activity.

Post below in the comments section if you have comments or questions about peer conferences, or want some help organizing one. I’d be happy to point you in the right direction!

Solving the real problem

This is a continuation of my post from back in August on the ISO/IEC/IEEE 29119 software testing standard in which I argued that the upcoming standard is a rather bad idea. This post is more about what I think we should do instead, to solve the problem the new ISO standard is failing to solve.

Another comment to the article I mentioned in my previous post asked whether or not it would be feasible to arrive at a better software testing standard through an open participation wiki page where a democratic process would be applied, and in the end get a consensus and gain widespread acceptance? My reply in short was: No, I don’t think that’s possible. Open and democratic processes are nice, but they don’t guarantee consensus. In fact, they rarely lead down that path.

But I’m not just being bleak and pessimistic here. There are other ways of achieving wide spread acceptance of ideas while also increasing our common testing toolbox!

My take on it is that we should instead (continue to) do the opposite of committee wrangling. That is, don’t worry about consensus up front. Just keep on noticing good practices that you come across yourself and share them. Codify them, create frameworks and talk about them with other testers, at conferences, on blogs, in white papers. Describe the problem and describe how it was solved, and then leave it in the hands of the global testing community to decide what to do with your ideas.

Ideas that are good will get adopted and will continue to evolve. In some cases they’ll become so universally applicable that they’ll start to “feel” almost like standards, at least for solving very specific problems. Nothing wrong with that. If it works, it works. As long as it’s still ok to question the practice and to leave bits out or add bits to it when the context requires it, and when it makes sense. And with ideas like the ones I’m talking about, experimentation is always ok, and even encouraged.

Ideas that turn out to not work, or that only work in narrow circumstances, won’t get picked up by the community at large. But they can still be a valid solution in certain niche situations. Keep on sharing those ideas too! Context matters and one day your context might require you to implement that weird idea you heard about when attending your last conference that you thought sounded ridiculous or that you’d never bother trying out.

What I don’t think works though, is to design a catch-all solution for every testing problems at once, from scratch, and by committee. Which is why I ranted about the ISO/IEC/IEEE 29119 in my previous post.

As an illustration, take a relatively small framework, like the session-based test management framework in its original form. Imagine what would have become of the session-based testing framework if its originators had decided to call together 20 people to come up with a consensus solution for (e.g.) adding more traceability to exploratory testing. My guess is that it would have been dead in the water before they were even able to agree on what traceability means. What happened in reality was that a basic framework was created by James and Jon Bach to solve a real-world problem and it was then presented at STARWest in 2000 and published as articles in different software engineering magazines. It gained popularity among practitioners and is today a widely known approach that others have adopted and modified to fit their contexts where needed. Peer reviewing at its best.

Session-based Test Management, or how to “manage testing based on sessions” as Carsten Feilberg (@Carsten_F) aptly described it once, has become one of several “standard-like” ways of approaching certain exploratory testing problems. The framework doesn’t have an authoritative organization controlling it though. If you want to measure something other than the ratio of time spent in setup, testing and bug reporting for instance, you can. Heck, you can decide to add or remove whatever you want if you think it will add value. Nobody’s going to stop you from trying. If you do, and if it turns out to add value, your peers will applaud you for adding to the craft rather than bashing you for going against the session-based testing “standard”. And if it doesn’t work, we’ll still applaud you for trying (and for showing us what not to do).

And therein lies my point I believe. I’m all for trying to gather our combined testing knowledge. Do it, and share it! I just don’t think it should be codified by a committee and I don’t believe it’s productive to have authoritative organizations trying to control how to apply that knowledge, or deciding what gets put in and what gets left out, regardless of how well-meaning that organization may have started out. It’s impractical and it slows down progress.

Martin Jansson (@martin_jansson) suggests a somewhat extreme, but interesting comparison here (in Swedish), which is to compare how some of us are fighting against standards in the testing world, with the difficulties Galileo went through when he tried to get the concept of heliocentrism accepted into the “standard” world view of the committee known as the Catholic Church back in the 17th century. Out situation is definitely better than that of Galileo, but still, there might be one or two amusing similarities to contemplate there.

As testers, semi-dogmatic organizations like the ISTQB are already out there, using different schemes and big dog tactics to try to dictate how testing should be done or how we should think. I believe we can do without adding more ammunition like a new ISO standard to their arsenal.

New additions to our common testing toolbox are added everyday by serious and passionate test practitioners. Anybody who calls themselves a professional tester should be expected to be able to pick the best possible tools for their context from that toolbox instead of looking for silver bullets by adopting an all-encompassing solution designed and controlled by a committee.

I think we need a greater number of responsible, creative individuals who are humble enough to draw on relevant experiences from other people, but who are also brave enough to take the risk of thinking for themselves and make their own decisions about what works or not for their particular problems. And then, after the project’s done, share their experiences with their peers. That is, I believe, how we solve the real problem of organizations and teams wanting to learn more about how to organize their testing.

Finally, and although I mentioned it in my EuroSTAR post just a few days ago, I think it’s worth once again to point you to James Christie’s take on testing standards. He hits the nail right on the head in my opinion, and expands on many of my own concerns in a very nice way.

P.S. If you’re in an idea sharing mood. The call for proposals for Let’s Test Oz 2014 is open until January 15th 2014. As an added bonus, you might get a good excuse to visit Sydney! Go for it!

Report from EuroSTAR 2013

I had the opportunity to speak at EuroSTAR this year, which made the decision to go a bit easier than it normally is. After all, EuroSTAR is a pretty pricey party to attend compared to many other conferences, such as e.g. Øredev which ran almost in parallel with EuroSTAR this year.

Anyway, this is a brief report from the conference with some of my personal take-aways, impressions and opinions about the whole thing.

People

First of all, to me, conferences is about the people you meet there. Sure, it’s good if there’s an engaging program and properly engaged speakers, but my main take-aways are usually from the hallway hangouts or late night discussions with whoever happens to be up for a chat. This year, I think the social aspect at EuroSTAR was great. I’ve been to EuroSTAR twice before, in 2008 and 2009, but this was the first one where I didn’t think that the size of the conference got in the way of meeting new and old friends. It also made a bit proud to see that the actual discussions and open seasons seemed pretty much dominated by the people in my community this year. This I think has to do with the way we normally interact with each other, and not because of any bias in the program. On the contrary, I think the program committee had put together a very well-balanced program with a lot of different view and testing philosophies being represented.

Tutorials

The first day and a half at EuroSTAR were devoted to tutorials. I rarely attend tutorials, unless I know they will be highly experiential, but this year I opted for one with relevance to my current testing field, medical software, namely “Questioning Auditors Questioning Testing, Or How To Win Friends And Influence Auditors” with James Christie. My main take-aways were not about how to relate to auditors though, but rather how to think about risk. James pointed out that a lot of times, we use variations of this traditional model to assess risk:

Risk Matrix

The problem with that model is that it scores high impact/low probability risks and low impact/high probability risks the same. Sure, if something is likely to happen we’d probably want to take care of that risk, even if it has only “low” impact. But is that really as important as fixing something that would be catastrophic but has only a small risk probability of happening? Sometimes yes, sometimes no, right? Either way, the model is too simplistic. The problem lies in our (in)ability to perceive and assess risk. Something I think is illustrated quite nicely in the following table.


O’Riordan, T, and Cox, P. 2001. Science, Risk, Uncertainty and Precaution. University of Cambridge.

This is an area I feel I want to dig deeper into. If you have any tips on reading material, please share in the comments.

By the way, James Christie also has a blog that I’ve started reading myself quite recently. His latest blog post is a real nugget for sure: Testing standards? Can we do better?

Keynotes & Sessions

New for this year of EuroSTAR is that the conference chair (Michael Bolton) had pushed for the use of K-cards and facilitated discussions after each talk and keynote. 30 minutes talks, 15 minutes of open season Q&A. Nice! I think that’s a very important improvement for EuroSTAR (though full hour slots would be even better). I mean, if you’re not given the opportunity to challenge a speaker on what he/she is saying, then what’s the point? Argument is a very important tool if we want to move our field forward, and it’s so rare that we in the global testing community get to argue face to face. We need facilitated discussions at every conference, not just a few. I’m glad to see that EuroSTAR is adopting what started at the LAWST peer workshops, and I do hope they stick with it!

All in all, I think the best sessions out of those I attended were:

Laurent Bossavit, who made a strong case against accepting unfounded claims. He did for instance bring up the age old “truth” about how fixing a bug becomes exponentially more expensive as it escapes from the requirements phase into the design phase (and so on) of a project. Turns out, the evidence for that truth is fairly poor, and only applies to certain types of bugs.

Keith Klain, who talked about overcoming organizational biases. His 5 point to follow when attempting to change company culture: 1. Determine your value system. 2. Define principles underpinned by your values. 3. Create objectives aligned to your business. 4. Be continually self-reflective. 5. Do not accept mediocrity. Changing culture is hard, but you might want (or need) to do it anyway. If you do, keep in mind point number 6: Manage your own expectations.

Ian Rowland, who gave a very entertaining talk about the power of IT, “Impossible Thinking”. Seemingly similar to lateral thinking (Edward DeBono), Impossible Thinking challenges you to not stop thinking about something just because it appears impossible, but rather move past that limitation and think about how the impossible might become possible. The thinking style can also be used to provoke creative thinking or new solutions, like how thinking about a phone that can only call 5 different phone numbers (a ridiculous idea at first glance) provoked the creation of a mobile subscription plan that let you call 5 friends free of charge. An idea that allegedly boosted sales for that particular carrier in a way that left competitors playing catch-up for months.

Rob Lambert, gave an experience report where he described in detail his company’s journey moving from releasing only a couple of times per year, down to releasing once per week. It was a very compelling story, but unfortunately I find myself currently working in a very different context. True experience reports are always a treat though.

Then of course I had my own presentation: Test Strategy – Why Should You Care?, where I tried to expand a bit on four main points: 1. Why most strategies I’ve seen are terrible and not helpful. 2. A model for thinking about strategies in a way that they might become helpful. 3. Characteristics of good strategy. 4. Arguments why you should care about good strategy. All in all, apart from maybe trying to pack too much into 30 minutes, I think it went ok. The room was packed too, which was nice.

I did sample quite a number of other sessions too, but it’s difficult to sum them all up with any sort of brevity so I won’t even try. Instead, I’ll provide a few quote from the talks I found the most rewarding:

“If it hurts, keep doing it.” – Rob Lambert (Learning/change heuristic)

Condense all the risks of the corporation into a single metric.” – Rick Buy, Enron (anti-pattern)

“Reality isn’t binary. We don’t know everything in advance. Observe, without a hypothesis to nullify.” – Rikard Edgren

“The questions we can answer with a yes or no, are probably those that don’t matter, or matter less.” – James Christie, paraphrased

Governance shouldn’t involve day to day operational management by full-time executives. – James Christie

“Comply or explain” vs. “comply or be damned”. – UK vs. US approach to auditing descirbed by James Christie

“Self-defense skill for testers: “citation needed”. Also: Curiosity, Skepticism, Tenacity. – Laurent Bossavit, paraphrased, warning against accepting unsubstantiated claims

“Do not accept mediocrity.” – Keith Klain

“Culture eats strategy for breakfast.” – Keith Klain

“When you start to think about automation for any other reason than to help testing, you might be boxing yourself in. – Iain McCowatt

“Rational thinking is good if you want rational results.” – Ian Rowland

“Thought Feeders.” – Michael Bolton proposed an improvement to the term Thought Leaders

The Good, the Bad and the Ugly

This was the best EuroSTAR to date in my experience. The program was better than ever and more diverse with a good mix of testing philosophies being represented. The facilitated discussions elevated the proceedings and prevented (most) speakers from running away from arguments, questions and contrasting ideas. I also liked that the community and social aspects of the conference appear to have been strengthened since my last EuroSTAR in 2009. The workshops, the do-over session and the community hub were all welcome additions to me. And the test lab looked as brilliant as ever, and I think it was a really neat idea to have it out in the open space it was in, rather than being locked away in a separate room. Expo space well used.

Camera Uploads

While I applaud the improvements, there are still things that bother me about some EuroSTAR fundamentals. The unreasonably large and hard to avoid Expo, which strangely enough is called “the true heart of the conference” in the conference pamphlet, is one such thing. Not having ample (or hardly any) opportunity to sit down and have my lunch at a table is another. Basic stuff, and I think the two are connected. Seated attendees wouldn’t be spending enough time in the Expo, so eating while standing is preferred to give the vendors enough face-time with attendees. To me, this is not only annoying, but I also think it’s actually a disadvantageous setup for both vendors and attendees. My advice: Have the Expo connected to the conference, but off to the side. Make it easy and fun for me to attend the Expo if I choose, but also easy for me to avoid. For attendees, the true heart of any conference is likely about conferring and we would appreciate having a truly free choice of where and how to spend our limited time at what was otherwise a great conference this year.

Oh, and Jerry Weinberg won a luminary award for his contributions to the field over the years. If you develop software and haven’t read his books yet, you’re missing out. He’s a legend, and rightly so. Just saying.

2013-11-06 20.21.49

Finally, if you haven’t had enough of EuroSTAR ramblings yet, my friend Carsten Feilberg has written a blog post of his own about his impressions at EuroSTAR that you can check out, or have a look at Kristoffer Nordström’s dito blog post.

A solution to a non-existent problem

Recently, there was a post made to the Swedish community site “TestZonen” with some information about how the work with the new ISO/IEC/IEEE 29119 standard for software testing is progressing. That post sparked a rather intense series of comments from different members of the Swedish testing community.

My personal stance on the whole thing has in the past been that I “don’t care”. I don’t care for it and I don’t care much about it. If you’re not familiar with the standard, here’s what the organization’s website says the goal of it is:

The aim of ISO/IEC/IEEE 29119 Software Testing is to provide one definitive standard for software testing that defines vocabulary, processes, documentation, techniques and a process assessment model for software testing that can be used within any software development life cycle.

That’s a pretty tall order if you ask me. And words like “definitive” definitely set off some alarm bells in my head. I also think it’s a near impossible goal to reach, at least if one by “standard” mean anything near what I interpret the word to mean. So when I say I don’t care, I partly mean that I think it’s a waste of time and won’t help the craft or the industry, so why bother? After having thought a bit more about it though, I’ve come to realize that I do care a little bit. Why? Because not only do I not think it would help our craft, I also think it can actually hurt the software testing industry. Now, of course you can always opt-out and simply not use the standard in your organization, but I don’t think it’s as easy as that for the industry as a whole.

What follows below is a not-quite-but-pretty-close-to literal translation of what my comments to the article were. It’s a bit long and maybe a bit of a rant, sorry about that. Look, even an introvert can go off on rants sometimes. It started off with me quoting a line from the article where the author gave some examples about what he thought the might be the outcome of the release of this new standard.

One possibility is that it will show up as a requirement, in procurement negotiations or when selecting partners, that you need to be certified according to ISO/IEC/IEEE 29119.

Yes, that’s one of the risks I see with this standard. As if it’s not bad enough that some buyers of testing services today are already throwing in must-have requirements that they don’t know the value (or cost) of. Here comes even more crap that a big number of test professionals must spend time and money getting certified on, or risk becoming disqualified when competing for contracts.

Another risk is that this will be yet another silver bullet or best practice that well-meaning but dangerously uninformed stakeholders will decide to implement rather than taking the time to understand what software development is really all about. Especially since this silver bullet has ISO and IEEE written all over it. How much will the time saved picking this “best practice” over developing a context-appropriate approach end up costing their shareholders in the end…?

Ok, so isn’t it then a good thing that we get a standard so that buyers and stakeholders can feel more secure and know what they’re betting their money on? No, not in this case. Software development is much too context-dependent for that. Just because there’s a standard out there that somebody’s cooked up, doesn’t mean that people will all the sudden become more knowledgeable about software testing. Instead, this will become an opportunity for people to keep ignoring to learn anything about all the choices and judgement calls that go into creating good software. Just use this standard and quality will flow forth from your projects, that’s all you need to know. I don’t see how it would ever be possible to automatically equate using this (or any other) standard with delivering valuable testing and useful information. But unfortunately I believe that’s what a lot of uninformed people will think it means.

What I think is needed instead is a continued dialogue with our stakeholders. We need to gain their understanding for our job and have them realize that creating software is a task filled with creativity, checking and exploration, and that we perform this exercise guided by sound principles and past experiences, but very rarely by standards. And we can of course talk about these principles and explain and defend them as well as use them when we’re asked to estimate amounts of work, or report on progress, or give a status report. But to standardize the practical application of a principle is as unthinkable to me as it is to tell a boxer that he or she will win against any opponent as long as a certain combination of punches is used, regardless of what the opponent is doing. But, teach the same boxer a few principles like how it’s generally a good idea to keep the guard up and keep moving, and their chance to win increases.

The principles and heuristics we use have very little to do with the kind of rigid process maps I find on the standard organization’s website though. I sincerely hope those are old drafts that have been discarded a long time ago. Judging by that material though, I think the best case scenario we’re looking at is that we’ll end up with a new alternative to RUP or some other fairly large and prescriptive process. And since this process will have ISO and IEEE in its name, it will likely be welcomed by many industries and once again they will go back to focusing on being “compliant” rather than making sure to deliver something valuable to customers. So again we’ll be back in that rigid grind where documents are being drawn up just for the heck of it, just because the process says so. Again.

Like many other comments to the article said: What’s the use of this standard? Where’s the value? Who is it really benefiting? To me, it seems like a solution to a non-existent problem!

The article asked if the standard will mean new opportunities or if it will mean that our everyday work will be less open to new ideas. I believe it will mean neither to me. Because I don’t see the opportunities, and I also don’t see that it’s even possible to do good work without being open to new ideas and having the freedom to try them out and then look back on the result and evaluate those ideas. What I see instead is yet another meaningless abbreviation that testers will get hit over the head with time and time again and that we will have to spend time and energy arguing against following, if we want to do valuable testing in our projects and organizations.

The standard is scheduled for release soon. You can choose to ignore it or you can choose to fight it. But please don’t adopt it. You’ll only make life harder for your peers and you’re unlikely to gain anything of value.