Another day of interesting talks in the agile testing world. There’s so much to process and think about. If I’m being honest, I’m still digesting all the information from yesterday’s talks!
Thankfully I have my laptop on hand to button-mash the notes together and reflect later. Below are write-ups of the talks I sat in on the second day of the conference.
Leo van der Aalst – 15 tips on testing in Scrum
I felt that this talk was more tailored towards those who were still in the period of transition from waterfall to agile. Whilst we didn’t get to cover all 15 points as advertised, there were some very good sharing of experiences from a number of testers in the room.
One of the more interesting discussions was about the ‘definition of done’ topic. It was interesting to see what it meant for the different attendees and how some even had the notion of “done-done” and how it doesn’t even make sense. For me, being ‘done’ typically means:
- The code has been dev-reviewed
- We have tests at the correct level(s)
- The tests pass reliably in CI
- We’ve had the chance to exploratory test the change
- We’ve updated the release notes as necessary
With this I was also introduced to two new terms for my agile/scrum vocabulary:
- Definition of Ready – The point of which a story is ready to be part of an upcoming sprint
- Definition of Shippable – The criteria which says we are we able to ship to customers or production
Tik Teuben – Modern skills and techniques for the modern age tester
This talk started in bizarre fashion, a poem recital about a lost blue Unicorn. This then lead to a rapturous applause of the audience, I may or may not have participated in this. There seems to be a big thing about getting Unicorns into your presentation talks in this conference. Anyway back to the actual talk…
Tik based a lot of the content on the consultancy work he does as part of helping teams adopt agile. One of the slides looked at research on the skill-sets which were required for testers both pre and post agile. Unsurprisingly there was a far greater importance placed on communication skills for the modern tester. Testers are informers, we connect with people and ensure the correct conversations are taking place at the right times.
Coming from a very agile/scrum work culture at Caplin, I guess I take for granted not having been “set in a way” of thinking. For me, always looking for continuous improvement should be a given in any work place.
Markus Gärtner – Adaptation and Improvisation – but your weakness is not your technique
This talk too, began with a random video clip of pink unicorns dancing on rainbows. I seriously need to figure out how this all started…
Those of you who are a fan of The Matrix would recognise that quote from the dojo fight scene in the first movie. Markus started off by reading a brilliant agile-testing adaptation of the quotes Morpheus uses to describe to Neo what the Matrix is.
Software development is changing. Agile, Scrum and XP have come on a really long way in terms of adoption. Even enterprise companies are now making a change in how they work. With this, testing too needs to change, adapt… improvise.
“If we have mastered a few of these fundamentals along with the habit of curious exploration, we can rediscover special techniques as we need them” – citing from Computer Programming Fundamentals, 1961
Markus spoke about the problems with management and in particular, management by measurement. It encourages supervision, relies on extrinsic motivators and more often than not, the supposed KPIs introduced encourage the wrong behaviour (E.g. code coverage).
A team needs to be self-organizing and adapt to it’s context, Markus referred to The five dysfunctions of a team. Trust underpins everything in the team, the harmony, team commitment, accountability for delivering and ultimately the result.
This was a very creative talk with a lot of the analogies working very well with the audience. It was very original and didn’t feel forced at all. The red pill / blue pill question gives plenty of food for thought.
Jason Ayers – Continuous Testing: Continuous Integration and TDD Extended
The premise of continuous testing is to have your tools automatically run tests locally as you make your incremental changes as part of TDD.
Research at MIT suggested that developers spend 10-15% of their time running test locally. Whether this is running a build step which eventually runs some tests at the end or if it’s executing a large suite of JUnit tests after making a bug fix.
There are tools available which can help you do this automatically when you save a file in your IDE. Some of the more sophisticated continuous testing tools would also try to figure out which tests are best candidates to be executed, as you may not necessarily want to execute ALL of your test suites, but perhaps just the ones that your changes affect.
Fast feedback is important – Of course, in order for this to ‘work’ and be effective, you also need to be aggressive in getting your test execution down to milliseconds from seconds (introducing more mocking etc). Jason sees this way of working as something that a lot of people would be looking to move towards.
Gojko Adzic – Reinventing software quality
This was by far the most thought-provoking talk of the day, my favourite so far. There were also unicorns.
“We’ve got software quality completely wrong” – Collecting the wrong data will hurt your company.
An example given was the American Internal Revenue Service began publishing data on how many people were avoiding tax in 2007. By 2009 the number reported as part of the publication had dropped but it was also noted that there was a huge spike in fraudulent activity, coincidence? The point being made was that measuring the wrong thing can lead to unintended consequences. The trend may have been what they were hoping for, but the actual result worked against their intention.
“This is what happens when you track bugs as KPIs” – Gojko was stressing that we too often measure the wrong thing, we don’t focus on what really matters, our progress towards business goals.
“Tracking bugs is a measure of your process, not your quality” – Bugs are just a health indicator for a system. An absence of bugs does not indicate the presence of quality – what good is a system with 0 bugs but is still useless for the user?
Gojko made reference to Maslow’s hierarchy of needs and related this back to the needs of a software system in a similar pyramid, I would highly recommend reading his blog post which goes in to a lot more detail.
In summary, after a certain threshold of functionality, performance and usability is met or “good enough”, Gojko suggests that you have very diminishing returns if you do not turn your turn your focus to the higher segments of the pyramid – usefulness and the definition of your success for your business goals (e.g. income, larger market adoption etc).
Gojko discussed how the team worked together and was self-organizing, they even tested each other’s code on printed paper and how things are very different today, citing reference back to Alberto Savoia from Google saying that ‘test is dead”.
So what does this mean for today’s ‘agile tester’?
This is plenty food for thought. What I’ve found is that for me it’s true that I don’t simply ‘just test’, I’m also actively engaging stakeholders outside the team and doing whatever I feel is best for team productivity. Depending on your skill-set and workload, you may also be able to chip away at several build tasks or even picking smaller development tasks if you’re comfortable in the code base.
I see my role as helping my team be productive, generally a lot of that time is helping with acceptance criteria, reviewing tests and finding edge cases – that’s what I’m good at but I don’t let my job title restrict what I can contribute to the team. Everyone has a different context that they work in, the challenge is finding the way of working that focuses on delivering value defined by business goals, not process goals.