BDD: Starting With Quality

By Shannon Kringen @ flickr
Quality with a star and stuff

Quality isn’t defined by a lack of defects or bugs. You wouldn’t get a cup of barista coffee and exclaim, “This coffee is quality because it doesn’t have a problem with it.” (No bugs in my coffee). You can get a cup of coffee from a petrol station that doesn’t have any problems with it. It’ll be hot, wet and coffee flavoured – there are no defects with this coffee. So, where does this idea of the “quality” of something come from? It’s a subjective thing. A quality cup of coffee to me may not be the same quality cup of coffee to you. If you favour a black americano, your definition of quality is not going to involve a milky heart drawn into the top of it, which is how I like my coffee.

When we talk about the quality of a feature, the fact that it doesn’t have any defects is an implicit part of that quality, but it’s not where quality starts or stops. What makes the quality of a feature is whether or not it does what it’s supposed to do and whether it provides the amount of value to the user that they expect. Again, this is different depending on your context: what you application does, who it does it for and why.

Quality Assurance (QA) plays a fairly key role in any software development team. I know some schools of thought suggest that there shouldn’t be a QA role, and while this is probably the subject of a separate blog post, I feel that this is wrong. We have a QA in the team, just the same as we have a designer in the team. It’s a specialist role that requires certain skills I don’t expect engineers to necessarily have.

That said, I’ve always been troubled with the way that the QA role is executed in a team. Let’s suppose that we’ve got a scrum team that performs well. They commit to a given number of independent stories, work on them sequentially, so they finish the first story before starting the second and so on. Once the feature has been completed, the work of the QA starts in earnest (until that point, the QA will put together a test execution plan and a strategy for dealing with the tests during the sprint). They will begin exploratory testing and creating or updating automated tests. This is all well and good and will ensure that the feature meets the minimum, implied, level of quality. In most cases, it’s enough that it’s free of defects.

For me, this is where the problem lies. But how do we solve the problem?

We realised that actually, we never really discussed what quality meant to a particular story or sprint. We had made assumptions about the quality based on the content of the story and the acceptance criteria. As long as the story met those acceptance criteria and didn’t have any defects, we assumed we were done. In reality, we weren’t really thinking about what constitutes quality but just what constitutes the feature.

So we decided to start with quality. It made sense to talk about what we thought quality meant to any particular story before we talked about anything else. At the beginning of planning a story in sprint planning, we would spend some time discussing what quality meant to this feature. Using the example of a login screen, the story might be:

  • As a user,
  • I need to log in to the site,
  • to access all the features.

Before we chose to start with quality, we might discuss what the feature looked like, or we may already have a design for it. But then we’d just jump straight into the technical planning: how do we implement it, what code do we need, database schemas – that kind of thing. Instead, now we talk about the feature from a users’ point of view:

  • What happens if they get their password wrong?
  • How do they reset their password?
  • How long should the password be? Should it have characters?
  • What happens if a user doesn’t have an account, how should we direct them to sign up?
  • What kind of error messages do we want to show?
  • Etc.

This opened up a whole new discovery phase. Product Owners cannot think of everything when writing stories and this discovery allowed us to offer our insight into how the feature works, ask questions about how it should work and these are often based on technical knowledge of the platform that the Product Owner may not have. We began by adding these new requirements to the conditions of satisfaction, but they soon become long and arduous to check. So we looked for a better solution than acceptance criteria.

The solution we chose was to use a new tool. BDD (Behaviour Driven Development) is a method of functional testing which allows you to describe the functionality of a feature in a “scenario” file in plain english:

  • Given I am on the login page
    • When I enter ‘mikepearce’ into the username field
    • And I enter ‘butteryballs’ into the password field
    • And I click “login”
    • Then I should see my dashboard.

We can then use a BDD framework to run these scenarios against our web application. We use Behat as a framework and it works really well (apart from our problem of data fixtures…). It pretends it’s a browser (and, will use Firefox if it needs to work with javascript) and does what your user would do with your system. This allows us to have automated functional testing too. Awesome.

So, when we’re doing this extra discovery step, we record our findings as these step definitions, instead of acceptance criteria:

  • Given I am on the login page
    • When I enter ‘mikepearce’ into the username field
    • And I enter ‘wrongpassword’ into the password field
    • Then I should see the message ‘Sorry, your password is wrong’
    • And I should see the link ‘Did you forget your password?’

We slowly build up a specification file for this feature, which is mostly centred around the “happy path” and add edge cases or problem scenarios if we think of them. It’s important to note that we don’t expect to think of EVERYTHING in this session as we time box it to ten minutes and expect other features or ideas to emerge during the sprint.

Once we’ve finished, we’ve got a specification file that we can run against the web app with Behat. The first time we run it, it will fail, because the feature isn’t there – but this is good! This is Test Driven Development for the masses! As the team slowly builds the feature and keeps running the Behat tests against it, it will slowly become more and more green. If new things emerge during the sprint, we add extra steps to the scenario file. By the end of the sprint, all Behat tests will be green and we can have confidence that, not only is the feature defect-free, but it also does what the user expects it to and provides them value.

So, now we have a way of assuring that our software has quality. Not only do we have a slick set of automated functional tests, but we’ve also added a low-friction, low effort step of discovery that allows us to really understand AND define what quality means to us in the context of this feature for our product.

I’d encourage you to try this in your teams. Actually having Behat (or any other BDD framework) isn’t really a requirement to get started. You can start by just writing your scenario file as the first step of your team planning a story and storing it somewhere for future reference. The value is in the discussion you have to define the quality. The artefact you create is useful afterwards for checking that you meet those requirements. The added benefit is that it’s written in a language that anyone can read – from your team and product owner, to stakeholders and anyone else in the business who is interested to learn more about your product.

(The benefits of using BDD are outside the scope of this article for more than I’ve described, there are plenty of articles on the web to suit your style, language and infastructure and I would encourage you to learn more about it. Start with this presentation from Gojko Adzic and this from Dan North)

Advertisements

Who tunes the guitars?

Black Cat by erin m on flickr
You need men like this, otherwise live music would sound awful.

I read a lot of blog posts, articles, papers and what-not on creating, maintaining and coaching teams. It’s the one thing that, for me, has no set formula. Each team is different, as they’re made from different individuals, each with their own set of morals, ethics and experiences that drive their behaviours. There is no ‘one-size-fits-all’ team guide, unfortunately.

So, what we’re left with is a miasma of different works, all decrying or promoting some way or other of how to form and maintain a team, how to help them become high-performing, how to retrospect, plan, work, play, love, eat, argue and play football together. None of them are right and, equally, none of them are really wrong. But the one thing I see frequently is suggestions, and often demands, that “if they don’t fit, you’re hiring wrong/replace them” and this is alarming.

The common theme I see is that, yes, there IS a formula for creating the winningest team and, if one of the members of the team you’re trying to engineer to stardom doesn’t fit your mould, then he’s out on his ear … hit the road jack, your awesome-coding-but-complete-lack-of-personal-skills ass is outta here!

Not everyone can be a rockstar, you need some roadies too.

Perhaps you know someone, or have someone in your team who doesn’t really have any great ideas, isn’t that keen on researching the latest javascript framework or building some wanky Hipstagram filter in Ruby, but who can build a data abstraction layer, fully unit tested in a day and always reviews your code and finds the one or two things you missed. Where would you be without this guy?

Or maybe there’s the guy who just REALLY enjoys honking about with MySQL queries, doesn’t really give a rats ass about what the business does, but cares deeply about the quality of his code and making his queries elegant and fast.

Some of you may know the guy who get’s a real kick out of cutting HTML all day, taking the photoshop files from your rockstar, sweater-vest wearing, macciato drinking, fixie-bike riding designer and making a cross browser compliant, responsive and even working in IE.

These guys may not be the fastest, or the bloggiest, or the githubbiest, they may not even really get on that well with agile estimation, sprinting or ‘stakeholders’, but they’re rock solid and dependable.

Everyone has a place, even if they’re wearing a christmas sweater in June, they’re part of your team. Find a place for them, or, better yet, let them find their own place and do what they do best, even if it doesn’t align with your idea, or the internets’ idea, of what a perfect team is.

They tune the guitars and your rockstars play ’em. That’s what a real team is.

Some of us inherit teams. We don’t have the luxury of hand picking the people we want to have. Maybe there just isn’t enough in the budget to fill your team with rockstars. Software gets written outside of Silicon Valley too!

This, for me, is the real notion of building a team. Not chucking money at a bunch of rockstars, but taking what you have, or building on a small budget and creating a live band – you need rockstars AND roadies if you want to wow your audience.

The Snoopers Charter

So, I wrote my MP, Sam Gyimah:

Hey Sam,

Haven’t written in a while, but this new ‘Communications Data Bill’ has really got me worried.

While I have absolutely nothing to hide, I believe that too many people having access to too much of my information – personal information – that can be easily accesses and held insecurely, is a terrible idea. I’m also fairly sure that, given the track record of data-loss by organisations in this country, the access will not be properly regulated.

Collecting and having access to this data by law-enforcement, with a warrant, is a good idea, but only in small measures and certainly not the broad strokes the bill lays out and without adequate safeguards.

Why is it the EVERYBODY’S data will be collected and stored – mine, yours, my mothers and not JUST those who are under suspicion? I pretty sure there is no good reason to collect this information about me without my consent and further, there can never be a reason why this data can be accessed so easily and without warrant. Who get’s to decide who accesses this data? Law enforcement. Do they need a reason? Probably not – that’s the meaning of warrantless.

Hiding under a shroud of “maintaining capability”, whatever THAT means, the bill, in it’s current form, is dangerous and alarming. Much of the data the bill makes reference too, held by services like Facebook, Twitter, LinkedIn etc, is much richer than that of a telephone conversation and, once mined, will probably highlight things about me that I don’t even know. This is invasive and most unwelcome.

This kind of snooping and data storage requires stringent oversight to make sure that people are not put at risk.

I have seen no evidence to support the need for this bill. It undermines the commitment of the coalition (which YOU are a part of Sam), to enhance our civil liberties.

I would like you, on my behalf, to oppose this bill unless it changes significantly. On top of that, I’d like you to make others aware of what dangers this poses and how it could be abused. Control over these kind of powers needs to be tightened, not loosened as this bill requires.

Yours Sincerely,

Mike Pearce

Ready, steady, ROWE!

Today, my department begins a trial of a new culture. It’s a new way of thinking about work. Alright, it’s not *new* but it is rare. Especially in the UK. The culture can be summed up with one sentence: ‘Employees are free to do what they want, when they want as long as they get the work done.

There will be a whole bunch of blog posts on this subject, so consider this one a bit of a summary of all the main points.

History

I’m talking about a Results-Only Work Environment, or ROWE for short. It started with Best Buy in the states. Two employees were tasked with making things better and they began with flexible working programs which quickly morphed into a Results-Oriented Work Environment, then settled on a Results-Only. They detail their journey in their book ‘Why work sucks and how to fix it‘. I devoured the book in one sitting and made copious notes and realised that this was what we needed.

The way business works now is outdated; why do we need to work between 9 and 5:30? Why do we need to go to one particular place to do it? It doesn’t matter if you’re a knowledge worker, or in sales, it’s the same thing: the focus should be on results, not when or where you are.

The trial

It’s not been easy getting this trial started. There are lots of questions around holiday allocation and remaining on the correct side of UK employment law and legislation. Essentially, we need to still allocate people the appropriate amount of holiday and then ensure that they remember to take it. This mostly covers the company, so if someone leaves on a bad note, they can’t claim to have not taken holiday, because, in ROWE, you take holiday whenever you need it, there’s no allocation really. Because employees get to choose when, where and how they work, the rules governing the Working Time Directive don’t count either. There’s no 48 hour waiver or whatever.

We’re starting with *almost* the exact ROWE for a three month trial, I say almost because we’ve made two concessions, which I detail in the guideposts below. After three months, if it’s still working, we’ll extend the trial. The idea is to see if vanilla ROWE as detailed in the book works in the UK. UK and US cultures are different and employment law and employee rights are also different, so we need to make sure it fits properly for us and our business. I’m 100% confident that this will work.

Freedom and culture

Ultimately, working in a ROWE is purely focussed on the results. But it gives employees absolute freedom to manage their lives they way they need to. We all have to work, there’s no way around that (unless you’re a millionaire playboy or whatever), so we should be free to fit that work into our lives as we see fit. You don’t need to take a half day holiday to visit the doctors, you don’t need to phone in sick unless you’re going to be letting someone down with a meeting or conversation and you don’t need to worry about where anyone is, you and your colleagues are available 24/7 by phone, voicemail, email, SMS, skype, irc, Google Hangout – lot’s of options!

ROWE isn’t an activity or an action, it’s a culture and a new one too. It’s about changing the attitudes people have towards work and challenging the long held beliefs that time plays an important role in measuring someones value (time is still relevant for deadlines and in some lines of work, but not in the web industry).

Productivity

I’m expecting to see productivity improvements too – I have no idea what they’ll be or in what format, but I’m sure they’ll be there. For one, I find I can get a lot of work done in the wee small hours, because I’m not being distracted and there’s nothing to procrastinate about when the kids and wife are in bed – so I can’t play PS3 or watch a movie, so I work and I can get more done in less time because I’m so focussed on what I’m doing.

The results

Our teams use Scrum to build our software, so we have a built in results measure. We’ll be looking at velocity for each team and the department as a whole, as well as defects, engagement and acceptance rate of stories (the percentage of story points accepted at the end of the sprint). On top of that, all the people in my team have individual goals and objectives; a mix of skills acquisition and platform or performance goals, so I can measure individuals as well as teams.

The Guideposts

ROWE is based on a set of principles called ‘Guideposts’ which enable the change of culture to happen with a purpose, the most contentious of which are ‘Unlimited Paid Holidays’ and ‘Every Meeting is Optional’. The holiday one is easy to cover in principle – it’s irrelevant how much holiday you take, or when/where you take it, as long as the work gets done. That said, we still need to make sure that we’re adhering UK legislation and in order to do that, we still need to allocate and record holiday taken. It’s a small price to pay for that much freedom though!

‘Every Meeting is Optional’ is also difficult for people new to the culture to get their heads round, it doesn’t mean ‘flip a coin to decide whether or not to go to a meeting’, it does mean, find out if you can get, or give value to the meeting, find out, or define (if it’s your meeting) the outcomes and then decide whether you need to be there in person, whether you can dial or Skype in, or whether you’re just required to give information which you can email to the organiser. As long as the work get’s done and you’re meeting the goals, objectives and targets set, it’s up to you whether you attend meetings and how you attend them.

The one concession we made to this guidepost was that, ALL meetings are optional, but pay particular attention to external client meetings or group-wide meetings. We’re only one department in one company doing a ROWE trial, we can’t expect others to change they way THEY work … just yet. It’s a small concession and, to be fair, will barely affect our department.

Metrics and the win condition

How do we know the trial is successful? If nothing changes. Like I said, we’re currently tracking:

  • Velocity: The average rolling department velocity over the last 4 sprints (and long term velocity, but this is less volatile and less useful)
  • Defect rates: how many defects are opened per two-week period and how long those defects stay open (as an aggregate and also by priority)
  • Acceptance rate: What percentage of story points is ‘accepted’ by the product owner at the end of the sprint.
  • Engagement: we’re using Murmur to track employee engagement with the company on many levels.

We’re also looking to measure the perception of the department from across the business.

The win condition will be if nothing changes. If none of the metrics change over the next three months, then the trial will be considered successful, the net benefits of ROWE will be over and above just the impact on those metrics. So, as long as nothing get’s worse, we’ve proved ROWE as a culture in the organisation.

On top of the team metrics, we’re also setting goals and objectives on an individual level – skills acquisition or just ‘stuff that needs doing’ for the platform, our tools or whatever. So, we can watch everything that’s happening and see if it’s making a difference.

Next?

Next I sit back and wait for a sprint or two and see what’s happening, I need to keep my eye on the metrics and the individual objectives, but really, it’s business as usual … or not, depends on how you look at it!

I’ll be blogging more as the trial progresses in the hope that others in the UK who are already in a ROWE, or those thinking about going ROWE can share their progress, problems, failures and successes.