Have you ever sat down with a pile of Legos and tried to build something with whichever pieces came to hand? I have, and it turned out looking something like this:
I didn’t really know where I was going with it, and then I couldn’t find any of the round pieces to use as wheels, and pretty soon my car became a house, then it was just sort of this rock/tree stump thing with stuff growing out of it. When I was a kid, though, I used to buy the Lego kits. You know, the ones that could build a spaceship or a race car or, if your parents were rich, a medieval castle. When I followed all of the steps, I ended up with this:
It took a lot of work, and the instructions weren’t always helpful, but I had a plan. I knew what I would have to show for my efforts, and with the application of some patience, concentration, and some judiciously applied glue to fix the pieces that I broke trying to force them into place, I got my spaceship (with bonus moon rovers!). I had a great sense of accomplishment when I finished that project, which lasted until my little sister played with it and broke it in half. Then my dad stepped on a moon rover and it was time for it all to go back in the big box with all the other pieces.
So what’s with all the Lego talk? As I look around at the state of the software industry, the metaphor seems apt. On the 15th anniversary of the Agile Manifesto, people started reassessing the value and the viability of Agile development practices. Many, including Dave Thomas, declared that at least the term “Agile” should be declared dead. Matthew Kern provided an exhaustive summary of the larger conversation along with some insightful analysis on how the original vision has been co-opted, but the Agile brand will live on for as long as consultancies can bill hours teaching it, practicing it, and building extensive methodologies around it.
I don’t know if Agile is dead yet, but I certainly see some signs of poor health. I’ve written before about the dangers of overly simplifying the messy process of software development and I’ve poked fun at the sunny-minded Agile zealots who think that a Scrum Master certification and a framed copy of the Agile Manifesto will solve all of life’s problems. Now I have another bone to pick with the Sacred Manifesto and its adherents.
Let’s clear something up first: I was agile before we had Agile. I developed my own iterative delivery methodology when I was a frustrated project manager trying to figure out why building software was so damn much more difficult than building a house, road, or nuclear submarine. I was teaching teams how to break functions down and deliver them in pieces when the consultants were still trying to figure out whether “software development” was a practice they wanted to add to their centers of excellence. In fact, I’m not even sure they had “centers of excellence” back then. They might have just called them “centers.” The excellence (and associated higher billing rates) came later. I see the value in acknowledging that software development is essentially a creative process in which continuous discovery plays an essential part. I get it, and I know that, even if “Agile” dies, agility, at the individual, team, and corporate level, is still critical to successfully delivering functional products.
I also live in the real world where, if you want people to pay you for something, you have to actually give it to them. And that’s where I see Agile Development as a practical methodology failing us today.
It all started with Line 4:
[We have come to value] Responding to Change over Following a Plan
On the face of it, this makes perfect sense. If your plan no longer matches reality, you need either a new plan or a new reality. Until science comes through with that multiverse-hopping technology they’ve been promising us for years — and I am absolutely canceling my Scientific Americansubscription this year if they don’t deliver — you’re only left with one practical option: change the plan. Clinging to a plan when it no longer makes sense is foolish. The essence of agility is the ability to respond to changing demands and market conditions for the good of the product.
Once again, though, we’ve gone too far. “Don’t let your plan overrule your common sense” has become “planning is futile.” Not only have we lost sight of the big picture, it seems like some teams are determined to deny that anything like a big picture could ever exist. Their battle cry of “the sprint is everything!” places them in a state of Agile existentialism, with no future, no past, only today. This defeatist approach to the fourth tenet of the Manifesto subverts its original purpose while using it as a club to pound the long-term memory out of an organization’s brain.
As often happens when ideas take on momentum, we’ve swung from one extreme to the other. Where the smart Agile team used to say, “I know where I want to go, but I can’t tell you exactly how long it will take to get there (because, uncertainty),” now the extremist Agile team says, “I don’t know where we’re going or how long it will take, but when we get there I’ll tell you what we did (because nothing is certain and planning is for evil management types).” This twisted logic leaves teams in a Kafkaesque nightmare of failing to deliver because they failed to plan and then choosing not to plan because delivery is so uncertain.
What would you do if you took your car to a mechanic and he said, “Well, I can see what’s wrong, but I won’t know if I can fix it until I take the whole engine apart and look at all the pieces to see if there’s anything else I haven’t seen yet. I’ll start taking it apart now and I’ll let you know the next step in a couple of days.” Would you leave your car with him or find someone who knew what he was talking about? Yet this is what overly Agile teams tell their customers on a regular basis: “We’ll start coding and we’ll let you know what you’re getting when we’re done building it.”
Contrary to popular opinion, “release planning” isn’t a sneaky way for project managers to make their teams commit to delivering features they don’t yet understand. Nor is it an even sneakier way of reverting to waterfall project management. Release planning is how you figure out what you’re going to build and roughly how long it’s going to take, so that you can decide whether the destination will be worth the trip before you leave the house. It’s what you’re supposed to do when you build your backlog and define your minimum viable product and before you begin sprint planning. It tells you what the finished product will look like and when you’ll have it, based upon what you know right now. In true Agile form, we aren’t committing anyone to anything, because we realize that change happens, but in service to agility, we also want to make sure that we’re on the right track before we write a line of code.
If you don’t bother to look at the big picture, bad things happen. Just a few things I’ve seen over the years:
A team was two months into a release before they realized that the way they had designed the features only worked for very specific use cases. They had to scrap what they’d built and start over from scratch.
Another team committed to an initial release of a minimum viable product within four months, but didn’t estimate beyond the next 2-week sprint. Three months into the project, they realized that there was no way they could deliver a viable product in only one more month and had to choose between releasing a stunted, useless product or asking for more time. After six months, they finally released a product that still had fewer features than they originally planned.
A team made commitments to a client to deliver a set of features within six months. While the list of features felt too long to successfully deliver, they didn’t think that they had a choice. To save time, they skipped the initial planning phase and jumped right into coding. After several false starts that cost them several weeks of time, they finally got going. When it was time to deliver the release, they rushed through testing in order to hit the deadline and spent the next two months stabilizing the product, effectively turning their six month release into eight months.
You could look at each one of these examples and say, “See? Software development is uncertain. You never know what could happen, so long-term planning is a waste of time.” Yet for every one of these sad stories I could give you two examples where a little bit of forward thinking and rough estimation kept a project from disaster. Release planning isn’t about forcing false certainty on an uncertain world, but rather about embracing uncertainty. It’s about stating what you know and what you don’t know, and solving for both.
Another advantage of putting a little energy into forward thinking: you can compartmentalize your features, deciding how much time you can afford to spend on each one as part of the entire release. This allows you to decide where to draw the line at “good enough” rather than continuing to polish a feature before moving on to the next one. Without this framing, it’s easy for a complex feature to shove the others aside, leaving you with a difficult conversation as the end of the release approaches.
There’s something to be said for flexibility and for the ability to meet every unforeseen circumstance with the full attention of a development team. But if you’ve given up on planning altogether, then you’ve gone from agility to chaos, from disciplined improvisation to making it up as you go and hoping for the best. Keep it up. and even though you might want a snazzy spaceship, you’re going to end up with an ugly rock.