Accurate estimation, really?

I ended up in a twitter conversation last weekend about estimation and velocity.  It started when Annu Augustine asked about what qualities to look for in a lead developer, other than technical skills.  One of the qualities I put forward was accurate estimations. This went around a bit and ended up, not surprisingly for me, at velocity.  There are a couple of points that I need to offer my opinion on:

I ended up in a twitter conversation last weekend about estimation and velocity.  It started when Annu Augustine asked about what qualities to look for in a lead developer, other than technical skills.  One of the qualities I put forward was accurate estimations. This went around a bit and ended up, not surprisingly for me, at velocity.  There are a couple of points that I need to offer my opinion on:

Accurate estimation is not an oxymoron.  Let’s just get over some technicalities first: “accurate” is not the same as “exact”.  Estimation in software is never exact, but the magnitude of accuracy is very significant.  If I say that something will take approximately 2 weeks to achieve, then I need to qualify “approximately”.  What is the magnitude hidden behind “approximately”?  Is it 10% accurate, 50% accurate, 80% accurate?  Let’s say it is 50% accurate.  This means that I can finish this work at best in 1 week or worst 3 weeks.  That’s a big swing.  As a good developer, it has to be better than 50%.  If it’s 50% or less, then it is an indicator that you are lacking understanding (i.e. knowledge) in a key part of the problem and/or solution domain.  Then you have to estimate on what it will take to overcome that ignorance, before committing to the original request.

An estimate is not a commitment.  If you mix these two concepts together, then you most likely will be in trouble sooner than later.  The estimate is based on something that you understand and that you can foresee with good accuracy.  In other words, an estimate is a prediction of the future.  The commitment is a promise that you make on part of the estimate in order to make that prediction come true.  If I predict that a problem can be solved in 3 days, then I may make a promise for some part of it by the end of the first day. This distinction may surprise some who have been using the poker game, or other estimation technique in Scrum. Scrum teams use estimates in planning make commitments for the entire sprint, then track and calibrate velocity via burndowns, which leads me to the next point.

Team velocity or relative estimation does not make for better estimation.  Knowing what a team’s velocity is based on tracking historical trends is only a measure of the energy the team has expended to date.  This expended energy (the area under the curve) is used to predict the potential energy of a team.  That’s all, nothing more nor less.  I will stick my neck out and go so far as to say that relative estimation in points is not at all about estimating, but a way to accurately predict the potential energy of a team.  I’ll go even further: relative sizing does not work because software development is about crunching new knowledge, and every new piece of work should be crunching more knowledge.  If it’s the same thing over again, then the design is flawed or at least at the wrong level of abstraction.  Jumping levels of abstraction, and re-shaping designs takes quite a lot of knowledge crunching and is not relative to anything else but your existing knowledge.  So, relative estimation does not make for better estimations and velocity just tells you the potential energy of the team.

Where does this leave me?  I’ve given up on relative sizing and estimation.  Knowing my own velocity has not added any value to my ability to estimate because every problem is unique in some way.  I estimate based on the knowledge that I have at that point in time, and force myself to be aware of my accuracy.  All of this, and more, has made me appreciate the value of one piece flow a lot more than time boxed flow.

SDTimes, Oredev2011, Rubyfuza2012, AgileIndia2012

Over the next few months or so, I will be quite active in the conference space.  Here’s what’s coming up soon …

  • SD Times’ Leaders of Agile Webinar: This is the fourth or fifth in this series hosted by Kent Beck.  It’s always very engaging for me, and Kent normally asks far too many difficult questions without warning me in advance.  It’s also the third time that I will be working with Henrik Kniberg.  This time we will be talking about Kanban, and should be another really fun 3 hours.  What I enjoy the most is the balance between practical experience, challenging the norm, and always the realisation that I know far too little.  It’s free and you can register at SD Times.
  • Oredev 2011:  I’ve been invited back to Oredev in Malmo, Sweden.  This is always a fun conference and I get some face time with my factor10 colleagues in Sweden again.  On this occassion, I will be giving two talks.  In the first I will explore the messes we make over and over again, yet we are missing some old teachings and learnings consistently.  The second is on the Java track and I will share my experiences on experimenting with Scala and Clojure, more from design and mental mindshifts, instead of the usual language and syntax changes.  Check it out at Oredev 2011.
  • rubyfuza 2012: This is the second edition of the only South African Ruby conference that I know of.  The 2011 edition was well put together with great local and overseas speakers.  Although the program is not finalised, I will be talking about using JavaScript and CoffeeScript to build parsers and grammars.  This is something that I’ve spent a lot of time on recently and it’s changed a lot of my thinking on domain driven design implementation options.  Check out the details at rubyfuza.
  • AgileIndia2012:  This is my first trip to India, and one that I am absolutely excited about.  I will be running two sessions.  One on product ownership which is a slight improvement on the sessions held for SD Times and the S.African Scrum Safari.  The other is a culmination of years of introspection on cultural differences that I face.  I explored this quite emotionally at Oredev last year and will run a more in-depth session for AgileIndia.  Check out the abstracts for my sessions Practical Product Ownership: balancing strategy and development and Collaboration lessons from the Rainbow Nation the early bird program.


Bigger stories with few people spanning two sprints

I came across this tweet by Karen Greeves.

Scrum Master fail… Improvement action: Bigger stories where only few members can participate should be scheduled to run over 2 sprints.

After a quick twitter conversation, Karen explained.

It removes the ability to measure progress via working software at the end of the sprint

My response was

Working software is not the only way to measure progress in a sprint. And what if it works? I think it can.

It may well not be the ScrumMaster that failed when it was decided to have a bulkier story span two sprints.  I appreciate that it is a whole lot better when a story is in one sprint, and that should be our default objective.  However, it’s often not so trivial.  Given that I don’t have a lot of context, I will be assuming a lot.

In this case, it is more likely a failure of the team that they (a) accept a story to develop over 2 sprints, or (b) was unable to do enough analysis to consider what can be done in one sprint.  It can also be a failure on the product owner for not entering into a meaningful conversation on the details of the story, and then, again, a team failure on not engaging the PO significantly to take the problem forward.

If I encounter a story that spans two sprints, and that is more often than you think (often discovered mid-sprint), then I’m not interested in working software but seek clarity of understanding in that sprint.  The outcome at the end of the immediate sprint is an unambiguous story (or maybe several stories) which is a statement of the problem domain, or even better, the solution domain.  That is what I mean by working software is not the only measure of progress in a story.  It is more important to “measure” increased understanding in each sprint, and good statements in the solution domain is at the heart of knowledge crunching.

The most neglected aspect of working software is a measure of understanding of the solution domain.  In my experience, many teams are great at expressing the problem domain in software and their code reflects the analysis of the problem.  Consequently, the code does not reflect the understanding of the solution.  The end result is a weak design.  Over many sprints that require work in the vicinity of the weak design, there will be a degradation in velocity because the code is just baking in the problem statement, and not a well crafted solution to the problem.

Next, let me deal with the case of just a few team members participate and not the entire team.  I think that is completely feasible approach.  I’ve done it many times to great effect and with great efficiency because the conversation is a lot more direct and contained.  It is later that the distilled knowledge is shared with the team.  This is largely crunching in the problem domain with some rough models in the solution domain.  When I’ve included the entire team in the analysis, then the effect is one of dilution and inefficiency – too many people over a longer period.

Lastly, I asked “What if it works?”.  While this may seem to be brash or provocative  question, it is meant literally.  What if it really does work? Hey, then what we’ve achieved as an odd case of delivery over two sprints instead of one.  If it happens often enough, then we need to adapt accordingly: increase sprint duration, have people dedicated to analysis (oops, bite me ‘cos I’m creating a silo) and maybe more if we just think a bit about it.

So, in my opinion, it is absolutely OK to attempt the proposed solution because it is an admittance of ignorance, but if the team does not understand the actual situation they’re in, then it is the failure of the team not the ScrumMaster.  I also think that we should understand the rules of the game a lot more deeply.  Like I’ve said in the past:

It’s not the rules that matter, it’s what we do with the rules that counts.

To each their own, and life goes on.

Stay in bed or come to SGZA

I will be hosting a 3 hour session at the South African Scrum Gathering titled “Live your principles or stay in bed”.  You can read the abstract here.  In my opinion, there is far too little focus on software development itself in Scrum.  So, this is unashamedly a developer session.  I will be present various snippets of code, and we will “live our principles” to transform the code into something that is less messy.
I often hear developers, and managers too, saying “It’s so much easier without, so why bother?”.  Well, design is hard.  Applying principles for life is harder.  But if you are professional developer and have a conscience about your design, your code, and your product then “an easy life without principles” is not an option.

If you are planning to come along, bring your laptop with your development environment.  I will likely have code samples in Java, C#, Ruby, Javascript, and even, yup, Basic (well, maybe).  All the samples should be very readable and you could easily translate them to something equivalent in your language pretty easily.  Better still, bring some of your own code along, that you want to share.

In reality, this is stuff that Scrum does not teach you, but need to know to avoid Scrum burnout.  Looking back, I should have done something like this sooner.

Upcoming talks

I’ve got a busy few weeks of preparation to get through for a few talks that I will be giving.  In September I will be speaking at the South African Scrum Gathering.  The one talk for Johannesburg is on product ownership and it is a combination of the content that I presented earlier for the SD Times webinars.  The other I hope to keep quite code centric and is aimed squarely at developers and architects.
Then in November, I’ve been kindly invited to speak at Oredev in Malmo, Sweden.  I will be talking on the Java track and Architecture track.  Being in Sweden, I get a chance to get some face time with my Scandinavian colleagues, and lots of offshore geek friends.  If you are a South African looking for a decent developer conference, then consider Oredev.  It has a good vibe and some very good content, and is generally good value for money.

You did what with your ESB!?

I’ve seen many enterprise service bus (ESB) implementations that are, well, quite extraordinary.  Sadly, they are extraordinary for more wrong reasons than right.  Given that we had a SOA frenzy not too long ago, many developers got caught in that feeding frenzy.  Hey, when you’re a shark off the south coast of Kwazulu-Natal in winter, everything looks like a sardine.  In fact, if you were a tiny sole, just wandering around at the bottom of a big sea, chances are you also wanted some of them some sardines.
That was a long time ago, but now that we get to see the extraordinary things that developers built with their ESB.  Here are my top five extraordinary things.

#5 Look, we can synchronize our databases with our ESB

You have two application databases and you want the changes in one to be propagated to the other.  The ESB seemed like a perfect way to spray data around.  Hey, just give me an end point and I’d pump some data through it.  Well, syncing data is a replication problem with different challenges such as change detection, conflict resolution, retry or abort strategies, and bulk materialization on both ends when things get horribly out of sync.

#4 You can call my stored proc from the other side of my ESB

Somewhere in your app you had something that called a stored procedure. The ESB seemed like a really easy way to wrap that SProc with a service and hand out a new end point.  Cool, now everyone can call your stored proc.  Well, perhaps that original thing that was calling your stored proc was a wrong architecture decision in the first place.  If it was the write decision, suddenly opening it up to multiple callers that you have no control over means you got to be certain about whether your SProc is re-entrant, can handle the concurrency, and a whole lot more. 

#3 My ESB now manages my transactions

Enough said.

#2 My authentication runs as a centralized service on my ESB

Login seems like an easy enough “hello world” service to get up and running.  Hey, since our ESB has all of these apps hanging off it, we can get our Login Service to do single sign on (SSO).  We just need to call all the login services for each app and front it with our single sign on service and then this service will dish out authentication tokens.  Oh, we might as well put in a centralised authorisation service too.  Well, firstly, login is not a service.  You can take that thought further from here.

#1 I replaced my call stack with my ESB

To be fair, most developers don’t know that they did this.  Let me explain.  You had some app that called a method that calls a few more methods and so on.  We all know that this builds up a call stack that gets popped on the return path.  When you take those classes that have those methods and dump them as services on your ESB, the call stack has not changed, but now there is an ESB in between.  Well, you can’t just shift classes wrapped as services to your ESB.  You actually have to get service oriented in your architecture.  Only then will that call stack change.

It’s not surprising that we see this now.  It takes a long time for an architectural style to be understood.  Unfortunately, there are some costly mistakes along the way.  We also can’t blame the ESB, but, beware, the sardine run happens every year.

What’s the point in Scrum?

Scrum people like to use points for estimating and measuring velocity.  I won’t go into detail about how points work and how to play those poker estimation games.  Just search around and you will find a ton of stuff.  So, back to this points stuff.  I have a divided relationship with the humble point.  I like it when a team switches to using points for the first time, because it gives them a chance to think a little bit deeper about what they want to do.  I don’t like it when we start inventing rules around points (and you can lump guidelines and best practices into the rules pot too).  When the rules appear, the thinking disappears.
In every team trying Scrum, there is bound to be a rule about points.  I dare you to put up a hand and say you have none.  These rules are things like “We can’t take anything over 13 points into a sprint”, “Our epics are 100 points”, “The login screen is our baseline of 3 points”, “Anything over 40 points must be broken down”.  So, I double dare you :-)

Sprint backlog shape with high shared understanding

I have different view of the humble point. A point may seem like a one dimensional thing, but it has a some facets built into it.  One facet is the “amount of effort to build something”.  Another facet is “amount of ignorance” and this has an inverse – “amount of shared knowledge”.  Sometimes I find it useful to make a judgement based on what I don’t know as opposed to what I do know.  Regardless of whether I choose to view the cup as half full or half empty, I cannot estimate effort to build something based upon what I don’t know.  So, effort tends to track the amount of knowledge, not ignorance.  As knowledge increases, my ignorance decreases and each point starts representing more and more of pure effort.

However, if I am in a state of complete ignorance, then it is completely impossible for me to make any judgement on effort to build.  I’d be simply speculating.  What I can do, though, is create a time box to explore the unknown so that I can start moving out of my state of ignorance.  This is also an estimate and I am not making an excuse for non-delivery either.  I need to understand some things and also show my understanding in some code.  Yes, the code that I produce may not have a visible user interface or some other convenient demo-friendly stuff, but I need to carefully plan my sprint review to express my understanding.

It’s all about gaining a SHARED understanding. This understanding is body of knowledge that I have learned which I need to confirm with others.  This act of confirmation can happen in several ways.  I can have a conversation and explain what I understand, I can draw a blocks and lines picture, or show a spreadsheet, and so on.  Regardless of the method of communication, I still use the opportunity of discovery to express my understanding in code as tests.  Another powerful way of expressing my understanding is to write out a story and a few scenarios.  Using BDD style grammar can be a great way of concisely expressing some things, that can be easily shared.  Yes, you heard me correctly – as a developer, I write the stories and scenarios.  When I am given a story and scenario by someone and asked to estimate, then I am attempting to estimate based on  another person’s expression of their understanding and my assumed understanding.

In a recent discussion with Jimmy Nilsson, he said that he prefered to call scenarios “examples”.  That really resonated with me.  I also do a lot of discovery by example, and then gradually introduce more a more into the examples, as I get more and more confident of my knowledge.

How do I know how much I don’t know? That’s a tough question.  What I do comes straight out of my TDD habits.  I create a list of questions – my test list.  For some questions, I will know the answer easily, some not all, and some are debatable.  The more that I can answer, the better I can estimate effort.  I can then turn the questions that I can answer into statements of fact.  The more facts I have, the less ignorant I am.

Recently, I worked with a team that wanted to get TDD going, and the most significant change that I introduced was in backlog grooming and sprint planning.  During these two ceremonies, we (as a team) threw questions madly at a requirement, regardless of whether we knew the answer or not.  We then worked through the questions (as a team) to establish how much we could answer.  The trend that emerged was that the original estimates where either half of the new estimate or double of the new estimate.  When they where halved, it was generally because we were able to negotiate some of the unknowns (the ignorant areas) to a future sprint with the product owner.  In some cases, the product owner was equally ignorant, and was reacting to the “business wants the feature” pressure.  When they were doubled, it was so much more was discovered than originally assumed.  At the end of the session, we always asked the meta-question “If we answer all these questions sufficiently, will we be done?”.  I call this style of working “test first backlog grooming” or “test first sprint planning”.

Often I discover more things I don’t know. Annoyingly, this happens in the middle of a sprint, but if it did not happen in that phase of work, then perhaps I was not digging deep enough.  When this happens, I just keep on adding them to my list of questions.  These new questions are raised at any time with others on the team, the customer or with whoever can help me understand a bit more.  Sometimes, it’s put on the table for negotiation to be dealt with at another time.  Nevertheless, standups still seem to be a good time to put new questions on the table, for discussion later.

There are several ripple effects of thinking about points in this manner – this notion of ignorance and shared knowledge gauges.

The first is about the possible shape of your sprint backlog. If you have deep understanding, then it is likely that you will be able to decompose complex problems into simple solutions, that take less effort.  The effect is that low point stories are in greater number in a sprint.

If you are highly ignorant, then the estimation points reflect that and there are more medium to high point stories in the sprint.

The second is about what you value in a story. You will find less value in the ontology of epics, themes and stories.  It is no longer about size of effort but degree of understanding or ignorance.  Instead, the shape of the product backlog is something that is constantly shifting from high uncertainty (big point numbers) to high certainty (low point numbers).  That’s what test first backlog grooming gives you.

The third is about continuous flow that is the nature of discovery.  When you work steadily at reducing your degree of ignorance, then you are steadily answering questions through answers expressed in code, and steadily discovering new questions that need answering.  This process of discovery is one of taking an example based on what you know in this moment and modeling it.  Then expanding that example with one or two more additional twists, and modeling that, and so it goes.

It also touches product ownership and software development. When you work in this way, then explicit estimation of effort becomes less significant.  Moments that have been earmarked as important  points in the life of the product become more significant.  Call them milestones.  These milestones are strategically and tactically defined, and become a dominant part of product ownership.  Software development becomes the act of having long running conversations with the customer.  Those milestones give context for the content of those conversations.  Ultimately, those conversations are then expressed as a set of organised thoughts in code.  If your code is not organised well, then perhaps you also don’t understand the problem or solution or both.

This is a long story for a short message. A high priority is to resolve the tension that exists in an estimation in the form of knowlege/ignorance fighting against effort.  When you release that tension through shared understanding, then you can deal with the tension that exists in the act of creating those significant milestones.  In my opinion, that’s the real wicked problem.

Product Ownership Webinar

On 12 May 2011 I will be joining Kent Beck and Henrik Kniberg in a free webinar hosted by SD Times to take a deeper look at product ownership as described by the Scrum methodology.  I think we all have a lot of questions, especially Kent, but I will also put forward some things that I have tried and some opinions of what I think should be tried. As usual, I welcome critical comment.
For a long time I have been wary of the way product ownership is “taught” in CSPO courses, and the way it is implemented in Scrum teams. I think the fundamental tension of product ownership is not being addressed.  So, at the heart of my talk, I want to explore the tension that a product owner needs to resolve and, maybe, some ways of resolving that tension.

Regardless of whether we offer workable solutions, I think the webinar will raise questions that are well worth discussing in larger groups.

A DVCS does not reduce problems in code

Let’s get this out of the way quickly.
Using any VCS requires you to do merges. The more frequently you commit, the less merge pain you have. With a distributed VCS like git or mercurial, you also have to fetch from a remote repository, merge and push. Merging does not disappear, so deal with it.

Why am I stating the obvious? Because I have seen a couple of cases now where people have switched from centralised VCS (cvs, svn style) to distributed and believed that it will help them reduce problems in code.  The thing that helps you reduce problems is tests. Running your tests under a CI server just creates a tighter, automated feedback loop. Of course, you have to update/fetch, merge, commit/push for your CI to be useful.

Hmmm, did you see what flashed by?  Your CI server does not give you a tighter feedback loop; it is frequent commits that gives you a really tight feedback loop.  The CI server will sit idle for as long as you don’t commit and push.

Rolling out a methodology is about design

Implementing a new methodology is a painful exercise.  Lots change, lots break, and lots of so-called “colateral damage”.  I have tried implementing new methodologies including XP and Scrum many times.  I have also witnessed a lot of attempts by other people, and been involved while others drove the initiative.  Every time, it has lead the organisation into a disruptive, stressful state.  The most common position taken by the implementors is:

Of course it is disruptive.  That’s part of the change-process.  We all knew that from the moment we started. How else do you think it’s going to get better?

In the past, I’ve been guilty of the same.  The end result is that I am left unsatisfied and unfulfilled, and so is the organisation. Yes, it may eventually get better.  Eventually I got sick of taking this high road.  Well, it only took two such situations a long time ago to realise that I was messing up royally.

In my quest to do things better, I drew inspiration from test driven development and dealing with old, messy legacy code.  Three very distinct things which are rooted in software development and changing legacy code is very, very apt.

  1. Rolling out a methodology is at the implementation level.  So, was there a design for the implementation in the first place?  Implementation without design always ends up in a mess.
  2. Even if we abstracted the design from one implementation, does the design support all implementations?  “Similar” does not equate to “same”.
  3. The existing methodology has a set of protocols by which the organisation functions, while the new methodology introduces a new set of protocols.  Just dumping down the new protocols is the equivalent of rip and replace – the grand rewrite without care for migration.  Is this the only way?

So, taking inspiration from code, here is something that you can try when attempting a new rollout.

Understand the existing implementation. Use a test based approach to concretely discover existing protocols within the organisation.  This may be as simple as playing out what-if scenarios that test the communication pathways.  Keep your eyes wide open for seams or boundaries.  These seams are candidates for incision points for introducing a new protocol facing in towards the area under change while honoring the old protocol facing out towards the other side that should not be changed (yet).

Design, design, design. Once you understand the existing protocols and how they behave under certain scenarios, you switch to design mode.  Look again at the dependency graph within the organisation for a particular protocol.  What is affected when a certain event occurs?  Then look at your candidate seams and incision points and design your wedge.  It may be a transformer that completely modifies information as it crosses over the seam.  Maybe it’s a buffer with a finite capacity that slows things down and trickle feeds info to the other side.  What about a filter that removes some info?  How about something that just decorates existing info with a just a wee bit more that is tolerable on the other side.

This design is mostly about designing feedback loops.  As such, you need to consider the temporal and synchronous aspects of feedback also. What is the expected latency when I stick in this new wedge?  Will it take one week or one day or one hour when something crosses this boundary?  Do we send off some info and wait for a response on the same pathway, or do we get informed of which other pathway to regularly check for the response?  Perhaps someone gets nudged when the response arrives.

Implement it test first. While it may seem like a lot of upfront work is necessary to get the ball rolling, it can be done in tiny steps.  You don’t need to fall into the analysis hole when looking for seams.  Nor do you need to get stuck looking for the perfect design.  It is better to remain concrete for long periods of time, than speculate at possibilities.  Doing a little bit at a time with some small tests helps you keep both feet on the ground.  For example, you want to switch from the old protocol of reporting progress with budgeted vs actual to burning down story points.  Some people still need to use the budget vs actual report and it is inefficient to have to maintain both (not to mention not DRY at all).  We need a way of transforming from burn down to budget vs actual.  Think about candidate tests that will shift you towards that goal?  Maybe it’s “I should be able to take existing budgeted tasks in a time frame and map it to tasks on the burndown”.  Perhaps it is “I should be able to introduce new time frames on the budgeted vs actuals that synchronise with new sprint boundaries”.

These are just some things that I think and try out.  It comes from me being sick and tired of subjecting people to stressful implementations and being party to messed up implementations too.  It’s just too easy to blame someone or something else for our own ineptitude.  I don’t take the high road anymore.  I take the middle road.  It doesn’t travel straight, and has branches with dead ends too, but it leaves me a lot more content.