Live with it for a while

Before I rush off and refactor my code, I like to live with my code for a while. The refactoring I do in the TDD cycle is to get rid of trivial duplication and, perhaps, some better naming. I deliberately delay extracting methods and classes and pushing up or down. For me, those are quite important design choices, and I want to make those decisions only when I have a good understanding of my problem.

What do I mean by “live with it for a while”?  Literally, I leave it alone and move along to something that is in it’s vicinity.  I choose something that is close enough that will need to interact or modify the “living-with” code.  This new use case, scenario or question is to further my understanding of the problem and  I choose it deliberately. If it turns out to be tangential, I don’t sweat it. I just pick something else that will move me closer.  The fact that my first choice was poor is always valuable insight into my lack of understanding of the problem.

Aside: The
simplicity of my solution is directly related to my depth of
understanding.  The deeper I understand the problem, the simpler I can
potentially get.  Shallow understanding leads to more complex solutions. This takes time, and “living with it” gives me freedom to play with the problem from several angles.

I don’t mean “ignore it after a while”.  Ignoring it is like noticing a crack on your lounge wall and after 2 weeks of doing nothing about it, you don’t see it anymore. So, living with my code is not giving myself permission to be sloppy.  It’s deliberate choice to look for a better, simpler solution as I increase my knowledge. Once I have a bit more knowledge, I can start making more adventurous design decisions.  I think it’s almost impossible to find a simple solution if you don’t have deep domain knowledge.

It means that I refactor a little at a time frequently.  Even if I know my code is not clean, I’d rather have a pulse for my code, and let it beat weakly.  All I’m doing is constantly looking for little things that will make that pulse beat a bit stronger.  I now realize that I refactor a little at a time, but almost continuously. I’m not searching for clean code. I’m searching for markers in the domain that confirm or refute my understanding.  The clean code will come with time.

Living with my design for a while has saved me lots of time, especially when I’m not confident of my knowledge of the problem.  Give it a try and let me know whether it works for you too.

What’s the point in Scrum?

Scrum people like to use points for estimating and measuring velocity.  I won’t go into detail about how points work and how to play those poker estimation games.  Just search around and you will find a ton of stuff.  So, back to this points stuff.  I have a divided relationship with the humble point.  I like it when a team switches to using points for the first time, because it gives them a chance to think a little bit deeper about what they want to do.  I don’t like it when we start inventing rules around points (and you can lump guidelines and best practices into the rules pot too).  When the rules appear, the thinking disappears.
In every team trying Scrum, there is bound to be a rule about points.  I dare you to put up a hand and say you have none.  These rules are things like “We can’t take anything over 13 points into a sprint”, “Our epics are 100 points”, “The login screen is our baseline of 3 points”, “Anything over 40 points must be broken down”.  So, I double dare you 🙂

Sprint backlog shape with high shared understanding

I have different view of the humble point. A point may seem like a one dimensional thing, but it has a some facets built into it.  One facet is the “amount of effort to build something”.  Another facet is “amount of ignorance” and this has an inverse – “amount of shared knowledge”.  Sometimes I find it useful to make a judgement based on what I don’t know as opposed to what I do know.  Regardless of whether I choose to view the cup as half full or half empty, I cannot estimate effort to build something based upon what I don’t know.  So, effort tends to track the amount of knowledge, not ignorance.  As knowledge increases, my ignorance decreases and each point starts representing more and more of pure effort.

However, if I am in a state of complete ignorance, then it is completely impossible for me to make any judgement on effort to build.  I’d be simply speculating.  What I can do, though, is create a time box to explore the unknown so that I can start moving out of my state of ignorance.  This is also an estimate and I am not making an excuse for non-delivery either.  I need to understand some things and also show my understanding in some code.  Yes, the code that I produce may not have a visible user interface or some other convenient demo-friendly stuff, but I need to carefully plan my sprint review to express my understanding.

It’s all about gaining a SHARED understanding. This understanding is body of knowledge that I have learned which I need to confirm with others.  This act of confirmation can happen in several ways.  I can have a conversation and explain what I understand, I can draw a blocks and lines picture, or show a spreadsheet, and so on.  Regardless of the method of communication, I still use the opportunity of discovery to express my understanding in code as tests.  Another powerful way of expressing my understanding is to write out a story and a few scenarios.  Using BDD style grammar can be a great way of concisely expressing some things, that can be easily shared.  Yes, you heard me correctly – as a developer, I write the stories and scenarios.  When I am given a story and scenario by someone and asked to estimate, then I am attempting to estimate based on  another person’s expression of their understanding and my assumed understanding.

In a recent discussion with Jimmy Nilsson, he said that he prefered to call scenarios “examples”.  That really resonated with me.  I also do a lot of discovery by example, and then gradually introduce more a more into the examples, as I get more and more confident of my knowledge.

How do I know how much I don’t know? That’s a tough question.  What I do comes straight out of my TDD habits.  I create a list of questions – my test list.  For some questions, I will know the answer easily, some not all, and some are debatable.  The more that I can answer, the better I can estimate effort.  I can then turn the questions that I can answer into statements of fact.  The more facts I have, the less ignorant I am.

Recently, I worked with a team that wanted to get TDD going, and the most significant change that I introduced was in backlog grooming and sprint planning.  During these two ceremonies, we (as a team) threw questions madly at a requirement, regardless of whether we knew the answer or not.  We then worked through the questions (as a team) to establish how much we could answer.  The trend that emerged was that the original estimates where either half of the new estimate or double of the new estimate.  When they where halved, it was generally because we were able to negotiate some of the unknowns (the ignorant areas) to a future sprint with the product owner.  In some cases, the product owner was equally ignorant, and was reacting to the “business wants the feature” pressure.  When they were doubled, it was so much more was discovered than originally assumed.  At the end of the session, we always asked the meta-question “If we answer all these questions sufficiently, will we be done?”.  I call this style of working “test first backlog grooming” or “test first sprint planning”.

Often I discover more things I don’t know. Annoyingly, this happens in the middle of a sprint, but if it did not happen in that phase of work, then perhaps I was not digging deep enough.  When this happens, I just keep on adding them to my list of questions.  These new questions are raised at any time with others on the team, the customer or with whoever can help me understand a bit more.  Sometimes, it’s put on the table for negotiation to be dealt with at another time.  Nevertheless, standups still seem to be a good time to put new questions on the table, for discussion later.

There are several ripple effects of thinking about points in this manner – this notion of ignorance and shared knowledge gauges.

The first is about the possible shape of your sprint backlog. If you have deep understanding, then it is likely that you will be able to decompose complex problems into simple solutions, that take less effort.  The effect is that low point stories are in greater number in a sprint.

If you are highly ignorant, then the estimation points reflect that and there are more medium to high point stories in the sprint.

The second is about what you value in a story. You will find less value in the ontology of epics, themes and stories.  It is no longer about size of effort but degree of understanding or ignorance.  Instead, the shape of the product backlog is something that is constantly shifting from high uncertainty (big point numbers) to high certainty (low point numbers).  That’s what test first backlog grooming gives you.

The third is about continuous flow that is the nature of discovery.  When you work steadily at reducing your degree of ignorance, then you are steadily answering questions through answers expressed in code, and steadily discovering new questions that need answering.  This process of discovery is one of taking an example based on what you know in this moment and modeling it.  Then expanding that example with one or two more additional twists, and modeling that, and so it goes.

It also touches product ownership and software development. When you work in this way, then explicit estimation of effort becomes less significant.  Moments that have been earmarked as important  points in the life of the product become more significant.  Call them milestones.  These milestones are strategically and tactically defined, and become a dominant part of product ownership.  Software development becomes the act of having long running conversations with the customer.  Those milestones give context for the content of those conversations.  Ultimately, those conversations are then expressed as a set of organised thoughts in code.  If your code is not organised well, then perhaps you also don’t understand the problem or solution or both.

This is a long story for a short message. A high priority is to resolve the tension that exists in an estimation in the form of knowlege/ignorance fighting against effort.  When you release that tension through shared understanding, then you can deal with the tension that exists in the act of creating those significant milestones.  In my opinion, that’s the real wicked problem.

Delusion: A firm belief despite contradiction in the face of reality

DevDays 2010 in Cape Town yesterday was slick.  Very slick.  It’s always slick.  Those guys really know how to put an a good show.
Most of the speakers are good.  Most demos were good.  Most jokes were funny.  The food was mostly good. The mood was mostly good too.   And the MS fan club was mostly impressed.  And most noobs were converted for ever.  And the improvements were mostly apologetic of the earlier short comings.  And code that you were promised you would write was mostly minimal.  And most absent was the word design.  Most ignored was TDD.  And most content presumed that we are dumb ass developers that don’t care about good code, good tools, good software.
If it was not for Bart de Smet’s two sessions on Core .NET 4.0 Enhancements and Language Enhancements in .NET 4.0, the day would have been mostly wasted.  Bart gave me a glimmer of hope, not for MS but for the manner in which he assumed we are not moron developers that can’t think.
The EF4 code first demo was explained completely as if the classes in the model are no different from entities in a table.  You may be have slides with the words “code first models”, but if you don’t actually do it for real, then you’re just leading dataset happy, marginally object oriented developers further away from the truth.
I understand that it’s a marketing game but, come on, MS South Africa, at least pretend that we are capable developers that care about being professional.  We care mostly about design. Mostly about clean code. Mostly about quality.  Mostly about getting projects done on time, within budget and mostly maintenance free.  We care mostly about being agile, being able to refactor code continuously, being able to test first, and not tossing code downstream.
Ahmed’s ping-pong of bugs is so irrelevant, when the developer is test-first infected and the tester is actually your continuous integration server.  Mostly we are developers that test our own code.
Glimmers of hope
– IE9 has developer support in mind
– EF4 has code first, but still so far from being a decent ORM
– DLR has a potential sweet spot

DevDays 2010 in Cape Town was slick.  Very slick.  You guys really know how to put an a good show.

Most of the speakers were good.  Most demos were good.  Most jokes were funny.  The food was mostly good. The mood was mostly good too.   And the fan club was mostly impressed.  And most noobs were converted forever.  And the new features were mostly so good, apparently, you won’t have to write so much code anymore.  And the most underused word was design.  Most absent words were TDD, refactoring, quality, and clean!!

I understand that DevDays is a product showcase but, come on, MS South Africa, at least cater for the entire spectrum of developers, just a little bit, and in a responsible manner.  How about pitching content that shows that you do care about design, about clean code and quality.  How pitching the new features in a way that shows a trend towards agility, to being able to refactor code continuously,  to test first and other vital aspects of professional software development.

Let me give you just an example to illustrate what I mean.  The EF4 code first demo was explained completely as if the classes in the model are no different from entities in a table.  Even the language used was “entities” and “keys”.  I don’t think I heard the word “class” or “object” once!  You may have slides with the words “Code First Model”, but if you don’t actually do it for real, then you’re just leading dataset-happy developers that are marginally object-oriented further away from good code and good architecture.  You need to explain why it’s better:  that it promotes better object orientation, that POCO models disconnected from an ORM can be done test-first, and you can evolve your model, instead of designing up-front, blah, blah, blah.

Another time there was a ping-pong table with a developer at one end and tester at another with a bug being batted between them.  That pulled quite a laugh from just about everyone.  While that is reality in many organisations, there are many of us that test our own code and deal with our own bugs.  The tester that we toss our code to is our automated continuous integration server.  Tossing code downstream when it’s too late for reprise is not very professional.  How about focusing on the testing tool, as opposed to pitching it in a manner that makes everyone believe that dealing with the bug downstream is just the way its meant to be.

Sure, I know that you need to show off the latest cool things and evangelise your products, but there is a sector of developers that you are blatantly ignoring.  It is the sector that is, perhaps, the most influential amongst other developers. We are those developers that value our craft of software development.  We evangelise the craft and the value that it brings to our lives, our teams, our projects, our clients and our organisations.

Perhaps I am just delusional.

Oh well, so long and thanks for the fish.

Test First TDD

I think that TDD is getting bastardized.  If you happen to use a Unit testing framework, it does not mean that you are test driven at all.  TDD is about test first to drive the rest – design, clean code, feedback, quality, and lot more.  Using a testing framework is easy.  Being test first driven is really difficult.  You may start off with the mechanics and focus on the cadence, but you only feel the value a lot later – when you have woven it as an attitude into your fabric of thinking.
That’s why I’m giving the TEST FIRST TDD course next week.  If you want to go beyond just learning about an xUnit API and step on the path of a personal journey to changing the way you create software, then come along.  I don’t have miracles but I can do better than just shining a light.  I will step into the darkness with you and help you move towards the light.

Driving through a red light can kill you

The other night I was driving home quite late from the airport.  For that hour, the roads are quiet and it’s a relaxing drive home that gives me a chance to think back on the the day’s events.  At a red traffic light, I stopped, but some idiot in the lane next to me rushed straight through.  You know what happened in the next 5 minutes.  Lots of honking, screeching tires and near crashes.  Fortunately, there were not accidents and nobody got hurt.
Really?

No!  Even though there wasn’t this big crash, someone did get a fright of their life, even I as an observer got a fright. And someone else did continue their journey quite shaken and certainly not in their same frame of mind.  The only person that seemed to be least affected was the person who jumped the red light.  But I am speculating, maybe he was under pressure and really needed to get to the end on time, and so he rushed ahead and saw in his rear view the potential destruction he left behind.  And maybe he regrets his decision.  But he certainly did not see how his actions affected the other people that were close by, myself included.

Driving through a red light can kill you and you can kill other people too.  The same thing happens when ignore the red light from your tests.  You can hurt yourself and you can hurt other people too.

Just think about that red light at the traffic light and in your code.  It’s telling you so much.

  • It’s telling you to slow down a bit.
  • To look around.
  • To think about the moment.
  • Don’t focus on the end, you will get there in good time.
  • If you ignore the red light, other things will break and other people will get hurt.
  • Maybe you don’t know why you must stop, but you need to stop first, before you can find out why.
  • It’s telling you to do the right thing, not the rushed thing.

Driving through a red light is another suicide pattern that I will add to my growing list.

And TDD is also part of ubuntu coding.  I am not telling you anything new.  I’m just telling you the same thing from another perspective.

Fast Track to Domain Driven Design

I finally got out of neutral and pulled together the first public offering our Domain Driven Design courses in Cape Town, South Africa.  Normally we give these courses on-site with people on the same development team but I thought it may be fun and inspiring to open it up to everyone for a change.  Now I’m all excited again and really looking forward to a diverse mixture of people. Hopefully, I will see some old faces and lots of new people.
The one thing I can tell you is that the course is a very immersive experience.  I really hate lecturing but I enjoy probing conversations and that’s how I give the course. I don’t have answers to the practical work and concerns are addressed as we go along.  As a result, the day takes unexpected turns and routes.  But in the end I get you to the right destination.  Come along; you will leave exhausted, but inspired!

Take the Fast Track to Domain Driven Design

about the coursecourse contents / should you attend? / register for the course

factor10 has expanded its services in South Africa to include our advanced and
expert level courses aimed for the software professional.  On September 8-9, 2009,
we will be offering a fast track to DDD for Architects at the BMW Pavilion in
Cape Town.

zz485d34cf

Who should attend?

This course is for software professionals that want to take the right steps towards
advanced and expert levels in their careers.  Register for this course if you want to …

  • learn more than just another syntax and set of tools
  • write software for large, long living systems
  • increase the quality and maintainability of your design
  • design high quality models and use code to represent those models effectively
  • develop applications with a good APIs
  • add a design edge to your skill set

zz3caeb696

Why should you learn DDD?

More and more developers and architects realise that learning every detail of a new
API just isn’t the way to deliver the best business value. It’s such a tough balancing
act; focus on the solving the business problem and focus on building working software
with your frameworks.

One way of taking a big leap in the right direction is to learn and apply domain driven
design. It is definitely not abstract and fluffy; it deals a lot with the code also. DDD
leads us to focus on understanding and to communicate that understanding very well;
in language, in design and in code. You will shift your focus away from designing for a
technology, and you will learn to design for the business domain; to the core of the
problems and solutions. Those are the most interesting parts and what your users
and customers really care about.

about the coursecourse contents / should you attend? / register for the course

Writing Specs is Writing Code is Designing

A team that I am coaching has settled on using BDD stories and scenarios for describing their requirements and specifications.  They’ve also chosen cucumber as their acceptance testing tool.  All well and good, but they are making very slow progress and seem to be really struggling with the change in workstyle.  I think I’ve spotted the reason for this.
The feedback loop is missing.  They view the stories as a spec that has been handed down.  And they have not made the connection that spec writing is design work that is intended to clearly illustrates concepts in a domain.  It is a form of writing code.  But it’s just that this code is, maybe, non-executable.

Here’s my workflow and how I close the loop.

  • write story and scenario
  • Sketch a design if needed – helps when pairing to be on the same page.
  • Start writing test for scenario
  • ooops … test is getting complicated? stuck?
  • maybe the domain is not understood enough? Dig deeper, improve scenario, design (as needed) and continue writing test
  • or maybe the scenario was badly written? Ignore scenario structure, continue writing test.  Refactor scenario later.  We’re in deep discovery mode here.
  • get test to pass
  • refactor code
  • refactor scenario
  • … cycle the red-green-refactor until happy.

Acknowledging when you’re in discovery mode and knowing that you are allowed to refactor requirements is the trick.  Nothing is cast in concrete.  That’s why I like frequent feedback loops with tight turning circles.

No feedback loop, no progress.

BTW, I really don’t like explaining such things as flow-charts and sequences.  You got to find your own style.  It’s not a recipe or rules thing.  The above is something that is about as close to what I do but it changes when the need arises.  That’s also another key feature of being agile – adapt or die in the waterfall.

Measuring the Clarity of Requirements

The last two geeky conversations I had, stumbled upon the same thing – how do you measure the effectiveness of requirements in describing the business to the business and describing the specification to the developer?
So, I posed the question “How far away are you from executing your requirements?”. If you are going to go through various steps and stages to get to compilation and then execution, then every step is an opportunity for valuable information being lost in translation. If you can compile your requirements immediately then nothing will be lost.

Each additional step between requirements description and compilation and execution is an opportunity to confuse the user and the developer and everyone in between.  That’s why fully dressed use cases are not so effective as fully dressed behavior driven stories.  And that’s why BDD is very agile and a great asset in DDD and use cases just don’t cut it anymore.

Right now, my favorite tool is Cucumber.  I can execute the requirements and that raises the clarity ranking of my requirements super high.

Heck! We should have paid attention that day.

I’ve been doing some internal DSL coaching recently, and my coaching-partner and I have been working BDD/TDD style.  It turned out to be quite a nice experience overall.  We started with a Cucumber based feature that described the behavior of a typical DSL script and we drove it down into rpsec where we tested each little part of the script.  In the end, I achieved what I set out: the person I was coaching grokked the whole “code is data” idea behind DSLs.
Then I started thinking about what it takes to coach external DSLs.  I got really scared – parsers, generators, abstract syntax trees.  Those were things that most people wanted to forget about the day after they wrote the exam for that horrible semester course in university.  What makes it worse is that I never did that in university – I did Electronic Engineering and we spent our coding time figuring out how to do Fast Fourier Transforms on digital signals.

But when I look at the progress that is being made in language workbenches to help us create DSLs, then I reckon we should dust off those books and start paying attention again.  Some of these language workbenches still leverage a host language such as Java or C#, but the act of using a structural editor that edits the AST directly is strangely weird.  But, some refactorings are just not a problem anymore.  For example, if you are changing the node on a tree, then all the references to that node are automatically aware of it.  Compare that to a text editor where you need refactoring wizardry in the tools to make sure all references are updated neatly.

I think that language workbenches may be a great tool for coaching DSLs because the things that made your head spin during that lex and yacc week, are made a whole lot simpler.  You can focus on designing the language and walk into ASTs with less fear.  But, be warned — language design is not easy, and you still need to know about ASTs and parsers and generators.  In fact, just write a parser and a generator and it will be a learning experience that goes beyond DSLs.  Better still, do it BDD/TDD style.

The relevancy of language oriented programming is just going to continually increase.  Those previously “irrelevant” courses are important again.  Watch out, the gap just got wider.

Øredev Presentations

My presentations from Oredev are finally available.  After working through almost all the export options on Keynote, I have settled on QuickTime as the distro format.  The “flying code” in the aspects presentation worked out best with QuickTime.  Note that it’s not a continuous playback and you have to click-through each frame.