Get up, Stand up.

I first got paid to write code in 1992.  The few years before were the heavy days of mass protest against apartheid.  Having a common enemy was not what drove people to change.  It was having a common belief that we are all equal and deserve to be treated equally. It is that simple.

In the years that followed, my career always surfaced a blatant fact.  I was always the minority in a white, male dominated industry.  In my first job I was hired by people that also despised apartheid.  For that I am grateful because it gave me a chance to learn without fighting for place.  But the places in which I plied my trade were dominated by white, male developers and managers with an attitude of superiority by entitlement, birth, law and any other decree.

And today I am still the minority in a white, male dominated industry.  I don't need hard stats and precise surveys when it is patently obvious from the demographics at community events, conferences, start-ups and cubicle farms that black software development has not gained a foothold in our industry.  It is especially so in Cape Town, and only marginally better in Johannesburg.  I can't speak for Durban because I haven't worked there since 2005 but it was not any better then.  It is not just South Africa.  It is the same in Europe and North America.

 There are at least two sides to any story. Here, the one side is with corporate South African software development as a conduit for opportunity or suppression thereof, and the other side lies with the black software developer with messed up priorities.  By the way, when I say "black" I mean "not white", and I everything I say is a generalization because I believe it is in the majority.

In my experience corporate South African software development perpetuates apartheid.  There are official policies and frameworks for employment of people and for furthering their careers.  There is nothing wrong with these frameworks. But like all rules, how we use them matters.

A few weeks ago I had another black developer share his entrance into a prominent consulting company.  He was offered an internship whilst a white male of equal qualification was offered a full contract.  When his internship was over, he was not offered a contract on the grounds that his line manager did not like his "work ethic".  Really? Who's work ethic needs to be questioned?  Today, this developer is employed as an equal in more progressive organisation with a highly respected brand that is determined to breakdown apartheid in South African software.

I am yet to meet a black software developer that is leading a team, making the call on design decisions.  Along the way, I have met many blacks in leadership positions. Many are tokens and lack the leadership and technical skill to justify their position.   This is also the fault of these individuals, exploiting the rules for self gain.

The majority of black software developers are either doing "maintenance" or being instructed on how to write to specification or requirement with a full blown design being given to them.  I have met many black developers who I've seen argue their design ideas only to be later classified as not being "team players".  This also occurs in the sugar-sweet agile "we-love-transformation-i-love-my-job" work places.

What gives credence to the "inferior" black South African software developer is that black software developers are apathetic.   I find it difficult to identify with someone who voluntarily accepts being treated as a second class citizen.  I can understand the risks of 20 or more years ago, but life today is different.  We have a voice and we have a deliberate choice of the battles that we can fight.  This is mental slavery all over again.  And to some extent this mental slavery is self imposed.

I can appreciate that someone with a history of poverty that is offered a monthly salary that their parents could only ever earn over several years, is driven by a more basic need and deep empathy.  It goes horribly wrong when that is all there is to it.  You earn your dough, but you don't earn your stripes.  I know several black developers that have doubled or tripled salaries in a few hops inside of two years.  These developers do nothing but give credence to the ingrained belief that black developers are not skilled.

I am fully aware of black developers working from a weaker maths and science base.  But I am also aware that there are white software developers that also don't have a strong maths and science base.  I have wondered about this difference: two people with equally weak programming prerequisites yet the black developer struggles and the white developer excels.  I don't think spoken language is a significant variable here.  The one thing that I have observed is that the white developer receives more opportunity and attention than the black developer by his white leadership.  The black developer in turn believes that they are inferior, and the lack of opportunity and attention furthers erodes self-belief.  Self-doubt is by the far the greatest obstacle to learning.

If you have read this far, there are several among you that have already formulated counter arguments, specific cases that justify your position too.  I am open to hear your side.  The problem with this blog post is that everything until now is just background to create context to put forward my belief.

I believe that there is a wealth of talented software developers in Africa that understands our history and our future. I believe that as a Pan African software community we are capable of designing frugal solutions for our people and for the world.

To achieve this, we must break free of our own mental slavery.  We need to build our own solid foundation that is unshakable at our core.  We must equip ourselves as masters of our craft.  We must let our voices be heard in Africa and the rest of the world as voices that speak substance.  And we must return to be with our people as no more than just people.  Our first step is to rid ourselves of apartheid in software development.  Like Marcus Garvey said and Bob Marley sang "none but ourselves can free our mind".

I dream that the next though leaders of software development will be African.

Reflections of BDD Stories

There was an interesting discussion on the AgileSA Linked-in group around the use of BDD stories, and whether they should contain technical references or not.  I found myself saying that I don't mind having, for example, a login story. To help Kevin Trethewey get over the shock and horror of this, I reflected on how my use of BDD stories has changed over time.

I remember Dan North explaining BDD stories to me when it was just a thought in his own mind.  That was around 2005 or 2006 and I remember being so inspired by the simplicity of the BDD grammar. So, seven or eight years later, let me share how my use has changed.  And, Kevin, I hope you enjoy your own journey too. It's a lot of fun.

On CRUD.  I agree that CRUD is bad, blah, blah, blah.  But there are times when CRUD can a valid, and reasonable design choice. I don't discount it, but it is not my first choice, and it is very rare for me. Oh, I sometimes just use it because I don't know anything else about the domain. Once I discover more, I noticed that those CRUD things, quite naturally, fade away.

Who is the best judge of a story?  The customer is unlikely the best person to articulate these stories, nor judge the quality of the story. I have to guide them and extract that. I now ask the following questions.

• Who are they?
• What do they need?
• What do they think they need?
• What do they really want?

What does the story describe? Of the above questions, the last is the most powerful for me.  It balances my perspective. It stimulates creativity and moves me from the problem space to solution space. The story then exists in the solution space; i.e. it is now reflects a design intention, not a requirement statement in the problem space.

BDD stories are great conversation artifacts. It's like a book on a coffee table. It stimulates conversation.  It is of same value as using a metaphor. In conversation with the customer, the story is mostly about things in the problem space. In other words, it is an analysis and clarifying tool. I found that direct, literal and very early use of this analysis statement as an executable specification results can result in brittle tests.

On the use of technical references. When I'm working in the design space and writing design stories, then I don't mind if there is reference to a technical implementation such as a login screen.  At some time, I have to get concrete. I like getting concrete very early and abstract from that. It's just how my mind works. So, if there is alternative authentication mechanism (say, LDAP or ActiveDirectory), then it is just another concrete implementation. If the authentication choice is an exclusive one, then, the abstraction of proprietary authentication and ActiveDirectory authentication doesn't offer any benefit. So, I'll just go for one of those and the story on task board will make reference to the technical aspects directly. It's a great reminder of a design choice that is explicit to everyone.

Most stories start out as bad stories. My early stories in an unfamiliar domain are awful.  Like code, the story should exhibit single responsibility. That takes a lot of domain insight and discipline. Unfortunately, refactoring stories towards single responsibility is not trivial. It's not as simple as extract class/method/variable. The result is that my story based test suite is in constant turmoil longer than it is calm with a few small ripples. For this reason, I use the story grammar as a conversation piece, but not as code artifact.

BDD Stories on the backlog. To avoid confusion about when the story is in the problem space or solution space, I don't use BDD stories on the backlog. I prefer the XP style of a short phrase as a reminder of something to discuss.

On the use of outside-in style testing. I like outside-in to analyse the problem space, but I often find it equally valuable to evolve the design from the assertions inside-out.  I oscillate between the two perspectives rapidly and quite often. I think I'm searching for that harmony in perspective.  I then make it a choice to use the BDD story as an executable test for the outside in perspective. Often, though I find it unnecessary because I already have tests that reflect that perspective; it's just not using the BDD grammar. Yet, the BDD grammar was a starting point. I just am not fixated on the BDD grammar being the ending point.

On the BDD grammar. From a language perspective, the BDD template is a general grammar that can be used to express any domain.  Just like we can use a general purpose language to solve any problem,  the BDD grammar can be used similarly. Yet, we have learned that domain specific languages can be more expressive of a particular domain.  Equivalently, I keep my mind wide open, looking for a story grammar that is domain specific.  For example, in a time sensitive domain such as investment portfolios, I might extract a grammar that expresses time as a first class citizen. There won't be a "When" clause. I might have something like "At time t(0), the portfolio capital amount is..., at t(n) the portfolio has ..., at t(n+1) the surrender value should be ...".

Remember, these are just reflections and observations about myself. Please don't treat it as a gospel. You have your own journey that takes different pathways. Just enjoy those excursions.

Are we there yet?

This is an interactive slide deck of my talk at Agile India 2013.  I explore the interplay and gap between the architect and the manager. Along the way, I point out some agile landmarks that we need to stop and reconsider. I will also highlight some tweaks that I make to find balance, finding a thread to narrow this divide.

Split stories as a design activity

"A story should be big enough to fit into a sprint, otherwise chop it up until it does" -- this is advice that is more or less given to Scrum teams during planning or backlog grooming. The problem is that this is not easy to do.  My friends at Growing Agile describe a few ways to achieve this (see their blog post Breaking Down User Stories). These techniques are not wrong in any particular way, and they will certainly result in smaller stories.  However, these are what I call "mechanized" techniques. When I've been mechanical about splitting stories, I've always ended up with weak fracture points in the problem space.  So, I prefer to look in the solution space for boundaries that promote or retain conceptual integrity of the software.

Below are just three techniques that are quite commonly used by Scrum teams.  I steer away from them at all costs.

• CRUD.  I find that thinking in terms of these database operations removes a lot of the richness in the domain.  Instead, I think about the life cycle of things.  For example, there is no CRUD for an invoice.  Instead a customer buys something which means that a sales person issues an invoice. The customer pays the invoice. Perhaps, a debtors clerk requests payment for an overdue invoice.  These are all different things that can be done with an invoice at different times in its life.  Note also that "creation" is a very special case in any life cycle, and to bring something into existence that maintains integrity is quite an effort.

• Dependent Stories.  I try to break all dependencies between stories.  I've found that looking to create "stand-alone" stories results in some very deep and powerful analysis of the domain.  Inadvertently, you will crack open a crucial part of the domain.  Often the concept which holds several stories together in sequence turns out to be orthogonal to the original stories.  For example, there is a workflow for invoices (issue, authorise, pay, remind, resend, etc) that can result in several dependent stories.  Alternatively, we can model the invoice state (and operations allowed for each state) independent of the sequence(s) of states.  Now we can build software that deals with specific sequences, independently of the operations for each state.  This separation can lead to such powerful discussions with the domain expert.
  

• Job Functions: I've never found job functions to yield useful modules.  Extending the invoice example above, a job function breakdown could be on sales (create, authorise), debtors (record payment, payment reminders), customer service (credit notes), marketing (cross sales campaigns).  Now each of those job functions work with invoices in some way, but the conceptual integrity and cohesion is stronger around the invoice and its states.  Designing good modules is by far, the hardest part of any software design effort.  Get it wrong and it will hurt.  More often than not, it is just too costly to try to create new cohesive modules from code that is already laid down along job functions (or any other weak boundary criteria).

There are significant consequences to splitting stories in the solution space.

• The product owner just needs a simple phrase or sentence that describes the problem space, but remains part of the feedback loop for the solution space stories.
  
• Backlog grooming becomes an exercise in understanding the problem space. 
• Sprint planning blitzes (one day or less) is not sufficient.
  
• To be effective, sprint planning becomes continuous; i.e. design is continuous
  
• Each story can (potentially) be released on completion
• Sprint boundaries (time boxes) become less important moments in time
• ... and you will tend towards continuous flow.
  

Live with it for a while

Before I rush off and refactor my code, I like to live with my code for a while. The refactoring I do in the TDD cycle is to get rid of trivial duplication and, perhaps, some better naming. I deliberately delay extracting methods and classes and pushing up or down. For me, those are quite important design choices, and I want to make those decisions only when I have a good understanding of my problem.

What do I mean by "live with it for a while"?  Literally, I leave it alone and move along to something that is in it's vicinity.  I choose something that is close enough that will need to interact or modify the "living-with" code.  This new use case, scenario or question is to further my understanding of the problem and  I choose it deliberately. If it turns out to be tangential, I don't sweat it. I just pick something else that will move me closer.  The fact that my first choice was poor is always valuable insight into my lack of understanding of the problem.

Aside: The simplicity of my solution is directly related to my depth of understanding.  The deeper I understand the problem, the simpler I can potentially get.  Shallow understanding leads to more complex solutions. This takes time, and "living with it" gives me freedom to play with the problem from several angles.

I don't mean "ignore it after a while".  Ignoring it is like noticing a crack on your lounge wall and after 2 weeks of doing nothing about it, you don't see it anymore. So, living with my code is not giving myself permission to be sloppy.  It's deliberate choice to look for a better, simpler solution as I increase my knowledge. Once I have a bit more knowledge, I can start making more adventurous design decisions.  I think it's almost impossible to find a simple solution if you don't have deep domain knowledge.

It means that I refactor a little at a time frequently.  Even if I know my code is not clean, I'd rather have a pulse for my code, and let it beat weakly.  All I'm doing is constantly looking for little things that will make that pulse beat a bit stronger.  I now realize that I refactor a little at a time, but almost continuously. I'm not searching for clean code. I'm searching for markers in the domain that confirm or refute my understanding.  The clean code will come with time.

Living with my design for a while has saved me lots of time, especially when I'm not confident of my knowledge of the problem.  Give it a try and let me know whether it works for you too.