Update on Grokking Functional Programming

Writing this book has been quite a journey.  Being a personal experiment in user experience design in a non-interactive medium has proven to be quite a challenge.  The constraints are significant from page size, typography and then to consider cognitive overload and leaps from concept to concept.  Yet, at the core I am still aiming for a pairing experience.  I want to create an experience that someone is working with me at the same computer and sheets of paper.

This is what is emerged as a workflow.

  • sketch out a plan for the chapter;
  • focus on what are the main takeaway points;
  • what pre-requisite knowledge is needed for the chapter; and
  • fill in some examples and exercise, pretty much faux-code

The gap that surfaced this week with my editors is that we swirl around the main takeaway points a lot.  The fundamental question that I now ask is “What is the one super power that we receive with this chapter?”.  That’s a hard question to answer and it forces me to ask very abstract and pointed questions about the main concept of the chapter.  Often, the answer is philosophical which forces me to reduce it to something practical.

Sometimes, answering the super power question has left me admitting “Well, that’s pretty unremarkable!”.  Initially, I was disappointed when that happened.  Now, I use that as a feedback to dig further.  It forces me to seek for my own deeper understanding.  The bottom line is that I am still hindered by my own ignorance.  It takes effort to break through ignorance barriers and it is not about pounding on the same door all the time.  It is about circling around and attempting to find tiny cracks to chip into.

In addition, I have now realised is that UX can only be solved after the philosophical and abstract are reduced to concrete.  Having tight UI constraints is actually a blessing for UX.  I now appreciate the limitations as it simply removes options not worth their bandwidth consumption.

Grokking FP JourneyWhere have I taken people on this journey so far?  On the right is what I’ve covered so far.  Next up are algebraic data types and then more in depth coverage of high order functions.  Very surprising for me is that this journey of constraints, user experience challenges, asking super power questions has led me to shifting this chapter on high order functions from being chapter 2 to chapter 7.  That was unexpected.

Keep an eye out for tweets on discounts on the book.  The latest promo from Manning is for 30 October 2014.  Using discount code dotd103014au at http://manning.com/khan will give you 50% off the list price.


A Book is Born

Grokking Functional Programming

Over the last year I have been spending some of my time writing a book on functional programming.  Finally, it is available on Manning’s early access program.  The first three chapters have been published and my commitment is to produce new content every month or sooner.

This has got to be one the most difficult pieces of work that I have ever attempted.  In many ways, this is an experimental book.  It takes my brain on two orthogonal paths.  The first is about the topic of functional programming (FP).  And the second is about cognitive learning.  FP is a difficult subject and large too.  I’ve attempted learning it several times and threw it to one side in frustration.  So this book is my crazy attempt to apply a few aspects of learning to a complex topic.  My hope is that it is an accessible book for most software developers.

Early and gracious thanks must go to the team at Manning for putting their trust in me on this experimental journey.  And to say that my editor has been patient is a gross understatement.

If you are going to read this book, please do provide feedback on the book’s forum.  I really want to know where you got confused, which sections are too fast paced, which pages made you take ridiculously giant mental leaps.

Update: Anybody can get 50% off the book by using the discount code mlkhan at manning.com/khan/.

Event: Inclusivity and Innovation in S.African Software

I am facilitating a panel discussion as one of Cape Town’s World Design Capital 2014 events, in collaboration with the IEEE.  Attendance is free.
This is the brief that I have given to the panelists.
Software development in South Africa is still a closed society.  It is also a consumer society.  It is closed because we are not inclusive nor embracing of our cultural diversity.  It is a consumer society because we are net importers of technology, fed by global technology companies motivated by profit in a third world market.  This is the lifestyle of software development – import ideas, import technology, and restricted participation. A desired future state is a lifestyle of software developers where we are net exporters of ideas, exporters technology and encouraging inclusive participation.  In this session, we want to explore why this future state is valuable, and what we need to do to achieve this.
And the panelists are …
  • Philip Copeman of Pink Software.  He is the custodian of TurboCash, most probably South Africa’s most successful open source software
  • Brain Leke of ThoughtWorks.  Brain is a long time champion of equality of demographics in African software communities.
  • Lorraine Steyn of local startup I’m Bored.  Lor has been in the trenches of helping young developers kick start their careers for over 25 years.

When, where …

  • Date: May 20, 2014
  • Time: 12.30PM to 1.30PM.  It’s deliberately being held over lunch, and it’s a free attendance.
  • Where: 3rd Floor, Bandwidth Barn, Woodstock Exchange
  • Address: 66-68 Albert Road, Woodstock, Cape Town (here’s the map)
  • Cost: Free

Why are we doing this?

… because it is an important for the sustainability of software development as a thriving sector, and this spurs further discussion and action too; and to a much wider audience.

… and because we should care.

James the Shoe Shiner

A few weeks back I was humbled by about 100 graduates.  They attended an event called Level Up hosted by ThoughtWorks in Johannesburg.  I was speaking last and everything that I wanted to say was said by the time I stood up.  So, I went impromptu and spoke about why we change jobs, that a focus on design will see you through in the world of software, and that life is a cycle and we will do well to do the same as is being done to us right now to start our careers.

Then I rushed off to the airport, and the content of my improv talk hit me fully with stark reality.  When the airport is quiet, like it was that Saturday evening, I know that business is bad for the shoe shiners.  Here was James, the only shoe-shiner around.  It was 6.30pm.  He finishes work at 7pm.  I asked him how much short was he of his daily quota.  It was a slow day, he said.  I opted for an expensive shoe shine so that James met his quota.  I know that if James didn’t meet his quota, it is taken off his weekly wages.

James, it turns out, is from the Eastern Cape.  He graduated last year with a Diploma in Electrical Engineering.  He came to Johannesburg to get a job.  It is a tough job market, but he was finally offered a job.  But he couldn’t take it because the job required him to have a driver’s license.  Now he shines shoes to save money to take driving lessons, then he enters the job market again.  That job won’t wait for him.

My talk from half hour before of why we change jobs hit me hard.  In John’s mind, the only thing he needed was an education, and everything will work out.  Then he discovers these hidden constraints.  He changes jobs – temporarily, he hopes.

This moment has lingered with me for weeks and it clarified three things only today.

  • Indignity is imposed, humility is a choice.  To make a person kneel before another to polish shoes is an indignity.  That a person has chosen to polish shoes as the needs to an end is humility.  The work we do is not what defines us.  We define our work.  I have heard several agile coaches telling people that if they are unhappy, then they should change their company or change their company.  Most have changed jobs feeling an indignity that was imposed upon them.  I wonder, though, whether life would have been better when that same “horrible” job was embraced with humility.  I know sometimes we have to flee.  In my life, when I fled too quickly, the real issue was never resolved.  It just surfaced some time later.
  • Living in the promise of the future or the longing for the past is to live in limbo.  The past ends in this moment, and the future starts in this moment too.  We live in the present.  That is part of being truly agile.  Just changing jobs because of something in the past or the lure of the future somewhere else is to lose what we have in the moment.  It is not enough, though, to live in the moment.  To be in the moment demands an awareness.  Adjusting a course based on awareness beats reacting to stimulus.  Reacting to a stimulus puts us into a fight or flight choice.  Adjusting with awareness keeps us in a contemplative place.  Agility is about continuous transitions, each so tiny, that we can’t see the jagged edges over time.
  • Unexpected constraints take us into uncharted places.  James needing a  driver’s license is his unknown constraint.  Now he shines shoes.  We don’t know who he might meet shining shoes that may take his life on a different path.  Similarly, the truly agile team will embrace an unexpected constraint and look for opportunity in the uncharted beyond.  The process religious agile team will try to reject the unexpected constraint, fearful of the great beyond – like anti-bodies fighting a virus.  To be truly agile, we need to just embrace the unexpected as it pops up.  I see many teams attempting agile software development by just tweaking settings on known constraints.  That works for a short time only.  Our  software, after all, is only one requirement away from being hopelessly invalidated.

One shoe at a time, James’ life is changing.  James is agile.  Working with humility, embracing the unexpected constraint of needing a driver’s license, living in the moment, being aware of an empty airport and short of his daily quota.  That is how we should build software – living in the moment with awareness, letting it change our life daily.

I believe this is possible.

Many teams co-operate, not collaborate

At Agile Africa 2013, I presented a session about lessons learned living through apartheid to democracy and how this manifest itself in teams in which I found myself during those times.  Each period has many little stories, all relevant to the way we behave when we work together, especially in quite diverse teams.  I’m not going to focus on these stories, but rather on the main theme of that talk.  The slide deck for this session is available here.

The first time I spoke about this publicly was in 2010.  It’s taken me a while to articulate this particular aspect.  This is the first time I’ve written about it and I want to share that.  It is my take on a concept that I find is handed out too easily in agile software teams – collaboration.  I have chosen specific words to describe different kinds of relationships.  My usage might differ from their dictionary meanings.

Now let us start with the first period – the days of apartheid.


In the apartheid years of S.Africa we had a distinct “us and them” political and social structure.  It was whites holding a superior position over blacks.  It was only beneficial to one – those that held the superior position.

I call this kind of relationship contempt.  There is no trust between people and no shared values too.  The only value system that matters is that of superior.  This is a case of oppression.  It is the oppression of another’s values and identity and of freedom of choice.  Hence, we feel imprisoned.  The consequence of this is the under development of the oppressed.

When have contempt, there is no chance of working for mutual benefit in any kind of way.  It is just enslavement.  In software teams, I’ve experienced this overtly, without any facade or subtleties.  Some have been extreme cases, reflecting the bitterness of racism entrenched in our bigger society.  More common, though, are subtle situations of contempt.  For example, a developer considers another developer as less skilled – stupid to be precise – then we have relationship of contempt.  Architects and developers, managers and team – these are instances of “us and them”.  We can have such titles, but it is the attitude that arises when we build a culture where one holds themselves superior to others.


In the early 1990’s South Africa entered a period of transition.  We were dismantling apartheid and building a new political and social system that ensured that we never go back to subhuman bondage.  This required enemies to find a way to work together for the survival of both.

I call this a co-operative relationship.  It is when both sides still retain their own values, but find a way to fulfil a common objective.  To achieve this common objective requires them to build trust.  I imagine trust as the payload carried in every action.

I sometimes explain it like this.  If we were renovating a kitchen, then the brick layer, the plumber and painter need to co-operate in order to complete the kitchen on time.  Each of them have their own work ethic, which is largely tucked away and insulated from the rest.  They need to agree on certain things like the bricklayer will complete the kitchen before the plumber can put in the taps, and the carpenter waits for the plumber before building the cupboards.  Should one not honour their commitment, then trust is destroyed because it affects the integrity of the others as seen through the eyes of the home owner.

This is the most common type of relationship that the majority of software teams practice.  We only need a co-operative relationship to build a reasonably efficient cross-functional team. Unlike contempt, people are no longer treated as inferiors.  Instead, everyone stands on their own pedestals of similar height.  The only thing that binds the team is the trust that comes from executing specific actions to achieve the common goal.

Most people call this collaboration, but I call it co-operation.  I can understand why we consider this collaboration.  It is the degree of trust that is necessary for this to emerge.  It is also reinforced by giving a team a name as a sense of common identity.  For me, that kind of naming leads to tribalism.  We gather to paint our faces in familiar markings, exhibiting the same brand.  Our behaviour becomes one of exclusivity than inclusivity.  It reminds me of the tribes that formed in the somewhat controversial book Lord of the Flies.  Pretty soon, we have intertribal tension.  This actually pops up quite quickly in multi-team Scrum implementations where all teams are working of a common backlog.  That backlog introduces real or invented dependencies between tribes, and so the tension starts emerging; just a shadow of first but easy enough to envelope all in darkness.

So, what then, do I consider collaboration?


After South Africa ushered in a new constitution, and successfully held its first democratic elections, the cracks started to appear.  We heard words such as “we need nation building”, “we are a rainbow nation”, and more recently “everyone can lead S.A.”.  All these words refer to one thing – we need to share a common identity.  That is what collaboration is for me – we have trust and now we focusing to share a set of values that exposes a powerful new identity.  It does not mean that we ignore or assume the presence of trust.  Our actions still carry a high payload of trust.  That never stops.

Building a shared value system is not easy, nor quick.  It is a challenge because each of us in this collaborative relationship have our own hierarchy of values.  Now we are being challenged to reconfigure personal hierarchies to overlap.  When we identify those intersections, then we are standing on the same pedestal.

When we work of a common platform – of even just one common value – then our basis for existence changes.  We stop becoming exclusive and starting behaving inclusively.  When a “stranger” walks in, seeking participation, then we look for that tiny tendril that allows them to plug into our shared values. Over time that tiny tendril grows stronger, and deep roots are planted.  Sometime the new person enlightens us and we expand our value system too.

What happens if that new person does not have that thin tendril that we can plug in?  This maybe a meta-value.  The one value that is necessary for collaboration to take root is that we respect all values; being prepared to explore the adoption of any value.  If we don’t have this single meta-value then we can’t bootstrap collaboration.  Without this meta-value we are unable to explore any value.  Collaboration comes to a halt.

The good news is that the each person does not have to have deep roots into the value system.  Tapping in lightly is enough.  The depth of the roots that are grown is tied to each person on that platform.  Those with shallow roots will move on to other teams that allows them to grow a deep root.  The period of time when they planted that root so shallow is still a wonderful time because we shared an identity.  That we cannot take away.  Inclusion is a way of growing.

When people anchor themselves deeply into the value system, we will find that those teams have a longer life. This is because we have a pathway to succession.  Having strong roots means that we create sufficient time for others to grow strong roots too.  In this collaboration, our sense of tribe is meaningless because the values will nourish and support generations of people.  Having a name becomes a label for the values that define us.

A closing thought agile adoption

I’ve tagged this with XP (extreme programming) as well as agile, and ubuntu – the zulu philosophy.  XP is what made me aware of the power of values in a team.  In my ignorance, I often attempted to instil values very early in a team.   I thought that I needed to build from a solid foundation.  It never worked.  I found that building a collaborative team is inverted.  We build the walls and the roof before we build the foundation.  The walls and the roof is trust that is needed, and the foundation is the values that are nurtured.

A collaborative team, as I describe it, is rare. It may take years to establish that foundation, with lots of corrections along the way.  That is ok.  That is the journey.  Enjoy it for exactly that.

Extreme Programming is as relevant now as before

Extreme ProgrammingLast year I met a few people that had never experienced a waterfall project.  Their first software project was with Scrum.   What stood out for me was that these developers had not read Kent Beck’s Extreme Programming.  I also discovered the same for several that were years into their agile development journey – never read the XP book.

Perhaps we are past the sceptical stage, and the notion of working in 2 to 3 week cycles is not considered so extreme any more.  XP may not be as popular as Scrum and Kanban at the moment, but it is still intrinsic to successful agile software development.

I confess that I find a lot of the XP practices are difficult and I have certainly not mastered any.  I think it might be simply that software development is just difficult.  Another reason may be that XP warrants that I change my approach, shifting my attitude to software development quite fundamentally.  That is the extreme part of XP for me, not the 3 week cadence.

Here are a few aspects of XP that I have grown to appreciate over time.

XP makes being a decent human explicit

XP has a lot to offer for the welfare of each person, and for our collective welfare too.  In a software world where we are learning to embrace our diversity, these insights are valuable for nurturing what is still a comparatively young sector.  Certainly, XP offers values.  However, just declaring a set of values does not mean that we live our values.  That’s where XP’s principles and practices kick in.  These exist to nurture those values.  Having a shared set of values and principles is essential in a team, because we use these to help us eliminate options and make decisions.  Value based decisions always seem like choosing the harder option, but the payback is often greater.

XP is about fine grained feedback

XP introduced me to the idea of feedback and made me aware of the theory of constraints – that we just shift problems around when we introduce or remove constraints.  Now we see the rise of lean which focuses on one piece flow, not short time cycles.  That is a different constraint and we have new feedback points like pivoting and waiting until  the last responsible moment.  All these require explicitly designed feedback paths.

Producing a piece of software with one piece flow means that we must bake the feedback into the software we produce.  For example, if we redesign the landing page for a web site, then we need to bake in measuring page views and time spent on the page as part of the release.  Without explicitly thinking about how we receive feedback, we just end up producing a chunk of software that has very little value.  Once we have feedback loops baked in, we are back to observing the shifting of constraints.

XP practices such as continuous integration, collective code ownership, simplifying vigorously, small and frequent releases all aid in building precise feedback loops.  These practices directly support one-piece flow, devops, and continuous delivery.  Yet, most people are unaware that some of these practices are there to increase collaboration.  For example, when there is a problem in the build pipeline, then people will swarm to fix the problem.  It is the high signal conversation after the failure that is valuable.  XP helps increase the signal to noise ratio in our feedback  loops.

XP is not just process centric

As a developer, I am yet to find a set of practices that appeal directly to my work at the most basic level – working with code.  I’ve noticed that many teams that embark on an agile journey tend to focus more on an agile process, than their code base. After a while their code base becomes their biggest bottleneck.  Not obvious is that the potential to regress to a waterfall-like development process is high.  When we have little control over the code base, then we may well reach for more grand planning upfront, less frequent and larger releases, and longer testing cycles.

The code base becoming a bottleneck is not an event that suddenly happens.  It is an accumulative effect.  It sneaks up on us.  Some symptoms are a drop in throughput; it becomes more painful to build, release and deploy; and introducing a new feature needs design contortions.  I’ve referred to this before as “trying to sprint in mud”.   We expend a lot of energy, without going far.

The consequences of a code base out of control is the real kicker.  The code we write affects everyone, not just developers and our customers.  It introduces stress to everyone associated with the software – analysts, managers, customers, family and friends too.  That’s why in XP there are practices that help us keep the code malleable, interwoven with practices that deal with our collective welfare.

With a code base out of control, we can forget  about one piece flow, lean style.  The same goes for continuous delivery and devops too.  Even with the best build pipeline tools, a badly managed code base will not make life easier.  It is like using really bad quality ingredients to make an exquisite meal.

XP keeps a focus on design all the time

XP introduced us to test driven development and continuous refactoring to keep our code bases under control.  Some people use test coverage as measure of that.  I don’t subscribe to coverage as a measure of control.  I prefer weighing up the cost of defects against cost of releasing late.  For example, if getting a feature out means securing a market position, then I would go for lower coverage because releasing late will hurt more than releasing early with defects.  This might be a controversial statement but it is still a viable feedback option.

By the way, it is worth noting that TDD does not yield good designs.  Instead, it raises warnings of problems with our design.  Good design comes from another XP practice of always aiming for simple designs.  Simplicity comes when we gain deep insights into the domain.  Acquiring knowledge takes time – it is a slower feedback loop.  Consequently, achieving a simpler design occurs slower than churning out software at the end of each cycle.

XP allows for code first agile adoption

I prefer to emphasise getting the code base under control as my main target in an agile adoption.  I lean on XP heavily for achieving this goal.  Along the way, the people and process aspects slot in quite naturally.  As the code base is brought under control, it creates breathing room to introduce other non-technical practices.  I’ve found that it is less painful to focus on the code base first, XP style, and slot in other practices along the way.

When I’ve tried it the other way – process before code – I’ve experienced a kind of agility that is only skin deep.  The reason is that the act of adopting the code centric practices requires significant change in developers.  Until developers embrace changes in themselves, in the way they approach coding, the agile adoption will remain superficial.  By “superficial” I mean that it does not penetrate down to the code itself.

Superficial agile is not a bad thing.  I know of some organisations that will benefit from this approach, but only for a while.  For some, that is sufficient.  If we really want to go beyond skin deep agility, then I think it is necessary to reach for XP’s practices that deal with code.  There really is nothing else that deals with writing code in short cycles or one piece.

Should you read Extreme Programming?

My answer is a resounding yes.  It is a wonderful book, written with a lightness and easiness.  Yet is dense with deep insights in every sentence.  The one last aspect of XP that I cherish is that I could ease into to it slowly and deliberately.  I did not have to apply the entire body of work instantaneously, in one huge swoop.  And I am still easing into it.

We are better than this

The people whose contributions to a public Twitter conversation that I mentioned in my last blog post were offended by my opinion on themes that we touched upon.  I’ve apologised over Twitter that my intention was not to be personal, negative nor offensive.  I’ve read my blog post again, taken in the feedback over Twitter and from Kevin’s blog.  I believe that I could have given better context by acknowledging the value of the code retreat and Kevin’s efforts in the community.  Also, I could have expressed my initial tweet differently or at a different time.  That it distracted Kevin from his code retreat was insensitive of me.

Events like Kevin’s code retreats, Black Girls Code, and other initiatives are important to improving our community and software development in South Africa, in general.  Yet, we are still facing glaring disparities in our sector and in our country that I find difficult to ignore.  My fear is if we all take defensive positions, we may never ever discuss such matters openly and respectfully.  Like I said in my twitter conversation to Kevin, this is not about him and not about me too.  It is about us and our future.

The fight for equality is so much tougher when we have freedom.

A question of demographics and then some

Update: Kevin Trethewey has extended this conversation on his blog.

I commented on a photograph that Kevin Trethewey shared of the code retreat that was hosted in Johannesburg, South Africa recently.  His response was very defensive and it clearly showed that he was taking my observation personally.  That was something that I didn’t anticipate, as I have not seen Kevin carry prejudices of other race groups.

What followed was quite an interesting conversation on the disproportionate representation of non-whites in the South African software community.  I’m going to comment on parts of this conversation.  For the full conversation, go here.

A point to note here is how quickly people focused on racial composition as opposed to gender composition of the group.  That is a problem in itself, even for the most gender-aware man in a software community.

This is a long, long, long post, mostly because it is a brain dump of the themes that were flowing in this conversation.

There  is adequate opportunity and access for everyone

Kevin believes that there is fair access and opportunity for all of us to attend an event such as the code retreat.  I disagree. For example, I know a few black software developers that don’t live in a conveniently accessible middle class suburb.  For them, getting to work is a challenge.  To get to a community event outside of the peak public transport service hours is a huge issue.  The same applies to people attending university.  A black kid living in a township with minimal to zero essential services loses at least 3 hours of study time a day when compared to her more privileged peers.  Accumulate that over a month and we find that she loses about 6 days of work a month.  Our lives are not equal in far more fundamental ways than we wish to acknowledge.

It is also a matter of survival, and not that people lack ambition to be better developers.  When we have less than enough of everything that is essential for living, then a meagre salary is a big deal.  Many black developers I know are supporting their entire family, if not an extended family.  When we’re in survival mode, ambition is lower priority.

I would love to see events hosted closer to the people that have greater logistical challenges.  That’s why Braamfontein was a good venue for Agile Africa.  Maybe in 2014, we will have a software conference in Soweto for the first time.  When I want to help someone out of a dark place, I prefer to go into the dark and walk out with them, taking the bumps together, than to just stand outside and shine a light in the distance to where they should go.

People are apathetic

Simon Stewart was of the opinion that it is a lack of caring – an apathy.  I don’t know which group of people Simon refers to here so I will steer clear of making assumptions.  What I have noticed is that the majority of software developers don’t care much for self-learning.  The code retreat that sparked this discussion is a case in point.  Twenty seven developers pitch up to practice their TDD skills.  I don’t know how many developers there are in Johannesburg, but I guess that this is less than 1% of the total.  Regardless, I don’t have a problem with this apathy.  Some people “give a shit” about different things, like earning a salary first.  Because they care less for self-learning, does not mean they don’t care at all.  It just means that their value system is stacked differently from others.

I don’t believe it is about software craftsmanship either.  I think software craftsmanship as a community appeals to people that share that value system and the way that community engages with others.  I’m not one for the craftsmanship movement for reasons that are irrelevant here.  However, I do respect that this community has a right to exist alongside all the others.  Not caring about software craftsmanship does not mean that a developer does not care about growing.  Some of the most accomplished, self-taught developers I know care far less for craftsmanship than I do.  We need to help people understand their potential, and provide diverse ways in which they can fulfil their potential.  Sometimes people will break through that ceiling, setting up a new potential to fulfil.

Lack of diversity is commonplace

Simon rightfully points out that the lack of diversity is common.  I concur.  I’ve spoken at a few conferences outside our continent and it is the same in most places, as much as it is within ours.  Again, that it is so does not mean that we should accept this.  Make up your own mind, but I cannot accept that.  I’ve been talking about diversity publicly for several years now.  I’ve been on projects where the direct result of us not working with our diversity resulted in a tragic cost of the human spirit.  I cannot accept that we are prepared to allow that to happen again and again and again.

I try to be a lot more aware, a lot more empathetic.  I’ve been fighting my own unsettling biases to appreciate our diversity, to celebrate our differences and embrace our uniqueness.  The scary part is this very process of appreciation demands that we drop all our prejudices.  To do that requires self-reflection and clarity to face the truths of our deeply internalised, and camouflaged stereotypes.  Be prepared to cry for ourselves.  Be prepared to feel lighter on the other side of that.

Quotas and Meritocracy

Steven asked what he thought was a silly question.  But it is not silly at all – “How should a code retreat represent the demographics of the country?”  For me it comes down to us building a significant software development sector in South Africa.  The white minority cannot satisfy that agenda alone because there aren’t enough of them.  We need black software developers too.  Parallel to this, we need fair and equal gender representation too.  South Africa has one of the most humane constitutions in the world.  Yet, I see many software developers adopt the non-inclusive behaviour of frat boys, country clubs and colonial style thinking of the west and north.  That is a learned behaviour that I refuse subscribe to.  Given our history of the inhumanity of apartheid, I find this behaviour equally abhorrent.  Incidentally, I was part of a very diverse group of developers that were reflecting on what we don’t like of the software development community, locally and globally.  What emerged were distinct pointers to this imported, inhumane subculture: elitism, arrogance, fear, bullying, sexism, racism, monopolies, nepotism and a lot more.

Steven also dangled the option of quotas.  South Africans are masters of quotas.  We’ve tried that in almost everything from sports to business.  It turns out that Steven is not offering quotas as an option.  What is relevant for me is the other side of the quota argument – inclusion by merit.  Meritocracy is a subtle form of exclusion.  In Africa, where we have significant disparity on multiple levels.  Meritocracy just widens the gap between those that have excelled and those that may excel but don’t have opportunity to do so.

I’ve been guilty of exercising meritocracy to the extent of building a company based on inclusion by merit.  Being elitist is one of the easiest ways of being exclusive, and oh so easy to cover up whilst staring in the face of everyone.  What changed for me was a change in my attitude.  I now carry an attitude where I respect the potential of each individual.  I realised that we need people to contribute in their own little way, acknowledge that contribution and then help that person raise their own ceiling.

We must challenge the bias of minorities

Here is Steven’s alternative to quotas.  I ask that we not flame him or attack his position.  This is his position and I want us to respect that.  If I want to change Steven’s mind, then I need to offer him alternatives, not antagonise him.

I agree that tech events are open to everyone.  I don’t accept that non-whites and women – the “minorities” – don’t attend because they hold prejudices against the pale males.  I think the problem is to do with accessibility, opportunity and a necessity for survival prioritised higher than self-learning, as discussed above.  By the way, Steven, as a white male, you are the minority.

Yet, there is a strong element of intimidation.  This, we cannot ignore.  I’ve been on the receiving side of intimidation.  It is subtle and, strangely, more powerful in its exclusionary effect.  It exhibits itself as quiet arrogance, and other times as open boorishness.  Tech conferences are testosterone events where geeks flex their muscles using absurd measurements of superiority like “my big data is bigger than your big data” or “I’ve got more pull requests accepted on blah-blah-blah repo than you”.  All it takes is one moment of acknowledgement, a moment where there is a little light shining on us and people smile and say that we are awesome.  It is that moment when we pack the soap box in our suitcase and take it everywhere.  From that moment, the superior position is taken and others struggle to fit in, white or not, male or not.  Tech events are open for those of us with soap boxes. Oh, and you must make sure your soap  box is bigger than other soap boxes.

This is why I promised myself to always step off the podium and be accessible to everyone in the room and beyond.  This topic of soapboxes, self appointed popes, cardinals and priests of industry, blogging, tweeting, etc deserves deeper discussion more than this already far too long blog post.  And, yes, I know that my irony is that I’m blogging and tweeting too.

Tell me when you’re done talking

I was uncertain by what Kevin meant when he asked me to let him know when I’m done talking.  It would have been easy for me to fire a flame attack along the lines of “WTF makes you think I’m all talk and no action?”.  That’s not useful because twitter is a horrible medium that draws assumptions out of people.  So, I just asked for clarity first.  This is important; a behaviour that I try to instil in myself all the time – pause, listen, think and ask for clarification if I have even the slightest doubt.  That double check saves me a lot of unnecessary flaming.  I also found that asking for clarification in the face of cynicism does defuse the situation. Then again, rushing around with a burning flamethrower, defending every single one my opinions is a horrible way to spend my life.  It doesn’t take me forward in the least bit.  Worse, still, is that it causes me to stagnate.

I do try my best to not draw public attention to the work that I do.  I respect the privacy of being in a group and have learnt a long time ago that it is self-serving to tweet about such work.  As a mentor to a few people for many years, I respect that unspoken code of confidentiality.  Being a mentor, for me, is a two way street.  I need to be equally open and honest as I expect of the other.  Sometimes, that means being quite personal and deeply private.  So, I don’t crow about my work.  But I blog and open conversations based on observations which I’ve extracted from my time with others.

Those that work closely with me know the extents to which I go to change the status quo.  Then there are those that are even closer that know the extent to which it pains me when I can’t change the status quo.  At a deeply philosophical level, is it possible to give unconditionally – without even the tiniest drop of self-serving satisfaction?

Celebrate the success

Mark Pearl, on a separate conversation stream, reminded me that that progress was being made.  That is true, there is progress being made.  I mentioned that at Agile Africa that the demographics are changing, slowly for the better.  I think we are moving forward and I am not so concerned about the pace either.  There is a part of me, though, that is cautious of grand celebrations for small steps.  That is just me.  I prefer constant acknowledgment of progress for each tiny step.  It keeps me humble, knowing that the big picture is still not fully painted.

Can I plug in, please?

Hearing that I am doing something, Kevin invited me to breakfast to hear more so that he can plug in.  I’m happy to share with most people, but this work isn’t about plugging in.  It is not a thing at all.  It is about my individual behaviour and attitude.  There are no recipes that I can offer to make this better.  It requires me to find a connection with myself.  It is about holding a heightened sense of self-awareness, of each thought, each action and each consequence.  It demands a level of honesty with myself that is, at times, extremely hard to face.  This is a journey that I’ve been trying to walk.  It is painful to look in the mirror and see beyond the reflection.  I am motivated by a belief that we have a primal encoding for decency and an inclusive humane culture.  Everything else is an invention that is designed to obscure that basic encoding.

For more than 50% of the time, I get it wrong – my behavior and attitude.  So, I cannot stop here and I must persevere with this.  This is why demographic representation is important.  It is a measure of our collective shedding of invented biases and a statement of us including each other without prejudice.

To Kevin and the others, thanks for opening a powerful, multi-faceted conversation.  And if you want to plug-in to this inclusive, humane culture, then you will need to plug-in to yourself first.  This is not something I can help others plug in to.  I can only behave in a way that I want our software community to behave.  I believe that others will change in step.

The Genocide of South African Software Developers

The South African Department of Basic Education has decided that the only Delphi and nothing else will be used to teach programming in our schools.  Many of us became aware via Derek Keats’  blog post. On his blog, is a scanned copy of Circular 9/2013 which communicates this decision.  Even the  Digital Portfolio Committee of the  Cape Chamber of Commerce has issued a statement on this matter and are convening a meeting on 16 October 2013 to address this.

That Delphi is old is irrelevant.  For me, the issue is that Delphi is not current.  Old and current is not the same.  For example, Python is old.  Python v1.0 was released in 1994, but it is still current.  It is still under very active development and a popular choice for building software in today’s world.  Delphi is old and has tried desperately to keep up with the times.  Delphi is no longer current.  Please do not bother pointing me to  Embarcadero’s Delphi XE5.  Do your own homework on its market share and developer mind share.

Currency makes the things that we wish to achieve today and tomorrow possible.  Currency of software languages and tools instills wonder and amazement for the current generation and next.  It was C that did it for me when I was starting out.  That I could make LEDs flash sequentially when connected to a parallel printer port, left me breathless.  Delphi excited some people in 1996.  It excites nobody anymore.  It has lost its currency decades ago.  It no longer inspires people that imagine possibilities in the world of cloud, mobile, touch and the Internet of things.  

Certainly, we can teach kids programming using Delphi and Pascal.  We’ve taught programming with Pascal for years.  And, yes, design is the hardest aspect of programming and we can argue that language is an implementation detail.  This argument may stand up in the generality of teaching programming.  But it falls flat when it comes to specifics of teaching programming in South Africa.

The reality is that South Africa does not have a significant software development sector.  As one of the BRICS we have an expectation that our software development sector should be comparable, in relative size, with our “developing” sisters.  Get real. We just don’t have enough developers in South Africa to build a significant software development sector.  What we do have is a marginal sector where the majority seats are occupied by a minority group.  I can only foresee a significant software development sector when the majority of seats are occupied by the majority group in South Africa.  Let me cut through the euphemisms.  The majority group are not white.  The minority group is predominantly white.  But this is not a race issue, it is much deeper than that.

We are in a software development crisis in South Africa.  We do not have enough software developers.  Forget talent, simply on quantity, the minority group cannot fix this situation.  We need black software developers to fill the seats, first with numbers.  The black talent drought is a different problem, but let’s just stop the black software developer drought.  We need the numbers first.  The majority of our kids go through state schools will be left uninspired by Delphi, and turn to other careers. Software development is not even close to being a common career option for all South African kids, and the Department slams the door shut. 

Kids in private, independent and previously advantaged schools will be ok.  I know this because my son attends a private, independent school.  These kids will be inspired.  They will feel that sense of wonder and will reach for a career in software development.  And we will continue fooling ourselves building a software development sector with a minority group.

I wonder whether the Department of Basic Education knows that it is responsible for the under-development of South Africa.  Yes, Mr. SG Padayachee, you and whoever is advising you are killing an entire generation of software developers.  It is a genocide of software developers.

Why Maths Matters

A few months ago I was working on a piece of code that polls a service for a set of data, then for each item in the set, sends a request to a second service.  This was just one part of a pipeline of data processing.  My first naive solution was to just take a guess at the polling interval and the buffer size of the sets and hope for the best.  Then, I made these two parameters configurable.  Now, someone else is responsible for the guesses.

The actual problem is that I was irresponsible.  The situation called for precision in my design and I was being casual.  I focused on the plumbing (i.e. HTTP requests) and not the underlying questions that needed answers.  For example,

  • How big a buffer do I need for the data set?
  • How frequently must I poll for new data?
  • How frequently can I push data out? 
  • How can I reduce the push rate if the receiver cannot accept data faster than I can give it?

One very precise model comes ready made and is known as the leaky bucket problem.  The leaky bucket solution is applied very often in low level networking solutions, but is just as applicable in my problem space.  Now before you roll your eyes and fake a yawn at the maths behind this, hear me out.

The moment we go into higher order control systems, we need higher order maths to build precise mathematical models of these systems. That is not the maths that I’m suggesting we chase.  Instead, with a little bit of creativity and some strategic design constraints, we may be able to reduce many classes of systems to first and second order systems. In other words, introduce constants until you have one or two variables that you can tweak.

This is exactly the situation that I had.  Sure, there were more than two buffers to be managed, but I was in a position to fix the size of some buffers or the flow rates.  Then, the buffers and flow rates that were critical to me could be modeled with straight forward linear equations.

Had I done this, then I would have been in a position to have built a system that could have adjusted itself to a steady state, without frequent reconfiguration.  The fact that I was casual about my design led directly to a case of the pipeline being in a non-deterministic state.  This problem was highlighted early when the users started asking for different kinds of “status reports” for data flowing through the pipeline.  Of course, being casual about it, I treated it as a feature request and implemented a few reports.

This is when maths and science make a difference in software development.  Unsurprisingly, mathematical models are generally deterministic, self contained (i.e. highly cohesive), and at the right level of abstraction for the problem.  And all of those characteristics lead to highly testable models that you can do in wee little steps, test first.

That’s why it matters to have a maths background.  And if you came into software development through some other route, then do some layman studying of algorithms, control systems and simple higher order maths.  It will serve you well forever. It will certainly give you a design advantage when you need it most.  Right now, I’ll take any advantage because design is just so darn difficult.