Category Archives: Software Development

Introducing the A-* Stack

Posted on by

Nowadays, software architecture and agile methodologies seem to be inextricably inter-twined. Everytime I have a chance for geek-talk with a bunch of software architects, there is always someone that will throw in some of the softer issues that deal with how we run our projects, how do we estimate, something about big design up front no-no’s, YAGNI, DRY and other buzzwords. Since architecture is full of metaphorical stacks of many kinds, I thought it might be useful to invent of an agile stack. Humor me, and let’s call it the A-* stack 🙂
I think there are several layers in A-*. I have no idea what is stacked on top of what, but Here is my A-* stack as I think of it right now, and we’ll try and refactor it later to gain deeper insight into the layers of responsibility and order that must evolve out of the chaos.

  • People Layer: This layer is responsible for establishing a team ethos. It is vital to creating a common work ethic in the team, shared values and principles. It is the lowest common denominator. In high conflict teams with high discord, things fall apart easily. Under these circumstances, you need to drop to philosophical introspection of your team values such as honesty, respect, reasons for existing on the project/or building the solution.
  • Project Management Layer: Managing a project founded on agile practices, even those that use agile practices partially, is no easy task. There is a dedicated layer of responsibility that keeps track of project velocity, prioritization of stories, facilitating feedback and managing change. Sure, we embrace change in agile projects but it still needs to be managed within the prioritized list of stories and other constraints of the project. This is different from traditional PMBOK style project management and deserves its own layer of responsibility.
  • Development Layer: This layer embraces the technical practices of the software developers. It includes niceties like continuous integration, test driven development, code refactoring, single code repositories that guarentee one version of the truth. This is, perhaps, the one layer that is best understood and have tangible actions at the code face.
  • Architecture and Design Layer: This layer is more than it’s really cool acronyms like YAGNI, DRY and BDUF. The focus is on gaining deeper insight into the problem domain. It very likely shares a gray and fuzzy area with the Development Layer and that’s ok. It really doesn’t matter that we have spillage into the development layer or vis versa. As long as we focus on gaining maximum understanding of the problem domain and modelling the solution as simply as as is possible.
  • Run-Time Layer: This is an oee one that I’ve dwelled on for a while. Sometimes the run-time environment really gets in the way and obstructs fluidity and rhythm in the development and architecture layer. It may well be the least agile of all layers in A-*. So, choose wisely … if you can. Let me explain a little further by example. The Ruby on Rails folk have made many screen casts that show how you can change code and you can just reload the page and, magically, the change is visible. Now compare that to someone writing EJB’s. Write, package, undeploy, deploy … it’s just painful, even if you are POJO+TDD inclined. The EJB container will bite you, eventually. So, in some respects the RoR runtime is more agile than the EJB runtime. (Aside: I think the only agile runtime in Java world is OSGi because it supports dynamic loading and unloading of classes and multiple versions classes in the same namespace. Now that’s agile!)
  • Environment Layer: The place where the team work is an equal contributor to agility. From how your workpace is layed out to desk configurations in an open plan office space, it is significant. Audible and visual communication is important and this may overlap ever so slightly with the people layer. I think the environment has a dedicated layer of responsibility in A-*.
  • Toolbox Layer: The tools you use can help you become more agile. I find that a flip chart, white board and multi-colored markers keeps me fluid and helps me progress rapidly, especially when I am working in the Architecture and Design Layer. We all have our special favorites that include the full blown IDE with our special key bindings, code diff tools, and even specialised items like shared whiteboards. I know of one team that has a Skype bot that acts as their JIRA interface – chatting to thet Skype bot allows them to update statuses and query JIRA. What tool ever keeps you agile has a place in this layer.

Perhaps, you have some other thoughts about what goes into A-* and what should be taken out. Maybe you have some real world insights that go beyond my meagre learning experiences. Drop me a note … I really would like to know how your A-* stacks up.

Technical Debt Does Not Exist

Posted on by

Metaphors may be a good way of getting to grips with a new domain.  It allows you to imagine the behavior of something that you don’t quite understand (yet) in terms of something with which you are quite familiar.  That’s were it should stop. I hate metaphors that extend beyond their purpose.  Once I have a good enough understanding, I drop the metaphor.  My reason is simple.  Metaphors force me to do a lot of energy sapping context switching.
Technical debt is one of those metaphors that have been extended so far, that it is believed by many to be something tangible.  Let’s get real here:  Technical debt does not exist.  It is just a metaphor for us to realise that our code base may cost us more money than it should, and that is a future view.  Sometimes our metaphors become euphemisms, and then it is dangerous.  When it comes to technical debt, the less I think of code problems as debt the more I am able to face the problems head on.

Ironically, I did a talk recently on dealing with technical debt.  My fundamental position is simple.  Either your code base has things that exist as a result of broken principles or it does not.  The more principles you break, the more potential problems you have in your code. It is not a problem right now, but it may be a problem in the future.  This future can be a minute away when I run my next test or a year away.  If the future is infinitely far away, then it is not a problem at all.

My first prize is to not break principles, so I don’t create potential problems.  My second prize is to deal with real problems and just leave the potential problems for the future.

(Warning: I’m drifting into my enlightenment self-reflection, so feel free to stop reading now.)

If I am part of the exploration that is looking for a solution (living in the moment and not outside it as an observer), then I should be aware of principles that I am breaking and I should change course immediately.  If I am part of the exploration that is dealing with the potential problem that is now a real problem, than I need to understand the principle that was broken and fix the problem by restoring the principle.  Restoring the principle could mean going on a search for the right solution to the original problem, not just trying to fix the problem that is a result of breaking a principle.  This problem that broke the principle may just be the wrong solution that we thought was the right solution because we ignored our principles in the first place.

Hmmm, that’s an interesting thought.  Karen Greaves asked me if a re-write can be justified and I mumbled something about technology, etc.  What tripe!  Now I think I’ve just reflected on when a rewrite is justified.  When refactoring will not fix the broken principle, then the right solution was never discovered in the first instance.  That is also what I mean when I say that clean code is necessary by not sufficient.  At factor10, we call this a radical makeover.  Radical makeovers are a viable way of getting rid of real problems, and restoring principles.

Heck!  This blog post is less about technical debt metaphors than I thought.  Oh well, the ride on this journey never stops.

By the way, a huge thank you to Steve for helping me realise that all problems come from breaking principles, in code and in life.

Trust Everything

Posted on by

Trust has popped up in so many of my conversations recently.  It came up at home, at a new school that Lia will be starting next term, in the DDD course that I gave earlier in the month, in Peter Hundermark’s scrum master certification course.  And I got a one line email that said this.

The entire world lives on trust. Every aspect in life moves with trust.

The more I think about situations in life that will prove this statement false, the more it seems to hold true.  Even in design it holds true.  Your most fundamental architectural decisions are based on trust and the implementations of that architecture work because of trust.

It’s true for code too.  If you don’t trust the code on which you build or depend, then you might as well write everything yourself, and give up your place on your team.

I was thinking about the AOP with DDD tutorial that I will be giving at OOPSLA this year, and this trust thing came up.  Here again, aspects and the classes into which they get woven, need a trust relationship.  It may seem like a stretch to make that statement, but I think it holds true again.

So, how do you gain trust?  I am not sure, but I think you have give up something first.  Maybe you need to show your vulnerability first, then it becomes easier to let someone into your space.  Then, perhaps, they will let you in to their space too.  When ego walls are erected, then trust finds it hard to grow.  By ego, I don’t mean arrogance, I mean awareness of your self that you hide from others for fear.  Perhaps, it is only when you show your true interface, that the other will worry less about hidden agendas.

In code, trust lies in interfaces and types, not in implementations.  It’s really about trusting the implementation that makes types worthy.  When you trust the type and send it a message and it behaves as expected, then you trust it.  If you request something of an abstract type and the message was received by an instance of a subclass, then you expect the subclass to behave like the abstract type.  You don’t hope that it does behave consistently, you trust that it does!

Trust is tied in with ubuntu too.  You can’t be part of a community nor allow yourself to be defined and shaped by the people around you, if you can’t trust them.  I think ubuntu coding needs trust as one of it’s values.  It’s already a value in XP, and Scrum, and families.  It needs to be in teams, and organisations, and communities and nations too.

Stranger Danger in the Cloud

Posted on by

There will always be someone who will scream that Cloud Computing is new.  And there’s someone else that will tell you it’s old-school-been-there-done-that.  And there will always be some vendor that will try to sell something too.
Since we shrink in body and mind to the state of children when thrown into a new context, beware of stranger danger.  It’s tempting to follow strangers for cheap candy in the cloud.

And yes, like SOA, we’ve seen and done this before.  But I think it is important that we let the passage of time to gel our experiences and ideas over and over again.  Why?  Because contexts change with time and we can use new contexts to innovate again and again and again.

You’re allowed to think … for yourself … and decide … for yourself.

Reflections on the JCSE Agile and Architecture Talk

Posted on by

It was really good to be part of a very topical subject at the JCSE Architecture Forum last night.  While these discussion are so valuable, the things that surface can only be glossed over, largely because of time constraints.  I end up feeling a very satisfied and energised but a part of me feels a bit hollow.
So here are some of the things that surfaced at the Forum, and my narrow, unworldly opinion on each (i.e. I’m just trying to fill that hollow feeling).

When we talk about architecture, we need to define what we mean by architecture?

In my talk it was a very simple view of architecture which, thinking back, I should have disclosed very early.  I am now applying from Kent Beck who talks about mutually beneficial relationships.   So I think of architecture as the mutually beneficial relationship between two or more things.  So what is a thing?  It could be  lines code in a method, methods in a class, classes in namespace, namespaces in code base, binaries in an application server, application servers in a cluster, … see where I am going?  Architecture is about creating beneficial relationships, and the 5 things I discussed are based on this view.  If you don’t know anything about the things, then you cannot create beneficial relationships.  From an agile perspective, the beneficial relationship that you create should only be beneficial based on your knowledge right now.  Tomorrow your knowledge changes, so the relationship may not be as beneficial as yesterday.  Time to change.

Building infrastructural architecture independently of functional requirements…

I am not convinced of the benefit of this approach.  In my limited experience, every business need defines the constraints or needs of the infrastructural architecture.  I find it hard to find the point of departure, yet there is a school of thought that suggests that function is orthogonal to the architecture.  Perhaps I just don’t understand this.  However from an agile perspective, I want to release early and there are many constraints on infrastructure from the business (for example, administrative processes like procurement of hardware).  I like to understand what these are early on, reach agreement on what we can release at the earliest and design accordingly.  Perhaps the first release is on lightweight infrastructure and that means we “limit” scalability.  So, I don’t design for beyond what I know is real.

Model Driven Architecture …

My view is more philosophical and abstract.  What is a model?  For me, a model is something intangible.  It is a way we understand something.  But we represent our models in many ways.  Through words in written or spoken conversation, in unstructured pictures, in structured notation like UML, even code is a representation of a model.

What do we mean by driven? I view it as a something that takes an input that produces an output.  In this case, we take an input, the model, and produce an output, an architecture.  So, I take an understanding of problem and use that to derive an architecture.  So, that’s nothing new here.  However, I don’t like to confuse driving out an architecture from a representation of the model.  That’s different.  Now we are going beyond thought processes  into mechanical processes.  Then the challenge is about how to apply the feedback to the representation of the model – and that is what will make you agile.  Too much for my small brain.

Plumbing …

Yup, we do too much hand crafted plumbing!  It’s something that we have been working on for a long, long time.  I think convention over configuration, dependency inversion, meta-programming are all attempts at addressing this problem.  Some early success that I have experienced is on taking a polyglot approach. I am not talking about mixing general purpose languages on one runtime only.  I am also including domain specific languages. I’ve had some early success where using DSL to describe functional intentions and then generating a large portion of the plumbing.  Where I’ve suffered is when I mix concepts from different domains.  There is the domain of plumbing and the domain of the business.  Whenever I’ve mixed the two, it pains later rather than sooner.  Right now, the only way I’ve had some success is with aspect orientation and meta-programming.

@StatelessSessionBean …

Chris Naidoo is right.  That thing called J2EE and subsequent versions is just horribly broken.  It’s broken encapsulation and a whole lot more.  The fact that we now must use an annotation and not implement an interface is immaterial.  Both result in the same pain – mixed concepts (see plumbing above).  Annotations should be specific to the business such as @RecalculateCostsOnRerouteOfCargo can be used as an interception point for injecting a rule on a class or method.

I would go even further and say that the POJO JavaBean specification is also broken.  Why on earth must I have a no-argument constructor and accessors and mutators.

Last thoughts …

I may have missed some of the other discussions but these are the ones that I woke up with this morning. In general, my observation is that we need to be very concrete very early if we want to be agile, even in architecture.

Balancing craftmanship and methodology

Posted on by

Carlo Kruger commented on my last blog post with reference to Martin Fowler’s blog post which is a concise view of a crazy blog and twitter war around software craftmanship.  For some reason, Carlo thinks that my last two blog posts (here and here) fall into this space. Either Carlo is baiting a hook for fun 😉 or he misunderstood me. Nevertheless, my dear friend, I shall reply.
Martin Fowler and Dan North put the customer’s derived value from the software as the focal point.  The craftmanship movement puts the code that derives the value as the focal point.  I don’t feel comfortable about either.  It is unbalanced.  We always need balance.  So, what will provide an equally powerful balancing force to either of these two focal points?  I think it is the economic viability of the code base.

In Let scrum die, the key message is that Scrum should not be the end goal but a means to get somewhere better.  At some point it will outlive it’s value and I propose a moment in time when that will happen, and one way to force this (i.e. plan the exit of the Scrum master).  I also describe “talented developers” which, perhaps, feels like a craftmanship-esque thing.  It’s not about that at all.  It’s about the balance that is described in the next to last paragraph: developers will represent the interest of the customer, and the code base is under control (i.e. it is economically viable).  Balance the focus of methodology with an the economic viability of the code base.

In You can’t let Scrum die, I draw attention to this lack of balance that leads to complacency.  I mention two craftmanship things to do with the code – remove duplication and express intentions.  Again, it’s not about this alone.  It’s about the balance that is needed.  That balance is described in the next to last paragraph: look at yourself and look at what you create (i.e. the code base).  The reason why I say look at the person next to you is because the economic viability of the code that you create will affect that other person also, including the entire organisation and its customers.  Balance the craftmanship with the methodology which is balanced by the economic viability of the code base.  This reduces to balancing craftmanship with the economic viability of the code base.

There is one area where I don’t know of a powerful enough balancing force.  In Politics of Software Delivery, I draw attention to organisational power grabbing that will always result in non-delivery.  I suggest raising the importance of the delivery value.  But I think this is still too weak a force to balance the power battle.  I don’t know how to fix that one.

Footnote: Jimmy Nilsson reminded me of an article he wrote about why TDD, Refactoring, DDD, and all such things matter.

Flowing in the Waves

Posted on by

I just had a quick Google Wave experience with Willem Odendaal and the experience of seeing the other person type was a bit weird for both of us.
Lesson to both of us: Think before you wave!

Also, I have to remind myself to not think about waves as email, or tweets or instant messages. It’s just something else! And it has a different spin on the time dimension of communication.

I suspect that Google Wave will force us to be better at the way we communicate, how we express ourselves and the relevance of the content to the conversation.  I can imagine a wave growing over time that describes a story started by a domain expert with feedback from a developer and a nice cadence emerging between them.  It all is in one nice wave, with playback that tells you how you got there in the first place.  I wonder if this will have an influence on effectiveness of remote pairing?

I also have a feeling that if you’re a waterfall type of person, then waves will not have an impact on you.  It’s all about feedback and dealing with the changes, which is at the heart of agility.

Now I just need someone to wave with to try out a slightly modified development flow.

Intention Revealing Names

Posted on by

Update: A small elaboration of this has been published on the Architects zone at DZone.com.  Read it at http://architects.dzone.com/news/intention-revealing-designs.
The value of naming of variables, methods and classes is highly underestimated. Given that code is read more than it is ever written, the readability of your code will get more attention than your original thought process that led you to the code. I will be so bold as to say that the reader really does not care how you arrived at those lines of code. The reader is actually focused on what you mean by the code and not how it does its magic.

Often the problem lies in carrying implementation details into the name of the intention. For example, assume you have designed a social network as a directed graph and you want to find all circles of friends that are in this social network (e.g. Joe knows Mary who knows Peter who knows Joe). Since it’s a graph, we can find all cycles in the graph, or closed loops. So, you write code such as:

class ObjectGraph {
  public Collection<CyclicGraph> findCyclicSubgraph() {
	  ...
  }
}

class CyclicGraph extends ObjectGraph {
  ...
}

This does not reveal any intention of finding circles of friends nor does it surface the domain adequately. The same code with intention revealing names just reads better:

class SocialNetwork {
	public Collection<SocialNetwork> findCirclesOfFriends() {
	  ...
	}
}

There’s a subtlety hidden in the refactored code. Both the circle of friends graph and the graph of the entire network are of the same type. SocialNetwork is a composite and has the advantage that all operations executed on a composite will yield a composite of the same type as well. In other words, all operations on SocialNetwork will yield another SocialNetwork. That is also intention revealing design as well as a bit of suppleness.

Here’s another example. You wish to set a boolean to indicate that an object is dirty in your cache.

public void setDirtyFlag(boolean dirtyFlag) {
	this.dirtyFlag = dirtyFlag;
}

But refactored with intention revealing names, the purpose is now explicit.

public void dirty() {
	this.dirtyFlag = true;
}

public void clean() {
	this.dirtyFlag = false;
}

Whenever you are writing code, ask yourself: “Is my intention clear and unambiguous?” If the answer is “No!” then your code is most probably lacking readability. There are refactoring techniques and many patterns that can be used to increase the suppleness of your design, but just spending a few extra brain cycles on the names that you use in your code can make an enormous difference in displaying your intention and not your implementation. This alone will increase readability and reduce maintenance of your code.

Patterns at the September SPIN Event

Posted on by

I suspect that many people did not understand what I meant about forces at play and that patterns describe a solution to bring some harmony among the forces in the problem in a particular context.  For the example of the airplane and wanting to serve the right meal to the right person, the challenge is to serve the meal without knowing about the seating arrangement on the plane, and still sequentially access each seat.  Let’s look at how to get rid of the need to know the seating arrangement.
We start with the solution where we need to know the arrangement of seating and number of seats too.  BTW, it’s ruby code.

for row in (0..29) do # 30 rows
    for pos in (0..5) do # seat A-F
       passenger = airplane.seats[row][pos]
       next if passenger.nil?
       passenger.serve_meal("nut free") if passenger.meal() == "nut free"
    end
end

Of course, if we know the seat number, for example, 15C, then we can do this.  But that does not help at all.

airplane.seats[14][2].serve_meal("nut free") if airplane.seats[14][2].meal() == "nut free"

But, we still expose the data structure of the seats.  So, let’s make it a little better by using the iterator pattern on the data structure for the seats.

airplane.seats.each do |row|
  row.each do | passenger |
    next if passenger.nil?
    passenger.serve_meal("nut free") if passenger.meal() == "nut free"
  end
end

We could be cuter and do something like this and hide the seats array, but we still expose the numbering scheme of the seating.

airplane.serve_meal("15C", "nut free")

So, we can have the iterator on the seats data structure, which helps a bit.  But we can make it a lot better if we put an iterator on the airplane itself. Now, we just care about occupied seats.

airplane.each_occupied_seat do |passenger|
  passenger.serve_meal("nut free") if passenger.meal()
 == "nut free"
end

In the airplane class, we have the following.

class airplane
  ...
  def each_occupied_seat &block
    @seats.each {|row| row.each { |pos| yield pos if not pos.nil?} }
  end
end

We are using iterators on the encapsulated seats structure and exposing a new iterator for the airplane. Also, we are only work with seat that has a passenger

So, we started out with a deep need to know the structure, layout and limits of the seating in the airplane. Then we started hiding the data structure for the seats, put iterators on this seats data structure which helped a bit.  But the real breakthrough happened when we started asking the airplane to just give us a way of sequentially accessing each seat that had a passenger.  No more conflicting forces, just a simple harmonious way to access each occupied seat on the plane.  And when the plane seating changes, we don’t care.

Two Angles to Sustainable Pace

Posted on by

At the Scrum User Group South Africa meeting last night Marius de Beer did a really good talk about Software Development Practices.  It’s been a long while since I saw anyone attempt to draw so much from such widely spread corners of wisdom.  In one slide Marius mentioned the practice of Sustainable Pace.  Many take the view that this is about cutting back on working overtime and that it supports the principles of energised work, and work-life balance.  Marius did make the same point, and it is correct.
But there is another angle to Sustainable Pace.  As a developer, you need to build a rhythm, or flow.  It’s a cadence that you establish as you are writing code.  It’s a cadence that TDD helps you establish itself.  This cadence is also sustainable pace.  And one thing that kills this cadence and your pace in a flash is a mid-stream meeting.

In the panel discussion afterwards, there was a question at the end regarding ways to reduce the number of meetings which I just glossed over.  If you schedule meetings with developers for only the first hour in the morning, you not only reduce the number of meetings but, also, you don’t destroy the sustainable pace built up from the morning.

So, don’t think about pace as just working 7 hours a day, it’s about what you do in the 7 hours that matters.  Get the rhythm going and be anal about things that can kill your flow mid-stream; especially meetings.