The architecture of business intelligence solutions is as archaic and carved in stone as a clay tablet from Mesopotamia.  It is old, stale, behind the times and controlled by a dictatorship that tries to portray a fake benevolence in all its propagandist messages.  The dictator is data itself.  Data has enslaved procedures and actions and treat them as second class citizens.  Every single time something needs doing, the data just force another action into slavery.
But everything is not everything.  A raucous revolt has been going on for a long time and there are just a few dictators hanging around clinging onto power.  The actions have created their own federated colony where they rule over the data.  “Rule” is a tough description.  Data are actually the protected citizens.  They are governed by a glorious constitution that ensures that their rights are not violated.  They behave within the guidelines of the constitution.

This new frontier is, indeed, a difficult world to understand.  It is even more difficult for the tourist companies putting together glossy brochures selling vacation trips to this world.  How do you describe a vacation utopia for something that is abstract.  Really, we can’t see an action but we can see data!  It is genuinely abstract; the action is a kind of energy that exists and binds the data.  It’s like saying that we can’t see the 4th dimension, but we understand it and know it exists because we can see the three dimensional shadows it casts into our universe.

If you’re still wondering what these extended metaphors are rambling about, then let me cut the meat closer to the bone.  BI architecture is stuck in stale techniques of moving and accessing data around the enterprise and it uses procedural techniques (the actions) to achieve this.  Everything is centered on data and data is centered on creating efficient actions as slaves.  In the world of equality – the one with the nice constitution – encapsulated behaviors (actions) protect the integrity of the data.  There are defined contracts and objects are the carriers for behavior and data.

There is so much that can be learned from this ignored world.  For example, the problems of scalability has been solved with statelessness.  Contention for shared resources have been solved with highly concurrent techniques that remove semaphores and other dead lock creating protection schemes.  The freedom to work with data in forms that are natural to the data is clean and efficient.  If data looks hierarchical in structure, then we can use use storage that naturally works with directed graphs, with lazy loading thrown in for good measure.  If data looks dimensional, then we can use a single multi-dimensional table that tolerates variant hash maps for individual columns.  If there is a need for massively parallel processing, then we can use map-reduction techniques over a distributed file system in a massive cluster of commodity hardware.

BI desperately needs change.  So throw away the clay tablets and start thinking laterally.  It’s a lot more flexible.  And if the tourist companies selling vacations want to take advantage of the new frontier, then they better understand the laws that govern this world.

Data exists because of the actions and not the other way around.  Get with the program and embrace the techniques, tools and agility of the new frontier to build better BI solutions, instead of trying to get everything to conform to that old clay tablet.  The body did not create consciousness, consciousness created the body.  Here’s the bottom line:  BI is stuck because it is a body without consciousness.

For me, I’m “just an earth bound misfit learning to fly” .   I’m still grappling with impedances and shear planes between these dichotomous worlds, trying to get others to see the value of using multiple disciplines for the symbiotic benefit of both.  I’m still “tongue tied and twisted” but a little step closer to “learning to fly”.

** Lyrics from “Learning to Fly” by Pink Floyd.

 Note: Most people know that I work for a company that crafts business intelligence solutions and that I work on the enterprise application development side of the company.  Unfortunately, I really do think that BI is lagging behind the times and that it needs a serious jolt.  The plethora of proprietary, non-agile tools and practices is still a problem.  Perhaps there are others out there that can enlighten me on the strides they have taken to built better BI architectures.