Monday, 19 May 2008

The elephant is in the room!


It's a big week for Kevin and I. Our book Eating the It Elephant: Moving from Greenfield Development to Brownfield is finally available in the US and the UK. It's been eighteen months of hard work from the submission of the proposal of the book to it being real. For a book that talks about re-engineering IT and coping with complexity whilst taking in semantics and business attractors, we've been reassuringly told that its easy to read. My kids even think the cartoons are funny. So if you feel the need to buy a book with a particularly attractive elephant on the cover, ours is the one to go for. If you also happen to work in complex IT environments the contents might be worth reading too!

Is there any more room in the bathtub?

I’ve been hearing some disturbing messages from my customers over the last few years. Some of them are beginning to regard IT as a burden and a blocker of change in their business. They complain that the systems they have are costing a lot to maintain and they are difficult to change. Rather than being the bright new thing enabling business change, IT is being seen as a dirty money grubbing blocker on their business. OK, I may be exaggerating somewhat, but if you talk to customer procurement people long enough you begin to believe them!
But they may have a point. I’ve been researching how much big companies spend on operating and maintaining their existing IT systems versus how much they spend on creating new business capabilities. A Forrester report I saw some time ago suggested that the maintenance budget was expanding and the amount of money being spent on regulatory compliance was also growing. The end result? A squeeze on the amount of IT spending available for business innovation. Now the annual squeeze was only a small incremental effect, but it was measurable.
The installed base of IT complexity in already complex environments is gradually but perceptibly growing. Costs of maintaining that complexity are increasing.
Adding a new channel to a system will add complexity. Layering SOA on top of existing systems will make them more useful and reusable, but adds to the overall complexity. Adding new function to an existing system without taking out any existing function will add to its complexity.
The IT industry stopped throwing the baby out with the bath water over 40 years ago when it became possible for business code to become portable without change between business systems. I suspect these days, our metaphorical industry bath tub is therefore overflowing with all ages from infant to middle-aged hippie. Figures I’ve seen suggest a bank or national government department will have 500,000 function points of installed complexity. That corresponds to 4,000 person years of delivery.
And that’s where the industry could hit a downward spiral. As IT environments grow and get more complex, we spend more time integrating and migrating from old to new. The number of potential (and actual) connections between systems gets larger. The testing burden grows and the reduced budget for innovation gets spent on integration and migration.
How bad is the problem? Well the very largest IT operations now spend 90% of their annual budget on the status quo. That’s only 10% left for innovation. Not only that, but we’re now beginning to see big projects where 60% of the overall costs are attributable to integration and migration, rather than the delivery of new function. With some basic high school maths that means only 4% of the IT budget is actually going to make a difference to the business. Ouch.
Now, not everywhere is as complex or hard as the kind of environments we’re talking about, but there is a danger that as time goes by the same increase in complexity will work its way further down the stack.
We need, as an industry, to do something about it and soon. We need to simplify, simplify, simplify – and that doesn’t mean sticking another layer of supposedly simplifying complexity on top of what we’ve got. If we do, the IT may make it back to being the shiny lithe hero of the story, rather than the obese unwashed villain.

Monday, 12 May 2008

Why do big IT projects fail?

It’s over thirty years since The Mythical Man Month and we’ve made huge strides in the IT industry in formalising methods and tooling. Nevertheless looking around, we still see projects failing. Why is this?
If you listen to industry leaders like Grady Booch, during the 9th Annual Turing Lecture, they will remind you that building complex systems is intrinsically hard. You’re engineering with peoples’ ideas - fundamentally abstract concepts. You’re then turning those rather amorphous things into instructions that can tell millions of tiny super hot switches whether to turn on or off. The gap in perspective is immense. We have an ever increasing number of layers and technologies to help us make the translation, but each layer adds some complexity to the ultimate solution.
Business has got more complex too, of course. The ability to put complex decisions and processes into computer systems makes it possible, for example, to assign a mobile phone to an individual in minutes. Each mobile that’s commissioned takes thousands of individual steps which need to be co-ordinated. It’s not just the phone and the network; it’s the billing and the accounting, plus the customer support and service. If it was down to a human, it would never happen reliably – and certainly not in minutes. Business is complex and technology has enabled it to become ever more so.
Therefore what we do in engineering big systems is hard and in many was it’s a minor miracle that any very large system gets delivered.
The troubled projects that Kevin and I see are failing mostly because of the need to co-ordinate people and the intrinsic complexity of what those people are trying to achieve. So my hypothesis of the day is: projects don’t tend to fail because of technology flaws or poor solutions; they fail because we don’t contain the complexity or enable it to be communicated.

Thursday, 8 May 2008

Drawing Parallels to Science

Brownfield Development approaches the understanding of computers in a manner that is not all that new in terms of science but is extremely new in terms of computers. Brownfield Development looks at the smallest computational building block and develops a detailed understanding of the system by fitting each piece of the computational puzzle together. This is not all that different from Chemistry where atoms (which are made up of even smaller building blocks) are combined into elements and elements under very specific combinations and parameters make up molecules as well as compounds. In the same way, Brownfield Development, is based upon the smallest computational unit which is called a triple. Triples are made of 3 elementary components a Subject, Predicate and Object. When triples are combined they will eventually result in the representation of complex computer system.

Everything in nature can ultimately be broken down into atoms in the same way all computational systems can be broken down into the triple building blocks that make them up. What is interesting is that molecular diagrams look very similar in structure to a diagrammed ontology. Please see the 2 images below, one is a graphical representation of an ontology, the other is a graphical representation of a molecule...what is interesting to note is the similarity in structure between the two diagram forms. Granted these are mankind’s attempt at visualizing things that are so complex that we're still learning new things about them today but that is the whole point of Brownfield, embracing complexity such that we as human being can better utilize the systems that we work with and that operate almost all facets our society today.


Figure 1 - Ontology Visualisation



Figure 2 - Molecule Visualisation


Computing as a whole could benefit from looking at the other sciences and gleaning from the years of work that have already been done. The natural world is in my opinion the ultimate example of engineering at it's best. The following are links to wikipedia pages for a couple of other interesting areas of science that might have some interesting parallels with Brownfield development as well.

Tuesday, 6 May 2008

Brownfield Overview - in 2 minutes!

The following YouTube video servers as a brief introduction to the Elephant Eaters way of doing things. Special thanks to Kevin MacLeod http://incompetech.com/m/c/royalty-free/ for the music.