Complex Landscapes

I have spent the last few weeks reviewing some application management support deals. It seems to me that business is being driven by short term goals rather than long term – and that approach is costing them dearly in the long term, both in terms of flexibility and money.

When a customer buys a hardware solution, one of the major criteria for the choice is the total cost of ownership – it will be cheaper, more economical, or whatever to run it on the latest, fastest more efficient chipset. However it seems when it comes to applications, speed to deploy and cost to deploy are the main considerations. Businesses look at the total cost of acquisition rather than the total cost of ownership.

Surely the quicker and cheaper to deploy is the best in any case? Well normally not. Ill thought out solutions often increase the complexity of your estate rather than reducing it. Ultimately this will end up costing you more in terms of money and flexibility. Let’s take a simple example of a partial Service Orientated Architecture (SOA) implementation that would wrap, rather than re-engineer, the legacy system to expose it’s offerings as services to the rest of the estate. It certainly makes it easier to interface to the legacy system, but at what cost? The solution has solved something by adding another layer (or more) into your estate. You will still have to run the legacy system, still have to it to maintain, you will also have to maintain the new SOA services. Your estate has become more complex.

I am not stating that SOA is a bad thing, it is just an example. There might be good reasons for doing the example I gave above. Perhaps it allows you to isolate the legacy system while you develop a suitable replacement - so that it is easier to replace, in which case this will make that transition easier. This would be a good use of the service – as a means of transitioning. However this is not the normal case in our world. Solutions are performed piecemeal with little long term strategy – what will be introduced as a strategy will be overturned in coming years when that money is required on even more functionality.

So we continue to make our estates harder and more difficult to maintain we increase the complexity of our estates, and we do it often by adding layers of technology which it is ‘supposed’ to make it simpler. In fact it does the opposite.

So what is the solution? Well it is reasonably obvious, we need to stop trying to hide things that are difficult to understand behind a curtain and get to understand them. It is odd that in no other field of business would a businessman say that area is too complex to understand – I’ll just ignore it! Yet the business – IT gap makes that seem acceptable and normal.

The bottom line is simple, just like one of the Brownfield beliefs – embrace complexity rather than ignoring it.

Do this and your world will become simpler.

Introducing Brownfield - Part One

Now that the blog is being relayed through to Amazon, it may be worth recapping what Brownfield and Elephant Eating is actually about. Brownfield came about because most organizations in the twenty-first century have an existing, very complex business and IT landscape. The introduction of 3GLs and families of computers such as the System/360 enabled the complexity of these landscapes to accumulate. In our fashion-driven industry, these programs have been augmented by the adoption of on-line processing, client server, object-oriented programming, the exploitation of the Internet, and so on. Each of these movements has had its own strategy for dealing with complexity, but in reality, each one added to it. Today’s IT systems are typically poorly documented and their maintenance is dependent clusters of knowledgeable ‘system experts’. Each new fashion has introduced its own experts, resulting in the costs of maintaining the status quo now absorbing 70%-90% of IT budgets. The budget for innovation is being squeezed between increased regulation and maintenance costs. As a result businesses are finding it harder to migrate to competitive, integrated architectures such as Web 2.0 and SOA.




To move to these new architectures and achieve demanding business targets this existing complexity often needs to be discovered, understood and harnessed. However no-one can understand the whole beast, so with traditional greenfield methods we rely upon the knowledge of many people to understand the whole. This means the environment for most big IT projects is an immense communications and co-ordination challenge because we are entangled in an almost uncountable number of poorly understood environmental constraints.

Not surprisingly we have observed that an increasing proportion of the effort of developing new business capabilities is spent on understanding and integrating with the existing complex system and business landscape rather than delivering value. It has been observed that up to 75% of overall project effort is now spent on software integration and migration rather than new function. Many of these large integration and migration projects fail expensively and publicly due to poorly understood complexity.

Engineering change in such environments has many parallels with the concerns of the construction industry in redeveloping industrial or contaminated sites. They are full of hazards, unexpected complexities and tend to be risky and expensive to redevelop. Today’s big IT projects are usually executed on such “contaminated” sites, where you need to be careful where and how you build; a change in one place can ripple through to other systems in unexpected ways. Such sites are more ‘brown’ than green. The accumulated complexity of IT environments has made them “Brownfield” sites.

Brownfield development recognises that any new software architecture must take into account and coexist with live software already in situ. It does this by making a number of improvements to conventional software engineering practices.

Current “Greenfield” tooling and methods use early, informal and often imprecise abstractions that essentially ignore complexity. However the devil is always in the detail. Early, poorly informed abstractions are usually wrong and are often detected late in construction, resulting in delays, expensive rework and even failed developments. A Brownfield oriented approach embraces existing complexity and focusses on understanding, communicating and harvesting it to reliably accelerate the overall solution engineering process. Such an approach can enable phased, incremental change where this was previously thought impossible.

Brownfield Development, starts by harvesting a detailed knowledge of the systems, services and data in the immediate vicinity of the solution under construction and then using that reverse engineered knowledge for forward engineering of the new solution. Rather than taking the conventional approach of starting with a Conceptual Model and driving down to Platform Specific Models and code generation, Brownfield starts by harvesting code and other existing artefacts and uses patterns to formally abstract upwards towards the Architecture and Business tier. It then uses all of these new layers of knowledge to perform the necessary transformation of the existing environment.

In the next post we’ll cover how we go about doing this with tooling using the VITA tooling architecture.