Lines‎ > ‎

Borg Architecture

posted Jul 13, 2012, 2:34 AM by Jerry Dawson   [ updated Jul 13, 2012, 3:04 AM ]
As I'm bringing the new blog-site online, here's a piece (slightly adapted) I wrote a couple of years ago:

One of the problems with implementing a new platform is the striving to make it future proof.  Already, in itself, that’s a
somewhat vain ambition. But a necessary one – the last thing we want is to deliver some all-singing, all-dancing solution,
only for it to become obsolete at the release of the next product version. Then we’re faced with the problem of yet more
investment to bring the platform up to scratch, or neglect the investment until our fancy solution becomes yet another
legacy millstone.
But the other side of the future-proofing coin is that in the striving, the underlying platform becomes so complex, that it
is difficult and expensive to enhance the system. A simple example: say our platform makes use of web services,
orchestration, metadata-driven transformations, slowly changing dimensions in a data warehouse, etc… Now to add a new
application into this platform requires its integration across most if not all aspects of the system.  And so the
developmental threshhold for delivering the new application can become prohibitive for all but the most extensive
The result is that the new app will invariably be tagged onto the side of the main platform, as some sort of unmentionable
growth. And the same happens with the next app, and the next one. Until once again we have a legacy situation of
organically grown exceptions to the rule. The de facto platform then isn’t our fancy future-proof, ever-sharp cutting edge
environment, but another mish-mash of stand alone systems cobbled together around a couple of key functions – such as a DB
cluster or a web server farm.
What I’m imagining is an extension of the principles of asynchronous composite applications. In this way, not only are the
components of the platform available to new or upgraded apps and systems, but those changes will enhance the core platform
itself. This means that the components must be small and loosely coupled enough to allow for major overhauls – even
complete replacement – without having any impact on other existing parts of the overall system.
The way to achieve this is not only through asynchronous services, but multiple layers of redundancy. Ideally, I'd see a
platform spread across two or more data centers where if a part is pulled out for repair or replacement, the whole carries
on with the existing structure until the improvement is brought online. Obviously there's a cost impliation to all of this,
but on the other hand, it simultaneously resolves all those issues around availability and business continuity.
The improvements to one application, process or system then become improvements to the entire architecture. The changes
become assimilated into the whole, and changes become part of the collective system. And those geeks among you will
understand then why I refer to this as Borg architecture...