Is Agile a true Darwinian Force?

I have just come across a very interesting post on Igor Lobanov’s Enterprise Systems Engineering blog: Can an architecture emerge? The gist of the article, as I was able to understand it, is in answering this question posed in the title by highlighting the analogy between agile development (on both micro- and macro-levels) and the Darwinian natural selection process. Igor’s conclusion is that both are capable of architecture emergence but produce results burdened with evolutionary/architectural debt. I would wholeheartedly agree with the debt part, but let us take a closer look at the analogy itself. It is an original and a very intriguing idea. There is no doubt that evolution drives outcomes that clearly look architected (hence propagating the meme of intelligent design). I would however argue that although somewhat similar, agile development has much weaker drive towards emergence of architecture than evolution for three primary reasons:

1.       Lack of parallelism - nature "tries" millions variations in parallel, while project managers, being a frugal bunch, typically allow only one development team/stream for each system being built. And although developers and organizations have some memory for past successes and failures which provides some degree of virtual parallelism (for every path taken developers would have considered and rejected a few alternatives that either failed or delivered suboptimal results in the past) it provides orders of magnitude sparser coverage of the solution space. Furthermore being human, we are naturally strongly biased towards remembering and replicating success, thus leading us to favoring local optimums and reusing past solutions and approaches outside the context where they originally emerged and were proven.

2.       Forgiving environment – setting aside the clichés of corporate jungle, most development occurs in much more sheltered and forgiving environment than those found in nature. I often argue that it is one of the underlying root causes for the lack of respect towards architecture in corporate ecosystems: at its core, architecture is about doing things right, so why bother with it if there is little downside to messing up. In software development almost working is often good enough and, in absence of a ticking bomb of some kind, a working solution is almost never reengineered just because there is a possibility to make it better. In nature however, not being up to scratch is punished very swiftly and even successful solutions are under constant pressure from the competition.

3.       Shorter time horizon – nature had three billion years to evolve to the present state which is quite a bit longer than most deadlines I have come across in the course of my career. Even if comparing the two processes in the units of their natural cycles (biological generations versus sprints of even code-build-deploy cycles) software development needs to arrive to the architecture emergence stage in no more than a few dozen generations.

If you accept the above differences, you would agree that emergence of meaningful architecture from a purely myopic (focused only on the current problem/iteration) agile development process is highly unlikely.