Every software project involves managing a complexity budget.
A complexity budget can be defined as:
An explicit or implicit allocation of complexity across the entire application
“Complexity” here is defined subjectively (rather than formally) and in Stewartian Terms: “I know it when I see it.”
Or, more specifically to software development: “I know it when I feel it.”
One of the primary jobs of an application architect is to manage a projects complexity budget:
An infuriating aspect of complexity is that that attempts to address it can, in fact, add more complexity.
A good example of this from experience was when a company I worked at added OSGi to the system to manage the increasing complexity of the project. It seemed like a reasonable approach, it offered a sophisticated module system, it was recommended by a newly hired architect, and it even says on the “What is OSGI page”:
OSGi significantly reduces complexity in almost all aspects of development: code is easier to write and test, reuse is increased, build systems become significantly simpler, deployment is more manageable, bugs are detected early, and the runtime provides an enormous insight into what is running.
What’s not to like?
Unfortunately, adding OSGi to that project effectively ground the entire project to a halt: it took a few of our best engineers out of normal application development for over a year, and when they were done the codebase was even more difficult to work with than when they started. Feature velocity, already teetering, collapsed.
This is not to say OSGi is universally bad. But, in this case, rather than boosting our development teams productivity, it effectively ended it.
A good software architect is someone who manages the software budget of their project effectively, either explicitly or implicitly.
My sense, admittedly without hard evidence, is that Stewartian Application Complexity grows roughly geometrically with the size of an application. Proper factoring by experienced developers can hold this curve down for quite some time.
However, this doesn’t change the fact that, somewhere out there, there is a Complexity Wall lurking.
And, if you aren’t careful, you will run headlong into it and grind your development velocity to a halt.
I have had multiple experiences with this: one day, inexplicably, development on a system that I was working on went from feeling “large, but manageable” to “this is impossible to deal with”.
Here are some tools for managing your complexity budget:
Unfortunately, experience shows that managing Stewartian Complexity is a subjective endeavor and that many talented and experience developers will disagree on the proper course of action at a given decision point.
Nonetheless, by making the concept of a complexity budget explicit in your software project, these conversations can be more productive and ultimately lead to better software outcomes.
Almost all mature applications are complex.
Finding a new codebase “complex” is not an excuse for tearing everything apart or aggressive refactoring. We must always bear in mind Chesterton’s Fence.
If an application is functioning well (or even reasonably) then we should assume that the complexity budget was well (or at least reasonably) managed.
And we must always remember that, with unfortunate frequency, big attempts at addressing complexity in existing, large applications often fail or, sadly, make things worse.