Thursday, 2 February 2012

Peak Abstraction

An interesting phenomena I've observed over my years as a programmer is the occurrence of "Peak Abstraction" within a young programmer - and their subsequent improvement.

There is a common pattern. It typically occurs after 3-4 years of professional practice. A programmer is growing in his confidence. He begins to tackle more complex, challenging problems. And, he reflects those problems with complex, challenging data structures. He has not yet learnt the fine art of simplicity.

Every member of every data structure is a pointer or a reference. There are no simple data types. Everything becomes a "value dictionary", a "graph-based visitor" or a "multi-dimensional ring buffer". Disassembly of his code reveals it to be little more than an infinite sequence of lwz, lwz, lwz, lwz and more lwz. Perhaps he/she develops a hobbyist interest in Java. They may develop an editor that connects boxes with lines - such editors are great for filling with lots and lots of abstraction! Often, she/he may favour Boost; perhaps he even becomes your office's Boost Guy. Even simple plain old data types such as integers are suddenly held within a smart-pointer container and reference counted. And with a pimpl to retrieve their value.

Those around them become increasingly desperate. They can see the code becoming inflated, inefficient, unmaintainable. Spending time reading this individual's code is like spending time exposed to a radioactive isotope. You can only tolerate a few moments before exhibiting signs of sickness. You attempt to modify their code. 478 pages of template errors dissuade you from considering this foolish course of action again.

And then it happens. The programmer realises the error of their ways. They realise that they are adding neither design value nor computational value. They realise they are adding overhead and maintenance trouble. Perhaps they undergo their first encounter with profiling software. Said software may disclose the full terror of the D$ dirty bomb they have unleashed upon the code. Perhaps they are "introduced" to the profiler by an irate lead in some sort of shotgun-wedding affair. Whatever, the programmer realises that solving complex problems does not require needlessly complex data structures. Such structures impress nobody. People are impressed by simple, elegant solutions. And thus the recovery begins. Data structures become simple, expressive, and maintainable. Data cache misses dip under 20ms/frame for the first time. Co-workers enjoy modifying their code. The compiler begins to emit instructions such as "add" or "mul". Sadly the compiler may be too fond of outputting "jump" instructions at this stage... The programmer themself becomes wary of others who are pre-Peak Abstraction.

One even wonders if, like it's namesake "Peak Oil", there may be a global Peak Abstraction. Perhaps the practising body of programmers may all undergo Peak Abstraction in the next 5-10 years and we will all left never having to refactor some template monstrosity.

But then, where's the fun in that? Don't programmers just love to be indignant about others' code?


  1. I see your point, but these days I find it impossible to run into such state of the mind if one spends at least some time reading tweets and blogs of experienced programmers. It's being stressed that simplicity is the king over and over again, so that I never actually had a chance to think that over-abstracting might be a good thing, and I have much less than 4 years of proffesional experience! (still, maybe it's yet ahead of me :))

  2. There tends to be one global peak where things really get out of control, whie alerts you to the problem, but you have to remain on guard. I find tackling significant problems in new languages and frameworks is a big help. This forces you to start up the abstraction curve over and over. Eventually you begin to recognize the inflection point in that curve that signals danger ahead. (For what it's worth, my abtraction peak was at an invention I called UFO's: Ultimate Folded Objects. Trust me, you really want to know any more...)

  3. Great article! It is so true. Unfortunately, there are lots of cases where programmers don't grow up and abandon their sick abstractions. As mihu said there are also developers, who seem not to fall in this complexity trap. I believe it is, because they already came across crappy code and know how it hurts.

  4. I went through the same thing, fortunately I never went deep into abstraction because I am too used to having to use old hardware where simplicity is directly linked to performance and I grew tired of debugging obscure bugs of colleagues that are deeply nested in multiple layers of classes, instances, singletons, etc. This realisation about having to simplify again only really comes from experience and hopefully wisdom which as we know only comes with age. Some people do not even reach this at ten years of programming. It also depends on in which computing era someone started programming.