Exposed Wiring

or

Millenium Falcon vs. Macintosh in modern code design

Programming nowadays has reached the levels of sophistication such
that peoples' expectation is no longer just that a piece of code
works, but that the experience has transcended the functional, into
the aesthetic.  People expect the code to look 'beautiful', to go
down easy, to speak in its own perfectly-tailored, English-like
seamless abstraction.  Seamless, in fact, in the literal sense.
Literally "magic is happening inside of this thing, which you
should not worry about, but for which we have gone through great pains
to ensure you get the experience we hand to you on a plate, but you
do not get to look inside the box."

In the technology world, of course, the modern Macbook stands as the
symbol of this design philosophy.  In a field that has taken decades
to try to _start_ to dredge itself out of supposed ugliness and
complexity, the Mac took the symbolic and business leap to cultivate
an aesthetic consumer experience, free of the details of exactly how
the magic box delivers cat pictures through the intertubes.  Because,
the truth is, almost nobody, numerically speaking, cares or wants to
deal with how the thing works.

But of course, you and I, dear reader, are not most people.  We are
the ones who are paid to understand how the sausage gets made, who
must, on a daily basis, tear open the box to try to attempt, on a
deadline, to figure out what the hell is segfaulting our supposedly
wonderfully sophisticated aesthetic experience.

More and better words have been spilled on this topic, by authors
whose words ring true across the years:

- [The Law of Leaky Abstractions] [spolsky]
- [Shopcraft as Soulcraft] [crawford]
- [Commodity fetishism] [marx]

[spolsky]: http://www.joelonsoftware.com/articles/LeakyAbstractions.html
[crawford]: http://www.amazon.com/Shop-Class-Soulcraft-Inquiry-Value/dp/0143117467
[marx]: http://en.wikipedia.org/wiki/Commodity_fetishism

The reductionist explanation of these works starts like this: trying
to build a beautiful facade around our constructions hurts us.  It
distracts from understanding where our objects came from, and the
brutality that was required to make them (think iPods made by child
laborers in China).  It disempowers us, by dissuading us from
understanding how our constructions work, and by in fact making the
operations of our constructions more complicated, because they try to
hide their details behind more layers of details, rather than making a
simple system that was meant to be laid bare.  When everything will be
exposed, the only way to simplify things for the human mind is to in
fact have few parts.

The opaque layers and complexities of our systems hurt us in many
places.  I think perhaps the most problematic complexity right now is
in our politics and economics, where people feel such lack of
understanding and despair in such a complex system, that they feel
like the system is now not understandable, not influencable, and
ultimately has become a static source of injustice, not justice, not
the hacker ethic or the American can-do spirit or even the
Enlightenment dream that freedom and rationality could make the world
a better place.  People have given up.

But I digress.

There is an opposite icon in tech culture to the Macintosh, one that
embodies the DIY hacker spirit rather than the polished aesthetic of
consumption, and that is the Millenium Falcon.  The Millenium Falcon
is, most commonly referred to as a piece of junk, and spends many
parts of most of the Star Wars movies being broken.  You can see all
of its parts, and many of its parts are frequently hanging out
exposed, on fire, being tinkered with by Han Solo and his heterosexual
life partner, Chewbacca.  But the Millenium Falcon is the fastest ship
in the fleet, it runs the most dangerous missions, and no matter how
much it's broken, a ragtag crew of Han or Chewie or R2D2 can get it
working again in the middle of a firefight, because they understand
how it works and they have access to the components and spare parts
that can patch the thing together, even out in the middle of an
asteroid field.

This is a significant point.

Taste is a significant thing in engineering design.  It's taste, by
and large, that is the first line of defense against confusion and
complexity in a design, because a mass of complexity _feels bad_
and evokes _terrible premonitions of dread_ when you can tell that
something is going to be hard to reason about, hard to debug,
hard to unfuck in six months when the original author has left the
company and it's become your job to fix the site outage in the middle
of the night.  Many smart people have the intellectual capacity to
figure out a complex design--you wouldn't have gotten anywhere in the
field without that kind of intellectual capacity.  But it's only the
people with some kernel of good taste and judgment who tend to reign
in the madness.

So having a taste for simplicity is important, extremely important in
trying to hold together our rapidly spiraling out of control world.
But, to try to reunite the circle, I think there's a kind of
fundamental difference between the simplicity that goes into trying to
make the Millenium Falcon a DIY spaceship, versus a frankly pernicious
so-called simplicity that makes a Macintosh a shiny disposable
consumer device.

You can see the forces at work in code, in two of the programming
communities that have some of the highest degree of taste: Ruby and
Lisp.  Both of these languages and ecosystems have a high degree
of expressive power, ethics of DRYness, metalinguistic mutability,
and aesthetics of 'simplicity.'  And it's here where the two
approaches to 'simplicity' have their most violent and nauseating
conflicts occur.

[Zen and the Art of Motorcycle Maintenance] [pirsig] has a great
refrain about words, words that can cut like a knife to try to divide
a concept and lay bare the difference between one side of the concept
and the other, to try to draw a distinction between _quality_ and _not
quality_.  For Pirsig, in fact, the most important distinction was not
about enumerating what had quality or not quality, but in fact the
distinction between being aware of quality at all and seeking it,
versus the mass of ordinary experience in which people don't even
realize they throw away their lifetimes, spent occupied on things
completely _orthogonal to_, _oblivious of_ whether they are even
engaged in acts of quality or not.  But the word, the conceptual
knife, could _reveal_ into clarity the differences in the world and
thus help us to pursue what we could now see.

[pirsig]: http://en.wikipedia.org/wiki/Zen_and_the_Art_of_Motorcycle_Maintenance

I think it's important to have a conceptual knife to illuminate the
distinction between what's going on here between the Macintosh vs. the
Millenium Falcon in our code, because I assert that it's real, and I
assert that the difference has true effects on our real day-to-day
struggles.  Here's how it goes.

In any given system, as the level of abstraction and encapsulation
increases, it's natural to want to hide seemingly unnecessary details
of implementation away from the consumer of a product.  This of course
comes with the kind of leaky abstraction assumptions listed above; the
assumptions that:

- my client is a _consumer_ and not a co-creator, co-understander,
  co-repairing of my product.  my client is too busy or too dumb
  to understand.
- my code is complete and perfect ; it does not need to be opened up,
  modified, debugged, or extended.  its outer face is the only thing
  anyone will ever need to understand or use.
- my conception is obvious and superior; it does not need to be
  opened up to be understood, it does not need to be compared to
  existing concepts with different names.  My names and concepts
  stand in their own self-contained glory, not the names and
  concepts that lesser systems have used.

Have you seen constructions that embody these attitudes?  You will
notice them in abudance among the most "sophisticated" purveyors
of Ruby and Lisp, because they are the attitude of the implicit
convention, the framework, and most of all the DSL.

Do you see the difference between the way that programming languages
work, and implicity conventions, frameworks and DSLs?  Where
programming languages, to the vast extent, give you explicit axioms of
general-purpose, composable constructs, all of these other constructs
give you narrow, opaque monotools.  Where programming languages, if
you've had some first year CS education, can be directly translated
into the operating all the way down to the underlying hardware, DSLs
are damn near impossible even to interpret into the next layer down in
the stack.

Have you ever tried to read, much less debug, the Rails source?

So let's cut deeper.  Everyone knows that abstraction is good, that
higher-order thinking only occurs because of abstraction.  What then
is the difference between abstraction that cures and abstraction that
kills?  The difference, I think, is at the very top-most layer.  The
layer where the creator makes his declaration of arrogance, that he
is the smarter better creator than everybody else, that _he_ sees
beyond the veil to the truth of the how the system works, while
everyone sees the facade.

You see, even the implementors of conventions, frameworks and DSLs
have to build their systems out of parts.  They see pluggable
components and wires and broken hyperdrives made out of ordinary
commodity parts, just the way that everyone else normally sees all of
their own code.  The makers of the platforms still have to use
ordinary broken disappointing tools to make their platforms--the only
difference is that at the end of the day, they wrap up their imperfect
product in a thick opaque layer, and declare to everyone else, fuck
you, I'm better than you, you have to speak my language, which will
clearly cover all cases, rather than ordinary components and
constructs of the underlying base language in which the framework is
actually built.

The underlying language, in almost all cases, was built to be simple
and general and composable--a Lego set to make the user be able to do
anything.  But the framework is meant to make the user do exactly what
the framework creator deemed acceptable.  The Framework actively
prevents or simply just breaks when tried to be composed with other
tools--why would evyer want or need other tools when you have The
Framework?

This I think is the sickness at the heart of some of our most
promising ecosystems; that peoples' attraction to having their thing
be pervasively The One True Thing actively distracts us from building
simple interoperable building blocks with which we can *work
together*.  Instead of giving each other tool sets, we build
widely-isolated fiefdoms and tell each other you're either with us or
them, and you'll never need another kidnapper, you'll learn to love us
and our One True Way instead.  This is what's implied by the
private-pervasive-metaphor approach to design rather than the simple
composable part philosophy.

It's all both simpler and more complex than it self.  Simpler because
half the time the one-true-abstraction is just a wrapper layer around
all the ordinary parts that we could come to learn and understand--you
could take apart all the DSLs and just have systems that dealt in
their underlying operations--which would all just be functions and
data structures, which are simple, publically known,
well-reasonable-about things.  But it's the design approach is also
more complex, because psychologically, sociologically,
architecturally, it implies and embodies hubris, arrogance,
privitization and competition within the field, rather than
collaboration and common effort to things that everyone can understand
and improve into the future, rather than reimplementing the same
bullshit in one framework vs. another.

So this I think is at the heart of the issue, right up at that
dividing membrane between the ugly guts on the inside and the
beautiful exterior of your aesthetic experience.  The ugly guts are
nothing special, we all could have written and understood them,
perhaps even better than you did.  But the outside is what makes them
exclusive, mysterious, uniquely yours in that special way--you were
the creator of the language, not Guido or Matz or Rob or Joe or Rich.
Other men have to speak your words, not you theirs.

I don't want to deny people their freedoms.  If you want to have it
your own way, I don't think Matz or Rich would care (though Guido or
Rob might).  But I do think that, objectively speaking, you help other
people get their jobs done easier, and help the state of the field
improve more, when you build interoperable parts based on common
foundations (e.g., libraries) rather than non-interoperable parts
based on privre ate foundations (e.g., DSLs).  And those goals, more
productive teams, evolving common tools and platforms, are really
what helps companies succeed privately and the industry succeed
publicly, not the consumer aesthetic optimization of whatever
particular micro-problem you've identified any given all-consuming
paradigm you're trying to impose.