A lot of people at the moment are talking about intelligent routing, the supposedly high-tech solution that Heroku promised, but failed to deliver to their customers, not least among them Rap Genius[1].

While the phrase intelligent routing has been thrown around by allcomers, with even Wired getting in on the act today[2], very few people seem to have actually sat down and figured out what intelligent routing actually means.

So, what the hell is intelligent routing? How does it differ from its inferior cousin, random routing? And most importantly of all, how do we mere mortals get the same secret sauce that Heroku (doesn't) sprinkle on their lucky customers' Rails deployments for our own apps?

Note. This post talks about Rails, but it applies equally well to many other web application frameworks.

Random routing
==============

Your Rails app is deployed to a server, and to cope with the traffic you've spun up a number of instances of the app. Let's say you've got eight instances, running on ports 8000 to 8007 inclusive.

Random routing, taken literally, will distribute each incoming request for a web page to a Rails instance at random. But hold on - why did nobody ever use the phrase random routing before a few weeks ago? Oh, that's because nobody really sets up random routing in real life, they're far more likely to use Round Robin routing which isn't exactly random - it's actually sequential and completely predictable - but does still result in the same distribution pattern as random routing.  Over time, each Rails instance will receive the same number of requests as any other.

Vitally, if one Rails instance is tied up with a slow-running request - a file upload perhaps, or a leisurely admin action - the instance may still be sent incoming requests. These requests will queue up behind the slow request in front of them, and take many seconds to complete which creates a poor user experience. This is despite there quite possibly being other, idle Rails processes sitting around doing nothing the whole time.

This is the gist of Rap Genius's complaint to Heroku.

Intelligent routing
===================

So how do you solve this problem? It's pretty easy actually. Enter intelligent routing. This is where a front-end web server or loadbalancer keeps track of the Rails instances with slow-running requests, and refuses to route more requests to such an instance until it's free again.

Once again, maybe it's just me but I've never once heard this load-balancing configuration referred to as "intelligent routing". That's just a buzzword that's used by people who don't understand how to set this up for themselves. But whatever you call it, here's how to set it up.

How-to set up intelligent routing for yourself.
===============================================

Here's a handy screencast written by (ironically enough) a former Heroku devops engineer. It's very informative and easy to follow.
http://37signals.com/svn/posts/1073-nuts-bolts-haproxy

Install haproxy[3], use it as a front-end web server or most likely between Nginx and your Rails instances.

Here is your haproxy configuration:

defaults
    balance roundrobin

backend nginx
    timeout server 30000
    timeout connect 30000
    log 127.0.0.1   local0 debug

    server thin0 127.0.0.1:8000 maxconn 1 check
    server thin1 127.0.0.1:8001 maxconn 1 check
    server thin2 127.0.0.1:8002 maxconn 1 check
    server thin3 127.0.0.1:8003 maxconn 1 check
    server thin4 127.0.0.1:8004 maxconn 1 check
    server thin5 127.0.0.1:8005 maxconn 1 check
    server thin6 127.0.0.1:8006 maxconn 1 check
    server thin7 127.0.0.1:8007 maxconn 1 check

The important bit is maxconn 1. This one tweak to your haproxy configuration will make haproxy behave in exactly the way that intelligent routing is meant to. Incoming requests will be routed on a round-robin basis, but no Rails instance will ever have more than one request sent to it concurrently. That means that no request will ever be queued behind a slow-running request ahead of it, unless every single Rails instance is currently busy. (Even then, haproxy will manage the queued requests efficiently and process each as quickly as possible).

It really is that simple. Now you hopefully have more of a clue how to weigh up this kerfuffle between Heroku and that rap startup too.

[1] http://rapgenius.com/James-somers-herokus-ugly-secret-lyrics
[2] http://www.wired.com/wiredenterprise/2013/03/hieroku/
[3] http://haproxy.1wt.eu