A few weeks ago I demoed a project I’d worked on called Oncloud.org. Oncloud was an offshoot project of something called CloudBridge, which was intended for use as a frontend load balancer for a hosting service I was thinking of working on. I’d worked on these projects about a year ago, and at the time they garnered a bit of interest, but not as much as I’d hoped so I kind of left them sitting for a while.

But this time it got a lot more interest. Probably because I explained it better. As the hits to oncloud mounted, I changed the text on the front page to more clearly identify what exactly it does for you. I’ve been really impressed and excited to see people using this project, so I decided I should explain a bit more about the concept behind them and how they work.

How The Web Works

You know those levels in Donkey Kong Country where you’re jumping between barrel canons, each shooting you to the next until you (hopefully) get where you’re going eventually? No? Check out this video.

The web is like that. You send out a request and it gets shot from the next canon and the next until finally a real backend web server takes your request and does something with it. These canons are called proxies (or more specifically HTTP proxies). Note that this is ignoring the lower level where the entire internet is like this. These days you can often count on your request being shot out of at least two canons: one that your isp runs (often invisibly) to speed up requests through caching, and one at the other end that does load balancing between backend machines.

While these two tasks are superficially quite similar, in reality they are quite different. The ISP proxy is blind as to the content it forwards. It knows you want something and it’ll go through the standard mechanism a browser would to find the server you really want and then forward it on.

The one at the other end, on the other hand, has a list of backends it knows will handle any request that reaches it. It forwards the request based on various potential heuristics, the simplest being round robin. It sends each request to the next backend on the list.

In that simplest configuration, it’s taking a crapshoot. The server in question might be down, it might be overloaded, or it might just plain be ignoring a completely underloaded server. More complex heuristics have been developed to manage this, including adding priority numbers, finding out the load average of all the backend servers, etc. Needless to say, these solutions become quite complex.

They especially get very complex if you’re trying to manage your cluster dynamically. If you need to bring up and down servers on a regular basis, that means changing the list on your load balancer(s) on a regular basis. Which also probably means confusing any kind of statistic gathering used to help prevent overloading a backend. This is especially a problem if you’re provisioning servers based on load through something like EC2.

Why CloudBridge is Different

Historically, this arrangement — the load balancer connecting to the backends — has been done because of the ready availability of high quality proxy implementations like squid and apache’s mod_proxy. These proxies weren’t originally meant for this task, but they’ve been modified to support it.

CloudBridge takes this arrangement and flips it. Rather than the load balancer connecting to the backends, the backends connect to the load balancer and signal availability to handle a request. They do this through an HTTP extension called a BRIDGE request. More technical details can be read on github in the protocol description.

To bring it back to the Donkey Kong Country example, where the existing web is a bunch of barrel canons, CloudBridge has the two sides meet in the middle forming a bridge. Hence the name.

This way, the cloudbridge server can be dumb. It doesn’t need a list of backends. In fact, it doesn’t even need a list of hosts it handles. Through the use of hash-based secret keys, your backends can be authorized to handle a domain without the cludbridge server needing to be restarted or re-read a configuration list.

And (though this hasn’t been implemented yet) you can also have the cloudbridge server give information on how many waiting requests on either side a given host has. If there are a whole bunch of client requests for a domain waiting to serve up, you know that that particular domain probably needs some extra backend servers and spin them up. Or if there are too many waiting backends, you can shut them down.

Which is exactly what you need to do if you’re hosting your servers on a system like EC2. Provisioning for peak all the time from EC2 is exactly what you don’t want to do. When there’s only one user on your site in the middle of the night, you want only one backend server running.

How That Turned Into OnCloud

Which leaves OnCloud. OnCloud was something I realized you could do part way through implementing cloudbridge. Given the properties above, you could run a cloudbridge as a sort of open proxy letting users connect to it from behind firewalls and then their development app would become available publicly. This would be much simpler than the more common practice of using an ssh tunnel or Heroku’s limited free hosting to achieve roughly the same results.

In particular, this is really good for developing facebook or twilio apps. But that’s a whole other blog post.

It’s not exactly fast, and it runs with a relatively low connection limit to prevent abuse, but it’s there and it works, and a lot of people have now used it (and continue to use it), and that makes me very very happy.

I’ve heard Twitter described as RSS 3.0, and I’d agree with that assessment. Twitter allows me to get new information on the basis of an ad-hoc network of people filtering that information. I follow people who are interested in similar things (and they do and so on) and bits of information get into my feed through retweets as well as original discoveries. In this way, all Twitter users are basically like the coffee beans in a coffee maker. A whole bunch of information (water) is poured in at the top, it gets absorbed into us, added to, and passed on, and then my follow list acts as a filter on the result. The disgusting gunk that’s left in the filter gets thrown out.

RSS, on the other hand, requires me to go to information sources and subscribe to them. I get either no diversity in this mechanism (just blog posts by the same people all the time) or far too much diversity (by following an rss feed aggregation that tends to include everything new under the sun — see also Digg/Slashdot — with no filtering at all). Twitter — and presumably other services like it — strike a perfect balance by allowing me to choose who I want to listen to. It’s a bit like finding your favorite movie reviewers and ignoring all the rest.

But sometimes there are threats to this filtering. Stuff like #moonfruit is one — by jamming as much noise into the pipeline as possible, it makes it hard to find any signal. Another, in my opinion, is FriendFeed. When someone uses FriendFeed as a client to Twitter, it does this very painful thing where it will turn links into ff.im links. Nothing wrong with a url shortener, but this one does something a little more.

Ff.im links have the painful problem of forcing me to go to a FriendFeed landing page before visiting the content. I have to be subjected to comments about a page I haven’t even seen yet before I can actually see the page. Not only does this not make sense, it also frustrates me. I get a lot of new links to look at on my twitter feed every day, and I almost never follow these ones all the way through to the content. At the moment of click, I usually know so little about the target that it’s just not worth it.

A big part of this is laziness, of course. I already admitted I am lazy above. But it goes a bit beyond that. FriendFeed is essentially advertising to me (I don’t mean banners, I mean they’re advertising their site/service to me) on every link that gets posted through them to twitter. As a non-user of their site (and I never will be one so long as they do this), I did not consent to be advertised to by them. Now that I recognize ff.im links for what they are, I just don’t follow them. And if someone I follow has this happen a lot on links I would otherwise be interested in, I unfollow them.

The first notable case of this was @Scobleizer, who provided a lot of the interesting links I read when I first joined twitter, but at some point all his links became ff.im links. I’m sure there are plenty of people who don’t care about this (I’m well known to care about things like this more than most people), but I’m sure there are also a lot of people out there who find this as frustrating and destructive as I do.

If FriendFeed really needs to tack on content to pages I view through their links, could they at least make it more like Facebook or Digg’s, with the bar along the top? As much as I despise those sorts of things too, they would be a huge improvement over being forced into an interstitial every time I click a link off twitter. Show me how many comments there are, allow me to expand that frame, and I’d probably sometimes even look.