By Zed A. Shaw

This is a short little post about the latest drop of Mongrel2 which is getting closer to an ALPHA release that regular folks can try out. This release features the beginning of it being a real server, and a nearly complete protocol format for 0MQ handlers no matter what kind of message is being sent. It also includes the rough code for two sample handlers that implement the chat demo and a basic HTTP handler.

If you want to try out the latest gear, read the Getting Started document that explains how to get up and running and do some demos and such. It runs through all the recent features, but is still definitely targeted at a programmer type who wants to try it out. Let's go over some of the big features.

GNU Make 4Life

I tried to like Premake4 but it just fails miserably on too many platforms. The fact that you need Premake4 to build Premake4 makes it nearly useless for many platforms, most notably OSX and a few Linux variants.

Rather than deal with it I just ditched Premake4 for a single gmake file that does the same damn thing with less build junk and works everywhere, even OSX.

Which leads me to....

It Builds On OSX

Mongrel2 now builds on many variants of OSX, so far 10.5 and 10.6 as well as 32 bit and 64 bit. That's in addition to various Linux variants but not any of the BSDs yet.

Daemonized Chroot Drop Priv

Mongrel2 now operates in a chroot required mode that will do the following:

1. chroot to the current working directory (configurable later).
2. Setup a log file and pid file as root.
3. Drop privileges to whoever owns the chroot directory so you can't get it wrong.
4. Start up the server internals.

Using this design, it'll be easy to keep Mongrel2 from accessing files it shouldn't, and help people keep their setup secure without having to do tons of configuration and get UIDs and GIDs right.

If this doesn't work the way you expect yet, submit a ticket so we can make it work better.

Status Dump SIGQUIT

If you hit a Mongrel2 with a SIGQUIT then it will dump a file into tmp/mongrel2_status.txt (if tmp exists in the chroot). This status file is basic right now, but has kind of an "internal process list" showing you the status of all the running tasks inside. This will improve to tell you what state each one is holding in the connection state machine as well.

More Correct Connection State

Connection state is now correct and hasn't needed to be updated yet. It will get timeouts and some additional statistics tracking gear for managing a server, but not much else.

Nearly Complete Universal Handler Format

This is probably the most important feature as it is the basis of how Mongrel2 talks to any 0MQ backend. It gives a complete protocol that can handle HTTP or JSSocket (or future request types) with the same format. The format looks something like this:

54c6755b-9628-40a4-9a2d-cc82a816345e 25 @chat 16:{"PATH":"@chat"},68:{"type":"msg",\n"msg":"yeah not many folks here",\n"user":"zedchrome"},

This is a sample from the chat handler and is of the format:

UUID connid path netstring(json(headers))netstring(body)

In this case the body is also JSON since it comes from the jssocket message, so an HTTP request looks like this:

'54c6755b-9628-40a4-9a2d-cc82a816345e 27 /handlertest 496:{"PATH":"/handlertest","METHOD":"GET","VERSION":"HTTP/1.1", "URI":"/handlertest","Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8", "Accept-Charset":"ISO-8859-1,utf-8;q=0.7,*;q=0.7","Accept-Encoding":"gzip,deflate", "Accept-Language":"en-us,en;q=0.5","Connection":"keep-alive","Host":"mongrel2.org:6767", "Keep-Alive":"115","Referer":"http://twitter.com/","User-Agent":"Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.6) Gecko/20100628 Ubuntu/10.04 (lucid) Firefox/3.6.6"},0:,'

In this case, the body is empty, so it's all headers. How this is parsed in Python is very simple:

def parse(msg):
    sender, conn_id, path, rest = msg.split(' ', 3)
    headers, rest = parse_netstring(rest)
    body, _ = parse_netstring(rest)
    headers = json.loads(headers) if headers else None

Format Limitations

Right now, there's no handling of HTTP request bodies, so you'll get an error if you try to do a POST or PUT. I'll be implementing a basic send body functionality that will have some form of limiter on it to prevent giant messages from choking the system. I'll then look at ways to allow for file uploads in some way that makes them scale, probably with some form of async callback mechanism.

For now, just pretend POST will work tomorrow.

Next, while this request format is working, the reply format needs work. That'll be working tomorrow most likely at which point I'll also drop a Python library that implements the whole thing.

Simple Examples

The last thing in this release is that I've included complete but simple and sort of crappy examples. They're in the examples/ directory and the GettingStarted document says how to try some. Included is the simple handler i that I have up, and the chat demo that we're using.

Next Whack Of Work

Pretty soon I'll be looking for contributors, probably tomorrow I'll have something written up for what I'm looking for in people to help out. Before that happens I want to get the protocol format straight, and finish a couple other little cleanups and features. That way people can understand the whole picture and contribute easier.

The next thing I'll be working on (after the format is complete) is the configuration system. Right now I've got just the bare bones SQL, which is not going to work for the average user. I'm going to try a few ideas I have and then get feedback from folks about it. I'm looking at a kind of "round trip capable" config file that you "commit" to the sqlite configuration, coupled with a nice command line tool or two.

For contributors I'm contemplating three things:

1. There is no "core", just contributors who come and go.
2. Focus on students who want to apprentice more than hardcore neckbeards with an agenda.
3. You have to commit to stay in. Once you stop committing you are in the Hall Of Fame and the slot opens for someone else to take over.

I think this plan will keep the project alive and make sure there's no dead weight, that way people get a chance to rotate in and learn something and bring new ideas. It'll also mean I'll keep the contributor list down fairly small, maybe 5 people at a time.

Anyway, just an idea, not sure if it will work out or not.