By Zed A. Shaw

I've managed to work out a decent "polling" abstraction so that I can compenstate for later designs and try out some different methods for handling events. Today I committed the code that makes poll or epoll work inside superpoll without much configuration. It works and I have good unit tests, but I have to tighten it up tomorrow and make it usable by other people. Also need to make the epoll compile out if your platform doesn't support it.

In this blog post I'll show you a few graphs and some data that shows some improvements I've made to the system, and then what I plan to do with it before Mongrel2 1.0.

Current SuperPoll Status

Right now it works, but I don't have the epoll actually turned on. There's a small bug in that Mongrel2 won't shutdown if epoll is used so I need to fix that. I have a decent unit test that runs the whole system using both poll and epoll and then tracks performance, and then I'm using that to tune it a bit. Finally all the code for superpoll is about 370 lines long.

It also reduced the code in the libtask code considerably and abstracts it away. I should be able to then put whatever I need behind this and adapt to different needs.

Tomorrow I'm going to clean it up some more, and then do the last thing I want to do for Mongrel2 to use it. I first thought I'd go all the way and make the poll and epoll do dynamic scaling and try out various profiles, but then I learned that ZeroMQ is working on something very sweet along the same lines so I'm going with an interim plan.

In the meantime, and hell this may be better, I'm just going to have the connection state machine (which knows a lot about what the current sockets will be doing) just explicitly say whether poll or epoll should be used. This will be pretty quick thing to implement and simple is always better.

I also, really just want to get epoll going and then stop working on this for now. I've been wasting tons of time making this right and I've got plans for a Mongrel2 1.0 release and going crazy on the research isn't going to help my plan. I've laid the ground work so hopefully it'll be easy for other people to try it out and optimize.

Final thing I have to do tomorrow is, well, make people without epoll work. No, I will not be implementing kqueue, /dev/poll or whatever else is out there. If the ZeroMQ stuff works out I'll just use that later. Otherwise, platforms without epoll will just use poll. Frankly, Linux is the platform everyone deploys to these days, and I'm sorry if your favorite platform isn't Linux, but I need to get something out soon.

Later versions of Mongrel2 might support every possible combination of event polling, but for now, I'm going for poll and epoll because those will be good enough for now.

Graphs For The Implementation

The design I went with for superpoll is simply that the epoll file descriptor is inside the zmq poll list. That'll work for a first cut, but I had to work with the two algorithms and getting it straight with libtask took a bit. I also just ripped off the pipetest.c code that I used for my first analysis and just make it the superpoll_test.c code. This is great because I can then compare superpoll to what I know from the other two and it's a complete test by itself.

After getting it working, I checked the performance by running a few different combinations of work loads then made a few graphs:

The first graph is just a plot showing you the range of event performance between poll vs. epoll, but inside superpoll. Right away this looks a bit weird since epoll doesn't perform that bad. The good part of this graph is that epoll has much better lower performance, but really it should be closer to the poll peformance.

The second graph is called a "coplot" and it basically puts two graphs into one graph where events are compared to ATR (active/total ratio) but given each event type, poll or epoll. You can also see that the performance of these two is all over the place as well.

What I wanted to do was fix these two graphs up by changing the algorithms a bit and getting the performance more consistent. At this stage in Mongrel2 raw performance isn't as much of a goal as very stable performance for all the different kinds of loads it'll handle. Very idle sockets and very active sockets. That was the next big project.

Graphs After Optimization

I got into the code and first started cleaning it up. Then I swapped out some really weird pointer swizzling for a simple linked list. This could be improved further, but for now I'll just go with a basic linked list and see how that improves things. Then I started cutting down the code as much as possible by running everything under valgrind and analyzing it with kcachegrind to find hot spots.

After working it, and thankfully I had a good test for this, I had these graphs:

This graph is what I was shooting for, and hopefully this holds up with further development. What you see is now epoll is at least as fast as poll, and has a tighter range, but they're both not all over the place like before. I've found that tight performance that's more consistent "seems" faster to people than a system that may have higher peaks but is all over the place. My belief on this is that, "Humans only remember the speed below the mean."

First, look at the first graph and notice that it looks like epoll wins out, because all you see is raw event performance, but not by what kind of ATR as a simulated load. You'll also see this fits what you'd expect that epoll probably couldn't get faster than poll because poll is called first, and poll probably can't be much faster since it has to call out to epoll periodically. That's alright since I was shooting for a consistent performance profile.

Next, look at the coplot for the rateK/ATR/event comparison:

Notice in this graph you can see that epoll only performs really good on the low ATR end, but poll performs better when the ATR is higher. You can also see that the performance measurements are much tighter. I can see epoll improving in the middle range ATR and then poll staying about the same.

What This Means For Mongrel2

I've wanted to get a 1.0 out really quick. I kind of get tired of dragging projects along and I think doing a sequence of X.0 releases along a more aggressive plan would be better.

That means I can't wank on this crap any longer. What the above work means is that after this weekend when I get the epoll and poll stuff put into Mongrel2 with just a bit more work I'm not going to touch it anymore. There's things I want to make with Mongrel2 and I want to get a 1.0 out soon. Later versions will continue the plan, but for now, this is it.

The next phase is simply the following features:

1. A few parameters for tuning the number of files. I've tried 65k and it works fine.
2. The ability to say Mongrel2 should just use epoll and STFU.
3. The connection state machine identifying when it's in a "mostly idle" vs. a "mostly active" mode so that poll or epoll can be picked.
4. Compiled away for platforms without epoll, so you get plain old poll.

That's it. No more wanking on this. The next weeks will be squashing bugs and testing the living hell out of Mongrel2.