By Zed A. Shaw

While working on Hype.la I wanted to do some photo scaling/cropping stuff for people's uploaded profile pictures. If there's one thing on the web that sucks more than uploading a photo, it's munging it for efficiency. Usually you're stuck with nasty API's that are written in C and just damn hard to use. Combine that with the range of image types and sizes people can upload and you've got lots additional headaches.

With Tir's new Tasks though I figured I could create a system that did the photo uploading, but did all the processing in async tasks off ZeroMQ. After figuring out how to actually crop photos in Lua with Imalib2 it was about a day of work and now gives me some nice room for the future.

Here's how I did it.

Step 1: Scale And Crop With Imlib2

I tried a bunch of libraries, and they're all still idiotic bad. The only one that was sort of alright was Imlib2 but the Lua library for it wasn't working quite right. Well, what's a boy to do? That's right, bust out alien and roll your own. Here's the whole code for doing simplistic photo scale/crop and it's all done with alien binding to Imlib2 on the fly.

module('model.images', package.seeall)
require 'alien'
local imlib = alien.load("Imlib2")
local Image = {}
Image.load = imlib.imlib_load_image
Image.load:types("pointer", "string")
Image.save = imlib.imlib_save_image
Image.save:types("void", "string")
Image.context_set_image = imlib.imlib_context_set_image
Image.context_set_image:types("void", "pointer")
Image.crop_and_scale = imlib.imlib_create_cropped_scaled_image
Image.crop_and_scale:types("pointer", "int","int","int","int","int","int")
Image.format = imlib.imlib_image_set_format
Image.format:types("void", "string")
Image.free = imlib.imlib_free_image_and_decache
Image.free:types("void")
Image.get_width = imlib.imlib_image_get_width
Image.get_width:types("int")
Image.get_height = imlib.imlib_image_get_height
Image.get_height:types("int")
function convert(infile, outname, format, x, y, width, height, final_width, final_height)
    assert(infile and outname and format and
            x and y and final_width and final_height, "Invalid function call.")
    local img = Image.load(infile)
    if not img then 
        return nil, "File not found." 
    end
    Image.context_set_image(img)
    if width == -1 then
        width = Image.get_width()
    end
    if height == -1 then
        height = Image.get_height()
    end
    print("IMAGE IS " .. width .. "x" .. height)
    local cropped = Image.crop_and_scale(x, y, width, height, final_width, final_height)
    Image.free()
    if not cropped then
        return nil, "Failed to scale image." 
    end
    Image.context_set_image(cropped)
    Image.format(format)
    Image.save(outname .. '.' .. format)
    Image.free()
    return true
end

Nothing big there other than having to read the Imlib2 API and the header file to figure out the signature of the functions I need. Then I wire up a simple module and wrote a convert function for what I needed.

Please, if you know this is vulnerable in some way, let me know. I'll probably firewall the crap out of the photo task that uses this just in case anyway.

Step 2: A Simple Photo Crunch Task

Once I had a library for modifying images I needed to make the background task that uses it. The code for this is very simple:

require 'tir/engine'
require 'model/member'
require 'model/images'
function cropper(req)
    print("CONVERTING", req.filename, "TO", req.outname, req.format)
    model.images.convert(
        req.filename, 
        req.outname, req.format,
        req.x, req.y, req.width,
        req.height,
        req.final_width, req.final_height)
end
Tir.Task.start { main = cropper, spec = 'ipc://run/photos' }

Here I'm just using Tir new Tir.Task module to setup a task that takes requests for photo actions and does them. I also wanted to use this in a few spots so I added this to my little member model:

PROFILE_DIMS = {x = 200, y = 200}
PIC_DIMS = {x = 64, y = 64}
function scale_crop(photo_task, pic_fname, dims)
    local outname, format = pic_fname:match('^(.*)%.([a-z]+)$')
    if not (outname and format) then
        print("ERROR: uploaded invalid file", outname, format)
        return false
    else
        photo_task:send("photos", {
            filename = pic_fname,
            outname = outname,
            format = format,
            x = 0, y = 0, width = -1, height = -1,
            final_width = dims.x,
            final_height = dims.y})
        return true
    end
end

The phototask parameter is a *Tir.Task.connect* that we make in the handler, picfname is the file to work, and dims is what dimensions we want. In my case I'm just doing a simple scale and no real cropping, but later I may expand this to do a nicer scale operation.

Step 3: Accept Uploaded Photos And Crunch

Finally I take an uploaded photo, using Tir's nice multipart MIME upload functionality, and work the photos people upload:

require 'tir/engine'
require 'model/member'
local member = model.member
local photo_task = Tir.Task.connect { spec = 'ipc://run/photos' }
local user_task = Tir.Task.connect { spec = 'ipc://run/users' }
local Upload = {
    form = Tir.form({"file"})
}
function Upload.profile_pic(web, req, params)
    if params.owner_id ~= web.owner.id then
        web:forbidden()
    else
        local pic = params[1]
        local owner = web.owner
        if pic.body and #pic.body > 0 then
            local ext = pic['content-disposition'].filename:match('".*%.(%w+)"')
            local pic_fname, profile_fname = member.set_picture(owner, ext, pic.body)
            if not member.scale_crop(photo_task, pic_fname, member.PIC_DIMS) then
                return web:bad_request('Your photo is jacked.')
            elseif not member.scale_crop(photo_task, profile_fname, member.PROFILE_DIMS) then
                return web:bad_request('Your photo is jacked.')
            end
        end
        db.Member.store(owner)
        user_task:send("update_landing", {owner_id = web.owner.id})
        web:redirect('/Profile/photo')
    end
end
Upload.config = {
    route='/Upload',
    before=member.login_check(false, '/Login')
}
Tir.evented(Upload)

This is an evented style handler which I'll expand out later for other kind of uploaded media. It also requires the user to be logged in and the form they submit should match the owner id of whoever's logged in.

After that I'm just taking the photo, setting it on the member, and then using the member.scale_crop I wrote above to do the work. You'll notice I'm also using a task for updating people's static profile pages. I use this task on anything that edits their adverts or their profile. That task is just doing an async rendering to HTML of a profile like mine here..

Unwinding The Stack

Alright, so you saw how it's built, but to figure out how it works we should "unwind the stack" and go backwards from the Upload handler to an updated photo:

1. The upload handler gets a requests and pulls the picture data out.
2. It then uses the member model to set that pic on the owner.
3. After that's settled it sends a request on the photo_task connection to have each file (pic and profile) crunched.
4. When it's done it saves the member and also sends the Tir.Task for updating the profile pages.
5. Now, the photo task gets the request, and it just calls images.convert to make it happen.
6. Finally, images.convert does all the grunt work of cranking on the photo with Imlib2.

And there you have it, simple background tasks making photo uploads work with asynchronous background crunching and profile updating.

Possible Improvements

I can definitely see improving the images code so that the scaling and cropping keeps the same image size ratio. Right now everyone looks a lot thinner than they really are.

I'd also want to find out the security implications of this code. I've got no idea if Imlib2 is built for this kind of access. The good news is it's gotta go through a few checks before Imlib2 handles it, but code like this just sort of creeps me out. Which leads to my next improvement, putting the photo task in a chroot so if it does get hacked I don't have to care so much.

I can also see moving some more of this processing into the photo task. I'd probably try to save the file one time somewhere temporary, do an "upload progress" javascript meter, and then when the photo is uploaded and scaled/cropped they get the message that it's done. Mongrel2's large file uploads should make that easier.

Finally, implement the large file upload stuff so that I can scale back the buffer size and do the uploads async as well. Right now I just set the max photo size at 80k and have Mongrel2 block there. It'd be nicer to allow larger photos and do progress.

Now that I have these simple little 0MQ tasks there's a lot of this kind of work that is easy to offload. Other frameworks usually don't even have a concept of this backed in, so you end up using fairly gnarly background task solutions late in the game. In Tir I'm pushing that you can get your architecture going with little tasks before you need them, and since they use ZeroMQ scaling them out won't be some weird bolt-on afterthought.