By Zed A. Shaw

Mongrel2 Cannot Defeat Ninjas

My favorite part of the sqlite3 design choice in Mongrel2 is the fact that the bikeshedders have latched onto it and waste their energy there. I thought they'd be wasting time with the ternary search tree but I guess that's too much computer science for the average bikeshedder. Nope, it's gotta be the config file.

A really funny thing that comes out of this, is summarized by this statement from Chris Stucchio:

I don't agree with all your choices, particularly sqlite for the configuration. I would have preferred something more human readable and more git friendly.

Odd, I mean, I put the configuration files in fossil all the time. I'm confused, is git just not able to store text files. Hmm. Let me check.

/me puts the config files in git.

Yep, works the same. Config files in git. Right there. Hmm, I don't get it.

The reason people have this fear of "not being able to put the config in git is apparently this bizarre paranoia of Ninjas. I summarized it in ticket 9e57f999d5 where we plan to create the Ninja Proof Configuration:

Determined this scenario:
1. You hire ninjas who secretly want to destroy your company.
2. Late at night, these ninjas go onto your servers where mongrel2 is
running, and they inject SQL queries directly into the config that fixes
problems, but they don't update the mongrel2.conf to reflect these
changes.  Their master plan is to wait until you have a problem and then
when you update the db from the mongrel2.conf *bam* your whole site goes
down because you don't have the changes they put in recently via SQL.
Devious. Brilliant.
3. To prevent this from happening, and you know actually it'd be useful
for a few other reasons, we need a "m2sh diff".  You would give it a
-db, a -config like normal, but it would compare what it knows about the
config to what's in the database and report any differences.
Then people can do this:
"OMG NINJAS!"
"Quick update the site it's down!"
"Holy crap, what if they changed the config when we weren't looking?"
"I know, I'll run m2sh diff and see if they've been sabotaging our site
with fixes that we don't know about."
"Aha! they have! Those bastards."

Yes, quite literally people want to have a text file be the only thing Mongrel2 uses to configure itself because they are worried that bad evil sysadmins will use SQL directly to sqlite3 to make hidden configurations. It's even more stupid because no matter where or how you store your configs, a bad sysadmin will still screw this up. Putting them in git doesn't solve the problem at all, and recovery from the problem is the same involving updating from git and regenerate the deployment.

But, this just boggles the mind. What kind of shop are people running where:

  1. You don't trust the people working for you this much, or
  2. You have people this completely idiotic working for you that you can't fire?

Think about it like that. You have people who actually think that some asshole is going to go onto a box, use SQL to "backdoor" a config change in, skip using the easier config file we've created, and then not update the config file to reflect the changes. People actually believe that lazy sysadmins will go to this great of a length to make their jobs harder.

Alright I'll give you that, sure let's say you've got some asshat who's like this. You can't fire him? Reprimand? If he didn't have this he'd find some other way to screw up your day. Why not get rid of him? Oh that's right, your company is so totally screwed up that you hired this guy in the first place, gave him access to trusted systems, and then when he totally and completely screws the pooch you can't fire him.

I'm sorry people, but if your house is so completely fucked that you have this problem, then no amount of technological engineering will ever help you. You are absolutely and totally screwed.

P.S.

Chris, thanks for helping, you're awesome man. Very well written explanation of what someone wants, with code, and I totally get where you're coming from. This post was not directed at you.