Hugo's Blog

A few years ago I developed a voting system for use in the residence I was staying at. It evolved a bit over time, but each implementation or partial rewrite was a rushed job (one, maybe two weekends). The goal was to get something robust and reliable and easy to use, a later iteration supporting ranked ballots and Maximum Majority Voting. Making it easy to set up was not something I had had time for. Consequently, you need an experienced Linux geek to do that part.

In the years after I left, I was still in town. So I could go help them set it up when the time comes each year. 2006, 2007... and then the fateful 2008. I recently accepted a job in Europe, so I was packing up. Didn't have the time for the rewrite, as I had since realised it would take quite a bit of effort. But I could still set it up, shortly before I fly. And do the rewrite for 2009.

That's where we learn reliability lessons, lessons that we all already know, but brought into sharp focus the hard way: you need a failover system. A few days after the election, I found out the computer I set it up on had died, and they didn't stand a chance getting it running again on another, so they went back to a paper election. <cringe>

Anyway, to cut a long story short, the time has come for the rewrite. The election usually takes place around August. I heard rumours they may use a different system (rumours of another system being developed). If they do, I don't blame them. If they do that, it would probably mean bye-bye to the Condorcet voting method? :-( But all of that is out of my hands. I'm rewriting my voting system anyway. It unfortunately spells delays for Other Important Personal Pet Projects (TM), as I'm setting an end-of-June deadline for this rewrite. (Optimism: I should be done by mid-June? At least with Part One of the rewrite plans.)

The rewrite will have two phases: the first is an implementation running on a local machine, to mimic the way we used it in the past. Setup needs to be "download and run". Secondary goal, if it proves easy, which I hope it does: support any operating system for which Python is available.

Second phase: implement it on Google AppEngine, then it will be available for everyone, everywhere. Which will not be diversely useful until I implement a more online way of handing out ballots, the original system is meant for real-world elections, people are ticked off on a list and given a ballot identified by a random number. An online vote has some other considerations to take care of.

Technical Details

The old (current) system had a crazy set of dependencies ­­— implemented on what I knew well at the time, rather than what would be easy to get up and running. Since I was there to run it. Dependencies:

• Python
• PostgreSQL — I used database constraints for pretty much all of the error checking. The Python code merely catches the exceptions and reports the errors.
• psycopg for the python adapter to connect to the database
• python-numeric for the 2D array class to tally the vote
• Quixote (first version 1, later version 2) as the web "framework"
• python-scgi for the server side of the scgi protocol
• Apache for the actual webserver — and I used Apache's authentication to do access control
• libapache-mod-scgi for connecting to the voting system via the scgi protocol

Installing all of these is pretty simple using a good Linux distribution. If you're already a Linux user and know your way around the packaging system. Though installing it yesterday on my Ubuntu laptop, I had to manually enable libapache-mod-scgi.

Thus, setting it up requires:

• Create the database, create the database user, set the user's password.
• Configure postgres to allow passworded logins from localhost.
• Run the createtables.sql script to create the tables needed.
• Configure Apache to serve the relevant URLs via SCGI
• Create two users and two groups for Apache's authentication
• Edit common.py to configure the database (username, password, database name)
• Install the vote system, and run it so that Apache can talk to it

This is all documented in the README, but written at a level that an "expert" Linux user (rather, a system administrator) can easily follow. (I mean, I could easily follow it year after year. :-P) But how many expert Linux users / SA's are there in a student res with 200 people? And this isn't the one known as the "nerd res" on campus? ;-)

Phase One of the rewrite will do the following:

• Chuck out Apache, chuck out Quixote. Use CherryPy instead, let CherryPy be both the framework and the web server. Who needs Apache, on such a small scale?
• Implement authentication in the Python code, we don't have Apache anymore.
• Chuck out PostgreSQL in favour of SQLite. Or anything else for that matter? Something that handles multiple threads or processes well enough though. I'll have to research this a little bit. (Any tips from any geeks that happen to be reading this? ;-) ) SQLite is included in Python these days, so it isn't an external dependency.
• Implement error checking in the Python code, since SQLite doesn't do database constraints and thus can't do my error checking for me.
• Without Apache or PostgreSQL, we don't need psycopg or scgi either.
• Chuck out python-numeric. NumPy is the way of the future / present / recent past, but I'll even go so far as to give standard Python datastructures a go first. Not that dependencies matter, if I'll be doing the next point as well:
• Try py2exe to make a self-sufficient executable for running in Windows. Otherwise, I may try to create a nice graphical installer. I have done that before.

Consequently, getting the voting system up and serving (as a website) should be as easy as "install it, run it". The rest of the configuration should happen via a web browser. It's going to be a lot more work than I've put into it in the past.

Phase Two drops SQLite and CherryPy in favour of Google AppEngine.