boingboing.net's excellent tribute

No, it’s not the iPad, the iPhone, or even the iPod. It’s definitely the Mac. Steve Jobs real gift to the world was bringing the Mac to the market in 1984. The Macintosh was a major step in making personal computers pleasant and easy enough to use to gain mass market appeal. If the Mac hadn’t been released then, the industry, our industry, would not be where it is today. You likely would not be reading this, you would not be on Facebook or Twitter, you wouldn’t know many of the people you know, especially those met online.

The Mac was never the market leader, but it’s existence opened peoples eyes. Computers HAD to be easy to use like the Macintosh, cryptic commands typed into dark screens weren’t going to cut it. This changed the game, the competition released competing products and the personal computer industry as we know it was born.

That’s was what Steve did. He didn’t invent anything brilliant. He took fantastic people and technologies and integrated them into products that created market sectors that didn’t exist. He took huge gambles, that industry experts almost always said would not pay off. Sometimes they didn’t. But others really did change the world. I know it’s trendy to hate Jobs, Apple, heck, anything popular with hipsters. Just don’t forget the value of what Jobs accomplished with his much too short life. Certainly the world would look very different today if the modern computer as we know it was still sitting in the halls of Xerox and the labs of Universities for another 5 or 10 years as everyone continued to type cryptic commands into their terminals…

Rest in peace Steve, you changed my life, and for that I will be eternally grateful.

First the good. The UI changes are great. I have always been a fan of tiny widgets and maximizing screen real-estate. In 10.6 and prior, I went to great lengths to shrink every font and widget. In 10.7 theres no need. Scroll bars are tiny and automatically fade out, you can full screen most apps, fonts and widgets are just smaller. Fantastic. New UI animations and transitions are everywhere and delay things a bit, hopefully there will be a TinkerTool or similar to disable them. They’re short and tolerable but as an example, I am typing a character or two into the ether when switching spaces during the animation.  I’m not really into the LaunchPad paradigm, but the MasterControl look works for me. Mail.app’s new UI is fantastic, iCal’s is a bit over the top. Mail.app’s performance (specifically around large operations and anything RSS related) is a complete train wreck. I expect a patch soon. Reverse scrolling took less than a day to adjust to (I am an iPad/iPhone user though). Autocorrect is a nice addition. The new Finder is great, Safari updates seem good, heck even the Terminal.app updates are nice! Grab an updated version of TerminalCopyOnSelect and away you go.

10.7 is a huge update, in the way 10.5 was to 10.4, and at $29. If Mail.app’s performance is fixed I’ll likely be very happy.

Here’s a very WTF bug though. I ran into it last night and figured it was just me, or I’d boned something up. I have a bunch of shell scripts that do SSH port forwards for accessing network appliances and infrastructure behind lock and key. Half of them stopped working on me, failing to resolve their respective hosts. How strange. Amusingly Brandon was complaining of basically the same problem this morning so we decided to take a look.

Some how OS X’s resolver library is no longer checking any hostname with a dot in it against the /etc/resolv.conf search directive in 10.7. Seriously. Let’s say I have a server called admin.omghi2u.com. My /etc/resolv.conf (and OS X network control panel) contain omghi2u.com in the search field. Surely, we can resolve (ping, ssh, web browse, whatever) to ‘admin’ since it can match that as admin.omghi2u.com. Cool. Now normally, in UNIX (heck even Windows land!) if I had admin.tor.omghi2u.com and admin.chi.omghi2u.com, simply hitting ‘admin.tor’ or ‘admin.chi’ would match the omghi2u.com hostnames. Not the case in 10.7. Something is clearly broken in the resolver in libSystem. Oddly despite being linked to the same library, the host command still functions properly… Maybe the search logic is handled by the command itself…

While it’s a simple bug fix, its an annoying oversight on Apples part. Breaking basic UNIX networking functionality is kinda shameful.

UPDATE (JULY 26th): Head over to Brandon’s site for a fix!

I’m not a huge bash guy,  in fact nine times out of ten I’ll chose perl. I like the way perl hands strings, escaping and its general syntax better. Sometimes however, it’s just better to use bash, say for a cron, especially if you can pull it off in one line (instead of maintaining a script on disk). This is definitely advantages when you are adding a cron to an appliance, it’s nice to maintain everything that isn’t provided by the managed distribution in a single location (i.e. root’s crontab)

I’m a big fan of F5, the Big-IP product line is fantastic as is their support. Theres definitely a lot of way to get alerting, the best of which would be SNMP, or even have the included alertd directly email your pagers. Personally, I don’t have an SNMP driven alerting system, 99% of our devices/systems are actively monitored by dedicated monitoring systems. Modifying alertd has the problem that you have to port your changes forward during any OS updates, and we are currently split between OS 9.x and OS 10.

I decided to write a quick little non-intrusive script (no changes OS configuration, nothing to maintain changes to), and keep it as a single cronable line, to comb the LTM logs (though this could easily be used for GTM logs as well) and email out notifications. The log looks like:


Jun 8 17:31:09 local/tmm notice tmm[1823]: 01070028:3: No members available for pool db-cluster.omghi2u.dev
Jun 8 19:12:17 local/ltm1 notice mcpd[3377]: 01070640:5: Node 192.168.110.82 monitor status down.
Jun 8 19:13:51 local/ltm1 notice mcpd[3377]: 01070728:5: Node 192.168.110.82 monitor status up.
Jun 8 19:13:51 local/tmm notice tmm[1823]: 01070028:3: No members available for pool web-cluster.sup2u.qa
Jun 8 19:15:08 local/ltm1 notice mcpd[3377]: 01070727:5: Pool member web2.sup2u.qa:80 monitor status up.


The log formatting in OS9 is a bit less verbose than OS10, but basically it always starts with a timestamp formatted %b %e %H:%M:%S, then the log entry (OS9 lacks the log level and context). We want to check every 5 minutes (could be bumped to every minute) for any new entries in the last 5 minutes, and email them out if they aren’t stuff we don’t care about.

Some cool stuff of note: if you want to do a for loop on new lines instead of any whitespace, you need to change the IFS variable around. Just make sure you unset it when you are done or you will screw your terminal (or cron run!) up. The code originally looked like:


export IFS=$(echo -en "\n\b"); guts=$(for i in `cat /var/log/ltm | awk '{print $1 " " $2 " " $3}'` ; do unixtime=$(date --date=$i +"%s"); if (( unixtime > `date --date="5 minutes ago" +%s` )); then grep `date --date="@$unixtime" +"%b %e %H:%M:%S"` /var/log/ltm; fi; done | sort | uniq | grep -v -f /root/ltm_excludes.txt); if [ -n "$guts" ]; then echo "$guts" | mail -s "$HOSTNAME logs" "ops@zomghi2u.com"; fi; unset IFS;


And worked like a charm on OS10, but OS9 is based on Redhat 3, and has an ancient version of the date command that didn’t support the @timestamp format. Lovely. Theres lots of people using the pure date command to get around this, but timezones become a problem and are messy. It’s way better to use awk’s wrapper to strftime() and thus our fully backward compatible cron is:


*/5 * * * * export IFS=$(echo -en "\n\b"); guts=$(for i in `cat /var/log/ltm | awk '{print $1 " " $2 " " $3}'` ; do unixtime=$(date --date=$i +"\%s"); if (( unixtime > `date --date="5 minutes ago" +\%s` )); then grep `date --date=\`echo | awk "{ print strftime(\"\%c\", $unixtime) }"\` +"\%b \%e \%H:\%M:\%S"` /var/log/ltm; fi; done | sort | uniq | grep -v -f /root/ltm_excludes.txt); if [ -n "$guts" ]; then echo "$guts" | mail -s "$HOSTNAME logs" "ops@zomghi2u.com"; fi; unset IFS;


Only thing I wasn’t totally happy with was that I had to do a dumb echo | into awk. I couldn’t figure out (from the man page and a quick googling) how to get awk to do it’s thing without stdin or a file. Oh well. That was a lot of fun to write. You could change it to a whitelist by making a /root/includes.txt kind of file and losing the -v on grep. In fact you could have two crons. General alerts goto your ops inbox, alerts you are worried about (like pools having no members left :D) go to your emergency inbox (pagers). Or many crons. Or actually just hack up the alertd.conf… Or alert on SNMP! Either way, happy scripting!

UPDATE: (June 21st, 2011) On a day filled with a particularly large number of port scans resulting in lots of grep-filtered RST response messages, we decided to move the grep -v up to the beginning instead of at the end, this increases performance immensely. No more spikes on the CPU0 graphs! Here’s the updated script, smarter logic this time ’round:

*/5 * * * * export IFS=$(echo -en "\n\b"); guts=$(for i in `grep -v -f /root/ltm_excludes.txt /var/log/ltm | awk '{print $1 " " $2 " " $3}'` ; do unixtime=$(date --date=$i +"\%s"); if (( unixtime > `date --date="5 minutes ago" +\%s` )); then grep `date --date=\`echo | awk "{ print strftime(\"\%c\", $unixtime) }"\` +"\%b \%e \%H:\%M:\%S"` /var/log/ltm; fi; done | sort | uniq ); if [ -n "$guts" ]; then echo "$guts" | mail -s "$HOSTNAME logs" "ops@zomghi2u.com"; fi; unset IFS;


Solution: Ask the switch who lives where, make perl do the heavy lifting!

Our HP ProCurves provide what they call ‘port address tables’ which basically tell you which MACs are seen on which ports. A bit of digging and that info was just an SNMP call away. The MACs are stored in decimal format, in the OID, and the integer value it points to is the port number. Ports after physical ports (on ProCurves) are VLANs and Trunks. (The mappings and locations of ports is probably completely vendor specific though!)

Once we have that MAC address, we can pretty it up, map it to an actual hostname, and then label the ports with that. No more will you have to check another table, database, text file, etc and say, hey where is XYZ plugged into… The 30 minute dirty perl code:

#!/usr/bin/perl
# switchLabel.pl - 5/10/2011 (dirty 30 minute hack version)
# label ports on an HP Procurve switch with their boxes MAC address (easy to lookup)
# and even their hostnames! --nick@fmpub.net

# configs
$debugMode = 1;

# code dont touch
if ( $#ARGV < 1 ) {
       print "usage: switchLabel.pl [switch IP/hostname] [community]\n";
       die("Please specify a switch IP and a valid r/w community string!\n");
} 

sub debug
{
        use vars qw($debugMode);
        if ($debugMode == 1) {
                print $_[0];
        }
}

my $switchIP = $ARGV[0];
my $snmpCommunity = $ARGV[1];

my $numPorts = 48; # after this is trunks and vlans, which we're going to ignore
                                  # label them manually kthx
                                  # change me for big chassis switches
                                  # could become another (optional) config variable

my @snmpWalk = `snmpwalk -On -c $snmpCommunity -v2c $switchIP 1.3.6.1.2.1.17.4.3.1.2`;
my %switchPorts = ();

foreach (@snmpWalk) {
 	$_ =~ s/.1.3.6.1.2.1.17.4.3.1.2.//; # lazy ass formating
	$_ =~ s/ = INTEGER: / /;
	chomp;
	($decMac, $port) = split(/ /, $_);
	if ($port <= $numPorts && $port > 0) {
		my @macOctets = split(/\./, $decMac);
		my $hexMac = "";
		foreach (@macOctets) {
			$_ = sprintf("%0.2X", $_);
		}
		$hexMac = join(':', @macOctets);
		debug("[found] $hexMac @ $switchIP: port $port\n");

		# put code in here to map MACs back to boxnames
		# or leave a MAC for now...
		my $machineName = $hexMac;

		if (!$switchPorts{$port}) {
			$switchPorts{$port} = $machineName;
		} else {
			$switchPorts{$port} .= ", " . $machineName;
		}
	}
}

for ($i = 1; $i <= $numPorts; $i++) {
	if ($switchPorts{$i}) {
		my $portName = $switchPorts{$i};
		if (length($portName) > 64) {
			$portName = substr($portName, 0, 61) . "...";
		}
		debug("[assign] $i => $portName\n\t");
                debug(`snmpset -c $snmpCommunity -v2c
                      $switchIP .1.3.6.1.2.1.31.1.1.1.18.$i s "$portName"`);
                 #the above line was broken in half for awful wordpress formatting
	}
}

I’ve removed our implementation specific MAC address to hostname mapping bit. Basically you can tackle that a few different ways, do an ARP lookup on a box that sees and knows all, query an SQL table where you do inventory, whatever you want. Just drop it in, and if (!machineName), put the hexMac in instead (i.e. you have an unknown box!) You could (and I probably will) extend this code to alert you if boxes move around (i.e. without you knowing), for audit-ability purposes.

This awesome reverse mapping of MAC addresses looks to be the same as Cisco’s dynamic CAM entries table, and they are on the same OID. (In fact I wager Cisco came up with that specification) That means this above code should work on a Catalyst with no modification but I haven’t tried it :) I have (and always will be) a Cisco big iron fanboy, but these ProCurves are totally awesome for anything other than edge and carrier distribution. All the SNMP goodies you’d ever need can be found on a sub-$250 HP 2510, and thats with a lifetime warranty and support…. (insane)

Grab the code and play around and always remember, snmpwalk is a network admins best friend!

It's a UNIX system, I know this!

I’ve spun up an IRIX machine at home to replace my long dead Linux based file server / general network management box. (It’s a 2-node IP45 Origin rack, so 8 sockets of R14000 lovin’) I don’t really like to have more than 1 active server and 1 active desktop at home at a time for simplicity / heat / space reasons. I’ve been an avid IRIX user since 1990, in the golden days of the Personal IRIS and 4D/480. It was my first UNIX and will always have a special place in my heart. SGI has retired the platform and it will go End of Support in 2013, and the final generation of IRIX hardware is becoming outdated on a GFLOP/watt basis now, and thusly affordable!

IRIX is POSIX, POSIX2 compliant, really strictly. It was based on SysV and later got several BSD extensions added, but anything that isn’t mandated with POSIX probably isn’t there unless they wanted it within the halls of SGI. It’s profiling support, native debugging tools and general sysadmin usability are still unmatched. I can still install and configure an SGI box in my sleep, I will definitely be sad the day the final IRIX hardware is thoroughly useless performance wise.

Enough of the background though, I didn’t want to turn this into a sappy rant over a dead platform. I’ve been porting stuff to IRIX for a decade at least, I find as an exercise it’s very enjoyable. I’ve often found myself porting stuff over only to never use it or even nuke it for the sake of the exercise itself. I find it generally makes you a more ‘aware’ C (and UNIX) programmer, especially of stuff outside of the realm of Linux and GCC. I ran into a really cool one late last night. I’ve been working with a bunch of caching engines, on all layers of the equation (object, code, front end), and while I do all of my actual performance testing in Linux land, I’ve been doing some IRIX ports for shiggles. I ran into in a pretty basic mmap which is what spurred me to write this article:

return mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);

Pretty basic, MAP_PRIVATE specifies that modifications to our memory map are private, that is our changes won’t make it to the file descriptor, it’s copy on write. MAP_ANONYMOUS (at least to me) always seemed like a given too, in Linux/BSD/Solaris it ignores the file descriptors and just gives you a zero’d memory. OS’ all seem to implement this differently, just reading the OS X man page it looks like you can pass flags to the Mach VM about tags and its purgability. IRIX (and HP-UX it seems) completely lacks it. No biggie! Sure enough we can accomplish the same thing thusly!

static int devZero;
devZero = open(“/dev/zero”, 2);
return mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_PRIVATE, devZero, 0);

Easy peasy. Open a file descriptor to /dev/zero, kernel then knows what to do. Worked like a charm, fast in memory caching was a go!

Ok so this is an OS X portability issue, but IRIX lacks daemon() too. The error was just too good to not include in a portability article :P

Let’s talk about some other Linux/IRIX/UNIX/whateverisms. I’m just going to skim the surface but here we go….

You also run into lots of cases where IRIX’s libc does not support many ‘givens’ in the Linux (and even FreeBSD and even sometimes Solaris’) libc. I was working on porting varnish to IRIX on Sunday morning, and while we’re still in the syscall tracing phase due to some misbehaving of its internal code compiler, but I’ll use it as an example as it had two of the most common functions that are easy substitutions I see all the time:

setenv() setenv was added to UNIX version 7 back in 1979, somehow IRIX completely lacks it. It’s pretty easy to throw putenv in for most cases though. Looking at the varnish source we swap:

        AZ(setenv("TZ", "UTC", 1));

for

        AZ(putenv("TZ=UTC"));

Same goes for wait4(), it’s part of System 4 but not specified by POSIX so IRIX completely lacks it. Thankfully it’s what waitpid() is implemented with on many platforms (BSD), so if the *rusage isn’t specified, we can do a direct swap thusly:

        r = wait4(v->pid, &status, 0, NULL);

becomes

        r = waitpid(v->pid, &status, 0);

Varnish needed a few other tweaks, stuff like IOV_MAX isn’t specified in any IRIX headers, but can be gleaned from sysconfig (1024). CLOCK_MONOTONIC is not defined, but thankfully they had Solaris conditionals around it that became Solaris and IRIX conditionals.  Fun.

Onto GCCisms. GCC is a great compiler suite. Supports a ton of software/hardware, but it will never been the fastest on any platform. On SGI’s we have MIPSpro and on PC’s you have Intel’s really awesome compiler suite. (We used to use Intel’s suite at Cedara for our x86 medical imaging software, strictly based on its killer performance). I tend to try and port stuff to MIPSpro unless you run into too many bad GCCisms. Sometimes I think that GCC being the defacto ‘nix compiler, has lead people (likely CS students) to believe thats just “the way things are”.

Zero length arrays i.e.

char contents[0];

Ok so it is a clever way to throw a place holder into a struct, but this could be totally handled with a char *contents; pointer too.. Not portable though…

Variable length arrays i.e.

char str[strlen (s1) + strlen (s2) + 1];

Again, its a clever way to do really basic memory handling without thinking about it, but ugh. C99 does support variable length arrays, but GCC allows them in C89 and C++ code too, just to muck shit up. Don’t use these outside of C99 code, it will not be portable.

There are so many more, but these are the most common I tend to see. GNU has a guide on their extensions to the C language, and totally there is lots of really useful stuff there, but just know that it breaks portability. This rant would have a lot more meaning if people used much other than GCC, but nowadays even embedded development and video game consoles are moving toward GCC. At this point in time I honestly don’t see another compiler suite beating GCC but who knows…

Code, port, hack. Expect more cooking stuff soon :P

The main difficulty is grabbing the To/From headers. You can use bash using the SHELL directive within a procmailrc file, however we don’t want to process all the email (since this is a relatively intensive process). So we wrap everything in a conditional for our domain. Once the { } brackets come out, procmail acts as if its a totally new .procmailrc script, and you can perform your shell scripting before delivery (or matching) as desired. The script:

SHELL=/bin/sh
LOGFILE=procmail.log

:0
* ^TO *mydomain.evul.net
{
    FROM=`formail -x"From " | sed 's/^\([^@]*[ <]\)//' | sed 's/\([ >]\).*$//'`
    TO=`formail -x "To:" | sed 's/^\([^@]*[ <]\)//' | sed 's/\([ >]\).*$//'`
    :0
    | /usr/bin/php /path/to/my/script.php ${FROM} ${TO}
}

Efficient? No. It’s a good 8 forks for every email, and I don’t wanna check how many fopen()s… Such a solution would not work for any high volume of email, but for a relatively low (say thousands/day) its good enough.

Now, /etc/security/limits.conf and /etc/security/limits.d are read by PAM’s pam_limits.so, so only things that use PAM will ever touch these. Apache starting on a server reboot is not affected by these limits, so Apache will have access to its full 106,496 threads. But if root restarts apache from a shell (/etc/init.d/httpd restart or service httpd restart), Apache will inherit root’s proc limit of 1024! This can be verified with a <?php passthru(‘ulimit -u’); ?> via mod_php.

1024 threads is totally not enough for reasonable operation in Apache. It will begin to kill children off as they seteid() to its designated user as the call fails with errno 11. (seteid() has not always performed an nproc check, this was added in some 2.6 kernel to prevent people from sneaking over the limit) You can work around this by dropping a ulimit -u <much higher number> into /etc/sysconfig/httpd, but my preferred solution is to remove (blank) RHEL6′s new /etc/security/limits.d/90-nproc.conf . (Or I guess you could just never restart your apache, best to reboot the system if theres a problem :P) Thanks Redhat.

Back to that large 106,496 number. Thats the maximum number of procs that nick as a user can spawn, even using ulimit -u he can’t go higher than that. The system’s maximum number of threads (visible in /proc/sys/kernel/threads-max) is 212992 (exactly double) is decided at boot time by the kernel. I decided to do some digging in the kernel source and the maximum number of threads is calculated thusly:

max_threads = totalram_pages / (8 * THREAD_SIZE / PAGE_SIZE);

(defined in kernel/fork.c, called by init/main.c)

PAGE_SIZE is architecture specific, for x86 it’s 4kb (and 4MB but thats a discussion for another day), and is calculated via:

#define __AC(X,Y)       (X##Y)
#define _AC(X,Y)        __AC(X,Y)
#define PAGE_SHIFT      12
#define PAGE_SIZE       (_AC(1,UL) << PAGE_SHIFT)

PAGE_SHIFT is a constant for x86, other architectures support alternate (larger) page sizes. THREAD_SIZE is also architecture specific:

#define THREAD_ORDER    1
#define THREAD_SIZE  (PAGE_SIZE << THREAD_ORDER)

THREAD_ORDER is 1 in x86, but can vary depending on the arch. These result in a PAGE_SIZE of 4096 and a THREAD_SIZE of 8192 for x86. Back to the formula  above, we can fill it in as:

max_threads = totalram_pages / (8 * 8192 / 4096);

becomes

max_threads = totalram_pages / 16 ;

Calculating totalram_pages is a bit tricky since its not just the full system ram allocated to pages. The easiest way to calculate it from /proc/zoneinfo, by summing the spanned pages:

cat /proc/zoneinfo | grep spanned | awk ‘{totalpages=totalpages+$2} END {print totalpages}’
3407872

So on my server with 12 gigabytes of memory we’ve got:

max_threads =  3407872 / 16;

becomes

max_threads = 212992;

Now the default maximum number of threads is designed to be able to only consume half the memory from threads alone. But one user should also not be able to consume all of the threads on the system. Back in kernel/fork.c we have:

init_task.signal->rlim[RLIMIT_NPROC].rlim_cur = max_threads/2;
init_task.signal->rlim[RLIMIT_NPROC].rlim_max = max_threads/2;

So there you have it, the ulimit -u for nick (or root) is now set to 106496 procs (threads). nick is unable to take up all of the processes on a box, even using ulimit, he cannot raise past max_threads/2 (106496, though root can). Unless modified via /proc/sys/kernel/threads-max, a maxed out system will not use more than 50% of the memory to store thread structures. Hopefully you too now know a little bit more about linux process limits! Big thanks to colleague and friend Jim Hull for his troubleshooting and major help last night.

The exact formulas may vary kernel to kernel, my findings were through the kernel 2.6.37 source and verified on RHEL5′s heavily modified 2.6.18-194 kernel

Another thing I love are QR codes. Original created by DENSO as a way of streamlining their huge automotive parts manufacturing business, they are essentially a high capacity barcode. QR codes became popular in the land of cell phone cameras (Japan) to point URLs for extra product info, free ringtones, images and for contests. This trend has of course spread here and is a lot of fun.

The premiss is simple. Anyone with a camera equipped phone (and nowadays this is almost every phone) can take a photo of your QR code, then using one of many free apps, translate it into a URL. One of my favourite QR code contests involved finding five different ads (for different flavours) of Maynards candy. You had to plug an email in for each one, and when you had obtained them all, you would be entered into a draw to win one of several prizes. I scoured the subway car I was in to find all the different ads (which were all different sizes), I was involved, I was smiling, it was a good execution. Of course I didn’t win but I enjoyed it anyway.

I recently encountered another QR based contest which was a give-away for a clothing store. It was pretty simple, you had to scan the code on the back of the catalog, and they would give away one gift card every day for a month and a half. My love of QR codes and these mobile contests lead me to enter it. Upon scanning the code I realized that the URL I was pointed to was non unique, that is, every catalog had the same QR code. When my phone actually visited the URL I was told I was not a winner. The experience was simple and quick which is definitely good. However when I began thinking about it, the simplicity of this contest is a serious flaw.

Since the contest declares an instant loss (or win), someone would just have to hit the contest repeatedly, looking for anything but a loss. Throw in some common mobile phone user-agents and a small pool of IPs in a roulette, you no longer look like a bot. Throw in randomized but short delays in between attempts and you are beginning to look human, moreover, like many humans. Pretty soon our poor contest has little defense against even a novice hacker…

The whole premiss of this contest is that there is one winner a day, however since you are declared a loser (or winner) right away, how do they randomly chose a winner? Basically they are rolling a huge dice with a very large number of sides, say a hundred thousand. A winner could be declared a minute after midnight making the entire days contest moot, or a minute before midnight. Once a winner is found, the contest stops rolling and declares everyone a loser. A true daily winner would require selecting one from the true pool of entrants during a 24 hour period. Asking for an email and randomly picking a winner at midnight accomplishes the goal and has real advantages for fraud prevention. We’ll talk more about using an email address later.

I think the #1 concern of any developer of a contest (web or otherwise) should be how to stop gaming, hacking or fraud, whatever name you chose to give it. :) You need to be able to accurately track entrants, and for the web this can be accomplished in at least three ways. The first is through an IP address. However this becomes a problem since many ISPs use proxy servers to keep their transit costs down. It becomes impossible for a primarily mobile contest as most phones do not get a unique IP, they are NAT’d out one (or a small handful) or IPs assigned to a specific cell tower. The second way to track entrants is through a tracking cookie, however defeating this for someone interested in gaming your contest is trivial. Do not rely on cookies to declare someone unique! The third solution is utilizing some sort of external tracking method, that is, do not track them through their hit of the contest URL. The best way to do this would be actual unique QR codes tied to each consumer’s catalog, with some real cryptography behind the unique string. Unfortunately this would add to the cost of printing the catalogs and may not be an option. However there are alternative methods of tracking entrants external to the actual web hit.

With most web based contests you would require an email address of an entrant. Creating fake email addresses isn’t difficult, but its a good hurdle and one thats fairly easy to detect. (Did you see a million entries from zomg48.info? it’s probably junk, filter it. Do a daily report on top email domains: audit audit audit!) However, since this is a mobile contest an email verification might break the flow of the app since you no longer declare that instant loss (or win), and it requires a person to enter a potentially large string (their email address) on a cell phone.

Alternatively, if you must declare an instant winner, you can always use a captcha. Captchas are not fool proof, but they are certainly a hurdle that would dissuade most fraud. The real draw back to captchas is that they annoy users enough on desktop web applications. Mobile web apps needs to be streamlined, and easy to use on small screens. Captchas aren’t particularly good at either of these things…

I really didn’t like the email approach (breaks the mobile flow), nor do I like captchas on mobile platforms. In talking to my good friend Brandon, he had a good suggestion. When a user hits the QR encoded URL, take them to a page where they enter their mobile number. From there you can either declare an instant winner via SMS or even SMS them at noon the next day for a true daily winner. SMS them a secure unique URL to hit to fill in their personal details, then dispatch their prize. An instant SMS wouldn’t break the flow, since SMS delivery is pretty much guaranteed as a capability of anyone entering your contest, where as email isn’t reliably instant for non-savvy smart phone users. Compared to an email address, mobile numbers are also much easier (and quicker) for cell phone users to enter on a page.

I guess to summarize the above: