Long live the average programmer



They say that great programmers are an order of magnitude more productive than average programmers. Wikipedia would want me to specify who "they" are (Frederick Brooks and Joel Spolsky for starters), but this is not a scientific assertion. Any programmer who has worked with really smart colleagues will, perhaps grudgingly, admit that it is by and large true.
It doesn't only go for programming, though. True excellence in any endeavour is rare (Mozart, Michelangelo, Monty Python). Excellent achievements really stand out. That's why we call them excellent. If excellent programmers weren't rare they would be average. So much for tautology.

Experience doesn't count for much. It doesn't take you a lifetime to become excellent at something, and that's a soothing though. You find out quickly enough whether you really have great talent. I started playing the piano almost thirty years ago and I quickly made progress during my early teens, but after that it kind of stagnated. Child prodigies, by comparison, don't stagnate after a few years. They start playing when they're four and by the time they're sixteen they baffle the crowds at the major concert halls. Great athletes don't win the Tour de France once or twice: they win it at least five times (Merckx, Hinault, Indurain, Armstrong).

There's no shame in being out of these people's league. Most of us are. You can still be a competent pianist and avoid the fiendishly difficult sonatas of Franz Liszt. You can be a competent programmer and not be able to reverse-engineer the Linux kernel.

Should a software company fire people like myself and hire a genius, who is ten times more productive but takes up the same amount of office space? Of course not. To begin with, geniuses are very rare and even if you paid them triple wages (which would make very good business sense) a genius is primarily motivated by the satisfaction that practising his art gives him, only secondarily by the amount of money he can make with it. If you have a genius in your team, you better make sure to give them some real challenges, or they get bored before you can say "private office with Aero chair".

Most programming work I have done over the years takes smart people, not geniuses. In fact, some of it was pretty mundane and boring. Most software jobs don't require inventing faster sorting algorithms or more reliable floating-point arithmetic. At least mine didn't. I would have been completely clueless anyhow. More likely they require building a user-friendly web form that handles phone numbers intuitively rather than insulting customers with "010-123456" does not match regex validPhoneNumber Javascript popups.

If it takes me a day to fix a simple task, the star could probably do it in half that time, but I guess the simpler the task, the smaller the difference. Partly because the star would get bored or consider it an insult of their intelligence. Molest me not with this pocket calculator stuff, they will reply haughtily when asked to hack a few AJAX web forms. Give them something really difficult to do, because that's where they shine. How much faster than myself could they reverse-engineer the Linux kernel? It does not compute, because I simply wouldn't be up to it. Not in this lifetime, with this set of brains at least.

The Page Paradigm

Following Jeff Atwood's tip I read Dan Ariely's excellent book Predictably Irrational, which gave me all sorts of revealing insights into my own irrational psyche.

Anchoring is the psychological phenomenon explaining how we human beings are reluctant to update our ingrained opinions. This relates to our sense of value (cheap or expensive) as well as our conception of the validity of doing things a certain way. We like to stick to what we know. If you grew up in the fifties, 200,000 Euro for a house will always feel like an obscene amount, and if the Commodore VIC20 was your first computer any file over a megabyte just feels huge. I remember a C programmer lambasting me in very derisive terms for programming with mod perl (an embedded Perl interpreter in the Apache web server) because it added a mere 4Mb to Apache's memory footprint. Note that this was seven years ago, in 2002.

If you built dynamic web sites around the turn of the century, you'll have HTTP, HTML, JavaScript with either Perl, JSP, PHP or ASP paradigm anchored in your brain and you'll be used to doing things in a very time-consuming and cumbersome way. I don't care what anyone may say to the contrary: building fast, usable, secure and reliable web applications was and is a very tough job.

You could fill entire bookshelves lamenting the inadequacies in terms of security, usability and browser compatibility nastiness, but what's the point? Clearly the benefits outweigh the drawbacks. Embedded Java applets had their chance and we didn't want them. Web applications are here to stay. The underlying technology has evolved impressively, yet for a revolution we should stop thinking of web applications in terms of pages. It's time to haul that anchor and move on.

Originally the web was conceived as an information universe of static content, randomly accessible through hyperlinks. The back button has always been an indispensable tool. It allows you to follow any link on a page and quickly retrieve your steps. It's the equivalent of the undo function and it works well for static content.

A typical web application however does not take kindly to it. The function of a web application is performing a mandatory sequence of actions in which the user and the remote server typically exchange data through HTML forms. You search and pick a book of your choice, you provide shipping and payment details and then you submit your order. Presenting these stages as navigable pages with a back button is asking for trouble. Your browser knows this. It will warn you that you are “attempting to submit a form twice”, or something to that effect, leaving the less technically inclined users clueless. Bad programmers will even book you three seats on the same flight as a penalty for hitting the refresh button a couple of times. Where's the logic in that? The back and forward buttons look like undo and redo functions, but they're not. You can even bookmark a submitted form like this: www.initrode.com/order.cgi?orderId=12345;creditCard=1234234534564567;type=VISA;valid=10_2015.

Don't get me started.

Ajax to the rescue. Browser technology has now progressed to the point where you can build an entire client-server application to be operated from a single HTML page. Embedded or linked JavaScript libraries generate page elements on the fly and communicate asynchronously with a remote server without having to refresh the page. No more form submissions. And no more quirky back buttons. The Google Web Toolkit (GWT) framework even has undo functionality built in. Users can hit the back button and the application will behave appropriately, which also means notifying the user when an action is not undoable.

Usability guru Jakob Nielsen still did not think much of AJAX technology in 2005, when it was still a budding technology http://www.usabilityviews.com/ajaxsucks.html but he's slowly coming round and for a thousand bucks he'll tell you why.

Existing page-based technologies like PHP and JSP remain anchored in the pages metaphor. If you're a Java developer with plenty of experience in Swing GUI programming you should give GWT a spin. It's the most radical departure from the old school and I absolutely love it.

The Curse of Mr Fixit

There are two kinds of programmers. There are those who get back from work, shove a frozen pizza in the microwave and scoff it down while coding their own LDAP server in a little used but cool language. Then there's the motley assortment of CS, physics, history and language graduates with a life that doesn't only involve computers. In the first category you will find Mr Fixit (I have yet to come across a Ms Fixit).

Mr Fixit has a voracious appetite for new technologies. He has a l'art pour l'art attitude when it comes to information technology, where writing your own LDAP server for an address book with thirty entries makes perfect sense. He likes to think himself a free spirit. Anything that impedes his artistic flow (like coding standards and testing procedures) he hates with a passion.

He likes to disparage lesser operating systems and programming languages with a destructive zeal comparable to that of orthodox Muslims looking at a sacrilegious drawing of the prophet Mohammed.

He relishes the kudos that comes from getting a job done, not necessarily from getting the job done properly. He´s not afraid to use duct tape to stop a leak, but the duct tape is never replaced by proper solder. Where's the fun in that? He shows great stamina when solving a nasty classpath issue in the web server, but he'll often be curt and impatient explaining his solution to lesser mortals. That's because he likes to be smarter than you.

However, our Mr Fixit is a person of flesh and blood and writing your own Microsoft killer in the wee hours of the morning is going to tell on you one day. When Mr Fixit suffers his first major burnout the company will realize what trouble they are in for not having kept him in better check. Suddenly everybody realizes how they have relied on Mr Fixit and nobody remembers where he left all these nifty undocumented Perl command line utilities to reboot the Oracle servers and archive the log files.

As a manager, you should not always avoid hiring a Mr Fixit. He can be extremely useful when facing a tight deadline, working ungodly hours when the rest of the team would rather save their marriage. But you cannot let him get away with the feeling that therefore the rules don't apply to him. If every programmer in your team has to provide unit tests and documentation, so does Mr Fixit.

Secondly, be aware that anynew software technology or working method has a learning curve. Programmers are smart people with usually a fine capacity for self study, but we can also be very conservative in our adherence to doing things a certain way and be prejudiced towards anything newfangled. If you let Mr Fixit install a new source repository and a new build server without proper training and evangelism within the team, you have made him the de facto guru and everyone will turn to him for the most stupid questions. Now you really pissed him off!