June 2011

In his essay A Case Study on the Influence of Gestural Computing, Nicholas O’Brien discusses a car commercial that incorporates touch screen gestures characteristic of an iPhone or iPad user: The hand of an otherwise unseen consumer enters the video frame to change the movement and point of view of your standard slick sports utility vehicle montage: swoop -- the SUV drives through western landscapes! point -- it sits proudly in the carport!

I took the ad mostly as a cross-promotion for Apple and Land Rover. It shows you the twin magics of driving a gas guzzler with that new car smell as well as the abracadabra gestures of surfing with an iPhone. To interpret any larger meaning in the commercial one must first accept that Apple's gestures have achieved total market penetration and are the new norm for general purpose computer navigation.

I offer as a counter-argument this curmudgeonly essay about the poor design of tablets and touchscreens and how their main best uses are as toys. The author is talking about the iPad, not the iPhone but the complaints about covering your work area with your hands seem applicable to both:

A Google Image search turns up [a] quite typical picture of a [medieval] scribe practicing his art. You'll notice that the scribe's desk contains two levels, where the topmost level holds an exemplar document and the bottom holds the document that he's actually working on. The scribe in the picture could be a copyist who's making a copy of the exemplar, or he could be a writer who's using the top copy as a source or reference. Either way, his basic work setup is the same as my modern monitor plus keyboard setup, in that it's vertically split into two planes, with the top plane being used for display and the bottom plane being used for input.

The key here is that the scribe's hands aren't in the way of his display, and neither are mine when I work at my desktop or laptop. My hands rest on a keyboard, comfortably out of sight and out of mind.

With a tablet, in contrast, my rather large hands end up covering some portion of the display as I try to manipulate it. In general, it's less optimal to have an output area that also doubles as an input area. This is why the mouse and keyboard will be with us for decades hence -- because they let you keep your hands away from what you're trying to focus on.

My ultimate point here, and the reason I started with an image of a scribe, is that this separation of our productive workspace into display and input planes has been with us since the dawn of writing, and is likely to stay with us as long as being productive involves making text and pictures. It's a fundamental reality of knowledge work, and it means that multitouch tablets will continue to be novelty/entertainment items.

I wouldn't offer such a confident prediction about the longevity of the "mouse and keyboard"; Nicholas O'Brien is obviously past them and many children growing up with phones will never feel comfortable splitting the output and input area. [For that matter artists have worked in their output area since the dawn of painting.] And none of this is to challenge O'Brien's critique of the ad:

To a certain extent [it] also highlights an impatience that consumers have developed as a result of the instantaneous gratification that the web engenders through [search results] filtration...* The fictitious browser that pages through the landscape seems as though they are never quite satisfied with the context their Land Rover occupies, or even what it should exactly be used best for. That permanent unsettled fidgeting seems emblematic of -- or at least closely tied to -- how gestural computing has influenced our behaviors online.

*O'Brien is referencing a TED talk [YouTube] about how Google is now tailoring your searches to what it thinks you might want to find.

Whatever practical value the "unsettled fidgeting" of the phone user may have, we have Apple to thank for at least providing us with a visual metaphor for consumer discontent. Swoop, collapse, let's move on.