Abstraction, the Body & Motion Controls

by tommy.rousse

Wii Tennis players. creative commons, Kevin Chen

From a discussion of the importance of controllers in understanding gameplay by Graeme Kirkpatrick:

The controller and its resistances are those of the game and its objects, compared with the screen image they are commonly a miniaturized and condensed instantiation of the game program. “Play involves exploring and altering the field of tension. When Steven Poole (2000) writes that the physical skills learned in the playing of ancient, spear-throwing games are ‘‘exactly those skills exercised by modern target videogames’’ (p. 174), he is obviously wrong, in the sense that throwing a spear is not the same action as holding down the ‘‘B’’ button on a controller, but he also conveys an important truth about controllers. Something of the experience of throwing a javelin—its tensions in the body, its discipline, its conscious manipulation of weight and energies—gets condensed into the hand. This is, perhaps, best understood as the form of the action.

The same forms are present when we play using a Wii-mote. The first feeling triggered by the Wii-mote is one of vertigo, because the tensions of play are not contained within the hand any longer. Some (not all) of the actions we have to do to play the game no longer have the controller to refer to, so to speak, and instead must occupy the empty space of the room we are in. We use more (a greater part) of our bodies to trace out the formal patterns that have to be enacted if we are to play the game but whereas traditional controllers involve a kind of condensation of the formal properties of the game in miniature, the Wii-mote generally makes the forms bigger, but more abstract.

This change in the phenomenology of controller use corresponds to the shift Don Ihde detects in the move from analogue to digital clocks. The conventional clock with hands communicates with us by establishing a kind of tension between the clock face, which represents blank, empty time, and the hands whose current positions signify only with the face as a background. The position of the hands is our position relative to time as a whole. When we move to digital clocks, which present us with a numeric representation of time, time loses some of its concreteness and becomes more abstract: digital clocks only tell us the time it is now, removing the current instant from any context. Ihde (1990, p. 68) links this to the historical trajectory of technology itself, which moves us towards increased detachment and decontextualization of information (see Feenberg, 2002). Driving a car with a digital speed display produces a similarly frictionless sense of speed, because the tension between the needle and the background, which here represents stillness, is no longer present and speed becomes more abstract.

However, this doesn’t change their fundamental character as forms because spinning one’s arm when holding a Wii-mote (to ‘‘serve’’ in tennis perhaps) makes the same kind of pattern we would make with our thumbs using a lever on a traditional controller, only now it is drawn out in empty space rather than against the resistance of a physical object.

—Graeme Kirkpatrick
Aesthetic Form in the Computer Game,” Games & Culture, 134-135.1

Kirkpatrick has many interesting things to say about videogames, but the passage above is problematic. In fact, it seems to be inconsistent internally, and with the rest of the essay as a whole, particularly the preceding section on hands and touch. While he makes a valuable contribution by discussing the importance of controllers to gameplay, he loses focus when it comes to motion controls.

Prima facie, motion controls in the Wii Sports paradigm are not more abstract than traditional button-based controls. 2 As experienced by most players, and especially those new to the experience, motion control is largely mimetic of the activity being simulated. Kirkpatrick’s analysis is caught up in some fundamental errors in regard to materiality and play as an enacted experience.  Above, he contrasts the “empty space” of moving the Wii controller to the “resistance” of button-based play.  Furthermore, he claims that “spinning one’s arm when holding a Wii-mote (to ‘‘serve’’ in tennis perhaps) makes the same kind of pattern we would make with our thumbs using a lever on a traditional controller.”

Unlike our virtual avatars, however, player movement always translates into resistance: the controller and our arms have weight, and we must expend energy to overcome gravity and move. While visibly empty, the space we move the Wii controller with is actually full of invisible resistance.  Extended sessions of Wii Sports quickly demonstrate this resistance; Wii Tennis elbow is not unheard of.

Kirkpatrick assumes that if this motion were translated into a traditional controller, the same pattern of movement would be preserved in the thumb. I’ve played quite a few tennis games over the years, but I’ve never actually encountered one where that is the case. Simply put, the thumb is not a very good replacement for the movement of an entire arm. In the traditional “cross plus two” layout immortalized by the Nintendo Entertainment System, that would leave no way of controlling the movements of the player. Instead of repeating that pattern of motion, the process is invariably abstracted to a much simpler combination of buttons.  Watch a bit of this recent adaptation of tennis to get a sense of what I mean:

In Jan Willem Nijman’s Tennnes 3, striking the ball is controlled by pressing a button and pointing the D-pad in the direction of its intended flight.  The D-pad is also used to control movement. No circular thumb-sweep for the “stroke” or other mimetic input is necessary; however, the stroke is memetically represented by a small box and a thwock noise that invokes the sound of a ball hitting strings.

As I see it, the move from button to motion-based controls is the inverse of Ihde’s shift from analog to digital displays.  Kirkpatrick doesn’t seem to heed the Focillon he quotes a few pages before:

We must never think of forms, in their different states, as simply suspended in some remote, abstract zone, above the earth and above man.  They mingle with life, whence they come; they translate into space certain movements of the mind. (p.60)

—quoted in ibid.

  1. I added paragraph breaks  to increase legibility in this format.
  2. However, this discussion is complicated by the fact that these mimetic movements are not strictly necessary for interacting with the game. Cf. Bart Simon’s discussion of gestural excess. Here, I will assume non-expert motion-controlled play.
  3. Nijman is 1/2 of Vlambeer. Tennnes was made at Nordic Game Jam 2012, and has been exhibited beside LAZA KNITEZ!! at Distortion Street Festival.