Speaking the Language: Thoughts on Controller Design

Many Mindsets for Interacting with Electronic Musical Instruments

Mike Metlay · 09/21/23

Editor's note: this article is a companion to Mike Metlay's recent article, Alternative MIDI Controllers: New Ways to Touch Music. Where that article acts as a flyover historical view of the last 50-plus years of interaction design in electronic musical instruments, this article strives to go deeper into the philosophical aspects of human-machine interaction in electronic music. How does it feel to play an alternative controller, and why would you want to? What mindset can you adopt in order to develop your own unique playing style? What makes a particular means of interaction design succeed or fail? Read on to find out.

The Mindsets of Interacting with Electronic Musical Instruments

Over the past 50-plus years, alternate controllers have been divided (roughly) into two groups: those that are designed to imitate the playing technique of traditional instruments that don’t use keyboards, such as guitars, winds, drums, etc., and those that are designed to be entirely new, with no traditional counterparts.

These two kinds of alternative control appeal to two different creative mindsets:

  • If you’re a skilled player on an instrument that doesn’t have a keyboard, an interface that plays like your weapon of choice will make it much more natural for you to play a synthesizer expressively.
  • If you’re looking to knock down all the barriers set up by a traditional mindset, you can work with an interface that has no history or rules for “proper technique,” and make up your own playing style from scratch.

Both of these approaches have their strengths and can open a lot of creative doors...but they share two sides of a common problem: the language of performance.

What Makes an Instrument Successful?

Successful instruments stand the test of time by developing a set of performance techniques that are commonly accepted and easily teachable, while leaving room at the edges for new techniques to develop and gradually become accepted. The process can take years, decades, or even centuries.

A violin student first learns how to hold a note on a violin’s neck and move the bow over the string to make a sound. Then, over time, they learn hand positions and how to master rapid melodic lines, different bow techniques to produce different tonal textures, and eventually get into more unusual things like plucking the strings rather than bowing them.

Eventually, they might get into using their violin with effects pedals to replace a lead guitar (thanks, Nash The Slash!), or triggering synthesizers with it (thanks, Jean-Luc Ponty!), or replacing the bow with a length of pre-recorded magnetic tape and the bridge with a tape playback head (thanks, Laurie Anderson!)—but they all start with the same language of violin performance, and build on that.

The same holds true for the flute, the sax, the congas, the kalimba, the oud, and every other successful instrument on Earth. Each has its own language: techniques that can be passed on to students (vocabulary), and standards for competency that separate an effective player from an inexpert one (grammar and style).

When Language is Broken by Technology

When an alternative controller attempts to replace a traditional instrument, the player may discover that in their language of performance, one or more pages have been torn out of the dictionary: their vocabulary—their arsenal of techniques—is stunted by the limitations of the technology.

Playing a note on a synthesizer using a keyboard is comparatively straightforward: play a key, send a gate or trigger and a pitch control voltage, hear a note at the right pitch, maybe add velocity or pressure for more expression. What about a guitar, though? Fret and pluck a string, hear a note—unless you’re trying to play a synthesizer, in which case the machine has to:

  • 1. detect the pluck and open a gate;
  • 2. generate a control voltage for the pitch, determined by either
    • a. analyzing the frequency of the vibrating string, or
    • b. sensing where the string is touching the fret;
  • 3. listen to the amplitude of the string vibration and close the gate when it drops below a certain level.
The ARP Avatar guitar synthesizer—widely accepted as a commercial failure, if not also as the design that bankrupted ARP altogether. Images via Perfect Circuit's archives.

[Above: the ARP Avatar guitar synthesizer—widely accepted as a commercial failure, if not also as the design that bankrupted ARP altogether. Images via Perfect Circuit's archives.]

That’s what you need to play a single note with only basic dynamics, being careful not to play any other notes—that’s how the ARP Avatar guitar synth works. Playing chords requires a pickup and pitch detection for each string, and six separate control paths—that’s how the Roland GR-500 and its various offshoots work. If you want to use any other technique beyond simply playing notes—like string bends or vibrato arm techniques, hammer-ons or pull-offs—things get even harder for the technology to handle, if it can handle them at all.

The player may not want to have to change their technique to suit the machine, but the machine can have real trouble understanding what the player wants. If it can’t deliver on the language, the player feels tongue-tied.

Every instrument has its own language—how hard you bow a cello, using lip pressure to control pitch on a sax, where and how hard you hit a drum head. Primitive synth control technology put hard limits on these techniques, but over time, advancements in sensors, controller design, and materials have been patiently gluing pages back into players’ dictionaries.

When Language Dies Even As It's Being Born

A completely unique new playing interface brings with it an all-encompassing new question: now that you’ve got it, how do you play it? The problem isn’t that the existing language is being stifled—it’s that there is no language at all. The player is writing the dictionary as they go, and is often unaware of all the other players out there, each writing their own dictionary with their own vocabulary that has nothing to do with anyone else’s.

[Above: a video demonstration of the Eigenharp.]

A colleague of mine who plays the Eigenharp once described to me what that player community’s occasional conventions are like: a large crowd of people comparing notes on their instruments and how they’re set up—more often than not finding someone else’s Eigenharp completely unplayable, even though it’s exactly the same instrument!

This leads to significant problems in terms of an instrument’s long-term survival. If nobody can agree on how to play an instrument, what are its chances of being more than a fad?

A classic example is the Trautonium, invented by Friedrich Trautwein in 1930 and developed and played for many years by Oskar Sala. He used the Trautonium to perform music by famed classical composers like Paul Hindemith, as well as on film soundtracks like Alfred Hitchcock’s The Birds. The problem is that Sala never taught anyone else to play it as well as he could. He died in 2002, and there are hardly any new Trautonium performers to take his place—so in a very short time, if no one learns to play it, the Trautonium will become a museum piece, its language lost forever.

Of course, this might not matter to you. One of the joys of such an instrument is that it’s yours and no one else’s—if it helps you realize your musical vision, that may well be enough! (Just don’t expect anyone to be able to perform your music accurately in 50 years, assuming that matters.)

When Language Grows from Healthy Roots

Alternative controllers have only been around for 100 years or so, but amidst all the various trials and failures, there are some important success stories. In those cases, the key to longevity seems to be a marrying of the two worlds. Controllers are successful if they give new dimension to existing technique...if the community has the courage to say, "This may be a new instrument, but there are rules you have to start with, just as with any other instrument."

[Above: video of Roger Linn demonstrating/performing with his masterpiece controller design, the LinnStrument.]

One great example is the LinnStrument. Roger Linn is frequently approached by people who want to redesign how the LinnStrument’s grid of notes works—usually asking for a “no wrong notes” grid layout like you can set up on the Novation Launchpad or Ableton Push.

Roger patiently explains to them that that’s never going to happen, because that’s not how a LinnStrument is meant to be played. It’s a chromatic instrument, and you can’t yank notes out of it any more than you can arbitrarily remove keys from a piano. If you want a key-specific grid, that’s fine, but you can get that from a Launchpad or a Push...they’re designed to be able to do that. As a result of this seemingly arbitrary limitation, LinnStrument players know what the instrument will and will not do from the very beginning. They start with a firm footing on basic technique, and develop their capabilities from there, leading to a growing and healthy community.

Another excellent example is the Ondes Martenot. It’s an unusual instrument, with its pressure “lozenge” to articulate envelopes and the au ruban technique of gliding between pitches by moving a ribbon to the left and right with a ring on your index finger. But for all that, it’s based on a piano keyboard, and it’s been mastered by many different players over the nearly 100 years it’s been in use.

[Above: Official product demonstration of the Expressive E Osmose keyboard/synthesizer/alternative controller in use.]

One of the most compelling modern examples would be the Expressive E Osmose. It’s firmly based in traditional keyboard technique, but its keys themselves allow for a variety of individually nuanced motions. It starts with the language every keyboardist knows, then encourages them to start scribbling notes in the margins of the dictionary, and eventually add pages of their own.

So, the next time you sit down with an alternative controller, take a moment to think seriously about its language. How does it want you to play it? What can it do? What can’t it do? Spend some time wrapping your head around that, and you’ll find that making music with it will be more productive, more quickly rewarding, and more satisfying in the end.

Check out Perfect Circuit's full selection of MIDI controllers and alternative controllers here!