Tricking Your Ears: Auditory Illusions

Psychoacoustic Tricks + How to Use Them In Your Music

Eldar Tagi · 08/30/23

It is quite fascinating that the combinations of abstract sounds we call music have such a powerful impact on our individual lives, and world cultures in general. Somewhere during our evolutionary process, our bodies became astonishingly receptive to sounds—starting an avalanche of emotional, cognitive, and physical responses. Yet more than that, each of us has unique ears with deeply individualized acoustic and perceptual architecture.

"Perceptual" is the key term in this article. Although we may all get together in the same room to listen to the same sound, each of us will experience it differently depending on a variety of factors. Furthermore, our sensory organs are limited, and are prone to error—which adds yet another layer of subjectivity to the experience. "Error," in this case, is not necessarily a bad thing, but rather an inherent potential to misinterpret what we hear or see, and in other cases a capacity to construct a completely imagined reality in response to (or sometimes independently of) external stimuli. We call these effects illusions when they are caused by external stimuli, and hallucinations when they happen on their own.

While both hallucinations and illusions have their own evolutionary causes (and value), the latter is much easier to study, as most people can predictively experience the effect, and the results are consistently repeatable. Throughout this article, we aim to investigate some of the more interesting auditory illusions discovered, and explore their potential for creative use in our artistic practices. As you will shortly see, the subject is tightly connected with electronic music, and particularly audio technology in general.

What is Psychoacoustics?

If acoustics is the study of how sound waves behave in liquids and gasses, then psychoacoustics tackles how they are interpreted by our minds. This fascinating field of study incorporates tools and methodology from a vast array of disciplines including psychology, physics, biology, electrical engineering, and computer science, collectively providing us with a better understanding of our internal relationship with sound, and music. The science grew out of the premise that quite a few more things are involved in the phenomenon of hearing besides the pure mechanics of traveling waves. Between the sound and its representation in our psyche, there are layers of active coding/decoding conducted by our cognitive apparatuses—which in effect are influenced by other sensory organs, environment, memory, age…well, evidently it gets quite complex.

One may wonder then, how can we use that information? On a plainly practical level, various discoveries in the field of psychoacoustics help us find real solutions to such problems as audio compression, sound reproduction, spatial sound, and even in things like image-guided surgery. From a different angle, psychoacoustic research opens up new creative possibilities for musical composition, and sound production. From iconic figures of electronic music history to modern Hollywood composers like Hans Zimmer—many have discovered applications for psychoacoustic effects in their work. Out of all the different discoveries within the field, auditory illusions hold a special place for artists, as they become powerful tools that can take the listener to a place they haven't been before.

The Psychology of Music

Another field of study that is of great importance to the topic of or article is music psychology—a cross between psychology and musicology that aims to understand musical experiences, and behaviors from both creative and listening sides. Undoubtedly, one of the first names associated with the field is Diana Deutsch—a professor of psychology at the University of California, San Diego whose contributions are so significant that much of her work has become foundational for the discipline as a whole. Auditory illusions, in particular, have remained central to her research since the very beginning.

Dr. Diana Deutsch, a foundational researcher in the field of music psychology and psychoacoustics Dr. Diana Deutsch, a foundational researcher in the field of music psychology and psychoacoustics

Deutsch's investigations and experiments led to a better understanding of such phenomena as perfect pitch, and her findings display a major link between speech, language, music, and handedness. While in the following sections of the article we will discuss some of the audio illusions discovered by Diana Deutsch, If you are interested to dive deeper into the research, we strongly recommend her books The Psychology of Music (1983), and most recently Musical Illusions and Phantom Words: How Music and Speech Unlock Mysteries of the Brain (2019).

So what are auditory illusions, and why do they happen? In essence, we consider something to be an auditory illusion when the sound that we perceive is different from what we are actually hearing. As Deutsch describes, there are two primary mechanisms that provoke illusions. The first one is related to our brain's remarkable ability to fill in the gaps when the information being presented is incomplete, and in doing so, arrive at peculiar conclusions. On the opposite end of the spectrum, some illusions are caused by biases formed by our knowledge, experience, and expectations. Also note, that for effective reproducibility of the effects, the listening environment is highly important—in some cases, headphones are required, in others well-spaced stereo speakers. We will certainly mention the desired configuration as we go through the examples.

While it is not feasible to list and explain all of the illusions discovered by Deutsch and others in this article, our primary goal is to identify a few that hold better potential for creative applications. Without a doubt, some of the ones mentioned will be familiar to some of you, but perhaps we can inspire you to take a second look at them, and see if they can spice up your artistic practice.

Binaural Audio

One of the first illusions that I personally heard about many years ago was the binaural audio or 3D audio effect—a unique stereo recording that when listened to on headphones creates a highly realistic sense of three-dimensional space. One of the most effective examples circulating at the time was the "virtual barber shop"—a binaural recording simulating a hairdresser giving you a haircut. While getting a haircut may sound like quite a mundane experience, what is important here is that binaural recording opened up a potential to create an auditory experience that is unbelievably (or, err…believably) personal for the listener.

The key to creating binaural audio is the recording process itself—which ideally requires a dummy head with two lavalier-type microphones placed inside the mockup ears (such as the Neumann KU 100 dummy head microphone, pictured above). Since we all have differently shaped heads, and ears, the same recording may work better on some people than others, but overall it is quite possible to achieve amazingly realistic results with such a technique. The applications of binaural audio are evidently greatly suited for virtual reality experiences, but they have also found wide use in now wide-spread ASMR-inducing A/V content.

Binaural Beats

Perhaps you have come across the phrase "binaural beats" somewhere in an article, a video, or a dedicated app that claimed to help with relaxation, meditation, sleep, etc. To understand what binaural beats are, we first need to understand "beating." Beating is an interference pattern that occurs when two sound waves whose frequencies are slightly different from each other are combined—imagine the pulsing effect when fine-tuning one guitar string while listening to the other string tuned to the same note.

Binaural beat using sine tones at 211 and 215Hz, respectively

The beat frequency is equal to the difference in frequencies between two tone generators. Thus if one source is 100Hz, and the other one is 102Hz, the beating frequency will be equal to 2Hz. The key here is that two sounds have to be combined acoustically for the beating to occur. However, the human brain is quite good at constructing things, and if we send two signals independently into each ear via headphones the beating effect still occurs, only now it happens inside our brains.

The effects of binaural beats have been studied quite a bit, and it is suggested that they can trigger specific neural synchronization patterns associated with a variety of particular states of mind including sleep, relaxation, creativity, focus, etc. With that said, the research on binaural beats is rather inconclusive, and there is no consensus on whether they truly are capable of retuning our minds. Nevertheless, this field of research is interesting, and we hope to hear more on the subject in the future.

And while we are at it, we would be remiss not to mention one of the relevant instruments that we carry here at Perfect Circuit—a binaural noise generator Quantum Ocean by Soma Laboratory, which directly aims to expand the consciousness of its user. [Editor's note / warning! One reviewer noted that, despite its quantum properties, Quantum Ocean caused notable damage to their aura, perhaps due to influence from a negative spectral presence from another plane—though this has not been confirmed. Use with caution!]

The McGurk Effect

Although we may be inclined to think that our sensory organs are working independently from each other, in truth, there is a lot of cross-influence going on. A piece of good evidence for such a link is the McGurk effect—an illusion where what we hear is changed depending on visual stimuli. In the experiment, we first see and hear a person saying some single syllable word like "Ga", and then we see and hear them say another word like "Fa". Only thing is that the audio in both segments is exactly the same, while the video of the person saying the word changes.

So what is happening here? Our brain showcases a clear prioritization of visual stimuli, and makes us hear a sound that better corresponds with the shape, and movement of the person's mouth despite the fact that the audio doesn't totally match. Here, once again, our biases and expectations alter the actual reality of things in order to make sense of the environment.

The McGurk effect is heavily utilized in foley—a post-production process of adding and altering sounds to accompany video. Thanks to this illusion, we can substitute the sound of fire with wrinkling paper, a heartbeat with rhythmically stretched fabric, or a walking robot with a few metallic objects hitting each other.

There is also another similar auditory illusion based on the dominance of sight over the sense of hearing—the ventriloquist effect. Here our brain is deceived in terms of the localization of the sound. If the sound we hear matches the visual image that we see we easily believe that that is where the sound is coming from, even if it is emitted from a totally different place. As the name of this effect suggests, this is precisely why we find the puppet shows believable.

Phantom Words

Discovered by Diana Deutsch, Phantom Words is an illusion where our brain creates meaning in places where there is none—specifically words. To create the illusion, we need to create a composite of two-syllable words for each of the two audio channels, which are then looped (well-spaced speakers are most effective for this illusion, and headphones don't work quite as well).

An audio example of the Phantom Words illusion

After some time of listening to the looping pattern, a completely new and unrelated word tends to emerge for many people. What is even more interesting is that words people hear directly relate to the language they speak, and it is common for individuals from different cultures to hear different words. Also, it was observed that people tend to report hearing words that are related to the condition of their mental state—e.g., stressed students reported words associated with stress and anxiety.

While it was not intentionally planned (at least to my knowledge), the effect of the phantom words can be experienced in early electronic compositions by Steve Reich, such as Come Out (1966), and It's Gonna Rain (1965).

Speech to Song

What is one of the key elements of any song? What makes a song memorable? Certainly, it is repetition. The act of repeating a pattern over and over strengthens it and increases its importance for the listener. But what is particularly strange is that by the nature of the way our minds work, we do tend to associate the act of repetition itself with musicality, even if the source of repeating pattern is not intentionally musical—such as our speech.

An audio example of the Speech to Song phenomenon used to musical effect

Diana Deutsch discovered that when a segment of speech is repeated, it is commonly experienced as sung, and not spoken. Deutsch performed several experiments to confirm the finding, and the results were consistently similar. In one of the experiments, Deutsch played the repeated phrase to eleven individuals with some degree of musical training independently, and asked them to repeat the phrase just as they heard it. Not only did all the subjects repeat the phrase in a singing melody, but when the individual voices were later layered on top of each other, the melodies they sang matched nearly perfectly into a cohesive choir. On the opposite side of this, when the phrase was played only once to a group of people, and they were asked to repeat it exactly as they heard it, the results were not nearly as musical.

Even before Diana Deutsch's discovery, several composers including Leoš Janáček and Percy Grainger noticed the inherent musicality of the conversational speech, and tried to extract musical patterns from common speech. Electronic music producers also utilize this effect, creating memorable hits out of repeated phrases—Disclosure's "When The Fire Starts To Burn" and AFX's "Children Talking" come to mind. In any case, this is a technique worth exploring, and it shows us that great musical gems can be hidden in even the most ordinary things we say. And as an extension to this, it seems aptly suggestable that any sounds repeated in a loop have the potential to be perceived as musical.

Shepard Tone, Shepard-Risset Glissando, & Risset Rhythm

Named after cognitive scientist Roger Shepard and French composer Jean-Claude Risset respectively, the Shepard Tone and Shepard-Risset glissando are perhaps the most well-known auditory illusions, and they have been extensively utilized in both classical music, and popular culture. The illusion is akin to the M.C. Escher's infinite stairs or the barber pole illusion, creating a sense of the tone or melody whose pitch is constantly rising or falling.

An audio example of Shepard tones at work

In its simplest form, a Shepard tone consists of several sound generators (at least three) layered on top of each other, separated by an octave in both directions. The frequencies and amplitudes of each sound generator are modulated in such a way that the middle tone slowly rises up an octave with the same amplitude level all the way through, while the higher octave tone gradually becomes quieter as the pitch rises, and the lower octave tone gradually becomes louder as it goes up. Since at any given point we hear at least two ascending (or descending) tones, we are tricked to believe that the tone rises infinitely, and this creates a very unsettling feeling.

One of the earliest examples of Shepard tone used in musical composition was James Tenney's "For Ann (Rising)," which consisted entirely of the effect alone—and in fact, in some circles, this phenomenon is referred to (almost out of principle) as the Tenney tone. Since then, the illusion has proven to be particularly useful for film composers, and once you know how it sounds, you can easily spot it in countless film trailers. Hans Zimmer's notoriously intense soundtrack for Dunkirk also makes significant use of the Shepard tone.

In the world of electronic musical instrument design, there are a few noteworthy Shepard tone generators. First developed in the early 2000s, the Buchla 260 Duophonic Pitch Class Generator is a dual digital oscillator; each oscillator's purpose is to create a single pitch class in all perceptible octaves at once. It can generate continuous or stepped Shepard tones/glissandi—and can even generate two at once. The more recent 260e features identical functionality, with a slightly re-designed panel and different display.

In the current day, Eurorack modular synthesizer users can explore Shepard-tone like behaviors with the Expert Sleepers General CV. Also, Make Noise's classic, now sadly discontinued, Telharmonic module had an Easter Egg Shepard tone mode, lovingly referred to as Spiratone (video above).

An audio example of Risset rhythms at work

A similar technique can also be applied to a rhythm—this is known as the Risset rhythm. Constructed in a similar way only using the speed parameter, the effect makes it seem as if the beat is constantly speeding up or slowing down. One of the classic examples of Risset rhythms can be heard in Autechre's "Fold4,Wrap5" from the album LP5.

Tricking Your Ears

While the list of auditory illusions mentioned in this article is far from complete, we hope that this information will inspire you to dive deeper into the work of Diana Deutsch, and others who have studied the strange world of interception between music and human perception. Also, the effects and techniques associated with them voiced above already provide a decent platform for creative exploration. Capable of being finely controlled, modern electronic musical instruments are particularly suited for such tasks, and if you happen to stumble upon something particularly interesting in your own experiments, make sure to let us know. Take care, and 'til next time!