One of the most exciting ways to enjoy music is to listen… with your eyes. It’s one of the main reasons you go to a rock concert or orchestra performance instead of just buying the recording. It’s why some pianists become famous and others go unnoticed.

As I become more deaf, I’m growing more aware of how much I use my eyes to hear. The notes from the top of the piano all the way down to middle C escape me now, and the notes below are often a muddy, scratchy rumble. But when I see my fingers press each key, the sound starts to clear and the blanks begin to fill in.

You are already listening with your eyes without even realizing it. Let’s say you’re watching a marching band in a movie and you see a trumpet player: he takes a deep breath, his lips puff, his face turns red, his fingers press down the valves, and… BLAST!

Watching the trumpeter inhale, you’re seeing “anticipatory movement.” It’s the same with watching a singer breathe, a conductor raise their arms, a violinist lift the bow, or a pianist prepare to strike the keys. Your eyes see this movement and they tell your brain, “Get ready for sound!”

Even seeing the trumpet itself, your brain starts to pull up the “trumpet sound” from its menu of memories. If you’ve never heard a trumpet before, your brain can still guess by the size of the instrument that it will be higher in pitch than a tuba, and because it is metal the tone will be different from a violin, and that because you blow on it instead of pluck it, the sound may be more round.

But if you listened to an audio recording of this same moment, you might not be able to tell if it was a trumpet, a sax, or a trombone. Even skilled musicians may have trouble distinguishing between a trumpet and similar sounding instruments, like the cornet. Films often use your eyes to trick you into thinking you’re hearing an orchestra instead of a synthesizer—the power of your eyes to “hear” often wins out over your ears. You see a trumpet, your brain thinks “trumpet.”

After telling your brain to get ready for sound and what sound it will be, your eyes pick up even more once the music starts. As the trumpeter plays and the band marches, the steps and movements give you clues to tempo, rhythm, and if you have musical training—you might even guess the notes by looking at their fingers! Try watching a video of musicians play with the sound off and see how much you can hear with your eyes.

* * * *

On a recent vacation, I watched Ilya Yakushev play Prokofiev’s Piano Concerto No. 3 with the Des Moines Symphony while following along with the score on my iPad (side note: getting approval to bring an iPad into a concert hall can be difficult; be prepared). While it’s impossible to instantly grasp every note and the complex harmonics (thanks, Prokofiev!), it’s quite easy to get a lot, especially with passionate performers. The quick strike of pizzicato, the loving roll of the hand in sensuous phrases, digging deep into a long trill—all of these are treats for your eyes. And Ilya did not disappoint.

But there are even more ways to listen with your eyes when you’re at the piano. Looking at the keyboard, you can see the rise and fall of the melody as your hand moves up and down the keys. You can sense chords by looking at the hand shapes. You can see volume in a soft caress or a hard strike.

For the ambitious musician, you can even learn to hear with your eyes when looking at music scores. The vertical staff layout makes it easy to see high notes at the top and low notes at the bottom. And with the five-line staff design based on the most common diatonic scale, it’s also easy to see the difference between major and minor chords—especially in simple keys like C, G, and F—the difference between open chords and clusters, and a variety of inverted and other chord types.

However, processing and “hearing” all of a score in real time solely with your eyes can be overwhelming with complex pieces. It’s best for simple songs and for focusing on one or two music elements at a time, like the melody and bass line.

For truly hearing all of a piece with your eyes alone, we need advancements in computer software and a defined visual music language. This is something I’ve been working on in the past few years, having explored live frequency analysis software in art exhibits, MIDI visualizations in video installations, and about every visual music program out there.

Try listening more with your eyes and notice how much they add to your experience. Can you tell when sound is coming? What sound it will be? How fast, how long, how loud? What relationship it has to the other sounds in each moment?

The more I listen with my eyes, the more I grasp. And I begin to see that music is not just about the rising and falling notes in the moment, it’s about the patterns created over time—what patterns came before, what patterns came after, and how those patterns are interwoven or conflicting. So keep listening with your ears, your body, and your eyes. But to fully understand music, we’ll need to add one more tool to this listening series: listening… with your memory.