Visualizing Music with JavaScript

Computer science has had an impact on every field, and music is no exception. Computer software now plays a crucial role in the ways we create music, edit music, share and distribute music, and even listen to music.

• Programs like Ableton allow artists to create and edit music with computer software.
• Programs like Soundcloud and Spotify allow musicians to distribute music and allow us fans to listen to music.
• For as long as digital music has been around, programs like iTunes and Windows Media Player and even the Atari Video Music have allowed us to visualize music as we play it.

Music visualization has been used to enhance the music listening experience in music videos, live performances, and even in education.

The Cooper Union in NYC is using music visualization to teach deaf children about sound. They have developed an interactive light studio in the American Sign Language and English Lower School in NYC. This consists of an interactive wall display that shows digital output created by sound and music. Children can trigger the playing of instruments with their movement, and they can watch the visual feedback from this music. (source: How Visuals Can Help Deaf Children 'Hear', 2014)

Nowadays, people create all kinds of music visualizations using code. Just search "#creativecoding music visualization" on Twitter to see some of the awesome creations people are making.

In this tutorial, we'll learn how to make our very own music visualizations with JavaScript. The possibilities are endless! Here's an example of what we'll be creating by the end of this tutorial:

First thing's first: let's write a program that plays some music.

To play an audio file in your program, we'll need 2 things:

1. A URL to the audio file. In this example we'll use "yesterday" by Jahzzar (CC BY-SA 4.0): https://codehs.com/uploads/8179d3793582c02c44438d0160d6f0a5
2. Create an Audio object from that URL and call the play() function on that object

To play your own audio file, Fork this program to your sandbox and use the More > Upload tab to upload your own audio file. Use the resulting URL to create your own Audio object on line 3.

Great! We have our music playing, but how can we visualize that music?

The big idea here is that the music playing through your speakers is just DATA.

All information stored in a computer is stored as digital data. Digital means stored as digits, aka numbers.

When a computer plays a music file, it's simply reading all the numbers in that file, and making noises through the speakers based on the numbers it reads.

So what do these numbers look like? We can picture a music file as a series of frames. A frame represents a single moment in the song, and each frame is a list of numbers that describe the notes being played at that moment.

The frame is just a list of numbers. The first numbers in the list represent how loud the low notes are being played. The middle numbers represent how loud the mid-range notes are being played. And the final numbers in the list represent how loud the high notes are being played.

Let's start off with a program that just prints out each frame in the audio file:

Let's break down what's happening here.

1. We create the Audio object, stored in a variable named song
2. We play the song
3. We add blank text to the screen
4. We update the text to show the current frame anytime song changes frames

The key to step 4 is the use of audioChangeMethod. audioChangeMethod is an event listener that listens for a special type of event - the given audio file changing frames.

If you've taken a JavaScript course on CodeHS before, you may be familiar with mouse event listeners:

mouseMoveMethod(myFunction) - calls myFunction anytime the mouse is moved
moveClickMethod(myFunction) - calls myFunction anytime the mouse is clicked
mouseDragMethod(myFunction) - calls myFunction anytime the mouse is dragged
mouseDownMethod(myFunction) - calls myFunction anytime the mouse button is pressed down
mouseUpMethod(myFunction) - calls myFunction anytime the mouse button is released

Here we are introducing a new event listener - specifically for audio events!

audioChangeMethod(song, myFunction) - calls myFunction anytime the given song changes to a new frame

Our job is to write a more interesting visualize function that draws shapes based on the given frame, rather than just printing Text based on the given frame.

Let's write a program that draws a vertical Rectangle for each number in the frame. The height of each Rectangle will be equal to each value in the frame, so quieter notes will result in shorter Rectangles, and louder notes will result in taller Rectangles:

Looking good!

But the visualization feels sort of "upside down"

Let's see if we can get the bars to rest at the bottom of the canvas, rather than the top.

Instead of using 0 for each bar's y-position, we want the y-position to be the bottom of the canvas MINUS the height of the bar:

Awesome!

From here we can make any customizations we'd like. The goal is to make shapes that are based in some way on the numbers in the given frame. Each shape's height can be based on the frame, or the width, or the color, or the position, or a combination!

Let's add one more customization:

The color of each bar will be based on the value in the frame. A large number will result in a more blue color, and a small number will result in a more red color.

Now it's your turn!

Try forking these examples and creating your own visualizations. Here are a few ideas to start with:

• Create circles for each frame instead of Rectangles
• Modify the position of each shape based on the values in the frame
• Create your own custom colors by modifying the amount of Red, Green, and Blue based on the values in the frame