Chapter 7 - Sounds
We have finally learned enough about programming that we can now talk about the main topic of our course: Digital Media. Sound files play a major role in most of our lives; I'm sure most all of you have listened to music through either your computer or phone at some point today. That's not the only way that we interact with sound files on a daily basis though. Talking on the phone, using Skype, etc. all use some form of sound file to communicate data between the people in the conversation. Today, we will start to learn more about the inner workings of how sound is stored in our computers.
7.1 Continuity
Sound is transmitted via waves. While they are quite analogous to waves you would see in an ocean, the medium via which they are transmitted is quite different: Sounds travel via air, as opposed to water. This means that we cannot really "see" sound. It is vibrations and perturbations that exist in the air that our ear drums convert into something that our brain interprets as sounds. That doesn't mean we cannot visualize sounds, however.
This is an image of the waveform generated by someone speaking the
phrase "This is what sound looks like." A waveform is a fancy physics term
used to describe any visualization of any wave. We will use it throughout the
rest of this chapter.
One of the most important things to note about a sound wave is its continuous nature. Sound is not made of discrete tokens which can describe each individual component. This poses quite the problem, since our computers are inherently discrete beings. They NEED things to be tokenized in order for them to get a grasp on what is going on. As such, our computers must use a technique called sampling in order to convert the continuous wave into discrete chunks.
7.2 - Sampling
The theory of how sampling works is relatively straight forward: break the wave into discrete chunks so that the computer can compute an approximation of the wave later. To accomplish this, computers create a list of values that are representative of the shape of the wave. The more of these values we generate, the better our later approximation of the sound will be.
Consider the above simple waveform. There are an infinite number of values which make up the entirety of the sound. However, most of them are very, very close to one another. The amount of information lost by removing some of the values can be negligible. To that end, we compute a fixed set of evenly spaced out samples. These samples are values which we consider "representative" of the waveform.
For each one of these points we compute the magnitude of the wave: the vertical distance from the baseline to a point on the wave. Notice that these values can be both positive and negative. In fact, there must be positive and negative values. If there are not, then we will not get any sound. Using the samples we computed, we can reconstruct an approximation of the wave by simply connecting the dots.
If we wish to modify properties of the sound (pitch, volume, etc.), we simply need to modify the values that represent the approximation of the sound wave.
7.3 - Reading Sounds
Your Blockly programming environment is already setup for you to be able to
read, modify, and play back sound files. Just as we started with Turtle, any
sound capable program MUST begin with a 
block. This indicates to blockly that you will be utilizing the sound
libraries built into the browser for modification of the sound files. Every
one of your blockly programs that begins with the above block must also be
closed with a 
, which will then attempt to
play the sound that was read by your program.
You will notice that these blocks change the behavior of the blockly interfaces slightly. Instead of creating a canvas (which the Turtle blocks did), it now creates a file dialog box and a play button. These will only pop up once you have pressed the run code button! Notice that the code you write does not execute until after you have selected an audio file through the newly created file dialog box. This is different from how the turtle software behaves, but is necessary for how code is executed via the browser. Once a file has been selected, Blockly will run your code, and attempt to play your new sound.
The values that are read from the specified file can be retrieved using the
block. This block will provide your
program with the list of all of the samples of the sound file loaded. While
the adapter for this block could fit in the slot of many of our other
command blocks, it should only be used in a variable assignment block.
Execises
There are no specific assignments you need to complete for this reading. However, you should use the below area to create a sound program which simply allows you to load a sound and them play it back afterwards.