Crafting musical instruments with the Web Audio API

It’s really amazing how much functionality is built into the browser platforms these days. WebGL, Web Push Notifications, WebRTC, View Transitions, and everything else enabling Progressive Web Apps comes to mind. The Lynx browser could never.

But the topic of this post is the Web Audio API which provides primitives for building digital audio applications entirely within the browser! Some folks have gone much, much deeper on the subject such as fellow Brooklynite Yotam Mann (see Tone.js) and Tero Parviainen. The browser APIs tend to be intentionally low-level and verbose so that folks like us can build useful higher-level abstractions over them for various use cases.

For example, this React hook I wrote below is quite nice for getting a feel for the API in a modern JS framework setting. The interplay between the native code and the JS code takes some getting used to as there is some amount of state being maintained in both places. The JS layer is responsible for the configuration of the audio, but the actual adjustments to the audio happen on the native threads. This means that adjustments to the configuration will not cause a re-render of the component using the hook! It’s sort of like working with uncontrolled form inputs.

Each usage of useTone() will create an audio node which can be turned on and off via .play() and .stop(). The oscillator doesn’t actually disconnect or unmount when stop is called. Instead, the volume (“gain”) is just set to 0 while it continues to run in the background, but this is an implementation detail that is abstracted away from the caller of the hook.

interface Tone {
  freq: number;
  volume: number;

  // play basically just turns up the volume
  play: () => void;
  // stop sets the volume to zero
  stop: () => void;

  // advanced users can dig into the audio primitives
  oscillatorNode: OscillatorNode;
  gainNode: GainNode;
}

export interface UseToneOptions {
  // frequency in hertz between 1 and 24000
  freq: number;
  // gain value between 0 and 1
  volume: number;
}

const defaultOptions: UseToneOptions = {
  freq: 440,
  volume: 1,
};

function useTone(options: Partial<UseToneOptions>): Tone {
  const { ctx } = useContext(AppAudioContext);
  const osc = useRef<OscillatorNode>(ctx.createOscillator()).current;
  const gain = useRef<GainNode>(ctx.createGain()).current;

  // derive final configuration
  const opts: UseToneOptions = {
    ...defaultOptions,
    ...options,
  };

  useEffect(() => {
    osc.connect(gain);
    gain.connect(ctx.destination);

    return () => {
      try {
        osc.disconnect(ctx.destination);
        gain.disconnect(ctx.destination);
      } catch (e) {}
    };
  }, []);

  useEffect(() => {
    osc.frequency.setValueAtTime(opts.freq, ctx.currentTime);
  }, [opts.freq]);

  useEffect(() => {
    gain.gain.setValueAtTime(opts.volume, ctx.currentTime);
  }, [opts.volume]);

  const play = () => {
    try {
      osc.start();
    } catch (e) {}
    gain.gain.setValueAtTime(opts.volume, ctx.currentTime);
  };

  const stop = () => {
    gain.gain.setValueAtTime(0, ctx.currentTime);
  };

  return {
    freq: osc.frequency.value,
    volume: gain.gain.value,
    play,
    stop,

    oscillatorNode: osc,
    gainNode: gain,
  };
}

export { useTone };

Next are a few examples of how I used this hook to build a small ensemble of instruments! Unfortunately, they do not work well on mobile (yet).

First, you need to click this big button to allow me to play audio because apparently that is a thing.

I am ready to jam