This is the last of three blog posts about building a synthesizer using Next.js's App router, the Web Audio API, and TypeScript. In the first post, I built a very basic synthesizer with a bank of three oscillators. In the second post, I made it a much more sophisticated instrument by adding a filter, envelopes, an LFO, and a compressor. This post continues to build on the same instrument, so I recommend reading/following along with those posts first because we're jumping into the middle of things here.

I won't really be touching much related to audio in this post. Instead, I'll be adding an Oscilloscope, which will let us visualize the shape of the soundwave that our synthesizer is producing. We'll use an HTML Canvas to illustrate the wave shape in real time, hooking into the existing Web Audio API context to add an AnalyserNode. On the React side of things, we'll hold the analyzer in state and use a ref to hold the canvas element. The bulk of the work will be a useEffect that draws a grid to the canvas element, normalizes the data from the AnalyserNode, converts that data into X and Y coordinates, and puts those coordinates on the canvas while a note is playing.

What we're building

Just like in each of the last two posts, I'll start off by showing what exactly we're going to build in this post:

next.js synth mk3

Wave Type

Octave

Detune 0

Volume 0.35

Wave Type

Octave

Detune 0

Volume 0.35

Wave Type

Octave

Detune 0

Volume 0.35

Master Volume 1

Wave Type

Rate 5

Depth 4

Attack 0.1

Decay 0.24

Sustain 0.44

Release 0.56

Filter Type

Frequency 350

Detune 0

Q - Lowpass, Highpass, Notch1

Gain - Lowshelf, Highshelf0

Env Amount 5000

Attack 0.1

Decay 0.2

Sustain 0.2

Release 0.5

Loading Synth V3...

Note: I recognize that this thing is a little squished within the blog post layout. It still works, and I've done my best to accommodate everything. A better full-screen version can be seen over at joeyreyes.dev/synth.

All of the audio controls work the same, I've just added the Oscilloscope into the mix and rearranged things to accommodate this new component in the layout.

Import Oscilloscope

Let's dive right in at the top of the instrument. In parent synth component, let's do the following:

Import a new Oscilloscope component.
Import an analyzer from useSynthEngine (alongside playNote, stopNote, updateMasterVolume, and updateFilter).
Drop the Oscilloscope component into the UI with analyzer as its sole argument

// index.tsx
"use client";
import React, { useRef, useCallback, useState } from "react";
import Oscillator from "./components/Oscillator";
import VolumeLFO from "./components/VolumeLFO";
import Filter from "./components/Filter";
import Envelope from "./components/Envelope";
import Oscilloscope from "./components/Oscilloscope";
import Keyboard from "./components/Keyboard";
// ...other imports, synthFont, OscillatorBank interface...

export default function Synth() {
  // ...state variables and ref...

  const {
    playNote: enginePlayNote,
    stopNote: engineStopNote,
    updateMasterVolume,
    updateFilter,
    analyzer, // import analyzer from useSynthEngine
  } = useSynthEngine(settingsRef);

  // ...playNote, stopNote, change handler functions...
  return (
    <div className={`synth-container ${synthFont.variable}`}>
      <div className="synth">
        <div className="title">
          <h1 className="synth-title">
            next.js synth mk3 - synth page
          </h1>
        </div>
        <div className="synth-padding">
          <div className="synth-modules">
            {/* ...Oscillators, VolumeLFO, Envelope, and Filter... */}
            <Envelope
              envelopeSettings={filterSettings.filterEnvelope}
              handleEnvelopeSettingsChange={(nextFilterEnvelope) => {
                handleFilterSettingsChange({
                  ...filterSettings,
                  filterEnvelope: nextFilterEnvelope,
                });
              }}
              variant="filter"
            />
            {/* add Oscilloscope */}
            <Oscilloscope analyzer={analyzer} />
            <Keyboard onKeyDown={playNote} onKeyUp={stopNote} />
          </div>
        </div>
      </div>
    </div>
  );
}

Add analyzer to useSynthEngine

The next thing we need to do is actually create the AnalyserNode from our AudioContext. As always, this lives over in our useSynthEngine custom hook.

The steps here are:

Import useState.
Create the AnalyserNode ("analyzer") and give it an fftSize.
Connect the compressor to the analyzer, then connect analyzer to the audio destination (computer speakers).
- Previously, compressor connected directly to the audio destination.
Set analyzer to state as an analyzerInstance and return this alongside playNote, stopNote, updateMasterVolume, and updateFilter.

// hooks/useSynthEngine.ts
"use client";
import React, { useRef, useEffect, useState } from "react"; // import useState
// ...other imports...

export function useSynthEngine(settingsRef: React.RefObject<SynthSettings>) {
  // ...audioCtx, masterGain, activeNots refs...

  const [analyzerInstance, setAnalyzerInstance] = useState<AnalyserNode | null>(
    null,
  );

  // ...octaveToFrequency function...

  useEffect(() => {
    if (!audioCtx.current) {
      audioCtx.current = new (
        window.AudioContext || (window as any).webkitAudioContext
      )();

      masterGain.current = audioCtx.current.createGain();

      const compressor = audioCtx.current.createDynamicsCompressor();
      compressor.threshold.setValueAtTime(-12, audioCtx.current.currentTime);
      compressor.knee.setValueAtTime(30, audioCtx.current.currentTime);
      compressor.ratio.setValueAtTime(12, audioCtx.current.currentTime);
      compressor.attack.setValueAtTime(0.003, audioCtx.current.currentTime);
      compressor.release.setValueAtTime(0.25, audioCtx.current.currentTime);

      // create Analyzer:
      const analyzer = audioCtx.current.createAnalyser();
      analyzer.fftSize = 2048;

      // connect compressor to analyzer
      masterGain.current.connect(compressor);
      compressor.connect(analyzer);
      // connect analyzer to context destination
      analyzer.connect(audioCtx.current.destination);
      // add analyzer to state
      setAnalyzerInstance(analyzer);
    }
    return () => {
      audioCtx.current?.close();
      audioCtx.current = null;
    };
  }, []);

  // ...playNote, stopNote, updateFilter functions...
  return {
    playNote,
    stopNote,
    updateMasterVolume,
    updateFilter,
    analyzer: analyzerInstance, // export the analyzer instance
  };
}

fftSize (or Fast Fourier Transform Size) is the size of audio data in time that we're going to analyze. 2048 is a pretty standard size here.

As an aside, I recognize I've been blending spelling here - analyzer vs. AnalyserNode. I am American, so I'm going with the American spelling ("analyzer") wherever possible, but the node is spelled AnalyserNode in the API, so it's unfortunate. Oh well.

Create Oscilloscope component

Now for the meat and potatoes of our Oscilloscope, the Oscilloscope component:

// components/Oscilloscope/index.tsx
"use client";
import React, { useRef, useEffect } from "react";

export default function Oscilloscope({
  analyzer,
}: {
  analyzer: AnalyserNode | null;
}) {
  const canvasRef = useRef<HTMLCanvasElement>(null);
  const requestRef = useRef<number | null>(null);
  const dimensionsRef = useRef({ width: 0, height: 0 });

  useEffect(() => {
    if (!analyzer || !canvasRef.current) {
      return;
    }

    const canvas = canvasRef.current;
    const ctx = canvas.getContext("2d");
    if (!ctx) {
      return;
    }

    function handleResize() {
      if (canvas && ctx) {
        const dpr = window.devicePixelRatio || 1;
        const rect = canvas.getBoundingClientRect();

        canvas.width = rect.width * dpr;
        canvas.height = rect.height * dpr;

        dimensionsRef.current = {
          width: rect.width,
          height: rect.height,
        };

        ctx.scale(dpr, dpr);
      }
    }

    window.addEventListener("resize", handleResize);
    handleResize();

    const bufferLength = analyzer.frequencyBinCount;
    const dataArray = new Uint8Array(bufferLength);

    function draw() {
      const currentCtx = ctx;
      const currentAnalyzer = analyzer;

      if (!currentCtx || !currentAnalyzer) {
        return;
      }

      requestRef.current = requestAnimationFrame(draw);
      currentAnalyzer.getByteTimeDomainData(dataArray);

      const { width, height } = dimensionsRef.current;
      if (width === 0 || height === 0) {
        return;
      }

      currentCtx.fillStyle = "#f7f3a2";
      currentCtx.fillRect(0, 0, width, height);

      currentCtx.strokeStyle = "#010101";
      currentCtx.lineWidth = 0.5;

      const numVerticalLines = 10;
      currentCtx.beginPath();

      for (let i = 0; i <= numVerticalLines; i += 1) {
        const x = (width / numVerticalLines) * i;
        currentCtx.moveTo(x, 0);
        currentCtx.lineTo(x, height);
      }
      currentCtx.stroke();

      const numHorizontalLines = 6;
      currentCtx.beginPath();

      for (let i = 0; i <= numHorizontalLines; i += 1) {
        const y = (height / numHorizontalLines) * i;
        currentCtx.moveTo(0, y);
        currentCtx.lineTo(width, y);
      }
      currentCtx.stroke();

      currentCtx.lineWidth = 1.5;
      currentCtx.strokeStyle = "#d9a443";
      currentCtx.beginPath();

      let triggerOffset = 0;
      const WAVEFORM_MIDPOINT = 128;
      const searchRange = bufferLength * 0.75;

      for (let i = 0; i < searchRange; i += 1) {
        const currentSample = dataArray[i];
        const nextSample = dataArray[i + 1];

        if (currentSample !== undefined && nextSample !== undefined) {
          if (
            currentSample < WAVEFORM_MIDPOINT &&
            nextSample >= WAVEFORM_MIDPOINT
          ) {
            triggerOffset = i;
            break;
          }
        }
      }

      const samplesToDraw = Math.min(
        bufferLength / 2,
        bufferLength - triggerOffset,
      );
      const sliceWidth = width / samplesToDraw;
      let x = 0;

      for (let i = triggerOffset; i < triggerOffset + samplesToDraw; i += 1) {
        const val = dataArray[i];

        if (val !== undefined) {
          const v = val / 128.0;
          const y = (v * height) / 2;

          if (i === triggerOffset) {
            currentCtx.moveTo(x, y);
          } else {
            currentCtx.lineTo(x, y);
          }
        }
        x += sliceWidth;
      }

      currentCtx.stroke();
    }

    draw();

    return () => {
      window.removeEventListener("resize", handleResize);
      if (requestRef.current !== null) {
        cancelAnimationFrame(requestRef.current);
      }
    };
  }, [analyzer]);

  return (
    <div className="module oscilloscope">
      <div className="header">
        <h2>Oscilloscope</h2>
      </div>
      <div className="canvas-container">
        <canvas ref={canvasRef} className="canvas" />
      </div>
    </div>
  );
}

Obviously there's quite a bit going on in the useEffect here, but it breaks down into a few key points:

The handleResize function handles DPR scaling to calculate devicePixelRatio as well as the proper height and width of the canvas. This does its best to keep things looking crisp.
- I recognize there are some weird canvas sizing issues when the viewport is resized. I've tried debugging this, but it's a bit beyond me. Refreshing the page seems to reliably do the trick
The draw function actually draws our canvas. It handles horizontal and vertical grid lines and the background color. More crucially, it takes the data from analyzer, fills a dataArray with values between 0 and 255 based on that analyzer data, creating the shape of the currently-playing soundwave.
We use a rising edge trigger to keep the waveform from jumping around the grid, instead keeping it locked to the center line of the Oscilloscope.
Loop over the audio data, converting them into X and Y coordinates, spreading them evenly across the width of the canvas.

This is obviously a very, very high level overview of what's happening here. I'll qualify all of the above with a callout that I am not an expert in working with canvas, and I worked more closely with AI on how the draw function works than anything else in this entire synthesizer.

Update styles

The last thing to do is to update the SCSS so that the Oscilloscope/canvas falls nicely in the layout:

/* synth.scss */
@use "../../../styles/breakpoints";

.synth-container {
  flex-grow: 1;
  display: flex;
  justify-content: center;
  align-items: center;
  + p {
    margin-top: 20px;
  }
}

.synth {
  h1,
  h2,
  h3,
  h4,
  h5,
  h6,
  label,
  p {
    font-family: var(--font-synth);
    font-size: 12px;
    text-transform: lowercase;
  }
  --accent-color: #d9a443;
  --input-background-color: #f7f3a2;
  --outline-color: #000000;
  --disabled-color: #525252;
  --synth-background-color: #f4f4d5;

  background-color: var(--synth-background-color);
  padding-bottom: 20px;

  .title {
    background-color: var(--accent-color);
    padding: 16px 20px;
    border-radius: 29px 29px 0 0;
    .synth-title {
      font-size: 16px;
    }
  }
  // layout
  .synth-padding {
    padding: 20px;
  }
  .synth-modules {
    display: grid;
    gap: 12px;
    grid-template-columns: repeat(2, minmax(0, 1fr));
    @include breakpoints.mobile {
      grid-template-areas:
        "oscillator1 oscillator2"
        "oscillator3 volumelfo"
        "envelopegain filter"
        "envelopefilter oscilloscope"
        "keyboard keyboard";
    }
    @include breakpoints.tablet {
      grid-template-columns: repeat(4, minmax(0, 1fr));
      grid-template-areas:
        "oscillator1 oscillator2 oscillator3 volumelfo"
        "envelopegain filter envelopefilter oscilloscope"
        "keyboard keyboard keyboard keyboard";
    }
    @include breakpoints.desktop {
      grid-template-columns: repeat(5, minmax(0, 1fr));
      grid-template-areas:
        "oscillator1 oscillator2 oscillator3 volumelfo envelopegain"
        "filter envelopefilter oscilloscope oscilloscope oscilloscope"
        "keyboard keyboard keyboard keyboard keyboard";
    }
    @include breakpoints.large-desktop {
      grid-template-columns: repeat(5, minmax(0, 1fr));
      grid-template-areas:
        "oscillator1 oscillator2 oscillator3 volumelfo envelopegain"
        "filter envelopefilter oscilloscope oscilloscope oscilloscope"
        "keyboard keyboard keyboard keyboard keyboard";
    }
  }
  height: 100%;
  border: 2px solid var(--accent-color);
  border-radius: 32px;

  // module
  .module {
    border: 1px solid var(--accent-color);
    border-radius: 12px;

    .header {
      min-height: 26px;
      background-color: var(--accent-color);
      display: flex;
      align-items: center;
      padding: 4px;
      border-radius: 10px 10px 0 0;
      &.header--no-border-radius {
        border-radius: 0;
      }
      h2 {
        margin-bottom: 0;
      }
      input:where([type="checkbox"][role="switch"]) {
        margin-left: auto;
        -webkit-appearance: none;
        -moz-appearance: none;
        appearance: none;
        position: relative;
        font-size: inherit;
        width: 2em;
        height: 1em;
        box-sizing: content-box;
        border: 1px solid;
        border-radius: 1em;
        vertical-align: text-bottom;
        color: inherit;
        &:checked {
          color: var(--disabled-color);
        }
        &:not(:checked) {
          &::before {
            left: 1em;
          }
        }
        &::before {
          content: "";
          position: absolute;
          top: 50%;
          left: 0;
          transform: translate(0, -50%);
          box-sizing: border-box;
          width: 0.7em;
          height: 0.7em;
          margin: 0 0.15em;
          border: 1px solid;
          border-radius: 50%;
          background: currentcolor;
        }
        &:focus {
          outline: 2px solid var(--outline-color);
          border-radius: 4px;
          border-color: var(--outline-color);
          box-shadow: 0 0 0 2px var(--outline-color);
        }
      }
      label:has(+ input[type="checkbox"]) {
        position: absolute;
        overflow: hidden;
        height: 1px;
        width: 1px;
        margin: -1px;
        padding: 0;
        border: 0;
        clip: rect(0 0 0 0);
        clip-path: inset(50%);
      }
    }

    .controls {
      padding: 8px;
      display: flex;
      flex-direction: column;
      gap: 12px;
    }
  }

  // label
  label {
    display: flex;
    padding-bottom: 4px;
    .right {
      margin-left: auto;
    }
  }

  // range slider
  .range-container {
    input[type="range"] {
      appearance: none;
      -webkit-appearance: none;
      width: 100%;
      height: 16px;
      background: var(--input-background-color);
      border: black solid 1px;
      border-radius: 4px;
      margin: 0;

      &::-webkit-slider-thumb {
        appearance: none;
        -webkit-appearance: none;
        background: black;
        width: 6px;
        height: 16px;
        opacity: 1;
      }
      &:focus {
        outline: none;
        border-color: var(--accent-color);
        box-shadow: 0 0 0 2px var(--accent-color);
      }
    }
  }

  // select
  .select-container {
    select {
      width: 100%;
      padding: 4px 4px;
      font-size: 12px;
      line-height: 1.5;
      border: 0px solid transparent;
      border-radius: 0;
      background-color: var(--input-background-color);
      border: var(--outline-color) solid 1px;
      border-radius: 4px;
      cursor: pointer;
      color: var(--outline-color);
      font-family: var(--font-synth);
      text-transform: lowercase;
      appearance: none;
      background-image: url("data:image/svg+xml;charset=utf8,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 448 512'%3E%3Cpath fill='%23000000' d='M443.5 162.6l-7.1-7.1c-4.7-4.7-12.3-4.7-17 0L224 351 28.5 155.5c-4.7-4.7-12.3-4.7-17 0l-7.1 7.1c-4.7 4.7-4.7 12.3 0 17l211 211.1c4.7 4.7 12.3 4.7 17 0l211-211.1c4.8-4.7 4.8-12.3.1-17z'%3E%3C/path%3E%3C/svg%3E");
      background-repeat: no-repeat, repeat;
      background-position: right 0.7em top 50%;
      background-size: 1em;
      &:hover {
        border-color: var(--accent-color);
      }
      &:focus {
        outline: none;
        border-color: var(--accent-color);
        box-shadow: 0 0 0 2px var(--accent-color);
      }
    }
  }

  // place modules on grid
  .oscillator {
    &-1 {
      grid-area: oscillator1;
    }
    &-2 {
      grid-area: oscillator2;
    }
    &-3 {
      grid-area: oscillator3;
    }
  }

  .volume-lfo {
    grid-area: volumelfo;
  }

  .envelope--gain {
    grid-area: envelopegain;
  }

  .filter {
    grid-area: filter;
  }

  .envelope--filter {
    grid-area: envelopefilter;
  }

  .keyboard {
    grid-area: keyboard;
    display: flex;
    justify-content: center;
    .keys {
      @include breakpoints.tablet {
        width: 500px !important;
      }
      @include breakpoints.desktop {
        width: 500px !important;
      }
      @include breakpoints.large-desktop {
        width: 500px !important;
      }
      ul {
        display: initial;
      }
    }
  }

  .oscilloscope {
    grid-area: oscilloscope;
    display: flex;
    flex-direction: column;
    height: 100%;

    .canvas-container {
      flex: 1;
      min-height: 0;
      position: relative;
    }

    .canvas {
      display: block;
      height: 100%;
      width: 100%;
      border-radius: 0 0 10px 12px;
      @include breakpoints.tablet {
        border-radius: 0 0 16px 16px;
      }
      @include breakpoints.desktop {
        border-radius: 0 0 16px 16px;
      }
      @include breakpoints.large-desktop {
        border-radius: 0 0 16px 16px;
      }
    }
  }
}

Next.js Synth: Adding an Oscilloscope

What we're building

next.js synth mk3

Oscillator 1

Oscillator 2

Oscillator 3

Volume

LFO

Gain envelope

Filter

filter envelope

Oscilloscope

Import Oscilloscope

Add analyzer to useSynthEngine

Create Oscilloscope component

Update styles

next.js synth mk3

Oscillator 1

Oscillator 2

Oscillator 3

Volume

LFO

Gain envelope

Filter

filter envelope

Oscilloscope