Visualizing a YouTube video's audio
Why a web page cannot read the sound of an embedded YouTube video, what getDisplayMedia can do instead, and how my audio visualizer falls back to a beat.
· Updated · 4 min read

A web page cannot read the audio of an embedded YouTube video. The sound comes from a cross-origin iframe, so it cannot be connected to an AnalyserNode. The only route in is to ask the visitor to share the tab's audio, which desktop Chromium supports. I have not yet tested a real share with audio. Everywhere else, you have to fake it from the tempo.
I built an audio visualizer, Resonance, with three ordinary sources and one hard one. This post is about the hard one, and about keeping the visuals within safe flash limits.
The easy sources#
Web Audio gives you an AnalyserNode. MDN describes it as a node that "passes the audio stream unchanged from the input to the output, but allows you to take the generated data" for visualizations. Resonance has three inputs that feed it directly:
- A built-in track: 104 BPM in A minor, synthesised live in code: a sine kick with a pitch drop, filtered-noise snare and hats, saw and sub bass, a pad and an arpeggio through an echo and a reverb whose impulse response is generated noise.
- The microphone: after a click. It connects to the analyser only and is never played back, so there is no feedback.
- A file: picked or dropped from the visitor's device.
The graph is source → bus → compressor → analyser → master gain → speakers. Volume and mute act after the analyser, so the picture still reacts at volume 0.
Turning an FFT into something you can draw#
The analyser uses a 2048-sample FFT, which gives 1,024 frequency bins; MDN says frequencyBinCount is half of fftSize. Linear bins are evenly spaced in Hz, but we hear pitch on a log scale, so most of 1,024 linear bins would land in the treble. So the bins are folded into 128 log-spaced bins from 30 Hz to 16 kHz. The bass gets as much room as the treble.
Those 128 bins and 128 waveform samples go to the GPU every frame as one 128 × 2 byte texture. Bass (30 to 160 Hz), mids (160 to 2,400 Hz) and highs (2.4 to 12 kHz) are the mean loudness of their bins, followed with a fast attack and a slower release.
The hard source: YouTube#
A YouTube embed plays inside an iframe from another origin, and a page cannot route a cross-origin frame's audio into its own audio graph. There is no createMediaElementSource for it, and no workaround that I know of.
The one door is screen sharing. On desktop Chromium the click that picks the video also calls:
const stream = await navigator.mediaDevices.getDisplayMedia({
audio: true,
video: true,
preferCurrentTab: true,
});
If the visitor ticks "Share tab audio", the stream should have an audio track. I send that track to the analyser only. It never reaches the speakers, because the tab already plays it, and I stop the video track at once.
MDN warns that the returned stream can have no audio track even when audio is requested, that browser support for audio tracks varies, and that the call needs a user gesture and a secure context. It also says browsers may ignore the hints and decide which sources to offer.
The fallback: follow the beat#
When the share is declined, has no audio, or the browser is not desktop Chromium, there is nothing to analyse. The orb then follows a synthetic beat at the song's published tempo for the preset videos, and at 120 BPM for a pasted link: a kick envelope on each beat and hats on the off-beats. The player card says "following the beat" and does not claim to be listening.
Embedding YouTube without paying for it up front#
The embed is a youtube-nocookie.com iframe created only after a click, because the player is about 1 MB. Links are parsed into an 11-character video ID, covering watch, youtu.be, Shorts, embed and live links, and only the ID reaches the embed URL. The parser has unit tests.
Keep the flashing safe#
Beat-reactive pictures can flash. WCAG 2.3.1 is a Level A criterion that says pages must not contain anything that flashes more than three times in any one second period, unless the flashes stay below its thresholds.
Resonance is designed to stay under it, though I have not checked it with a flash-analysis tool. A beat needs a bass sample clearly above its running mean, on a rising edge, at least 340 ms after the last beat. That is fewer than three a second. The pulse is a 40 ms rise and an exponential fall, and it lifts the overall exposure by at most 12 percent.
Lifecycle#
- No audio plays before a click. The
AudioContextis created by the first Play, Track, Mic or File press. - Pausing suspends the context. Leaving the page stops the synth, the file, every microphone and tab-capture track, and closes the context.
- The frame loop stops off screen and in hidden tabs.
- Under reduced motion the orb is a still frame before play, and beats do not pulse.
What I have not tried yet#
I have not tried a real tab share with audio in desktop Chrome on a variety of machines, and I have not tested YouTube in Safari or on phones. In headless Chromium the share was declined, so I only saw the beat fallback. I did not measure how close the synthetic beat is to a real track.