Running FFmpeg in the Browser with WebAssembly

FFmpeg is the Swiss Army knife of video processing — but it's a C project with tens of millions of lines. Running the whole thing inside a browser sounds like science fiction. Yet that's exactly what ffmpeg.wasm does every day, and it's the technical foundation behind every "processed locally in your browser, nothing uploaded" tool on this site. These are the notes from our journey taking it from "it runs" to "it's usable."

Why WASM Instead of a Server

Server-side transcoding (upload → run ffmpeg → send back) has three unavoidable problems: bandwidth costs, queueing delays, and privacy concerns. A user who just wants to cut 10 seconds off the front of a video would have to upload gigabytes of raw footage to the cloud.

WebAssembly offers a third path: compile FFmpeg to .wasm bytecode and execute it at near-native speed inside the browser sandbox. The costs:

The single-file core is about 30 MB — slow on first load
Memory is capped by the browser's 32-bit WASM limit (roughly 2–4 GB)
Multithreading requires cross-origin isolation headers (more below)

For "small files, single-step jobs, privacy-sensitive" use cases (trimming, format conversion, compression, audio extraction), this trade is well worth it.

Loading Strategy: CDN First, Local Fallback

The core is 30 MB, and your loading strategy determines the first-run experience. Ours: CDN first, fall back to a self-hosted copy on failure:

import { FFmpeg } from '@ffmpeg/ffmpeg'
import { toBlobURL } from '@ffmpeg/util'

const CDN_BASE = 'https://unpkg.com/@ffmpeg/core@0.12.10/dist/esm'

async function loadCore(ffmpeg: FFmpeg) {
  try {
    // CDN first: tens of milliseconds on a cache hit
    await ffmpeg.load({
      coreURL: await toBlobURL(`${CDN_BASE}/ffmpeg-core.js`, 'text/javascript'),
      wasmURL: await toBlobURL(`${CDN_BASE}/ffmpeg-core.wasm`, 'application/wasm'),
    })
  } catch {
    // You MUST terminate() before falling back, or the worker is left half-initialized
    ffmpeg.terminate()
    await ffmpeg.load({
      coreURL: await toBlobURL('/ffmpeg-core/ffmpeg-core.js', 'text/javascript'),
      wasmURL: await toBlobURL('/ffmpeg-core/ffmpeg-core.wasm', 'application/wasm'),
    })
  }
}

Three details worth emphasizing:

Call terminate() before the fallback. A failed load() can leave behind a half-initialized worker, and a second load() on top of it produces bizarre errors.
Use toBlobURL instead of passing the URL directly. Loading the worker script from a blob URL sidesteps cross-origin restrictions and keeps CDN and local resources perfectly symmetric in the loading logic.
Download progress and processing progress are two different things. Progress during ffmpeg.load() is the 30 MB core download; progress during ffmpeg.exec() is actual processing. Surface them separately in the UI, or users will stare at a progress bar stuck at 0% while the core downloads.

SharedArrayBuffer and Cross-Origin Isolation

This is the biggest deployment trap. The multithreaded build of ffmpeg.wasm depends on SharedArrayBuffer, and browsers only allow it on cross-origin isolated pages. Your nginx needs two response headers:

add_header Cross-Origin-Opener-Policy same-origin;
add_header Cross-Origin-Embedder-Policy require-corp;

Without them, nothing errors out — the page works, but SharedArrayBuffer is undefined and ffmpeg.wasm silently degrades to single-threaded (or fails to load, depending on the version). These "performance quietly halved" issues are the hardest to diagnose. Before shipping, verify in the console:

console.log(typeof SharedArrayBuffer !== 'undefined') // must be true

Watch the side effect: with COEP: require-corp, every cross-origin resource on the page (third-party images, fonts, analytics scripts) must come with CORS authorization, or it fails to load outright. Keep that in mind whenever you add a third-party service.

MemFS: Doing the Memory Math on Large Files

ffmpeg.wasm's file system is an in-memory virtual FS (MemFS); every read and write goes through the JS heap:

// Write: JS memory → MemFS (note: this copies the data)
await ffmpeg.writeFile('input.mp4', await fetchFile(file))

// Execute
await ffmpeg.exec(['-i', 'input.mp4', '-ss', '10', '-c', 'copy', 'output.mp4'])

// Read back: MemFS → JS memory (yet another copy)
const data = await ffmpeg.readFile('output.mp4')

Do the math: for a 500 MB video, after writeFile there are already two copies in memory (the File object buffer plus the MemFS copy), and with the WASM heap's processing overhead on top, peak usage easily approaches the browser tab's memory ceiling — cross it and you get a merciless AbortError: out of memory.

Practical countermeasures:

Use -c copy whenever possible. Operations like trimming and container swapping don't re-encode; both memory and time drop by an order of magnitude.
Delete as soon as you're done. Call deleteFile() right after exec finishes to clear MemFS. Long sessions (processing file after file) accumulate without cleanup.
Cap input size at the product level. Our tools suggest desktop FFmpeg for files beyond a certain size — acknowledging the boundary beats pretending you can handle everything.

Performance: From "It Runs" to "It's Acceptable"

Single-threaded ffmpeg.wasm runs at roughly 20%–50% of native speed depending on the operation. A few optimizations that actually work:

Multithreaded core: @ffmpeg/core-mt plus the cross-origin isolation headers unlocks multiple cores for a 2–4× speedup (the core is larger, too).
The -threads flag: with the MT core, pass ['-threads', String(navigator.hardwareConcurrency ?? 4)] explicitly.
Precise trimming: prefer -ss (input-side seek) over output-side trimming — the amount of encoding work can differ by an order of magnitude.
Avoid pointless re-encoding: container conversion (MP4↔MKV) and lossless cuts go through stream copy; pay the full price only when re-encoding is genuinely required.

Pitfall Cheat Sheet

Symptom	Root cause	Fix
Second load fails after CDN failure	Worker half-initialized	`terminate()` before falling back
Production several times slower than dev	Missing COOP/COEP headers, silently single-threaded	Add the isolation headers, verify SharedArrayBuffer
`out of memory` on large files	MemFS + JS heap holding multiple copies	Prefer `-c copy`, `deleteFile` promptly, cap input size
Progress bar stuck at 0%	Core download progress mistaken for processing progress	Separate `load` and `exec` progress phases
Zero-byte output file	exec argument error with unchecked return code	Check the `exec` return value; validate before `readFile`

When Not to Use WASM

Honestly, the browser isn't a silver bullet: re-encoding long videos over 1 GB, batch jobs that need GPU acceleration, and license-sensitive codecs like H.265 are still better served by a server or a local CLI. The answer is a hybrid strategy — lightweight single-step operations happen locally in the browser, and heavy jobs get clear guidance to the right tool.

Want to see it in action? The FFmpeg Command Lab lets you run arbitrary FFmpeg commands in your browser, and the Video Trimmer is a typical application of this stack — your files never leave your computer.

FAQ

Q: Does ffmpeg.wasm support every FFmpeg filter and encoder? A: Not all of them. The official core omits some encoders (such as the multithreaded libx265 build) and anything requiring system dependencies. Verify your target encoder exists in the build before committing to it.

Q: Do files really never touch a server? A: Correct. All the core logic runs inside the browser sandbox; the server only serves static HTML/JS/wasm assets. That's exactly why we're comfortable putting "no uploads" in our product copy.

Q: Does Safari work? A: Yes, but older versions had compatibility issues combining OffscreenCanvas with Workers, and the multithreaded core's Safari behavior deserves real-world verification.