This is the story of a decision that took three days, broke our streaming engine twice, and ultimately reshaped the entire Conductor Live architecture. If you're building a browser-based production tool and evaluating streaming platforms, this might save you the same three days.

The original plan: Mux

Mux was the obvious first choice. Every developer tutorial about video streaming points you to Mux. The API is clean. The documentation is excellent. The dashboard is beautiful. When we started building the streaming layer for Conductor, Mux was the default assumption.

Our architecture at the time was straightforward: the director's browser composites multiple video streams onto a canvas element, calls canvas.captureStream(30) to get a MediaStream, and pushes that stream to a streaming platform for delivery to viewers. The canvas gives us compositing control — scene switching, graphics overlays, layout changes — and the streaming platform handles the hard part of encoding and delivery at scale.

We built a useWhipStream hook that would take the canvas MediaStream and push it via WHIP (WebRTC HTTP Ingest Protocol) to Mux's endpoint. WHIP is the emerging standard for browser-to-server video ingest — it's essentially WebRTC without the complexity of peer-to-peer negotiation. Clean, simple, browser-native.

There was just one problem.

Mux doesn't support WHIP

We discovered this the hard way. Mux has a real-time product called Mux Spaces that uses WebRTC, but their live streaming product — the one that gives you HLS delivery, CDN, and the viewer scale we needed — only accepts RTMP and SRT for ingest. There is no WHIP endpoint.

This is a fundamental problem when your compositor runs in a browser. Browsers don't speak RTMP. They don't speak SRT. They speak WebRTC. If the streaming platform doesn't accept WebRTC ingest, you need something in the middle to translate.

So we built a relay.

The MediaMTX relay

MediaMTX is an open-source media server that speaks almost every protocol. The plan was elegant on paper:

Browser canvas → WHIP → MediaMTX → FFmpeg → SRT → Mux

We spun up an AWS Lightsail instance, ran MediaMTX in Docker, configured it to accept WHIP on port 8889, and set up an FFmpeg pipeline to transcode the incoming WebRTC stream to SRT for Mux ingest. Cost: $10/month for the Lightsail instance.

It worked in testing. Then we tried it with actual multi-participant video compositing and the video track kept dropping. The relay was adding a hop that introduced instability — the browser's WebRTC connection to MediaMTX would occasionally renegotiate, and during renegotiation the video track would disappear. FFmpeg would lose its input. The SRT stream to Mux would die. The viewer would see a black screen.

We spent a full day debugging this. Adjusted ICE candidate gathering. Tuned FFmpeg buffer sizes. Tried RTSP instead of WHIP as the internal protocol. Nothing was reliable enough for a production broadcast tool.

The relay architecture was adding complexity to solve a problem that shouldn't exist in the first place. The streaming platform should accept what the browser can send.

Finding IVS

Amazon Interactive Video Service was not on our original evaluation list. We found it while researching WHIP ingest alternatives. IVS has a feature that changed everything for us: the IVS Web Broadcast SDK.

The SDK runs in the browser. You give it a MediaStream — exactly what canvas.captureStream() produces — and it handles the WebRTC connection to Amazon's ingest infrastructure directly. No relay. No protocol translation. No FFmpeg. The browser talks to IVS over WebRTC on port 4443, and IVS handles encoding, packaging, and CDN delivery.

Browser canvas → IVS Web Broadcast SDK → Amazon IVS → CloudFront CDN → Viewer
One hop. Zero relay infrastructure.

We ripped out the Mux integration, the MediaMTX relay, and the useWhipStream hook in a single session. The Lightsail instance went from being a critical piece of infrastructure to unnecessary. The new streaming path was browser-native and worked on the first test.

What IVS actually gives you

Beyond solving the ingest problem, IVS turned out to be the right choice for several reasons we didn't anticipate:

Per-show channel provisioning. IVS lets you create channels programmatically via API. Each Conductor show gets its own dedicated IVS channel with its own stream key and playback URL. When the show ends, the channel can be torn down. This maps perfectly to our show lifecycle model — every show is isolated, every show has dedicated infrastructure.

Low-Latency mode. IVS offers a low-latency channel type that delivers HLS at 2-3 seconds rather than the typical 8-15 seconds. For a KOL advisory board where the moderator asks a question and expects a timely response from the audience, 2-3 seconds is workable. 15 seconds is not.

CloudFront CDN included. IVS streams are delivered via Amazon CloudFront's 300+ edge locations globally. We didn't need to set up a separate CDN or worry about geographic distribution. It's included in the IVS pricing.

Timed metadata. IVS supports injecting timed metadata into the stream that arrives at the viewer synchronized with the video frame. This is how we'll implement synchronized graphics — a lower third appears on the viewer's screen at exactly the frame the director takes it live, not 3 seconds later when the HLS segment boundary arrives.

What we gave up

Mux has better analytics, a more polished dashboard, and a developer experience that's genuinely a pleasure to work with. Their player SDK is more mature. Their documentation is some of the best in the industry.

IVS's documentation is AWS documentation, which means it's comprehensive but dense. The console is the AWS console, which means it's functional but not beautiful. The developer experience is "AWS developer experience," which is a specific flavor that some people love and some people tolerate.

We also gave up Mux's excellent video analytics — viewer engagement metrics, quality of experience scoring, attention tracking. IVS has basic CloudWatch metrics but nothing approaching Mux's analytics depth. We'll likely add Mux back as a delivery layer for analytics in the future, using IVS for ingest and Mux for viewer-facing delivery and metrics.

The plot twist: Ant Media

IVS solved the ingest problem but introduced a new one. The IVS Web Broadcast SDK uses the browser's software H.264 encoder, and on macOS it doesn't use the hardware encoder even when one is available. At 1080p30, the software encoder consumed so much CPU that it starved the canvas compositor. We dropped to 720p24 as a mitigation, but that's not broadcast quality.

The real fix — and where the architecture is heading — is server-side GPU compositing via Ant Media Server. But that's a story for the next post.

The streaming platform decision was ultimately about one thing: does it accept what the browser can send? Mux doesn't. IVS does. Everything else followed from that.

Next in this series: Hitting the CPU wall: why browser compositing broke at 1080p