openvibers/openvibe.live Commits

RS relay: forward only the media SSRC (root cause of the video cut-outs)

OP OpenVibers committed 18 Aug 2026 · c89a85db62 · 2 files, +127 −7

mediasoup's RTP probation generator emits bandwidth-probing padding on a
fixed ssrc=1234 / payloadType=127, and it lands on the same plain-transport
socket as the media — measured live at ~1.6/s beside the real H264 ssrc:

  ssrc=378058538  pt=103  5262 pkts  seq 39276..44537   (contiguous)
  ssrc=1234       pt=127    16 pkts  seq 62728..62743

emitVideo() forwarded everything on that socket. It rewrote the payload type
to 103 so the padding looked like video, werift rewrote the ssrc to the
sender's, and the probation packet's SEQUENCE NUMBER went out untouched. So
RobotStreamer received one video stream whose sequence jumped ~18000 back and
forth several times a second.

RS runs a pre-3.12 mediasoup (it still advertises the framemarking extensions
removed in 3.12). RtpStreamRecv::UpdateSeq treats a jump past MaxDropout(3000)
as "sequence made a very large jump": discard the packet, arm badSeq, and keep
computing `expected` off a max_seq dragged into the probation range. Loss
inflates to phantom levels — RS reported 462203 packets, ~72% — the producer
score collapses to 0 and its consumers go INACTIVE. Video stops reaching
viewers, and because inactive consumers do not request keyframes RS goes quiet
instead of PLI-storming. Audio rides a separate ssrc with no probation on it
and plays throughout: the reported symptom exactly. The phantom gaps also
drove the NACK ranges behind the earlier retransmit storm.

Measured with a viewer probe that joins RS as a browser does: a 37.6s video
blackout with audio at a steady 50 p/s and max audio gap 154ms, while relay
telemetry showed vIn 529/s == vOut 529/s and a keyframe every second
throughout. Nothing on the sending side was wrong; we were poisoning RS's
sequence accounting with packets that were never media.

Take the consumer's own ssrc as authoritative rather than first-seen, so a
probation packet arriving first can never define the stream, and drop anything
else on both the video and audio ingests.

Co-Authored-By: Claude Opus 5 <[email protected]>

Changes

ddiff --git a/server/integrations/rs-passthrough-relay.js b/server/integrations/rs-passthrough-relay.js
iindex 0b1cb9a3..686f1cc7 100644
---- a/server/integrations/rs-passthrough-relay.js
++++ b/server/integrations/rs-passthrough-relay.js
@@@ -272,7 +272,7 @@ async function openPlainIngest(sfu, roomId, producerId) {
const info = await sfu.createPlainConsumer(roomId, producerId, '127.0.0.1', port, port + 1);
return {
socket, port, transportId: info.transportId, consumerId: info.consumerId,
-- payloadType: info.payloadType, kind: info.kind,
++ payloadType: info.payloadType, kind: info.kind, ssrc: info.ssrc,
mimeType: info.mimeType, clockRate: info.clockRate, channels: info.channels,
codecParameters: info.codecParameters || {},
};
@@@ -483,7 +483,7 @@ class RsPassthroughRelay {
aIn: 0, aOut: 0, aErr: 0, // audio: same
pli: 0, lostIn: 0, kf: 0, maxKfGap: 0, inj: 0,
rtxDeep: 0, rtxMiss: 0, rtxTooOld: 0, gopDamaged: 0, lateFill: 0,
-- ingestStalls: 0, ingestStallMs: 0,
++ ingestStalls: 0, ingestStallMs: 0, vAlien: 0, aAlien: 0,
};
const vPt = +vMedia.pts[0];
@@@ -668,13 +668,41 @@ class RsPassthroughRelay {
reqKey(); // pull a keyframe now …
setTimeout(() => reqKeyNow(250), 300); // … and a backstop in case it's lost too
};
++ // ── Only the media SSRC may be forwarded ────────────────────────────────────────
++ // mediasoup's own RTP probation generator emits bandwidth-probing padding on a FIXED
++ // ssrc=1234 / payloadType=127, and those packets land on this same plain-transport
++ // socket (measured: ~1.6/s alongside the real H264 ssrc). emitVideo() used to forward
++ // them like any other packet — rewriting PT 127 to 103 so they looked like video, while
++ // werift rewrote the ssrc to the sender's. What it could NOT rewrite is the sequence
++ // number, so RobotStreamer saw ONE video stream whose sequence jumped ~18000 between
++ // the media range and the probation range, several times a second.
++ //
++ // For RS's (pre-3.12) mediasoup that is fatal. RtpStreamRecv::UpdateSeq treats a jump
++ // past MaxDropout(3000) as "sequence made a very large jump": it discards the packet and
++ // arms badSeq, and its `expected` count is computed off a max_seq that keeps getting
++ // yanked into the probation range. The result is enormous PHANTOM loss — RS reported
++ // 462203 packets lost, ~72% — which collapses the producer's score to 0. At score 0 its
++ // consumers go INACTIVE: video stops reaching viewers and, because inactive consumers do
++ // not request keyframes, RS goes quiet rather than PLI-storming (exactly the flat pliRx
++ // observed across a measured 37.6s blackout). Audio rides a separate ssrc with no
++ // probation traffic on it, so it plays throughout — the reported symptom precisely.
++ // Its NackGenerator also NACKs the phantom gaps, which is what fed the retransmit storm.
++ //
++ // Forwarding a foreign SSRC was never correct: probation padding is not media, and the
++ // consumer's own ssrc is authoritative. Prefer it over first-seen so a probation packet
++ // arriving first can never define the stream.
++ const vSsrcExpect = videoIn.ssrc;
videoIn.socket.on('message', (buf) => {
const now = Date.now();
const timeGap = (now - _lastVideoAt) > VIDEO_GAP_MS;
-- stats.vIn++;
-- noteIngest('video', now);
let p;
try { p = RtpPacket.deSerialize(buf); } catch { return; /* drop malformed */ }
++ if (vSsrcExpect != null && p.header.ssrc !== vSsrcExpect) {
++ if (stats.vAlien++ === 0) log(sid, `dropping non-media RTP on video ingest: ssrc=${p.header.ssrc} pt=${p.header.payloadType} (mediasoup probation/rtx; forwarding it corrupts RS's sequence accounting)`);
++ return;
++ }
++ stats.vIn++;
++ noteIngest('video', now);
// Reorder-tolerant loss detection on the MAIN video stream only (ignore RTX/other SSRCs
// and out-of-order arrivals — those were inflating the count and spamming keyframes).
let burst = false;
@@@ -740,11 +768,16 @@ class RsPassthroughRelay {
}
if (audioIn && aMedia) {
const aPt = +aMedia.pts[0];
++ const aSsrcExpect = audioIn.ssrc;
audioIn.socket.on('message', (buf) => {
-- stats.aIn++;
-- noteIngest('audio', Date.now());
try {
const p = RtpPacket.deSerialize(buf);
++ if (aSsrcExpect != null && p.header.ssrc !== aSsrcExpect) {
++ if (stats.aAlien++ === 0) log(sid, `dropping non-media RTP on audio ingest: ssrc=${p.header.ssrc} pt=${p.header.payloadType}`);
++ return;
++ }
++ stats.aIn++;
++ noteIngest('audio', Date.now());
p.header.payloadType = aPt; p.header.extensions = [];
audioTrack.writeRtp(p); stats.aOut++;
} catch (e) { if (stats.aErr++ === 0) log(sid, 'audio writeRtp failed (first of run):', e.message); }
@@@ -773,7 +806,7 @@ class RsPassthroughRelay {
const stalled = _videoStalledSince ? ` STALLED-${Date.now() - _videoStalledSince}ms` : '';
log(sid, `flow: vIn ${Math.round((stats.vIn - lastVIn) / 10)}/s vOut ${Math.round((stats.vOut - lastVOut) / 10)}/s ` +
`aIn ${Math.round((stats.aIn - lastAIn) / 10)}/s aOut ${Math.round((stats.aOut - lastAOut) / 10)}/s${stalled} ` +
-- `| writeErr v=${stats.vErr} a=${stats.aErr} | stalls=${stats.ingestStalls} (${Math.round(stats.ingestStallMs / 1000)}s total) ` +
++ `| writeErr v=${stats.vErr} a=${stats.aErr} alienSsrc v=${stats.vAlien} a=${stats.aAlien} | stalls=${stats.ingestStalls} (${Math.round(stats.ingestStallMs / 1000)}s total) ` +
`| kf ${stats.kf - lastKf}/10s maxKfGap ${stats.maxKfGap}ms injSPS/PPS=${stats.inj}${INJECT_PARAMS ? '' : '(off)'} gopDmg=${stats.gopDamaged} ` +
`| lossIN ${stats.lostIn - lastLostIn} (goosely→relay) lateFill ${stats.lateFill - lastLateFill} ` +
`| RS lost=${rsLostPct}% pktsLost=${s.remotePacketsLost ?? '?'} pliRx=${s.pliCount ?? '?'} firRx=${s.firCount ?? '?'} nackRx=${s.nackCount ?? '?'} retx=${s.retransmittedPacketsSent ?? '?'} deepRtx=${stats.rtxDeep} rtxMiss=${stats.rtxMiss} rtxTooOld=${stats.rtxTooOld} rtt=${Math.round((s.rtt || 0) * 1000)}ms ` +
ddiff --git a/test/rs-passthrough-ssrc-filter.test.js b/test/rs-passthrough-ssrc-filter.test.js
nnew file mode 100644
iindex 00000000..b9b13b65
---- /dev/null
++++ b/test/rs-passthrough-ssrc-filter.test.js
@@@ -0,0 +1,87 @@
++/**
++ * Regression: the RS passthrough relay must forward ONLY the consumer's media SSRC.
++ *
++ * mediasoup's RTP probation generator emits bandwidth-probing padding on a fixed
++ * ssrc=1234 / payloadType=127, and it arrives on the same plain-transport socket as the
++ * media. The relay used to forward every packet on that socket: it rewrote the payload
++ * type to the video PT so the padding looked like H264, werift rewrote the SSRC to the
++ * sender's, and the probation packet's SEQUENCE NUMBER went out unchanged.
++ *
++ * RobotStreamer therefore saw one video stream whose sequence jumped ~18000 back and
++ * forth several times a second. Their mediasoup (pre-3.12 — it still advertises the
++ * framemarking extensions removed in 3.12) treats a jump past MaxDropout(3000) as a
++ * "very large jump": discard the packet, arm badSeq, and keep computing `expected` from a
++ * max_seq that has been dragged into the probation range. That inflates loss to ~72%,
++ * collapses the producer score to 0, and deactivates every consumer — video stops
++ * reaching viewers while audio (a separate SSRC, no probation on it) keeps playing.
++ *
++ * This test asserts the filter drops foreign SSRCs and that the surviving sequence
++ * numbers stay monotonic, which is the property RS's sequence accounting depends on.
++ */
++const assert = require('assert');
++
++const MEDIA_SSRC = 378058538;
++const PROBATION_SSRC = 1234; // RTC::RtpProbationGenerator
++const PROBATION_PT = 127;
++const VIDEO_PT = 103;
++
++/** The relay's ingest guard, mirrored exactly. */
++function accept(expectedSsrc, pkt) {
++ return !(expectedSsrc != null && pkt.ssrc !== expectedSsrc);
++}
++
++// A realistic ingest burst: contiguous media interleaved with probation padding whose
++// sequence numbers sit ~18000 away, which is what was measured on the live socket.
++const ingest = [];
++let mediaSeq = 39276, probSeq = 62728;
++for (let i = 0; i < 600; i++) {
++ ingest.push({ ssrc: MEDIA_SSRC, pt: VIDEO_PT, seq: mediaSeq++ & 0xffff });
++ if (i % 40 === 39) ingest.push({ ssrc: PROBATION_SSRC, pt: PROBATION_PT, seq: probSeq++ & 0xffff });
++}
++
++// ── Without the filter: reproduce the corruption ────────────────────────────────────
++const unfiltered = ingest.slice();
++let bigJumps = 0, prev = null;
++for (const p of unfiltered) {
++ if (prev !== null) {
++ const adv = (p.seq - prev) & 0xffff;
++ if (adv >= 3000 && adv <= 65536 - 100) bigJumps++; // mediasoup MaxDropout / MaxMisorder
++ }
++ prev = p.seq;
++}
++assert.ok(bigJumps > 0, 'expected the unfiltered stream to contain mediasoup-fatal sequence jumps');
++console.log(`OK A: unfiltered ingest produces ${bigJumps} sequence jumps past MaxDropout — the corruption`);
++
++// ── With the filter: clean, monotonic media only ────────────────────────────────────
++const filtered = ingest.filter(p => accept(MEDIA_SSRC, p));
++assert.strictEqual(filtered.length, 600, 'filter must keep every media packet');
++assert.ok(filtered.every(p => p.ssrc === MEDIA_SSRC), 'no foreign SSRC may survive the filter');
++assert.ok(filtered.every(p => p.pt === VIDEO_PT), 'no foreign payload type may survive the filter');
++
++let jumpsAfter = 0; prev = null;
++for (const p of filtered) {
++ if (prev !== null) {
++ const adv = (p.seq - prev) & 0xffff;
++ if (adv >= 3000 && adv <= 65536 - 100) jumpsAfter++;
++ }
++ prev = p.seq;
++}
++assert.strictEqual(jumpsAfter, 0, 'filtered stream must contain NO mediasoup-fatal sequence jumps');
++console.log('OK B: filtered ingest is contiguous — zero jumps past MaxDropout');
++
++// ── The guard must not disarm when the SSRC is unknown ──────────────────────────────
++assert.ok(accept(null, { ssrc: PROBATION_SSRC }), 'unknown expected SSRC must fall back to forwarding');
++assert.ok(!accept(MEDIA_SSRC, { ssrc: MEDIA_SSRC + 1 }), 'a near-miss SSRC must still be rejected');
++assert.ok(accept(MEDIA_SSRC, { ssrc: MEDIA_SSRC }), 'the media SSRC must always pass');
++console.log('OK C: guard edge cases (unknown ssrc, near-miss, exact match)');
++
++// ── The relay must actually contain the guard ───────────────────────────────────────
++const fs = require('fs'), path = require('path');
++const src = fs.readFileSync(path.join(__dirname, '../server/integrations/rs-passthrough-relay.js'), 'utf8');
++assert.ok(src.includes('const vSsrcExpect = videoIn.ssrc'), 'relay must derive the expected video SSRC from the consumer');
++assert.ok(src.includes('p.header.ssrc !== vSsrcExpect'), 'relay must drop video packets from a foreign SSRC');
++assert.ok(src.includes('p.header.ssrc !== aSsrcExpect'), 'relay must drop audio packets from a foreign SSRC');
++assert.ok(/ssrc: info\.ssrc/.test(src), 'openPlainIngest must propagate the consumer SSRC');
++console.log('OK D: rs-passthrough-relay.js carries the SSRC guard on both ingests');
++
++console.log('✅ RS passthrough SSRC-filter regression test passed');