Media: boot recovery skips the jobs this process is running (a request's job was failed as interrupted, then its result refused) (#99)

OP OpenVibers committed 12 Oct 2026 · 3a9b6f59e2 · 1 file, +3 −1

`worker.start()` runs boot recovery over every row marked `running`, on the assumption that they belong to a process that died. It did not skip the jobs this process was running itself, unlike `tick()`'s own recovery, which already passes `except: running`. Such a job (for example a `runNow` from a request, or anything claimed before `start()`) was failed as "interrupted by a restart". Its real result then arrived and was refused as stale.

`jobs.test.js` hit this under load, on main as well as on branches: the async thumbnail job ended failed, and the log showed exactly this sequence:

    [Jobs] 1 job(s) interrupted by the restart were requeued or failed
    [Jobs] refused a stale succeed of mjob_… from a holder that lost it (it is failed now)

**The change:** `start()` now passes `except: new Set(running.keys())`, the same rule `tick()` uses. In production `start()` runs at boot, so the window is small, but a request that runs a thumbnail before the worker starts would have been failed the same way.

I could not reproduce the race on demand once the machine was quiet. `ov test --repeat 5 jobs` passes 5 of 5 with the fix, and also 5 of 5 without it at the time, so this is hardening, matched to the logged failure, rather than a proven root-cause fix. If it recurs, the log above will show whether the recovered job was this process's own.

Changes

ddiff --git a/server/jobs/worker.js b/server/jobs/worker.js
iindex c6c3770..9d33107 100644
---- a/server/jobs/worker.js
++++ b/server/jobs/worker.js
@@@ -198,7 +198,9 @@ function abort(id) {
  async function start() {
      if (started || !cfg().enabled) return false;
      started = true;
--    const n = await queue.recoverInterrupted({ all: true });
++    // Every running row is from a dead process — except one this process is running right now (runNow on a request,
++    // or a job claimed before start()): failing it here would refuse its own result as stale (jobs.test.js flaked on it).
++    const n = await queue.recoverInterrupted({ all: true, except: new Set(running.keys()) });
      if (n) console.log(`[Jobs] ${n} job(s) interrupted by the restart were requeued or failed`);
      lastScheduleAt = Date.now();           // the first scheduled scan waits a few minutes after boot
      timer = setInterval(() => { tick().catch(() => {}); }, cfg().pollMs);