openvibers/openvibe.media Commits
Operator retire: the v1 row decides publicness for a legacy VOD/clip (#327)
Changes
@@@ -3,10 +3,12 @@## Unreleased- Operator retirement for an object with no good copy (`scripts/no-good-copy-report.js --retire <med_id> [--apply]`):-- the dry run changes nothing; only an object in the report, not public and under 1 MiB is retired (public, good-copy,-- 1 MiB+ and held objects are refused with the reason). The delete is the ordinary soft delete (bytes kept,-- retention, `media.object.deleted`, restorable), and a projected VOD/clip has its v1 row made private so the inherited-- route stops serving it. Test: `test/retire-no-good-copy.test.js`.++ the dry run changes nothing; only an object in the report, not public and under 1 MiB is retired (a public v1 row, a++ native public object, good-copy, 1 MiB+ and held objects are refused with the reason). For a legacy VOD/clip the v1++ row decides: a private or unlisted row makes the object not public even when its `media_objects` row still says++ public. The delete is the ordinary soft delete (bytes kept, retention, `media.object.deleted`, restorable), and a++ projected VOD/clip has its v1 row made private so the inherited route stops serving it. Test:++ `test/retire-no-good-copy.test.js`.- A legacy Live paste screenshot can be retired for OpenVibe.Community (2026-11-06). The internal`POST /internal/community/legacy-paste-screenshots/:id/retire` route is loopback and service-token only@@@ -617,7 +617,7 @@ that only moved to B2/R2 keeps it. Test: `test/content-hash.test.js`.node scripts/no-good-copy-report.js --retire <med_id> [--retire <med_id> ...] [--apply]```-- The dry run (no `--apply`) prints what it would do and writes nothing. Only an object that is in the report (no present or pending copy), is not public and is under 1 MiB is retired; anything else — a public object, one with a good copy, one of 1 MiB or more, one under a retention hold — is refused with the reason and nothing is changed. The delete is the ordinary soft delete (`model.softDelete`): the bytes stay, `metadata.retention_until` is set, the same `media.object.deleted` event is queued as for any other delete, and `POST /:id/restore` brings the object back inside the retention period. A projected VOD or clip also has its v1 row made private (`visibility private`, `is_public 0`) so `/v/:id` and `/c/:id` stop serving it; the row is kept, not removed, so the retirement stays reversible. A file object is refused because `/f/:key` serves every file without auth. Test: `test/retire-no-good-copy.test.js`.++ The dry run (no `--apply`) prints what it would do and writes nothing. Only an object that is in the report (no present or pending copy), is not public and is under 1 MiB is retired; anything else — a public v1 row, a native public object, one with a good copy, one of 1 MiB or more, one under a retention hold — is refused with the reason and nothing is changed. For a legacy VOD or clip object the v1 row decides: when its `vods`/`clips` row is private or unlisted the object counts as not public even if its `media_objects` row still says public, because that row is what `/v/:id` and `/c/:id` serve. The delete is the ordinary soft delete (`model.softDelete`): the bytes stay, `metadata.retention_until` is set, the same `media.object.deleted` event is queued as for any other delete, and `POST /:id/restore` brings the object back inside the retention period. A projected VOD or clip also has its v1 row made private (`visibility private`, `is_public 0`) so `/v/:id` and `/c/:id` stop serving it; the row is kept, not removed, so the retirement stays reversible. A file object is refused because `/f/:key` serves every file without auth. Test: `test/retire-no-good-copy.test.js`.## Public object-size invariant@@@ -7,7 +7,9 @@* "scheduled verification"):** - it is in the no-good-copy report (listNoGoodCopy: ready, not virtual, no present/pending copy);-- * - it is not public (a public object, or a public v1 row, is refused);++ * - it is not public (a native public object, or a public v1 row, is refused); a legacy VOD/clip object whose++ * v1 row is private counts as not public even when its media_objects envelope still says public, because that++ * row is what /v/:id and /c/:id serve;* - it is under 1 MiB: a stub or a broken small upload, never a real recording;* - no retention hold applies.*@@@ -62,17 +64,20 @@ async function assess(id, { q = null } = {}) {const reads = q || query();const inReport = (await copyReport.listNoGoodCopy(reads, { appId: obj.app_id })).some((o) => o.id === obj.id);if (!inReport) return { ok: false, id, obj, reason: 'not in the no-good-copy report (it has a good copy, is virtual, or is not ready)' };-- if (obj.visibility === 'public') return { ok: false, id, obj, reason: 'it is public' };-- const size = Number(obj.size_bytes) || 0;-- if (size >= MAX_BYTES) return { ok: false, id, obj, reason: `${humanBytes(size)} is not under 1 MiB` };-- if (await model.isHeld(obj.id)) return { ok: false, id, obj, reason: 'it is under a retention hold' };--const refs = await copyReport.references(reads, obj);-- if (refs.files.length) return { ok: false, id, obj, reason: `its files/${refs.files[0].key} row is served publicly by /f/:key` };const publicVod = refs.vods.find((r) => rowVisibility(r) === 'public');if (publicVod) return { ok: false, id, obj, reason: `its vods#${publicVod.id} row is public` };const publicClip = refs.clips.find((r) => rowVisibility(r) === 'public');if (publicClip) return { ok: false, id, obj, reason: `its clips#${publicClip.id} row is public` };++ // The v1 row decides for a projected VOD/clip: an older write can leave the envelope at public after the row is++ // private, and the row is what /v/:id and /c/:id serve. A native object has no row, so its own visibility stands.++ if (obj.visibility === 'public' && !refs.vods.length && !refs.clips.length) {++ return { ok: false, id, obj, reason: 'it is public' };++ }++ const size = Number(obj.size_bytes) || 0;++ if (size >= MAX_BYTES) return { ok: false, id, obj, reason: `${humanBytes(size)} is not under 1 MiB` };++ if (await model.isHeld(obj.id)) return { ok: false, id, obj, reason: 'it is under a retention hold' };++ if (refs.files.length) return { ok: false, id, obj, reason: `its files/${refs.files[0].key} row is served publicly by /f/:key` };return {ok: true, id, obj, size,@@@ -1,8 +1,9 @@'use strict';// Operator retirement of a no-good-copy object (scripts/no-good-copy-report.js --retire): the dry run prints and// changes nothing; a stub is soft-deleted (retention metadata, restorable, media.object.deleted staged) with its v1--// VOD row hidden, and the readiness count (the same count /api/ready's object_copies check reads) goes to 0; public--// objects, objects with a good copy, objects of 1 MiB or more and held objects are refused with the reason.++// VOD row hidden, and the readiness count (the same count /api/ready's object_copies check reads) goes to 0; a legacy++// object whose envelope says public but whose v1 row is private is still retired (the row decides); native public++// objects, public v1 rows, objects with a good copy, objects of 1 MiB or more and held objects are refused.const assert = require('assert');const fs = require('fs');const os = require('os');@@@ -79,15 +80,34 @@ const path = require('path');assert.strictEqual((await verifyJob.status()).no_good_copy, 0);console.log('✅ retires the stub: soft delete, retention, hidden v1 row, media.object.deleted, readiness count 0');++ // ── A legacy envelope still saying public, whose v1 row is private (the production drift): the row decides ──++ const stale = await vod({ title: 'stale envelope' });++ await d.prepare("UPDATE media_objects SET visibility = 'public' WHERE id = ?").run(stale.objectId);++ assert.strictEqual((await model.getObject(stale.objectId)).visibility, 'public', 'the legacy object row still says public');++ assert.strictEqual(await copyReport.countNoGoodCopy(db), 1);++ const staleDry = await run(['--retire', stale.objectId]);++ assert.strictEqual(staleDry.code, 0, staleDry.err);++ assert.match(staleDry.out, /hide: vods#\d+ \(visibility private, is_public 0\)/);++ assert.match(staleDry.out, /dry run: nothing was changed/);++ const staleApplied = await run(['--retire', stale.objectId, '--apply']);++ assert.strictEqual(staleApplied.code, 0, staleApplied.err);++ assert.strictEqual((await model.getObject(stale.objectId)).lifecycle_status, 'deleted');++ const staleRow = await d.prepare('SELECT visibility, is_public FROM vods WHERE object_id = ?').get(stale.objectId);++ assert.deepStrictEqual([staleRow.visibility, Number(staleRow.is_public)], ['private', 0], 'the private v1 row stays private');++ assert.strictEqual(await copyReport.countNoGoodCopy(db), 0);++ console.log('✅ retires a legacy VOD whose object row says public while its v1 row is private');++// ── Refusals: nothing is changed, the reason says why ──const good = await vod({ title: 'good', size: 100, bytes: 100 });const publicVod = await vod({ title: 'public', visibility: 'public', isPublic: 1 });++ const nativePublic = { objectId: await model.createObject({ app_id: 'live', owner_user_id: 5, kind: 'vod', visibility: 'public', lifecycle_status: 'ready', size_bytes: 1000 }) };const big = await vod({ title: 'big', size: MB });const held = await vod({ title: 'held' });await model.placeHold({ object_id: held.objectId, kind: 'admin', reason: 'test hold', created_by: 'test' });-- assert.strictEqual(await copyReport.countNoGoodCopy(db), 3, 'public, big and held; not the one with a good copy');++ assert.strictEqual(await copyReport.countNoGoodCopy(db), 4, 'public v1 row, native public, big and held; not the one with a good copy');for (const [v, reason] of [-- [publicVod, /it is public/],++ [publicVod, /its vods#\d+ row is public/],++ [nativePublic, /it is public/],[good, /not in the no-good-copy report/],[big, /not under 1 MiB/],[held, /retention hold/],@@@ -99,12 +119,12 @@ const path = require('path');assert.match(r.out, /Nothing was changed/);assert.deepStrictEqual(await state(v), before, 'a refused retirement writes nothing');}-- console.log('✅ refuses a public object, one with a good copy, one of 1 MiB and one under a hold, with the reason');++ console.log('✅ refuses a public v1 row, a native public object, one with a good copy, one of 1 MiB and one under a hold, with the reason');// ── Restorable ──const back = await model.restore(await model.getObject(stub.objectId));assert.strictEqual(back.lifecycle_status, 'ready', 'a restore inside retention brings it back');-- assert.strictEqual(await copyReport.countNoGoodCopy(db), 4, 'and it is a no-good-copy object again');++ assert.strictEqual(await copyReport.countNoGoodCopy(db), 5, 'and it is a no-good-copy object again');console.log('✅ restorable inside the retention period');await db.close();
@@ -3,10 +3,12 @@
@@ -3,10 +3,12 @@
## Unreleased
## Unreleased
- Operator retirement for an object with no good copy (`scripts/no-good-copy-report.js --retire <med_id> [--apply]`):
- Operator retirement for an object with no good copy (`scripts/no-good-copy-report.js --retire <med_id> [--apply]`):
- the dry run changes nothing; only an object in the report, not public and under 1 MiB is retired (public, good-copy,
- 1 MiB+ and held objects are refused with the reason). The delete is the ordinary soft delete (bytes kept,
- retention, `media.object.deleted`, restorable), and a projected VOD/clip has its v1 row made private so the inherited
- route stops serving it. Test: `test/retire-no-good-copy.test.js`.
+ the dry run changes nothing; only an object in the report, not public and under 1 MiB is retired (a public v1 row, a
+ native public object, good-copy, 1 MiB+ and held objects are refused with the reason). For a legacy VOD/clip the v1
+ row decides: a private or unlisted row makes the object not public even when its `media_objects` row still says
+ public. The delete is the ordinary soft delete (bytes kept, retention, `media.object.deleted`, restorable), and a
+ projected VOD/clip has its v1 row made private so the inherited route stops serving it. Test:
+ `test/retire-no-good-copy.test.js`.
- A legacy Live paste screenshot can be retired for OpenVibe.Community (2026-11-06). The internal
- A legacy Live paste screenshot can be retired for OpenVibe.Community (2026-11-06). The internal
`POST /internal/community/legacy-paste-screenshots/:id/retire` route is loopback and service-token only
`POST /internal/community/legacy-paste-screenshots/:id/retire` route is loopback and service-token only
@@ -617,7 +617,7 @@ that only moved to B2/R2 keeps it. Test: `test/content-hash.test.js`.
@@ -617,7 +617,7 @@ that only moved to B2/R2 keeps it. Test: `test/content-hash.test.js`.
node scripts/no-good-copy-report.js --retire <med_id> [--retire <med_id> ...] [--apply]
node scripts/no-good-copy-report.js --retire <med_id> [--retire <med_id> ...] [--apply]
```
```
- The dry run (no `--apply`) prints what it would do and writes nothing. Only an object that is in the report (no present or pending copy), is not public and is under 1 MiB is retired; anything else — a public object, one with a good copy, one of 1 MiB or more, one under a retention hold — is refused with the reason and nothing is changed. The delete is the ordinary soft delete (`model.softDelete`): the bytes stay, `metadata.retention_until` is set, the same `media.object.deleted` event is queued as for any other delete, and `POST /:id/restore` brings the object back inside the retention period. A projected VOD or clip also has its v1 row made private (`visibility private`, `is_public 0`) so `/v/:id` and `/c/:id` stop serving it; the row is kept, not removed, so the retirement stays reversible. A file object is refused because `/f/:key` serves every file without auth. Test: `test/retire-no-good-copy.test.js`.
+ The dry run (no `--apply`) prints what it would do and writes nothing. Only an object that is in the report (no present or pending copy), is not public and is under 1 MiB is retired; anything else — a public v1 row, a native public object, one with a good copy, one of 1 MiB or more, one under a retention hold — is refused with the reason and nothing is changed. For a legacy VOD or clip object the v1 row decides: when its `vods`/`clips` row is private or unlisted the object counts as not public even if its `media_objects` row still says public, because that row is what `/v/:id` and `/c/:id` serve. The delete is the ordinary soft delete (`model.softDelete`): the bytes stay, `metadata.retention_until` is set, the same `media.object.deleted` event is queued as for any other delete, and `POST /:id/restore` brings the object back inside the retention period. A projected VOD or clip also has its v1 row made private (`visibility private`, `is_public 0`) so `/v/:id` and `/c/:id` stop serving it; the row is kept, not removed, so the retirement stays reversible. A file object is refused because `/f/:key` serves every file without auth. Test: `test/retire-no-good-copy.test.js`.
## Public object-size invariant
## Public object-size invariant
@@ -7,7 +7,9 @@
@@ -7,7 +7,9 @@
* "scheduled verification"):
* "scheduled verification"):
*
*
* - it is in the no-good-copy report (listNoGoodCopy: ready, not virtual, no present/pending copy);
* - it is in the no-good-copy report (listNoGoodCopy: ready, not virtual, no present/pending copy);
- * - it is not public (a public object, or a public v1 row, is refused);
+ * - it is not public (a native public object, or a public v1 row, is refused); a legacy VOD/clip object whose
+ * v1 row is private counts as not public even when its media_objects envelope still says public, because that
+ * row is what /v/:id and /c/:id serve;
* - it is under 1 MiB: a stub or a broken small upload, never a real recording;
* - it is under 1 MiB: a stub or a broken small upload, never a real recording;
* - no retention hold applies.
* - no retention hold applies.
*
*
@@ -62,17 +64,20 @@ async function assess(id, { q = null } = {}) {
@@ -62,17 +64,20 @@ async function assess(id, { q = null } = {}) {
const reads = q || query();
const reads = q || query();
const inReport = (await copyReport.listNoGoodCopy(reads, { appId: obj.app_id })).some((o) => o.id === obj.id);
const inReport = (await copyReport.listNoGoodCopy(reads, { appId: obj.app_id })).some((o) => o.id === obj.id);
if (!inReport) return { ok: false, id, obj, reason: 'not in the no-good-copy report (it has a good copy, is virtual, or is not ready)' };
if (!inReport) return { ok: false, id, obj, reason: 'not in the no-good-copy report (it has a good copy, is virtual, or is not ready)' };
- if (obj.visibility === 'public') return { ok: false, id, obj, reason: 'it is public' };
- const size = Number(obj.size_bytes) || 0;
- if (size >= MAX_BYTES) return { ok: false, id, obj, reason: `${humanBytes(size)} is not under 1 MiB` };
- if (await model.isHeld(obj.id)) return { ok: false, id, obj, reason: 'it is under a retention hold' };
-
const refs = await copyReport.references(reads, obj);
const refs = await copyReport.references(reads, obj);
- if (refs.files.length) return { ok: false, id, obj, reason: `its files/${refs.files[0].key} row is served publicly by /f/:key` };
const publicVod = refs.vods.find((r) => rowVisibility(r) === 'public');
const publicVod = refs.vods.find((r) => rowVisibility(r) === 'public');
if (publicVod) return { ok: false, id, obj, reason: `its vods#${publicVod.id} row is public` };
if (publicVod) return { ok: false, id, obj, reason: `its vods#${publicVod.id} row is public` };
const publicClip = refs.clips.find((r) => rowVisibility(r) === 'public');
const publicClip = refs.clips.find((r) => rowVisibility(r) === 'public');
if (publicClip) return { ok: false, id, obj, reason: `its clips#${publicClip.id} row is public` };
if (publicClip) return { ok: false, id, obj, reason: `its clips#${publicClip.id} row is public` };
+ // The v1 row decides for a projected VOD/clip: an older write can leave the envelope at public after the row is
+ // private, and the row is what /v/:id and /c/:id serve. A native object has no row, so its own visibility stands.
+ if (obj.visibility === 'public' && !refs.vods.length && !refs.clips.length) {
+ return { ok: false, id, obj, reason: 'it is public' };
+ }
+ const size = Number(obj.size_bytes) || 0;
+ if (size >= MAX_BYTES) return { ok: false, id, obj, reason: `${humanBytes(size)} is not under 1 MiB` };
+ if (await model.isHeld(obj.id)) return { ok: false, id, obj, reason: 'it is under a retention hold' };
+ if (refs.files.length) return { ok: false, id, obj, reason: `its files/${refs.files[0].key} row is served publicly by /f/:key` };
return {
return {
ok: true, id, obj, size,
ok: true, id, obj, size,
@@ -1,8 +1,9 @@
@@ -1,8 +1,9 @@
'use strict';
'use strict';
// Operator retirement of a no-good-copy object (scripts/no-good-copy-report.js --retire): the dry run prints and
// Operator retirement of a no-good-copy object (scripts/no-good-copy-report.js --retire): the dry run prints and
// changes nothing; a stub is soft-deleted (retention metadata, restorable, media.object.deleted staged) with its v1
// changes nothing; a stub is soft-deleted (retention metadata, restorable, media.object.deleted staged) with its v1
-// VOD row hidden, and the readiness count (the same count /api/ready's object_copies check reads) goes to 0; public
-// objects, objects with a good copy, objects of 1 MiB or more and held objects are refused with the reason.
+// VOD row hidden, and the readiness count (the same count /api/ready's object_copies check reads) goes to 0; a legacy
+// object whose envelope says public but whose v1 row is private is still retired (the row decides); native public
+// objects, public v1 rows, objects with a good copy, objects of 1 MiB or more and held objects are refused.
const assert = require('assert');
const assert = require('assert');
const fs = require('fs');
const fs = require('fs');
const os = require('os');
const os = require('os');
@@ -79,15 +80,34 @@ const path = require('path');
@@ -79,15 +80,34 @@ const path = require('path');
assert.strictEqual((await verifyJob.status()).no_good_copy, 0);
assert.strictEqual((await verifyJob.status()).no_good_copy, 0);
console.log('✅ retires the stub: soft delete, retention, hidden v1 row, media.object.deleted, readiness count 0');
console.log('✅ retires the stub: soft delete, retention, hidden v1 row, media.object.deleted, readiness count 0');
+ // ── A legacy envelope still saying public, whose v1 row is private (the production drift): the row decides ──
+ const stale = await vod({ title: 'stale envelope' });
+ await d.prepare("UPDATE media_objects SET visibility = 'public' WHERE id = ?").run(stale.objectId);
+ assert.strictEqual((await model.getObject(stale.objectId)).visibility, 'public', 'the legacy object row still says public');
+ assert.strictEqual(await copyReport.countNoGoodCopy(db), 1);
+ const staleDry = await run(['--retire', stale.objectId]);
+ assert.strictEqual(staleDry.code, 0, staleDry.err);
+ assert.match(staleDry.out, /hide: vods#\d+ \(visibility private, is_public 0\)/);
+ assert.match(staleDry.out, /dry run: nothing was changed/);
+ const staleApplied = await run(['--retire', stale.objectId, '--apply']);
+ assert.strictEqual(staleApplied.code, 0, staleApplied.err);
+ assert.strictEqual((await model.getObject(stale.objectId)).lifecycle_status, 'deleted');
+ const staleRow = await d.prepare('SELECT visibility, is_public FROM vods WHERE object_id = ?').get(stale.objectId);
+ assert.deepStrictEqual([staleRow.visibility, Number(staleRow.is_public)], ['private', 0], 'the private v1 row stays private');
+ assert.strictEqual(await copyReport.countNoGoodCopy(db), 0);
+ console.log('✅ retires a legacy VOD whose object row says public while its v1 row is private');
+
// ── Refusals: nothing is changed, the reason says why ──
// ── Refusals: nothing is changed, the reason says why ──
const good = await vod({ title: 'good', size: 100, bytes: 100 });
const good = await vod({ title: 'good', size: 100, bytes: 100 });
const publicVod = await vod({ title: 'public', visibility: 'public', isPublic: 1 });
const publicVod = await vod({ title: 'public', visibility: 'public', isPublic: 1 });
+ const nativePublic = { objectId: await model.createObject({ app_id: 'live', owner_user_id: 5, kind: 'vod', visibility: 'public', lifecycle_status: 'ready', size_bytes: 1000 }) };
const big = await vod({ title: 'big', size: MB });
const big = await vod({ title: 'big', size: MB });
const held = await vod({ title: 'held' });
const held = await vod({ title: 'held' });
await model.placeHold({ object_id: held.objectId, kind: 'admin', reason: 'test hold', created_by: 'test' });
await model.placeHold({ object_id: held.objectId, kind: 'admin', reason: 'test hold', created_by: 'test' });
- assert.strictEqual(await copyReport.countNoGoodCopy(db), 3, 'public, big and held; not the one with a good copy');
+ assert.strictEqual(await copyReport.countNoGoodCopy(db), 4, 'public v1 row, native public, big and held; not the one with a good copy');
for (const [v, reason] of [
for (const [v, reason] of [
- [publicVod, /it is public/],
+ [publicVod, /its vods#\d+ row is public/],
+ [nativePublic, /it is public/],
[good, /not in the no-good-copy report/],
[good, /not in the no-good-copy report/],
[big, /not under 1 MiB/],
[big, /not under 1 MiB/],
[held, /retention hold/],
[held, /retention hold/],
@@ -99,12 +119,12 @@ const path = require('path');
@@ -99,12 +119,12 @@ const path = require('path');
assert.match(r.out, /Nothing was changed/);
assert.match(r.out, /Nothing was changed/);
assert.deepStrictEqual(await state(v), before, 'a refused retirement writes nothing');
assert.deepStrictEqual(await state(v), before, 'a refused retirement writes nothing');
}
}
- console.log('✅ refuses a public object, one with a good copy, one of 1 MiB and one under a hold, with the reason');
+ console.log('✅ refuses a public v1 row, a native public object, one with a good copy, one of 1 MiB and one under a hold, with the reason');
// ── Restorable ──
// ── Restorable ──
const back = await model.restore(await model.getObject(stub.objectId));
const back = await model.restore(await model.getObject(stub.objectId));
assert.strictEqual(back.lifecycle_status, 'ready', 'a restore inside retention brings it back');
assert.strictEqual(back.lifecycle_status, 'ready', 'a restore inside retention brings it back');
- assert.strictEqual(await copyReport.countNoGoodCopy(db), 4, 'and it is a no-good-copy object again');
+ assert.strictEqual(await copyReport.countNoGoodCopy(db), 5, 'and it is a no-good-copy object again');
console.log('✅ restorable inside the retention period');
console.log('✅ restorable inside the retention period');
await db.close();
await db.close();