openvibers/openvibe.media Commits
Operator retire: a delisted v1 row (is_public 0) is not public (#328)
Changes
@@@ -5,8 +5,9 @@- Operator retirement for an object with no good copy (`scripts/no-good-copy-report.js --retire <med_id> [--apply]`):the dry run changes nothing; only an object in the report, not public and under 1 MiB is retired (a public v1 row, anative public object, good-copy, 1 MiB+ and held objects are refused with the reason). For a legacy VOD/clip the v1-- row decides: a private or unlisted row makes the object not public even when its `media_objects` row still says-- public. The delete is the ordinary soft delete (bytes kept, retention, `media.object.deleted`, restorable), and a++ row decides: a delisted row (`is_public` 0, even with `visibility` still public after the health quarantine), or a++ private/unlisted one, makes the object not public even when its `media_objects` row still says public. The delete is++ the ordinary soft delete (bytes kept, retention, `media.object.deleted`, restorable), and aprojected VOD/clip has its v1 row made private so the inherited route stops serving it. Test:`test/retire-no-good-copy.test.js`.@@@ -617,7 +617,7 @@ that only moved to B2/R2 keeps it. Test: `test/content-hash.test.js`.node scripts/no-good-copy-report.js --retire <med_id> [--retire <med_id> ...] [--apply]```-- The dry run (no `--apply`) prints what it would do and writes nothing. Only an object that is in the report (no present or pending copy), is not public and is under 1 MiB is retired; anything else — a public v1 row, a native public object, one with a good copy, one of 1 MiB or more, one under a retention hold — is refused with the reason and nothing is changed. For a legacy VOD or clip object the v1 row decides: when its `vods`/`clips` row is private or unlisted the object counts as not public even if its `media_objects` row still says public, because that row is what `/v/:id` and `/c/:id` serve. The delete is the ordinary soft delete (`model.softDelete`): the bytes stay, `metadata.retention_until` is set, the same `media.object.deleted` event is queued as for any other delete, and `POST /:id/restore` brings the object back inside the retention period. A projected VOD or clip also has its v1 row made private (`visibility private`, `is_public 0`) so `/v/:id` and `/c/:id` stop serving it; the row is kept, not removed, so the retirement stays reversible. A file object is refused because `/f/:key` serves every file without auth. Test: `test/retire-no-good-copy.test.js`.++ The dry run (no `--apply`) prints what it would do and writes nothing. Only an object that is in the report (no present or pending copy), is not public and is under 1 MiB is retired; anything else — a public v1 row, a native public object, one with a good copy, one of 1 MiB or more, one under a retention hold — is refused with the reason and nothing is changed. For a legacy VOD or clip object the v1 row decides: when its `vods`/`clips` row is not public — `is_public` 0 (the health quarantine leaves `visibility` public), or private/unlisted — the object counts as not public even if its `media_objects` row still says public. The delete is the ordinary soft delete (`model.softDelete`): the bytes stay, `metadata.retention_until` is set, the same `media.object.deleted` event is queued as for any other delete, and `POST /:id/restore` brings the object back inside the retention period. A projected VOD or clip also has its v1 row made private (`visibility private`, `is_public 0`) so `/v/:id` and `/c/:id` stop serving it; the row is kept, not removed, so the retirement stays reversible. A file object is refused because `/f/:key` serves every file without auth. Test: `test/retire-no-good-copy.test.js`.## Public object-size invariant@@@ -7,9 +7,9 @@* "scheduled verification"):** - it is in the no-good-copy report (listNoGoodCopy: ready, not virtual, no present/pending copy);-- * - it is not public (a native public object, or a public v1 row, is refused); a legacy VOD/clip object whose-- * v1 row is private counts as not public even when its media_objects envelope still says public, because that-- * row is what /v/:id and /c/:id serve;++ * - it is not public (a native public object, or a v1 row that is public, is refused); a legacy VOD/clip object++ * counts as not public when its v1 row is not public (is_public 0, even if visibility still says public — the++ * health quarantine's shape), even when its media_objects envelope still says public;* - it is under 1 MiB: a stub or a broken small upload, never a real recording;* - no retention hold applies.*@@@ -45,6 +45,15 @@ function rowVisibility(row) {return row.visibility || (row.is_public ? 'public' : 'private');}++/**++ * Whether a v1 row is public for retirement purposes: the route would show it, and it is not delisted. The health++ * quarantine sets is_public = 0 and leaves visibility = 'public'; such a row is not listed anywhere, so it is not++ * public here and its object can be retired.++ */++function rowPublic(row) {++ return rowVisibility(row) === 'public' && (row.is_public == null || Number(row.is_public) !== 0);++}++function humanBytes(n) {n = Number(n) || 0;if (n < 1024) return `${n} B`;@@@ -65,12 +74,13 @@ async function assess(id, { q = null } = {}) {const inReport = (await copyReport.listNoGoodCopy(reads, { appId: obj.app_id })).some((o) => o.id === obj.id);if (!inReport) return { ok: false, id, obj, reason: 'not in the no-good-copy report (it has a good copy, is virtual, or is not ready)' };const refs = await copyReport.references(reads, obj);-- const publicVod = refs.vods.find((r) => rowVisibility(r) === 'public');++ const publicVod = refs.vods.find(rowPublic);if (publicVod) return { ok: false, id, obj, reason: `its vods#${publicVod.id} row is public` };-- const publicClip = refs.clips.find((r) => rowVisibility(r) === 'public');++ const publicClip = refs.clips.find(rowPublic);if (publicClip) return { ok: false, id, obj, reason: `its clips#${publicClip.id} row is public` };-- // The v1 row decides for a projected VOD/clip: an older write can leave the envelope at public after the row is-- // private, and the row is what /v/:id and /c/:id serve. A native object has no row, so its own visibility stands.++ // The v1 row decides for a projected VOD/clip: an older write or the health quarantine (is_public 0, visibility++ // still public) can leave the envelope at public after the row is delisted. A native object has no row, so its++ // own visibility stands.if (obj.visibility === 'public' && !refs.vods.length && !refs.clips.length) {return { ok: false, id, obj, reason: 'it is public' };}@@@ -2,7 +2,7 @@// Operator retirement of a no-good-copy object (scripts/no-good-copy-report.js --retire): the dry run prints and// changes nothing; a stub is soft-deleted (retention metadata, restorable, media.object.deleted staged) with its v1// VOD row hidden, and the readiness count (the same count /api/ready's object_copies check reads) goes to 0; a legacy--// object whose envelope says public but whose v1 row is private is still retired (the row decides); native public++// object whose envelope says public but whose v1 row is delisted (is_public 0) is still retired (the row decides); native public// objects, public v1 rows, objects with a good copy, objects of 1 MiB or more and held objects are refused.const assert = require('assert');const fs = require('fs');@@@ -80,10 +80,14 @@ const path = require('path');assert.strictEqual((await verifyJob.status()).no_good_copy, 0);console.log('✅ retires the stub: soft delete, retention, hidden v1 row, media.object.deleted, readiness count 0');-- // ── A legacy envelope still saying public, whose v1 row is private (the production drift): the row decides ──++ // ── The production drift: the media_objects row says public and the v1 row's health quarantine left visibility++ // public but is_public 0. The delisted row decides, so the object can be retired ──const stale = await vod({ title: 'stale envelope' });++ await d.prepare("UPDATE vods SET visibility = 'public', is_public = 0 WHERE id = ?").run(stale.id);await d.prepare("UPDATE media_objects SET visibility = 'public' WHERE id = ?").run(stale.objectId);assert.strictEqual((await model.getObject(stale.objectId)).visibility, 'public', 'the legacy object row still says public');++ const staleBefore = await d.prepare('SELECT visibility, is_public FROM vods WHERE id = ?').get(stale.id);++ assert.deepStrictEqual([staleBefore.visibility, Number(staleBefore.is_public)], ['public', 0], 'the v1 row is delisted but still says visibility public');assert.strictEqual(await copyReport.countNoGoodCopy(db), 1);const staleDry = await run(['--retire', stale.objectId]);assert.strictEqual(staleDry.code, 0, staleDry.err);@@@ -95,7 +99,7 @@ const path = require('path');const staleRow = await d.prepare('SELECT visibility, is_public FROM vods WHERE object_id = ?').get(stale.objectId);assert.deepStrictEqual([staleRow.visibility, Number(staleRow.is_public)], ['private', 0], 'the private v1 row stays private');assert.strictEqual(await copyReport.countNoGoodCopy(db), 0);-- console.log('✅ retires a legacy VOD whose object row says public while its v1 row is private');++ console.log('✅ retires a legacy VOD whose object row says public while its v1 row is is_public 0');// ── Refusals: nothing is changed, the reason says why ──const good = await vod({ title: 'good', size: 100, bytes: 100 });
@@ -5,8 +5,9 @@
@@ -5,8 +5,9 @@
- Operator retirement for an object with no good copy (`scripts/no-good-copy-report.js --retire <med_id> [--apply]`):
- Operator retirement for an object with no good copy (`scripts/no-good-copy-report.js --retire <med_id> [--apply]`):
the dry run changes nothing; only an object in the report, not public and under 1 MiB is retired (a public v1 row, a
the dry run changes nothing; only an object in the report, not public and under 1 MiB is retired (a public v1 row, a
native public object, good-copy, 1 MiB+ and held objects are refused with the reason). For a legacy VOD/clip the v1
native public object, good-copy, 1 MiB+ and held objects are refused with the reason). For a legacy VOD/clip the v1
- row decides: a private or unlisted row makes the object not public even when its `media_objects` row still says
- public. The delete is the ordinary soft delete (bytes kept, retention, `media.object.deleted`, restorable), and a
+ row decides: a delisted row (`is_public` 0, even with `visibility` still public after the health quarantine), or a
+ private/unlisted one, makes the object not public even when its `media_objects` row still says public. The delete is
+ the ordinary soft delete (bytes kept, retention, `media.object.deleted`, restorable), and a
projected VOD/clip has its v1 row made private so the inherited route stops serving it. Test:
projected VOD/clip has its v1 row made private so the inherited route stops serving it. Test:
`test/retire-no-good-copy.test.js`.
`test/retire-no-good-copy.test.js`.
@@ -617,7 +617,7 @@ that only moved to B2/R2 keeps it. Test: `test/content-hash.test.js`.
@@ -617,7 +617,7 @@ that only moved to B2/R2 keeps it. Test: `test/content-hash.test.js`.
node scripts/no-good-copy-report.js --retire <med_id> [--retire <med_id> ...] [--apply]
node scripts/no-good-copy-report.js --retire <med_id> [--retire <med_id> ...] [--apply]
```
```
- The dry run (no `--apply`) prints what it would do and writes nothing. Only an object that is in the report (no present or pending copy), is not public and is under 1 MiB is retired; anything else — a public v1 row, a native public object, one with a good copy, one of 1 MiB or more, one under a retention hold — is refused with the reason and nothing is changed. For a legacy VOD or clip object the v1 row decides: when its `vods`/`clips` row is private or unlisted the object counts as not public even if its `media_objects` row still says public, because that row is what `/v/:id` and `/c/:id` serve. The delete is the ordinary soft delete (`model.softDelete`): the bytes stay, `metadata.retention_until` is set, the same `media.object.deleted` event is queued as for any other delete, and `POST /:id/restore` brings the object back inside the retention period. A projected VOD or clip also has its v1 row made private (`visibility private`, `is_public 0`) so `/v/:id` and `/c/:id` stop serving it; the row is kept, not removed, so the retirement stays reversible. A file object is refused because `/f/:key` serves every file without auth. Test: `test/retire-no-good-copy.test.js`.
+ The dry run (no `--apply`) prints what it would do and writes nothing. Only an object that is in the report (no present or pending copy), is not public and is under 1 MiB is retired; anything else — a public v1 row, a native public object, one with a good copy, one of 1 MiB or more, one under a retention hold — is refused with the reason and nothing is changed. For a legacy VOD or clip object the v1 row decides: when its `vods`/`clips` row is not public — `is_public` 0 (the health quarantine leaves `visibility` public), or private/unlisted — the object counts as not public even if its `media_objects` row still says public. The delete is the ordinary soft delete (`model.softDelete`): the bytes stay, `metadata.retention_until` is set, the same `media.object.deleted` event is queued as for any other delete, and `POST /:id/restore` brings the object back inside the retention period. A projected VOD or clip also has its v1 row made private (`visibility private`, `is_public 0`) so `/v/:id` and `/c/:id` stop serving it; the row is kept, not removed, so the retirement stays reversible. A file object is refused because `/f/:key` serves every file without auth. Test: `test/retire-no-good-copy.test.js`.
## Public object-size invariant
## Public object-size invariant
@@ -7,9 +7,9 @@
@@ -7,9 +7,9 @@
* "scheduled verification"):
* "scheduled verification"):
*
*
* - it is in the no-good-copy report (listNoGoodCopy: ready, not virtual, no present/pending copy);
* - it is in the no-good-copy report (listNoGoodCopy: ready, not virtual, no present/pending copy);
- * - it is not public (a native public object, or a public v1 row, is refused); a legacy VOD/clip object whose
- * v1 row is private counts as not public even when its media_objects envelope still says public, because that
- * row is what /v/:id and /c/:id serve;
+ * - it is not public (a native public object, or a v1 row that is public, is refused); a legacy VOD/clip object
+ * counts as not public when its v1 row is not public (is_public 0, even if visibility still says public — the
+ * health quarantine's shape), even when its media_objects envelope still says public;
* - it is under 1 MiB: a stub or a broken small upload, never a real recording;
* - it is under 1 MiB: a stub or a broken small upload, never a real recording;
* - no retention hold applies.
* - no retention hold applies.
*
*
@@ -45,6 +45,15 @@ function rowVisibility(row) {
@@ -45,6 +45,15 @@ function rowVisibility(row) {
return row.visibility || (row.is_public ? 'public' : 'private');
return row.visibility || (row.is_public ? 'public' : 'private');
}
}
+/**
+ * Whether a v1 row is public for retirement purposes: the route would show it, and it is not delisted. The health
+ * quarantine sets is_public = 0 and leaves visibility = 'public'; such a row is not listed anywhere, so it is not
+ * public here and its object can be retired.
+ */
+function rowPublic(row) {
+ return rowVisibility(row) === 'public' && (row.is_public == null || Number(row.is_public) !== 0);
+}
+
function humanBytes(n) {
function humanBytes(n) {
n = Number(n) || 0;
n = Number(n) || 0;
if (n < 1024) return `${n} B`;
if (n < 1024) return `${n} B`;
@@ -65,12 +74,13 @@ async function assess(id, { q = null } = {}) {
@@ -65,12 +74,13 @@ async function assess(id, { q = null } = {}) {
const inReport = (await copyReport.listNoGoodCopy(reads, { appId: obj.app_id })).some((o) => o.id === obj.id);
const inReport = (await copyReport.listNoGoodCopy(reads, { appId: obj.app_id })).some((o) => o.id === obj.id);
if (!inReport) return { ok: false, id, obj, reason: 'not in the no-good-copy report (it has a good copy, is virtual, or is not ready)' };
if (!inReport) return { ok: false, id, obj, reason: 'not in the no-good-copy report (it has a good copy, is virtual, or is not ready)' };
const refs = await copyReport.references(reads, obj);
const refs = await copyReport.references(reads, obj);
- const publicVod = refs.vods.find((r) => rowVisibility(r) === 'public');
+ const publicVod = refs.vods.find(rowPublic);
if (publicVod) return { ok: false, id, obj, reason: `its vods#${publicVod.id} row is public` };
if (publicVod) return { ok: false, id, obj, reason: `its vods#${publicVod.id} row is public` };
- const publicClip = refs.clips.find((r) => rowVisibility(r) === 'public');
+ const publicClip = refs.clips.find(rowPublic);
if (publicClip) return { ok: false, id, obj, reason: `its clips#${publicClip.id} row is public` };
if (publicClip) return { ok: false, id, obj, reason: `its clips#${publicClip.id} row is public` };
- // The v1 row decides for a projected VOD/clip: an older write can leave the envelope at public after the row is
- // private, and the row is what /v/:id and /c/:id serve. A native object has no row, so its own visibility stands.
+ // The v1 row decides for a projected VOD/clip: an older write or the health quarantine (is_public 0, visibility
+ // still public) can leave the envelope at public after the row is delisted. A native object has no row, so its
+ // own visibility stands.
if (obj.visibility === 'public' && !refs.vods.length && !refs.clips.length) {
if (obj.visibility === 'public' && !refs.vods.length && !refs.clips.length) {
return { ok: false, id, obj, reason: 'it is public' };
return { ok: false, id, obj, reason: 'it is public' };
}
}
@@ -2,7 +2,7 @@
@@ -2,7 +2,7 @@
// Operator retirement of a no-good-copy object (scripts/no-good-copy-report.js --retire): the dry run prints and
// Operator retirement of a no-good-copy object (scripts/no-good-copy-report.js --retire): the dry run prints and
// changes nothing; a stub is soft-deleted (retention metadata, restorable, media.object.deleted staged) with its v1
// changes nothing; a stub is soft-deleted (retention metadata, restorable, media.object.deleted staged) with its v1
// VOD row hidden, and the readiness count (the same count /api/ready's object_copies check reads) goes to 0; a legacy
// VOD row hidden, and the readiness count (the same count /api/ready's object_copies check reads) goes to 0; a legacy
-// object whose envelope says public but whose v1 row is private is still retired (the row decides); native public
+// object whose envelope says public but whose v1 row is delisted (is_public 0) is still retired (the row decides); native public
// objects, public v1 rows, objects with a good copy, objects of 1 MiB or more and held objects are refused.
// objects, public v1 rows, objects with a good copy, objects of 1 MiB or more and held objects are refused.
const assert = require('assert');
const assert = require('assert');
const fs = require('fs');
const fs = require('fs');
@@ -80,10 +80,14 @@ const path = require('path');
@@ -80,10 +80,14 @@ const path = require('path');
assert.strictEqual((await verifyJob.status()).no_good_copy, 0);
assert.strictEqual((await verifyJob.status()).no_good_copy, 0);
console.log('✅ retires the stub: soft delete, retention, hidden v1 row, media.object.deleted, readiness count 0');
console.log('✅ retires the stub: soft delete, retention, hidden v1 row, media.object.deleted, readiness count 0');
- // ── A legacy envelope still saying public, whose v1 row is private (the production drift): the row decides ──
+ // ── The production drift: the media_objects row says public and the v1 row's health quarantine left visibility
+ // public but is_public 0. The delisted row decides, so the object can be retired ──
const stale = await vod({ title: 'stale envelope' });
const stale = await vod({ title: 'stale envelope' });
+ await d.prepare("UPDATE vods SET visibility = 'public', is_public = 0 WHERE id = ?").run(stale.id);
await d.prepare("UPDATE media_objects SET visibility = 'public' WHERE id = ?").run(stale.objectId);
await d.prepare("UPDATE media_objects SET visibility = 'public' WHERE id = ?").run(stale.objectId);
assert.strictEqual((await model.getObject(stale.objectId)).visibility, 'public', 'the legacy object row still says public');
assert.strictEqual((await model.getObject(stale.objectId)).visibility, 'public', 'the legacy object row still says public');
+ const staleBefore = await d.prepare('SELECT visibility, is_public FROM vods WHERE id = ?').get(stale.id);
+ assert.deepStrictEqual([staleBefore.visibility, Number(staleBefore.is_public)], ['public', 0], 'the v1 row is delisted but still says visibility public');
assert.strictEqual(await copyReport.countNoGoodCopy(db), 1);
assert.strictEqual(await copyReport.countNoGoodCopy(db), 1);
const staleDry = await run(['--retire', stale.objectId]);
const staleDry = await run(['--retire', stale.objectId]);
assert.strictEqual(staleDry.code, 0, staleDry.err);
assert.strictEqual(staleDry.code, 0, staleDry.err);
@@ -95,7 +99,7 @@ const path = require('path');
@@ -95,7 +99,7 @@ const path = require('path');
const staleRow = await d.prepare('SELECT visibility, is_public FROM vods WHERE object_id = ?').get(stale.objectId);
const staleRow = await d.prepare('SELECT visibility, is_public FROM vods WHERE object_id = ?').get(stale.objectId);
assert.deepStrictEqual([staleRow.visibility, Number(staleRow.is_public)], ['private', 0], 'the private v1 row stays private');
assert.deepStrictEqual([staleRow.visibility, Number(staleRow.is_public)], ['private', 0], 'the private v1 row stays private');
assert.strictEqual(await copyReport.countNoGoodCopy(db), 0);
assert.strictEqual(await copyReport.countNoGoodCopy(db), 0);
- console.log('✅ retires a legacy VOD whose object row says public while its v1 row is private');
+ console.log('✅ retires a legacy VOD whose object row says public while its v1 row is is_public 0');
// ── Refusals: nothing is changed, the reason says why ──
// ── Refusals: nothing is changed, the reason says why ──
const good = await vod({ title: 'good', size: 100, bytes: 100 });
const good = await vod({ title: 'good', size: 100, bytes: 100 });