openvibers/openvibe.media Commits
Operator retirement for a no-good-copy object (#323)
Changes
@@@ -2,6 +2,12 @@## Unreleased++- Operator retirement for an object with no good copy (`scripts/no-good-copy-report.js --retire <med_id> [--apply]`):++ the dry run changes nothing; only an object in the report, not public and under 1 MiB is retired (public, good-copy,++ 1 MiB+ and held objects are refused with the reason). The delete is the ordinary soft delete (bytes kept,++ retention, `media.object.deleted`, restorable), and a projected VOD/clip has its v1 row made private so the inherited++ route stops serving it. Test: `test/retire-no-good-copy.test.js`.++- A legacy Live paste screenshot can be retired for OpenVibe.Community (2026-11-06). The internal`POST /internal/community/legacy-paste-screenshots/:id/retire` route is loopback and service-token only(`media.object.delete`, principal `svc:community`) and soft-deletes exactly a `legacy:live:paste:<slug>`@@@ -610,6 +610,14 @@ that only moved to B2/R2 keeps it. Test: `test/content-hash.test.js`.For each object it prints the owner (app, `owner_user_id`, `owner_subject`), `legacy_ref`, the `vods`/`clips`/`files` rows that point at it, its relationships, variants and holds, every location with its state and `verified_at`, and the last verification. The report opens the database read-only. It exits 1 when there are any.++ Retire one of those objects instead of only listing it:++++ ```++ node scripts/no-good-copy-report.js --retire <med_id> [--retire <med_id> ...] [--apply]++ ```++++ The dry run (no `--apply`) prints what it would do and writes nothing. Only an object that is in the report (no present or pending copy), is not public and is under 1 MiB is retired; anything else — a public object, one with a good copy, one of 1 MiB or more, one under a retention hold — is refused with the reason and nothing is changed. The delete is the ordinary soft delete (`model.softDelete`): the bytes stay, `metadata.retention_until` is set, the same `media.object.deleted` event is queued as for any other delete, and `POST /:id/restore` brings the object back inside the retention period. A projected VOD or clip also has its v1 row made private (`visibility private`, `is_public 0`) so `/v/:id` and `/c/:id` stop serving it; the row is kept, not removed, so the retirement stays reversible. A file object is refused because `/f/:key` serves every file without auth. Test: `test/retire-no-good-copy.test.js`.++## Public object-size invariantThis is roadmap W4 deliverable 5, stated as **policy** rather than a copied constant:@@@ -8,19 +8,66 @@** node scripts/no-good-copy-report.js [--app live] [--json] [--out report.json]*-- * Exit code 0 when there are none, 1 when some were found, 2 on error.++ * Retire one or more of those objects instead (scripts/no-good-copy-report.js --retire):++ *++ * node scripts/no-good-copy-report.js --retire <med_id> [--retire <med_id> ...] [--apply]++ *++ * The dry run (no --apply) prints what it would do and changes nothing. Only an object that is in the++ * report, not public and under 1 MiB is retired, through the ordinary soft delete (bytes kept for the++ * retention period, media.object.deleted queued, restorable) with its v1 VOD/clip row hidden. Anything++ * else is refused with the reason and nothing is changed.++ *++ * Exit code: 0 when the report found none (or every named object can be / was retired), 1 when the++ * report found some (or one named object was refused), 2 on error.*/const fs = require('fs');--const path = require('path');(async () => {await require('../server/db/database').initDb(); // PostgreSQL (DATABASE_URL), as the serviceconst args = process.argv.slice(2);const has = (name) => args.includes(`--${name}`);const arg = (name, dflt) => { const i = args.indexOf(`--${name}`); return i >= 0 ? args[i + 1] : dflt; };++ const retireIds = [];++ for (let i = 0; i < args.length; i++) {++ if (args[i] !== '--retire') continue;++ const value = args[i + 1];++ if (!value || value.startsWith('--')) { retireIds.push(''); continue; }++ retireIds.push(...String(value).split(','));++ }try {-- // Read-only: the report runs in a READ ONLY transaction (PostgreSQL refuses any write inside it).++ // ── Retire named objects (the one write path of this script) ──++ if (retireIds.length) {++ const { assess, describe, retire } = require('../server/objects/retire');++ const ids = [...new Set(retireIds.map((s) => s.trim()).filter(Boolean))];++ if (!ids.length) {++ console.error('--retire needs one or more med_… object ids');++ process.exit(2);++ }++ const plans = [];++ for (const id of ids) plans.push(await assess(id));++ const refused = plans.filter((p) => !p.ok);++ if (refused.length) {++ for (const p of refused) console.log(`${p.id} — refused: ${p.reason}`);++ console.log('Nothing was changed.');++ process.exit(1);++ }++ if (!has('apply')) {++ for (const p of plans) console.log(`${describe(p)}\n dry run: nothing was changed; re-run with --apply to retire it\n`);++ process.exit(0);++ }++ const done = await retire(plans);++ for (const d of done) {++ console.log(`${d.id} retired: ${d.lifecycle_status}; retention until ${d.retention_until || `the end of the ${require('../server/config').objects.retentionDays}-day period`}; restorable`);++ }++ process.exit(0);++ }++ if (has('apply')) {++ console.error('--apply only goes with --retire <med_id>');++ process.exit(2);++ }++++ // ── Report (read-only: the report runs in a READ ONLY transaction, PostgreSQL refuses any write) ──const { buildReport, formatReport } = require('../server/objects/copy-report');const d = require('../server/db/database').getDb();const q = {@@@ -105,4 +105,4 @@ function formatReport(report) {return out.join('\n');}--module.exports = { NO_GOOD_COPY_WHERE, countNoGoodCopy, listNoGoodCopy, buildReport, formatReport };++module.exports = { NO_GOOD_COPY_WHERE, countNoGoodCopy, listNoGoodCopy, references, buildReport, formatReport };nnew file mode 100644@@@ -0,0 +1,121 @@++'use strict';++/**++ * Operator retirement of a no-good-copy object (scripts/no-good-copy-report.js --retire).++ *++ * A ready object with no good copy cannot be repaired when every copy it has is missing or corrupt. The report can++ * only list it; this is the one operator path that retires it, and only the safe shape (docs/object-model.md,++ * "scheduled verification"):++ *++ * - it is in the no-good-copy report (listNoGoodCopy: ready, not virtual, no present/pending copy);++ * - it is not public (a public object, or a public v1 row, is refused);++ * - it is under 1 MiB: a stub or a broken small upload, never a real recording;++ * - no retention hold applies.++ *++ * The delete is the ordinary soft delete (model.softDelete), the same one Media's own delete uses: lifecycle_status++ * becomes deleted, the bytes stay for MEDIA_DELETE_RETENTION_DAYS (metadata.retention_until), the media_objects++ * trigger queues the same media.object.deleted event as any other delete (so Live and Search forget the item), and++ * POST /:id/restore brings it back inside the retention period.++ *++ * A projected VOD or clip is also hidden from its inherited route the way the owning app's delete hides it: its v1++ * row is made private (visibility = private, is_public = 0) through db.setVodVisibility / setClipVisibility, so the++ * row and the object agree. The row is kept, not removed, which is what keeps the retirement reversible; a VOD that++ * is public is refused rather than hidden. A file row keeps being served by /f/:key whatever its object says, so a++ * file object is refused too.++ */++const db = require('../db/database');++const model = require('./model');++const copyReport = require('./copy-report');++++const MAX_BYTES = 1024 * 1024; // strictly under 1 MiB++const BY = 'operator:no-good-copy-report';++const MED_ID = /^med_[0-9A-HJKMNP-TV-Z]{26}$/;++++function query() {++ const d = db.getDb();++ return {++ all: async (sql, params = []) => await d.prepare(sql).all(...params),++ get: async (sql, params = []) => await d.prepare(sql).get(...params),++ };++}++++/** The v1 route's own visibility rule (vods/clips row → public, unlisted or private). */++function rowVisibility(row) {++ return row.visibility || (row.is_public ? 'public' : 'private');++}++++function humanBytes(n) {++ n = Number(n) || 0;++ if (n < 1024) return `${n} B`;++ if (n < 1048576) return `${(n / 1024).toFixed(1)} KB`;++ return `${(n / 1048576).toFixed(1)} MB`;++}++++/**++ * Decide whether an object may be retired. Reads only. Returns++ * { ok: true, id, obj, size, rows, locations } or { ok: false, id, obj?, reason }.++ */++async function assess(id, { q = null } = {}) {++ if (!MED_ID.test(String(id || ''))) return { ok: false, id, reason: 'not a med_… object id' };++ const obj = await model.getObject(id);++ if (!obj) return { ok: false, id, reason: 'no such object' };++ if (obj.lifecycle_status === 'deleted') return { ok: false, id, obj, reason: 'it is already deleted' };++ const reads = q || query();++ const inReport = (await copyReport.listNoGoodCopy(reads, { appId: obj.app_id })).some((o) => o.id === obj.id);++ if (!inReport) return { ok: false, id, obj, reason: 'not in the no-good-copy report (it has a good copy, is virtual, or is not ready)' };++ if (obj.visibility === 'public') return { ok: false, id, obj, reason: 'it is public' };++ const size = Number(obj.size_bytes) || 0;++ if (size >= MAX_BYTES) return { ok: false, id, obj, reason: `${humanBytes(size)} is not under 1 MiB` };++ if (await model.isHeld(obj.id)) return { ok: false, id, obj, reason: 'it is under a retention hold' };++++ const refs = await copyReport.references(reads, obj);++ if (refs.files.length) return { ok: false, id, obj, reason: `its files/${refs.files[0].key} row is served publicly by /f/:key` };++ const publicVod = refs.vods.find((r) => rowVisibility(r) === 'public');++ if (publicVod) return { ok: false, id, obj, reason: `its vods#${publicVod.id} row is public` };++ const publicClip = refs.clips.find((r) => rowVisibility(r) === 'public');++ if (publicClip) return { ok: false, id, obj, reason: `its clips#${publicClip.id} row is public` };++++ return {++ ok: true, id, obj, size,++ rows: [++ ...refs.vods.map((r) => ({ table: 'vods', id: r.id })),++ ...refs.clips.map((r) => ({ table: 'clips', id: r.id })),++ ],++ locations: await model.listLocations(obj.id),++ };++}++++/** What a dry run (or the apply itself) does, in operator-readable lines. */++function describe(plan) {++ const o = plan.obj;++ const out = [`${o.id} ${o.kind} ${humanBytes(plan.size)} ${o.visibility} ${o.legacy_ref || '(native object)'}`];++ out.push(` action: soft delete (bytes kept, ${require('../config').objects.retentionDays}-day retention; restorable; this script deletes no bytes)`);++ out.push(' event: media.object.deleted (Live and Search forget the item)');++ if (plan.rows.length) out.push(` hide: ${plan.rows.map((r) => `${r.table}#${r.id} (visibility private, is_public 0)`).join(', ')}`);++ if (!plan.locations.length) out.push(' report: no good copy (no locations recorded)');++ else out.push(` report: no good copy (${plan.locations.map((l) => `${l.provider} ${l.state}`).join(', ')})`);++ return out.join('\n');++}++++/**++ * Retire the assessed plans in one transaction: hide their v1 rows, then soft-delete each object through++ * model.softDelete (with the event and retention metadata). Returns the deleted objects' key facts.++ */++async function retire(plans) {++ const d = db.getDb();++ await d.tx(async () => {++ for (const plan of plans) {++ for (const row of plan.rows) {++ if (row.table === 'vods') await db.setVodVisibility(row.id, 'private');++ else if (row.table === 'clips') await db.setClipVisibility(row.id, 'private');++ }++ await model.softDelete(plan.obj, { by: BY });++ }++ });++ return await Promise.all(plans.map(async (plan) => {++ const obj = await model.getObject(plan.obj.id);++ const md = model.parseJson(obj.metadata, {});++ return { id: obj.id, lifecycle_status: obj.lifecycle_status, deleted_at: obj.deleted_at || null, retention_until: md.retention_until || null };++ }));++}++++module.exports = { MAX_BYTES, BY, MED_ID, assess, describe, retire, rowVisibility, humanBytes };nnew file mode 100644@@@ -0,0 +1,114 @@++'use strict';++// Operator retirement of a no-good-copy object (scripts/no-good-copy-report.js --retire): the dry run prints and++// changes nothing; a stub is soft-deleted (retention metadata, restorable, media.object.deleted staged) with its v1++// VOD row hidden, and the readiness count (the same count /api/ready's object_copies check reads) goes to 0; public++// objects, objects with a good copy, objects of 1 MiB or more and held objects are refused with the reason.++const assert = require('assert');++const fs = require('fs');++const os = require('os');++const path = require('path');++++(async () => {++ const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-media-retire-'));++ const dir = (n) => { const d = path.join(tmp, n); fs.mkdirSync(d, { recursive: true }); return d; };++ const env = {++ VOD_PATH: dir('vods'), CLIPS_PATH: dir('clips'), FILES_PATH: dir('files'),++ THUMBNAILS_PATH: dir('thumbnails'), PASTES_PATH: dir('pastes'), OBJECTS_PATH: dir('objects'), ASSETS_PATH: dir('assets'),++ MEDIA_PUBLIC_URL: 'https://media.test', OV_NETWORK_URL: 'http://127.0.0.1:9',++ };++ for (const k of ['MEDIA_B2_ENDPOINT', 'MEDIA_B2_BUCKET', 'MEDIA_R2_ENDPOINT', 'MEDIA_R2_BUCKET', 'EVENTS_URL', 'MEDIA_VERIFY_ENABLED']) process.env[k] = '';++ Object.assign(process.env, env);++++ const db = require('../server/db/database');++ const model = require('../server/objects/model');++ const copyReport = require('../server/objects/copy-report');++ const verifyJob = require('../server/objects/verify-job');++ const vodStorage = require('../server/vod/vod-storage');++ const { runScript } = require('./helpers/run-script');++ await vodStorage.tierConfig.init(vodStorage.DEFAULTS);++ await require('../server/objects/tier-policy').init();++ const d = db.getDb();++ const MB = 1024 * 1024;++++ let nextVod = 6027;++ const insertVod = d.prepare(`INSERT INTO vods (id, app_id, user_id, title, file_path, file_size, duration_seconds, visibility, is_public, storage_provider, clips_only, is_recording, created_at)++ OVERRIDING SYSTEM VALUE VALUES (?, 'live', 5, ?, ?, ?, 1, ?, ?, 'local', 0, 0, '2026-11-01 00:00:00') RETURNING id`);++ /** A VOD row projected onto its object; the file exists only when `bytes` is written. */++ async function vod({ title = 'V', size = 8855, visibility = null, isPublic = 0, bytes = null }) {++ const id = nextVod++;++ const p = path.join(env.VOD_PATH, `vod-${id}.webm`);++ if (bytes != null) { fs.mkdirSync(path.dirname(p), { recursive: true }); fs.writeFileSync(p, Buffer.alloc(bytes, 7)); }++ await insertVod.run(id, title, p, size, visibility, isPublic);++ await model.syncVod(id);++ const objectId = (await d.prepare('SELECT object_id FROM vods WHERE id = ?').get(id)).object_id;++ return { id, objectId, obj: await model.getObject(objectId) };++ }++++ const script = path.join(__dirname, '..', 'scripts', 'no-good-copy-report.js');++ const run = async (a) => await runScript(script, a, { env });++ const state = async (v) => { const o = await model.getObject(v.objectId); return [o.lifecycle_status, o.visibility, o.deleted_at || null]; };++++ // ── The stub (the report's shape): a 1-second, 8.6 KB recording whose only copy is gone, never public ──++ const stub = await vod({ title: 'recording stub' });++ assert.strictEqual(stub.obj.visibility, 'private');++ assert.strictEqual(await copyReport.countNoGoodCopy(db), 1);++ assert.strictEqual((await verifyJob.status()).no_good_copy, 1);++++ // ── Dry run ──++ const dry = await run(['--retire', stub.objectId]);++ assert.strictEqual(dry.code, 0, dry.err);++ assert.match(dry.out, /soft delete/);++ assert.match(dry.out, /vods#6027 \(visibility private, is_public 0\)/);++ assert.match(dry.out, /dry run: nothing was changed/);++ assert.deepStrictEqual(await state(stub), ['ready', 'private', null]);++ console.log('✅ the dry run prints what it would do and writes nothing');++++ // ── Apply: the stub is soft-deleted, its v1 row hidden, the readiness count is 0 ──++ const applied = await run(['--retire', stub.objectId, '--apply']);++ assert.strictEqual(applied.code, 0, applied.err);++ const gone = await model.getObject(stub.objectId);++ assert.strictEqual(gone.lifecycle_status, 'deleted');++ assert.ok(gone.deleted_at, 'deleted_at is set');++ const md = model.parseJson(gone.metadata, {});++ assert.ok(Date.parse(md.retention_until) > Date.now(), `retention_until is in the future (${md.retention_until})`);++ const row = await d.prepare('SELECT visibility, is_public FROM vods WHERE id = 6027').get();++ assert.deepStrictEqual([row.visibility, Number(row.is_public)], ['private', 0], 'the v1 VOD row is hidden');++ const change = await d.prepare("SELECT change FROM media_object_changes WHERE object_id = ? ORDER BY id DESC LIMIT 1").get(stub.objectId);++ assert.strictEqual(change && change.change, 'deleted', 'media.object.deleted is staged for the object');++ assert.strictEqual(await copyReport.countNoGoodCopy(db), 0, 'the readiness object_copies count is 0');++ assert.strictEqual((await verifyJob.status()).no_good_copy, 0);++ console.log('✅ retires the stub: soft delete, retention, hidden v1 row, media.object.deleted, readiness count 0');++++ // ── Refusals: nothing is changed, the reason says why ──++ const good = await vod({ title: 'good', size: 100, bytes: 100 });++ const publicVod = await vod({ title: 'public', visibility: 'public', isPublic: 1 });++ const big = await vod({ title: 'big', size: MB });++ const held = await vod({ title: 'held' });++ await model.placeHold({ object_id: held.objectId, kind: 'admin', reason: 'test hold', created_by: 'test' });++ assert.strictEqual(await copyReport.countNoGoodCopy(db), 3, 'public, big and held; not the one with a good copy');++ for (const [v, reason] of [++ [publicVod, /it is public/],++ [good, /not in the no-good-copy report/],++ [big, /not under 1 MiB/],++ [held, /retention hold/],++ ]) {++ const before = await state(v);++ const r = await run(['--retire', v.objectId]);++ assert.strictEqual(r.code, 1, `${v.objectId} refused ${r.err || ''}`);++ assert.match(r.out, reason);++ assert.match(r.out, /Nothing was changed/);++ assert.deepStrictEqual(await state(v), before, 'a refused retirement writes nothing');++ }++ console.log('✅ refuses a public object, one with a good copy, one of 1 MiB and one under a hold, with the reason');++++ // ── Restorable ──++ const back = await model.restore(await model.getObject(stub.objectId));++ assert.strictEqual(back.lifecycle_status, 'ready', 'a restore inside retention brings it back');++ assert.strictEqual(await copyReport.countNoGoodCopy(db), 4, 'and it is a no-good-copy object again');++ console.log('✅ restorable inside the retention period');++++ await db.close();++ fs.rmSync(tmp, { recursive: true, force: true });++ console.log('retire-no-good-copy: all checks passed');++ process.exit(0);++})().catch((err) => { console.error(err); process.exit(1); });
@@ -2,6 +2,12 @@
@@ -2,6 +2,12 @@
## Unreleased
## Unreleased
+- Operator retirement for an object with no good copy (`scripts/no-good-copy-report.js --retire <med_id> [--apply]`):
+ the dry run changes nothing; only an object in the report, not public and under 1 MiB is retired (public, good-copy,
+ 1 MiB+ and held objects are refused with the reason). The delete is the ordinary soft delete (bytes kept,
+ retention, `media.object.deleted`, restorable), and a projected VOD/clip has its v1 row made private so the inherited
+ route stops serving it. Test: `test/retire-no-good-copy.test.js`.
+
- A legacy Live paste screenshot can be retired for OpenVibe.Community (2026-11-06). The internal
- A legacy Live paste screenshot can be retired for OpenVibe.Community (2026-11-06). The internal
`POST /internal/community/legacy-paste-screenshots/:id/retire` route is loopback and service-token only
`POST /internal/community/legacy-paste-screenshots/:id/retire` route is loopback and service-token only
(`media.object.delete`, principal `svc:community`) and soft-deletes exactly a `legacy:live:paste:<slug>`
(`media.object.delete`, principal `svc:community`) and soft-deletes exactly a `legacy:live:paste:<slug>`
@@ -610,6 +610,14 @@ that only moved to B2/R2 keeps it. Test: `test/content-hash.test.js`.
@@ -610,6 +610,14 @@ that only moved to B2/R2 keeps it. Test: `test/content-hash.test.js`.
For each object it prints the owner (app, `owner_user_id`, `owner_subject`), `legacy_ref`, the `vods`/`clips`/`files` rows that point at it, its relationships, variants and holds, every location with its state and `verified_at`, and the last verification. The report opens the database read-only. It exits 1 when there are any.
For each object it prints the owner (app, `owner_user_id`, `owner_subject`), `legacy_ref`, the `vods`/`clips`/`files` rows that point at it, its relationships, variants and holds, every location with its state and `verified_at`, and the last verification. The report opens the database read-only. It exits 1 when there are any.
+ Retire one of those objects instead of only listing it:
+
+ ```
+ node scripts/no-good-copy-report.js --retire <med_id> [--retire <med_id> ...] [--apply]
+ ```
+
+ The dry run (no `--apply`) prints what it would do and writes nothing. Only an object that is in the report (no present or pending copy), is not public and is under 1 MiB is retired; anything else — a public object, one with a good copy, one of 1 MiB or more, one under a retention hold — is refused with the reason and nothing is changed. The delete is the ordinary soft delete (`model.softDelete`): the bytes stay, `metadata.retention_until` is set, the same `media.object.deleted` event is queued as for any other delete, and `POST /:id/restore` brings the object back inside the retention period. A projected VOD or clip also has its v1 row made private (`visibility private`, `is_public 0`) so `/v/:id` and `/c/:id` stop serving it; the row is kept, not removed, so the retirement stays reversible. A file object is refused because `/f/:key` serves every file without auth. Test: `test/retire-no-good-copy.test.js`.
+
## Public object-size invariant
## Public object-size invariant
This is roadmap W4 deliverable 5, stated as **policy** rather than a copied constant:
This is roadmap W4 deliverable 5, stated as **policy** rather than a copied constant:
@@ -8,19 +8,66 @@
@@ -8,19 +8,66 @@
*
*
* node scripts/no-good-copy-report.js [--app live] [--json] [--out report.json]
* node scripts/no-good-copy-report.js [--app live] [--json] [--out report.json]
*
*
- * Exit code 0 when there are none, 1 when some were found, 2 on error.
+ * Retire one or more of those objects instead (scripts/no-good-copy-report.js --retire):
+ *
+ * node scripts/no-good-copy-report.js --retire <med_id> [--retire <med_id> ...] [--apply]
+ *
+ * The dry run (no --apply) prints what it would do and changes nothing. Only an object that is in the
+ * report, not public and under 1 MiB is retired, through the ordinary soft delete (bytes kept for the
+ * retention period, media.object.deleted queued, restorable) with its v1 VOD/clip row hidden. Anything
+ * else is refused with the reason and nothing is changed.
+ *
+ * Exit code: 0 when the report found none (or every named object can be / was retired), 1 when the
+ * report found some (or one named object was refused), 2 on error.
*/
*/
const fs = require('fs');
const fs = require('fs');
-const path = require('path');
(async () => {
(async () => {
await require('../server/db/database').initDb(); // PostgreSQL (DATABASE_URL), as the service
await require('../server/db/database').initDb(); // PostgreSQL (DATABASE_URL), as the service
const args = process.argv.slice(2);
const args = process.argv.slice(2);
const has = (name) => args.includes(`--${name}`);
const has = (name) => args.includes(`--${name}`);
const arg = (name, dflt) => { const i = args.indexOf(`--${name}`); return i >= 0 ? args[i + 1] : dflt; };
const arg = (name, dflt) => { const i = args.indexOf(`--${name}`); return i >= 0 ? args[i + 1] : dflt; };
+ const retireIds = [];
+ for (let i = 0; i < args.length; i++) {
+ if (args[i] !== '--retire') continue;
+ const value = args[i + 1];
+ if (!value || value.startsWith('--')) { retireIds.push(''); continue; }
+ retireIds.push(...String(value).split(','));
+ }
try {
try {
- // Read-only: the report runs in a READ ONLY transaction (PostgreSQL refuses any write inside it).
+ // ── Retire named objects (the one write path of this script) ──
+ if (retireIds.length) {
+ const { assess, describe, retire } = require('../server/objects/retire');
+ const ids = [...new Set(retireIds.map((s) => s.trim()).filter(Boolean))];
+ if (!ids.length) {
+ console.error('--retire needs one or more med_… object ids');
+ process.exit(2);
+ }
+ const plans = [];
+ for (const id of ids) plans.push(await assess(id));
+ const refused = plans.filter((p) => !p.ok);
+ if (refused.length) {
+ for (const p of refused) console.log(`${p.id} — refused: ${p.reason}`);
+ console.log('Nothing was changed.');
+ process.exit(1);
+ }
+ if (!has('apply')) {
+ for (const p of plans) console.log(`${describe(p)}\n dry run: nothing was changed; re-run with --apply to retire it\n`);
+ process.exit(0);
+ }
+ const done = await retire(plans);
+ for (const d of done) {
+ console.log(`${d.id} retired: ${d.lifecycle_status}; retention until ${d.retention_until || `the end of the ${require('../server/config').objects.retentionDays}-day period`}; restorable`);
+ }
+ process.exit(0);
+ }
+ if (has('apply')) {
+ console.error('--apply only goes with --retire <med_id>');
+ process.exit(2);
+ }
+
+ // ── Report (read-only: the report runs in a READ ONLY transaction, PostgreSQL refuses any write) ──
const { buildReport, formatReport } = require('../server/objects/copy-report');
const { buildReport, formatReport } = require('../server/objects/copy-report');
const d = require('../server/db/database').getDb();
const d = require('../server/db/database').getDb();
const q = {
const q = {
@@ -105,4 +105,4 @@ function formatReport(report) {
@@ -105,4 +105,4 @@ function formatReport(report) {
return out.join('\n');
return out.join('\n');
}
}
-module.exports = { NO_GOOD_COPY_WHERE, countNoGoodCopy, listNoGoodCopy, buildReport, formatReport };
+module.exports = { NO_GOOD_COPY_WHERE, countNoGoodCopy, listNoGoodCopy, references, buildReport, formatReport };
new file mode 100644
new file mode 100644
@@ -0,0 +1,121 @@
@@ -0,0 +1,121 @@
+'use strict';
+/**
+ * Operator retirement of a no-good-copy object (scripts/no-good-copy-report.js --retire).
+ *
+ * A ready object with no good copy cannot be repaired when every copy it has is missing or corrupt. The report can
+ * only list it; this is the one operator path that retires it, and only the safe shape (docs/object-model.md,
+ * "scheduled verification"):
+ *
+ * - it is in the no-good-copy report (listNoGoodCopy: ready, not virtual, no present/pending copy);
+ * - it is not public (a public object, or a public v1 row, is refused);
+ * - it is under 1 MiB: a stub or a broken small upload, never a real recording;
+ * - no retention hold applies.
+ *
+ * The delete is the ordinary soft delete (model.softDelete), the same one Media's own delete uses: lifecycle_status
+ * becomes deleted, the bytes stay for MEDIA_DELETE_RETENTION_DAYS (metadata.retention_until), the media_objects
+ * trigger queues the same media.object.deleted event as any other delete (so Live and Search forget the item), and
+ * POST /:id/restore brings it back inside the retention period.
+ *
+ * A projected VOD or clip is also hidden from its inherited route the way the owning app's delete hides it: its v1
+ * row is made private (visibility = private, is_public = 0) through db.setVodVisibility / setClipVisibility, so the
+ * row and the object agree. The row is kept, not removed, which is what keeps the retirement reversible; a VOD that
+ * is public is refused rather than hidden. A file row keeps being served by /f/:key whatever its object says, so a
+ * file object is refused too.
+ */
+const db = require('../db/database');
+const model = require('./model');
+const copyReport = require('./copy-report');
+
+const MAX_BYTES = 1024 * 1024; // strictly under 1 MiB
+const BY = 'operator:no-good-copy-report';
+const MED_ID = /^med_[0-9A-HJKMNP-TV-Z]{26}$/;
+
+function query() {
+ const d = db.getDb();
+ return {
+ all: async (sql, params = []) => await d.prepare(sql).all(...params),
+ get: async (sql, params = []) => await d.prepare(sql).get(...params),
+ };
+}
+
+/** The v1 route's own visibility rule (vods/clips row → public, unlisted or private). */
+function rowVisibility(row) {
+ return row.visibility || (row.is_public ? 'public' : 'private');
+}
+
+function humanBytes(n) {
+ n = Number(n) || 0;
+ if (n < 1024) return `${n} B`;
+ if (n < 1048576) return `${(n / 1024).toFixed(1)} KB`;
+ return `${(n / 1048576).toFixed(1)} MB`;
+}
+
+/**
+ * Decide whether an object may be retired. Reads only. Returns
+ * { ok: true, id, obj, size, rows, locations } or { ok: false, id, obj?, reason }.
+ */
+async function assess(id, { q = null } = {}) {
+ if (!MED_ID.test(String(id || ''))) return { ok: false, id, reason: 'not a med_… object id' };
+ const obj = await model.getObject(id);
+ if (!obj) return { ok: false, id, reason: 'no such object' };
+ if (obj.lifecycle_status === 'deleted') return { ok: false, id, obj, reason: 'it is already deleted' };
+ const reads = q || query();
+ const inReport = (await copyReport.listNoGoodCopy(reads, { appId: obj.app_id })).some((o) => o.id === obj.id);
+ if (!inReport) return { ok: false, id, obj, reason: 'not in the no-good-copy report (it has a good copy, is virtual, or is not ready)' };
+ if (obj.visibility === 'public') return { ok: false, id, obj, reason: 'it is public' };
+ const size = Number(obj.size_bytes) || 0;
+ if (size >= MAX_BYTES) return { ok: false, id, obj, reason: `${humanBytes(size)} is not under 1 MiB` };
+ if (await model.isHeld(obj.id)) return { ok: false, id, obj, reason: 'it is under a retention hold' };
+
+ const refs = await copyReport.references(reads, obj);
+ if (refs.files.length) return { ok: false, id, obj, reason: `its files/${refs.files[0].key} row is served publicly by /f/:key` };
+ const publicVod = refs.vods.find((r) => rowVisibility(r) === 'public');
+ if (publicVod) return { ok: false, id, obj, reason: `its vods#${publicVod.id} row is public` };
+ const publicClip = refs.clips.find((r) => rowVisibility(r) === 'public');
+ if (publicClip) return { ok: false, id, obj, reason: `its clips#${publicClip.id} row is public` };
+
+ return {
+ ok: true, id, obj, size,
+ rows: [
+ ...refs.vods.map((r) => ({ table: 'vods', id: r.id })),
+ ...refs.clips.map((r) => ({ table: 'clips', id: r.id })),
+ ],
+ locations: await model.listLocations(obj.id),
+ };
+}
+
+/** What a dry run (or the apply itself) does, in operator-readable lines. */
+function describe(plan) {
+ const o = plan.obj;
+ const out = [`${o.id} ${o.kind} ${humanBytes(plan.size)} ${o.visibility} ${o.legacy_ref || '(native object)'}`];
+ out.push(` action: soft delete (bytes kept, ${require('../config').objects.retentionDays}-day retention; restorable; this script deletes no bytes)`);
+ out.push(' event: media.object.deleted (Live and Search forget the item)');
+ if (plan.rows.length) out.push(` hide: ${plan.rows.map((r) => `${r.table}#${r.id} (visibility private, is_public 0)`).join(', ')}`);
+ if (!plan.locations.length) out.push(' report: no good copy (no locations recorded)');
+ else out.push(` report: no good copy (${plan.locations.map((l) => `${l.provider} ${l.state}`).join(', ')})`);
+ return out.join('\n');
+}
+
+/**
+ * Retire the assessed plans in one transaction: hide their v1 rows, then soft-delete each object through
+ * model.softDelete (with the event and retention metadata). Returns the deleted objects' key facts.
+ */
+async function retire(plans) {
+ const d = db.getDb();
+ await d.tx(async () => {
+ for (const plan of plans) {
+ for (const row of plan.rows) {
+ if (row.table === 'vods') await db.setVodVisibility(row.id, 'private');
+ else if (row.table === 'clips') await db.setClipVisibility(row.id, 'private');
+ }
+ await model.softDelete(plan.obj, { by: BY });
+ }
+ });
+ return await Promise.all(plans.map(async (plan) => {
+ const obj = await model.getObject(plan.obj.id);
+ const md = model.parseJson(obj.metadata, {});
+ return { id: obj.id, lifecycle_status: obj.lifecycle_status, deleted_at: obj.deleted_at || null, retention_until: md.retention_until || null };
+ }));
+}
+
+module.exports = { MAX_BYTES, BY, MED_ID, assess, describe, retire, rowVisibility, humanBytes };
new file mode 100644
new file mode 100644
@@ -0,0 +1,114 @@
@@ -0,0 +1,114 @@
+'use strict';
+// Operator retirement of a no-good-copy object (scripts/no-good-copy-report.js --retire): the dry run prints and
+// changes nothing; a stub is soft-deleted (retention metadata, restorable, media.object.deleted staged) with its v1
+// VOD row hidden, and the readiness count (the same count /api/ready's object_copies check reads) goes to 0; public
+// objects, objects with a good copy, objects of 1 MiB or more and held objects are refused with the reason.
+const assert = require('assert');
+const fs = require('fs');
+const os = require('os');
+const path = require('path');
+
+(async () => {
+ const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-media-retire-'));
+ const dir = (n) => { const d = path.join(tmp, n); fs.mkdirSync(d, { recursive: true }); return d; };
+ const env = {
+ VOD_PATH: dir('vods'), CLIPS_PATH: dir('clips'), FILES_PATH: dir('files'),
+ THUMBNAILS_PATH: dir('thumbnails'), PASTES_PATH: dir('pastes'), OBJECTS_PATH: dir('objects'), ASSETS_PATH: dir('assets'),
+ MEDIA_PUBLIC_URL: 'https://media.test', OV_NETWORK_URL: 'http://127.0.0.1:9',
+ };
+ for (const k of ['MEDIA_B2_ENDPOINT', 'MEDIA_B2_BUCKET', 'MEDIA_R2_ENDPOINT', 'MEDIA_R2_BUCKET', 'EVENTS_URL', 'MEDIA_VERIFY_ENABLED']) process.env[k] = '';
+ Object.assign(process.env, env);
+
+ const db = require('../server/db/database');
+ const model = require('../server/objects/model');
+ const copyReport = require('../server/objects/copy-report');
+ const verifyJob = require('../server/objects/verify-job');
+ const vodStorage = require('../server/vod/vod-storage');
+ const { runScript } = require('./helpers/run-script');
+ await vodStorage.tierConfig.init(vodStorage.DEFAULTS);
+ await require('../server/objects/tier-policy').init();
+ const d = db.getDb();
+ const MB = 1024 * 1024;
+
+ let nextVod = 6027;
+ const insertVod = d.prepare(`INSERT INTO vods (id, app_id, user_id, title, file_path, file_size, duration_seconds, visibility, is_public, storage_provider, clips_only, is_recording, created_at)
+ OVERRIDING SYSTEM VALUE VALUES (?, 'live', 5, ?, ?, ?, 1, ?, ?, 'local', 0, 0, '2026-11-01 00:00:00') RETURNING id`);
+ /** A VOD row projected onto its object; the file exists only when `bytes` is written. */
+ async function vod({ title = 'V', size = 8855, visibility = null, isPublic = 0, bytes = null }) {
+ const id = nextVod++;
+ const p = path.join(env.VOD_PATH, `vod-${id}.webm`);
+ if (bytes != null) { fs.mkdirSync(path.dirname(p), { recursive: true }); fs.writeFileSync(p, Buffer.alloc(bytes, 7)); }
+ await insertVod.run(id, title, p, size, visibility, isPublic);
+ await model.syncVod(id);
+ const objectId = (await d.prepare('SELECT object_id FROM vods WHERE id = ?').get(id)).object_id;
+ return { id, objectId, obj: await model.getObject(objectId) };
+ }
+
+ const script = path.join(__dirname, '..', 'scripts', 'no-good-copy-report.js');
+ const run = async (a) => await runScript(script, a, { env });
+ const state = async (v) => { const o = await model.getObject(v.objectId); return [o.lifecycle_status, o.visibility, o.deleted_at || null]; };
+
+ // ── The stub (the report's shape): a 1-second, 8.6 KB recording whose only copy is gone, never public ──
+ const stub = await vod({ title: 'recording stub' });
+ assert.strictEqual(stub.obj.visibility, 'private');
+ assert.strictEqual(await copyReport.countNoGoodCopy(db), 1);
+ assert.strictEqual((await verifyJob.status()).no_good_copy, 1);
+
+ // ── Dry run ──
+ const dry = await run(['--retire', stub.objectId]);
+ assert.strictEqual(dry.code, 0, dry.err);
+ assert.match(dry.out, /soft delete/);
+ assert.match(dry.out, /vods#6027 \(visibility private, is_public 0\)/);
+ assert.match(dry.out, /dry run: nothing was changed/);
+ assert.deepStrictEqual(await state(stub), ['ready', 'private', null]);
+ console.log('✅ the dry run prints what it would do and writes nothing');
+
+ // ── Apply: the stub is soft-deleted, its v1 row hidden, the readiness count is 0 ──
+ const applied = await run(['--retire', stub.objectId, '--apply']);
+ assert.strictEqual(applied.code, 0, applied.err);
+ const gone = await model.getObject(stub.objectId);
+ assert.strictEqual(gone.lifecycle_status, 'deleted');
+ assert.ok(gone.deleted_at, 'deleted_at is set');
+ const md = model.parseJson(gone.metadata, {});
+ assert.ok(Date.parse(md.retention_until) > Date.now(), `retention_until is in the future (${md.retention_until})`);
+ const row = await d.prepare('SELECT visibility, is_public FROM vods WHERE id = 6027').get();
+ assert.deepStrictEqual([row.visibility, Number(row.is_public)], ['private', 0], 'the v1 VOD row is hidden');
+ const change = await d.prepare("SELECT change FROM media_object_changes WHERE object_id = ? ORDER BY id DESC LIMIT 1").get(stub.objectId);
+ assert.strictEqual(change && change.change, 'deleted', 'media.object.deleted is staged for the object');
+ assert.strictEqual(await copyReport.countNoGoodCopy(db), 0, 'the readiness object_copies count is 0');
+ assert.strictEqual((await verifyJob.status()).no_good_copy, 0);
+ console.log('✅ retires the stub: soft delete, retention, hidden v1 row, media.object.deleted, readiness count 0');
+
+ // ── Refusals: nothing is changed, the reason says why ──
+ const good = await vod({ title: 'good', size: 100, bytes: 100 });
+ const publicVod = await vod({ title: 'public', visibility: 'public', isPublic: 1 });
+ const big = await vod({ title: 'big', size: MB });
+ const held = await vod({ title: 'held' });
+ await model.placeHold({ object_id: held.objectId, kind: 'admin', reason: 'test hold', created_by: 'test' });
+ assert.strictEqual(await copyReport.countNoGoodCopy(db), 3, 'public, big and held; not the one with a good copy');
+ for (const [v, reason] of [
+ [publicVod, /it is public/],
+ [good, /not in the no-good-copy report/],
+ [big, /not under 1 MiB/],
+ [held, /retention hold/],
+ ]) {
+ const before = await state(v);
+ const r = await run(['--retire', v.objectId]);
+ assert.strictEqual(r.code, 1, `${v.objectId} refused ${r.err || ''}`);
+ assert.match(r.out, reason);
+ assert.match(r.out, /Nothing was changed/);
+ assert.deepStrictEqual(await state(v), before, 'a refused retirement writes nothing');
+ }
+ console.log('✅ refuses a public object, one with a good copy, one of 1 MiB and one under a hold, with the reason');
+
+ // ── Restorable ──
+ const back = await model.restore(await model.getObject(stub.objectId));
+ assert.strictEqual(back.lifecycle_status, 'ready', 'a restore inside retention brings it back');
+ assert.strictEqual(await copyReport.countNoGoodCopy(db), 4, 'and it is a no-good-copy object again');
+ console.log('✅ restorable inside the retention period');
+
+ await db.close();
+ fs.rmSync(tmp, { recursive: true, force: true });
+ console.log('retire-no-good-copy: all checks passed');
+ process.exit(0);
+})().catch((err) => { console.error(err); process.exit(1); });