openvibers/openvibe.events Commits

A claim stops at the first pending delivery instead of sorting them all (#216)

OP OpenVibers committed 11 days ago · d0f6711e0b · 2 files, +30 −0

The claim reads a subscription's pending deliveries in (priority, seq) order and takes the first
claimable one; with no index in that order it read and sorted every pending row. With 74,000
pending for Search (a product re-sending its documents) each claim took 265 ms, one PostgreSQL
core sat at 100% and Search received ~190 deliveries a minute. Migration 0011 adds
idx_deliveries_claim (subscription_id, priority, seq) WHERE pending or failed: 0.4 ms. It was
created CONCURRENTLY on production the same day; IF NOT EXISTS makes the migration a no-op there.

Changes

ddiff --git a/migrations/0011_claim_index.sql b/migrations/0011_claim_index.sql
nnew file mode 100644
iindex 0000000..2b1e760
---- /dev/null
++++ b/migrations/0011_claim_index.sql
@@@ -0,0 +1,7 @@
++-- phase: expand
++-- The claim walks a subscription's pending deliveries in (priority, seq) order and takes the first claimable one
++-- (store.js claimDeliveries / next). With only (status, next_attempt_at) and (subscription_id, status, seq) to use, it
++-- read every pending row of the subscription and sorted them: 265 ms a claim with 74,000 pending for Search on
++-- 2026-11-02 (one PostgreSQL core at 100%, ~190 deliveries a minute). This index lets the claim stop at the first
++-- row: 0.4 ms. Created CONCURRENTLY on production the same day; IF NOT EXISTS makes this a no-op there.
++CREATE INDEX IF NOT EXISTS idx_deliveries_claim ON deliveries (subscription_id, priority, seq) WHERE status IN ('pending', 'failed');
ddiff --git a/test/claim-index.test.js b/test/claim-index.test.js
nnew file mode 100644
iindex 0000000..7fa551d
---- /dev/null
++++ b/test/claim-index.test.js
@@@ -0,0 +1,23 @@
++'use strict';
++/**
++ * The claim walks a subscription's pending deliveries in (priority, seq) order and stops at the first claimable one
++ * (migration 0011, idx_deliveries_claim). Without it, a claim read and sorted every pending row of the subscription:
++ * 265 ms each with 74,000 pending for Search on 2026-11-02.
++ *
++ *   node test/claim-index.test.js
++ */
++const assert = require('assert');
++const { testDb } = require('./db');
++
++(async () => {
++    const d = await testDb();
++    try {
++        const idx = await d.db.prepare("SELECT indexdef FROM pg_indexes WHERE tablename = 'deliveries' AND indexname = 'idx_deliveries_claim'").get();
++        assert.ok(idx, 'the claim index exists after the migrations');
++        assert.match(idx.indexdef, /\(subscription_id, priority, seq\)/);
++        assert.match(idx.indexdef, /WHERE \(status = ANY \(ARRAY\['pending'::text, 'failed'::text\]\)\)/);
++    } finally {
++        await d.close();
++    }
++    console.log('claim index: all checks passed');
++})().catch((err) => { console.error(err); process.exit(1); });