openvibers/openvibe.food Commits

A food's page shows USDA ERS's annual fruit and vegetable price for 16 items beside BLS (#211)

OP OpenVibers committed 8 days ago · 88995b2f6e · 8 files, +327 −4

* A food's page shows USDA ERS's annual fruit and vegetable price for 16 items beside BLS

Read from OpenVibe.Sources (source usda-ers-fruit-vegetable-prices, category food) on its own change-feed cursor into food_usda_prices (expand 0012), never from ERS. ERS's national average is shown with its 2023 data year and its warning that the annual data must not be read as price changes over time; five of the 16 also have a BLS price and both figures show. The list's own 2025 band stays as it is.

* The USDA price note in plain words: one year's national average, not a trend

The note quoted ERS's terms at the shopper, Circana included. It now says whose figure it is and that it is a
snapshot of one year; the page never names Circana, which is how ERS's attribution term is met (tested).

Changes

ddiff --git a/README.md b/README.md
iindex ce41f4e..16fd0c6 100644
---- a/README.md
++++ b/README.md
@@@ -167,6 +167,15 @@ published average-price file once a day (source `bls-average-prices`) and Food f
`food_official_prices`. BLS is cited with its note that it cannot vouch for data once they leave BLS.gov. The list's
own band stays as it is.
++**USDA prices** — the U.S. Department of Agriculture's Economic Research Service publishes annual national-average
++retail prices for 155 fresh and processed fruits and vegetables (Fruit and Vegetable Prices, 2023 retail scanner data,
++public domain); 16 of Food's items are among them, and an item's page shows ERS's figure for the food's pack beside
++BLS's where both price it ([server/food/usda-prices.js](server/food/usda-prices.js)). Food never calls ERS:
++OpenVibe.Sources reads the two published CSV tables once a week (source `usda-ers-fruit-vegetable-prices`) and Food
++follows its change feed into `food_usda_prices`. The page gives the data year (2023) and repeats ERS's warning that the
++data should not be used for making inferences about price changes over time; the annual figure is never shown as a
++month's price or a trend.
++
**Open Food Facts labels** — Open Food Facts' daily product export (ODbL), read by OpenVibe.Sources as a streamed
bulk dataset (source `open-food-facts`), is matched by product name to the 42 foods on the list. A food's page shows
the most complete matching product's per-100 g figures, its Nutri-Score and a link to the product, in a separate
ddiff --git a/migrations/0012_usda_prices.sql b/migrations/0012_usda_prices.sql
nnew file mode 100644
iindex 0000000..b8054dd
---- /dev/null
++++ b/migrations/0012_usda_prices.sql
@@@ -0,0 +1,14 @@
++-- phase: expand
++-- OpenVibe.Food's USDA ERS prices (server/food/usda-prices.js): the Economic Research Service's annual national-average
++-- retail prices for fruits and vegetables, as OpenVibe.Sources collects them (source usda-ers-fruit-vegetable-prices,
++-- category food). Food never calls ERS: it follows Sources' change feed from its own cursor in food_ingest_cursor and
++-- keeps each row as ERS publishes it. US federal data (public domain in the US); nothing here belongs to a person.
++CREATE TABLE food_usda_prices (
++ identity text COLLATE "C" PRIMARY KEY, -- "Apples | Fresh": Sources' identity (ERS product and form)
++ item_id text COLLATE "C" NOT NULL UNIQUE, -- the Sources item
++ revision integer NOT NULL, -- the Sources item's revision this row holds
++ value numeric(10, 4) NOT NULL CHECK (value > 0), -- dollars per the row's unit, as ERS states it
++ unit text COLLATE "C" NOT NULL, -- lb (every row Food maps is priced per pound)
++ vintage text COLLATE "C" NOT NULL, -- the data year ERS published (2023 today)
++ retrieved_at text COLLATE "C" NOT NULL -- when Sources last read it, ISO 8601 UTC
++);
ddiff --git a/server/app.js b/server/app.js
iindex ac7a46b..942fe6d 100644
---- a/server/app.js
++++ b/server/app.js
@@@ -98,6 +98,8 @@ async function createApp(opts = {}) {
ctx.recalls = require('./food/recalls').createRecalls({ config, s, fetchImpl, log, source: opts.recallSource || null });
// BLS's US average prices for list items, from OpenVibe.Sources (server/food/official-prices.js); a test passes opts.priceSource.
ctx.officialPrices = require('./food/official-prices').createOfficialPrices({ config, s, fetchImpl, log, source: opts.priceSource || null });
++ // USDA ERS's annual fruit and vegetable prices beside BLS (server/food/usda-prices.js); opts.usdaPriceSource.
++ ctx.usdaPrices = require('./food/usda-prices').createUsdaPrices({ config, s, fetchImpl, log, source: opts.usdaPriceSource || null });
// Open Food Facts products as label data beside the list, from OpenVibe.Sources (server/food/off.js); opts.offSource.
ctx.off = require('./food/off').createOff({ config, s, fetchImpl, log, source: opts.offSource || null });
ddiff --git a/server/food/usda-prices.js b/server/food/usda-prices.js
nnew file mode 100644
iindex 0000000..609b3aa
---- /dev/null
++++ b/server/food/usda-prices.js
@@@ -0,0 +1,173 @@
++'use strict';
++
++/**
++ * USDA prices: the Economic Research Service's annual national-average retail prices for fruits and vegetables, read
++ * from OpenVibe.Sources (source usda-ers-fruit-vegetable-prices, category food), never from ERS. Sources reads ERS's
++ * two published all-fruits and all-vegetables tables once a week under its robots and terms checks and keeps one item
++ * per product and form (the identity is "product | form"); Food follows Sources' change feed from its own cursor
++ * (openvibe-publishing/ingest, the same food_ingest_cursor table as the recalls and BLS) and keeps one row per
++ * Sources identity in food_usda_prices.
++ *
++ * The food list's own prices stay what they are, a hand-kept 2025 band to plan by; a food's page shows the USDA figure
++ * beside the BLS one, for ERS's per-pound price and worked out for the item's pack. ERS publishes annual averages, not
++ * a monthly series, and says the data "should not be used for making inferences about price changes over time": the
++ * page says the data year (2023) and NOTE repeats both, so the figure is never read as a trend or as this month's price.
++ *
++ * Pulls a minute after start and then every six hours. Off without OV_SOURCES_INTERNAL_URL and the OAuth client secret
++ * (a test passes its own source).
++ */
++const ingest = require('openvibe-publishing/ingest');
++
++const SOURCE_KEY = 'usda-ers-fruit-vegetable-prices';
++const CURSOR = 'sources.food.usda.after';
++const EVERY_MS = 6 * 60 * 60 * 1000;
++const FIRST_MS = 60 * 1000;
++// ERS's current tables use 2023 retail scanner data; the files were last updated 2025-12-09. ERS updates them each
++// year, subject to data availability: re-check both when the source's page changes (the seed's terms_note says so too).
++const VINTAGE = '2023';
++const URL = 'https://www.ers.usda.gov/data-products/fruit-and-vegetable-prices/';
++// In plain words for a shopper: whose figure it is, that it is one year's national average, and that it is not a
++// trend (ERS: "should not be used for making inferences about price changes over time"). ERS also asks that its
++// findings not be attributed to Circana, whose scanner data it uses: the page never names Circana, which meets that.
++const NOTE = 'A yearly national average from the USDA Economic Research Service (public domain): a snapshot of one year, not a price trend, so it is not compared with other years.';
++
++// Sources identity ("ERS product | form") → the food it prices, how many pounds the food's pack holds (null when the
++// pack states no weight), and the plain words the page uses for the product. Every row mapped here is priced per pound.
++const PRODUCTS = {
++ 'Apples | Fresh': { slug: 'apples-3lb', pack: 3, label: 'apples' },
++ 'Bananas | Fresh': { slug: 'bananas', pack: 1, label: 'bananas' },
++ 'Oranges | Fresh': { slug: 'oranges-3lb', pack: 3, label: 'oranges' },
++ 'Strawberries | Fresh': { slug: 'strawberries', pack: 1, label: 'strawberries' },
++ 'Potatoes | Fresh': { slug: 'potatoes-5lb', pack: 5, label: 'potatoes' },
++ 'Onions | Fresh': { slug: 'onion-3lb', pack: 3, label: 'onions' },
++ 'Carrots, baby | Fresh': { slug: 'baby-carrots', pack: 1, label: 'baby carrots' },
++ 'Avocados | Fresh': { slug: 'avocado', pack: null, label: 'avocados' },
++ 'Grapes (raisins) | Dried': { slug: 'raisins', pack: 0.375, label: 'raisins' },
++ 'Peaches, in syrup or water | Canned': { slug: 'canned-fruit', pack: 15 / 16, label: 'peaches in syrup or water' },
++ 'Pinto beans | Dried': { slug: 'dried-beans', pack: 1, label: 'pinto beans' },
++ 'Black beans | Canned': { slug: 'canned-beans', pack: 15 / 16, label: 'black beans' },
++ 'Corn | Canned': { slug: 'canned-corn', pack: 15 / 16, label: 'corn' },
++ 'Green beans | Canned': { slug: 'canned-green-beans', pack: 14.5 / 16, label: 'green beans' },
++ 'Mixed vegetables, carrots, peas, corn, and green beans | Frozen': { slug: 'frozen-mixed-veg', pack: 12 / 16, label: 'mixed vegetables' },
++ 'Tomatoes | Canned': { slug: 'canned-tomatoes', pack: 14.5 / 16, label: 'tomatoes' },
++};
++
++const text = (v, max) => (v == null || v === '' ? null : String(v).replace(/\s+/g, ' ').trim().slice(0, max) || null);
++
++/** A Sources item → a food_usda_prices row, or null when it is not a row this list maps (or its unit is not a pound). */
++function rowOf(item) {
++ const f = (item && item.fields) || {};
++ const identity = text(item && item.identity, 80);
++ if (!identity || !Object.prototype.hasOwnProperty.call(PRODUCTS, identity)) return null;
++ const value = Number(f.price);
++ // Every mapped food is priced per pound; any other unit would make the pack arithmetic wrong, so hold it.
++ if (!Number.isFinite(value) || value <= 0 || value > 1000 || text(f.unit, 20) !== 'per pound') return null;
++ return {
++ identity,
++ item_id: String(item.id),
++ revision: Number(item.revision) || 1,
++ value: Math.round(value * 10000) / 10000,
++ unit: 'lb',
++ vintage: VINTAGE,
++ retrieved_at: (item.provenance && item.provenance.retrieved_at) || new Date(0).toISOString(),
++ };
++}
++
++/** What a food's page shows for one row: ERS's figure and data year, and the price worked out for the food's pack. */
++function shape(row) {
++ const def = PRODUCTS[row.identity];
++ const value = Number(row.value);
++ return {
++ slug: def.slug,
++ identity: row.identity,
++ item: def.label,
++ form: row.identity.slice(row.identity.indexOf(' | ') + 3).toLowerCase(),
++ unit: 'lb',
++ value,
++ pack_price: def.pack == null ? null : Math.round(value * def.pack * 100) / 100,
++ year: row.vintage,
++ url: URL,
++ };
++}
++
++function createUsdaPrices({ config, s, fetchImpl = globalThis.fetch, log = console, source = null }) {
++ const url = String((config.sources && config.sources.internalUrl) || '').replace(/\/+$/, '');
++ const client = source || (url && config.oauth.clientSecret
++ ? ingest.createSourcesClient({ config: { ...config, sources: { internalUrl: url, category: 'food' } }, fetchImpl })
++ : null);
++ const cursor = ingest.createChangeCursor(s.db, { prefix: 'food', now: s.now });
++ const stats = { pulls: 0, applied: 0, removed: 0, last_at: null, last_error: null };
++ let byFood = new Map(); // food slug → shape(row), refreshed after each pull
++ let timer = null;
++ let first = null;
++ let running = false;
++
++ async function apply(item, t) {
++ if (!item || item.source_key !== SOURCE_KEY || item.kind !== 'price') return 'hold';
++ if (item.removed) {
++ await t.query('DELETE FROM food_usda_prices WHERE item_id = $1', [String(item.id)]);
++ return 'removed';
++ }
++ const row = rowOf(item);
++ if (!row) return 'hold';
++ const cols = Object.keys(row);
++ // A replayed page (or an older revision arriving late) never takes a row back.
++ await t.query(`INSERT INTO food_usda_prices (${cols.join(', ')}) VALUES (${cols.map((_, i) => `$${i + 1}`).join(', ')})
++ ON CONFLICT (identity) DO UPDATE SET ${cols.filter((c) => c !== 'identity').map((c) => `${c} = EXCLUDED.${c}`).join(', ')}
++ WHERE food_usda_prices.item_id <> EXCLUDED.item_id OR food_usda_prices.revision <= EXCLUDED.revision`, cols.map((c) => row[c]));
++ return 'applied';
++ }
++
++ async function load() {
++ const rows = await s.db.many('SELECT identity, value, unit, vintage FROM food_usda_prices');
++ byFood = new Map(rows.filter((r) => PRODUCTS[r.identity]).map((r) => [PRODUCTS[r.identity].slug, shape(r)]));
++ return byFood.size;
++ }
++
++ /** Follow Sources' change feed to its end. Never throws: the caller is a timer or a test. */
++ async function pull() {
++ if (!client || running) return { skipped: true };
++ running = true;
++ try {
++ const out = await ingest.pullChanges({ db: s.db, cursor, source: client, name: CURSOR, apply });
++ stats.pulls++;
++ stats.applied += out.applied;
++ stats.removed += out.removed;
++ stats.last_at = s.iso();
++ stats.last_error = null;
++ await load();
++ if (out.applied || out.removed) log.log(`[Prices] ${out.applied} USDA rows updated, ${out.removed} removed (Sources cursor ${out.after})`);
++ return out;
++ } catch (err) {
++ stats.last_error = text(err && err.message, 300);
++ log.warn(`[Prices] pull of USDA prices from OpenVibe.Sources failed: ${stats.last_error}`);
++ return { error: stats.last_error };
++ } finally { running = false; }
++ }
++
++ return {
++ enabled: Boolean(client),
++ stats,
++ pull,
++ apply,
++ load,
++ /** The USDA figure for a food on the list, or null when ERS prices nothing like it (or nothing is read yet). */
++ forFood(slug) { return byFood.get(slug) || null; },
++ start() {
++ load().catch(() => { /* the first pull loads it */ });
++ if (!client || timer) return;
++ first = setTimeout(() => { pull(); }, FIRST_MS);
++ timer = setInterval(() => { pull(); }, EVERY_MS);
++ if (first.unref) first.unref();
++ if (timer.unref) timer.unref();
++ },
++ stop() {
++ if (first) clearTimeout(first);
++ if (timer) clearInterval(timer);
++ first = null;
++ timer = null;
++ },
++ };
++}
++
++module.exports = { createUsdaPrices, rowOf, shape, PRODUCTS, SOURCE_KEY, NOTE, VINTAGE, URL };
ddiff --git a/server/http/pages.js b/server/http/pages.js
iindex 64fc128..3af7931 100644
---- a/server/http/pages.js
++++ b/server/http/pages.js
@@@ -30,6 +30,7 @@ const { markdown } = require('../render/markdown');
const { send } = require('../render/layout');
const foods = require('../food/foods');
const { NOTE: OFFICIAL_NOTE } = require('../food/official-prices');
++const { NOTE: USDA_NOTE } = require('../food/usda-prices');
const { planMeals } = require('../food/planner');
const { searchPlaces, safeUrl, ATTRIBUTION, NEAR_CITIES } = require('../food/places');
const { matchPantry } = require('../food/pantry-match');
@@@ -594,6 +595,7 @@ ${list.length ? sections : notice('Nothing matched. Try a shorter word.', 'warn'
r.get('/foods/:slug', (req, res) => {
const f = foods.getFood(req.params.slug);
const official = f && ctx.officialPrices ? ctx.officialPrices.forFood(f.slug) : null;
++ const usda = f && ctx.usdaPrices ? ctx.usdaPrices.forFood(f.slug) : null;
const off = f && ctx.off ? ctx.off.forFood(f.slug) : null;
if (!f) return page(req, res, {
title: 'No such food', crumbs: [{ label: 'Food list', href: '/foods' }, { label: 'Not found' }],
@@@ -621,6 +623,11 @@ ${table(['Store', 'A pack'], f.prices.map((p) => [p.store_name, money(p.price)])
${official ? html`<h2>US average price</h2>
<p><b>${money(official.pack_price)}</b> for this pack at the US city average for ${official.month}: ${money(official.value)} per ${official.unit} of ${official.item.charAt(0).toLowerCase() + official.item.slice(1)} (<a href="${official.url}" rel="noopener">BLS series ${official.series_id}</a>).</p>
<p class="muted small">${OFFICIAL_NOTE}</p>` : ''}
++${usda ? html`<h2>USDA average price</h2>
++<p>${usda.pack_price == null
++ ? html`<b>${money(usda.value)}</b> per ${usda.unit} of ${usda.item} (${usda.form}) at the USDA ERS national average for ${usda.year} (<a href="${usda.url}" rel="noopener">USDA ERS Fruit and Vegetable Prices</a>).`
++ : html`<b>${money(usda.pack_price)}</b> for this pack at the USDA ERS national average for ${usda.year}: ${money(usda.value)} per ${usda.unit} of ${usda.item} (${usda.form}) (<a href="${usda.url}" rel="noopener">USDA ERS Fruit and Vegetable Prices</a>).`}</p>
++<p class="muted small">${USDA_NOTE}</p>` : ''}
${off ? html`<h2>Open Food Facts label</h2>
<p><b>${off.name}</b>${off.brands ? html` · ${off.brands}` : ''}${off.nutriscore_grade ? html` · Nutri-Score ${off.nutriscore_grade}` : ''}</p>
${table(['Per 100 g', 'As the label states'], [
ddiff --git a/server/index.js b/server/index.js
iindex 42b2999..13c17b5 100644
---- a/server/index.js
++++ b/server/index.js
@@@ -47,14 +47,15 @@ async function start() {
// The recipe library writes a theme every six hours (server/food/library.js); off without OpenVibe.AI.
ctx.library.start();
ctx.recipeSearch.start();
-- // Food recalls, BLS's average prices and Open Food Facts products follow OpenVibe.Sources' change feed
-- // (server/food/recalls.js, server/food/official-prices.js, server/food/off.js).
++ // Food recalls, BLS's average prices, USDA ERS's prices and Open Food Facts products follow OpenVibe.Sources'
++ // change feed (server/food/recalls.js, server/food/official-prices.js, server/food/usda-prices.js, server/food/off.js).
ctx.recalls.start();
ctx.officialPrices.start();
++ ctx.usdaPrices.start();
ctx.off.start();
// The home's "food near" cities: their area searches refreshed in the background (server/food/near-warm.js).
ctx.nearWarm.start();
-- const extra = [() => { if (subscriptions) subscriptions.stop(); }, () => ctx.library.stop(), () => ctx.recipeSearch.stop(), () => ctx.recalls.stop(), () => ctx.officialPrices.stop(), () => ctx.off.stop(), () => ctx.nearWarm.stop()];
++ const extra = [() => { if (subscriptions) subscriptions.stop(); }, () => ctx.library.stop(), () => ctx.recipeSearch.stop(), () => ctx.recalls.stop(), () => ctx.officialPrices.stop(), () => ctx.usdaPrices.stop(), () => ctx.off.stop(), () => ctx.nearWarm.stop()];
createLifecycle({ server, ctx, extra });
return { server, ctx };
}
ddiff --git a/test/helpers/boot.js b/test/helpers/boot.js
iindex 43486f1..954a764 100644
---- a/test/helpers/boot.js
++++ b/test/helpers/boot.js
@@@ -50,7 +50,7 @@ async function boot(opts = {}) {
const { createStore } = require('../../server/db');
const testdb = await require('./db').testDb();
const store = createStore(testdb.db, { now: opts.now });
-- const built = await createApp({ config, store, log, telemetrySink: opts.telemetrySink, limitsNow: opts.limitsNow, callerLimits: opts.callerLimits === true, accountSend: opts.accountSend, ai: opts.ai, barcodes: opts.barcodes, recallSource: opts.recallSource, priceSource: opts.priceSource, offSource: opts.offSource });
++ const built = await createApp({ config, store, log, telemetrySink: opts.telemetrySink, limitsNow: opts.limitsNow, callerLimits: opts.callerLimits === true, accountSend: opts.accountSend, ai: opts.ai, barcodes: opts.barcodes, recallSource: opts.recallSource, priceSource: opts.priceSource, usdaPriceSource: opts.usdaPriceSource, offSource: opts.offSource });
await built.ctx.keys.ensure();
const server = await new Promise((resolve) => { const s = http.createServer(built.app); s.listen(0, '127.0.0.1', () => resolve(s)); });
const base = `http://127.0.0.1:${server.address().port}`;
ddiff --git a/test/usda-prices.test.js b/test/usda-prices.test.js
nnew file mode 100644
iindex 0000000..d8adb6d
---- /dev/null
++++ b/test/usda-prices.test.js
@@@ -0,0 +1,117 @@
++'use strict';
++/**
++ * USDA ERS prices (server/food/usda-prices.js): the Economic Research Service's annual fruit and vegetable prices from
++ * OpenVibe.Sources.
++ * - a usda-ers-fruit-vegetable-prices item for a mapped product and form becomes one row; another identity, unit,
++ * kind or source is held;
++ * - the pull follows the change feed from its own cursor (BLS and the recalls keep theirs), a removed item is
++ * deleted, an older revision never wins;
++ * - a food's page shows the figure beside the BLS one with its data year and the ERS note; a food ERS does not price
++ * shows none.
++ */
++const assert = require('assert');
++const { boot, check, done } = require('./helpers/boot');
++const foods = require('../server/food/foods');
++const { rowOf, shape, PRODUCTS, VINTAGE } = require('../server/food/usda-prices');
++
++const usdaItem = (identity, value, n, over = {}) => ({
++ id: `itm_u${n}`, source_key: 'usda-ers-fruit-vegetable-prices', category: 'food', kind: 'price', identity,
++ canonical_url: null, title: identity.split(' | ')[0], summary: null,
++ fields: { name: identity.split(' | ')[0], form: identity.split(' | ')[1], price: String(value), unit: 'per pound' },
++ revision: 1, provenance: { retrieved_at: '2026-11-05T06:00:00.000Z' }, removed: null, change_seq: n, ...over,
++});
++
++const blsItem = (series, value, n) => ({
++ id: 'itm_b1', source_key: 'bls-average-prices', category: 'food', kind: 'price', identity: series, canonical_url: null,
++ title: series, summary: null, fields: { value: String(value), year: '2026', period: 'M09' }, revision: 1,
++ provenance: { retrieved_at: '2026-11-05T06:00:00.000Z' }, removed: null, change_seq: n,
++});
++
++function feed(items) {
++ return {
++ items,
++ async listItems({ after = 0, limit = 100 }) {
++ const rest = items.filter((x) => x.change_seq > after).sort((a, b) => a.change_seq - b.change_seq);
++ const page = rest.slice(0, limit);
++ return { items: page, next_after: page.length ? page[page.length - 1].change_seq : after, more: rest.length > limit, sources: {} };
++ },
++ };
++}
++
++(async () => {
++ const usda = feed([
++ usdaItem('Apples | Fresh', 1.86093095, 1),
++ usdaItem('Bananas | Fresh', 0.609594742, 2),
++ usdaItem('Strawberries | Fresh', 3.078734363, 3),
++ usdaItem('Avocados | Fresh', 2.262402684, 4),
++ usdaItem('Kale | Fresh', 3.57, 5), // a row Food does not map
++ { ...usdaItem('Oranges | Fresh', 1.44, 6), source_key: 'bls-average-prices' }, // another source
++ ]);
++ const bls = feed([blsItem('APU0000711211', 0.633, 1)]);
++ const t = await boot({ priceSource: bls, usdaPriceSource: usda });
++ try {
++ await check('every mapped identity prices a food on the list; a row is the item as ERS states it', async () => {
++ for (const [identity, def] of Object.entries(PRODUCTS)) assert.ok(foods.getFood(def.slug), `${identity} → ${def.slug}`);
++ const row = rowOf(usdaItem('Apples | Fresh', 1.86093095, 1));
++ assert.deepStrictEqual([row.identity, row.value, row.unit, row.vintage], ['Apples | Fresh', 1.8609, 'lb', VINTAGE]);
++ assert.strictEqual(rowOf(usdaItem('Kale | Fresh', 3.57, 1)), null, 'a row nobody maps');
++ assert.strictEqual(rowOf(usdaItem('Apples | Fresh', 1.86, 1, { fields: { name: 'Apples', form: 'Fresh', price: '1.86', unit: 'per pint' } })), null, 'per pint would make the pack arithmetic wrong');
++ assert.strictEqual(rowOf(usdaItem('Apples | Fresh', -1, 1)), null);
++ const apples = shape({ identity: 'Apples | Fresh', value: '1.8609', unit: 'lb', vintage: '2023' });
++ assert.deepStrictEqual([apples.slug, apples.item, apples.form, apples.pack_price, apples.year], ['apples-3lb', 'apples', 'fresh', 5.58, '2023'], 'a 3 lb bag');
++ assert.strictEqual(shape({ identity: 'Strawberries | Fresh', value: '3.0787', unit: 'lb', vintage: '2023' }).pack_price, 3.08, 'a 1 lb pack');
++ assert.strictEqual(shape({ identity: 'Grapes (raisins) | Dried', value: '4.3283', unit: 'lb', vintage: '2023' }).pack_price, 1.62, 'a 6 oz box is 0.375 lb');
++ assert.strictEqual(shape({ identity: 'Avocados | Fresh', value: '2.2624', unit: 'lb', vintage: '2023' }).pack_price, null, 'a pack that states no weight gets no pack price');
++ });
++
++ await check('the pull follows its own cursor; anything else is held; BLS keeps its own cursor', async () => {
++ assert.strictEqual((await t.get('/foods/apples-3lb')).text.includes('USDA average price'), false, 'nothing before the first pull');
++ const out = await t.ctx.usdaPrices.pull();
++ assert.deepStrictEqual([out.applied, out.hold], [4, 2]);
++ assert.strictEqual(Number(await t.ctx.s.db.value("SELECT cursor FROM food_ingest_cursor WHERE name = 'sources.food.usda.after'")), 6);
++ assert.strictEqual(await t.ctx.s.db.value("SELECT cursor FROM food_ingest_cursor WHERE name = 'sources.food.prices.after'"), null, 'the BLS cursor is theirs');
++ assert.deepStrictEqual([t.ctx.usdaPrices.forFood('apples-3lb').value, t.ctx.usdaPrices.forFood('apples-3lb').year], [1.8609, '2023']);
++ assert.strictEqual(t.ctx.usdaPrices.forFood('white-rice'), null, 'a food ERS does not price');
++ const blsOut = await t.ctx.officialPrices.pull();
++ assert.strictEqual(blsOut.applied, 1);
++ assert.strictEqual(Number(await t.ctx.s.db.value("SELECT cursor FROM food_ingest_cursor WHERE name = 'sources.food.prices.after'")), 1);
++ assert.strictEqual(Number(await t.ctx.s.db.value("SELECT cursor FROM food_ingest_cursor WHERE name = 'sources.food.usda.after'")), 6);
++ });
++
++ await check('a new value replaces the old; an older revision arriving late and a removed item behave', async () => {
++ usda.items.push(usdaItem('Apples | Fresh', 2.0, 7, { id: 'itm_u1', revision: 3 })); // the same Sources item, a new value
++ usda.items.push(usdaItem('Apples | Fresh', 9.99, 8, { id: 'itm_u1', revision: 2 })); // a replayed older revision
++ usda.items.push({ ...usdaItem('Strawberries | Fresh', 3.0787, 9), id: 'itm_u3', removed: { at: '2026-11-05T07:00:00.000Z', reason: 'replaced' }, revision: 2 });
++ const out = await t.ctx.usdaPrices.pull();
++ assert.strictEqual(out.removed, 1);
++ assert.strictEqual(t.ctx.usdaPrices.forFood('apples-3lb').value, 2);
++ assert.strictEqual(t.ctx.usdaPrices.forFood('strawberries'), null);
++ });
++
++ await check('a food\'s page shows the USDA figure with its data year and the ERS note beside BLS', async () => {
++ const apples = await t.get('/foods/apples-3lb');
++ assert.strictEqual(apples.status, 200);
++ assert.ok(apples.text.includes('USDA average price'));
++ assert.ok(apples.text.includes('$6.00</b> for this pack at the USDA ERS national average for 2023: $2.00 per lb of apples (fresh)'), apples.text.slice(apples.text.indexOf('USDA average'), apples.text.indexOf('USDA average') + 400));
++ assert.ok(apples.text.includes('https://www.ers.usda.gov/data-products/fruit-and-vegetable-prices/'));
++ assert.ok(apples.text.includes('a snapshot of one year, not a price trend'), 'ERS: no inference about change over time');
++ assert.ok(!/Circana/.test(apples.text), 'ERS: the findings are not attributed to Circana (the page never names it)');
++ assert.ok(!apples.text.includes('US average price'), 'BLS prices no apples');
++ const bananas = await t.get('/foods/bananas');
++ assert.ok(bananas.text.includes('US average price'), 'BLS on the same page');
++ assert.ok(bananas.text.includes('September 2026'), 'BLS month');
++ assert.ok(bananas.text.includes('USDA average price'));
++ assert.ok(bananas.text.includes('for 2023'), 'USDA data year');
++ assert.ok(bananas.text.includes('per lb of bananas (fresh)'));
++ const avocado = await t.get('/foods/avocado');
++ assert.ok(avocado.text.includes('$2.26</b> per lb of avocados (fresh)'), 'no pack weight, so no pack price');
++ assert.ok(!avocado.text.includes('for this pack at the USDA'), 'no invented pack weight');
++ const tuna = await t.get('/foods/canned-tuna');
++ assert.ok(!tuna.text.includes('USDA average price'));
++ assert.ok(!tuna.text.includes('US average price'));
++ });
++ } finally {
++ await t.close();
++ }
++ done();
++})();