openvibers/openvibe.sources Commits

USDA ERS fruit and vegetable prices: record the published CSVs as fixtures (#214)

OP OpenVibers committed 7 days ago · b42a91a0cf · 5 files, +259 −4

The source's seed mapping already reads ERS's two all-fruits and all-vegetables CSVs (62 + 93 rows) with format csv and identity_fields. Record both files byte-for-byte in test/fixtures (BOM, CRLF and all) and prove the seed's own endpoints still parse them: 155 distinct identities, none skipped, and all 16 identities Food maps are present per pound. The README's food row now says the source is enabled in production after its 2026-11-06 terms review.

Changes

ddiff --git a/.gitattributes b/.gitattributes
nnew file mode 100644
iindex 0000000..e3b279e
---- /dev/null
++++ b/.gitattributes
@@@ -0,0 +1,2 @@
++# Recorded HTTP bodies: committed byte-for-byte, BOM, CRLF and all (git would otherwise normalize the line endings).
++test/fixtures/usda-ers-*.csv -text
ddiff --git a/README.md b/README.md
iindex 7a75f1a..0b8a1a3 100644
---- a/README.md
++++ b/README.md
@@@ -25,7 +25,7 @@ decide — under their own review and indexability rules — what, if anything,
  ```bash
  npm install
  cp .env.example .env
--npm run seed           # the 65 sources in seeds/sources.json, created disabled
++npm run seed           # the 67 sources in seeds/sources.json, created disabled
  npm run dev            # http://127.0.0.1:4720
  npm test               # every test/*.test.js against local stub sites; no internet
  ```
@@@ -362,10 +362,10 @@ for the N-1 rollback window.
  
  ## Seeds
  
--[seeds/sources.json](seeds/sources.json) holds the 66 registered sources, each with its robots, terms and licence notes (the production rows are the record; the terms notes match them as of the 2026-10-23 review).
++[seeds/sources.json](seeds/sources.json) holds the 67 registered sources, each with its robots, terms and licence notes (the production rows are the record; the terms notes match them as of the 2026-10-23 review).
  `npm run seed` creates the ones that do not exist yet, **all disabled**, and never changes an existing one; a person
  enables a source after reading its terms (`PATCH /api/v1/sources/:key {"enabled": true}`), and the fetcher re-checks
--robots.txt on every run. Production on 2026-10-22:
++robots.txt on every run. Production on 2026-10-22 (the food row checked 2026-11-06):
  
  | Category | Sources (type) | Enabled in production |
  |---|---|---|
@@@ -378,6 +378,7 @@ robots.txt on every run. Production on 2026-10-22:
  | hosting | `porkbun-tld-pricing`, `ovhcloud-vps`, `ovhcloud-baremetal`, `linode-types`, `aws-lightsail-bundles`, `vultr-plans` (api); `lowendspirit-offers`, `lowendtalk-offers`, `lowendbox-posts` (rss) | 5 of 9: `vultr-plans` is off (its API host's robots.txt disallows every path), LowEndTalk's and LowEndBox's feeds answer 403 to the production host, and `ovhcloud-baremetal` is seeded disabled for review |
  | crowdfunding | `opencollective-accounts` (api) | 1 of 1 |
  | games | `steam-top-sellers`, `steam-new-releases`, `steam-specials` (api: the store's featured lists, in their order) | 3 of 3 |
++| food | `open-food-facts`, `openfda-food-enforcement`, `bls-average-prices`, `usda-ers-fruit-vegetable-prices` (api); `themealdb` | 4 of 5: `themealdb` is off (it needs a key, which is the owner's call); `usda-ers-fruit-vegetable-prices` was enabled on 2026-11-06 after its terms review, and its 155 prices are recorded in `test/fixtures/` |
  
  Candidates rejected on 2026-09-22: CheapShark's API (`/api/1.0/` is disallowed in its
  robots.txt) and EDGAR's `getcurrent` Atom feed (`/cgi-bin` disallowed). The seeded feeds were
@@@ -492,7 +493,7 @@ through Events.
  - conditional GET with ETag and Last-Modified (`test/ingest.test.js`)
  - loopback/private/metadata addresses, DNS answers and redirects into them refused (`test/robots-ssrf.test.js`)
  - credentials by variable name, never stored, not forwarded cross-origin (`test/ingest.test.js`)
--- parsers against fixtures (`test/adapters.test.js`); API, registry rules, paging, manual items,
++- parsers against fixtures (`test/adapters.test.js`, and ERS's two published CSVs in `test/usda-ers.test.js`); API, registry rules, paging, manual items,
    removal (`test/api.test.js`); scheduler, backoff, relay to Events (`test/scheduler-events.test.js`);
    proposals and seeds (`test/proposals.test.js`)
  - the public host shows only an honest "internal service" page and the merchant opt-in pages, and the vhost keeps the
ddiff --git a/test/fixtures/usda-ers-fruits.csv b/test/fixtures/usda-ers-fruits.csv
nnew file mode 100644
iindex 0000000..1b904c0
---- /dev/null
++++ b/test/fixtures/usda-ers-fruits.csv
@@@ -0,0 +1,63 @@
++Fruit,Form,AverageRetailPrice,AverageRetailPriceUnitOfMeasure,PreparationYieldFactor,SizeOfACupEquivalent ,CupEquivalentUnitOfMeasure,AveragePricePerCupEquivalent

++Apples,Fresh,1.86093095,per pound,0.9,0.2425,pounds,0.501435057

++"Apples, applesauce",Canned,1.208099427,per pound,1,0.5401,pounds,0.652533815

++"Apples, ready to drink",Juice,0.951115064,per pint (16 fluid ounces ready to drink),1,0.5,pints,0.475557532

++"Apples, frozen concentrate",Juice,2.668334171,per pint (16 fluid ounces concentrate),4,0.5,pints,0.333541771

++Apricots,Fresh,3.803731821,per pound,0.93,0.3638,pounds,1.487802023

++"Apricots, in juice",Canned,2.107757083,per pound,1,0.5401,pounds,1.138468192

++"Apricots, in syrup or water",Canned,2.556248226,per pound,0.65,0.4409,pounds,1.734019282

++Apricots,Dried,8.634283691,per pound,1,0.1433,pounds,1.237296915

++Bananas,Fresh,0.609594742,per pound,0.64,0.3307,pounds,0.31498274

++"Berries, mixed",Frozen,4.448419152,per pound,1,0.3307,pounds,1.471062824

++Blackberries,Fresh,6.83446877,per pound,0.96,0.3197,pounds,2.275808903

++Blackberries,Frozen,4.705827184,per pound,1,0.3307,pounds,1.55618596

++Blueberries,Fresh,5.132473096,per pound,0.95,0.3197,pounds,1.727051697

++Blueberries,Frozen,4.199260815,per pound,1,0.3307,pounds,1.388667808

++Cantaloupe,Fresh,0.791924175,per pound,0.51,0.3748,pounds,0.58196465

++Cherries,Fresh,3.659903242,per pound,0.92,0.3417,pounds,1.359401467

++"Cherries, in syrup or water",Canned,6.308314259,per pound,0.65,0.4409,pounds,4.279216099

++Clementines,Fresh,1.58557324,per pound,0.77,0.463,pounds,0.9533429

++Cranberries,Dried,5.263878609,per pound,1,0.1232,pounds,0.648316528

++Dates,Dried,5.88786111,per pound,1,0.1653,pounds,0.973538385

++Figs,Dried,8.049055012,per pound,0.96,0.1653,pounds,1.386338185

++"Fruit cocktail, in juice",Canned,2.650530966,per pound,1,0.5401,pounds,1.431638029

++"Fruit cocktail, in syrup or water",Canned,2.312737347,per pound,0.65,0.4409,pounds,1.568834792

++Grapefruit,Fresh,1.395775718,per pound,0.49,0.463,pounds,1.31878231

++"Grapefruit, ready to drink",Juice,1.21936719,per pint (16 fluid ounces ready to drink),1,0.5,pints,0.609683595

++Grapes,Fresh,2.248436594,per pound,0.96,0.3307,pounds,0.77452409

++Grapes (raisins),Dried,4.328343983,per pound,1,0.1653,pounds,0.715677379

++"Grapes, ready to drink",Juice,1.153972604,per pint (16 fluid ounces ready to drink),1,0.5,pints,0.576986302

++"Grapes, frozen concentrate",Juice,3.295747122,per pint (16 fluid ounces concentrate),4,0.5,pints,0.41196839

++Honeydew,Fresh,1.250052787,per pound,0.46,0.3748,pounds,1.018482806

++Kiwi,Fresh,2.834707401,per pound,0.76,0.3858,pounds,1.439020409

++Mangoes,Fresh,1.458675758,per pound,0.71,0.3638,pounds,0.747340675

++Mangoes,Dried,10.10968525,per pound,1,0.1253,pounds,1.266365955

++Nectarines,Fresh,2.475306142,per pound,0.91,0.3197,pounds,0.869540448

++Oranges,Fresh,1.439704016,per pound,0.68,0.4079,pounds,0.863515806

++"Oranges, ready to drink",Juice,1.154882364,per pint (16 fluid ounces ready to drink),1,0.5,pints,0.577441182

++"Oranges, frozen concentrate",Juice,3.401422791,per pint (16 fluid ounces concentrate),4,0.5,pints,0.425177849

++Papaya,Fresh,1.305512939,per pound,0.62,0.3086,pounds,0.649907855

++Papaya,Dried,8.174637154,per pound,1,0.1543,pounds,1.2615393

++Peaches,Fresh,2.457099438,per pound,0.96,0.3417,pounds,0.874616079

++"Peaches, in juice",Canned,2.486758071,per pound,1,0.5401,pounds,1.343178959

++"Peaches, in syrup or water",Canned,2.211851186,per pound,0.65,0.4409,pounds,1.500399127

++Peaches,Frozen,3.759783663,per pound,1,0.3307,pounds,1.243335617

++Pears,Fresh,1.916739424,per pound,0.9,0.3638,pounds,0.774709301

++"Pears, in juice",Canned,2.329038064,per pound,1,0.5401,pounds,1.25798925

++"Pears, in syrup or water",Canned,2.316963272,per pound,0.65,0.4409,pounds,1.571701429

++Pineapple,Fresh,0.668427034,per pound,0.51,0.3638,pounds,0.47676244

++"Pineapple, in juice",Canned,1.721465817,per pound,1,0.5401,pounds,0.929819708

++"Pineapple, in syrup or water",Canned,1.82757447,per pound,0.65,0.4409,pounds,1.239726776

++Pineapple,Dried,7.947966878,per pound,1,0.1543,pounds,1.22655873

++"Pineapple, ready to drink",Juice,1.374156691,per pint (16 fluid ounces ready to drink),1,0.5,pints,0.687078346

++"Pineapple, frozen concentrate",Juice,3.039046885,per pint (16 fluid ounces concentrate),4,0.5,pints,0.379880861

++Plum,Fresh,2.658223036,per pound,0.94,0.3638,pounds,1.028683485

++Plum (prunes),Dried,6.906399803,per pound,1,0.1874,pounds,1.294210446

++"Plum (prune), ready to drink",Juice,1.794818097,per pint (16 fluid ounces ready to drink),1,0.5,pints,0.897409048

++Pomegranate,Fresh,2.475577389,per pound,0.56,0.3417,pounds,1.510617244

++"Pomegranate, ready to drink",Juice,3.187428646,per pint (16 fluid ounces ready to drink),1,0.5,pints,1.593714323

++Raspberries,Fresh,7.966754529,per pound,0.96,0.3197,pounds,2.652848596

++Raspberries,Frozen,6.209611042,per pound,1,0.3307,pounds,2.053477346

++Strawberries,Fresh,3.078734363,per pound,0.94,0.3197,pounds,1.046999868

++Strawberries,Frozen,3.46750705,per pound,1,0.3307,pounds,1.146681673

++Watermelon,Fresh,0.405825415,per pound,0.52,0.3307,pounds,0.258084199

ddiff --git a/test/fixtures/usda-ers-vegetables.csv b/test/fixtures/usda-ers-vegetables.csv
nnew file mode 100644
iindex 0000000..313121b
---- /dev/null
++++ b/test/fixtures/usda-ers-vegetables.csv
@@@ -0,0 +1,94 @@
++Vegetable,Form,AverageRetailPrice,AverageRetailPriceUnitOfMeasure,PreparationYieldFactor,SizeOfACupEquivalent ,CupEquivalentUnitOfMeasure,AveragePricePerCupEquivalent

++Acorn squash,Fresh,1.241438589,per pound,0.4586,0.4519,pounds,1.223533225

++Artichoke,Fresh,2.506855868,per pound,0.375,0.3858,pounds,2.080460852

++Artichoke,Canned,3.505101483,per pound,0.65,0.3858,pounds,2.579113206

++Asparagus,Fresh,3.212453364,per pound,0.4938,0.3968,pounds,2.581435739

++Asparagus,Canned,3.676296303,per pound,0.65,0.3968,pounds,2.244418891

++Asparagus,Frozen,6.780904794,per pound,1.0335,0.3968,pounds,2.603589725

++Avocados,Fresh,2.262402684,per pound,0.7408,0.3197,pounds,0.976334492

++Beets,Canned,1.257920081,per pound,0.65,0.3748,pounds,0.725308679

++Black beans,Canned,1.306312699,per pound,0.65,0.3858,pounds,0.775364834

++Black beans,Dried,1.658510355,per pound,2.4692,0.3858,pounds,0.259142243

++Blackeye peas,Canned,1.42085767,per pound,0.65,0.3748,pounds,0.819257451

++Blackeye peas,Dried,2.125516477,per pound,2.5397,0.3858,pounds,0.322886618

++Broccoli florets,Fresh,3.083131203,per pound,1,0.3417,pounds,1.053556824

++Broccoli heads,Fresh,3.023555459,per pound,0.78,0.3417,pounds,1.324613857

++Broccoli,Frozen,2.572513699,per pound,0.8818,0.3417,pounds,0.996849058

++Brussels sprouts,Fresh,3.039541765,per pound,1.06,0.3417,pounds,0.979869427

++Brussels sprouts,Frozen,2.885028256,per pound,0.96,0.3417,pounds,1.026939351

++Butternut squash,Fresh,1.270953317,per pound,0.714,0.4519,pounds,0.804487884

++"Cabbage, green",Fresh,0.826627116,per pound,0.7788,0.3307,pounds,0.351003021

++"Cabbage, red",Fresh,1.260671568,per pound,0.7791,0.3307,pounds,0.535094606

++"Cabbage, sauerkraut",Canned,1.630119528,per pound,0.65,0.3086,pounds,0.774048884

++"Carrots, cooked whole",Fresh,1.00554135,per pound,0.8158,0.3197,pounds,0.394004108

++"Carrots, raw whole",Fresh,1.00554135,per pound,0.89,0.2756,pounds,0.311353821

++"Carrots, baby",Fresh,1.463044321,per pound,1,0.2756,pounds,0.403182576

++Carrots,Canned,1.320606207,per pound,0.65,0.3197,pounds,0.649474702

++Carrots,Frozen,2.037831594,per pound,0.98,0.3197,pounds,0.664728772

++Cauliflower florets,Fresh,3.971328618,per pound,0.9702,0.2756,pounds,1.127980976

++Cauliflower heads,Fresh,2.124437278,per pound,0.8926,0.2756,pounds,0.655876448

++Cauliflower,Frozen,2.719124778,per pound,0.9304,0.2756,pounds,0.805427956

++"Celery, trimmed bunches",Fresh,1.327844847,per pound,0.73,0.2646,pounds,0.48121591

++Celery sticks,Fresh,2.650279098,per pound,1,0.2646,pounds,0.701143831

++Collard greens,Fresh,2.483338628,per pound,1.16,0.2866,pounds,0.613558

++Collard greens,Canned,1.416270591,per pound,0.65,0.3638,pounds,0.792594554

++Collard greens,Frozen,2.723966393,per pound,0.8818,0.3638,pounds,1.123636137

++Corn,Fresh,2.102763641,per pound,0.54,0.3638,pounds,1.416494534

++Corn,Canned,1.218782103,per pound,0.65,0.3638,pounds,0.68207309

++Corn,Frozen,1.825089043,per pound,0.963,0.3638,pounds,0.689422372

++Cucumbers with peel,Fresh,1.092848382,per pound,0.97,0.2646,pounds,0.298059992

++Cucumbers without peel,Fresh,1.092848382,per pound,0.73,0.2646,pounds,0.396052318

++Great northern beans,Canned,1.167251314,per pound,0.65,0.3858,pounds,0.692824637

++Great northern beans,Dried,1.872836969,per pound,2.4692,0.3858,pounds,0.292630776

++Green beans,Fresh,2.659189993,per pound,0.8466,0.2756,pounds,0.865621743

++Green beans,Canned,1.182933022,per pound,0.65,0.2976,pounds,0.54164511

++Green beans,Frozen,2.173343764,per pound,0.903,0.2976,pounds,0.716312032

++Green peas,Canned,1.421894276,per pound,0.65,0.3527,pounds,0.771628378

++Green peas,Frozen,2.191018173,per pound,0.8924,0.3527,pounds,0.866015088

++Green peppers,Fresh,1.503334589,per pound,0.82,0.2646,pounds,0.485017382

++Kale,Fresh,3.57112371,per pound,1.05,0.2866,pounds,0.974749919

++Kale,Frozen,2.28053379,per pound,0.8995,0.3638,pounds,0.922274695

++Kidney beans,Canned,1.199920223,per pound,0.65,0.3858,pounds,0.712215341

++Kidney beans,Dried,1.99633282,per pound,2.4692,0.3858,pounds,0.311927003

++Lentils,Dried,1.957697337,per pound,2.4692,0.3858,pounds,0.305890209

++"Lettuce, iceberg",Fresh,1.478386411,per pound,0.95,0.2425,pounds,0.377390794

++"Lettuce, romaine, heads",Fresh,2.43850297,per pound,0.94,0.2094,pounds,0.543317008

++"Lettuce, romaine, hearts",Fresh,3.029270006,per pound,0.85,0.2094,pounds,0.746409097

++Lima beans,Canned,1.627066597,per pound,0.65,0.3748,pounds,0.938156201

++Lima beans,Frozen,2.834837939,per pound,1.097,0.3748,pounds,0.403215155

++Lima beans,Dried,2.654307766,per pound,2.5397,0.3858,pounds,0.968493669

++"Mixed vegetables, peas and carrots",Canned,1.613335486,per pound,0.65,0.3392,pounds,0.841845185

++"Mixed vegetables, peas and carrots",Frozen,2.058864926,per pound,0.99,0.3392,pounds,0.705364443

++"Mixed vegetables, carrots, peas, corn, and green beans",Frozen,1.919265369,per pound,0.97,0.334,pounds,0.660927187

++"Mixed vegetables, broccoli, cauliflower, and carrots",Frozen,2.374324338,per pound,0.97,0.3105,pounds,0.760053601

++"Mushrooms, whole",Fresh,4.154521771,per pound,0.97,0.1543,pounds,0.660969781

++"Mushrooms, sliced",Fresh,4.006266272,per pound,1,0.1543,pounds,0.618261368

++Mustard greens,Canned,1.414656693,per pound,0.65,0.3307,pounds,0.719719419

++Mustard greens,Frozen,2.920188174,per pound,0.7478,0.3307,pounds,1.291356799

++Navy beans,Canned,1.234459073,per pound,0.65,0.3858,pounds,0.732715953

++Navy beans,Dried,1.705845988,per pound,2.4692,0.3858,pounds,0.266538436

++Okra,Fresh,5.680314126,per pound,0.7695,0.3527,pounds,2.603951096

++Okra,Frozen,2.652850328,per pound,0.8995,0.3748,pounds,1.105354303

++Olives,Canned,5.989003318,per pound,1,0.2976,pounds,1.782471447

++Onions,Fresh,1.062206305,per pound,0.9,0.3527,pounds,0.416313609

++Pinto beans,Canned,1.2121028,per pound,0.65,0.3858,pounds,0.719446337

++Pinto beans,Dried,1.502439243,per pound,2.4692,0.3858,pounds,0.234756132

++Potatoes,Fresh,0.921544199,per pound,0.8113,0.2646,pounds,0.300503543

++"Potatoes, french fries",Frozen,2.520906337,per pound,0.776,0.3417,pounds,1.110058188

++Potatoes,Canned,1.103743481,per pound,0.65,0.3417,pounds,0.580257486

++Pumpkin,Canned,2.080888289,per pound,1,0.5401,pounds,1.123955482

++Radish,Fresh,1.850976729,per pound,0.9,0.2756,pounds,0.566764607

++Red peppers,Fresh,1.705233347,per pound,0.82,0.2646,pounds,0.550155514

++"Spinach, boiled",Fresh,4.500420718,per pound,0.77,0.3307,pounds,1.932804413

++"Spinach, eaten raw",Fresh,4.500420718,per pound,1,0.1543,pounds,0.694521053

++Spinach,Canned,1.670872006,per pound,0.65,0.3748,pounds,0.96341412

++Spinach,Frozen,2.468972167,per pound,0.776,0.3748,pounds,1.192401331

++Sweet potatoes,Fresh,1.164626523,per pound,0.8818,0.4409,pounds,0.582313262

++"Tomatoes, grape and cherry",Fresh,4.155557011,per pound,0.91,0.3748,pounds,1.711476867

++"Tomatoes, Roma and plum",Fresh,1.235007071,per pound,0.91,0.3748,pounds,0.508640846

++"Tomatoes, large round",Fresh,2.142409751,per pound,0.91,0.3748,pounds,0.882356978

++Tomatoes,Canned,1.208159726,per pound,1,0.5401,pounds,0.652566384

++Turnip greens,Fresh,2.972110955,per pound,0.75,0.3197,pounds,1.266794055

++Turnip greens,Canned,1.397443153,per pound,0.65,0.3527,pounds,0.758359332

++Turnip greens,Frozen,2.919789615,per pound,0.776,0.3527,pounds,1.327177098

++Zucchini,Fresh,1.639076186,per pound,0.7695,0.3968,pounds,0.845273553

ddiff --git a/test/usda-ers.test.js b/test/usda-ers.test.js
nnew file mode 100644
iindex 0000000..78cd0ff
---- /dev/null
++++ b/test/usda-ers.test.js
@@@ -0,0 +1,95 @@
++'use strict';
++/**
++ * The USDA ERS fruit and vegetable prices source, end to end at the adapter: its seed's two endpoint mappings read the
++ * two CSVs ERS published on 2026-11-06 (recorded byte-for-byte in test/fixtures/), one item per product and form.
++ * This is the source Food's 16 mapped pages read; it is seeded disabled like every other seed and enabled in
++ * production by a person after the terms review (README "Seeds").
++ *
++ *   node test/usda-ers.test.js
++ */
++const assert = require('assert');
++const fs = require('fs');
++const path = require('path');
++const api = require('../server/adapters/api');
++const { validateSource } = require('../server/registry');
++const { suite } = require('./helpers');
++
++const t = suite('usda-ers');
++const seed = require('../seeds/sources.json').sources.find((s) => s.key === 'usda-ers-fruit-vegetable-prices');
++const source = validateSource(seed);
++const fixture = (name) => fs.readFileSync(path.join(__dirname, 'fixtures', name), 'utf8');
++
++// Food's 16 mapped identities (server/food/usda-prices.js PRODUCTS on 2026-11-06). This test fails if a seed mapping
++// or ERS's columns stop producing one of them, so the Food pages never silently lose their USDA price.
++const FOOD_IDENTITIES = [
++    'Apples | Fresh',
++    'Avocados | Fresh',
++    'Bananas | Fresh',
++    'Black beans | Canned',
++    'Carrots, baby | Fresh',
++    'Corn | Canned',
++    'Grapes (raisins) | Dried',
++    'Green beans | Canned',
++    'Mixed vegetables, carrots, peas, corn, and green beans | Frozen',
++    'Onions | Fresh',
++    'Oranges | Fresh',
++    'Peaches, in syrup or water | Canned',
++    'Pinto beans | Dried',
++    'Potatoes | Fresh',
++    'Strawberries | Fresh',
++    'Tomatoes | Canned',
++];
++
++const parse = (endpoint, file) => api.parse(fixture(file), { url: endpoint.url, endpoint, maxItems: 500 });
++
++t('the seed has ERS\'s two CSV files, one item per product and form, still disabled for review', () => {
++    assert.ok(seed, 'the source must be seeded');
++    assert.strictEqual(source.enabled, false, 'seeding is never permission to fetch');
++    assert.strictEqual(source.review_required, true);
++    assert.strictEqual(source.category, 'food');
++    assert.deepStrictEqual(source.endpoints.map((ep) => [ep.url, ep.format, ep.identity_fields]), [
++        ['https://www.ers.usda.gov/media/6210/all-fruits-average-prices-csv-format.csv', 'csv', ['Fruit', 'Form']],
++        ['https://www.ers.usda.gov/media/6240/all-vegetables-average-prices-csv-format.csv', 'csv', ['Vegetable', 'Form']],
++    ]);
++    for (const ep of source.endpoints) assert.strictEqual(api.checkMapping(ep), null, `${ep.url} must be a valid api mapping`);
++});
++
++t('the recorded fruits file: 62 rows, none skipped, every price and unit kept as published', () => {
++    const r = parse(source.endpoints[0], 'usda-ers-fruits.csv');
++    assert.strictEqual(r.items.length, 62);
++    assert.strictEqual(r.skipped, 0);
++    const units = {};
++    for (const item of r.items) units[item.fields.unit] = (units[item.fields.unit] || 0) + 1;
++    assert.deepStrictEqual(units, {
++        'per pound': 51,
++        'per pint (16 fluid ounces ready to drink)': 7,
++        'per pint (16 fluid ounces concentrate)': 4,
++    });
++    const apples = r.items.find((i) => i.identity === 'Apples | Fresh');
++    assert.deepStrictEqual([apples.title, apples.fields.price, apples.fields.unit], ['Apples', '1.86093095', 'per pound']);
++    assert.ok(r.items.some((i) => i.identity === 'Apples, applesauce | Canned'), 'a quoted comma belongs to the name');
++});
++
++t('the recorded vegetables file: 93 rows, none skipped, all priced per pound', () => {
++    const r = parse(source.endpoints[1], 'usda-ers-vegetables.csv');
++    assert.strictEqual(r.items.length, 93);
++    assert.strictEqual(r.skipped, 0);
++    assert.deepStrictEqual([...new Set(r.items.map((i) => i.fields.unit))], ['per pound']);
++    const onions = r.items.find((i) => i.identity === 'Onions | Fresh');
++    assert.deepStrictEqual([onions.title, onions.fields.price], ['Onions', '1.062206305']);
++});
++
++t('both files together: 155 distinct identities and every one Food maps is there, per pound', () => {
++    const items = [source.endpoints[0], source.endpoints[1]].flatMap((ep, i) => parse(ep, i ? 'usda-ers-vegetables.csv' : 'usda-ers-fruits.csv').items);
++    assert.strictEqual(items.length, 155);
++    const byIdentity = new Map(items.map((i) => [i.identity, i]));
++    assert.strictEqual(byIdentity.size, 155, 'no identity repeats across the two tables');
++    for (const identity of FOOD_IDENTITIES) {
++        const item = byIdentity.get(identity);
++        assert.ok(item, `Food maps ${identity}, which the recorded ERS files no longer state`);
++        assert.strictEqual(item.fields.unit, 'per pound', `${identity} must be per pound for Food's pack arithmetic`);
++        assert.ok(Number(item.fields.price) > 0);
++    }
++});
++
++t.run();