openvibers/openvibe.rent Commits
Rent: BLS CPI rent as a dated national and metro trend line (#152)
Changes
@@@ -26,6 +26,13 @@ the one for that size. Rent reads them from OpenVibe.Sources (`hud-fair-market-rarea a city lies in is `server/data/city-fmr-areas.json`, built once by `node tools/build-fmr-areas.js` from each city'sCensus point on HUD's own FMR layer.++Each city page also draws a **BLS CPI rent trend line** for the last 13 months, with the 12-month change ("Rents++in <metro> rose X% in the last year (BLS CPI, <month>)"), and the fair check repeats the figure for the city.++The national line is always shown; a city BLS publishes a metro rent series for gets its metro line too. Rent++reads it from OpenVibe.Sources (`bls-cpi-rent`, category `housing`; the national series and the 19 largest metro++series), never from BLS ([server/housing/rent-cpi.js](server/housing/rent-cpi.js), `rent_cpi_points`), with BLS's++citation and its note that it cannot vouch for data once they leave BLS.gov.++**OpenVibe.Homes** (since 2026-10-15, https://openvibe.homes): the buying side, served by this process for its own host(`server/homes`, `deploy/nginx/openvibe.homes.conf`).- **Search:** one place opens every for-sale site's own search (Zillow, Realtor.com, Redfin by ZIP, Homes.com, Trulia,@@@ -123,6 +130,9 @@ Rent's own data is its PostgreSQL database ([server/db.js](server/db.js)); the `per FMR area (HUD's code and name, the five monthly rents), and the cursor in Sources' change feed; an area Sourcesremoves is deleted. Public domain; nothing about a person([migrations/0010_fair_market_rents.sql](migrations/0010_fair_market_rents.sql)).++- **`rent_cpi_points`** (0011) — BLS CPI rent of primary residence, one row per series and month, for the national++ and 19 metro trend lines; a point Sources removes is deleted. Public domain; nothing about a person++ ([migrations/0011_rent_cpi.sql](migrations/0011_rent_cpi.sql)).- **`telemetry_samples`** — request telemetry (`platform.telemetry-sample@1`): each request by route template, methodand status class, never the raw URL. Once a minute one `http.request` sample per key, plus the active-requests,p95 latency and event-loop-lag gauges, is written; rows over 14 days old are deleted daily. `/shared` files,nnew file mode 100644@@@ -0,0 +1,19 @@++-- phase: expand++-- OpenVibe.Rent: BLS CPI rent of primary residence (source bls-cpi-rent, category housing), the national series and the++-- 19 largest metro series (server/housing/rent-cpi.js). Sources reads BLS's own flat file under its robots and terms++-- checks; Rent follows Sources' change feed from its rent_ingest_cursor and keeps one point per series and month, so a++-- city page can draw a dated trend line beside HUD's reference. BLS data are public domain; nothing here belongs to a++-- person.++++CREATE TABLE rent_cpi_points (++ series_id text COLLATE "C" NOT NULL, -- BLS series id (CUUR0000SEHA, CUURS12ASEHA, …)++ year integer NOT NULL,++ period text COLLATE "C" NOT NULL, -- M01..M12, as BLS writes it++ value double precision NOT NULL, -- the index value, as BLS states it++ item_id text COLLATE "C" NOT NULL UNIQUE,++ revision integer NOT NULL,++ retrieved_at text COLLATE "C" NOT NULL, -- when Sources last read it, ISO 8601 UTC++ PRIMARY KEY (series_id, year, period)++);++++CREATE INDEX rent_cpi_points_series ON rent_cpi_points (series_id, year, period);@@@ -87,6 +87,8 @@ async function createApp(opts = {}) {ctx.fmr = require('./housing/fair-market-rents').createFairMarketRents({ config, s, fetchImpl, log, source: opts.fmrSource || null });// HUD's Small Area Fair Market Rents, read from OpenVibe.Sources (source hud-safmr) and kept in memory.ctx.safmr = require('./housing/safmr').createSafmr({ config, s, fetchImpl, log, source: opts.safmrSource || null });++ // BLS CPI rent of primary residence (source bls-cpi-rent): the national and metro trend line, kept in rent_cpi_points.++ ctx.rentCpi = require('./housing/rent-cpi').createRentCpi({ config, s, fetchImpl, log, source: opts.rentCpiSource || null });// Account export and deletion (ADR-033, ./identity/account-data.js): the three tables that hold a person's rows.// The sender posts to Network's internal export/deletion routes with this service's own client-credentials token;nnew file mode 100644@@@ -0,0 +1,235 @@++'use strict';++++/**++ * BLS CPI rent of primary residence, read from OpenVibe.Sources (source bls-cpi-rent, category housing), never from++ * BLS. Sources streams BLS's published CPI flat file under its robots and terms checks and keeps one item per matching++ * series and month; Rent follows Sources' change feed from its own cursor and keeps one point per series and month in++ * rent_cpi_points, so a city page can show a dated trend line beside HUD's Fair Market Rent.++ *++ * The series are the US city average (CUUR0000SEHA) and the 19 largest metro rent series BLS publishes in that file.++ * A city maps to a BLS metro by the area name's own cities and states ("New York-Newark-Jersey City, NY-NJ-PA"); a city++ * BLS does not publish a rent series for shows the national line and says so. The 12-month change is the latest index++ * against the same month a year earlier, never a guess from a shorter run.++ *++ * BLS asks to be cited and says it cannot vouch for the data once they leave BLS.gov: NOTE says both wherever a figure++ * shows. Pulls 35 seconds after start and then every six hours. Off without OV_SOURCES_INTERNAL_URL and the OAuth++ * client secret (a test passes its own source).++ */++const ingest = require('openvibe-publishing/ingest');++++const SOURCE_KEY = 'bls-cpi-rent';++const CURSOR = 'sources.housing.rentcpi.after';++const EVERY_MS = 6 * 60 * 60 * 1000;++const FIRST_MS = 35 * 1000;++const NOTE = 'Rent of primary residence CPI from the U.S. Bureau of Labor Statistics (public domain). BLS.gov cannot vouch for the data or analyses derived from these data after the data have been retrieved from BLS.gov.';++const BLS_URL = 'https://data.bls.gov/timeseries/';++const MONTHS = ['January', 'February', 'March', 'April', 'May', 'June', 'July', 'August', 'September', 'October', 'November', 'December'];++++const NATIONAL = { series: 'CUUR0000SEHA', name: 'US city average' };++const METROS = [++ { series: 'CUURS12ASEHA', name: 'New York-Newark-Jersey City, NY-NJ-PA' },++ { series: 'CUURS49ASEHA', name: 'Los Angeles-Long Beach-Anaheim, CA' },++ { series: 'CUURS23ASEHA', name: 'Chicago-Naperville-Elgin, IL-IN-WI' },++ { series: 'CUURS37ASEHA', name: 'Dallas-Fort Worth-Arlington, TX' },++ { series: 'CUURS37BSEHA', name: 'Houston-The Woodlands-Sugar Land, TX' },++ { series: 'CUURS35BSEHA', name: 'Miami-Fort Lauderdale-West Palm Beach, FL' },++ { series: 'CUURS35CSEHA', name: 'Atlanta-Sandy Springs-Roswell, GA' },++ { series: 'CUURS12BSEHA', name: 'Philadelphia-Camden-Wilmington, PA-NJ-DE-MD' },++ { series: 'CUURS35ASEHA', name: 'Washington-Arlington-Alexandria, DC-VA-MD-WV' },++ { series: 'CUURS11ASEHA', name: 'Boston-Cambridge-Newton, MA-NH' },++ { series: 'CUURS24ASEHA', name: 'Minneapolis-St.Paul-Bloomington, MN-WI' },++ { series: 'CUURS23BSEHA', name: 'Detroit-Warren-Dearborn, MI' },++ { series: 'CUURS48ASEHA', name: 'Phoenix-Mesa-Scottsdale, AZ' },++ { series: 'CUURS48BSEHA', name: 'Denver-Aurora-Lakewood, CO' },++ { series: 'CUURS49BSEHA', name: 'San Francisco-Oakland-Hayward, CA' },++ { series: 'CUURS49DSEHA', name: 'Seattle-Tacoma-Bellevue, WA' },++ { series: 'CUURS49ESEHA', name: 'San Diego-Carlsbad, CA' },++ { series: 'CUURA104SEHA', name: 'Pittsburgh, PA' },++ { series: 'CUURA210SEHA', name: 'Cleveland-Akron, OH' },++];++const BY_SERIES = new Map([NATIONAL, ...METROS].map((m) => [m.series, m]));++++const text = (v, max) => (v == null || v === '' ? null : String(v).replace(/\s+/g, ' ').trim().slice(0, max) || null);++const norm = (v) => String(v == null ? '' : v).toLowerCase().replace(/[^a-z0-9]+/g, ' ').trim();++const monthKey = (year, period) => Number(year) * 12 + Number(String(period).slice(1)) - 1;++++/** A city of the Rent list → the BLS metro area whose own name lists that city and state, or null. */++function metroFor(city) {++ if (!city || !city.name || !city.state) return null;++ const name = norm(city.name);++ const state = String(city.state).toUpperCase();++ for (const metro of METROS) {++ const [cities, states] = metro.name.split(',');++ if (!cities || !states) continue;++ if (!states.trim().split('-').includes(state)) continue;++ if (cities.split('-').map(norm).includes(name)) return metro;++ }++ return null;++}++++/** A Sources item → a rent_cpi_points row, or null when it is not a series and month this trend uses. */++function rowOf(item) {++ if (!item || item.source_key !== SOURCE_KEY || item.kind !== 'rent_cpi') return null;++ const f = item.fields || {};++ const series = text(f.series_id, 20);++ if (!series || !BY_SERIES.has(series)) return null;++ const year = Number(f.year);++ const period = text(f.period, 3);++ const value = Number(f.value);++ if (!Number.isInteger(year) || year < 2000 || year > 2100) return null;++ if (!/^M(0[1-9]|1[0-2])$/.test(period || '')) return null;++ if (!Number.isFinite(value) || value <= 0 || value > 100000) return null;++ return {++ series_id: series,++ year,++ period,++ value: Math.round(value * 1000) / 1000,++ item_id: String(item.id || ''),++ revision: Number(item.revision) || 1,++ retrieved_at: (item.provenance && item.provenance.retrieved_at) || new Date(0).toISOString(),++ };++}++++function pointOf(row) {++ return { year: Number(row.year), period: row.period, value: Number(row.value), month: monthKey(row.year, row.period), label: `${MONTHS[Number(row.period.slice(1)) - 1]} ${row.year}` };++}++++/** The latest point and the 12-month change, with the points a trend line draws (up to the last 13 months). */++function trendOf(rows) {++ const points = rows.map(pointOf).sort((a, b) => a.month - b.month);++ if (!points.length) return null;++ const latest = points[points.length - 1];++ const earlier = points.find((p) => p.month === latest.month - 12) || null;++ const change_pct = earlier && earlier.value ? Math.round((latest.value / earlier.value - 1) * 1000) / 10 : null;++ return {++ points: points.slice(-13),++ latest,++ earlier,++ change_pct,++ series: rows[0].series_id,++ };++}++++function phrasing(area, trend) {++ if (!trend) return null;++ const month = trend.latest.label;++ if (trend.change_pct == null) return `BLS's rent index for ${area} stands at ${trend.latest.value} for ${month}; a 12-month change needs the same month a year earlier.`;++ const verb = trend.change_pct > 0 ? 'rose' : trend.change_pct < 0 ? 'fell' : 'was unchanged';++ const amount = trend.change_pct === 0 ? '' : ` ${Math.abs(trend.change_pct).toFixed(1)}%`;++ return `Rents in ${area} ${verb}${amount} in the last year (BLS CPI, ${month}).`;++}++++function createRentCpi({ config, s, fetchImpl = globalThis.fetch, log = console, source = null }) {++ const url = String((config.sources && config.sources.internalUrl) || '').replace(/\/+$/, '');++ const client = source || (url && config.oauth.clientSecret++ ? ingest.createSourcesClient({ config: { ...config, sources: { internalUrl: url, category: 'housing' } }, fetchImpl })++ : null);++ const cursor = ingest.createChangeCursor(s.db, { prefix: 'rent', now: s.now });++ const stats = { pulls: 0, applied: 0, removed: 0, last_at: null, last_error: null };++ let bySeries = new Map();++ let timer = null;++ let first = null;++ let running = false;++++ async function apply(item, t) {++ if (!item || item.source_key !== SOURCE_KEY || item.kind !== 'rent_cpi') return 'hold';++ if (item.removed) {++ await t.query('DELETE FROM rent_cpi_points WHERE item_id = $1', [String(item.id)]);++ return 'removed';++ }++ const row = rowOf(item);++ if (!row) return 'hold';++ const cols = Object.keys(row);++ await t.query(`INSERT INTO rent_cpi_points (${cols.join(', ')}) VALUES (${cols.map((_, i) => `$${i + 1}`).join(', ')})++ ON CONFLICT (series_id, year, period) DO UPDATE SET ${cols.filter((c) => !['series_id', 'year', 'period'].includes(c)).map((c) => `${c} = EXCLUDED.${c}`).join(', ')}++ WHERE rent_cpi_points.item_id <> EXCLUDED.item_id OR rent_cpi_points.revision <= EXCLUDED.revision`, cols.map((c) => row[c]));++ return 'applied';++ }++++ async function load() {++ const rows = await s.db.many('SELECT series_id, year, period, value, item_id, revision, retrieved_at FROM rent_cpi_points');++ const grouped = new Map();++ for (const row of rows) {++ if (!BY_SERIES.has(row.series_id)) continue;++ if (!grouped.has(row.series_id)) grouped.set(row.series_id, []);++ grouped.get(row.series_id).push(row);++ }++ bySeries = new Map([...grouped.entries()].map(([series, list]) => [series, trendOf(list)]));++ return bySeries.size;++ }++++ /** Follow Sources' change feed to its end. Never throws: the caller is a timer or a test. */++ async function pull() {++ if (!client || running) return { skipped: true };++ running = true;++ try {++ const out = await ingest.pullChanges({ db: s.db, cursor, source: client, name: CURSOR, apply });++ stats.pulls++;++ stats.applied += out.applied;++ stats.removed += out.removed;++ stats.last_at = s.iso();++ stats.last_error = null;++ await load();++ if (out.applied || out.removed) log.log(`[RentCpi] ${out.applied} BLS rent points updated, ${out.removed} removed (Sources cursor ${out.after})`);++ return out;++ } catch (err) {++ stats.last_error = text(err && err.message, 300);++ log.warn(`[RentCpi] pull from OpenVibe.Sources failed: ${stats.last_error}`);++ return { error: stats.last_error };++ } finally { running = false; }++ }++++ /** A trend for one series: the latest point, its 12-month change and the line's points; null when unread. */++ function forSeries(series) {++ const trend = bySeries.get(series);++ if (!trend) return null;++ const def = BY_SERIES.get(series);++ return { series, area: def.name, ...trend, text: phrasing(def.name, trend), url: `${BLS_URL}${series}` };++ }++++ return {++ enabled: Boolean(client),++ stats,++ pull,++ apply,++ load,++ forSeries,++ metroFor,++ /** The national line and, where BLS publishes one, the city's metro line; the metro line is the one quoted. */++ trendForCity(city) {++ const national = forSeries(NATIONAL.series);++ const metro = metroFor(city);++ const local = metro ? forSeries(metro.series) : null;++ const picked = local || national;++ if (!picked) return null;++ return {++ scope: local ? 'metro' : 'national',++ area: picked.area,++ text: picked.text,++ series: picked.series,++ url: picked.url,++ points: picked.points,++ latest: picked.latest,++ change_pct: picked.change_pct,++ national,++ metro: local,++ };++ },++ start() {++ load().catch(() => { /* the first pull loads it */ });++ if (!client || timer) return;++ first = setTimeout(() => { pull(); }, FIRST_MS);++ timer = setInterval(() => { pull(); }, EVERY_MS);++ if (first.unref) first.unref();++ if (timer.unref) timer.unref();++ },++ stop() {++ if (first) clearTimeout(first);++ if (timer) clearInterval(timer);++ first = null;++ timer = null;++ },++ };++}++++module.exports = { createRentCpi, rowOf, trendOf, metroFor, phrasing, METROS, NATIONAL, NOTE, BLS_URL, SOURCE_KEY };@@@ -34,11 +34,36 @@ const cell = (c, key) => {};const placeLabel = (c) => `${c.name}, ${c.state}`;const { HUD_URL } = require('../housing/fair-market-rents');++const { NOTE: CPI_NOTE, NATIONAL: CPI_NATIONAL } = require('../housing/rent-cpi');const fmrText = (n) => (n == null ? null : `$${n.toLocaleString('en-US')}`);/** What a Fair Market Rent is, under the city table (HUD's own definition and use, in short). */const fmrSource = (area) => html`<p class="small muted city-source">HUD Fair Market Rent for the ${area.name}: what a modest rental there costs, rent plus essential utilities, by HUD's estimate. HUD sets them each October; Housing Choice Vouchers pay up to 90% to 110% of them. <a href="${HUD_URL}" rel="noopener">HUD</a> (public domain), through OpenVibe.Sources.</p>`;--function mountCityPages(r, { config, page, fmr = null }) {++/** A small line for a BLS CPI trend: one point per month, the last 13 at most. */++function trendSvg(trend, label) {++ const points = (trend && trend.points) || [];++ if (points.length < 2) return '';++ const values = points.map((p) => Number(p.value));++ const min = Math.min(...values);++ const max = Math.max(...values);++ const span = max - min || 1;++ const x = (i) => 4 + (i * 312) / (points.length - 1);++ const y = (v) => 60 - ((v - min) / span) * 56;++ const poly = points.map((p, i) => `${x(i).toFixed(1)},${y(p.value).toFixed(1)}`).join(' ');++ return html`<svg class="rent-trend" viewBox="0 0 320 64" width="320" height="64" role="img" aria-label="${label}"><polyline fill="none" stroke="currentColor" stroke-width="2" points="${poly}"/></svg>`;++}++++/** The BLS trend lines a page shows: the national line and, where BLS publishes one, a city's metro line. */++function cpiBlock(cpi) {++ if (!cpi) return '';++ const lines = [cpi.national, cpi.metro].filter(Boolean).filter((t, i, all) => all.findIndex((x) => x.series === t.series) === i);++ if (!lines.length) return '';++ return html`<section class="sc-sec rent-cpi" aria-labelledby="h-cpi"><h2 id="h-cpi">Are rents rising?</h2>++${lines.map((t) => html`<div class="rent-cpi-line"><p>${t.text} <a href="${t.url}" rel="noopener">BLS series ${t.series}</a></p>${trendSvg(t, `Rent of primary residence CPI, ${t.area}, last 13 months`)}</div>`)}++<p class="small muted city-source">Rent of primary residence CPI from the <a href="${lines[0].url}" rel="noopener">U.S. Bureau of Labor Statistics</a> (public domain), through OpenVibe.Sources. ${CPI_NOTE}</p></section>`;++}++++function mountCityPages(r, { config, page, fmr = null, rentCpi = null }) {const base = () => String(config.baseUrl).replace(/\/+$/, '');const crumbsLd = (items) => seo.jsonLd.breadcrumbs(items.map((i) => ({ name: i.label, url: `${base()}${i.href}` })));@@@ -52,6 +77,7 @@ ${city ? html`<p class="small muted fair-how">Within the Census' margin of the m// ── /in: every state ──r.get('/in', (req, res) => {const largest = [...cities.cities].sort((a, b) => b.population - a.population).slice(0, 30);++ const national = rentCpi ? rentCpi.forSeries(CPI_NATIONAL.series) : null;page(req, res, {title: 'Rent by city: typical rents across the US',description: `What rent usually costs in ${cities.cities.length} US cities, by number of bedrooms, from the Census Bureau, and every rental site's search for each city in one place.`,@@@ -65,7 +91,7 @@ ${city ? html`<p class="small muted fair-how">Within the Census' margin of the m<div class="table-wrap"><table class="city-table"><caption class="sr-only">Typical rents in the 30 largest US cities</caption><thead><tr><th scope="col">City</th><th scope="col">1 bedroom</th><th scope="col">2 bedrooms</th><th scope="col">3 bedrooms</th></tr></thead><tbody>${largest.map((c) => html`<tr><th scope="row"><a href="${c.path}">${placeLabel(c)}</a></th><td>${cell(c, 'b1')}</td><td>${cell(c, 'b2')}</td><td>${cell(c, 'b3')}</td></tr>`)}</tbody></table></div>--${source()}</section>`,++${source()}</section>${cpiBlock({ national, metro: null })}`,});});@@@ -104,6 +130,7 @@ ${source()}const near = cities.nearby(c, 8);const two = cities.rentText(c.rent.b2);const area = fmr ? fmr.forCity(c) : null;++ const cpi = rentCpi ? rentCpi.trendForCity(c) : null;page(req, res, {title: `Rent in ${where}: typical rents and every rental site`,description: `${two ? `A typical 2-bedroom in ${where} rents for ${two} a month. ` : ''}Every rental site's search for ${c.name} in one place: Craigslist, Zillow, Apartments.com, Zumper and more, with typical rents by size and affordable housing nearby.`,@@@ -125,6 +152,7 @@ ${cities.SIZES.map((s) => html`<tr><th scope="row">${s.text}</th><td>${cell(c, s${source()}`}</section><section class="fair-card" aria-labelledby="h-fair"><h2 id="h-fair">Is a rent fair?</h2><p class="small muted">Compare a price from any site with the middle rent in ${c.name}.</p>${fairForm({}, { city: c })}</section></div>++${cpiBlock(cpi)}<section class="sc-sec" aria-labelledby="h-sites"><h2 id="h-sites">Apartments in ${c.name}, on every site</h2><ul class="everywhere-sites">${links.map((l) => html`<li class="ew-site"><a class="ew-open" href="${l.url}" target="_blank" rel="noopener nofollow"><span class="ew-name">${l.name}</span><span class="ew-host">${hostOf(l.url)} ↗</span></a><p class="ew-best">${l.best}</p></li>`)}</ul><nav class="everywhere-kinds" aria-label="Other things to rent in ${c.name}">${everywhere.KINDS.filter((k) => k !== 'apartment').map((k) => html`<a class="ew-kind" href="${kindHref(k)}">${everywhere.KIND_TEXT[k]}</a>`)}</nav>@@@ -177,6 +205,7 @@ ${source()}`,/** The verdict and a bar: the middle rent with its margin, and where the asked rent falls. */function fairResult(city, f, rent) {const area = fmr ? fmr.forCity(city) : null;++ const cpi = rentCpi ? rentCpi.trendForCity(city) : null;const max = Math.max(rent, f.median) * 1.35;const pct = (n) => Math.max(0, Math.min(100, (n / max) * 100)).toFixed(1);const lo = Math.max(0, f.median - f.moe); const hi = f.median + f.moe;@@@ -186,6 +215,7 @@ ${source()}`,<div class="fair-bar" aria-hidden="true"><span class="fair-band" style="left:${pct(lo)}%;width:${(pct(hi) - pct(lo)).toFixed(1)}%"></span><span class="fair-median" style="left:${pct(f.median)}%"></span><span class="fair-you" style="left:${pct(rent)}%"></span></div><p class="fair-legend small"><span class="lg lg-median"></span> Middle rent ${f.topped ? '$3,500 or more' : `$${f.median.toLocaleString('en-US')}`}${f.moe && !f.topped ? ` (±$${f.moe.toLocaleString('en-US')})` : ''} <span class="lg lg-you"></span> This rent $${rent.toLocaleString('en-US')}</p>${area && fmrText(area.rents[f.size.key]) ? html`<p class="small">HUD's Fair Market Rent for ${f.size.long} in the ${area.name} is ${fmrText(area.rents[f.size.key])} a month, rent plus essential utilities: the figure a Housing Choice Voucher is measured against.</p>` : ''}++${cpi && cpi.text ? html`<p class="small">${cpi.text} <a href="${cpi.url}" rel="noopener">BLS series ${cpi.series}</a>.</p>` : ''}<p class="small"><a href="${city.path}">Typical rents and every rental site in ${city.name}</a></p></section>`;}@@@ -903,7 +903,7 @@ ${it.newCount ? html` <span class="badge info">${it.newCount} new</span>` : html// ── The board: listings kept from any site (http/board-pages.js) ──mountBoardPages(r, { config, s, page, signedIn, requesterOf, mustSignIn, budgeted, sameSite, signInPrompt });-- mountCityPages(r, { config, page, fmr: ctx.fmr });++ mountCityPages(r, { config, page, fmr: ctx.fmr, rentCpi: ctx.rentCpi });// ── Reporting ────────────────────────────────────────────r.post('/listings/:id/reports', mustSignIn('/listings'), budgeted('rent.listing.report'), async (req, res) => {@@@ -44,6 +44,7 @@ async function start() {ctx.screen.start(); // the AI listing screen's sweep (off without OpenVibe.AI)ctx.fmr.start(); // HUD's Fair Market Rents: loaded now, then followed from OpenVibe.Sources every six hoursctx.safmr.start(); // HUD's Small Area Fair Market Rents: followed from OpenVibe.Sources every six hours++ ctx.rentCpi.start(); // BLS CPI rent: the national and metro trend line, followed from OpenVibe.Sources every six hours// One sweep at boot (a restart must not leave a stale listing up a moment longer than it should), then a timer.const expiry = startExpiryTimer(ctx.s);@@@ -56,7 +57,7 @@ async function start() {// Subscribe to the two ADR-033 topics at OpenVibe.Events (idempotent; off without RENT_EVENTS_URL and// RENT_EVENTS_SECRET). The consumer itself is mounted in server/app.js.const subscriptions = require('./events-consumer').startSubscriptions({ config, port: config.port, secret: config.events.secrets[0] || '' });-- const extra = [() => { if (subscriptions) subscriptions.stop(); }, () => ctx.fmr.stop(), () => ctx.safmr.stop()];++ const extra = [() => { if (subscriptions) subscriptions.stop(); }, () => ctx.fmr.stop(), () => ctx.safmr.stop(), () => ctx.rentCpi.stop()];createLifecycle({ server, ctx, timers: [expiry.timer, ctx.housingPruneTimer, ...(hpiJob ? [hpiJob.timer] : [])], extra });return { server, ctx, expiry };}@@@ -38,7 +38,7 @@ async function boot(opts = {}) {const { createStore } = require('../../server/db');const testdb = await require('./db').testDb();const store = createStore(testdb.db, { now: opts.now });-- const built = await createApp({ config, store, log, ai: opts.ai, telemetrySink: opts.telemetrySink, indexnow: opts.indexnow, limitsNow: opts.limitsNow, callerLimits: opts.callerLimits === true, accountSend: opts.accountSend, housingFetchImpl: opts.housingFetchImpl, fmrSource: opts.fmrSource });++ const built = await createApp({ config, store, log, ai: opts.ai, telemetrySink: opts.telemetrySink, indexnow: opts.indexnow, limitsNow: opts.limitsNow, callerLimits: opts.callerLimits === true, accountSend: opts.accountSend, housingFetchImpl: opts.housingFetchImpl, fmrSource: opts.fmrSource, rentCpiSource: opts.rentCpiSource });await built.ctx.keys.ensure();const server = await new Promise((resolve) => { const s = http.createServer(built.app); s.listen(0, '127.0.0.1', () => resolve(s)); });const base = `http://127.0.0.1:${server.address().port}`;nnew file mode 100644@@@ -0,0 +1,118 @@++'use strict';++/**++ * BLS CPI rent (server/housing/rent-cpi.js): the national and metro trend lines from OpenVibe.Sources.++ * - a bls-cpi-rent item becomes one point per series and month; another series, kind or source is held;++ * - the 12-month change is the latest index against the same month a year earlier;++ * - a city maps to its BLS metro by the area name's own cities and states, else the national line is used;++ * - the city and fair pages show "Rents in <area> rose X% in the last year (BLS CPI, <month>)" with a line++ * and the BLS credit; a city page with no metro line shows the national one.++ */++const assert = require('assert');++const { boot, check, done } = require('./helpers/boot');++const cities = require('../server/aggregate/cities');++const { createRentCpi, rowOf, trendOf, metroFor, phrasing, NOTE } = require('../server/housing/rent-cpi');++++const item = (series, year, period, value, over = {}) => ({++ id: `itm_${series}_${year}${period}`, source_key: 'bls-cpi-rent', category: 'housing', kind: 'rent_cpi',++ identity: `${series}-${year}${period}`, canonical_url: null, title: series, summary: null,++ fields: { series_id: series, year: String(year), period, value: String(value) },++ revision: 1, provenance: { retrieved_at: '2026-10-30T06:00:00.000Z' }, removed: null, ...over,++});++++function feed(items) {++ return {++ items,++ async listItems({ after = 0, limit = 100 }) {++ const rest = items.filter((x) => x.change_seq > after).sort((a, b) => a.change_seq - b.change_seq);++ const page = rest.slice(0, limit);++ return { items: page, next_after: page.length ? page[page.length - 1].change_seq : after, more: rest.length > limit, sources: {} };++ },++ };++}++++(async () => {++ const items = [++ { ...item('CUUR0000SEHA', 2025, 'M09', 400), change_seq: 1 },++ { ...item('CUUR0000SEHA', 2026, 'M09', 420), change_seq: 2 },++ { ...item('CUURS12ASEHA', 2025, 'M09', 500), change_seq: 3 },++ { ...item('CUURS12ASEHA', 2026, 'M09', 520), change_seq: 4 },++ { ...item('CUUR0000SEHA', 2026, 'M08', 419), change_seq: 5 },++ { ...item('CUURS99XSEHA', 2026, 'M09', 999), change_seq: 6 }, // not one of the 19 metro series++ { ...item('CUUR0000SEHA', 2026, 'M07', 418, { kind: 'record' }), change_seq: 7 },++ { ...item('CUUR0000SEHA', 2026, 'M06', 417, { source_key: 'hud-fmr' }), change_seq: 8 },++ ];++ const source = feed(items);++ const t = await boot({ rentCpiSource: source });++ const ny = cities.match('New York, NY');++ const boise = cities.match('Boise City, ID');++ try {++ await check('one point per series and month; the other series and kinds are held', async () => {++ const row = rowOf(item('CUUR0000SEHA', 2026, 'M09', 450.402));++ assert.deepStrictEqual([row.series_id, row.year, row.period, row.value], ['CUUR0000SEHA', 2026, 'M09', 450.402]);++ assert.strictEqual(rowOf(item('CUURS99XSEHA', 2026, 'M09', 999)), null, 'a series Rent does not map');++ assert.strictEqual(rowOf(item('CUUR0000SEHA', 2026, 'M13', 450)), null, 'M13 is the annual average, not a month');++ assert.strictEqual(rowOf(item('CUUR0000SEHA', 2026, 'M09', -1)), null);++ const out = await t.ctx.rentCpi.pull();++ assert.deepStrictEqual([out.applied, out.hold], [5, 3]);++ assert.strictEqual(Number(await t.ctx.s.db.value("SELECT cursor FROM rent_ingest_cursor WHERE name = 'sources.housing.rentcpi.after'")), 8);++ });++++ await check('the 12-month change compares the same month a year earlier', async () => {++ const national = t.ctx.rentCpi.forSeries('CUUR0000SEHA');++ assert.deepStrictEqual([national.latest.label, national.latest.value, national.change_pct], ['September 2026', 420, 5]);++ assert.strictEqual(national.text, 'Rents in US city average rose 5.0% in the last year (BLS CPI, September 2026).');++ const nyTrend = t.ctx.rentCpi.forSeries('CUURS12ASEHA');++ assert.strictEqual(nyTrend.change_pct, 4);++ assert.strictEqual(trendOf([]), null);++ assert.strictEqual(phrasing('Somewhere', { latest: { value: 1, label: 'September 2026' }, change_pct: null }).includes('12-month change'), true);++ });++++ await check('a city maps to its BLS metro by the area name; another falls back to the national line', async () => {++ assert.strictEqual(metroFor({ name: 'New York', state: 'NY' }).series, 'CUURS12ASEHA');++ assert.strictEqual(metroFor({ name: 'Jersey City', state: 'NJ' }).series, 'CUURS12ASEHA');++ assert.strictEqual(metroFor({ name: 'Boise City', state: 'ID' }), null);++ assert.strictEqual(metroFor({ name: 'St. Paul', state: 'MN' }).series, 'CUURS24ASEHA');++ const local = t.ctx.rentCpi.trendForCity(ny);++ assert.deepStrictEqual([local.scope, local.area, local.change_pct], ['metro', 'New York-Newark-Jersey City, NY-NJ-PA', 4]);++ const national = t.ctx.rentCpi.trendForCity(boise);++ assert.deepStrictEqual([national.scope, national.area, national.change_pct], ['national', 'US city average', 5]);++ });++++ await check('the city page draws both lines with the BLS credit; a city BLS does not publish shows the national one', async () => {++ const page = await t.get(ny.path);++ assert.strictEqual(page.status, 200);++ assert.ok(page.text.includes('Are rents rising?'));++ assert.ok(page.text.includes('Rents in New York-Newark-Jersey City, NY-NJ-PA rose 4.0% in the last year (BLS CPI, September 2026).'));++ assert.ok(page.text.includes('Rents in US city average rose 5.0% in the last year (BLS CPI, September 2026).'));++ assert.ok(page.text.includes('BLS series CUURS12ASEHA'));++ assert.ok(page.text.includes('<svg class="rent-trend"'), 'a trend line');++ assert.ok(page.text.includes('BLS.gov cannot vouch'));++ const boisePage = await t.get(boise.path);++ assert.ok(boisePage.text.includes('Rents in US city average rose 5.0%'));++ assert.ok(!boisePage.text.includes('CUURS12ASEHA'), 'no metro line where BLS has none');++ const all = await t.get('/in');++ assert.ok(all.text.includes('Are rents rising?') && all.text.includes('BLS series CUUR0000SEHA'), 'the national line is on /in too');++ });++++ await check('the fair page names the same 12-month change', async () => {++ const page = await t.get(`/fair?city=${encodeURIComponent(`${ny.state.toLowerCase()}/${ny.slug}`)}&beds=1&rent=2000`);++ assert.strictEqual(page.status, 200);++ assert.ok(page.text.includes('Rents in New York-Newark-Jersey City, NY-NJ-PA rose 4.0% in the last year (BLS CPI, September 2026).'));++ assert.ok(page.text.includes('BLS series CUURS12ASEHA'));++ });++++ await check('the points survive a restart, and a removed item is deleted', async () => {++ const again = createRentCpi({ config: { sources: { internalUrl: '' } }, s: t.ctx.s, log: { log() {}, warn() {} } });++ assert.strictEqual(await again.load(), 2);++ assert.strictEqual(again.forSeries('CUUR0000SEHA').latest.value, 420);++ source.items.push({ ...item('CUURS12ASEHA', 2026, 'M09', 520, { id: 'itm_CUURS12ASEHA_2026M09', removed: { at: '2026-10-30T07:00:00.000Z', reason: 'licence' }, revision: 2 }), change_seq: 9 });++ const out = await t.ctx.rentCpi.pull();++ assert.strictEqual(out.removed, 1);++ assert.strictEqual(t.ctx.rentCpi.forSeries('CUURS12ASEHA').latest.label, 'September 2025', 'only the 2025 point remains');++ assert.ok(NOTE.includes('BLS.gov cannot vouch'));++ });++ } finally {++ await t.close();++ }++ done();++})().catch((err) => { console.error(err); process.exit(1); });
@@ -26,6 +26,13 @@ the one for that size. Rent reads them from OpenVibe.Sources (`hud-fair-market-r
@@ -26,6 +26,13 @@ the one for that size. Rent reads them from OpenVibe.Sources (`hud-fair-market-r
area a city lies in is `server/data/city-fmr-areas.json`, built once by `node tools/build-fmr-areas.js` from each city's
area a city lies in is `server/data/city-fmr-areas.json`, built once by `node tools/build-fmr-areas.js` from each city's
Census point on HUD's own FMR layer.
Census point on HUD's own FMR layer.
+Each city page also draws a **BLS CPI rent trend line** for the last 13 months, with the 12-month change ("Rents
+in <metro> rose X% in the last year (BLS CPI, <month>)"), and the fair check repeats the figure for the city.
+The national line is always shown; a city BLS publishes a metro rent series for gets its metro line too. Rent
+reads it from OpenVibe.Sources (`bls-cpi-rent`, category `housing`; the national series and the 19 largest metro
+series), never from BLS ([server/housing/rent-cpi.js](server/housing/rent-cpi.js), `rent_cpi_points`), with BLS's
+citation and its note that it cannot vouch for data once they leave BLS.gov.
+
**OpenVibe.Homes** (since 2026-10-15, https://openvibe.homes): the buying side, served by this process for its own host
**OpenVibe.Homes** (since 2026-10-15, https://openvibe.homes): the buying side, served by this process for its own host
(`server/homes`, `deploy/nginx/openvibe.homes.conf`).
(`server/homes`, `deploy/nginx/openvibe.homes.conf`).
- **Search:** one place opens every for-sale site's own search (Zillow, Realtor.com, Redfin by ZIP, Homes.com, Trulia,
- **Search:** one place opens every for-sale site's own search (Zillow, Realtor.com, Redfin by ZIP, Homes.com, Trulia,
@@ -123,6 +130,9 @@ Rent's own data is its PostgreSQL database ([server/db.js](server/db.js)); the `
@@ -123,6 +130,9 @@ Rent's own data is its PostgreSQL database ([server/db.js](server/db.js)); the `
per FMR area (HUD's code and name, the five monthly rents), and the cursor in Sources' change feed; an area Sources
per FMR area (HUD's code and name, the five monthly rents), and the cursor in Sources' change feed; an area Sources
removes is deleted. Public domain; nothing about a person
removes is deleted. Public domain; nothing about a person
([migrations/0010_fair_market_rents.sql](migrations/0010_fair_market_rents.sql)).
([migrations/0010_fair_market_rents.sql](migrations/0010_fair_market_rents.sql)).
+- **`rent_cpi_points`** (0011) — BLS CPI rent of primary residence, one row per series and month, for the national
+ and 19 metro trend lines; a point Sources removes is deleted. Public domain; nothing about a person
+ ([migrations/0011_rent_cpi.sql](migrations/0011_rent_cpi.sql)).
- **`telemetry_samples`** — request telemetry (`platform.telemetry-sample@1`): each request by route template, method
- **`telemetry_samples`** — request telemetry (`platform.telemetry-sample@1`): each request by route template, method
and status class, never the raw URL. Once a minute one `http.request` sample per key, plus the active-requests,
and status class, never the raw URL. Once a minute one `http.request` sample per key, plus the active-requests,
p95 latency and event-loop-lag gauges, is written; rows over 14 days old are deleted daily. `/shared` files,
p95 latency and event-loop-lag gauges, is written; rows over 14 days old are deleted daily. `/shared` files,
new file mode 100644
new file mode 100644
@@ -0,0 +1,19 @@
@@ -0,0 +1,19 @@
+-- phase: expand
+-- OpenVibe.Rent: BLS CPI rent of primary residence (source bls-cpi-rent, category housing), the national series and the
+-- 19 largest metro series (server/housing/rent-cpi.js). Sources reads BLS's own flat file under its robots and terms
+-- checks; Rent follows Sources' change feed from its rent_ingest_cursor and keeps one point per series and month, so a
+-- city page can draw a dated trend line beside HUD's reference. BLS data are public domain; nothing here belongs to a
+-- person.
+
+CREATE TABLE rent_cpi_points (
+ series_id text COLLATE "C" NOT NULL, -- BLS series id (CUUR0000SEHA, CUURS12ASEHA, …)
+ year integer NOT NULL,
+ period text COLLATE "C" NOT NULL, -- M01..M12, as BLS writes it
+ value double precision NOT NULL, -- the index value, as BLS states it
+ item_id text COLLATE "C" NOT NULL UNIQUE,
+ revision integer NOT NULL,
+ retrieved_at text COLLATE "C" NOT NULL, -- when Sources last read it, ISO 8601 UTC
+ PRIMARY KEY (series_id, year, period)
+);
+
+CREATE INDEX rent_cpi_points_series ON rent_cpi_points (series_id, year, period);
@@ -87,6 +87,8 @@ async function createApp(opts = {}) {
@@ -87,6 +87,8 @@ async function createApp(opts = {}) {
ctx.fmr = require('./housing/fair-market-rents').createFairMarketRents({ config, s, fetchImpl, log, source: opts.fmrSource || null });
ctx.fmr = require('./housing/fair-market-rents').createFairMarketRents({ config, s, fetchImpl, log, source: opts.fmrSource || null });
// HUD's Small Area Fair Market Rents, read from OpenVibe.Sources (source hud-safmr) and kept in memory.
// HUD's Small Area Fair Market Rents, read from OpenVibe.Sources (source hud-safmr) and kept in memory.
ctx.safmr = require('./housing/safmr').createSafmr({ config, s, fetchImpl, log, source: opts.safmrSource || null });
ctx.safmr = require('./housing/safmr').createSafmr({ config, s, fetchImpl, log, source: opts.safmrSource || null });
+ // BLS CPI rent of primary residence (source bls-cpi-rent): the national and metro trend line, kept in rent_cpi_points.
+ ctx.rentCpi = require('./housing/rent-cpi').createRentCpi({ config, s, fetchImpl, log, source: opts.rentCpiSource || null });
// Account export and deletion (ADR-033, ./identity/account-data.js): the three tables that hold a person's rows.
// Account export and deletion (ADR-033, ./identity/account-data.js): the three tables that hold a person's rows.
// The sender posts to Network's internal export/deletion routes with this service's own client-credentials token;
// The sender posts to Network's internal export/deletion routes with this service's own client-credentials token;
new file mode 100644
new file mode 100644
@@ -0,0 +1,235 @@
@@ -0,0 +1,235 @@
+'use strict';
+
+/**
+ * BLS CPI rent of primary residence, read from OpenVibe.Sources (source bls-cpi-rent, category housing), never from
+ * BLS. Sources streams BLS's published CPI flat file under its robots and terms checks and keeps one item per matching
+ * series and month; Rent follows Sources' change feed from its own cursor and keeps one point per series and month in
+ * rent_cpi_points, so a city page can show a dated trend line beside HUD's Fair Market Rent.
+ *
+ * The series are the US city average (CUUR0000SEHA) and the 19 largest metro rent series BLS publishes in that file.
+ * A city maps to a BLS metro by the area name's own cities and states ("New York-Newark-Jersey City, NY-NJ-PA"); a city
+ * BLS does not publish a rent series for shows the national line and says so. The 12-month change is the latest index
+ * against the same month a year earlier, never a guess from a shorter run.
+ *
+ * BLS asks to be cited and says it cannot vouch for the data once they leave BLS.gov: NOTE says both wherever a figure
+ * shows. Pulls 35 seconds after start and then every six hours. Off without OV_SOURCES_INTERNAL_URL and the OAuth
+ * client secret (a test passes its own source).
+ */
+const ingest = require('openvibe-publishing/ingest');
+
+const SOURCE_KEY = 'bls-cpi-rent';
+const CURSOR = 'sources.housing.rentcpi.after';
+const EVERY_MS = 6 * 60 * 60 * 1000;
+const FIRST_MS = 35 * 1000;
+const NOTE = 'Rent of primary residence CPI from the U.S. Bureau of Labor Statistics (public domain). BLS.gov cannot vouch for the data or analyses derived from these data after the data have been retrieved from BLS.gov.';
+const BLS_URL = 'https://data.bls.gov/timeseries/';
+const MONTHS = ['January', 'February', 'March', 'April', 'May', 'June', 'July', 'August', 'September', 'October', 'November', 'December'];
+
+const NATIONAL = { series: 'CUUR0000SEHA', name: 'US city average' };
+const METROS = [
+ { series: 'CUURS12ASEHA', name: 'New York-Newark-Jersey City, NY-NJ-PA' },
+ { series: 'CUURS49ASEHA', name: 'Los Angeles-Long Beach-Anaheim, CA' },
+ { series: 'CUURS23ASEHA', name: 'Chicago-Naperville-Elgin, IL-IN-WI' },
+ { series: 'CUURS37ASEHA', name: 'Dallas-Fort Worth-Arlington, TX' },
+ { series: 'CUURS37BSEHA', name: 'Houston-The Woodlands-Sugar Land, TX' },
+ { series: 'CUURS35BSEHA', name: 'Miami-Fort Lauderdale-West Palm Beach, FL' },
+ { series: 'CUURS35CSEHA', name: 'Atlanta-Sandy Springs-Roswell, GA' },
+ { series: 'CUURS12BSEHA', name: 'Philadelphia-Camden-Wilmington, PA-NJ-DE-MD' },
+ { series: 'CUURS35ASEHA', name: 'Washington-Arlington-Alexandria, DC-VA-MD-WV' },
+ { series: 'CUURS11ASEHA', name: 'Boston-Cambridge-Newton, MA-NH' },
+ { series: 'CUURS24ASEHA', name: 'Minneapolis-St.Paul-Bloomington, MN-WI' },
+ { series: 'CUURS23BSEHA', name: 'Detroit-Warren-Dearborn, MI' },
+ { series: 'CUURS48ASEHA', name: 'Phoenix-Mesa-Scottsdale, AZ' },
+ { series: 'CUURS48BSEHA', name: 'Denver-Aurora-Lakewood, CO' },
+ { series: 'CUURS49BSEHA', name: 'San Francisco-Oakland-Hayward, CA' },
+ { series: 'CUURS49DSEHA', name: 'Seattle-Tacoma-Bellevue, WA' },
+ { series: 'CUURS49ESEHA', name: 'San Diego-Carlsbad, CA' },
+ { series: 'CUURA104SEHA', name: 'Pittsburgh, PA' },
+ { series: 'CUURA210SEHA', name: 'Cleveland-Akron, OH' },
+];
+const BY_SERIES = new Map([NATIONAL, ...METROS].map((m) => [m.series, m]));
+
+const text = (v, max) => (v == null || v === '' ? null : String(v).replace(/\s+/g, ' ').trim().slice(0, max) || null);
+const norm = (v) => String(v == null ? '' : v).toLowerCase().replace(/[^a-z0-9]+/g, ' ').trim();
+const monthKey = (year, period) => Number(year) * 12 + Number(String(period).slice(1)) - 1;
+
+/** A city of the Rent list → the BLS metro area whose own name lists that city and state, or null. */
+function metroFor(city) {
+ if (!city || !city.name || !city.state) return null;
+ const name = norm(city.name);
+ const state = String(city.state).toUpperCase();
+ for (const metro of METROS) {
+ const [cities, states] = metro.name.split(',');
+ if (!cities || !states) continue;
+ if (!states.trim().split('-').includes(state)) continue;
+ if (cities.split('-').map(norm).includes(name)) return metro;
+ }
+ return null;
+}
+
+/** A Sources item → a rent_cpi_points row, or null when it is not a series and month this trend uses. */
+function rowOf(item) {
+ if (!item || item.source_key !== SOURCE_KEY || item.kind !== 'rent_cpi') return null;
+ const f = item.fields || {};
+ const series = text(f.series_id, 20);
+ if (!series || !BY_SERIES.has(series)) return null;
+ const year = Number(f.year);
+ const period = text(f.period, 3);
+ const value = Number(f.value);
+ if (!Number.isInteger(year) || year < 2000 || year > 2100) return null;
+ if (!/^M(0[1-9]|1[0-2])$/.test(period || '')) return null;
+ if (!Number.isFinite(value) || value <= 0 || value > 100000) return null;
+ return {
+ series_id: series,
+ year,
+ period,
+ value: Math.round(value * 1000) / 1000,
+ item_id: String(item.id || ''),
+ revision: Number(item.revision) || 1,
+ retrieved_at: (item.provenance && item.provenance.retrieved_at) || new Date(0).toISOString(),
+ };
+}
+
+function pointOf(row) {
+ return { year: Number(row.year), period: row.period, value: Number(row.value), month: monthKey(row.year, row.period), label: `${MONTHS[Number(row.period.slice(1)) - 1]} ${row.year}` };
+}
+
+/** The latest point and the 12-month change, with the points a trend line draws (up to the last 13 months). */
+function trendOf(rows) {
+ const points = rows.map(pointOf).sort((a, b) => a.month - b.month);
+ if (!points.length) return null;
+ const latest = points[points.length - 1];
+ const earlier = points.find((p) => p.month === latest.month - 12) || null;
+ const change_pct = earlier && earlier.value ? Math.round((latest.value / earlier.value - 1) * 1000) / 10 : null;
+ return {
+ points: points.slice(-13),
+ latest,
+ earlier,
+ change_pct,
+ series: rows[0].series_id,
+ };
+}
+
+function phrasing(area, trend) {
+ if (!trend) return null;
+ const month = trend.latest.label;
+ if (trend.change_pct == null) return `BLS's rent index for ${area} stands at ${trend.latest.value} for ${month}; a 12-month change needs the same month a year earlier.`;
+ const verb = trend.change_pct > 0 ? 'rose' : trend.change_pct < 0 ? 'fell' : 'was unchanged';
+ const amount = trend.change_pct === 0 ? '' : ` ${Math.abs(trend.change_pct).toFixed(1)}%`;
+ return `Rents in ${area} ${verb}${amount} in the last year (BLS CPI, ${month}).`;
+}
+
+function createRentCpi({ config, s, fetchImpl = globalThis.fetch, log = console, source = null }) {
+ const url = String((config.sources && config.sources.internalUrl) || '').replace(/\/+$/, '');
+ const client = source || (url && config.oauth.clientSecret
+ ? ingest.createSourcesClient({ config: { ...config, sources: { internalUrl: url, category: 'housing' } }, fetchImpl })
+ : null);
+ const cursor = ingest.createChangeCursor(s.db, { prefix: 'rent', now: s.now });
+ const stats = { pulls: 0, applied: 0, removed: 0, last_at: null, last_error: null };
+ let bySeries = new Map();
+ let timer = null;
+ let first = null;
+ let running = false;
+
+ async function apply(item, t) {
+ if (!item || item.source_key !== SOURCE_KEY || item.kind !== 'rent_cpi') return 'hold';
+ if (item.removed) {
+ await t.query('DELETE FROM rent_cpi_points WHERE item_id = $1', [String(item.id)]);
+ return 'removed';
+ }
+ const row = rowOf(item);
+ if (!row) return 'hold';
+ const cols = Object.keys(row);
+ await t.query(`INSERT INTO rent_cpi_points (${cols.join(', ')}) VALUES (${cols.map((_, i) => `$${i + 1}`).join(', ')})
+ ON CONFLICT (series_id, year, period) DO UPDATE SET ${cols.filter((c) => !['series_id', 'year', 'period'].includes(c)).map((c) => `${c} = EXCLUDED.${c}`).join(', ')}
+ WHERE rent_cpi_points.item_id <> EXCLUDED.item_id OR rent_cpi_points.revision <= EXCLUDED.revision`, cols.map((c) => row[c]));
+ return 'applied';
+ }
+
+ async function load() {
+ const rows = await s.db.many('SELECT series_id, year, period, value, item_id, revision, retrieved_at FROM rent_cpi_points');
+ const grouped = new Map();
+ for (const row of rows) {
+ if (!BY_SERIES.has(row.series_id)) continue;
+ if (!grouped.has(row.series_id)) grouped.set(row.series_id, []);
+ grouped.get(row.series_id).push(row);
+ }
+ bySeries = new Map([...grouped.entries()].map(([series, list]) => [series, trendOf(list)]));
+ return bySeries.size;
+ }
+
+ /** Follow Sources' change feed to its end. Never throws: the caller is a timer or a test. */
+ async function pull() {
+ if (!client || running) return { skipped: true };
+ running = true;
+ try {
+ const out = await ingest.pullChanges({ db: s.db, cursor, source: client, name: CURSOR, apply });
+ stats.pulls++;
+ stats.applied += out.applied;
+ stats.removed += out.removed;
+ stats.last_at = s.iso();
+ stats.last_error = null;
+ await load();
+ if (out.applied || out.removed) log.log(`[RentCpi] ${out.applied} BLS rent points updated, ${out.removed} removed (Sources cursor ${out.after})`);
+ return out;
+ } catch (err) {
+ stats.last_error = text(err && err.message, 300);
+ log.warn(`[RentCpi] pull from OpenVibe.Sources failed: ${stats.last_error}`);
+ return { error: stats.last_error };
+ } finally { running = false; }
+ }
+
+ /** A trend for one series: the latest point, its 12-month change and the line's points; null when unread. */
+ function forSeries(series) {
+ const trend = bySeries.get(series);
+ if (!trend) return null;
+ const def = BY_SERIES.get(series);
+ return { series, area: def.name, ...trend, text: phrasing(def.name, trend), url: `${BLS_URL}${series}` };
+ }
+
+ return {
+ enabled: Boolean(client),
+ stats,
+ pull,
+ apply,
+ load,
+ forSeries,
+ metroFor,
+ /** The national line and, where BLS publishes one, the city's metro line; the metro line is the one quoted. */
+ trendForCity(city) {
+ const national = forSeries(NATIONAL.series);
+ const metro = metroFor(city);
+ const local = metro ? forSeries(metro.series) : null;
+ const picked = local || national;
+ if (!picked) return null;
+ return {
+ scope: local ? 'metro' : 'national',
+ area: picked.area,
+ text: picked.text,
+ series: picked.series,
+ url: picked.url,
+ points: picked.points,
+ latest: picked.latest,
+ change_pct: picked.change_pct,
+ national,
+ metro: local,
+ };
+ },
+ start() {
+ load().catch(() => { /* the first pull loads it */ });
+ if (!client || timer) return;
+ first = setTimeout(() => { pull(); }, FIRST_MS);
+ timer = setInterval(() => { pull(); }, EVERY_MS);
+ if (first.unref) first.unref();
+ if (timer.unref) timer.unref();
+ },
+ stop() {
+ if (first) clearTimeout(first);
+ if (timer) clearInterval(timer);
+ first = null;
+ timer = null;
+ },
+ };
+}
+
+module.exports = { createRentCpi, rowOf, trendOf, metroFor, phrasing, METROS, NATIONAL, NOTE, BLS_URL, SOURCE_KEY };
@@ -34,11 +34,36 @@ const cell = (c, key) => {
@@ -34,11 +34,36 @@ const cell = (c, key) => {
};
};
const placeLabel = (c) => `${c.name}, ${c.state}`;
const placeLabel = (c) => `${c.name}, ${c.state}`;
const { HUD_URL } = require('../housing/fair-market-rents');
const { HUD_URL } = require('../housing/fair-market-rents');
+const { NOTE: CPI_NOTE, NATIONAL: CPI_NATIONAL } = require('../housing/rent-cpi');
const fmrText = (n) => (n == null ? null : `$${n.toLocaleString('en-US')}`);
const fmrText = (n) => (n == null ? null : `$${n.toLocaleString('en-US')}`);
/** What a Fair Market Rent is, under the city table (HUD's own definition and use, in short). */
/** What a Fair Market Rent is, under the city table (HUD's own definition and use, in short). */
const fmrSource = (area) => html`<p class="small muted city-source">HUD Fair Market Rent for the ${area.name}: what a modest rental there costs, rent plus essential utilities, by HUD's estimate. HUD sets them each October; Housing Choice Vouchers pay up to 90% to 110% of them. <a href="${HUD_URL}" rel="noopener">HUD</a> (public domain), through OpenVibe.Sources.</p>`;
const fmrSource = (area) => html`<p class="small muted city-source">HUD Fair Market Rent for the ${area.name}: what a modest rental there costs, rent plus essential utilities, by HUD's estimate. HUD sets them each October; Housing Choice Vouchers pay up to 90% to 110% of them. <a href="${HUD_URL}" rel="noopener">HUD</a> (public domain), through OpenVibe.Sources.</p>`;
-function mountCityPages(r, { config, page, fmr = null }) {
+/** A small line for a BLS CPI trend: one point per month, the last 13 at most. */
+function trendSvg(trend, label) {
+ const points = (trend && trend.points) || [];
+ if (points.length < 2) return '';
+ const values = points.map((p) => Number(p.value));
+ const min = Math.min(...values);
+ const max = Math.max(...values);
+ const span = max - min || 1;
+ const x = (i) => 4 + (i * 312) / (points.length - 1);
+ const y = (v) => 60 - ((v - min) / span) * 56;
+ const poly = points.map((p, i) => `${x(i).toFixed(1)},${y(p.value).toFixed(1)}`).join(' ');
+ return html`<svg class="rent-trend" viewBox="0 0 320 64" width="320" height="64" role="img" aria-label="${label}"><polyline fill="none" stroke="currentColor" stroke-width="2" points="${poly}"/></svg>`;
+}
+
+/** The BLS trend lines a page shows: the national line and, where BLS publishes one, a city's metro line. */
+function cpiBlock(cpi) {
+ if (!cpi) return '';
+ const lines = [cpi.national, cpi.metro].filter(Boolean).filter((t, i, all) => all.findIndex((x) => x.series === t.series) === i);
+ if (!lines.length) return '';
+ return html`<section class="sc-sec rent-cpi" aria-labelledby="h-cpi"><h2 id="h-cpi">Are rents rising?</h2>
+${lines.map((t) => html`<div class="rent-cpi-line"><p>${t.text} <a href="${t.url}" rel="noopener">BLS series ${t.series}</a></p>${trendSvg(t, `Rent of primary residence CPI, ${t.area}, last 13 months`)}</div>`)}
+<p class="small muted city-source">Rent of primary residence CPI from the <a href="${lines[0].url}" rel="noopener">U.S. Bureau of Labor Statistics</a> (public domain), through OpenVibe.Sources. ${CPI_NOTE}</p></section>`;
+}
+
+function mountCityPages(r, { config, page, fmr = null, rentCpi = null }) {
const base = () => String(config.baseUrl).replace(/\/+$/, '');
const base = () => String(config.baseUrl).replace(/\/+$/, '');
const crumbsLd = (items) => seo.jsonLd.breadcrumbs(items.map((i) => ({ name: i.label, url: `${base()}${i.href}` })));
const crumbsLd = (items) => seo.jsonLd.breadcrumbs(items.map((i) => ({ name: i.label, url: `${base()}${i.href}` })));
@@ -52,6 +77,7 @@ ${city ? html`<p class="small muted fair-how">Within the Census' margin of the m
@@ -52,6 +77,7 @@ ${city ? html`<p class="small muted fair-how">Within the Census' margin of the m
// ── /in: every state ──
// ── /in: every state ──
r.get('/in', (req, res) => {
r.get('/in', (req, res) => {
const largest = [...cities.cities].sort((a, b) => b.population - a.population).slice(0, 30);
const largest = [...cities.cities].sort((a, b) => b.population - a.population).slice(0, 30);
+ const national = rentCpi ? rentCpi.forSeries(CPI_NATIONAL.series) : null;
page(req, res, {
page(req, res, {
title: 'Rent by city: typical rents across the US',
title: 'Rent by city: typical rents across the US',
description: `What rent usually costs in ${cities.cities.length} US cities, by number of bedrooms, from the Census Bureau, and every rental site's search for each city in one place.`,
description: `What rent usually costs in ${cities.cities.length} US cities, by number of bedrooms, from the Census Bureau, and every rental site's search for each city in one place.`,
@@ -65,7 +91,7 @@ ${city ? html`<p class="small muted fair-how">Within the Census' margin of the m
@@ -65,7 +91,7 @@ ${city ? html`<p class="small muted fair-how">Within the Census' margin of the m
<div class="table-wrap"><table class="city-table"><caption class="sr-only">Typical rents in the 30 largest US cities</caption>
<div class="table-wrap"><table class="city-table"><caption class="sr-only">Typical rents in the 30 largest US cities</caption>
<thead><tr><th scope="col">City</th><th scope="col">1 bedroom</th><th scope="col">2 bedrooms</th><th scope="col">3 bedrooms</th></tr></thead>
<thead><tr><th scope="col">City</th><th scope="col">1 bedroom</th><th scope="col">2 bedrooms</th><th scope="col">3 bedrooms</th></tr></thead>
<tbody>${largest.map((c) => html`<tr><th scope="row"><a href="${c.path}">${placeLabel(c)}</a></th><td>${cell(c, 'b1')}</td><td>${cell(c, 'b2')}</td><td>${cell(c, 'b3')}</td></tr>`)}</tbody></table></div>
<tbody>${largest.map((c) => html`<tr><th scope="row"><a href="${c.path}">${placeLabel(c)}</a></th><td>${cell(c, 'b1')}</td><td>${cell(c, 'b2')}</td><td>${cell(c, 'b3')}</td></tr>`)}</tbody></table></div>
-${source()}</section>`,
+${source()}</section>${cpiBlock({ national, metro: null })}`,
});
});
});
});
@@ -104,6 +130,7 @@ ${source()}
@@ -104,6 +130,7 @@ ${source()}
const near = cities.nearby(c, 8);
const near = cities.nearby(c, 8);
const two = cities.rentText(c.rent.b2);
const two = cities.rentText(c.rent.b2);
const area = fmr ? fmr.forCity(c) : null;
const area = fmr ? fmr.forCity(c) : null;
+ const cpi = rentCpi ? rentCpi.trendForCity(c) : null;
page(req, res, {
page(req, res, {
title: `Rent in ${where}: typical rents and every rental site`,
title: `Rent in ${where}: typical rents and every rental site`,
description: `${two ? `A typical 2-bedroom in ${where} rents for ${two} a month. ` : ''}Every rental site's search for ${c.name} in one place: Craigslist, Zillow, Apartments.com, Zumper and more, with typical rents by size and affordable housing nearby.`,
description: `${two ? `A typical 2-bedroom in ${where} rents for ${two} a month. ` : ''}Every rental site's search for ${c.name} in one place: Craigslist, Zillow, Apartments.com, Zumper and more, with typical rents by size and affordable housing nearby.`,
@@ -125,6 +152,7 @@ ${cities.SIZES.map((s) => html`<tr><th scope="row">${s.text}</th><td>${cell(c, s
@@ -125,6 +152,7 @@ ${cities.SIZES.map((s) => html`<tr><th scope="row">${s.text}</th><td>${cell(c, s
${source()}`}</section>
${source()}`}</section>
<section class="fair-card" aria-labelledby="h-fair"><h2 id="h-fair">Is a rent fair?</h2><p class="small muted">Compare a price from any site with the middle rent in ${c.name}.</p>${fairForm({}, { city: c })}</section>
<section class="fair-card" aria-labelledby="h-fair"><h2 id="h-fair">Is a rent fair?</h2><p class="small muted">Compare a price from any site with the middle rent in ${c.name}.</p>${fairForm({}, { city: c })}</section>
</div>
</div>
+${cpiBlock(cpi)}
<section class="sc-sec" aria-labelledby="h-sites"><h2 id="h-sites">Apartments in ${c.name}, on every site</h2>
<section class="sc-sec" aria-labelledby="h-sites"><h2 id="h-sites">Apartments in ${c.name}, on every site</h2>
<ul class="everywhere-sites">${links.map((l) => html`<li class="ew-site"><a class="ew-open" href="${l.url}" target="_blank" rel="noopener nofollow"><span class="ew-name">${l.name}</span><span class="ew-host">${hostOf(l.url)} ↗</span></a><p class="ew-best">${l.best}</p></li>`)}</ul>
<ul class="everywhere-sites">${links.map((l) => html`<li class="ew-site"><a class="ew-open" href="${l.url}" target="_blank" rel="noopener nofollow"><span class="ew-name">${l.name}</span><span class="ew-host">${hostOf(l.url)} ↗</span></a><p class="ew-best">${l.best}</p></li>`)}</ul>
<nav class="everywhere-kinds" aria-label="Other things to rent in ${c.name}">${everywhere.KINDS.filter((k) => k !== 'apartment').map((k) => html`<a class="ew-kind" href="${kindHref(k)}">${everywhere.KIND_TEXT[k]}</a>`)}</nav>
<nav class="everywhere-kinds" aria-label="Other things to rent in ${c.name}">${everywhere.KINDS.filter((k) => k !== 'apartment').map((k) => html`<a class="ew-kind" href="${kindHref(k)}">${everywhere.KIND_TEXT[k]}</a>`)}</nav>
@@ -177,6 +205,7 @@ ${source()}`,
@@ -177,6 +205,7 @@ ${source()}`,
/** The verdict and a bar: the middle rent with its margin, and where the asked rent falls. */
/** The verdict and a bar: the middle rent with its margin, and where the asked rent falls. */
function fairResult(city, f, rent) {
function fairResult(city, f, rent) {
const area = fmr ? fmr.forCity(city) : null;
const area = fmr ? fmr.forCity(city) : null;
+ const cpi = rentCpi ? rentCpi.trendForCity(city) : null;
const max = Math.max(rent, f.median) * 1.35;
const max = Math.max(rent, f.median) * 1.35;
const pct = (n) => Math.max(0, Math.min(100, (n / max) * 100)).toFixed(1);
const pct = (n) => Math.max(0, Math.min(100, (n / max) * 100)).toFixed(1);
const lo = Math.max(0, f.median - f.moe); const hi = f.median + f.moe;
const lo = Math.max(0, f.median - f.moe); const hi = f.median + f.moe;
@@ -186,6 +215,7 @@ ${source()}`,
@@ -186,6 +215,7 @@ ${source()}`,
<div class="fair-bar" aria-hidden="true"><span class="fair-band" style="left:${pct(lo)}%;width:${(pct(hi) - pct(lo)).toFixed(1)}%"></span><span class="fair-median" style="left:${pct(f.median)}%"></span><span class="fair-you" style="left:${pct(rent)}%"></span></div>
<div class="fair-bar" aria-hidden="true"><span class="fair-band" style="left:${pct(lo)}%;width:${(pct(hi) - pct(lo)).toFixed(1)}%"></span><span class="fair-median" style="left:${pct(f.median)}%"></span><span class="fair-you" style="left:${pct(rent)}%"></span></div>
<p class="fair-legend small"><span class="lg lg-median"></span> Middle rent ${f.topped ? '$3,500 or more' : `$${f.median.toLocaleString('en-US')}`}${f.moe && !f.topped ? ` (±$${f.moe.toLocaleString('en-US')})` : ''} <span class="lg lg-you"></span> This rent $${rent.toLocaleString('en-US')}</p>
<p class="fair-legend small"><span class="lg lg-median"></span> Middle rent ${f.topped ? '$3,500 or more' : `$${f.median.toLocaleString('en-US')}`}${f.moe && !f.topped ? ` (±$${f.moe.toLocaleString('en-US')})` : ''} <span class="lg lg-you"></span> This rent $${rent.toLocaleString('en-US')}</p>
${area && fmrText(area.rents[f.size.key]) ? html`<p class="small">HUD's Fair Market Rent for ${f.size.long} in the ${area.name} is ${fmrText(area.rents[f.size.key])} a month, rent plus essential utilities: the figure a Housing Choice Voucher is measured against.</p>` : ''}
${area && fmrText(area.rents[f.size.key]) ? html`<p class="small">HUD's Fair Market Rent for ${f.size.long} in the ${area.name} is ${fmrText(area.rents[f.size.key])} a month, rent plus essential utilities: the figure a Housing Choice Voucher is measured against.</p>` : ''}
+${cpi && cpi.text ? html`<p class="small">${cpi.text} <a href="${cpi.url}" rel="noopener">BLS series ${cpi.series}</a>.</p>` : ''}
<p class="small"><a href="${city.path}">Typical rents and every rental site in ${city.name}</a></p>
<p class="small"><a href="${city.path}">Typical rents and every rental site in ${city.name}</a></p>
</section>`;
</section>`;
}
}
@@ -903,7 +903,7 @@ ${it.newCount ? html` <span class="badge info">${it.newCount} new</span>` : html
@@ -903,7 +903,7 @@ ${it.newCount ? html` <span class="badge info">${it.newCount} new</span>` : html
// ── The board: listings kept from any site (http/board-pages.js) ──
// ── The board: listings kept from any site (http/board-pages.js) ──
mountBoardPages(r, { config, s, page, signedIn, requesterOf, mustSignIn, budgeted, sameSite, signInPrompt });
mountBoardPages(r, { config, s, page, signedIn, requesterOf, mustSignIn, budgeted, sameSite, signInPrompt });
- mountCityPages(r, { config, page, fmr: ctx.fmr });
+ mountCityPages(r, { config, page, fmr: ctx.fmr, rentCpi: ctx.rentCpi });
// ── Reporting ────────────────────────────────────────────
// ── Reporting ────────────────────────────────────────────
r.post('/listings/:id/reports', mustSignIn('/listings'), budgeted('rent.listing.report'), async (req, res) => {
r.post('/listings/:id/reports', mustSignIn('/listings'), budgeted('rent.listing.report'), async (req, res) => {
@@ -44,6 +44,7 @@ async function start() {
@@ -44,6 +44,7 @@ async function start() {
ctx.screen.start(); // the AI listing screen's sweep (off without OpenVibe.AI)
ctx.screen.start(); // the AI listing screen's sweep (off without OpenVibe.AI)
ctx.fmr.start(); // HUD's Fair Market Rents: loaded now, then followed from OpenVibe.Sources every six hours
ctx.fmr.start(); // HUD's Fair Market Rents: loaded now, then followed from OpenVibe.Sources every six hours
ctx.safmr.start(); // HUD's Small Area Fair Market Rents: followed from OpenVibe.Sources every six hours
ctx.safmr.start(); // HUD's Small Area Fair Market Rents: followed from OpenVibe.Sources every six hours
+ ctx.rentCpi.start(); // BLS CPI rent: the national and metro trend line, followed from OpenVibe.Sources every six hours
// One sweep at boot (a restart must not leave a stale listing up a moment longer than it should), then a timer.
// One sweep at boot (a restart must not leave a stale listing up a moment longer than it should), then a timer.
const expiry = startExpiryTimer(ctx.s);
const expiry = startExpiryTimer(ctx.s);
@@ -56,7 +57,7 @@ async function start() {
@@ -56,7 +57,7 @@ async function start() {
// Subscribe to the two ADR-033 topics at OpenVibe.Events (idempotent; off without RENT_EVENTS_URL and
// Subscribe to the two ADR-033 topics at OpenVibe.Events (idempotent; off without RENT_EVENTS_URL and
// RENT_EVENTS_SECRET). The consumer itself is mounted in server/app.js.
// RENT_EVENTS_SECRET). The consumer itself is mounted in server/app.js.
const subscriptions = require('./events-consumer').startSubscriptions({ config, port: config.port, secret: config.events.secrets[0] || '' });
const subscriptions = require('./events-consumer').startSubscriptions({ config, port: config.port, secret: config.events.secrets[0] || '' });
- const extra = [() => { if (subscriptions) subscriptions.stop(); }, () => ctx.fmr.stop(), () => ctx.safmr.stop()];
+ const extra = [() => { if (subscriptions) subscriptions.stop(); }, () => ctx.fmr.stop(), () => ctx.safmr.stop(), () => ctx.rentCpi.stop()];
createLifecycle({ server, ctx, timers: [expiry.timer, ctx.housingPruneTimer, ...(hpiJob ? [hpiJob.timer] : [])], extra });
createLifecycle({ server, ctx, timers: [expiry.timer, ctx.housingPruneTimer, ...(hpiJob ? [hpiJob.timer] : [])], extra });
return { server, ctx, expiry };
return { server, ctx, expiry };
}
}
@@ -38,7 +38,7 @@ async function boot(opts = {}) {
@@ -38,7 +38,7 @@ async function boot(opts = {}) {
const { createStore } = require('../../server/db');
const { createStore } = require('../../server/db');
const testdb = await require('./db').testDb();
const testdb = await require('./db').testDb();
const store = createStore(testdb.db, { now: opts.now });
const store = createStore(testdb.db, { now: opts.now });
- const built = await createApp({ config, store, log, ai: opts.ai, telemetrySink: opts.telemetrySink, indexnow: opts.indexnow, limitsNow: opts.limitsNow, callerLimits: opts.callerLimits === true, accountSend: opts.accountSend, housingFetchImpl: opts.housingFetchImpl, fmrSource: opts.fmrSource });
+ const built = await createApp({ config, store, log, ai: opts.ai, telemetrySink: opts.telemetrySink, indexnow: opts.indexnow, limitsNow: opts.limitsNow, callerLimits: opts.callerLimits === true, accountSend: opts.accountSend, housingFetchImpl: opts.housingFetchImpl, fmrSource: opts.fmrSource, rentCpiSource: opts.rentCpiSource });
await built.ctx.keys.ensure();
await built.ctx.keys.ensure();
const server = await new Promise((resolve) => { const s = http.createServer(built.app); s.listen(0, '127.0.0.1', () => resolve(s)); });
const server = await new Promise((resolve) => { const s = http.createServer(built.app); s.listen(0, '127.0.0.1', () => resolve(s)); });
const base = `http://127.0.0.1:${server.address().port}`;
const base = `http://127.0.0.1:${server.address().port}`;
new file mode 100644
new file mode 100644
@@ -0,0 +1,118 @@
@@ -0,0 +1,118 @@
+'use strict';
+/**
+ * BLS CPI rent (server/housing/rent-cpi.js): the national and metro trend lines from OpenVibe.Sources.
+ * - a bls-cpi-rent item becomes one point per series and month; another series, kind or source is held;
+ * - the 12-month change is the latest index against the same month a year earlier;
+ * - a city maps to its BLS metro by the area name's own cities and states, else the national line is used;
+ * - the city and fair pages show "Rents in <area> rose X% in the last year (BLS CPI, <month>)" with a line
+ * and the BLS credit; a city page with no metro line shows the national one.
+ */
+const assert = require('assert');
+const { boot, check, done } = require('./helpers/boot');
+const cities = require('../server/aggregate/cities');
+const { createRentCpi, rowOf, trendOf, metroFor, phrasing, NOTE } = require('../server/housing/rent-cpi');
+
+const item = (series, year, period, value, over = {}) => ({
+ id: `itm_${series}_${year}${period}`, source_key: 'bls-cpi-rent', category: 'housing', kind: 'rent_cpi',
+ identity: `${series}-${year}${period}`, canonical_url: null, title: series, summary: null,
+ fields: { series_id: series, year: String(year), period, value: String(value) },
+ revision: 1, provenance: { retrieved_at: '2026-10-30T06:00:00.000Z' }, removed: null, ...over,
+});
+
+function feed(items) {
+ return {
+ items,
+ async listItems({ after = 0, limit = 100 }) {
+ const rest = items.filter((x) => x.change_seq > after).sort((a, b) => a.change_seq - b.change_seq);
+ const page = rest.slice(0, limit);
+ return { items: page, next_after: page.length ? page[page.length - 1].change_seq : after, more: rest.length > limit, sources: {} };
+ },
+ };
+}
+
+(async () => {
+ const items = [
+ { ...item('CUUR0000SEHA', 2025, 'M09', 400), change_seq: 1 },
+ { ...item('CUUR0000SEHA', 2026, 'M09', 420), change_seq: 2 },
+ { ...item('CUURS12ASEHA', 2025, 'M09', 500), change_seq: 3 },
+ { ...item('CUURS12ASEHA', 2026, 'M09', 520), change_seq: 4 },
+ { ...item('CUUR0000SEHA', 2026, 'M08', 419), change_seq: 5 },
+ { ...item('CUURS99XSEHA', 2026, 'M09', 999), change_seq: 6 }, // not one of the 19 metro series
+ { ...item('CUUR0000SEHA', 2026, 'M07', 418, { kind: 'record' }), change_seq: 7 },
+ { ...item('CUUR0000SEHA', 2026, 'M06', 417, { source_key: 'hud-fmr' }), change_seq: 8 },
+ ];
+ const source = feed(items);
+ const t = await boot({ rentCpiSource: source });
+ const ny = cities.match('New York, NY');
+ const boise = cities.match('Boise City, ID');
+ try {
+ await check('one point per series and month; the other series and kinds are held', async () => {
+ const row = rowOf(item('CUUR0000SEHA', 2026, 'M09', 450.402));
+ assert.deepStrictEqual([row.series_id, row.year, row.period, row.value], ['CUUR0000SEHA', 2026, 'M09', 450.402]);
+ assert.strictEqual(rowOf(item('CUURS99XSEHA', 2026, 'M09', 999)), null, 'a series Rent does not map');
+ assert.strictEqual(rowOf(item('CUUR0000SEHA', 2026, 'M13', 450)), null, 'M13 is the annual average, not a month');
+ assert.strictEqual(rowOf(item('CUUR0000SEHA', 2026, 'M09', -1)), null);
+ const out = await t.ctx.rentCpi.pull();
+ assert.deepStrictEqual([out.applied, out.hold], [5, 3]);
+ assert.strictEqual(Number(await t.ctx.s.db.value("SELECT cursor FROM rent_ingest_cursor WHERE name = 'sources.housing.rentcpi.after'")), 8);
+ });
+
+ await check('the 12-month change compares the same month a year earlier', async () => {
+ const national = t.ctx.rentCpi.forSeries('CUUR0000SEHA');
+ assert.deepStrictEqual([national.latest.label, national.latest.value, national.change_pct], ['September 2026', 420, 5]);
+ assert.strictEqual(national.text, 'Rents in US city average rose 5.0% in the last year (BLS CPI, September 2026).');
+ const nyTrend = t.ctx.rentCpi.forSeries('CUURS12ASEHA');
+ assert.strictEqual(nyTrend.change_pct, 4);
+ assert.strictEqual(trendOf([]), null);
+ assert.strictEqual(phrasing('Somewhere', { latest: { value: 1, label: 'September 2026' }, change_pct: null }).includes('12-month change'), true);
+ });
+
+ await check('a city maps to its BLS metro by the area name; another falls back to the national line', async () => {
+ assert.strictEqual(metroFor({ name: 'New York', state: 'NY' }).series, 'CUURS12ASEHA');
+ assert.strictEqual(metroFor({ name: 'Jersey City', state: 'NJ' }).series, 'CUURS12ASEHA');
+ assert.strictEqual(metroFor({ name: 'Boise City', state: 'ID' }), null);
+ assert.strictEqual(metroFor({ name: 'St. Paul', state: 'MN' }).series, 'CUURS24ASEHA');
+ const local = t.ctx.rentCpi.trendForCity(ny);
+ assert.deepStrictEqual([local.scope, local.area, local.change_pct], ['metro', 'New York-Newark-Jersey City, NY-NJ-PA', 4]);
+ const national = t.ctx.rentCpi.trendForCity(boise);
+ assert.deepStrictEqual([national.scope, national.area, national.change_pct], ['national', 'US city average', 5]);
+ });
+
+ await check('the city page draws both lines with the BLS credit; a city BLS does not publish shows the national one', async () => {
+ const page = await t.get(ny.path);
+ assert.strictEqual(page.status, 200);
+ assert.ok(page.text.includes('Are rents rising?'));
+ assert.ok(page.text.includes('Rents in New York-Newark-Jersey City, NY-NJ-PA rose 4.0% in the last year (BLS CPI, September 2026).'));
+ assert.ok(page.text.includes('Rents in US city average rose 5.0% in the last year (BLS CPI, September 2026).'));
+ assert.ok(page.text.includes('BLS series CUURS12ASEHA'));
+ assert.ok(page.text.includes('<svg class="rent-trend"'), 'a trend line');
+ assert.ok(page.text.includes('BLS.gov cannot vouch'));
+ const boisePage = await t.get(boise.path);
+ assert.ok(boisePage.text.includes('Rents in US city average rose 5.0%'));
+ assert.ok(!boisePage.text.includes('CUURS12ASEHA'), 'no metro line where BLS has none');
+ const all = await t.get('/in');
+ assert.ok(all.text.includes('Are rents rising?') && all.text.includes('BLS series CUUR0000SEHA'), 'the national line is on /in too');
+ });
+
+ await check('the fair page names the same 12-month change', async () => {
+ const page = await t.get(`/fair?city=${encodeURIComponent(`${ny.state.toLowerCase()}/${ny.slug}`)}&beds=1&rent=2000`);
+ assert.strictEqual(page.status, 200);
+ assert.ok(page.text.includes('Rents in New York-Newark-Jersey City, NY-NJ-PA rose 4.0% in the last year (BLS CPI, September 2026).'));
+ assert.ok(page.text.includes('BLS series CUURS12ASEHA'));
+ });
+
+ await check('the points survive a restart, and a removed item is deleted', async () => {
+ const again = createRentCpi({ config: { sources: { internalUrl: '' } }, s: t.ctx.s, log: { log() {}, warn() {} } });
+ assert.strictEqual(await again.load(), 2);
+ assert.strictEqual(again.forSeries('CUUR0000SEHA').latest.value, 420);
+ source.items.push({ ...item('CUURS12ASEHA', 2026, 'M09', 520, { id: 'itm_CUURS12ASEHA_2026M09', removed: { at: '2026-10-30T07:00:00.000Z', reason: 'licence' }, revision: 2 }), change_seq: 9 });
+ const out = await t.ctx.rentCpi.pull();
+ assert.strictEqual(out.removed, 1);
+ assert.strictEqual(t.ctx.rentCpi.forSeries('CUURS12ASEHA').latest.label, 'September 2025', 'only the 2025 point remains');
+ assert.ok(NOTE.includes('BLS.gov cannot vouch'));
+ });
+ } finally {
+ await t.close();
+ }
+ done();
+})().catch((err) => { console.error(err); process.exit(1); });