openvibers/openvibe.rent AGENTS.md

AGENTS.md

main 11.4 KB · 157 lines · markdown

Raw HistoryBlame Repository
1# AGENTS.md — OpenVibe.Rent
2
3## Project Overview
4
5OpenVibe.Rent is the rental aggregator: every major rental site's own search for one place and filter set
6(`/everywhere`), person-posted listings that anyone can search, city rent pages with Census/HUD/BLS/FHFA context,
7"Is this rent fair?", an "Affordable housing near you" map-free guide from HUD open data, a personal board of
8listings from any site, and a scam screen. The same process also serves **OpenVibe.Homes** (`openvibe.homes`) for
9buying, chosen by Host header.
10
11- **Domain:** `openvibe.rent` (+ `openvibe.homes` in-process) · **Port:** 5010 · **Service id:** `rent` ·
12 **Env prefix:** `RENT`
13- **Runtime:** Node 22, Express 4, PostgreSQL 18 through `openvibe-sdk/db` (PGlite in development/tests), Valkey
14 optional for shared limits.
15- **Read first:** [README.md](README.md), [STATUS.json](STATUS.json), [.env.example](.env.example).
16
17**Network context:** accounts, sign-in and the staff role are OpenVibe.Network (OAuth client `rent`, PKCE S256; Homes
18has its own redirect URI on `openvibe.homes`). Account export/deletion arrives from OpenVibe.Events at
19`/internal/events`. The listing scam screen calls OpenVibe.AI (`rent.screen_listing`). Public market data is fetched
20from OSM Nominatim, HUD ArcGIS, FHFA and BLS — paced, cached and attributed. Rent stores no account, no role, no file
21or image, and owns no event delivery.
22
23## Commands
24
25```bash
26npm ci # install (contracts/sdk/shared pinned by tag)
27npm run dev # http://localhost:5010, PGlite under data/
28npm start # node server/index.js
29npm test # every test/*.test.js on temp PGlite, mock Network/AI/upstreams
30npm run test:pg # the same through PostgreSQL + PgBouncer
31npm test -- board # only test files whose name contains the word
32node --check <file.js> # syntax check (no linter configured)
33```
34
35No build step: pages are server-rendered; CSS only under [public/css/](public/css/) (`app`, `board`, `cities`,
36`everywhere`, `home`, `homes`, `housing`). Every page works without JavaScript.
37
38## Architecture at a Glance
39
40- **Entry:** [server/index.js](server/index.js) — listener, telemetry, the listing-expiry sweep and the HPI timer,
41 Events subscriptions and graceful stop. `RENT_DRILL` keeps boot-time work off; the expiry sweep is started **only**
42 here, never by a page render.
43- **App factory:** [server/app.js](server/app.js) — CSP, SSO with per-host auth routes (Rent and Homes), `/auth`,
44 `/internal/events`, `/shared`+static, `/api/v1`, host routing for openvibe.homes, pages, 404/error handler.
45- **Config:** [server/config.js](server/config.js) — `load(env)` is pure; production `/etc/openvibe/rent.env`.
46- **Listings:** [server/listings/](server/listings/) — `listings.js` (rules/validation), `store.js`, `service.js`,
47 `screen.js` (AI scam screen), `staff.js`, `expiry.js`.
48- **Market data:** [server/housing/](server/housing/) — `places.js`, `upstream.js` (paced/identified), `store.js`,
49 `hpi.js` (FHFA), `fair-market-rents.js` (HUD), `safmr.js`, `rent-cpi.js` (BLS); city aggregation
50 [server/aggregate/cities.js](server/aggregate/cities.js), [server/aggregate/everywhere.js](server/aggregate/everywhere.js).
51- **Homes:** [server/homes/](server/homes/) — `pages.js`, `city-pages.js`, `sites.js`, `mortgage.js`.
52- **Board and saved searches:** [server/board/board.js](server/board/board.js).
53- **HTTP:** [server/http/api.js](server/http/api.js), [server/http/pages.js](server/http/pages.js),
54 [server/http/city-pages.js](server/http/city-pages.js), [server/http/board-pages.js](server/http/board-pages.js),
55 [server/http/principal.js](server/http/principal.js), [server/http/discovery.js](server/http/discovery.js),
56 [server/http/router.js](server/http/router.js), [server/http/caller-limits.js](server/http/caller-limits.js).
57- **Auth/account data:** [server/auth/sso.js](server/auth/sso.js), [server/auth/keys.js](server/auth/keys.js),
58 [server/identity/account-data.js](server/identity/account-data.js), [server/events-consumer.js](server/events-consumer.js).
59- **Frontend/rendering:** [server/render/](server/render/) (`layout.js`, `html.js`, `home-blocks.js`, `short.js`,
60 `markdown.js`), agent twins [server/agent-pages.js](server/agent-pages.js),
61 [server/agent-descriptor.js](server/agent-descriptor.js), Homes descriptor
62 [server/agent-descriptor.homes.js](server/agent-descriptor.homes.js).
63- **Database:** [server/db.js](server/db.js) and [migrations/](migrations/): `rent_listings`, `rent_reports`,
64 `rent_listing_checks`, `rent_board_items`, `rent_saved_searches`, `housing_geo_cache`, `area_price_index`,
65 `rent_fmr_areas`, `rent_ingest_cursor`, `rent_cpi_points`, `rent_index_revisions`, `event_outbox`,
66 `account_data_events`, `telemetry_samples`.
67- **Deploy:** [deploy/systemd/openvibe-rent.service](deploy/systemd/openvibe-rent.service) (127.0.0.1:5010,
68 `/etc/openvibe/rent.env`, `/opt/openvibe.rent`), [deploy/nginx/openvibe.rent.conf](deploy/nginx/openvibe.rent.conf)
69 and [deploy/nginx/openvibe.homes.conf](deploy/nginx/openvibe.homes.conf). `sudo ovhost deploy rent`; database
70 `ov_rent`; rollback when `/api/ready` fails.
71
72## Ownership and Boundaries
73
74Rent owns listings, reports, AI screens, the board, saved searches, the market caches (geocode, HPI, FMR, CPI), the
75index sequencer and its outbox. It does **not** own accounts/roles (Network), the site frame (Network), event delivery
76(Events), files/images (Media), the external market data (OSM/HUD/FHFA/BLS/Census, read-only and attributed), or the
77host/database (Host). A listing carries **no image**; `contact_url` is text and is never fetched. The interface says
78plainly that Rent takes no payment from either side of a listing.
79
80## Auth and Capabilities
81
82- **No `rent.*` capability exists and none is needed.** Every write is a person acting for themself (or staff by
83 Network role); an app/agent/service token cannot post, edit, report or save. `CAPABILITIES` in
84 [server/http/principal.js](server/http/principal.js) is empty on purpose — do not add one without Contracts.
85- A person is a Network Bearer token or the `rent_at` session; a cookie write must be same-origin. Homes has its own
86 OAuth redirect and auth host.
87- Outbound: `events.subscription.manage` only (Network/Events endpoints); the AI screen uses a client-credentials token
88 for `rent.screen_listing`. No other service capability.
89- Listing rules: 10 active listings, 5 posted a day, 30 days each, 3 reports hide one until staff review, 30 per
90 search page. **A street address is refused** — listings are area-level; never add an exact-address field.
91- AI screen: reads the listing text only — **never the poster or the contact link** — and a `review` flag puts it in
92 the staff queue with the signs and quoted words. It changes nothing else for anyone; staff can mark it fine.
93- Errors are RFC 9457 problem+json with stable codes.
94
95## API (`/api/v1`)
96
97`GET /ping`, `GET /housing?q=&kind=`, `GET /listings?q=&kind=&city=&country=&min=&max=&period=&limit=&before=`,
98`GET /listings/:id` (contact link only for a signed-in person), `POST /listings`, `PATCH|DELETE /listings/:id`,
99`POST /listings/:id/renew`, `POST /listings/:id/reports`,
100`GET|POST /saved-searches`, `DELETE /saved-searches/:id`, `GET|POST /board`, `PATCH|DELETE /board/:id`,
101staff: `GET /staff/reports`, `POST /listings/:id/state`. Machine endpoints `/api/health`, `/api/ready`,
102`/release.json`, `/metrics` are loopback only.
103
104Pages: `/`, `/listings`, `/listings/:id`, `/post`, `/mine`, `/saved`, `/safety`, `/board`, `/staff`, `/updates`,
105`/in`, `/in/<state>`, `/in/<state>/<city>`, `/fair`, `/everywhere`, the housing pages and the openvibe.homes pages;
106discovery (`/llms.txt`, sitemaps, robots, Markdown twins) is generated per host.
107
108## Conventions
109
110- CommonJS, 4-space indent, single quotes, semicolons; no linter/formatter. Async handlers through the router.
111- New schema = a new `migrations/NNNN_name.sql`; never edit one that has run. Keep `rent_`/`housing_` prefixes.
112- External data is paced (`RENT_HOUSING_MIN_INTERVAL_MS`, one request a second per host), cached and identified with
113 `RENT_USER_AGENT` ("OpenVibeRent/…", never a browser's). HUD and FHFA are public domain; OSM is ODbL and credited;
114 Census/HUD/BLS lines stay separate and labelled — never averaged into one number.
115- Never fetch a user-supplied URL (a listing link, a board link): store it, render it as a link, and leave it alone.
116- A contact URL is shown only to a signed-in person; a hostile title/description is escaped everywhere (page, API,
117 JSON-LD, Search document).
118- CSP: add origins to `helmet` in [server/app.js](server/app.js) per host. No Font Awesome; shared glyphs.
119- Phone first: listing, board, city and housing pages fit 390 px; empty states use the shared kit.
120- "Calm reduces, only Off removes": a hidden/expired listing is suppressed, not deleted; only the owner's explicit
121 delete (or staff `removed`) removes it.
122- A fair-rent answer shows the median, its margin of error and the size it applies to; no invented precision.
123
124## Key Pitfalls
125
126- **Expiry** runs only from [server/listings/expiry.js](server/listings/expiry.js), started by
127 [server/index.js](server/index.js). Do not move it into a page or an API read (tests would race it).
128- **Reports hide at three** (`rent_reports_once` unique per person per listing); staff restore, and the count is never
129 shown to the public.
130- **Board:** one row per person per link, at most 500; a repeated link updates the same row (200), a different existing
131 link is 409 `board.duplicate`, full is 409 `board.full`. Board links come from any site; they are never fetched.
132- **Saved searches:** one row per distinct filter set; `new_count` compares against `last_seen_at`.
133- **AI screen** is bounded, never includes the poster/contact, caches the content hash so the same words are not
134 screened twice, and is off without the AI URL. A failed run is retried by the sweep, not silently ignored.
135- **Homes is a second host in this process:** add routes to the Homes router and its own auth/redirect, not to Rent's
136 pages; the two hosts must not leak each other's pages or discovery.
137- **Drill mode** (`RENT_DRILL`) keeps the expiry sweep, HPI timer, subscriptions and AI sweep off.
138- **`event_outbox`** needs its rejected partial index ([migrations/0012_outbox_rejected_index.sql](migrations/0012_outbox_rejected_index.sql)).
139- Account deletion removes a person's listings/board/saved searches and anonymizes public rows; redelivery changes
140 nothing.
141
142## Testing
143
144`npm test` runs each file in its own process on temp PGlite with mock Network/AI/upstreams: listing rules and caps,
145street-address refusal, search/facets/paging, contact-link visibility, reports and staff restore, expiry, owner-only
146edits, same-origin, saved searches, board, city pages, fair rent, housing/HUD/FMR/SAFMR/HPI/CPI, markets, AI screen,
147homes, error pages, discovery, SSO/JWKS, limits, secret-leak/session/redirect, service kit and budgets.
148`npm run test:pg` for PostgreSQL. Run the narrowest file first, then the suite; always `node --check` changed files.
149
150## Documentation
151
152- [README.md](README.md) — what it is, owns/does-not-own, data sources and terms, capabilities, what works, API,
153 configuration, acceptance, deploy, account data, security.
154- [STATUS.json](STATUS.json) — stage, deployed flag, features and not-yet list.
155- [.env.example](.env.example) — variable names only.
156- [CHANGELOG.md](CHANGELOG.md), [SECURITY.md](SECURITY.md), [tools/](tools/).
157