README.md

main 11.3 KB · 178 lines · markdown

1	# OpenVibe.Shoes
2
3	> Every shoe, every market, the right price.
4
5	Live site: https://open.shoes · API docs: https://openvibe.services · For AI: https://open.shoes/llms.txt
6	Run it: `npm ci && npm run dev` (Node 22 or newer) · Tests: `npm test`
7
8	Status: v1 written and tested (see [STATUS.json](STATUS.json)).
9	Domain: `open.shoes` · Port: 5180 · Service id: `shoes` · Env prefix: `SHOES`
10	License: AGPL-3.0-only (same as every OpenVibe service).
11
12	OpenVibe.Shoes is the network's shoe resource: one page per model (`/shoe/<brand>-<model>`) with every store's
13	offer in one all-in price, a price history from Shoes' own daily snapshots, the size-price curve where sizes are
14	known, and cited specs, fit notes, release facts and care notes — or an honest "not known yet". A person can ask
15	to be told when the model drops below a price at their size, or paste any listing link and have OpenVibe.Watch
16	keep it. Official release dates are a drops calendar with "remind me" on OpenVibe.Today. Shoes never enters a
17	raffle or a draw, never scrapes Facebook Marketplace, and carries no affiliate links.
18
19	## What it reads
20
21	\| Source \| What \| Key \|
22	\|---\|---\|---\|
23	\| OpenVibe.Deals \| the network's own offer aggregator: its public newest-offer pages and each product's public record (brand). No key. \| no \|
24	\| eBay Browse API \| official item summaries, only when a developer key pair is set \| `SHOES_EBAY_CLIENT_ID` + `SHOES_EBAY_CLIENT_SECRET` \|
25	\| A brand's or a store's own release page \| the official release calendar (Nike SNKRS by default), parsed by a pure parser that never invents a date \| no \|
26	\| OpenVibe.Watch \| a watch for a person, created with their own Network token \| the person's session \|
27	\| OpenVibe.Today \| a drop reminder on the person's own agenda (their Network token) \| the person's session \|
28
29	Every offer keeps its store, its currency, the price, the shipping the store stated, the condition, the size and
30	colourway when the source names them, the source's own check time and a stale flag. Every price is all-in:
31	price + shipping when shipping is stated. A missing key turns that part off with a plain note; it is never an
32	error. No source is a scrape: Deals reads stores through the network's sources, eBay is the official API, and a
33	pasted Marketplace link is handed to Watch, which applies its own fetch rules.
34
35	## Pages
36
37	\| Route \| What it shows \|
38	\|---\|---\|
39	\| `/` \| the room's front door: hero, live proof, the biggest price drops today, dropping this week, popular models, the three scenes (a size-price curve, a drop reminder on Today, a pasted link), explore, developers, questions \|
40	\| `/shoes?q=&brand=&size=&condition=&sort=&page=` \| the catalogue with facets; every offer all-in and dated \|
41	\| `/shoe/:slug` \| one model: offers by store, the daily price history (SVG from Shoes' own snapshots), the size-price curve, colourways, cited facts or "not known yet", and the watch form \|
42	\| `/drops` \| the official release calendar, grouped by day, with "remind me" (on Today) and the never-automated-entries note \|
43	\| `/track` \| paste a listing link; it becomes a Watch \|
44	\| `/watches` \| a signed-in person's watches (private, no-store) \|
45	\| `/developers` \| the API, sources and examples \|
46	\| `/laces` · `/lace-length` \| lacing diagrams and a source-cited length estimate from measured eyelet spacing \|
47	\| `/socks` \| brand-specific size conversion and use/material guidance \|
48	\| `/care` \| maker-cited care notes by material, with unknown compatibility stated clearly \|
49	\| `/kids` · `/kids/measure` \| manufacturer charts, a calibrated printable ruler, and an opt-in private Today remeasure reminder \|
50	\| `/api/openapi.json` \| the OpenAPI 3.1 document \|
51
52	All pages are server-rendered through `openvibe-shared/shell` and the room kit (`rooms.js`, `kit.card`,
53	`kit.rail`, `sc.room`), with the room id `shoes` and the accent `#FF7E76` (dark) / `#B02A2D` (light). Every page is
54	complete without JavaScript; forms are plain POSTs.
55
56	## API
57
58	Everything under `/api/v1`; errors are `application/problem+json` with a `code`. Public reads take no key.
59
60	\| Route \| Answer \|
61	\|---\|---\|
62	\| `GET /api/v1/shoes?q&brand&size&condition&store&sort&page&limit` \| `{ shoes, facets, page, per_page, total, sort, filters }`; sorts `recent`, `drop`, `price`, `price_desc`, `popular`, `name` \|
63	\| `GET /api/v1/shoes/:slug` \| `{ shoe, offers, history, curve, facts, drops }` \|
64	\| `GET /api/v1/shoes/:slug/price?size=&condition=` \| `{ slug, size, condition, price, currency, checked_at, offers }` — the URL a Watch polls \|
65	\| `GET /api/v1/products?kind=&q=&size=` \| shared catalog envelope with kinds `shoe`, `laces`, `socks`, `care`, `insoles`, `kids` (`shoes.product@1`) \|
66	\| `GET /api/v1/products/:id` · `/offers` \| one cited product or its offer list \|
67	\| `GET /api/v1/tools/lace-length?eyelet_pairs=&method=&horizontal_cm=&vertical_cm=&ends_cm=` \| measured lace estimate in cm/inches using a cited formula \|
68	\| `GET /api/v1/sizes/convert?brand=darn-tough&system=men\\|women\\|kids&size=` \| brand-chart conversion to sock size \|
69	\| `GET /api/v1/drops?from&to` \| `{ from, to, drops }`, soonest first \|
70	\| `GET /api/v1/drops/:id` \| one drop \|
71	\| `GET /api/v1/me/watches` \| session only: the person's watches \|
72	\| `POST /api/v1/me/watches` \| session only: `{ slug, size?, condition?, threshold, currency? }` → a price watch in OpenVibe.Watch \|
73	\| `POST /api/v1/me/track` \| session only: `{ url }` → a Watch for a pasted listing, through Watch's own rules \|
74	\| `GET /api/v1/status` \| the last read of each source, the counts, and whether eBay/Watch are configured \|
75	\| `GET /api/openapi.json` \| OpenAPI 3.1 \|
76
77	A signed-in write made with the session cookie must come from `open.shoes` (`Sec-Fetch-Site: same-origin`, or an
78	`Origin` equal to the site): another site cannot set a watch for a visitor. A bad Bearer is refused, never
79	downgraded to the session or anonymous.
80
81	## Jobs (`server/jobs/*`)
82
83	Each refuses to start under `SHOES_DRILL=1`; tests and `scripts/seed.js` call their `runOnce()`.
84
85	\| Job \| Reads \| Writes \| Default \|
86	\|---\|---\|---\|---\|
87	\| `offers` \| OpenVibe.Deals' public newest offers, then each candidate's public product for its brand \| a model, an offer with its all-in inputs, one snapshot per offer per day \| every 30 min, 20 pages \|
88	\| `drops` \| the official release calendar and any configured RSS feeds \| a drop with its source page and read time \| every 6 h \|
89	\| `ebay` \| the official Browse API (only with a key pair) \| an offer, the same as any other source \| every 60 min, off without keys \|
90
91	A job records each run in `shoe_job_runs` (source, start, finish, counts, error); `/api/v1/status` and the home FAQ
92	show it. The importer stops at the first page Deals answers empty, so a quiet source costs nothing.
93
94	## Database
95
96	PostgreSQL through `openvibe-sdk/db`, numbered migrations (`migrations/0001_initial.sql`). The tests use PGlite;
97	`npm run test:pg` runs the same suite through PgBouncer.
98
99	\| Table \| Holds \|
100	\|---\|---\|
101	\| `shoe_models` \| `shm_` + ULID, unique slug, brand, model, first seen \|
102	\| `shoe_offers` \| `sho_` + ULID, source + source id (unique), store, title, URL, price, list price, shipping, condition, size, colourway, checked time, stale, raw public item \|
103	\| `shoe_snapshots` \| `shs_` + ULID, offer, model, price, checked time, day — unique per offer per day (the first check wins) \|
104	\| `shoe_drops` \| `shd_` + ULID, unique slug, release time, price, sizes, source name/URL/key, read time \|
105	\| `shoe_watches` \| `shw_` + ULID, subject, kind `price\\|link`, model, size, condition, threshold, source URL, remote Watch id, status, note \|
106	\| `shoe_facts` \| cited specs/fit/release/care: kind, label, value, source name/URL, read time \|
107	\| `shoe_job_runs` \| one row per job run: source, start, finish, ok, counts, error \|
108	\| `account_data_events`, `telemetry_samples` \| the SDK's account receipts and request telemetry \|
109
110	## Environment
111
112	Only names in code and docs; values live in `/etc/openvibe/shoes.env` (see [.env.example](.env.example)).
113
114	- Service: `NODE_ENV`, `PORT` (5180), `HOST`, `BASE_URL`, `TRUST_PROXY`, `COOKIE_SECURE`, `SHOES_DRILL`.
115	- Database: `DATABASE_URL`, `DATABASE_DIRECT_URL`, `SHOES_PGLITE_DIR`, `VALKEY_URL`, `VALKEY_PREFIX`.
116	- Network: `OV_NETWORK_URL`, `OV_NETWORK_INTERNAL_URL`, `OV_NETWORK_ISSUER`, `OV_OAUTH_CLIENT_ID`,
117	`OV_OAUTH_CLIENT_SECRET`, `OV_OAUTH_REDIRECT_URI`, `OV_SESSION_AUDIENCE`, `SHOES_AUDIENCE`.
118	- Events: `SHOES_EVENTS_SECRET`, `SHOES_EVENTS_URL`, `SHOES_EVENTS_ENDPOINT`, `SHOES_EVENTS_INTERVAL_MS`.
119	- Sources: `SHOES_DEALS_URL`, `SHOES_OFFERS_PAGES`, `SHOES_PRODUCT_LOOKUPS`, `SHOES_STALE_HOURS`,
120	`SHOES_UPSTREAM_TIMEOUT_MS`, `SHOES_REQUEST_GAP_MS`, `SHOES_DROPS_URL`, `SHOES_DROPS_SOURCE_NAME`, `SHOES_DROP_FEEDS`,
121	`SHOES_EBAY_CLIENT_ID`, `SHOES_EBAY_CLIENT_SECRET`, `SHOES_EBAY_MARKETPLACE`, `SHOES_EBAY_CATEGORY_ID`,
122	`SHOES_EBAY_LIMIT`.
123	- Watch: `SHOES_WATCH_URL`, `SHOES_WATCH_INTERNAL_URL`, `SHOES_WATCH_EVERY_SEC`.
124	- Today: `SHOES_TODAY_URL`, `SHOES_TODAY_INTERNAL_URL`.
125	- Jobs: `SHOES_OFFERS_INTERVAL_MS`, `SHOES_DROPS_INTERVAL_MS`, `SHOES_EBAY_INTERVAL_MS`.
126
127	## Run it
128
129	```bash
130	npm ci --include=dev
131	npm test # every unit and HTTP test on PGlite
132	npm run test:pg # the same through PostgreSQL + PgBouncer
133	npm run seed # one first run of every job against the live sources (needs the network)
134	npm start # http://127.0.0.1:5180
135	```
136
137	Without `DATABASE_URL`, development uses an embedded PGlite database in `data/pglite` (one process only). A
138	production start needs `DATABASE_URL` and `DATABASE_DIRECT_URL`; the owner role migrates on boot.
139
140	## Deploy
141
142	Copy `deploy/systemd/openvibe-shoes.service` to `/etc/systemd/system/` and
143	`deploy/nginx/open.shoes.conf` to `/etc/nginx/sites-available/`; the unit reads `/etc/openvibe/shoes.env`, works in
144	`/opt/openvibe.shoes`, runs as `ubuntu:ubuntu` on `127.0.0.1:5180`, and the vhost proxies `/api/`, `/auth/` and the
145	forms with their own limits. The `.gitignore` keeps `node_modules`, `data`, `.env` and keys out of the repo.
146
147	## First run (seed)
148
149	`npm run seed` reads the live sources once and prints what it wrote. A source that does not answer is reported as
150	a plain note and does not fail the other jobs. The first local run (2026-10-31, 60 Deals pages) wrote:
151
152	\| Source \| Result \|
153	\|---\|---\|
154	\| OpenVibe.Deals \| 1,500 offers read, 182 footwear candidates, 165 product lookups, 178 offers \|
155	\| Nike SNKRS launch calendar \| 16 official releases \|
156	\| eBay \| off (no developer key pair) \|
157	\| Catalogue \| 169 models · 178 offers · 178 daily snapshots · 16 drops · 0 facts \|
158
159	The same counts are in [STATUS.json](STATUS.json) under `seed`. The catalogue opens with real models and real
160	releases; the price history is one day deep and grows with every daily check, and the "biggest price drops today"
161	section says so plainly when no store has cut a price yet.
162
163	## What v1 does not do
164
165	- No raffle, draw, reservation or checkout is automated, ever; the drops calendar links the official page and
166	adds a reminder on Today.
167	- Specs/fit/release/care are shown only from cited sources; with none, the page says "not known yet".
168	- eBay is off until the host sets a developer key pair; Watch is called with the signed-in person's own Network
169	token (a session without one still saves the watch on Shoes, marked "not connected").
170	- No affiliate links, no sponsored order, no Marketplace scraping.
171
172	<!-- versions:start -->
173	- openvibe-contracts: v0.419.0
174	- openvibe-sdk: v0.161.0
175	- openvibe-shared: v4.241.0
176	- openvibe-publishing: v1.7.0
177	<!-- versions:end -->
178