openvibers/openvibe.bot Commits

Bot: the Live controls export reads Live's PostgreSQL (the conversion can run) (#80)

OP OpenVibers committed 11 Oct 2026 · 2bbb2fbf61 · 2 files, +46 −56

`scripts/live-controls-export.sh` read Live's SQLite file, which has not existed since Live moved to PostgreSQL on 2026-10-08, so the Live controls → Bot conversion (plan T15 R9 step 5) could not run its export. It now reads `ov_live` with `psql`:

- **Read-only:** the session is `default_transaction_read_only`, and the export is one statement, so one snapshot.
- **The argument:** a database name or connection string; production runs `sudo -u postgres scripts/live-controls-export.sh ov_live`.
- **Unchanged:** the same JSON document shape, so the converter and its fixture are unchanged.
- **Subjects:** taken from `linked_accounts`, with `subject_projection` as the fallback; Live's PostgreSQL schema has no `users.subject_id`.

**Proof against production today:** the export ran (4 configs, 25 buttons, 1 whitelist entry, 19 latest-stream controls, 3 owners, all with a Network subject). The converter's dry run plans 3 robots on `relay.generic` ("OBS Control" and "Cozmo Hobo" for one owner, "Cozmo" for another) and skips one config with no enabled button. Nothing was applied: applying creates robots in those people's accounts, so it waits for a go-ahead.

`docs/live-controls-migration.md` shows the PostgreSQL command. shellcheck passes; `live-controls-convert` passes.

Changes

ddiff --git a/docs/live-controls-migration.md b/docs/live-controls-migration.md
iindex 817384f..67f57c4 100644
---- a/docs/live-controls-migration.md
++++ b/docs/live-controls-migration.md
@@@ -51,7 +51,7 @@ lines) let viewers drive a streamer's hardware from the channel page:
existing Pi script keeps working with a one-line change of where it connects. Local policy and the kill switch
apply as for every plugin.
5. **Built — the conversion**, `scripts/live-controls-export.sh` and `scripts/convert-live-controls.js` in Bot:
-- an operator runs the export on the Live host (read-only) and the converter here — dry-run first, then
++ an operator runs the export against Live's PostgreSQL (read-only) and the converter here — dry-run first, then
`--apply`. For each `control_configs` row it creates the robot for the owner's Network subject (Live's
`linked_accounts`/`subject_projection`, the person's `usr_…`), a robot-local profile from its buttons
(profile `relay.generic`, the catalogue's bare relay robot: buttons and video clicks, no drive), operators
@@@ -69,11 +69,12 @@ lines) let viewers drive a streamer's hardware from the channel page:
## Running the conversion
--On the Live host, where `DB_PATH` points at Live's SQLite file (read-only — no journal is written, nothing in
--Live changes):
++On the Live host, against Live's PostgreSQL database (`ov_live`, since 2026-10-08). The session is read-only
++(`default_transaction_read_only`) and reads one snapshot, so nothing in Live changes:
```sh
--scripts/live-controls-export.sh /path/to/live.sqlite > export.json
++sudo -u postgres scripts/live-controls-export.sh ov_live > export.json
++# or any connection string psql takes: scripts/live-controls-export.sh 'postgresql://reader@host/ov_live'
```
That one document carries only what the conversion needs: `configs`, `buttons`, `whitelist`,
ddiff --git a/scripts/live-controls-export.sh b/scripts/live-controls-export.sh
iindex 3838c9c..7cb1c81 100755
---- a/scripts/live-controls-export.sh
++++ b/scripts/live-controls-export.sh
@@@ -1,9 +1,12 @@
#!/bin/sh
# Export OpenVibe.Live's stream controls for the one-time conversion into Bot robots (plan T15 R9 step 5).
#
--# scripts/live-controls-export.sh <path to Live's SQLite file> > export.json
++# sudo -u postgres scripts/live-controls-export.sh ov_live > export.json
++# scripts/live-controls-export.sh 'postgresql://reader@host/ov_live' > export.json
#
--# Read-only: every read is `sqlite3 -readonly -json`, so the file is never written and no journal is made.
++# Live runs on PostgreSQL (ov_live, since 2026-10-08). The argument is anything `psql -d` takes: a database name
++# (with the caller's own access, as above) or a connection string. Read-only: the session is opened with
++# default_transaction_read_only, so nothing can be written, and the one statement reads one snapshot.
# The one document carries only what the conversion needs:
#
# configs every control_configs row (id, user_id, name)
@@@ -12,66 +15,52 @@
# latest_stream_controls the stream_controls rows of each config owner's most recent stream only
# owners only the config owners and whitelisted users: id, username, subject_id
#
--# No other personal data (no email, no password hash, no addresses). Run it on the Live host, where DB_PATH
--# points at the file; it prints one JSON document to stdout and nothing else.
++# No other personal data (no email, no password hash, no addresses). It prints one JSON document to stdout and
++# nothing else.
set -eu
DB=${1:-}
if [ -z "$DB" ]; then
-- echo "usage: live-controls-export.sh <path to Live's SQLite file>" >&2
++ echo "usage: live-controls-export.sh <database name or connection string for Live's PostgreSQL (ov_live)>" >&2
exit 2
fi
--if [ ! -f "$DB" ]; then
-- echo "live-controls-export: no such database file: $DB" >&2
-- exit 1
--fi
--command -v sqlite3 >/dev/null 2>&1 || { echo "live-controls-export: sqlite3 is not installed" >&2; exit 1; }
--
--# Rows as a JSON array (sqlite3's -json; an empty result prints nothing, normalised below).
--rows() { sqlite3 -readonly -json "$DB" "$1"; }
--# One scalar.
--one() { sqlite3 -readonly "$DB" "$1"; }
++command -v psql >/dev/null 2>&1 || { echo "live-controls-export: psql is not installed" >&2; exit 1; }
--# Which columns/tables this Live database has: the Network subject of a Live account lives in
--# linked_accounts (service 'network'), with subject_projection as the fallback for an account whose link
--# row is missing the subject; a future users.subject_id is honoured when present.
--users_has_subject=$(one "SELECT COUNT(*) FROM pragma_table_info('users') WHERE name = 'subject_id';")
--has_linked=$(one "SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = 'linked_accounts';")
--has_projection=$(one "SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = 'subject_projection';")
--
--SUBJECT='NULL'
--if [ "$users_has_subject" = "1" ]; then SUBJECT="NULLIF(u.subject_id, '')"; fi
--if [ "$has_linked" = "1" ]; then
-- LINKED="(SELECT la.subject_id FROM linked_accounts la WHERE la.service = 'network' AND la.user_id = u.id AND substr(la.subject_id, 1, 4) = 'usr_' LIMIT 1)"
-- if [ "$SUBJECT" = 'NULL' ]; then SUBJECT="$LINKED"; else SUBJECT="COALESCE($SUBJECT, $LINKED)"; fi
-- if [ "$has_projection" = "1" ]; then
-- PROJECTED="(SELECT sp.subject_id FROM subject_projection sp WHERE sp.network_user_id = CAST((SELECT la.service_user_id FROM linked_accounts la WHERE la.service = 'network' AND la.user_id = u.id LIMIT 1) AS INTEGER) AND substr(sp.subject_id, 1, 4) = 'usr_' LIMIT 1)"
-- SUBJECT="COALESCE($SUBJECT, $PROJECTED)"
-- fi
--fi
++# The Network subject of a Live account: linked_accounts (service 'network') first, then subject_projection by the
++# linked Network user id, for an account whose link row is missing the subject.
++SUBJECT="COALESCE(
++ (SELECT la.subject_id FROM linked_accounts la WHERE la.service = 'network' AND la.user_id = u.id AND left(la.subject_id, 4) = 'usr_' LIMIT 1),
++ (SELECT sp.subject_id FROM subject_projection sp
++ WHERE sp.network_user_id::text = (SELECT la.service_user_id FROM linked_accounts la WHERE la.service = 'network' AND la.user_id = u.id LIMIT 1)
++ AND left(sp.subject_id, 4) = 'usr_' LIMIT 1))"
# The config owners: an owner's channel is unique (channels.user_id UNIQUE), which is what the whitelist is
# keyed by. Whitelisted users are only those on a config owner's channel.
OWNERS="SELECT user_id FROM control_configs UNION SELECT w.user_id FROM control_whitelist w JOIN channels c ON c.id = w.channel_id WHERE c.user_id IN (SELECT user_id FROM control_configs)"
--CONFIGS=$(rows "SELECT id, user_id, name FROM control_configs ORDER BY id;")
--BUTTONS=$(rows "SELECT b.id, b.config_id, b.label, b.command, b.control_type, b.key_binding, b.cooldown_ms, b.is_enabled, b.sort_order FROM control_config_buttons b JOIN control_configs c ON c.id = b.config_id ORDER BY b.config_id, b.sort_order, b.id;")
--WHITELIST=$(rows "SELECT w.id, w.user_id, c.user_id AS owner_user_id FROM control_whitelist w JOIN channels c ON c.id = w.channel_id WHERE c.user_id IN (SELECT user_id FROM control_configs) ORDER BY w.id;")
--# The most recent stream per config owner (a live session's started_at, else when the row was created),
--# then that stream's controls. stream_config_id says which config the copy came from, so the conversion can
--# tell an override of this config from a stream bound to another one.
--LATEST=$(rows "WITH latest AS (
-- SELECT s.user_id, s.id AS stream_id, s.control_config_id,
-- ROW_NUMBER() OVER (PARTITION BY s.user_id ORDER BY COALESCE(s.started_at, s.created_at) DESC, s.id DESC) AS rn
-- FROM streams s WHERE s.user_id IN (SELECT user_id FROM control_configs)
-- )
-- SELECT sc.id, sc.stream_id, latest.user_id AS owner_user_id, latest.control_config_id AS stream_config_id,
-- sc.label, sc.command, sc.control_type, sc.key_binding, sc.cooldown_ms, sc.is_enabled, sc.sort_order
-- FROM latest JOIN stream_controls sc ON sc.stream_id = latest.stream_id
-- WHERE latest.rn = 1
-- ORDER BY sc.stream_id, sc.sort_order, sc.id;")
--OWNER_ROWS=$(rows "SELECT u.id, u.username, $SUBJECT AS subject_id FROM users u WHERE u.id IN ($OWNERS) ORDER BY u.id;")
++# The most recent stream per config owner (a live session's started_at, else when the row was created), then that
++# stream's controls. stream_config_id says which config the copy came from, so the conversion can tell an override
++# of this config from a stream bound to another one.
++SQL="SELECT json_build_object(
++ 'exported_at', to_char(now() AT TIME ZONE 'UTC', 'YYYY-MM-DD\"T\"HH24:MI:SS\"Z\"'),
++ 'configs', COALESCE((SELECT json_agg(t ORDER BY t.id) FROM (SELECT id, user_id, name FROM control_configs) t), '[]'::json),
++ 'buttons', COALESCE((SELECT json_agg(t ORDER BY t.config_id, t.sort_order, t.id) FROM (
++ SELECT b.id, b.config_id, b.label, b.command, b.control_type, b.key_binding, b.cooldown_ms, b.is_enabled, b.sort_order
++ FROM control_config_buttons b JOIN control_configs c ON c.id = b.config_id) t), '[]'::json),
++ 'whitelist', COALESCE((SELECT json_agg(t ORDER BY t.id) FROM (
++ SELECT w.id, w.user_id, c.user_id AS owner_user_id FROM control_whitelist w JOIN channels c ON c.id = w.channel_id
++ WHERE c.user_id IN (SELECT user_id FROM control_configs)) t), '[]'::json),
++ 'latest_stream_controls', COALESCE((SELECT json_agg(t ORDER BY t.stream_id, t.sort_order, t.id) FROM (
++ WITH latest AS (
++ SELECT s.user_id, s.id AS stream_id, s.control_config_id,
++ ROW_NUMBER() OVER (PARTITION BY s.user_id ORDER BY COALESCE(s.started_at, s.created_at) DESC, s.id DESC) AS rn
++ FROM streams s WHERE s.user_id IN (SELECT user_id FROM control_configs))
++ SELECT sc.id, sc.stream_id, latest.user_id AS owner_user_id, latest.control_config_id AS stream_config_id,
++ sc.label, sc.command, sc.control_type, sc.key_binding, sc.cooldown_ms, sc.is_enabled, sc.sort_order
++ FROM latest JOIN stream_controls sc ON sc.stream_id = latest.stream_id
++ WHERE latest.rn = 1) t), '[]'::json),
++ 'owners', COALESCE((SELECT json_agg(t ORDER BY t.id) FROM (
++ SELECT u.id, u.username, $SUBJECT AS subject_id FROM users u WHERE u.id IN ($OWNERS)) t), '[]'::json)
++);"
--printf '{"exported_at":"%s","configs":%s,"buttons":%s,"whitelist":%s,"latest_stream_controls":%s,"owners":%s}\n' \
-- "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
-- "${CONFIGS:-[]}" "${BUTTONS:-[]}" "${WHITELIST:-[]}" "${LATEST:-[]}" "${OWNER_ROWS:-[]}"
++PGOPTIONS='-c default_transaction_read_only=on' psql -X -q -A -t -v ON_ERROR_STOP=1 -d "$DB" -c "$SQL"