openvibers/openvibe.media Commits

Media S3: a HEAD that Cloudflare forwards as GET is served (header-signed only; presigned HEAD URLs never read the body) (#110)

OP OpenVibers committed 29 days ago · 73d7e9051d · 3 files, +40 −4

Found during the production proof of OpenVibe.Zone. The first `HeadObject` after a `PutObject` through `https://s3.openvibe.network` failed. nginx logged that request as a **GET** (403 `SignatureDoesNotMatch`), because Cloudflare forwards a HEAD as a GET for paths it may cache (by extension: `.bin`, `.jpg`, `.zip` …). SigV4 signs the method, so the signature was a HEAD's.

**Change (`server/s3/sigv4.js`):**
- A request that arrives as GET, is header-signed, has no body (empty-payload hash or `UNSIGNED-PAYLOAD`) and verifies only as HEAD is accepted and served as the GET. The proxy returns only the headers to the client.
- No new capability: the key must still be allowed to read, and its holder could send that GET themselves.
- **Never for a presigned URL:** one may be shared for a HEAD alone, so turning it into a GET would read the body. A presigned HEAD URL fetched as GET stays 403.
- Only GET can stand in for HEAD. A HEAD signature sent as DELETE, or a PUT sent as GET, is refused.

**Edge caching checked in production:** with the probe key, signed GETs of a `.jpg` answer `Cache-Control: private, no-store` with `cf-cache-status: BYPASS`, and an unsigned GET of the same URL is 403 without the bytes. Nothing private is cached at the edge.

The real fix at the edge is a DNS-only `s3.openvibe.media`, which waits for a Cloudflare token (owner). Until then this keeps rclone, the AWS CLI and SDK `HeadObject` calls working through Cloudflare.

**Tests:** `test/s3-zones.test.js` adds a header-signed HEAD sent as GET (served, right length), the same sent as DELETE (403, object kept), a presigned HEAD URL as HEAD (200) and as GET (403), and a PUT sent as GET (403). `s3-zones` and `s3-sigv4` pass. `docs/s3.md` explains the rule.

Changes

ddiff --git a/docs/s3.md b/docs/s3.md
iindex 126a33e..b4bf56a 100644
---- a/docs/s3.md
++++ b/docs/s3.md
@@@ -36,6 +36,7 @@ a named object zone of one project environment. Zones are made and deleted only
- **Overwrite.** The new bytes are stored first. Then, in one transaction under the tenant's quota lock, the previous object goes through Media's deletion path and the new one becomes ready. A held object is neither overwritten nor deleted (`AccessDenied`).
- **Quotas.** A write is checked against the zone's `quota_bytes` and the environment's quota, before the bytes arrive and again after.
- **Zone deletion.** When Zone pushes state `deleting`, the zone answers `NoSuchBucket` at once. The minute sweep (`zones.sweepDeleting`) tombstones every object with no restore window and purges every copy. The status reports `objects_remaining` and `held_objects` until nothing is left. Then Zone marks the zone `deleted`: its name is retired, and keys scoped only to it are revoked.
++- **HEAD through a proxy.** Cloudflare forwards a HEAD as a GET for paths it may cache (by extension: `.jpg`, `.bin`, `.zip` …), but SigV4 signs the method. A header-signed request with no body that arrives as GET and verifies only as HEAD is served as the GET, and the proxy returns just the headers. A presigned URL never gets this, because one may be shared for a HEAD alone; nor does any other method. Every S3 response is `Cache-Control: private, no-store`, so the edge never caches one (`cf-cache-status: BYPASS`). The fix at the edge itself is a DNS-only `s3.openvibe.media`.
- **No browser execution.** Every response carries `Content-Security-Policy: sandbox; default-src 'none'` and `nosniff`, whatever type the object declares.
## Tests
ddiff --git a/server/s3/sigv4.js b/server/s3/sigv4.js
iindex 48e26b1..614906f 100644
---- a/server/s3/sigv4.js
++++ b/server/s3/sigv4.js
@@@ -138,14 +138,14 @@ function amzDateMs(amzDate) {
return m ? Date.UTC(+m[1], +m[2] - 1, +m[3], +m[4], +m[5], +m[6]) : NaN;
}
--/** The signature the client should have sent, given the secret. */
--function expectedSignature(req, parsed, secret) {
++/** The signature the client should have sent, given the secret (for `method`, by default the request's own). */
++function expectedSignature(req, parsed, secret, method = req.method) {
const headers = parsed.signedHeaders.map((h) => {
const v = h === 'host' ? (headerValue(req, 'host') || '') : headerValue(req, h);
return `${h}:${v === undefined ? '' : v}\n`;
}).join('');
const canonical = [
-- req.method,
++ method,
canonicalPath(signedPath(req)),
canonicalQuery(parsed.pairs),
headers,
@@@ -176,7 +176,11 @@ function verify(req, parsed, secret, { now = Date.now() } = {}) {
if (at - now > SKEW_MS) return { code: 'AccessDenied', message: 'Request is not valid yet' };
if (now > at + parsed.expires * 1000) return { code: 'AccessDenied', message: 'Request has expired' };
}
-- const { signature, key } = expectedSignature(req, parsed, secret);
++ let { signature, key } = expectedSignature(req, parsed, secret);
++ if (!safeEqual(signature, parsed.signature) && headSentAsGet(req, parsed)) {
++ ({ signature, key } = expectedSignature(req, parsed, secret, 'HEAD'));
++ if (safeEqual(signature, parsed.signature)) parsed.signedAsHead = true;
++ }
if (!safeEqual(signature, parsed.signature)) {
return { code: 'SignatureDoesNotMatch', message: 'The request signature we calculated does not match the signature you provided' };
}
@@@ -184,6 +188,16 @@ function verify(req, parsed, secret, { now = Date.now() } = {}) {
return null;
}
++/**
++ * A HEAD that a proxy forwarded as GET: Cloudflare turns HEAD into GET for paths it may cache (by extension: .jpg,
++ * .bin, .zip …), and SigV4 signs the method, so the signature is a HEAD's. Taken only for a header-signed request
++ * with no body; never for a presigned URL, which may be shared for a HEAD alone. Serving it as a GET gives the
++ * signer nothing their key could not read with a GET of its own, and the proxy returns only the headers to them.
++ */
++function headSentAsGet(req, parsed) {
++ return parsed.kind === 'header' && req.method === 'GET' && (parsed.payloadHash === EMPTY_SHA256 || parsed.payloadHash === 'UNSIGNED-PAYLOAD');
++}
++
/** Is the payload an aws-chunked stream (x-amz-content-sha256 says so)? */
function isStreaming(parsed) {
return STREAMING.has(parsed.payloadHash);
ddiff --git a/test/s3-zones.test.js b/test/s3-zones.test.js
iindex e9bf461..0c3eb0d 100644
---- a/test/s3-zones.test.js
++++ b/test/s3-zones.test.js
@@@ -204,6 +204,27 @@ const { Readable } = require('stream');
assert.deepStrictEqual([res.status, /Request has expired/.test(await res.text())], [403, true], 'an expired URL is refused');
console.log('✅ presigned GET and PUT; a URL reaches its own key only and expires');
++ // ── A HEAD that a proxy forwarded as GET (Cloudflare does for cacheable extensions) ──
++ const asGet = client(full);
++ asGet.middlewareStack.addRelativeTo((next) => (args) => { args.request.method = 'GET'; return next(args); },
++ { relation: 'after', toMiddleware: 'awsAuthMiddleware', name: 'headSentAsGet' });
++ out = await asGet.send(new S3.HeadObjectCommand({ Bucket: 'assets', Key: 'readme.txt' }));
++ assert.strictEqual(Number(out.ContentLength), 'readme.txt'.length, 'a header-signed HEAD that arrives as GET is served');
++ const asDelete = client(full);
++ asDelete.middlewareStack.addRelativeTo((next) => (args) => { args.request.method = 'DELETE'; return next(args); },
++ { relation: 'after', toMiddleware: 'awsAuthMiddleware', name: 'headSentAsDelete' });
++ await assert.rejects(asDelete.send(new S3.HeadObjectCommand({ Bucket: 'assets', Key: 'readme.txt' })), (e) => e.$metadata.httpStatusCode === 403, 'only GET stands in as HEAD');
++ await s3.send(new S3.HeadObjectCommand({ Bucket: 'assets', Key: 'readme.txt' }));
++ const headUrl = await getSignedUrl(s3, new S3.HeadObjectCommand({ Bucket: 'assets', Key: 'readme.txt' }), { expiresIn: 60 });
++ assert.strictEqual((await fetch(headUrl, { method: 'HEAD' })).status, 200, 'a presigned HEAD reads the headers');
++ res = await fetch(headUrl);
++ assert.deepStrictEqual([res.status, /SignatureDoesNotMatch/.test(await res.text())], [403, true], 'a presigned HEAD URL never reads the body');
++ const putAsGet = client(full);
++ putAsGet.middlewareStack.addRelativeTo((next) => (args) => { args.request.method = 'GET'; return next(args); },
++ { relation: 'after', toMiddleware: 'awsAuthMiddleware', name: 'putSentAsGet' });
++ await assert.rejects(putAsGet.send(new S3.PutObjectCommand({ Bucket: 'assets', Key: 'readme.txt', Body: 'nope' })), (e) => e.$metadata.httpStatusCode === 403);
++ console.log('✅ a header-signed HEAD forwarded as GET is served; presigned HEAD URLs and other methods are not');
++
// ── Isolation and permissions ──
await assert.rejects(client(scoped).send(new S3.GetObjectCommand({ Bucket: 'assets', Key: 'readme.txt' })), /AccessDenied/, 'a zone-scoped key cannot reach another zone');
await client(scoped).send(new S3.PutObjectCommand({ Bucket: 'logs', Key: 'a.log', Body: 'x' }));