openvibers/openvibe.design Commits

Add repeatable icon generator comparison worksheet (#276)

OP OpenVibers committed 2 days ago · 12a01362d6 · 3 files, +34 −0

Changes

ddiff --git a/bench/icon-generator/README.md b/bench/icon-generator/README.md
nnew file mode 100644
iindex 0000000..87c8eed
---- /dev/null
++++ b/bench/icon-generator/README.md
@@@ -0,0 +1,21 @@
++# Icon generator comparison
++
++Use this worksheet to compare the Design glyph draft with the owner's Fonos reference. Keep the same prompt, style, and generation count for both products. Do not infer a Fonos score from another icon product; enter its product URL and capture the exact version or date used.
++
++## Run
++
++1. Record the Design release and Fonos URL/version in `scorecard.csv`. Use a signed-out Design session for the free tier and a signed-in session for the Anthropic route. Do not buy credits for this comparison; mark a product unavailable when its free allowance does not cover a prompt.
++2. For each row in `prompts.json`, generate three candidates per product with the same prompt and style. Save the returned SVG or screenshot with the prompt id, product, and candidate number. Record the model shown by each product, elapsed time, and any visible price. Keep failures and refusals.
++3. Place each candidate at 16, 24, 32, and 48 CSS pixels on both white and near-black backgrounds. Score the criteria below from 0 to 2 without knowing which product produced the candidate. Two reviewers score independently; keep both rows and their notes in `scorecard.csv`.
++4. Report each criterion's median across the ten prompts and both reviewers, plus the number of failures, free generations available, median elapsed time, and the displayed cost. Include candidate files and the filled scorecard with the report.
++
++## Rubric
++
++| Criterion | 0 | 1 | 2 |
++| --- | --- | --- | --- |
++| Alignment | Visibly off center or clipped at any size | Small offset or uneven margin | Centered with balanced margins at every size |
++| Legibility | Meaning is unclear at 24 px | Meaning is clear at 32 px | Meaning is clear at 16 and 24 px |
++| Consistency | Stroke, corners, or fill conflict within one icon | One visible mismatch | Coherent stroke, corner, and fill style throughout the icon |
++| Reuse | Cannot export a usable vector or transparent asset | Export needs manual cleanup | Ready to use in a UI without cleanup |
++
++The prompt set exercises simple forms, diagonals, thin details, and negative space. A higher score needs the saved outputs, not just the products' previews. Design's `test/glyph-fit.test.js` separately checks its 24-unit alignment invariant; this worksheet measures what a person sees after rendering.
ddiff --git a/bench/icon-generator/prompts.json b/bench/icon-generator/prompts.json
nnew file mode 100644
iindex 0000000..bfd8b8a
---- /dev/null
++++ b/bench/icon-generator/prompts.json
@@@ -0,0 +1,12 @@
++[
++ {"id":"search","style":"line","prompt":"A magnifying glass with a round lens and a short handle, as a simple interface icon"},
++ {"id":"settings","style":"line","prompt":"A gear with a circular center, as a simple interface icon"},
++ {"id":"upload","style":"line","prompt":"An upward arrow entering a tray, as a simple interface icon"},
++ {"id":"calendar","style":"line","prompt":"A calendar page with two top tabs and one marked day, as a simple interface icon"},
++ {"id":"audio","style":"line","prompt":"A speaker with two sound waves, as a simple interface icon"},
++ {"id":"shield","style":"solid","prompt":"A shield with a clear checkmark cut out of its center, as a simple interface icon"},
++ {"id":"cloud","style":"solid","prompt":"A cloud with an upward arrow cut out of it, as a simple interface icon"},
++ {"id":"compass","style":"solid","prompt":"A compass needle pointing northeast inside a circle, as a simple interface icon"},
++ {"id":"chat","style":"line","prompt":"A speech bubble with two short lines of text, as a simple interface icon"},
++ {"id":"bookmark","style":"line","prompt":"A bookmark ribbon with a V-shaped notch at the bottom, as a simple interface icon"}
++]
ddiff --git a/bench/icon-generator/scorecard.csv b/bench/icon-generator/scorecard.csv
nnew file mode 100644
iindex 0000000..655c9e0
---- /dev/null
++++ b/bench/icon-generator/scorecard.csv
@@@ -0,0 +1 @@
++prompt_id,product,product_url,release_or_version,session,model,candidate,reviewer,alignment_0_2,legibility_0_2,consistency_0_2,reuse_0_2,elapsed_seconds,displayed_cost,free_generation,output_file,notes