From 9213e1bcd1f5da640ebaa05cd63b7f6b500c2c31 Mon Sep 17 00:00:00 2001 From: Paul Bakaus Date: Tue, 21 Jul 2026 20:07:25 -0700 Subject: [PATCH] Fix stale section pointer and critique score denominators Two true positives from the Bugbot review on PR #397. document.md seed mode told the agent to run "Select one direction" for paths A, D, or E. new-work.md has neither that heading nor the A/D/E lettering since the workshop was restructured into named subsections, so a literal read could skip the world-and-surface flow entirely. Point at "Create or replace the visual world" and "Commit the world" instead. critique.md let the heuristic table renormalize to an applicable maximum when heuristics are scored n/a, but the report template hardcoded ??/40, the rating bands only mapped raw numbers out of 40, and the persisted meta carried total_score with no denominator. Trends could silently compare 24/32 against 30/40 as if they were the same scale. The template now prints the applicable max, the bands fall back to percentages for partial sets, the snapshot records max_score and na_heuristics, and the trend line states its denominator or breaks it out per run when they disagree. critique-storage.mjs serializes frontmatter key-agnostically, so the new keys need no code change. Co-Authored-By: Claude --- skill/reference/critique.md | 18 ++++++++++++------ skill/reference/document.md | 2 +- 2 files changed, 13 insertions(+), 7 deletions(-) diff --git a/skill/reference/critique.md b/skill/reference/critique.md index f16f50fa4..726fc1657 100644 --- a/skill/reference/critique.md +++ b/skill/reference/critique.md @@ -132,11 +132,13 @@ Present the Nielsen's 10 heuristics scores as a table: | 8 | Aesthetic and Minimalist Design | ? | | | 9 | Error Recovery | ? | | | 10 | Help and Documentation | ? | | -| **Total** | | **??/40** | **[Rating band]** | +| **Total** | | **??/[applicable max]** | **[Rating band]** | -Be honest with scores. A 4 means genuinely excellent. Most real interfaces score 20-32. +The applicable maximum is 4 times the number of heuristics you actually scored: **/40** when all ten apply, **/32** when two are `n/a`. Never print `/40` over a partial set. -**Mode applicability**: heuristics 7 (Flexibility and Efficiency) and 10 (Help and Documentation) may be scored `n/a` on Persuade and Experience surfaces (landing pages, campaigns, portfolios, bodies of work), as may any other heuristic that genuinely cannot apply to the surface under review. Write `n/a` in the Score cell with a one-line reason, and renormalize the total to the applicable maximum (e.g. **24/32** when two heuristics are n/a) so the rating band stays proportional. The persisted snapshot must record which heuristics were scored n/a. +Be honest with scores. A 4 means genuinely excellent. Most real interfaces score 20-32 out of 40. + +**Mode applicability**: heuristics 7 (Flexibility and Efficiency) and 10 (Help and Documentation) may be scored `n/a` on Persuade and Experience surfaces (landing pages, campaigns, portfolios, bodies of work), as may any other heuristic that genuinely cannot apply to the surface under review. Write `n/a` in the Score cell with a one-line reason, and renormalize the total to the applicable maximum (e.g. **24/32** when two heuristics are n/a) so the rating band stays proportional. The persisted snapshot must record the applicable maximum and which heuristics were scored n/a. #### Design Specificity Verdict @@ -214,10 +216,10 @@ Skip this step if the Setup slug was null (vague or root-level target). 2. **Pass the structured metadata** through `IMPECCABLE_CRITIQUE_META` (JSON), then run the write command: ```bash - IMPECCABLE_CRITIQUE_META='{"target":"","total_score":,"p0_count":,"p1_count":}' \ + IMPECCABLE_CRITIQUE_META='{"target":"","total_score":,"max_score":,"na_heuristics":"","p0_count":,"p1_count":}' \ node {{scripts_path}}/critique-storage.mjs write "" ``` - The helper prints the absolute path it wrote. + `max_score` is the applicable maximum from the heuristic table (40 when every heuristic applied), so a later run can tell a renormalized total from a full one. The helper prints the absolute path it wrote. 3. **Delete the temp body file** after the write attempt completes, whether the write succeeded or failed. If deletion fails, mention `temp-file cleanup failed: ` briefly in the final output, but do not block the critique. @@ -229,9 +231,11 @@ Skip this step if the Setup slug was null (vague or root-level target). 5. **Append a single line to the user-visible output**, after the report and before the questions: - > **Trend for `` (last 5 runs): 24 → 28 → 32 → 29 → 32** + > **Trend for `` (last 5 runs): 24 → 28 → 32 → 29 → 32 (out of 40)** > Wrote `.impeccable/critique/`. + Read `max_score` on each trend entry. When every entry shares one maximum, state it once as above. When they differ, print each score with its own denominator (`24/32 → 30/40`) and note that the runs scored different heuristic sets, so the line is not a like-for-like comparison. Treat a missing `max_score` on an older entry as 40. + If this is the first run for the slug, the trend is just one score; say so: "First run for this target, no trend yet." This is fire-and-forget. Do not show the user the helper's JSON output; only the human-readable trend line and the written path. Failures here should not block the rest of the flow; print the error and move on. @@ -623,6 +627,8 @@ Even if the system is usable without docs, help should be easy to find, task-foc | 12–19 | Poor | Major UX overhaul required; core experience broken | | 0–11 | Critical | Redesign needed; unusable in current state | +When heuristics were scored `n/a`, the maximum is lower than 40; read the band off the percentage instead of the raw number (90%+ Excellent, 70%+ Good, 50%+ Acceptable, 30%+ Poor, below that Critical). 24/32 is 75%, so Good. + --- #### Issue Severity (P0–P3) diff --git a/skill/reference/document.md b/skill/reference/document.md index 725d56db2..ce7f19287 100644 --- a/skill/reference/document.md +++ b/skill/reference/document.md @@ -355,7 +355,7 @@ For projects with no visual system to extract yet. Produces a user-chosen visual PRODUCT.md is the prerequisite. If it is missing, load [init.md](init.md) and complete its product interview first. Do not create a visual identity without durable product context. -If PRODUCT.md exists, load [new-work.md](new-work.md) and resolve visual authority. Seed mode requires a concrete first surface: use the target the user named, or ask what they want to make first. Run **Select one direction** for A, D, or E so the visual world and its first expression are chosen together. Stop after the directional DESIGN.md seed and surface brief; do not implement. A structured simulated user counts as the user and must get the same choice. +If PRODUCT.md exists, load [new-work.md](new-work.md) and resolve visual authority. Seed mode requires a concrete first surface: use the target the user named, or ask what they want to make first. Run new-work's **Create or replace the visual world** flow, then **Commit the world**, so the visual world and its first expression are chosen together. Stop after the directional DESIGN.md seed and surface brief; do not implement. A structured simulated user counts as the user and must get the same choice. If new-work already completed the workshop in this session, use its chosen direction directly. Do not ask again.