pbakaus_impeccable/docs/DESIGN-CONTEXT-ARCHITECTURE.md

# Design context architecture

How the design interview, the design context document, and everything the user
gives us during the seed process are organized, and why.

## The problem this layout solves

The design context is one product concept. Before this layout it was scattered
across two directories with no owner:

- `.impeccable/design-interview/` held the questionnaire submission, uploaded
  fonts, staged brand assets, and two runtime files.
- `.impeccable/visual-cues/` held the cue generation workspace, and also, buried
  inside `cues.json`, the `context` object carrying the entire chat half of the
  interview plus the `modes` set. Neither is a generation artifact.
- The design context document was not a file at all. It was a client-side render
  that existed only after a submit, died with its session, and could not be
  reopened.
- Three scripts hardcoded the storage directory independently, with no shared
  constant. The agent-side poll CLI treats a missing session file as a clean
  exit, so a half-finished rename would fail silently and no test would catch it.

Export, import, resume, rebuild, and edit each had to reassemble state from those
places with implicit coupling. One canonical store with one code owner replaces
that.

## File layout, in the user's project

```
.impeccable/design-context/          the store: the one exportable unit
  context.json    { schemaVersion, modes, context }   the chat half of the run
  answers.json    the questionnaire submission, flat form-field shape
  assets/         brand files the user supplied
  fonts/          font faces the user uploaded
  cue.png         the chosen hero, copied at submit
  runtime/        session.json, journal.jsonl, draft.json   (gitignored)
  exports/        design-context.md, design-context.bundle.json   (gitignored)

.impeccable/visual-cues/             the generation workspace, unchanged
  brief.md, <slug>.png x6, cues.json (cues + palette only), fonts.json
```

The reasoning, in the order the decisions were made:

1. **Durable versus regenerable is the axis.** It matches the policy this repo's
   own `.gitignore` states: generated sidecars and config may be tracked, but
   runtime recovery state and local assets stay local. Everything in the store
   above `runtime/` is a user decision or a user-supplied file, so it is
   trackable and exportable. The workspace is regenerable process output and
   stays ignored.
2. **`context.json` moves the chat half out of `cues.json`.** The context object
   was never a cue artifact. It rode there because that was the only file the
   picker served. With its own home it survives cue regeneration, imports without
   dragging six hero images along, and can be edited by the save flow without
   touching the generation workspace.
3. **`cue.png` makes the document self-sufficient.** The Color article renders
   the chosen cue only when the cue manifest still lists it, which means the
   document breaks once the workspace is cleaned or in a project that received
   the context by import. Copying the one chosen hero into the store at submit
   time costs a few hundred kilobytes and buys rendering independence. The five
   unpicked heroes stay in the workspace as art direction leftovers.
4. **`runtime/` isolates ephemera.** Session discovery, the journal, and the
   mid-questionnaire draft never belong in an export or a commit. One gitignore
   line covers all of it permanently.
5. **PRODUCT.md and DESIGN.md stay at the project root.** They are the canonical
   documents the whole toolchain reads. `context.json` carries distilled copies
   with provenance, never replacements. Edits that touch product truth reconcile
   back into PRODUCT.md through the agent rather than the store growing a second
   product record.

## Code layout

The shape mirrors `skill/scripts/live/`, which solves the same class of problem.

```
skill/scripts/design-context/
  store.mjs            the only code that knows store paths or writes store files
  bindings.mjs         the editable-field registry for the document
  session-routes.mjs   HTTP handlers for the document edit session
  portability.mjs      the export bundle format and its import validation

skill/scripts/
  picker-server.mjs      static serving, boot contract, submit, autosave, spawn
  picker-doc-session.mjs the session shell: http server, timers, token
  picker-doc-poll.mjs    the agent's poll CLI
  design-context-export.mjs / design-context-import.mjs

picker/scripts/
  boot.js              one memoized fetch of the boot contract
  hydrate.js           restoring a previous run into the questionnaire
  palette-picker.js    the questionnaire; owns running hydration
  design-context.js    the document; document mode, pending ledger, save bar
```

Three ownership rules keep this correct as it grows.

**store.mjs is the single writer of store files.** Server, session, and import
all go through it. Every write is atomic: write a temporary file beside the
target, then rename over it, so a reader never sees a torn file.

**Reads come off disk per request.** Files are the truth. Process memory is a
cache at best. This is live mode's stat-keyed cache rule in its simplest form,
which is all a small store needs.

**During a live session, only the session process writes store files.** The agent
edits its own documents, DESIGN.md and PRODUCT.md, directly. Any answer or
context updates it needs to make ride in its reply payload for the session to
apply. Without that split, an agent doing a read, modify, write cycle on the
answers file can silently drop a change the session wrote in between.

## The live session

The document is a working surface, not a report. The patterns below are taken
from live mode, scaled to one page, one store, and one agent.

1. **Append-only journal with pure replay.** The session appends one line per
   render-relevant event to `runtime/journal.jsonl`: applied changes, batch
   transitions, and request transitions. A booting session replays the file to
   recover its sequence number and any unacknowledged batch. Lines it cannot use,
   including records written by an older release that carry no sequence number,
   become diagnostics rather than failures.
2. **Stage, then apply.** Edits accumulate in a client-side ledger keyed by field
   binding. Re-editing a field updates the new value but keeps the first original,
   so the record of what changed stays true across repeated edits.
3. **Delivery that survives a dead agent.** One batch is outstanding at a time and
   is journaled. A poll leases it, a reply acknowledges it, and a session restart
   re-offers anything unacknowledged. An invalid reply returns a corrective hint
   while the lease holds, so the agent can fix its own message.
4. **Server-owned truth, DOM as cache.** The state endpoint carries the journal
   sequence and the current batch. The client compares sequence numbers and
   re-fetches when the server has moved ahead. That re-fetch and re-render is the
   hot reload. The count of staged changes before Apply is the client's own,
   because nothing has reached the server yet.
5. **One in-flight lock.** While a batch is unacknowledged the document's edit
   affordances are disabled. A re-render triggered by anything else re-applies
   still-staged edits onto the fresh DOM, so a background event never wipes work
   the user has not saved.
6. **Loopback discovery with a per-run token.** The session records its process
   id, port, and token; readers probe liveness before trusting the record; the
   token, not the origin, is the security boundary.

What was deliberately left out, because each answers a question this surface does
not ask: the evidence gathering and verification pipeline and the repair loop
(live cannot know which source file a DOM edit belongs to, while every field here
has an authoritative address and applying is a deterministic write); server-sent
events and reload-resume state (a two-second poll already covers it); poll lanes
and priorities and chunking (one agent, one batch); the roots manifest (one fixed
path under the project root).

## Data flow

```
SEED
  agent stages assets into the store, runs the cue pipeline, writes the
  workspace and the store's context.json, then launches the picker
  client boots, prefills from a draft or a previous submission, autosaves
  submit writes answers, copies the chosen hero, forks the session, exits 0
  agent seeds DESIGN.md from the answers, then enters the edit loop

REOPEN
  the picker server runs in document mode, reusing or spawning a session
  the client renders the document directly, with no submit
  the session ending is the agent's completion signal

EDIT
  staged changes leave the ledger as one batch, the session applies them to
  the store and queues the downstream work, the agent reconciles DESIGN.md
  and PRODUCT.md prose and replies, the sequence advances, the tab re-fetches
  and re-renders

EXPORT AND IMPORT
  the store compiles to a readable markdown document plus a lossless bundle
  importing a bundle rebuilds a working store in another project
```