From 453a6e150030ee5ea4f58327f7acfca1c767e936 Mon Sep 17 00:00:00 2001 From: Paul Bakaus Date: Thu, 3 Sep 2026 10:49:42 -0700 Subject: [PATCH] bundle: the page JS and the bundler become a library crate downstream packs can reuse The in-page bundle, the extension pieces, the registry JSON and the wasm-pack call were reachable only through `cargo xtask bundle`, which read `browser-bundle/*.js` from the repo root. A downstream crate that links impeccable-core + impeccable-wasm with its own rule pack had to copy the page JS to produce a detector bundle for its module. They move to `impeccable-bundle` (crates/bundle), which embeds every `browser-bundle/*.js` with `include_str!` and exposes `in_page_bundle`, `extension_pieces`, `registry_json`, `check_capture_contract` and `wasm_pack_build`. Nothing writes files or exits the process; the caller places the bytes. `registry_json` now reads `all_antipatterns()`, so an installed pack's rows land in `antipatterns.json` too (no built-in change). xtask becomes the workspace's caller and writes the same files to the same places; `cargo xtask bundle` is byte-identical, tracked live asset included. `IMPECCABLE_BUNDLE_SKIP_WASM_PACK` is the skip switch's new name, the old `IMPECCABLE_XTASK_SKIP_WASM_PACK` still works. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01Vau2X53xGTjjTCXWMVBoNY --- CLAUDE.md | 2 +- Cargo.lock | 13 +- Cargo.toml | 1 + browser-bundle/README.md | 16 +- crates/bundle/Cargo.toml | 11 + crates/bundle/src/lib.rs | 441 +++++++++++++++++++++++++++++++++++++++ crates/xtask/Cargo.toml | 4 +- crates/xtask/src/main.rs | 271 +++--------------------- docs/ENGINE.md | 33 ++- 9 files changed, 539 insertions(+), 253 deletions(-) create mode 100644 crates/bundle/Cargo.toml create mode 100644 crates/bundle/src/lib.rs diff --git a/CLAUDE.md b/CLAUDE.md index 0122c92f5..31fca9388 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -120,7 +120,7 @@ bun run rebuild:release # Clean and rebuild with root harness sync bun run fetch:engine # Download the pinned engine binary for this machine into skill/scripts/bin/ ``` -The skill's `scripts/` payload is copied verbatim to every provider (launcher with its executable bit, `impeccable.cmd`, `VERSION`, `command-metadata.json`, page JS); nothing under `skill/scripts/bin/` is read as source. The in-page detector bundle and the extension's detector pieces are produced by `cargo xtask bundle`, which `bun run build:extension` runs. +The skill's `scripts/` payload is copied verbatim to every provider (launcher with its executable bit, `impeccable.cmd`, `VERSION`, `command-metadata.json`, page JS); nothing under `skill/scripts/bin/` is read as source. The in-page detector bundle and the extension's detector pieces are produced by `cargo xtask bundle`, which `bun run build:extension` runs; the page JS and the bundling itself live in the `impeccable-bundle` library crate (`crates/bundle`) so a downstream rule pack can build the same artifacts for its own wasm module. Source files use placeholders that get replaced per-provider: - `{{model}}` — Model name (Claude, Gemini, GPT, etc.) diff --git a/Cargo.lock b/Cargo.lock index edf432995..0a5b368fb 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -540,6 +540,15 @@ dependencies = [ "url", ] +[[package]] +name = "impeccable-bundle" +version = "0.1.0" +dependencies = [ + "base64", + "impeccable-core", + "serde_json", +] + [[package]] name = "impeccable-common" version = "0.1.0" @@ -1672,9 +1681,7 @@ checksum = "3ad82d2a33cdc9674dc7465672f271e096168fcdbe0f799d9e6db8c5892679dc" name = "xtask" version = "0.1.0" dependencies = [ - "base64", - "impeccable-core", - "serde_json", + "impeccable-bundle", ] [[package]] diff --git a/Cargo.toml b/Cargo.toml index 6c9dccac7..dcca399c8 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -23,6 +23,7 @@ impeccable-context = { path = "crates/context" } impeccable-hook = { path = "crates/hook" } impeccable-comp = { path = "crates/comp" } impeccable-comp-verbs = { path = "crates/comp-verbs" } +impeccable-bundle = { path = "crates/bundle" } serde = { version = "1", features = ["derive"] } serde_json = { version = "1", features = ["preserve_order"] } thiserror = "2" diff --git a/browser-bundle/README.md b/browser-bundle/README.md index 3d011d7b8..460ef3f96 100644 --- a/browser-bundle/README.md +++ b/browser-bundle/README.md @@ -10,10 +10,18 @@ Two consumers: - `crates/browser` embeds `15-snapshot.js` (the snapshot producer the URL engine injects; no WebAssembly runs in the page). -- `cargo xtask bundle` concatenates these files, in filename order, with the - wasm core into `dist/detect-antipatterns-browser.js`, copies that bundle to - the tracked `crates/live/assets/detect-antipatterns-browser.js` the engine - embeds, and writes the extension's `extension/detector/` pieces. +- `crates/bundle` (the `impeccable-bundle` library) embeds every file here + with `include_str!` and concatenates them, in filename order, with the wasm + core into the in-page bundle plus the extension's `extension/detector/` + pieces. `cargo xtask bundle` is its caller inside this workspace: it writes + `dist/detect-antipatterns-browser.js`, copies that bundle to the tracked + `crates/live/assets/detect-antipatterns-browser.js` the engine embeds, and + writes the extension pieces. A downstream crate with its own rule pack + calls the library directly (`docs/ENGINE.md`). + +Because the files are embedded, a new one here has to be added to +`PAGE_JS` in `crates/bundle/src/lib.rs` (and to the order it is concatenated +in); a test fails when the two lists disagree. `15-snapshot.js` lists the computed-style properties the rules read; the bundle build checks that list against the core's and fails when they drift. diff --git a/crates/bundle/Cargo.toml b/crates/bundle/Cargo.toml new file mode 100644 index 000000000..8e64eaf17 --- /dev/null +++ b/crates/bundle/Cargo.toml @@ -0,0 +1,11 @@ +[package] +name = "impeccable-bundle" +edition.workspace = true +version.workspace = true +license.workspace = true +publish.workspace = true + +[dependencies] +impeccable-core = { workspace = true } +serde_json = { workspace = true } +base64 = "0.22" diff --git a/crates/bundle/src/lib.rs b/crates/bundle/src/lib.rs new file mode 100644 index 000000000..229966031 --- /dev/null +++ b/crates/bundle/src/lib.rs @@ -0,0 +1,441 @@ +//! impeccable-bundle: the page JS and the bundler behind `cargo xtask bundle`. +//! +//! Every browser artifact the detector ships is this crate's output: the +//! in-page bundle (`dist/detect-antipatterns-browser.js`, also the tracked +//! `crates/live/assets/` copy the engine embeds and serves as `/detect.js`) +//! and the five `extension/detector/` pieces. The page JS is embedded with +//! `include_str!`, so a crate that depends on this one gets it without +//! copying `browser-bundle/*.js` anywhere. +//! +//! A downstream crate that links `impeccable-core` + `impeccable-wasm` plus +//! its own [`RulePack`](impeccable_core::rule_pack::RulePack) into one wasm +//! module builds the same artifacts for its own module: +//! +//! ```no_run +//! use std::path::Path; +//! let (glue, wasm) = impeccable_bundle::wasm_pack_build( +//! Path::new("crates/my-wasm"), +//! Path::new("target/wasm-bundle"), +//! &[], +//! )?; +//! let js = impeccable_bundle::in_page_bundle(&glue, &wasm); +//! let registry = impeccable_bundle::registry_json(); +//! let ext = impeccable_bundle::extension_pieces(&glue, &wasm, ®istry); +//! # Ok::<(), String>(()) +//! ``` +//! +//! The pack's rows reach [`registry_json`] once the pack is installed +//! (`impeccable_core::rule_pack::install`), because the registry reads +//! built-ins plus every registered slice. +//! +//! Nothing here writes files or exits the process: the caller decides where +//! the bytes go and how a failure is reported. + +use base64::Engine as _; +use std::borrow::Cow; +use std::path::{Path, PathBuf}; +use std::process::Command; + +macro_rules! page_js { + ($($name:literal),* $(,)?) => { + /// The page JS, in the order [`in_page_bundle`] concatenates it: + /// `(file name, source)`. Embedded at compile time from + /// `browser-bundle/`. + pub const PAGE_JS: &[(&str, &str)] = &[ + $(($name, include_str!(concat!("../../../browser-bundle/", $name))),)* + ]; + }; +} + +page_js![ + "00-header.js", + "10-probe.js", + "15-snapshot.js", + "30-scan-common.js", + "35-visual.js", + "40-overlay.js", + "50-scan.js", + "60-offscreen.js", + "99-footer.js", +]; + +/// One embedded page file by name. +pub fn page_js(name: &str) -> Option<&'static str> { + PAGE_JS.iter().find(|(n, _)| *n == name).map(|(_, src)| *src) +} + +fn src(name: &str) -> Cow<'static, str> { + let s = page_js(name).unwrap_or_else(|| panic!("browser-bundle/{name}: not embedded")); + if s.ends_with('\n') { + Cow::Borrowed(s) + } else { + Cow::Owned(format!("{s}\n")) + } +} + +/// In-page bundle concatenation order. `@@GLUE@@` is the wasm-bindgen glue, +/// `@@WASM@@` the embedded module + synchronous instantiation. +const ORDER: &[&str] = &[ + "00-header.js", + "10-probe.js", + "15-snapshot.js", + "@@GLUE@@", + "@@WASM@@", + "30-scan-common.js", + "35-visual.js", + "40-overlay.js", + "50-scan.js", + "99-footer.js", +]; + +/// The extension pieces, each an IIFE over a subset of the same sources plus +/// a `window.*` export line. `core.js` is the wasm glue + an async loader +/// (the module ships beside it as `core_bg.wasm`; no base64: the offscreen +/// document fetches it) + the scan plumbing and visual-contrast orchestration +/// + the offscreen session protocol. +const EXT_SNAPSHOT: &[&str] = &["15-snapshot.js"]; +const EXT_OVERLAY: &[&str] = &["40-overlay.js"]; +const EXT_CORE: &[&str] = &["@@GLUE@@", "@@LOADER@@", "30-scan-common.js", "35-visual.js", "60-offscreen.js"]; + +fn push_glue(out: &mut String, glue_js: &str) { + out.push_str("// --- wasm-bindgen glue (generated by cargo xtask bundle) ---\n"); + out.push_str(glue_js); + if !glue_js.ends_with('\n') { + out.push('\n'); + } +} + +/// The in-page bundle: `dist/detect-antipatterns-browser.js` and the tracked +/// `crates/live/assets/` copy. `glue_js` and `wasm` are one wasm-pack +/// `--target no-modules` build (see [`wasm_pack_build`]); the module is +/// embedded as base64 and instantiated synchronously at load. +/// +/// Deterministic: the same glue and module produce the same bytes. +pub fn in_page_bundle(glue_js: &str, wasm: &[u8]) -> String { + let b64 = base64::engine::general_purpose::STANDARD.encode(wasm); + let mut out = String::new(); + for part in ORDER { + match *part { + "@@GLUE@@" => push_glue(&mut out, glue_js), + "@@WASM@@" => { + out.push_str("// --- impeccable_wasm module (generated by cargo xtask bundle) ---\n"); + out.push_str(&format!("const __IMPECCABLE_WASM_BYTES = {};\n", wasm.len())); + out.push_str("const __IMPECCABLE_WASM_B64 = \""); + out.push_str(&b64); + out.push_str("\";\n"); + out.push_str(WASM_INIT); + } + name => out.push_str(&src(name)), + } + } + out +} + +/// What goes into `extension/detector/`, from [`extension_pieces`]. +pub struct ExtensionPieces { + /// `snapshot.js`: the content-script page snapshot producer. + pub snapshot_js: String, + /// `overlay.js`: the content-script overlay UI. + pub overlay_js: String, + /// `core.js`: the offscreen-document wasm loader and scan session. + pub core_js: String, + /// `core_bg.wasm`: the module `core.js` fetches beside itself. + pub core_bg_wasm: Vec, + /// `antipatterns.json`: the registry slice the extension panel reads. + pub antipatterns_json: String, +} + +/// The extension pieces for one wasm build. `registry_json` is written +/// through unchanged, so a caller can pass [`registry_json()`] or its own. +pub fn extension_pieces(glue_js: &str, wasm: &[u8], registry_json: &str) -> ExtensionPieces { + let piece = |parts: &[&str], exports: &str, what: &str| -> String { + let mut p = format!( + "/**\n * Impeccable extension: {what}\n * Copyright (c) 2026 Paul Bakaus\n *\n * GENERATED -- do not edit. Source: browser-bundle/*.js (+ crates/core, crates/wasm for core.js).\n * Rebuild: cargo xtask bundle\n */\n" + ); + p.push_str("(function () {\n"); + for part in parts { + match *part { + "@@GLUE@@" => push_glue(&mut p, glue_js), + "@@LOADER@@" => p.push_str(EXT_CORE_LOADER), + name => p.push_str(&src(name)), + } + } + p.push_str(exports); + p.push_str("})();\n"); + p + }; + ExtensionPieces { + snapshot_js: piece( + EXT_SNAPSHOT, + "window.__impeccableSnapshot = __impeccableSnapshot;\nwindow.__impeccableCreateDrawableIO = __createDrawableIO;\n", + "snapshot.js, the content-script page snapshot producer (measurement only)", + ), + overlay_js: piece( + EXT_OVERLAY, + "window.__impeccableCreateOverlay = createImpeccableOverlay;\n", + "overlay.js, the content-script overlay UI (draws a findings list; no rules)", + ), + core_js: piece( + EXT_CORE, + "", + "core.js, the offscreen-document WASM core loader + scan session (rules run in core_bg.wasm)", + ), + core_bg_wasm: wasm.to_vec(), + antipatterns_json: registry_json.to_string(), + } +} + +/// The `atob` + synchronous instantiation that runs at bundle load. Chrome +/// (verified on 151, headless and headed) compiles multi-MB modules +/// synchronously on the main thread; the historical 4 KB limit no longer +/// applies to `new WebAssembly.Module`. A CSP without 'wasm-unsafe-eval' +/// throws here; consumers handle that per docs/WASM-BUNDLE.md. +const WASM_INIT: &str = r#"function __impeccableWasmBytes() { + const bin = atob(__IMPECCABLE_WASM_B64); + const bytes = new Uint8Array(bin.length); + for (let i = 0; i < bin.length; i++) bytes[i] = bin.charCodeAt(i); + return bytes; +} +let __impeccable = null; +let __impeccableInitError = null; +try { + wasm_bindgen.initSync({ module: __impeccableWasmBytes() }); + __impeccable = wasm_bindgen; +} catch (e) { + __impeccableInitError = e; +} +"#; + +/// The offscreen document's core loader: fetch the module beside the script +/// and instantiate asynchronously (the extension's own CSP carries +/// 'wasm-unsafe-eval'). Also stubs the live-page probe namespace the glue +/// imports, so a call that would need a page fails loudly instead of with a +/// ReferenceError. +const EXT_CORE_LOADER: &str = r#"const __impeccableDom = new Proxy({}, { + get() { throw new Error('[impeccable] the offscreen document has no live page; load a snapshot first'); }, +}); +async function __impeccableLoadCore() { + await wasm_bindgen({ module_or_path: chrome.runtime.getURL('detector/core_bg.wasm') }); + return wasm_bindgen; +} +"#; + +/// The capture contract: the property and state lists in `15-snapshot.js` +/// must equal the core's (`STYLE_PROPS`, `PSEUDO_PROPS` in snapshot.rs; +/// `STATE_PSEUDOS` in selector.rs), or a rule reads a column the capture did +/// not write. Returns the mismatch report on drift. +pub fn check_capture_contract() -> Result<(), String> { + let snapshot_js = page_js("15-snapshot.js").expect("15-snapshot.js embedded"); + fn js_list(src: &str, name: &str) -> Vec { + let start = src + .find(&format!("const {name} = [")) + .unwrap_or_else(|| panic!("15-snapshot.js: {name} not found")); + let rest = &src[start..]; + let end = rest.find("];").expect("list end"); + rest[..end] + .split('"') + .skip(1) + .step_by(2) + .map(|s| s.to_string()) + .collect() + } + let pairs: [(&str, Vec); 3] = [ + ("__SNAP_STYLE_PROPS", impeccable_core::browser::snapshot::STYLE_PROPS.iter().map(|s| s.to_string()).collect()), + ("__SNAP_PSEUDO_PROPS", impeccable_core::browser::snapshot::PSEUDO_PROPS.iter().map(|s| s.to_string()).collect()), + ("__SNAP_STATE_PSEUDOS", impeccable_core::browser::selector::STATE_PSEUDOS.iter().map(|s| s.to_string()).collect()), + ]; + for (name, want) in pairs { + let have = js_list(snapshot_js, name); + if have != want { + return Err(format!( + "browser-bundle/15-snapshot.js {name} differs from the core's list\n js: {have:?}\n core: {want:?}" + )); + } + } + Ok(()) +} + +/// `antipatterns.json`: `{ id, name, category, description }` per rule, in +/// registry order (built-ins first, then any installed rule pack's rows), as +/// 2-space JSON with a trailing newline. This is the shape `bun run build` +/// and the extension panel read. +pub fn registry_json() -> String { + let rows: Vec = impeccable_core::registry::all_antipatterns() + .map(|ap| { + serde_json::json!({ + "id": ap.id, + "name": ap.name, + "category": ap.category, + "description": ap.description, + }) + }) + .collect(); + let mut s = serde_json::to_string_pretty(&rows).expect("registry json"); + s.push('\n'); + s +} + +/// Build the wasm module for `crate_dir` and read back the wasm-bindgen glue +/// and the `.wasm`: +/// +/// ```text +/// wasm-pack build --target no-modules --release \ +/// --no-typescript --no-pack --out-dir --out-name impeccable +/// ``` +/// +/// `extra_cargo_args` are passed to cargo after `--` (the workspace uses this +/// for `--features pure-exports`). `WASM_PACK` names the binary when it is +/// not on PATH; `IMPECCABLE_BUNDLE_SKIP_WASM_PACK=1` (or the older +/// `IMPECCABLE_XTASK_SKIP_WASM_PACK=1`) reuses whatever `out_dir` already +/// holds, for iterating on the page JS alone. +/// +/// A downstream crate passes its own crate dir, so the module it gets back is +/// the engine plus its rule pack. +pub fn wasm_pack_build( + crate_dir: &Path, + out_dir: &Path, + extra_cargo_args: &[&str], +) -> Result<(String, Vec), String> { + let skip = std::env::var_os("IMPECCABLE_BUNDLE_SKIP_WASM_PACK").is_some() + || std::env::var_os("IMPECCABLE_XTASK_SKIP_WASM_PACK").is_some(); + if !skip { + let wasm_pack = std::env::var("WASM_PACK").unwrap_or_else(|_| "wasm-pack".to_string()); + let mut cmd = Command::new(&wasm_pack); + cmd.arg("build") + .arg(crate_dir) + .arg("--target") + .arg("no-modules") + .arg("--release") + .arg("--no-typescript") + .arg("--no-pack") + .arg("--out-dir") + .arg(out_dir) + .arg("--out-name") + .arg("impeccable") + // Size profile for the module; native builds keep opt-level 3. + .env("CARGO_PROFILE_RELEASE_OPT_LEVEL", "z") + // wasm-pack refuses to build a `cdylib` whose Cargo.toml lives in + // a workspace with `[profile.*]` overrides only when it cannot + // find the target dir; keep it explicit. + .env("CARGO_TARGET_DIR", target_dir_for(crate_dir)); + if !extra_cargo_args.is_empty() { + cmd.arg("--"); + for arg in extra_cargo_args { + cmd.arg(arg); + } + } + let status = cmd + .status() + .map_err(|e| format!("wasm-pack build: failed to spawn {wasm_pack}: {e}"))?; + if !status.success() { + return Err(format!("wasm-pack build failed ({status})")); + } + } + let glue_path = out_dir.join("impeccable.js"); + let wasm_path = out_dir.join("impeccable_bg.wasm"); + let glue = std::fs::read_to_string(&glue_path) + .map_err(|e| format!("{}: {e}", glue_path.display()))?; + let wasm = std::fs::read(&wasm_path).map_err(|e| format!("{}: {e}", wasm_path.display()))?; + Ok((glue, wasm)) +} + +/// `CARGO_TARGET_DIR` for a wasm-pack run: the caller's, else the target dir +/// of the workspace `crate_dir` sits in (nearest ancestor with a +/// `Cargo.lock`), else the crate's own. +fn target_dir_for(crate_dir: &Path) -> PathBuf { + if let Some(dir) = std::env::var_os("CARGO_TARGET_DIR") { + return PathBuf::from(dir); + } + let abs = std::fs::canonicalize(crate_dir).unwrap_or_else(|_| crate_dir.to_path_buf()); + abs.ancestors() + .find(|dir| dir.join("Cargo.lock").exists()) + .unwrap_or(&abs) + .join("target") +} + +#[cfg(test)] +mod tests { + use super::*; + + fn bundle_dir() -> PathBuf { + Path::new(env!("CARGO_MANIFEST_DIR")).join("../../browser-bundle") + } + + #[test] + fn embedded_files_match_the_directory() { + let mut on_disk: Vec = std::fs::read_dir(bundle_dir()) + .expect("browser-bundle/") + .map(|e| e.expect("dir entry").file_name().to_string_lossy().into_owned()) + .filter(|name| name.ends_with(".js")) + .collect(); + on_disk.sort(); + let mut embedded: Vec = PAGE_JS.iter().map(|(n, _)| n.to_string()).collect(); + embedded.sort(); + assert_eq!( + embedded, on_disk, + "browser-bundle/*.js changed: update PAGE_JS (and ORDER, if the file belongs in the page bundle)" + ); + } + + #[test] + fn embedded_sources_match_the_files_on_disk() { + for (name, src) in PAGE_JS { + let disk = std::fs::read_to_string(bundle_dir().join(name)).expect(name); + assert_eq!(*src, disk, "{name} drifted from disk"); + } + } + + #[test] + fn in_page_bundle_has_the_expected_skeleton() { + let out = in_page_bundle("const glue = 1;\n", &[0, 97, 115, 109]); + assert!(out.starts_with(page_js("00-header.js").unwrap())); + assert!(out.ends_with(page_js("99-footer.js").unwrap())); + assert!(out.contains("// --- wasm-bindgen glue (generated by cargo xtask bundle) ---\nconst glue = 1;\n")); + assert!(out.contains("const __IMPECCABLE_WASM_BYTES = 4;\n")); + // base64 of the four-byte wasm preamble. + assert!(out.contains("const __IMPECCABLE_WASM_B64 = \"AGFzbQ==\";\n")); + assert!(out.contains("wasm_bindgen.initSync({ module: __impeccableWasmBytes() });")); + // 60-offscreen.js is an extension-only piece. + assert!(!out.contains(page_js("60-offscreen.js").unwrap())); + // Order: probe and snapshot before the glue, scan after it. + let glue_at = out.find("const glue = 1;").unwrap(); + assert!(out.find("__SNAP_STYLE_PROPS").unwrap() < glue_at); + assert!(out.rfind("createImpeccableOverlay").unwrap() > glue_at); + } + + #[test] + fn extension_pieces_are_iifes_over_their_sources() { + let ext = extension_pieces("const glue = 1;\n", &[0, 97, 115, 109], "[]\n"); + for (what, js) in [("snapshot", &ext.snapshot_js), ("overlay", &ext.overlay_js), ("core", &ext.core_js)] { + assert!(js.starts_with("/**\n * Impeccable extension: "), "{what}"); + assert!(js.contains("(function () {\n"), "{what}"); + assert!(js.ends_with("})();\n"), "{what}"); + } + assert!(ext.snapshot_js.contains("window.__impeccableSnapshot = __impeccableSnapshot;")); + assert!(ext.overlay_js.contains("window.__impeccableCreateOverlay = createImpeccableOverlay;")); + assert!(ext.core_js.contains("chrome.runtime.getURL('detector/core_bg.wasm')")); + // The offscreen core fetches its module; nothing is base64 in there. + assert!(!ext.core_js.contains("__IMPECCABLE_WASM_B64")); + assert_eq!(ext.core_bg_wasm, vec![0, 97, 115, 109]); + assert_eq!(ext.antipatterns_json, "[]\n"); + } + + #[test] + fn capture_contract_holds() { + check_capture_contract().unwrap(); + } + + #[test] + fn registry_json_is_two_space_rows() { + let json = registry_json(); + assert!(json.ends_with("]\n")); + assert!(json.contains("\n {\n \"id\": ")); + let rows: Vec = serde_json::from_str(&json).expect("valid json"); + assert_eq!(rows.len(), impeccable_core::registry::all_antipatterns().count()); + for row in &rows { + for key in ["id", "name", "category", "description"] { + assert!(row.get(key).is_some(), "row missing {key}: {row}"); + } + } + } +} diff --git a/crates/xtask/Cargo.toml b/crates/xtask/Cargo.toml index 0b75dfa2c..5fcb97dea 100644 --- a/crates/xtask/Cargo.toml +++ b/crates/xtask/Cargo.toml @@ -10,6 +10,4 @@ name = "xtask" path = "src/main.rs" [dependencies] -impeccable-core = { workspace = true } -serde_json = { workspace = true } -base64 = "0.22" +impeccable-bundle = { workspace = true } diff --git a/crates/xtask/src/main.rs b/crates/xtask/src/main.rs index a53fd091e..31438b091 100644 --- a/crates/xtask/src/main.rs +++ b/crates/xtask/src/main.rs @@ -1,11 +1,15 @@ //! Workspace tasks that need no Node. //! -//! `cargo xtask bundle` builds the in-page detector bundle: +//! `cargo xtask bundle` builds the in-page detector bundle. The bundling +//! itself lives in `impeccable-bundle` (`crates/bundle`), the library a +//! downstream rule pack reuses for its own wasm module; this task is the +//! workspace's caller of it: //! 1. `wasm-pack build crates/wasm --target no-modules --release` //! (opt-level z via `CARGO_PROFILE_RELEASE_OPT_LEVEL`, wasm-opt from //! the crate metadata), into `target/wasm-bundle/`; -//! 2. concatenates the page JS (`browser-bundle/*.js`) in a fixed order -//! with the wasm-bindgen glue and the .wasm embedded as base64; +//! 2. concatenates the page JS (`browser-bundle/*.js`, embedded in +//! `impeccable-bundle`) in a fixed order with the wasm-bindgen glue and +//! the .wasm embedded as base64; //! 3. writes `dist/detect-antipatterns-browser.js` (deterministic: same //! sources, same bytes) and `dist/antipatterns.json` (the registry //! slice the extension panel reads), and copies the bundle to @@ -24,9 +28,7 @@ //! `cargo xtask bundle --check` rebuilds and fails when the tracked live //! asset differs (CI staleness gate). -use base64::Engine; use std::path::{Path, PathBuf}; -use std::process::Command; fn root() -> PathBuf { Path::new(env!("CARGO_MANIFEST_DIR")) @@ -35,30 +37,6 @@ fn root() -> PathBuf { .expect("workspace root") } -/// browser-bundle/*.js concatenation order. `@@GLUE@@` is the wasm-bindgen -/// glue, `@@WASM@@` the embedded module + synchronous instantiation. -const ORDER: &[&str] = &[ - "00-header.js", - "10-probe.js", - "15-snapshot.js", - "@@GLUE@@", - "@@WASM@@", - "30-scan-common.js", - "35-visual.js", - "40-overlay.js", - "50-scan.js", - "99-footer.js", -]; - -/// The extension pieces (`dist/extension/`), each an IIFE over a subset of -/// the same sources plus a `window.*` export line. `core.js` is the wasm -/// glue + an async loader (the module ships beside it as `core_bg.wasm`; -/// no base64: the offscreen document fetches it) + the scan plumbing and -/// visual-contrast orchestration + the offscreen session protocol. -const EXT_SNAPSHOT: &[&str] = &["15-snapshot.js"]; -const EXT_OVERLAY: &[&str] = &["40-overlay.js"]; -const EXT_CORE: &[&str] = &["@@GLUE@@", "@@LOADER@@", "30-scan-common.js", "35-visual.js", "60-offscreen.js"]; - fn main() { let args: Vec = std::env::args().skip(1).collect(); match args.first().map(String::as_str) { @@ -73,126 +51,28 @@ fn main() { } } -fn run(cmd: &mut Command, what: &str) { - let status = cmd.status().unwrap_or_else(|e| panic!("{what}: failed to spawn: {e}")); - if !status.success() { - eprintln!("{what} failed ({status})"); - std::process::exit(1); - } +fn die(message: String) -> ! { + eprintln!("{message}"); + std::process::exit(1); } /// `pure`: also compile the `pure_*` exports (feature `pure-exports`). fn bundle(check: bool, pure: bool) { let root = root(); let out_dir = root.join("target/wasm-bundle"); - let wasm_pack = std::env::var("WASM_PACK").unwrap_or_else(|_| "wasm-pack".to_string()); - let mut cmd = Command::new(&wasm_pack); - cmd.current_dir(&root) - .arg("build") - .arg("crates/wasm") - .arg("--target") - .arg("no-modules") - .arg("--release") - .arg("--no-typescript") - .arg("--no-pack") - .arg("--out-dir") - .arg(&out_dir) - .arg("--out-name") - .arg("impeccable") - // Size profile for the module; native builds keep opt-level 3. - .env("CARGO_PROFILE_RELEASE_OPT_LEVEL", "z") - // wasm-pack refuses to build a `cdylib` whose Cargo.toml lives in a - // workspace with `[profile.*]` overrides only when it cannot find - // the target dir; keep it explicit. - .env("CARGO_TARGET_DIR", root.join("target")); - if pure { - cmd.arg("--").arg("--features").arg("pure-exports"); - } - if std::env::var_os("IMPECCABLE_XTASK_SKIP_WASM_PACK").is_none() { - run(&mut cmd, "wasm-pack build"); + let cargo_args: &[&str] = if pure { &["--features", "pure-exports"] } else { &[] }; + let (glue, wasm) = + impeccable_bundle::wasm_pack_build(&root.join("crates/wasm"), &out_dir, cargo_args) + .unwrap_or_else(|e| die(e)); + + if let Err(mismatch) = impeccable_bundle::check_capture_contract() { + die(mismatch); } - let glue = std::fs::read_to_string(out_dir.join("impeccable.js")).expect("wasm-bindgen glue"); - let wasm = std::fs::read(out_dir.join("impeccable_bg.wasm")).expect("wasm module"); - let b64 = base64::engine::general_purpose::STANDARD.encode(&wasm); - let bundle_dir = root.join("browser-bundle"); - let src = |name: &str| -> String { - let s = std::fs::read_to_string(bundle_dir.join(name)) - .unwrap_or_else(|e| panic!("browser-bundle/{name}: {e}")); - if s.ends_with('\n') { - s - } else { - s + "\n" - } - }; - check_capture_contract(&src("15-snapshot.js")); + let out = impeccable_bundle::in_page_bundle(&glue, &wasm); + let registry = impeccable_bundle::registry_json(); + let ext = impeccable_bundle::extension_pieces(&glue, &wasm, ®istry); - let mut out = String::new(); - for part in ORDER { - match *part { - "@@GLUE@@" => { - out.push_str("// --- wasm-bindgen glue (generated by cargo xtask bundle) ---\n"); - out.push_str(&glue); - if !glue.ends_with('\n') { - out.push('\n'); - } - } - "@@WASM@@" => { - out.push_str("// --- impeccable_wasm module (generated by cargo xtask bundle) ---\n"); - out.push_str(&format!("const __IMPECCABLE_WASM_BYTES = {};\n", wasm.len())); - out.push_str("const __IMPECCABLE_WASM_B64 = \""); - out.push_str(&b64); - out.push_str("\";\n"); - out.push_str(WASM_INIT); - } - name => out.push_str(&src(name)), - } - } - - // Extension pieces. - let ext_piece = |parts: &[&str], exports: &str, header: &str| -> String { - let mut p = String::new(); - p.push_str(header); - p.push_str("(function () {\n"); - for part in parts { - match *part { - "@@GLUE@@" => { - p.push_str("// --- wasm-bindgen glue (generated by cargo xtask bundle) ---\n"); - p.push_str(&glue); - if !glue.ends_with('\n') { - p.push('\n'); - } - } - "@@LOADER@@" => p.push_str(EXT_CORE_LOADER), - name => p.push_str(&src(name)), - } - } - p.push_str(exports); - p.push_str("})();\n"); - p - }; - let ext_header = |what: &str| { - format!( - "/**\n * Impeccable extension: {what}\n * Copyright (c) 2026 Paul Bakaus\n *\n * GENERATED -- do not edit. Source: browser-bundle/*.js (+ crates/core, crates/wasm for core.js).\n * Rebuild: cargo xtask bundle\n */\n" - ) - }; - let ext_snapshot = ext_piece( - EXT_SNAPSHOT, - "window.__impeccableSnapshot = __impeccableSnapshot;\nwindow.__impeccableCreateDrawableIO = __createDrawableIO;\n", - &ext_header("snapshot.js, the content-script page snapshot producer (measurement only)"), - ); - let ext_overlay = ext_piece( - EXT_OVERLAY, - "window.__impeccableCreateOverlay = createImpeccableOverlay;\n", - &ext_header("overlay.js, the content-script overlay UI (draws a findings list; no rules)"), - ); - let ext_core = ext_piece( - EXT_CORE, - "", - &ext_header("core.js, the offscreen-document WASM core loader + scan session (rules run in core_bg.wasm)"), - ); - - let registry = registry_json(); let dist = root.join("dist"); // The one tracked generated file: live mode embeds it (include_str! in // crates/live/src/browser_assets.rs) and serves it as /detect.js. @@ -214,111 +94,24 @@ fn bundle(check: bool, pure: bool) { // extension/detector/: gitignored, vendored by `bun run build:extension`. let ext_dir = root.join("extension/detector"); std::fs::create_dir_all(&ext_dir).expect("extension dir"); - std::fs::write(ext_dir.join("snapshot.js"), &ext_snapshot).expect("write snapshot.js"); - std::fs::write(ext_dir.join("overlay.js"), &ext_overlay).expect("write overlay.js"); - std::fs::write(ext_dir.join("core.js"), &ext_core).expect("write core.js"); - std::fs::write(ext_dir.join("core_bg.wasm"), &wasm).expect("write core_bg.wasm"); - std::fs::write(ext_dir.join("antipatterns.json"), ®istry).expect("write registry"); + std::fs::write(ext_dir.join("snapshot.js"), &ext.snapshot_js).expect("write snapshot.js"); + std::fs::write(ext_dir.join("overlay.js"), &ext.overlay_js).expect("write overlay.js"); + std::fs::write(ext_dir.join("core.js"), &ext.core_js).expect("write core.js"); + std::fs::write(ext_dir.join("core_bg.wasm"), &ext.core_bg_wasm).expect("write core_bg.wasm"); + std::fs::write(ext_dir.join("antipatterns.json"), &ext.antipatterns_json).expect("write registry"); println!( "extension/detector/: snapshot.js {} KB, overlay.js {} KB, core.js {} KB, core_bg.wasm {} KB", - ext_snapshot.len() / 1024, - ext_overlay.len() / 1024, - ext_core.len() / 1024, - wasm.len() / 1024 + ext.snapshot_js.len() / 1024, + ext.overlay_js.len() / 1024, + ext.core_js.len() / 1024, + ext.core_bg_wasm.len() / 1024 ); + let b64_len = wasm.len().div_ceil(3) * 4; println!( "dist/detect-antipatterns-browser.js: {} KB (wasm {} KB, base64 {} KB, js {} KB)", out.len() / 1024, wasm.len() / 1024, - b64.len() / 1024, - (out.len() - b64.len()) / 1024 + b64_len / 1024, + (out.len() - b64_len) / 1024 ); } - -/// The `atob` + synchronous instantiation that runs at bundle load. Chrome -/// (verified on 151, headless and headed) compiles multi-MB modules -/// synchronously on the main thread; the historical 4 KB limit no longer -/// applies to `new WebAssembly.Module`. A CSP without 'wasm-unsafe-eval' -/// throws here; consumers handle that per docs/WASM-BUNDLE.md. -const WASM_INIT: &str = r#"function __impeccableWasmBytes() { - const bin = atob(__IMPECCABLE_WASM_B64); - const bytes = new Uint8Array(bin.length); - for (let i = 0; i < bin.length; i++) bytes[i] = bin.charCodeAt(i); - return bytes; -} -let __impeccable = null; -let __impeccableInitError = null; -try { - wasm_bindgen.initSync({ module: __impeccableWasmBytes() }); - __impeccable = wasm_bindgen; -} catch (e) { - __impeccableInitError = e; -} -"#; - -/// The offscreen document's core loader: fetch the module beside the script -/// and instantiate asynchronously (the extension's own CSP carries -/// 'wasm-unsafe-eval'). Also stubs the live-page probe namespace the glue -/// imports, so a call that would need a page fails loudly instead of with a -/// ReferenceError. -const EXT_CORE_LOADER: &str = r#"const __impeccableDom = new Proxy({}, { - get() { throw new Error('[impeccable] the offscreen document has no live page; load a snapshot first'); }, -}); -async function __impeccableLoadCore() { - await wasm_bindgen({ module_or_path: chrome.runtime.getURL('detector/core_bg.wasm') }); - return wasm_bindgen; -} -"#; - -/// The capture contract: the property and state lists in 15-snapshot.js -/// must equal the core's (`STYLE_PROPS`, `PSEUDO_PROPS` in snapshot.rs; -/// `STATE_PSEUDOS` in selector.rs), or a rule reads a column the capture -/// did not write. -fn check_capture_contract(snapshot_js: &str) { - fn js_list(src: &str, name: &str) -> Vec { - let start = src - .find(&format!("const {name} = [")) - .unwrap_or_else(|| panic!("15-snapshot.js: {name} not found")); - let rest = &src[start..]; - let end = rest.find("];").expect("list end"); - rest[..end] - .split('"') - .skip(1) - .step_by(2) - .map(|s| s.to_string()) - .collect() - } - let pairs: [(&str, Vec); 3] = [ - ("__SNAP_STYLE_PROPS", impeccable_core::browser::snapshot::STYLE_PROPS.iter().map(|s| s.to_string()).collect()), - ("__SNAP_PSEUDO_PROPS", impeccable_core::browser::snapshot::PSEUDO_PROPS.iter().map(|s| s.to_string()).collect()), - ("__SNAP_STATE_PSEUDOS", impeccable_core::browser::selector::STATE_PSEUDOS.iter().map(|s| s.to_string()).collect()), - ]; - for (name, want) in pairs { - let have = js_list(snapshot_js, name); - if have != want { - eprintln!("browser-bundle/15-snapshot.js {name} differs from the core's list"); - eprintln!(" js: {have:?}"); - eprintln!(" core: {want:?}"); - std::process::exit(1); - } - } -} - -/// `dist/antipatterns.json`: `{ id, name, category, description }` per rule, -/// as `scripts/build-extension.js` writes it (2-space JSON, trailing newline). -fn registry_json() -> String { - let rows: Vec = impeccable_core::registry::ANTIPATTERNS - .iter() - .map(|ap| { - serde_json::json!({ - "id": ap.id, - "name": ap.name, - "category": ap.category, - "description": ap.description, - }) - }) - .collect(); - let mut s = serde_json::to_string_pretty(&rows).expect("registry json"); - s.push('\n'); - s -} diff --git a/docs/ENGINE.md b/docs/ENGINE.md index 72b96c64b..22ce71cc5 100644 --- a/docs/ENGINE.md +++ b/docs/ENGINE.md @@ -39,8 +39,9 @@ crates/ the browser rule adapters, the visual-contrast decisions wasm wasm-bindgen exports over `core` (the in-page bundle and the extension's offscreen core) - xtask `cargo xtask bundle`: builds the in-page bundle and the - extension pieces + bundle the page JS plus the bundler: in-page bundle, extension + pieces, registry JSON, the wasm-pack call + xtask `cargo xtask bundle`: the workspace's caller of `bundle` ``` `crates/core` re-exports the foundation modules under its own paths, so every @@ -96,7 +97,7 @@ sources, same bytes. `wasm-pack` is the one extra tool this needs (`cargo install wasm-pack --locked`) plus the `wasm32-unknown-unknown` target, which -`rust-toolchain.toml` requests. `IMPECCABLE_XTASK_SKIP_WASM_PACK=1` reuses +`rust-toolchain.toml` requests. `IMPECCABLE_BUNDLE_SKIP_WASM_PACK=1` reuses whatever is already in `target/wasm-bundle/`, for iterating on the page JS alone. `IMPECCABLE_EXTENSION_SKIP_BUNDLE=1` lets `bun run build:extension` skip the bundle step when `extension/detector/` is already complete, for CI @@ -105,6 +106,32 @@ matrices that pre-built it. Run `cargo xtask bundle` after touching `crates/core`, `crates/wasm`, or `browser-bundle/`, and commit the refreshed live asset. +### Reusing the bundler downstream + +None of that lives in the task. `impeccable-bundle` (`crates/bundle`) embeds +`browser-bundle/*.js` with `include_str!` and owns the assembly, so a crate +that links `impeccable-core` + `impeccable-wasm` plus its own rule pack into +one wasm module builds the same artifacts for that module without copying a +file out of this repo: + +```rust +let (glue, wasm) = impeccable_bundle::wasm_pack_build( + Path::new("crates/my-wasm"), // engine + pack, not crates/wasm + Path::new("target/wasm-bundle"), + &[], // extra cargo args, after `--` +)?; +let js = impeccable_bundle::in_page_bundle(&glue, &wasm); // /detect.js +let registry = impeccable_bundle::registry_json(); // built-ins + pack rows +let ext = impeccable_bundle::extension_pieces(&glue, &wasm, ®istry); +impeccable_bundle::check_capture_contract()?; // snapshot/core drift +``` + +Nothing there writes files or exits: the caller places the bytes and reports +its own failures. The pack's registry rows appear in `registry_json` once the +pack is installed, since the registry reads built-ins plus every registered +slice. `IMPECCABLE_BUNDLE_SKIP_WASM_PACK=1` is the library's name for the +skip switch (the old `IMPECCABLE_XTASK_SKIP_WASM_PACK=1` still works). + ## Rule packs The built-in rules are compiled in and always run. A **rule pack** is how a