From 1e9fd74ea77d3042493b1fd3bb15cfd7e8dc6458 Mon Sep 17 00:00:00 2001 From: Magnus Hedemark Date: Thu, 21 May 2026 22:46:32 -0400 Subject: [PATCH] =?UTF-8?q?feat:=20add=20systematic-debugging=20skill=20?= =?UTF-8?q?=E2=80=94=204-phase=20root=20cause=20protocol?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adapted from obra/superpowers (Jesse Vincent, MIT) and expanded with real-world debugging patterns from production use. Core additions: Phase 1 sub-steps for common failure modes: - Schema/environment divergence (test vs production schema diffing) - Exception type specificity (sibling exception traps in Pyhton) - Progressive characterization grid (isolate retrieval failures variable by variable) - Dependency source detection (editable dev forks causing schema drift) - Systematic web research protocol (structured search before guessing) - macOS sandboxed app debugging (containers, XPC, TCC, iCloud sync) - Trace data flow upstream from the symptom Key heuristics: - Rule of Three: 3+ failed fixes = question the architecture - Red Flags table: 12 rationalizations to catch yourself making - Investigation Flow: keep pushing, don't break momentum to ask questions Includes two worked example references (dev fork detection, macOS Books.app). Signed-off-by: Jasper --- systematic-debugging/SKILL.md | 451 ++++++++++++++++++ .../references/dependency-source-example.md | 63 +++ .../references/macos-sandbox-debug-example.md | 125 +++++ 3 files changed, 639 insertions(+) create mode 100644 systematic-debugging/SKILL.md create mode 100644 systematic-debugging/references/dependency-source-example.md create mode 100644 systematic-debugging/references/macos-sandbox-debug-example.md diff --git a/systematic-debugging/SKILL.md b/systematic-debugging/SKILL.md new file mode 100644 index 0000000..0fcb12e --- /dev/null +++ b/systematic-debugging/SKILL.md @@ -0,0 +1,451 @@ +--- +name: systematic-debugging +description: >- + 4-phase root cause debugging protocol: understand bugs before fixing. + Use for ANY technical issue — test failures, production bugs, unexpected + behavior, performance problems, build failures, or integration issues. + ESPECIALLY when under time pressure, when "one quick fix" seems obvious, + or when previous fix attempts have failed. +license: MIT +compatibility: >- + Platform-agnostic. Some sections cover macOS-specific sandbox debugging. + Requires access to source code, version control (git), and testing tools + appropriate to the project. +metadata: + tags: + - debugging + - troubleshooting + - problem-solving + - root-cause + - investigation + source: >- + Adapted from obra/superpowers (Jesse Vincent, MIT). + Expanded with real-world debugging patterns from production use. +--- + +# Systematic Debugging + +## Overview + +Random fixes waste time and create new bugs. Quick patches mask underlying issues. + +**Core principle:** ALWAYS find root cause before attempting fixes. Symptom fixes are failure. + +## The Iron Law + +``` +NO FIXES WITHOUT ROOT CAUSE INVESTIGATION FIRST +``` + +If you haven't completed Phase 1, you cannot propose fixes. + +## When to Use + +Use for ANY technical issue: +- Test failures +- Bugs in production +- Unexpected behavior +- Performance problems +- Build failures +- Integration issues + +**ESPECIALLY when:** +- Under time pressure (emergencies make guessing tempting) +- "Just one quick fix" seems obvious +- You've already tried multiple fixes +- Previous fix didn't work +- You don't fully understand the issue + +**Don't skip when:** +- Issue seems simple (simple bugs have root causes too) +- You're in a hurry (rushing guarantees rework) +- Someone wants it fixed NOW (systematic is faster than thrashing) + +## The Four Phases + +Complete each phase before proceeding to the next. + +--- + +## Phase 1: Root Cause Investigation + +**BEFORE attempting ANY fix:** + +### 1. Read Error Messages Carefully + +- Don't skip past errors or warnings — they often contain the exact solution +- Read stack traces completely. Note line numbers, file paths, error codes +- **Action:** Read the relevant source files at the error locations and search the codebase for the error string to find all related code paths. + +### 2. Reproduce Consistently + +- Can you trigger it reliably? What are the exact steps? +- If not reproducible → gather more data, don't guess +- **Action:** Run the failing test or trigger the bug: + +```bash +pytest tests/test_module.py::test_name -v --tb=long +``` + +### 3. Check Recent Changes + +- What changed that could cause this? Git diff, recent commits, new dependencies, config changes +- **Action:** + +```bash +git log --oneline -10 +git diff +git log -p --follow src/problematic_file.py | head -100 +``` + +### 4. Gather Evidence in Multi-Component Systems + +**WHEN system has multiple components (API → service → database, CI → build → deploy):** + +Add diagnostic instrumentation BEFORE proposing fixes. For EACH component boundary: +- Log what data enters the component +- Log what data exits the component +- Verify environment/config propagation +- Check state at each layer + +Run once to gather evidence showing WHERE it breaks. THEN analyze to identify the failing component. + +### 5a. Check Schema and Environment Divergence + +**WHEN a bug reproduces in production but not in tests:** + +1. Compare the test fixture schema against the production schema — `PRAGMA table_info()`, `\\.schema`, or equivalent +2. Look for missing NOT NULL columns, foreign keys, unique constraints, or defaults +3. Check for differences in SQLite journal mode, connection flags, or PRAGMA settings +4. Verify the test data matches production shape — not just column names but constraints + +Simplified test schemas are a common source of hidden bugs. If the production table has `model TEXT NOT NULL` but the test table only has `vector BLOB`, a bug that only fires on NOT NULL violation will pass tests cleanly. + +**Action:** Run `PRAGMA table_info(table_name)` against both databases side-by-side and diff the output. + +### 5b. Check Exception Type Specificity in Fallback Chains + +**WHEN a try/except fallback isn't catching the error you see in logs:** + +```python +try: + # Primary path — can raise OperationalError + conn.execute("INSERT INTO t (model_name) VALUES (?)", ...) +except sqlite3.OperationalError: + # Fallback — can raise IntegrityError (sibling, not child) + conn.execute("INSERT INTO t (vector) VALUES (?)", ...) +``` + +`OperationalError` and `IntegrityError` are **siblings** — both inherit from `DatabaseError`, which inherits from `Error`. Catching one does NOT catch the other. + +The fix is either: +- Catch the parent class (`except sqlite3.DatabaseError`) if both paths can fail with different subtypes +- Catch `Exception` as last resort (broader but safer than a gap) +- Handle each expected error type explicitly + +### 5c. Progressive Characterization — Tool & API Behavior + +**WHEN investigating a retrieval system, API, or knowledge graph that returns empty or inconsistent results:** + +Start from what works and expand until it breaks. This isolates the variable causing failure. + +| Input | Expected | Actual | Diagnosis | +|-------|----------|--------|-----------| +| Single known term (e.g. `python`) | Hits | Hits | Tool works, connection OK | +| Two-word phrase from same doc (`type safety`) | Hits | Hits | Short phrase retrieval works | +| Related two-word phrase (`generic types`) | Hits | 0 hits | **Boundary found** — issue is phrase-specific | +| Longer query with same terms | 0 hits | 0 hits | Confirms: not a fluke | + +**Do NOT skip characterization:** Jumping straight to "the embedding model is broken" is guessing. The grid eliminates variables one at a time. + +### 5d. Check Dependency Source Before Fixing Library Code + +**WHEN you trace a bug into a third-party dependency:** + +Before patching the library code, verify WHERE it's installed from: + +```bash +pip show +``` + +Key fields: +- **Editable project location** — If present, this is a `pip install -e` dev copy. Was this intentional? +- **Location** — Is it in site-packages (production) or a temp/dev directory? +- **Version** — Compare against latest on PyPI: `pip index versions ` + +**The rule:** If the package is installed as an editable dev copy and you didn't put it there intentionally, STOP and ask. The symptom may be caused by code diverging from upstream — and the right fix is to switch to the production package, not to patch the fork. + +**Example:** A background worker crashed with `table X has no column named Y`. Investigation traced it to an editable install from `/private/tmp/some-fork/`. A dev fork had added the column name to INSERT statements but never added the schema migration. The correct fix wasn't to add the migration to the fork — it was to switch to the production PyPI package and delete the dev copy. + +### 6a. Research Before Guessing — Systematic Web Search + +**WHEN you've gathered all local evidence but still don't understand the root cause:** + +Do NOT guess solutions. Use structured web research: + +1. **Search with the exact error message** — Quote the error, include error codes and function names +2. **Search with symptom + platform context** — Combine what broke + what OS/tool version +3. **Search with the component/service name** — XPC services, daemons, subsystems often have documented bugs +4. **Prioritize recent threads** — Filter for current versions. Old solutions may not apply +5. **Read the full thread** before proposing solutions — partial reading causes partial fixes +6. **Check for a confirmed workaround** at the thread's end, not just the initial diagnosis + +### 6b. macOS App Troubleshooting — Sandboxed Applications + +**WHEN debugging a macOS app (especially sandboxed ones like Books, Music, or App Store apps):** + +The app is confined to a sandbox container under `~/Library/Containers//`. + +#### Locate the Container + +```bash +ls ~/Library/Containers// +# Data/Library/ — preferences, caches, databases +# Data/Documents/ — user-visible content, import queues +``` + +#### Check for an XPC Service Companion + +Many Apple apps use a background XPC service for file operations: + +```bash +# XPC services live in the framework bundle or app bundle: +/System/Library/PrivateFrameworks/.framework/XPCServices/ +/System/Applications/.app/Contents/XPCServices/ +ps aux | grep -i "" +``` + +#### Read the Database Directly + +Sandboxed apps often use SQLite/CoreData: + +```bash +sqlite3 ~/Library/Containers//Data/Documents/.sqlite ".tables" +sqlite3 ~/Library/Containers//Data/Documents/.sqlite "SELECT * FROM ZTABLE LIMIT 10;" +``` + +#### Check System Logs + +```bash +log show --predicate 'process == "AppName"' --last 10m --style compact +log stream --predicate 'process == "AppName"' --style compact +``` + +#### Reset TCC Permissions + +If the app can't access files outside its sandbox (silent import failures): + +```bash +tccutil reset All com.apple.bundle-id +``` + +#### Know the I/O Boundary + +- **Security-scoped bookmarks** (from drag-drop or `open` command) are one-time-use. If the import fails, the bookmark is consumed and subsequent attempts fail silently. +- **NSOpenPanel** (File > Import dialog) creates fresh bookmarks — more reliable for testing. +- If `open -b bundle-id file.ext` works from Downloads but not Desktop, it's likely a TCC/tiered-access issue (macOS gives Downloads more permissive access). + +#### Differentiate Local vs Cloud Sync Corruption + +Container resets fix local state but NOT iCloud sync corruption. Signs of cloud issues: +- Problem persists after full container deletion and reinstall +- Import works once after restart then degrades +- Same issue across multiple devices + +**Action:** If local reset doesn't fix it, the iCloud sync state may be corrupted. System Settings → Apple ID → iCloud → Manage Storage → [App] → Delete All Data is the nuclear option. + +#### Recovery Options (in order of escalation) + +1. Kill and restart the XPC service (`kill -9 `) — temporary, XPC respawns +2. Reset the app container (`rm -rf ~/Library/Containers//`) +3. Reset TCC permissions (`tccutil reset All `) +4. Reboot the Mac +5. Delete iCloud data for the app +6. Create a test macOS user — if the app works there, it's your user library, not the system + +### 6c. Trace Data Flow + +**WHEN error is deep in the call stack:** + +- Where does the bad value originate? +- What called this function with the bad value? +- Keep tracing upstream until you find the source +- Fix at the source, not at the symptom + +**Action:** Search the codebase for function references and variable assignments to trace the data path. + +### Phase 1 Completion Checklist + +- [ ] Error messages fully read and understood +- [ ] Issue reproduced consistently +- [ ] Recent changes identified and reviewed +- [ ] Evidence gathered (logs, state, data flow) +- [ ] Problem isolated to specific component/code +- [ ] Root cause hypothesis formed + +**STOP:** Do not proceed to Phase 2 until you understand WHY it's happening. + +--- + +## Phase 2: Pattern Analysis + +**Find the pattern before fixing:** + +### 1. Find Working Examples + +- Locate similar working code in the same codebase +- What works that's similar to what's broken? + +### 2. Compare Against References + +- If implementing a pattern, read the reference implementation COMPLETELY — don't skim +- Understand the pattern fully before applying + +### 3. Identify Differences + +- What's different between working and broken? +- List every difference, however small +- Don't assume "that can't matter" +- **Action:** Search the codebase for similar patterns to compare. + +### 4. Understand Dependencies + +- What other components does this need? +- What settings, config, environment? +- What assumptions does it make? + +--- + +## Phase 3: Hypothesis and Testing + +**Scientific method:** + +### 1. Form a Single Hypothesis + +- State clearly: "I think X is the root cause because Y" +- Write it down. Be specific, not vague. + +### 2. Test Minimally + +- Make the SMALLEST possible change to test the hypothesis +- One variable at a time +- Don't fix multiple things at once + +### 3. Verify Before Continuing + +- Did it work? → Phase 4 +- Didn't work? → Form NEW hypothesis +- DON'T add more fixes on top + +### 4. When You Don't Know + +- Say "I don't understand X" — don't pretend to know +- Ask for help. Research more. + +--- + +## Phase 4: Implementation + +**Fix the root cause, not the symptom:** + +### 1. Create Failing Test Case + +- Simplest possible reproduction. Automated test if possible. +- MUST have before fixing. +- Test environment must mirror production — audit test fixtures against real schema. + +### 2. Implement Single Fix + +- Address the root cause identified. ONE change at a time. +- No "while I'm here" improvements. No bundled refactoring. + +### 3. Verify Fix + +```bash +# Run the specific regression test +pytest tests/test_module.py::test_name -v + +# Run full suite — no regressions +pytest tests/ -q +``` + +### 4. If Fix Doesn't Work — The Rule of Three + +- **STOP.** +- Count: How many fixes have you tried? +- If < 3: Return to Phase 1, re-analyze with new information +- **If ≥ 3: STOP and question the architecture (step 5 below)** +- DON'T attempt Fix #4 without architectural discussion + +### 5. If 3+ Fixes Failed: Question Architecture + +**Pattern indicating an architectural problem:** +- Each fix reveals new shared state/coupling in a different place +- Fixes require "massive refactoring" to implement +- Each fix creates new symptoms elsewhere + +**STOP and question fundamentals:** +- Is this pattern fundamentally sound? +- Are you "sticking with it through sheer inertia"? +- Should you refactor the architecture vs. continue fixing symptoms? + +**Discuss before attempting more fixes.** This is NOT a failed hypothesis — this is a wrong architecture. + +--- + +## Red Flags — STOP and Follow Process + +If you catch yourself thinking: +- "Quick fix for now, investigate later" +- "Just try changing X and see if it works" +- "Add multiple changes, run tests" +- "Skip the test, I'll manually verify" +- "It's probably X, let me fix that" +- "I don't fully understand but this might work" +- "Here are the main problems: [lists fixes without investigation]" +- Proposing solutions before tracing data flow +- **"One more fix attempt" (when already tried 2+)** +- **Each fix reveals a new problem in a different place** + +**ALL of these mean: STOP. Return to Phase 1.** + +**If 3+ fixes failed:** Question the architecture (Phase 4, step 5). + +## Investigation Flow — Keep Forward Momentum + +When the investigation is active and the user says things like "we made progress but we're not done": + +Do NOT break flow by asking clarifying questions. Keep pushing — gather more evidence, try the next diagnostic step, check another angle. A paused investigation that asks "what happened when you tried X?" wastes the user's attention. + +**Signals to keep pushing:** +- "we made progress but we're not done" — continue investigating +- User ignores a clarify question — they want action, not questions +- "ok" or "still broken" — try next step, don't ask for permission + +**What to do instead:** Derive information from logs, databases, or file state — don't ask the user to be your instrumentation layer. Only ask questions when you genuinely cannot proceed without input AND you've exhausted all self-service options. + +## Common Rationalizations + +| Excuse | Reality | +|--------|---------| +| "Issue is simple, don't need process" | Simple issues have root causes too. Process is fast for simple bugs. | +| "Emergency, no time for process" | Systematic debugging is FASTER than guess-and-check thrashing. | +| "Just try this first, then investigate" | First fix sets the pattern. Do it right from the start. | +| "I'll write test after confirming fix works" | Untested fixes don't stick. Test first proves it. | +| "Multiple fixes at once saves time" | Can't isolate what worked. Causes new bugs. | +| "I see the problem, let me fix it" | Seeing symptoms ≠ understanding root cause. | +| "One more fix attempt" (after 2+ failures) | 3+ failures = architectural problem. Question the pattern. | + +## Quick Reference + +| Phase | Key Activities | Success Criteria | +|-------|---------------|------------------| +| **1. Root Cause** | Read errors, reproduce, check changes, gather evidence, trace data flow | Understand WHAT and WHY | +| **2. Pattern** | Find working examples, compare, identify differences | Know what's different | +| **3. Hypothesis** | Form theory, test minimally, one variable at a time | Confirmed or new hypothesis | +| **4. Implementation** | Create regression test, fix root cause, verify | Bug resolved, all tests pass | + +## References + +- [references/dependency-source-example.md](references/dependency-source-example.md) — Worked example of detecting an editable dev fork that caused schema drift in a library dependency. Read when Phase 1 step 5d leads you to a third-party package as the suspected source of a bug. +- [references/macos-sandbox-debug-example.md](references/macos-sandbox-debug-example.md) — Complete walkthrough of debugging Apple Books.app import failures, demonstrating the macOS sandbox techniques from Phase 1 step 6b. Read when debugging a macOS sandboxed application. diff --git a/systematic-debugging/references/dependency-source-example.md b/systematic-debugging/references/dependency-source-example.md new file mode 100644 index 0000000..40d7723 --- /dev/null +++ b/systematic-debugging/references/dependency-source-example.md @@ -0,0 +1,63 @@ +# Worked Example: Detecting an Editable Dev Fork + +Demonstrates the "Check Dependency Source" pattern (Phase 1, step 5d). + +## The Symptom + +A background worker crashed on every invocation: + +``` +Traceback ... + File "core/session.py", line 456, in _create_node + cursor.execute("""INSERT INTO thought_nodes + (id, content, node_type, timestamp, confidence, source_file, ...) +sqlite3.OperationalError: table thought_nodes has no column named confidence +``` + +## The Investigation Path + +1. **Read the traceback** — the error is in `core/session.py` (a third-party library), line 456. It's trying to insert a `confidence` column that doesn't exist in the database. + +2. **Check the schema** — `sqlite3 brain.db ".schema thought_nodes"` confirmed no `confidence` column in the actual table. The schema had `id, content, node_type, timestamp, source_file, decayed, ...` but no `confidence`. + +3. **Check the source** — `python3 -c "import core.session; print(core.session.__file__)"` revealed the library was loading from `/private/tmp/some-fork/core/session.py` — not from site-packages. + +4. **Check dependency source** — `pip show cashew-brain` revealed: + - `Editable project location: /private/tmp/some-fork` + - `Version: 1.0.0` + - This is a `pip install -e` dev copy in a temp directory + +5. **Compare to upstream** — `pip index versions ` showed a newer version on PyPI. The installed dev copy was behind and had un-merged changes. + +## The Fix + +The initial instinct was to patch the migration code in the dev copy — treating it as a bug in the code. The correct answer was: the dev copy shouldn't be there. The fix was: + +```bash +pip uninstall -y +pip install # install from PyPI +python3 -c "import ; print(.__file__)" +# → site-packages// ✓ production path +``` + +After switching to production, the INSERT no longer referenced `confidence` at all — the dev fork's schema drift was gone. + +## Key Lesson + +The `confidence` column had been added to INSERT statements in the dev fork, but the corresponding `ALTER TABLE ADD COLUMN` migration was never written. This is classic schema drift from an unmaintained fork. The symptom looked like a code bug, but the root cause was dependency management — running the wrong version of the library. + +## Verification Commands + +```bash +# Check install source +pip show + +# Check import path +python3 -c "import ; print(.__file__)" + +# Check PyPI for comparison +pip index versions + +# Check DB schema for drift +sqlite3 ".schema " +``` diff --git a/systematic-debugging/references/macos-sandbox-debug-example.md b/systematic-debugging/references/macos-sandbox-debug-example.md new file mode 100644 index 0000000..2611d10 --- /dev/null +++ b/systematic-debugging/references/macos-sandbox-debug-example.md @@ -0,0 +1,125 @@ +# Worked Example: Apple Books.app Import Pipeline (macOS) + +Demonstrates the "macOS App Troubleshooting" and "Research Before Guessing" patterns (Phase 1 steps 6a/6b). + +## The Symptom + +Importing EPUBs into Apple Books fails silently. Three failure modes: +1. **Silent drag-and-drop** — file lands in app, nothing happens +2. **Double-click in Finder** — focus shifts to Books, but import never starts +3. **File → Open dialog** — selecting a file does nothing + +No error dialogs. No crash reports. Books stays open and responsive. + +## Investigation Path + +### Phase 1 — Evidence Gathering + +**1. Check the database (BKLibrary)** + +The main library database is at: +``` +~/Library/Containers/com.apple.iBooksX/Data/Documents/BKLibrary/BKLibrary-1-091020131601.sqlite +``` + +The `ZBKLIBRARYASSET` table tracks all books. Key columns: +- `ZSTATE` — 3=local file exists, 5=cloud-only +- `ZCONTENTTYPE` — 1=ebook, 5=book store, 6=audiobook/other +- `ZPATH` — full path to local file in BKAgentService container +- `ZASSETID` / `ZSTOREID` — Apple Store identifiers + +**2. Check the XPC service (BKAgentService)** + +File storage lives in: +``` +~/Library/Containers/com.apple.BKAgentService/Data/Documents/iBooks/Books/ +``` + +Books are stored as **unzipped directory bundles** (`.epub` extension on a directory, containing `META-INF/`, `OEBPS/`, and `mimetype`). Apple's Books app unzips EPUBs on import and stores the raw directory structure. This means standard EPUB tools (Calibre, etc.) cannot directly read from this storage — the files must be re-zipped with `mimetype` as the first entry. + +**3. Check the import queue** + +``` +~/Library/Containers/com.apple.iBooksX/Data/Library/Caches/Inbox/ +``` + +Files that have been draggged or opened but not yet processed appear here. Files can get stuck indefinitely. + +**4. Read the system logs** + +```bash +log show --predicate 'process == "Books"' --last 30m --style compact +``` + +The critical error: +``` +BKResolveAssetForImportOperation: Unable to access url +BKResolveAssetForImportOperation: User cancelled import of cloud asset. +importBookFromURL: BKResolveAssetForImportOperation failed. +``` + +The "User cancelled" message is misleading — it's the app's internal interpretation of an NSFileCoordinator claim failure (Code=3072 "The operation was cancelled"), likely caused by a sandbox permission issue or XPC service state corruption. + +**5. Check for container migration artifacts** + +A `Data.old/` directory inside the BKAgentService container indicates a failed sandbox container migration during a macOS update: +``` +~/Library/Containers/com.apple.BKAgentService/Data.old/ +``` + +This can contain old book files and plists from a previous container version, creating orphaned state. + +### The Root Cause + +The user had deleted EPUB files from the Books local storage folder to free disk space. This broke the concordance between the CoreData database (which tracks book metadata) and the actual file store. Two layers of corruption resulted: + +1. **Database inconsistency** — entries had `ZSTATE=3` (local file exists) but the file was gone +2. **Container migration ghost** — `Data.old/` from an OS update left orphaned files +3. **XPC service degradation** — BKAgentService would process exactly one import after restart, then silently stop + +### Solutions Tested + +**What worked (partially):** +- Container deletion (`rm -rf ~/Library/Containers/com.apple.iBooksX/` and `BKAgentService/`) — books re-downloaded from iCloud, but import pipeline remained fragile +- Full process kill (Books + BKAgentService + BooksThumbnail) — one successful import, then degradation + +**What didn't work:** +- SQL-level database fixes (CoreData cached state and overwrote changes) +- TCC permission reset (`tccutil reset All com.apple.iBooksX`) — removed file access prompts +- BKAgentService selective kill — XPC respawns with broken state + +**Confirmed workaround:** +- **iPhone iCloud Drive workaround** — upload EPUB to iCloud Drive, open on iPhone/iPad in Files app, share to Books. Syncs to Mac via iCloud, bypassing local import pipeline entirely. +- **iCloud Books data reset** — System Settings → Apple ID → iCloud → Manage Storage → Books → Delete All Data (forces full iCloud sync state reset) + +## Books.app Import Pipeline + +``` +User drags EPUB → Powerbox creates security-scoped bookmark + → Books.app receives URL via AppleEvent + → BKResolveAssetForImportOperation copies file to Caches/Inbox + → BKAgentService XPC picks it up from Inbox + → BKAgentService unzips EPUB into bundle directory under Books/ + → BKAgentService updates Books.plist with metadata + → BKLibrary CoreData store records the asset + → iCloud sync pushes to other devices +``` + +### Where It Breaks + +1. Sandbox security-scoped bookmarks fail → "Unable to access url" (TCC issue) +2. BKAgentService XPC degrades over time → silent import failures +3. Database/file concordance breaks → Books thinks it has files it doesn't +4. iCloud sync state corrupts → phantom entries, cross-device sync fails + +### Diagnostic Quick-Reference + +| Check | Command | +|-------|---------| +| App log | `log show --predicate 'process == "Books"' --last 10m` | +| XPC log | `log show --predicate 'process == "com.apple.BKAgentService"' --last 10m` | +| Database state | `sqlite3 .../BKLibrary-*.sqlite "SELECT ZSTATE,COUNT(*) FROM ZBKLIBRARYASSET GROUP BY ZSTATE;"` | +| Container size | `du -sh ~/Library/Containers/com.apple.iBooksX/` | +| Import queue | `ls ~/Library/Containers/com.apple.iBooksX/Data/Library/Caches/Inbox/` | +| Book files | `ls ~/Library/Containers/com.apple.BKAgentService/Data/Documents/iBooks/Books/` | +| Migration ghosts | `ls -d ~/Library/Containers/com.apple.BKAgentService/Data.old 2>/dev/null` |