magnus919_agent-skills/anydoc/references/workflows.md

Workflows: recipes for converting documents to markdown

All recipes use the pinned CLI npx -y @firecrawl/anydoc@0.1.6 (ground truth) and the skill's wrapper scripts/anydoc where it adds value. Commands are shown relative to the repository root; anydoc/fixtures/... paths can be replaced with any document path. The vault-ingestion recipe (section 5) is written to be run from a temp or vault directory holding your own documents. Each raw-CLI invocation converts exactly one document — there is no batch mode.

1. Single conversion

# Markdown to stdout
npx -y @firecrawl/anydoc@0.1.6 anydoc/fixtures/fixture-handmade-outline.docx

# Markdown to a file (stdout stays silent; existing file is overwritten)
npx -y @firecrawl/anydoc@0.1.6 anydoc/fixtures/fixture-handmade-outline.docx -o outline.md

# Same jobs through the wrapper
python3 anydoc/scripts/anydoc convert anydoc/fixtures/fixture-handmade-outline.docx
python3 anydoc/scripts/anydoc convert anydoc/fixtures/fixture-handmade-outline.docx -o outline.md

Expected result: exit code 0, empty stderr, and GitHub-Flavored Markdown on stdout (or written to the -o output file) containing #/##/### heading lines.

2. Force the input format

# Extensionless or mislabeled file: name the format explicitly
npx -y @firecrawl/anydoc@0.1.6 ./data --format csv
npx -y @firecrawl/anydoc@0.1.6 ./report --format docx

Use --format <name> only when detection cannot work (CSV from stdin, or a missing/wrong extension). Aliases resolve: --format xls, --format docm, --format ppsx are accepted. An invalid name exits 2 with anydoc: invalid format 'bogus'; expected one of: ....

3. Read a document from stdin

# CSV from stdin requires --format csv (no signature, no extension)
printf 'name,role\nAlice,Engineer\n' | npx -y @firecrawl/anydoc@0.1.6 - --format csv

# Any document type can come from stdin; detection reads the bytes
curl -s https://example.com/paper.pdf | npx -y @firecrawl/anydoc@0.1.6 -

The wrapper supports the same: cat data.csv | python3 anydoc/scripts/anydoc convert - -f csv.

Piping notes:

• Markdown goes to stdout only; diagnostics are the single anydoc: <message> stderr line.
• EPIPE is handled: if the downstream pipe closes early (... anydoc@0.1.6 big.xlsx | head -n 1), the CLI exits 0 with no stderr noise — piping into head is safe and is not a failure.

4. Batch conversion (raw CLI)

The raw CLI takes one document per invocation, so batch with a shell loop:

mkdir -p out
for f in anydoc/fixtures/*.docx; do
  npx -y @firecrawl/anydoc@0.1.6 "$f" -o "out/$(basename "${f%.docx}").md"
done

Each failed document (error fixtures, scanned PDFs, encrypted files) exits 1 with its anydoc: <message> on stderr and produces no output file; the loop continues with the next input. Handle or route those per errors.md.

Or the wrapper, which is built for this (per-file status, continues past failures, summary, and a non-zero exit when any input failed):

python3 anydoc/scripts/anydoc batch \
  anydoc/fixtures/fixture-handmade-outline.docx \
  anydoc/fixtures/fixture-sheet.csv \
  --out-dir out/

batch --dry-run --json prints the plan (input → output, dry-run marker) without converting or creating anything:

python3 anydoc/scripts/anydoc batch anydoc/fixtures/fixture-handmade-outline.docx \
  anydoc/fixtures/fixture-sheet.csv --out-dir out/ --dry-run --json

5. Vault-ingestion pattern

Convert a folder of mixed office documents to markdown for ingestion into a vault or knowledge base:

1. Collect the documents into a folder (mixed docx/xlsx/pptx/csv/odt/pdf is fine — text-based PDFs only; see the no-OCR caveat in errors.md).

2. Batch-convert with the wrapper into a markdown folder:

   python3 anydoc/scripts/anydoc batch notes/*.docx notes/*.xlsx notes/*.csv --out-dir vault/inbox/
   

   (or the raw-CLI loop above if you are not using the wrapper).

   Run this from a temp or vault directory — never from the agent-skills repo root. The glob matches whatever directory you name, and the repository tracks a top-level docs/ directory (distinct from the documents/ skill): globbing docs/*.docx there, or deleting/cleaning those matches, would damage tracked repository files. Keep the source documents in their own folder (here notes/) and convert into a separate vault/inbox/ folder.

3. Verify each output (step 6) — at minimum confirm exit 0 and that the structural markers your formats produce are present (headings for Word/PDF, | tables for spreadsheets/CSV).

4. Failures are per-file: the batch summary names what failed; route those files per errors.md (scanned PDF → OCR tooling, encrypted → unencrypted copy, unsupported → check extension) and re-run only the failures.

6. Output verification

Before treating a conversion as done:

1. Exit code 0 — the CLI produced markdown. Exit 1: read the anydoc: <message> stderr line and match it against errors.md. Exit 2: fix the command (usage error).
2. Structural markers — check the markers your format actually produces:
   • Word / ODT / RTF / text-based PDF: #/## heading lines (grep -E '^#{1,6} ' out.md).
   • Spreadsheets (xlsx/xls/ods) and CSV: ## <sheet> headings and |-delimited rows (grep -E '^\|' out.md).
   • Presentations (pptx/odp): slide titles as plain paragraphs, > blockquote speaker notes, | table rows (legacy .ppt has no | rows — that is by design, not an error).
   • EPUB: # chapter headings and [text](#fragment) internal links.
3. Tables survived? If the source had tables and the output has no | rows, check the caveats: PDF and legacy .ppt flatten tables by design.
4. Large outputs: convert with -o out.md and inspect the file rather than streaming everything into context.

Use the committed fixtures to sanity-check an environment once:

npx -y @firecrawl/anydoc@0.1.6 anydoc/fixtures/fixture-handmade-outline.docx   # headings
npx -y @firecrawl/anydoc@0.1.6 anydoc/fixtures/sheet.xlsx                      # ## Values + table
npx -y @firecrawl/anydoc@0.1.6 anydoc/fixtures/fixture-text.pdf                # headings, no table

7. Large files and resource limits

• Conversion is not streaming — the document is read and processed as a whole, and safety limits protect against decompression and nesting bombs.
• Zip/image bombs are rejected via max_entry_bytes with exit 1 and the prefix anydoc: resource limit exceeded (max_entry_bytes): (full examples in errors.md). This is by design — do not try to bypass it.
• -o out.md is recommended for large documents so the output is written to a reviewable file instead of filling stdout/context; you can then read the parts you need.
• Genuinely large real documents (as opposed to bombs) convert normally; the per-document limit only rejects entries whose declared decompressed size exceeds the cap.
• If a resource-limit error fires on a real file, the archive is malformed or hostile — re-export the document rather than disabling the limit.

8. Startup cost and performance

Each npx -y @firecrawl/anydoc@0.1.6 invocation costs roughly 0.33–0.55 s of warm-cache startup (npm/npx process startup) on top of the conversion itself, which is a few milliseconds (measured ~5 ms for a PDF, <1 ms for a DOCX once the process is warm). There is no progress output; conversions are effectively instant. Plan for ~0.5 s per document in batch loops, and prefer a single npx process per document (you cannot batch inside one invocation).

9. Offline / cold-cache behavior

• The first npx run downloads the package plus the native binary (network required once); later runs use the npm cache. A cold-cache offline run fails with a clear npx fetch error before anydoc executes.
• For permanent or fully offline use, install once: npm install -g @firecrawl/anydoc, then call anydoc <file> directly.
• The wrapper always invokes npx with -y (non-interactive), so it never hangs on npx's install prompt — even on a cold cache it fails fast if the package cannot be fetched.