skills/magnus919_agent-skills
1
0
Fork 0
mirror of https://github.com/magnus919/agent-skills.git synced 2026-09-11 19:47:12 +03:00
Code Issues Packages Projects Releases Wiki Activity

feat: add FFmpeg expert skill

Browse Source 
Adds a source-grounded FFmpeg command-line skill with focused references, a capability preflight, and six output-quality eval cases. Closes #426.
This commit is contained in:
Magnus Hedemark
2026-09-01 08:33:26 -04:00
committed by GitHub
parent 516c233d33
commit 02d6189135
17 changed files with 818 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.claude-plugin/marketplace.json
+9
View File
@@ -401,6 +401,15 @@
"strict": false,
"description": "Build, configure, flash, test, debug, and recover firmware for ESP32-family boards, including ESP-IDF C/C++, Arduino/PlatformIO, MicroPython, CircuitPython, ESPHome, Zephyr, Rust, and NuttX. Use when identifying an ESP32 board, choosing a framework, wiring GPIO or peripheral buses, integrating sensors or actuators, diagnosing serial/boot/power/network failures, or planning OTA and production security. Do not use as a substitute for the exact board schematic, SoC datasheet, or attached component datasheet."
},
{
"name": "ffmpeg",
"source": "./",
"skills": [
"./ffmpeg"
],
"strict": false,
"description": "Use this skill when an agent needs to inspect, convert, remux, transcode, filter, combine, stream, or troubleshoot audio and video with the FFmpeg command-line tools, especially ffmpeg and ffprobe. It teaches explicit stream selection, filtergraph construction, timestamp diagnosis, build-aware commands, safe scripting, and post-run verification. Do not use it for libav API programming, professional color-management certification, DRM circumvention, or untested platform-specific capture hardware; route those to specialized guidance."
},
{
"name": "financial-modeling",
"source": "./",
.codex-plugin/plugin.json
+1
View File
@@ -62,6 +62,7 @@
"./enterprise-architecture",
"./epub",
"./esp32-development",
"./ffmpeg",
"./financial-modeling",
"./fireflies",
"./flaresolverr",
README.md
+4
View File
@@ -185,6 +185,10 @@ fixed-layout, accessibility, and media overlays. Portable across any AgentSkills
Build, configure, flash, test, debug, and recover complete ESP32 systems across ESP-IDF C/C++, Arduino/PlatformIO, MicroPython, CircuitPython, ESPHome, Zephyr, Rust, and NuttX. Covers board and pin identification, electrical safety, sensors, actuators, buses, networking, sleep, OTA, production security, native command-line tools, and evidence-first troubleshooting. Ships focused references, safe bring-up templates, a non-mutating host preflight, and safety evals.
### [ffmpeg](ffmpeg/SKILL.md)
Expert FFmpeg command-line guidance for inspecting, remuxing, transcoding, filtering, combining, streaming, and troubleshooting media with explicit mappings, build-aware assumptions, and post-run verification.
### [financial-modeling](financial-modeling/SKILL.md)
Build and review assumptions-led financial models, unit economics, pricing, fundraising scenarios, and SaaS operating metrics.
ffmpeg/README.md
+55
View File
@@ -0,0 +1,55 @@
# FFmpeg Expert Skill
A practical FFmpeg command-line skill for agents that need to understand media files, choose safe transformations, and verify the result instead of blindly copying recipes.
## Why Install This Skill
FFmpeg is powerful but its failures often happen at boundaries: a command selects the wrong stream, an option applies to the wrong input, a filter is missing from the installed build, or timestamps make a seemingly correct cut unusable. This skill gives an agent a repeatable way to inspect the media first and explain what the command will actually do.
After installation, an agent can reason about remuxing versus transcoding, construct explicit filtergraphs, diagnose timing and concatenation problems, write safer batch operations, and validate outputs against the intended player, editor, receiver, or archive. The guidance is grounded in official FFmpeg manuals, with version and build caveats called out clearly.
## What You Get
| Path | Purpose |
|---|---|
| `SKILL.md` | Trigger boundaries and the core inspect-decide-run-verify workflow |
| `references/core-model-and-command-anatomy.md` | Containers, streams, codecs, mapping, option scope, and timestamps |
| `references/filters-and-transformations.md` | Simple and complex filtergraphs, audio/video filters, and graph debugging |
| `references/intermediate-workflows.md` | Trimming, concat, metadata, subtitles, scripting, pipes, and streaming |
| `references/advanced-operations-and-safety.md` | Hardware, synchronization, reproducibility, and operational safety |
| `references/command-cookbook.md` | Short, assumption-labeled commands |
| `references/source-inventory.md` | Primary and secondary sources with evidence boundaries |
| `references/local-verification.md` | Recorded local-build experiments and their limits |
| `references/learning-summary.md` | Learning progression and consolidated mental model |
| `evals/evals.json` | Portable output-quality cases for the skill |
## Quick Start
Install FFmpeg with your platform's package manager, then verify both tools:
```sh
ffmpeg -version
ffprobe -version
ffprobe -v error -show_format -show_streams -of json input.mp4
```
Ask your agent to inspect the input before selecting a command. During exploration, write to a new output path and use `-n` to refuse accidental overwrites.
## Triggers
Load this skill when the task involves:
- Inspecting or explaining a media file's streams, codecs, container, metadata, or timestamps
- Converting, remuxing, transcoding, filtering, trimming, joining, extracting, or subtitle handling
- Building FFmpeg batch scripts, pipe workflows, or network streaming commands
- Checking filter, encoder, protocol, or hardware-acceleration availability
- Diagnosing synchronization, concat, mapping, muxing, or playback failures
Do not load it as the primary skill for libav API development, DRM circumvention, professional color management, or a named platform's account/API operations.
## Requirements
- `ffmpeg` and `ffprobe` on `PATH` for execution
- A shell for the examples, with careful quoting for filenames and filter expressions
- Network access only when consulting linked online documentation or exercising a network protocol
- Hardware acceleration requires the relevant device, drivers, compiled FFmpeg support, and a tested end-to-end path
ffmpeg/SKILL.md
+56
View File
@@ -0,0 +1,56 @@
---
name: ffmpeg
description: >-
Use this skill when an agent needs to inspect, convert, remux, transcode, filter,
combine, stream, or troubleshoot audio and video with the FFmpeg command-line
tools, especially ffmpeg and ffprobe. It teaches explicit stream selection,
filtergraph construction, timestamp diagnosis, build-aware commands, safe
scripting, and post-run verification. Do not use it for libav API programming,
professional color-management certification, DRM circumvention, or untested
platform-specific capture hardware; route those to specialized guidance.
license: MIT
compatibility: Requires ffmpeg and ffprobe for execution; exact filters, codecs, protocols, and hardware backends vary by build and version.
---
# FFmpeg Expert
Treat FFmpeg commands as typed media pipelines, not incantations. Start from what the input actually contains, choose the smallest operation that satisfies the output contract, and verify the resulting artifact at the boundary that matters.
## When Not to Use
- Do not use this skill for libav*/FFmpeg C API application development.
- Do not use it as a complete codec encyclopedia or a professional color-management certification guide.
- Do not use it to circumvent DRM or to document capture hardware that has not been tested on the target platform.
- For a named hosting or media platform's API, use that platform skill and use this skill only for the local media transformation.
## Operating Loop
1. **Inspect first.** Run `ffprobe -v error -show_format -show_streams -of json INPUT` and identify streams, codecs, dimensions, rates, durations, time bases, metadata, and start timestamps.
2. **Classify the operation.** Choose remux/stream copy, transcode, filter, combine, extract, or protocol/pipe output. Remuxing changes packaging; transcoding decodes and re-encodes.
3. **Check capabilities.** Use `ffmpeg -formats`, `-codecs`, `-encoders`, `-filters`, and `-hwaccels`. Never assume a tutorial's filter, encoder, or hardware backend exists locally.
4. **Make selection explicit.** Use `-map` for multiple inputs, tracks, or complex graphs. Remember that options generally apply to the next input or output, so order matters.
5. **Protect the source.** Use `-n` while exploring, write to a new path, avoid untrusted shell concatenation, and keep credentials out of command lines and logs.
6. **Probe and exercise the result.** Check the output with `ffprobe`, then test the actual player, editor, receiver, archive rule, or API consumer. Exit code and container validity are necessary but not sufficient.
## Choose the Right Reference
- Read `references/core-model-and-command-anatomy.md` for containers, streams, codecs, option scope, mapping, copy/transcode, and timestamps.
- Read `references/filters-and-transformations.md` for simple and complex filtergraphs, labels, audio/video processing, and incremental graph debugging.
- Read `references/intermediate-workflows.md` for seeking, trimming, concatenation, metadata, subtitles, batch scripts, pipes, and streaming.
- Read `references/advanced-operations-and-safety.md` for hardware acceleration, synchronization diagnosis, reproducibility, network and overwrite safety, and failure boundaries.
- Read `references/command-cookbook.md` for short examples with stated assumptions. Adapt them only after inspection and capability checks.
- Read `references/learning-summary.md` for the newcomer-first progression and consolidated mental model.
- Read `references/source-inventory.md` when assessing evidence, choosing authoritative documentation, or refreshing version-sensitive guidance.
- Read `references/local-verification.md` when interpreting the recorded local FFmpeg 8.1.2 evidence. It is a host-specific observation, not a universal capability claim.
- Run `scripts/ffmpeg-preflight --json` before automating a version-sensitive workflow. It reports whether `ffmpeg` and `ffprobe` are available and captures the first diagnostic line for local filters, encoders, and hardware inventory.
## Debugging Rules
- For missing filters, encoders, or protocols, reproduce with `ffmpeg -filters`, `-encoders`, or the relevant inventory before changing the command.
- For drift, bad cuts, concat jumps, or unexpected duration, compare timestamps and stream properties before adding flags. `-copyts`, `-start_at_zero`, synchronization controls, `setpts`, `asetpts`, `aresample`, and `avoid_negative_ts` solve different problems.
- Build filtergraphs incrementally: baseline transcode, one filter, then labels/branches. Distinguish parser, availability, format negotiation, timestamp, encoder, and muxer failures.
- Treat examples as conditional on input, target, build, version, and downstream consumer. State those conditions in explanations and scripts.
## Completion
Stop when the requested artifact exists, the relevant output probe and downstream-boundary check pass, and any untested capability or compatibility gap is stated explicitly. If execution is blocked, report the exact layer and evidence rather than substituting a plausible result.
ffmpeg/evals/evals.json
+84
View File
@@ -0,0 +1,84 @@
{
"schema_version": 1,
"skill_name": "ffmpeg",
"evals": [
{
"id": "inspect-before-remux",
"prompt": "I have an MKV with several tracks and need an MP4 without re-encoding. Give me a safe approach.",
"expected_output": "Explain that this is a remux/stream-copy operation, probe the input first, make stream selection explicit, use no-overwrite behavior during exploration, and probe the output because container compatibility is not guaranteed.",
"assertions": [
"distinguishes remuxing from transcoding",
"uses ffprobe before the operation",
"includes explicit mapping or explains why mapping is needed",
"mentions container and stream compatibility",
"includes post-run output verification"
],
"case_set": "dev"
},
{
"id": "build-aware-filter",
"prompt": "This command fails with No such filter: drawtext. Replace it with something that will work everywhere.",
"expected_output": "Do not claim a universal replacement. Explain that filter availability is build-dependent, show how to inspect the installed filter inventory/help, and offer a conditional path: use an available equivalent, install a build containing the filter, or omit the transformation.",
"assertions": [
"does not fabricate universal filter availability",
"checks ffmpeg filter capabilities",
"treats the error as a build boundary",
"offers conditional next steps rather than silently substituting",
"preserves the need to rerun and verify the output"
],
"case_set": "regression"
},
{
"id": "explicit-complex-mapping",
"prompt": "Overlay a logo image on the first video stream while keeping the first input's audio. Show the important FFmpeg structure.",
"expected_output": "Use a complex filtergraph with two labeled video inputs, label the overlay output, map that label exactly once, map the selected audio explicitly, and explain that filtering requires decoding and video re-encoding while audio may be copied.",
"assertions": [
"uses filter_complex with two inputs",
"labels and maps the filtered video output",
"maps the intended audio stream explicitly",
"explains filter and stream-copy boundaries",
"does not imply automatic mapping is safe here"
],
"case_set": "dev"
},
{
"id": "timestamp-concat-diagnosis",
"prompt": "Two clips have matching extensions but concatenation produces a jump and audio drift. What should I inspect and which concat mechanism should I choose?",
"expected_output": "Treat the issue as a stream-compatibility and timestamp problem. Compare codecs, dimensions, rates, channel layouts, start times, durations, time bases, and packet/frame behavior; distinguish concat demuxer, concat filter, and concat protocol; normalize timestamps and formats when using the filter.",
"assertions": [
"does not use file extension as the compatibility test",
"lists relevant audio/video stream properties",
"distinguishes concat demuxer from concat filter and protocol",
"identifies timestamps as a diagnosis surface",
"recommends probing inputs and output"
],
"case_set": "regression"
},
{
"id": "safe-batch-network",
"prompt": "Write a shell-loop recipe that converts every file in a directory and sends results to a network endpoint.",
"expected_output": "Before giving a recipe, set safe boundaries: use arrays or null-delimited discovery, avoid shell concatenation of untrusted filenames, refuse overwrites by default, record command/version/exit status, probe each output, and treat network protocol, timeout, authentication, and destination validation as explicit requirements.",
"assertions": [
"addresses shell quoting and untrusted filenames",
"uses a deliberate overwrite policy",
"requires per-output verification and logging",
"calls out network protocol and timeout assumptions",
"does not expose credentials in command arguments"
],
"case_set": "release"
},
{
"id": "hardware-pipeline-caveat",
"prompt": "NVENC is enabled. Does that prove hardware acceleration will make this workflow faster on my Mac?",
"expected_output": "No. Explain that decoding, filtering, memory transfers, and encoding are separate pipeline decisions, that NVIDIA-specific NVENC guidance does not transfer to macOS, and that the exact local build/device/pixel-format path must be inspected and benchmarked end to end.",
"assertions": [
"rejects encoder-only speed conclusions",
"separates decode/filter/encode and memory-transfer decisions",
"recognizes platform-specific hardware guidance",
"requires local capability inspection",
"requires reproducible end-to-end benchmarking"
],
"case_set": "release"
}
]
}
ffmpeg/references/advanced-operations-and-safety.md
+62
View File
@@ -0,0 +1,62 @@
# Advanced Operations and Safety
## Hardware acceleration is a pipeline decision
Hardware acceleration may affect decoding, filtering, and encoding separately. A hardware encoder alone does not guarantee faster end-to-end processing. Transfers between system memory and device memory can erase gains, and hardware encoders can have different quality, feature, and rate-control behavior from software encoders.
First inspect the local build:
```sh
ffmpeg -hwaccels
ffmpeg -encoders
ffmpeg -filters | grep -E 'cuda|vaapi|qsv|videotoolbox|vulkan'
```
The local macOS build lists `videotoolbox`. NVIDIA CUDA examples from the vendor guide do not apply to this host. Verify the exact device, pixel formats, filter path, and encoder before benchmarking.
## Timestamp and synchronization diagnosis
When audio drifts, video freezes, concat jumps, or duration is wrong, collect:
```sh
ffprobe -v error -show_streams -show_format -of json input
ffmpeg -loglevel verbose -i input -f null -
```
Compare stream start times, durations, time bases, frame rates, sample rates, packet ordering, and whether a muxer is buffering sparse streams. Avoid cargo-culting timestamp flags. Options such as `-start_at_zero`, `-copyts`, `-vsync`/the modern synchronization controls, `setpts`, `asetpts`, `aresample`, and `avoid_negative_ts` solve different problems and can interact.
## Reproducible experiments
Pin the binary version, record `ffmpeg -version` and `-buildconf`, preserve the exact input or synthetic generator, and probe both sides. Run more than once when measuring speed. Separate wall-clock throughput from output quality and compatibility. If a tutorial omits hardware, build, driver, codec settings, or measurement method, treat its performance claim as incomplete.
## Security and operational boundaries
Do not feed untrusted media to an experimental decoder or enable permissive protocol behavior without understanding the exposure. Avoid secrets in command-line arguments when process listings or logs can expose them. Restrict network protocols and destinations. Refuse overwrites by default during development, write to a new path, and keep the original until the output is independently verified.
A syntax check, successful process exit, valid container, or local playback test proves only that layer. Acceptance should match the real boundary: target player, editor, streaming receiver, archival standard, or API consumer.
## Advanced learning resources
Use the official filter, codec, format, protocol, utility, scaler, and resampler manuals as the reference corpus. Use `slhck/ffmpeg-encoding-course` for a structured intermediate bridge, `amiaopensource/ffmprovisr` for preservation-oriented practice, and the NVIDIA guide for a vendor-specific hardware path. The official Trac wiki is useful but was inaccessible during this research pass, so its examples remain leads rather than verified evidence.
SOURCES (LAYER 3 NAVIGATION)
https://ffmpeg.org/ffmpeg-codecs.html
-> Codec options, rate control, time bases, error detection, and encoder-specific behavior.
https://ffmpeg.org/ffmpeg-protocols.html
-> Protocol options, network I/O, timeouts, and whitelists.
https://ffmpeg.org/ffmpeg-formats.html
-> Probing, interleaving, timestamp shifting, and muxer/demuxer behavior.
https://docs.nvidia.com/video-technologies/video-codec-sdk/13.0/ffmpeg-with-nvidia-gpu/index.html
-> NVIDIA-specific CUDA/NVENC pipeline examples and performance cautions.
https://github.com/slhck/ffmpeg-encoding-course
-> Maintained secondary course for encoding concepts and practical progression.
https://github.com/amiaopensource/ffmprovisr
-> Secondary preservation/media workflow recipe collection.
https://ffmpeg.org/download.html
-> Official release and source-build guidance.
ffmpeg/references/command-cookbook.md
+90
View File
@@ -0,0 +1,90 @@
# Command Cookbook
These are learning commands, not universal production defaults. Check the installed build, input streams, target requirements, and output with `ffprobe`.
## Inspect
```sh
ffprobe -v error -show_format -show_streams -of json input.mp4
```
## List local capabilities
```sh
ffmpeg -hide_banner -formats
ffmpeg -hide_banner -codecs
ffmpeg -hide_banner -filters
ffmpeg -hide_banner -encoders
ffmpeg -hide_banner -hwaccels
```
## Remux without re-encoding
```sh
ffmpeg -n -i input.mkv -map 0 -c copy output.mp4
```
## Explicit transcode
```sh
ffmpeg -n -i input.mov -map 0:v:0 -map 0:a:0 \
-c:v libx264 -crf 20 -preset medium \
-c:a aac -b:a 160k output.mp4
```
## Scale and resample
```sh
ffmpeg -n -i input.mp4 -vf 'scale=1280:-2' -ar 48000 output.mp4
```
## Overlay a second input
```sh
ffmpeg -n -i video.mp4 -i logo.png \
-filter_complex '[0:v][1:v]overlay=20:20[v]' \
-map '[v]' -map 0:a:0 -c:v libx264 -c:a copy output.mp4
```
## Fast exploratory cut
```sh
ffmpeg -n -ss 00:01:00 -i input.mp4 -t 00:00:20 -c copy cut.mp4
```
## Extract audio
```sh
ffmpeg -n -i input.mp4 -map 0:a:0 -vn -c:a flac output.flac
```
## Generate a synthetic test asset
```sh
ffmpeg -f lavfi -i 'testsrc2=size=320x180:rate=30' \
-f lavfi -i 'sine=frequency=440:sample_rate=48000' \
-t 2 -c:v libx264 -pix_fmt yuv420p -c:a aac test.mp4
```
## Diagnostic null output
```sh
ffmpeg -hide_banner -loglevel verbose -i input.mp4 -f null -
```
## Build-aware filter check
```sh
ffmpeg -filters | grep -E 'scale|fps|drawtext|subtitles'
ffmpeg -h filter=scale
```
SOURCES (LAYER 3 NAVIGATION)
https://ffmpeg.org/ffmpeg.html
-> Primary command syntax, mapping, copy/transcode, seeking, and filtering options.
https://ffmpeg.org/ffprobe.html
-> Primary inspection and machine-readable output.
03-dossiers/local-verification.md
-> Commands actually exercised on the local FFmpeg 8.1.2 build.
ffmpeg/references/core-model-and-command-anatomy.md
+76
View File
@@ -0,0 +1,76 @@
# Core Model and Command Anatomy
## The pipeline
The `ffmpeg` command line accepts global options, one or more input blocks, and one or more output blocks:
```sh
ffmpeg [global_options] {[input_options] -i input_url} ... {[output_options] output_url} ...
```
Options generally apply to the next input or output, so order matters. Input and output indexes are zero-based. Stream specifiers such as `:v`, `:a`, `:s`, and `:1` narrow an option to a type or stream index.
The conceptual pipeline is:
```text
input URL -> protocol/IO -> demuxer -> streams -> decode -> filters -> encode -> muxer -> output URL
```
Stream copy skips decode, filtering, and encode for the copied stream. It is therefore fast and lossless with respect to the encoded stream, but cannot perform transformations on that stream.
## Selection and mapping
Automatic stream selection is convenient for simple files but unsafe for multi-input work. `-map 0:v:0 -map 1:a:0` selects the first video from input 0 and first audio from input 1. `-map 0` asks for all streams from input 0, subject to output-format limits. A complex filtergraph's labeled outputs must be mapped exactly once.
Use per-stream options deliberately:
```sh
ffmpeg -i input.mkv -map 0:v:0 -map 0:a:0 -c:v libx264 -crf 20 -c:a aac -b:a 160k output.mp4
```
This is an illustrative transcode, not a universal quality or bitrate recommendation. The correct settings depend on source, target, motion, delivery constraints, and playback support.
## Container and codec are different decisions
A container packages streams. A codec encodes one stream. Renaming a file does not convert it. A remux can change packaging without re-encoding:
```sh
ffmpeg -i input.mkv -map 0 -c copy output.mp4
```
The command can fail or produce an unsuitable file when the selected streams, metadata, or timing do not fit the target container. Probe both input and output.
## Quality and generation loss
Re-encoding is required for filtering, changing many codec properties, or adapting an incompatible stream. Each lossy generation may discard information. Prefer stream copy when the operation is only a compatible container change, but do not force copy when the target needs a different codec, pixel format, sample format, or timing structure.
## Time and timestamps
Video and audio use timestamps expressed in stream-specific time bases. FFmpeg documentation defines a time base as the fundamental time unit for frame timestamps. Fixed-frame-rate video commonly uses a time base related to the frame rate, while an MP4 stream may use a finer muxer time base. Do not compare raw PTS integers from different streams without rescaling them into a common time unit.
Seeking, trimming, synchronization, and concat problems are timestamp problems until proven otherwise. Record the relevant `start_time`, `duration`, `time_base`, frame rate, and packet/frame behavior with `ffprobe`.
## Probe before and after
```sh
ffprobe -v error -show_format -show_streams -of json input.mkv
ffprobe -v error -show_entries stream=index,codec_type,codec_name,width,height,sample_rate,channels,time_base,duration -of json output.mp4
```
Machine-readable output is preferable in scripts. Treat human-readable stderr as diagnostic evidence, not as a stable parsing interface.
SOURCES (LAYER 3 NAVIGATION)
https://ffmpeg.org/ffmpeg.html
-> Command syntax, option scope, stream selection, stream copy, transcoding, and mapping.
https://ffmpeg.org/ffprobe.html
-> Inspection, stream specifiers, machine-readable writers, and intervals.
https://ffmpeg.org/ffmpeg-codecs.html
-> Codec options, rate control, time bases, and error detection.
https://ffmpeg.org/ffmpeg-formats.html
-> Demuxers, muxers, probing, interleaving, and timestamp-related format behavior.
https://ffmpeg.org/ffmpeg-utils.html
-> Duration syntax, rational numbers, expressions, and quoting/escaping.
ffmpeg/references/filters-and-transformations.md
+69
View File
@@ -0,0 +1,69 @@
# Filters and Media Transformations
## Simple versus complex graphs
A simple video or audio filter can be attached with `-vf` or `-af`:
```sh
ffmpeg -i input.mp4 -vf 'scale=1280:-2,fps=30' -af 'loudnorm' output.mp4
```
The filter string is a graph, even when it is a linear chain. Filters consume frames or audio samples and produce new ones, so filtering implies decoding and re-encoding for that stream.
Use `-filter_complex` when multiple inputs, branches, overlays, joins, or separately labeled outputs are involved:
```sh
ffmpeg -i video.mp4 -i logo.png \
-filter_complex '[0:v][1:v]overlay=20:20[vout]' \
-map '[vout]' -map 0:a:0 -c:v libx264 -c:a copy output.mp4
```
Labels are graph edges. Every labeled output that should reach an output file must be mapped. Unmapped filtered output is not a harmless detail: it changes what reaches the muxer or causes an error.
## Common transformation classes
- `scale` changes dimensions and may require an explicit pixel format or aspect-ratio policy.
- `fps` changes frame cadence and can affect duration, motion, and synchronization.
- `crop`, `pad`, `transpose`, and `setsar` change geometry or display interpretation.
- `trim` and `atrim` select time ranges but generally need timestamp normalization such as `setpts` or `asetpts` before concatenation.
- `volume`, `loudnorm`, `aresample`, and `aformat` alter audio level, loudness, sample rate, or format.
- `subtitles` and `drawtext` render text into pixels when available, unlike a copied subtitle stream.
- `overlay`, `hstack`, `vstack`, `concat`, `amix`, and `amerge` combine streams and therefore require compatible timing and formats.
These names are not guarantees. Confirm local availability:
```sh
ffmpeg -filters
ffmpeg -h filter=scale
ffmpeg -h filter=drawtext
```
## Audio and video are separate graphs
A video filter cannot repair audio synchronization by itself. Treat audio and video as separate streams with separate clocks and sample/frame formats. When combining or transcoding them, inspect sample rate, channel layout, start timestamps, duration, and encoder delay.
The resampler can convert sample rates, channel layouts, and sample formats. The scaler converts image dimensions and pixel formats. Both have quality and range choices that should be explicit when the result matters.
## Local failure as evidence
The local FFmpeg 8.1.2 build did not include `drawtext`. A command that combined `scale`, `fps`, and `drawtext` stopped with `No such filter: 'drawtext'` before creating output. This demonstrates why a tutorial's command must be treated as a recipe conditioned on a build, not as a universal API.
## Debugging filtergraphs
Build graphs incrementally. First transcode without filters, then add one filter, then add labels and branches. Use short synthetic inputs, verbose logging, and explicit `-map`. If a graph fails, distinguish parser errors, missing filters, format negotiation errors, timestamp errors, and encoder/muxer errors.
SOURCES (LAYER 3 NAVIGATION)
https://ffmpeg.org/ffmpeg-filters.html
-> Filtergraph syntax, pads, labels, filter families, framesync, and filter options.
https://ffmpeg.org/ffmpeg-scaler.html
-> Image scaling and pixel-format conversion.
https://ffmpeg.org/ffmpeg-resampler.html
-> Audio resampling, rematrixing, formats, dithering, and compensation.
https://trac.ffmpeg.org/wiki/FilteringGuide
-> Official wiki tutorial lead for filtergraphs; access was blocked by Anubis during this research and commands require verification.
03-dossiers/local-verification.md
-> Local filter inventory and verified missing-filter failure.
ffmpeg/references/intermediate-workflows.md
+72
View File
@@ -0,0 +1,72 @@
# Intermediate Workflows
## Inspection and selective conversion
Start with `ffprobe`, then make selection explicit. For scripts, use JSON and fail if the expected stream is absent. For a simple conversion:
```sh
ffmpeg -i input.mov -map 0:v:0 -map 0:a:0 -c:v libx264 -c:a aac output.mp4
```
For a compatible remux:
```sh
ffmpeg -i input.mkv -map 0 -c copy output.mp4
```
Do not call remuxing a conversion of the encoded media. It changes packaging only.
## Trim and seek
`-ss` can be placed before input for fast input seeking or after input for output-side behavior with different accuracy and cost. `-t` limits duration; `-to` specifies an endpoint in the relevant command context. Test the actual cut, especially with inter-frame codecs, nonzero start timestamps, and audio.
```sh
ffmpeg -ss 00:01:00 -i input.mp4 -t 00:00:20 -c copy quick-cut.mp4
```
Stream-copy cuts may begin on keyframe boundaries and can preserve awkward timestamps. Re-encode when frame-accurate filtering or predictable normalization is more important than speed.
## Concatenation
There are distinct mechanisms:
- The concat demuxer reads a script describing files and is appropriate when streams are compatible and the files meet its safety and timestamp assumptions.
- The concat filter operates on decoded audio/video and can join segments after normalizing dimensions, formats, and timestamps.
- The concat protocol is physical byte/resource concatenation and is not a general-purpose media join.
Never choose a concat method solely because files share an extension. Inspect codecs, dimensions, frame rates, sample rates, channel layouts, time bases, and metadata.
## Metadata and subtitles
Metadata can be copied, mapped, or rewritten. FFmpeg's format documentation describes the `ffmetadata` muxer/demuxer for round-tripping metadata. Subtitle streams can be copied when the target container supports them, or rendered into video with a subtitle filter when permanent pixels are intended. Those are different deliverables.
## Batch scripting
Use shell quoting carefully and never construct commands by concatenating untrusted filenames into `sh -c`. Prefer arrays in Bash/Zsh, null-delimited file discovery, and explicit output paths. Probe each result and preserve stderr logs. Use `-n` to refuse overwriting during exploratory runs; use `-y` only when an overwrite policy is intentional.
A robust batch worker records input, exact command, FFmpeg version, exit status, output path, and a post-run probe. A zero exit status is not a complete acceptance gate if the downstream consumer has stricter requirements.
## Pipes and streaming
An output URL can be a file, pipe, or network protocol. `-f` can force the muxer when the output URL does not provide a useful extension. `-re` is relevant when reading a file at approximately its native rate for streaming demonstrations, not as a universal speed setting. Network operations need bounded timeouts, known protocols, authentication handling, and a safe destination.
The protocols manual documents protocol-specific options, including `rw_timeout`, protocol whitelists, UDP, and concat. Validate the receiving side independently.
SOURCES (LAYER 3 NAVIGATION)
https://ffmpeg.org/ffmpeg.html
-> Seeking, stream copy, mapping, input/output option scope, and transcoding.
https://ffmpeg.org/ffmpeg-formats.html
-> Demuxers, muxers, concat-related format behavior, metadata, probing, and interleaving.
https://ffmpeg.org/ffmpeg-protocols.html
-> File, pipe, concat, UDP, HTTP, and network protocol options.
https://ffmpeg.org/ffmpeg-utils.html
-> Time expressions and quoting/escaping needed for scripts.
https://shotstack.io/learn/how-to-use-ffmpeg/
-> Secondary practical examples; verify all commands against current official manuals.
https://en.wikibooks.org/wiki/FFMPEG_An_Intermediate_Guide
-> Secondary intermediate topic map; examples are version/build-sensitive.
ffmpeg/references/learning-summary.md
+47
View File
@@ -0,0 +1,47 @@
# Learning FFmpeg Without Memorizing Recipes
FFmpeg is best understood as a graph of media transformations. An input URL is demuxed into streams, streams may be selected and either copied or decoded, decoded frames can pass through audio/video filters, encoders turn processed frames back into packets, and a muxer writes those packets to an output URL. `ffprobe` is the inspection tool that tells you what the container and streams actually contain.
## The practical mental model
- A **container** such as MP4, Matroska, MPEG-TS, or WAV packages one or more streams and their metadata.
- A **codec** describes the encoded elementary stream, such as H.264, AV1, AAC, Opus, or PCM.
- A **stream** is one typed track, usually video, audio, subtitles, data, or attachments.
- **Remuxing** changes the container while copying encoded streams. It is fast and avoids generation loss, but only works when the target container accepts those streams.
- **Transcoding** decodes and re-encodes. It enables filtering and format changes, but costs time and can reduce quality.
- **Mapping** makes stream selection explicit. Use it whenever multiple inputs, multiple tracks, or complex filters make automatic selection ambiguous.
- A **filtergraph** is a directed graph of named inputs, filters, and outputs. A labeled output from `-filter_complex` must be mapped explicitly.
## A reliable operating loop
1. Inspect: `ffprobe -v error -show_format -show_streams -of json input.mkv`.
2. Decide: remux, stream-copy, transcode, filter, or combine inputs.
3. Verify capabilities: `ffmpeg -formats`, `-codecs`, `-encoders`, `-filters`, and `-hwaccels`.
4. Build a minimal command with explicit stream selectors and output options.
5. Run without overwriting first, capture stderr, and probe the output.
6. Test playback and the intended downstream consumer. A successful exit code and valid container are not universal compatibility proof.
## Learning path
Start with inspection and the command line, then understand stream selection and copy/transcode. Add simple filters before learning labeled filtergraphs. Next learn seeking, timestamps, concat, metadata, subtitles, pipes, and shell loops. Only then move to streaming, hardware acceleration, and complex debugging.
## What local testing changed
The local macOS Homebrew FFmpeg 8.1.2 build successfully generated and probed an H.264/AAC MP4. A subsequent scale/fps/text-filter experiment failed because this build did not contain `drawtext`. That failure is part of the lesson: online recipes are not portable promises. Check the installed build and verify the final artifact.
## Implications
FFmpeg becomes predictable when commands are treated as typed pipelines rather than incantations. Most difficult failures occur at boundaries: stream selection, option scope, timestamps, filter availability, codec/container constraints, shell escaping, or hardware/software memory transfer. Make those boundaries explicit and debugging becomes a sequence of observable checks.
SOURCES (LAYER 2 NAVIGATION)
02-analysis/core-model-and-command-anatomy.md
-> Detailed model of containers, streams, codecs, option scope, mapping, and copy/transcode.
02-analysis/filters-and-transformations.md
-> Filtergraph construction and audio/video transformation boundaries.
02-analysis/intermediate-workflows.md
-> Inspection, joining, metadata, scripting, and streaming workflows.
02-analysis/advanced-operations-and-safety.md
-> Hardware acceleration, timestamps, reproducibility, and diagnosis.
ffmpeg/references/local-verification.md
+38
View File
@@ -0,0 +1,38 @@
# Local Verification Dossier
**Access/test date:** 2026-09-01
**Host:** macOS
**Binary:** Homebrew-installed `ffmpeg`
**Version:** FFmpeg 8.1.2, libavutil 60.26.102, libavcodec 62.28.102
**Build evidence:** `ffmpeg -version` reports `--enable-videotoolbox`, `--enable-audiotoolbox`, libx264, libx265, libsvtav1, libvmaf, libopus, libmp3lame, libdav1d, and libvpx.
## Inventory
The local build reported 488 filters, 201 encoders, and 2 hardware acceleration methods in the captured inventories. The exact lists are preserved in `local-filters.txt`, `local-encoders.txt`, and `local-hwaccels.txt`. Availability is build-specific: never assume a filter, encoder, protocol, or hardware backend exists just because an online example uses it.
## Successful experiment
A synthetic 320x180, 30 fps test video and 48 kHz mono sine-wave audio were generated with lavfi and encoded to MP4 using libx264 and native AAC. `ffprobe` verified a 2.00-second MP4 containing H.264 video and AAC audio. The source log is `../03-dossiers/local-probe.json` and the command output is recorded in the run log outside this dossier.
The intended follow-up transcode used `-ss 0.5 -t 0.75`, scaling to 160 pixels wide, reducing to 15 fps, and adding `drawtext`. It failed before writing output because this local build reported `No such filter: 'drawtext'`. This is useful evidence: filter names and compiled capabilities must be checked with `ffmpeg -filters` or `ffmpeg -h filter=<name>` before placing them in automation. The failed and successful run logs were preserved in the local study record; their host-specific paths are intentionally omitted here.
## Verification lesson
A command that looks portable can still fail at the filter-availability boundary. The correct response is not to silently substitute a different filter or claim success. Inspect the local build, choose an available equivalent, or install/use a build with the required feature, then rerun and probe the resulting media.
## Reproduction commands
```sh
ffmpeg -version
ffmpeg -filters
ffmpeg -encoders
ffmpeg -hwaccels
ffprobe -v error -show_format -show_streams -of json input.mp4
```
SOURCES (LAYER 3 NAVIGATION)
https://ffmpeg.org/ffprobe.html
-> Official description of machine-readable media inspection.
https://ffmpeg.org/ffmpeg.html
-> Official command-line processing model and option semantics.
ffmpeg/references/source-inventory.md
+63
View File
@@ -0,0 +1,63 @@
# FFmpeg Source Inventory
**Research access date:** 2026-09-01
## Retained primary sources
| # | Source | Authority | Topics supported | Caveat |
|---|---|---|---|---|
| 1 | [FFmpeg documentation](https://ffmpeg.org/documentation.html) | Official project index | Current command-line and API documentation; versioned manuals | Online docs track the newest revision; use installed manuals for older builds. |
| 2 | [ffmpeg tool manual](https://ffmpeg.org/ffmpeg.html) | Official | Command anatomy, input/output order, stream selection, mapping, codecs, filtering, timestamps | Option scope is order-sensitive and many examples are contextual. |
| 3 | [ffprobe manual](https://ffmpeg.org/ffprobe.html) | Official | Container/stream inspection, machine-readable output, stream specifiers, intervals | Probe output describes the file; it does not prove universal playback compatibility. |
| 4 | [ffmpeg-all manual](https://ffmpeg.org/ffmpeg-all.html) | Official | Consolidated reference and examples across tools and libraries | Very large; use for lookup after learning the model. |
| 5 | [Filters manual](https://ffmpeg.org/ffmpeg-filters.html) | Official | Filtergraphs, pads, labels, audio/video filters, framesync, hardware filters | Availability and options depend on build and version. |
| 6 | [Formats manual](https://ffmpeg.org/ffmpeg-formats.html) | Official | Demuxers, muxers, probing, interleaving, timestamp handling, concat | Container behavior varies; codec and container compatibility are separate. |
| 7 | [Codecs manual](https://ffmpeg.org/ffmpeg-codecs.html) | Official | Encoder/decoder options, bitrate, time base, rate control, error detection | Private options are encoder-specific. |
| 8 | [Protocols manual](https://ffmpeg.org/ffmpeg-protocols.html) | Official | File, pipe, network, concat, UDP, HTTP and other I/O protocols | Network commands need explicit timeout, security, and endpoint assumptions. |
| 9 | [Utilities manual](https://ffmpeg.org/ffmpeg-utils.html) | Official | Duration syntax, quoting/escaping, expressions, rational numbers, channel layouts | Shell quoting is a second language layered over FFmpeg quoting. |
| 10 | [Scaler manual](https://ffmpeg.org/ffmpeg-scaler.html) | Official | Scaling, pixel-format conversion, algorithms, range and gamma | Quality depends on source/destination formats and range. |
| 11 | [Resampler manual](https://ffmpeg.org/ffmpeg-resampler.html) | Official | Sample-rate conversion, channel rematrixing, sample formats, dithering, sync compensation | Optional SoX support and defaults are build-dependent. |
| 12 | [NVIDIA FFmpeg GPU guide](https://docs.nvidia.com/video-technologies/video-codec-sdk/13.0/ffmpeg-with-nvidia-gpu/index.html) | Vendor primary source | CUDA decode, GPU surfaces, scale_cuda/scale_npp, NVENC | NVIDIA-specific; do not transfer commands to macOS or non-NVIDIA hosts. |
| 13 | [FFmpeg Git repository](https://git.ffmpeg.org/ffmpeg.git) | Official source | Source-level confirmation and release history | Source inspection does not replace testing the installed binary. |
| 14 | [FFmpeg download/release guidance](https://ffmpeg.org/download.html) | Official | Releases, source builds, release cadence and signed tags | Distribution packages may lag or differ in enabled components. |
## Secondary learning resources
- [FFmpeg Filtering Guide](https://trac.ffmpeg.org/wiki/FilteringGuide) - official project wiki guide. Automated retrieval was blocked by the site's Anubis proof-of-work page on this date, so treat it as a reading lead and verify commands against the filters manual.
- [FFmpeg Ultimate Guide](https://img.ly/blog/ultimate-guide-to-ffmpeg/) - readable secondary overview with practical transcoding examples; vendor context means commands should be checked against official manuals.
- [Shotstack FFmpeg guide](https://shotstack.io/learn/how-to-use-ffmpeg/) - broad examples and scripting orientation; secondary and service-oriented.
- [FFmpeg intermediate guide on Wikibooks](https://en.wikibooks.org/wiki/FFMPEG_An_Intermediate_Guide) - useful topic map; community-maintained and examples need current-build verification.
- [FFmpeg short guide and examples](https://github.com/term7/FFmpeg-A-short-Guide) - practical GitHub notes; inspect freshness before relying on a recipe.
- [slhck FFmpeg encoding course](https://github.com/slhck/ffmpeg-encoding-course) - maintained educational repository discovered during delegated research; strong intermediate bridge, but still secondary to the official manuals.
- [amiaopensource/ffmprovisr](https://github.com/amiaopensource/ffmprovisr) - preservation-oriented recipe collection; useful for real media workflows, with strong need for format-specific verification.
## Rejected or limited sources
- FFmpeg Trac pages for Concatenate, H.264, AAC, HWAccelIntro, StreamingGuide, Encode/YouTube, and CompilationGuide were not retained as evidence because retrieval encountered the site's JavaScript proof-of-work page. This is an access limitation, not evidence that the pages are wrong.
- DeepWiki pages were not used as primary evidence because they are generated/secondary explanations of source code.
- Search-result snippets and anonymous cheat sheets were used only for discovery, not evidence.
## Recommended learning order
1. `ffmpeg` synopsis and `ffprobe` inspection.
2. Containers, streams, codecs, demux/mux, decode/filter/encode, and stream mapping.
3. Remuxing versus transcoding, codec selection, quality controls, and stream specifiers.
4. Simple filters, then labeled `-filter_complex` graphs and explicit `-map`.
5. Time, seeking, trimming, concat demuxer versus concat filter, timestamps, and synchronization.
6. Metadata, subtitles, image sequences, pipes, and shell scripting.
7. Streaming protocols and latency controls, with explicit endpoint and timeout handling.
8. Hardware acceleration only after understanding software pipelines and verifying local capabilities.
9. Debugging with verbose logs, `ffprobe`, minimal reproductions, and build inventories.
SOURCES (LAYER 3 NAVIGATION)
https://ffmpeg.org/documentation.html
-> Official documentation index and version links.
https://ffmpeg.org/ffmpeg.html
-> Primary command-line semantics.
https://ffmpeg.org/ffmpeg-filters.html
-> Primary filtergraph reference.
https://ffmpeg.org/ffprobe.html
-> Primary inspection reference.
ffmpeg/scripts/ffmpeg-preflight
Executable
+55
View File
@@ -0,0 +1,55 @@
#!/usr/bin/env python3
"""Report local FFmpeg/ffprobe capability facts as JSON or readable text."""
from __future__ import annotations
import argparse
import json
import shutil
import subprocess
from typing import Any
def run(binary: str, *args: str) -> dict[str, Any]:
path = shutil.which(binary)
if not path:
return {"available": False, "error": f"{binary} was not found on PATH"}
result = subprocess.run(
[path, *args], capture_output=True, text=True, check=False
)
output = result.stdout + result.stderr
first_line = next((line for line in output.splitlines() if line.strip()), "")
return {
"available": result.returncode == 0,
"path": path,
"returncode": result.returncode,
"first_line": first_line,
}
def main() -> int:
parser = argparse.ArgumentParser(
description="Check local FFmpeg tools before using version-sensitive recipes."
)
parser.add_argument("--json", action="store_true", help="emit machine-readable JSON")
args = parser.parse_args()
report = {
"ffmpeg": run("ffmpeg", "-version"),
"ffprobe": run("ffprobe", "-version"),
"filters": run("ffmpeg", "-filters"),
"encoders": run("ffmpeg", "-encoders"),
"hardware_acceleration": run("ffmpeg", "-hwaccels"),
}
if args.json:
print(json.dumps(report, indent=2, sort_keys=True))
else:
for name, result in report.items():
state = "available" if result["available"] else "unavailable"
detail = result.get("first_line") or result.get("error", "")
print(f"{name}: {state} - {detail}")
return 0 if report["ffmpeg"]["available"] and report["ffprobe"]["available"] else 1
if __name__ == "__main__":
raise SystemExit(main())
ffmpeg/scripts/test_ffmpeg_preflight.py
+36
View File
@@ -0,0 +1,36 @@
#!/usr/bin/env python3
"""Smoke-test the FFmpeg preflight script without media or network access."""
import json
import os
import subprocess
import sys
import tempfile
from pathlib import Path
def test_preflight_with_fake_tools():
script = Path(__file__).resolve().parent / "ffmpeg-preflight"
with tempfile.TemporaryDirectory() as directory:
fake_bin = Path(directory)
for name in ("ffmpeg", "ffprobe"):
tool = fake_bin / name
tool.write_text("#!/bin/sh\nprintf \"%s\\n\" \"$0 $*\"\n")
tool.chmod(0o755)
env = os.environ.copy()
env["PATH"] = str(fake_bin)
result = subprocess.run(
[sys.executable, str(script), "--json"],
capture_output=True,
text=True,
env=env,
)
assert result.returncode == 0, result.stderr
report = json.loads(result.stdout)
assert report["ffmpeg"]["available"] is True
assert report["ffprobe"]["available"] is True
assert report["filters"]["available"] is True
assert report["encoders"]["available"] is True
assert report["hardware_acceleration"]["available"] is True
print("ffmpeg preflight smoke test passed")
llms.txt
+1
View File
@@ -46,6 +46,7 @@
- [enterprise-architecture](enterprise-architecture/SKILL.md): Design and evolve enterprise architectures by connecting business capabilities, value streams, applications, information, technology, operating models, and transition choices. Use when mapping an enterprise portfolio, comparing current and target states, sequencing transition architectures, or defining federated architecture decision rights and stakeholder communication. Do not use for system or solution design, API or data-platform design, organizational or talent design, product roadmaps, technology adoption, or corporate strategy; route those to the named specialist skills.
- [epub](epub/SKILL.md): Read, write, and edit EPUB2/EPUB3 ebooks as an EPUB file format expert. Extract text, metadata, structure, and knowledge from EPUB files for enrichment or memory. Create valid EPUBs from scratch. Validate against the EPUB specification. Use when the user mentions epub, ebook, EPUB file, ebook format, read epub, write epub, create ebook, extract from epub, epub to text, or ebook structure.
- [esp32-development](esp32-development/SKILL.md): Build, configure, flash, test, debug, and recover firmware for ESP32-family boards, including ESP-IDF C/C++, Arduino/PlatformIO, MicroPython, CircuitPython, ESPHome, Zephyr, Rust, and NuttX. Use when identifying an ESP32 board, choosing a framework, wiring GPIO or peripheral buses, integrating sensors or actuators, diagnosing serial/boot/power/network failures, or planning OTA and production security. Do not use as a substitute for the exact board schematic, SoC datasheet, or attached component datasheet.
- [ffmpeg](ffmpeg/SKILL.md): Use this skill when an agent needs to inspect, convert, remux, transcode, filter, combine, stream, or troubleshoot audio and video with the FFmpeg command-line tools, especially ffmpeg and ffprobe. It teaches explicit stream selection, filtergraph construction, timestamp diagnosis, build-aware commands, safe scripting, and post-run verification. Do not use it for libav API programming, professional color-management certification, DRM circumvention, or untested platform-specific capture hardware; route those to specialized guidance.
- [financial-modeling](financial-modeling/SKILL.md): Build and review assumptions-led financial models, unit economics, pricing, fundraising scenarios, and SaaS operating metrics. Use when calculating CAC, LTV, payback, runway, ARR, churn, NDR, Rule of 40, or sales efficiency; when modeling revenue, costs, cash flow, pricing, cap tables, or financing.
- [fireflies](fireflies/SKILL.md): Query Fireflies.ai meeting transcripts, meeting notes, summaries, contacts, channels, AI meeting analytics, AskFred, audio uploads, and webhook signatures through its GraphQL API. Use when a user mentions Fireflies, Fireflies.ai, meeting transcripts or notes stored in Fireflies, AskFred, or Fireflies webhooks. Do not use for local audio transcription, calendar management, or meetings that are not Fireflies data.
- [flaresolverr](flaresolverr/SKILL.md): Use the minimal FlareSolverr wrapper for a one-off health check or browser-backed GET/POST when ordinary retrieval is blocked by Cloudflare or DDoS-GUARD. Choose flaresolverr-cli instead for named session lifecycle, cookie-only returns, dry-run planning, or the full operational command surface.