mirror of
https://github.com/magnus919/agent-skills.git
synced 2026-09-11 19:47:12 +03:00
fix(ci): align skill test discovery with coverage guardrail
Discover skill-local test dirs from git ls-files so nested bundle sub-skill scripts/ dirs are covered, and force python_files=test_*.py so pytest collection matches the guardrail's covered model everywhere (skills with a local pytest.ini would otherwise fall back to the default collection). Also soften the docs' guardrail claims to describe the enforced naming convention precisely instead of overclaiming. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
This commit is contained in:
co-authored by
factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
parent
98eb5e4c4d
commit
4255d4429f
@@ -73,12 +73,10 @@ jobs:
|
||||
- name: Run skill-local script tests
|
||||
run: |
|
||||
set -euo pipefail
|
||||
for dir in */scripts; do
|
||||
if find "$dir" -name 'test_*.py' | grep -q .; then
|
||||
while IFS= read -r dir; do
|
||||
echo "=== $dir ==="
|
||||
python3 -m pytest "$dir" -o "addopts=-ra --strict-markers --tb=short" -v --durations=10
|
||||
fi
|
||||
done
|
||||
python3 -m pytest "$dir" -o "addopts=-ra --strict-markers --tb=short" -o "python_files=test_*.py" -v --durations=10
|
||||
done < <(git ls-files | grep -E '/scripts/test_[^/]*\.py$' | sed 's#/[^/]*$##' | sort -u)
|
||||
- name: Run skill-local shell test scripts
|
||||
run: python3 scripts/check-skill-tests.py --run
|
||||
- name: Check skill test coverage
|
||||
|
||||
@@ -166,7 +166,7 @@ Skills that perform diagnosis, planning, or multi-step work must state when they
|
||||
|
||||
### Skill Script Tests
|
||||
|
||||
Skills that ship executable scripts must ship pytest-runnable tests in `scripts/test_*.py`; CI auto-discovers them, so Python test files must use the exact `test_*.py` name. Shell-based tests are the exception: register them in `scripts/check-skill-tests.py` as a `run` or `manual` entry. CI enforces this via `python3 scripts/check-skill-tests.py --check`, which fails on any unregistered skill test file, stale registry entry, or registry inconsistency.
|
||||
Skills that ship executable scripts must name their script tests `scripts/test_*.py`; CI auto-discovers and runs them, so Python test files must use the exact `test_*.py` name. Shell-based tests are the exception: register them in `scripts/check-skill-tests.py` as a `run` or `manual` entry. CI enforces the naming convention via `python3 scripts/check-skill-tests.py --check`: a test-like file under a skill's `scripts/` directory that is neither a Python `test_*.py` nor registered fails the check.
|
||||
|
||||
## Validate Your Output
|
||||
|
||||
|
||||
+1
-1
@@ -79,7 +79,7 @@ If a skill includes executable scripts or a package, run its documented checks a
|
||||
|
||||
### Skill script tests
|
||||
|
||||
Every skill that ships executable scripts must ship pytest-runnable tests named `scripts/test_*.py`; pytest auto-discovers these in CI for every skill's `scripts/` directory, so Python test files must use the exact `test_*.py` name. Shell-based tests are the exception: register them in `scripts/check-skill-tests.py` as a `run` entry (executed in CI with `bash`) or a `manual` entry (documented only, when the test needs network access, credentials, or external tooling). The guardrail `python3 scripts/check-skill-tests.py --check` fails CI on any unregistered skill test file, on stale registry entries, and on registry inconsistencies, so a new, un-run shell test will not appear silently.
|
||||
Every skill that ships executable scripts must name its script tests `scripts/test_*.py`; pytest auto-discovers and runs them in CI for every skill's `scripts/` directory, so Python test files must use the exact `test_*.py` name. Shell-based tests are the exception: register them in `scripts/check-skill-tests.py` as a `run` entry (executed in CI with `bash`) or a `manual` entry (documented only, when the test needs network access, credentials, or external tooling). CI enforces the naming convention: a file under any skill `scripts/` directory whose name matches the test conventions (`test_*.py`, `test*.sh`, `*_test.*`, `*-test.*`, or `.bats`) must be a Python `test_*.py` (auto-run) or be registered in `scripts/check-skill-tests.py`; `python3 scripts/check-skill-tests.py --check` fails otherwise.
|
||||
|
||||
## Deprecating a skill
|
||||
|
||||
|
||||
Reference in New Issue
Block a user