Magnus HedemarkandGitHub
08fafacd6c
Two new reference files extending qa-methodology into the reactive side of its domain — diagnosing failures rather than designing strategy. ci-failure-triage.md: systematic CI failure diagnosis — runner availability checks, log triage (gh run view), exit 137 / container termination evidence-first procedure, pre-existing vs regression classification, flaky test management, and compose readiness corollary. Distilled from accumulated CI-failure incident notes. test-debugging.md: diagnosing broken tests — mock path binding after module-to-package refactors, FastAPI startup race (mock state set before TestClient context is overwritten), httpx mock transport pattern, test execution integrity (collection count vs exit code), deterministic integration seeds, API signature change fixture recovery, and uv lockfile hygiene. Distilled from accumulated test-debugging incident notes. Both are technique libraries serving qa-methodology's existing domain, not new standalone skills. SKILL.md reference table updated. Signed-off-by: Magnus Hedemark <magnus919@pm.me>
3.9 KiB
name, description, license, metadata
| name | description | license | metadata | ||||
|---|---|---|---|---|---|---|---|
| qa-methodology | Quality assurance methodology — test strategy design, test automation patterns, regression testing, CI quality gates, test data management, and quality metrics. Grounded in practical patterns for teams that want confident shipping. | MIT |
|
QA Methodology
Quality assurance is the practice of making confident shipping routine. This methodology covers test strategy, automation, regression management, and quality metrics that scale with a project's complexity.
The QA Engineer's Domain
| You own | You don't own |
|---|---|
| Test strategy — what to test, at what level, with what priority | Code review — that's the reviewer |
| Test automation — framework selection, test harness setup, CI integration | Root cause analysis of bugs — that's the debugger |
| Regression testing — suites that catch regressions without becoming brittle | Feature implementation — that's the developer |
| Quality gates — CI integration, pass/fail criteria, blocking vs non-blocking | Kanban workflow design — that's the kanban strategist |
| Test data management — fixtures, factories, synthetic data | Production monitoring — that's SRE |
| Quality metrics — coverage analysis, defect density, MTD | Verdict on completion — that's the verifier |
Reference Files
| Reference | When to load |
|---|---|
references/test-strategy.md |
Designing a test strategy for a new project or feature — test levels, risk analysis, prioritization, automation targets |
references/test-automation-gates-metrics.md |
Test automation framework selection, CI integration (parallel execution, sharding, flaky management), quality gate design (pass/fail criteria, blocking vs advisory, evolution), and quality metrics (coverage, defect density, MTTD/MTTR) |
references/regression-testing.md |
Building and maintaining regression suites — selection criteria, prioritization, suite evolution, false positive management |
references/test-data-management.md |
Test data strategy — fixtures vs factories, isolation, synthetic data, PII rules, external service mocking, volume testing |
references/performance-testing.md |
Performance testing — load/stress/soak/spike types, k6 patterns, metrics interpretation, CI integration |
references/security-testing.md |
Security testing — SAST/DAST/dependency audit, OWASP Top 10 test patterns, container scanning, CI gates |
references/ci-failure-triage.md |
CI is red — systematic diagnosis: runner availability, log triage, exit code taxonomy (137/OOM), flake vs real failure, pre-existing vs regression classification |
references/test-debugging.md |
A test that should pass is failing — mock path binding after package refactors, FastAPI startup races, httpx mock patterns, fixture recovery, execution integrity |
Core Principles
If it isn't tested, it's broken — Untested code is not working code; it's code whose failure mode hasn't been discovered yet.
Quality is a property of the process, not the artifact — Testing at the end doesn't create quality. Quality is designed in through test strategy, automation, and gating throughout the development cycle.
Test behavior, not implementation — Tests coupled to implementation details break on refactoring. Tests coupled to behavior survive it. Prefer testing what the system does, not how it does it.
Fast feedback wins — A test that takes 30 seconds to run gets run more often than a test that takes 30 minutes. Invest in test speed proportional to feedback frequency.
Flaky tests are worse than no tests — A test that fails nondeterministically trains teams to ignore failures. Fix or remove flaky tests on detection.