skills/magnus919_agent-skills
1
0
Fork 0
mirror of https://github.com/magnus919/agent-skills.git synced 2026-09-11 19:47:12 +03:00
Code Issues Packages Projects Releases Wiki Activity

feat(skill): add PostgreSQL operational skill (#245) (#265)

Browse Source 
Add a one-tool PostgreSQL operations skill: configuration review, index and
query-plan analysis, vacuum/bloat, WAL archiving + point-in-time recovery,
replication/failover, extensions, upgrades, and evidence-based diagnostics.
Ships the read-only pgdiag collector (stdlib, --json, --plan-for, --help
without a cluster), 9 dated references, tests, a human README, 6 eval cases,
and the top-level index + regenerated catalogs. Routes app data access to
backend-engineering and schema design to data-architect/data-engineering.

Closes #245

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
This commit is contained in:
Magnus Hedemark
2026-08-03 18:22:28 -04:00
committed by GitHub
co-authored by factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
parent ac1beb117d
commit cd14da26cc
18 changed files with 1622 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.claude-plugin/marketplace.json
+9
View File
@@ -680,6 +680,15 @@
"strict": false,
"description": "Operate Playwright for browser automation end to end: author and debug E2E test suites (robust locators, network interception and mocking, parallel workers, accessibility snapshot checks), wire them into CI, and drive headless browsing and scraping with an extract -> validate -> save loop. Use when writing, running, fixing, or scraping with Playwright, when a Playwright CI failure or JSON report needs triage, or when the bundled pwrun script should analyze a run. Do not use for QA strategy or test framework selection (route to qa-methodology), for frontend component or architecture design (route to frontend-engineering), or for Cloudflare/DDoS-GUARD challenge bypass (use flaresolverr)."
},
{
"name": "postgres",
"source": "./",
"skills": [
"./postgres"
],
"strict": false,
"description": "Operate PostgreSQL instances safely: configuration review, index and query-plan analysis, vacuum and bloat management, WAL archiving and point-in-time recovery, replication and failover, extensions, major-version upgrades, and evidence-based diagnostics with the bundled read-only pgdiag script. Use when running or inspecting a PostgreSQL server, diagnosing performance or backup health, or planning an upgrade or failover. Do not use for application-level data access patterns (that's backend-engineering) or schema design (that's data-architect/data-engineering)."
},
{
"name": "privacy-engineering",
"source": "./",
.codex-plugin/plugin.json
+1
View File
@@ -98,6 +98,7 @@
"./peertube",
"./platform-engineering",
"./playwright",
"./postgres",
"./privacy-engineering",
"./product-adoption",
"./product-analytics-and-measurement",
README.md
+4
View File
@@ -317,6 +317,10 @@ Infrastructure as code, CI/CD, container orchestration, service networking — m
Operate Playwright for browser automation: E2E test authoring with robust locators, network interception and mocking, parallel workers, CI integration, headless scraping, and accessibility snapshot checks. Ships a `pwrun` smoke harness (`--json`), a test-suite scaffold, 8 references, and 6 evals.
### [postgres](postgres/SKILL.md)
Operate PostgreSQL safely: configuration review, index and query-plan analysis, vacuum and bloat management, WAL archiving and point-in-time recovery, replication and failover, extensions, major-version upgrades, and evidence-based diagnostics. Ships the read-only `pgdiag` collector (`--json`, `--plan-for`, `--help` without a cluster), 9 dated references, tests, and 6 evals. Routes app data access to backend-engineering and schema design to data-architect/data-engineering.
### [privacy-engineering](privacy-engineering/SKILL.md)
Translate privacy principles and legal requirements into verifiable engineering artifacts — data-lifecycle records, privacy acceptance criteria, data-flow and access maps, retention/deletion verification plans, and privacy change reviews. Covers seven privacy dimensions (purpose, lifecycle/retention, access, deletion, tenant/isolation, residency, consent) as structured concerns, with specific guidance for agent traces and product analytics telemetry. Routes legal interpretation to legal-strategy and security implementation to secure-software-engineering. Ships 1 discovery brief, 5 templates, and 6 evals.
llms.txt
+1
View File
@@ -77,6 +77,7 @@
- [peertube](peertube/SKILL.md): Browse PeerTube federated video from the terminal: view videos and channels, search across instances, check server stats, and manage your account. Uses OAuth2 authentication with token persistence. Use when the user mentions PeerTube, federated video, decentralized video platforms, or browsing/uploading to a PeerTube instance.
- [platform-engineering](platform-engineering/SKILL.md): Use this skill when building or operating internal developer platforms: infrastructure as code, CI/CD, container orchestration, service networking, secrets, and observability. Do not use it to define release process, promotion, rollout, or rollback policy; use release-engineering for that delivery model.
- [playwright](playwright/SKILL.md): Operate Playwright for browser automation end to end: author and debug E2E test suites (robust locators, network interception and mocking, parallel workers, accessibility snapshot checks), wire them into CI, and drive headless browsing and scraping with an extract -> validate -> save loop. Use when writing, running, fixing, or scraping with Playwright, when a Playwright CI failure or JSON report needs triage, or when the bundled pwrun script should analyze a run. Do not use for QA strategy or test framework selection (route to qa-methodology), for frontend component or architecture design (route to frontend-engineering), or for Cloudflare/DDoS-GUARD challenge bypass (use flaresolverr).
- [postgres](postgres/SKILL.md): Operate PostgreSQL instances safely: configuration review, index and query-plan analysis, vacuum and bloat management, WAL archiving and point-in-time recovery, replication and failover, extensions, major-version upgrades, and evidence-based diagnostics with the bundled read-only pgdiag script. Use when running or inspecting a PostgreSQL server, diagnosing performance or backup health, or planning an upgrade or failover. Do not use for application-level data access patterns (that's backend-engineering) or schema design (that's data-architect/data-engineering).
- [privacy-engineering](privacy-engineering/SKILL.md): Translate privacy principles and legal requirements into data-flow, lifecycle, acceptance, and verification artifacts. Map data classification, purpose, processing, access, retention, deletion, residency, and consent; define verifiable privacy acceptance criteria; and produce data-lifecycle records, retention/deletion verification plans, and privacy change reviews. Use when engineering privacy into a system, feature, or data flow — not for legal advice, jurisdiction-specific regulatory interpretation, or replacing security engineering or incident response.
- [product-adoption](product-adoption/SKILL.md): Diagnose and design product adoption for onboarding, activation, behavior change, feature discovery, rollout, and sustained use after launch. Covers consumer, B2B, internal-tool, and public-service contexts with evidence-based decision points and cohort segmentation. Does not own acquisition campaigns or marketing conversion (routes to go-to-market), product-analytics instrumentation (routes to product-analytics-and-measurement), or customer-success account management and health scoring (routes to conditional-customer-success).
- [product-analytics-and-measurement](product-analytics-and-measurement/SKILL.md): Define observable, governed evidence for product outcomes through metric trees, event/tracking plans, instrumentation QA, and measurement governance. Use when defining product metrics, designing tracking plans, building event taxonomies, running product funnels or cohort analysis, setting up dashboard contracts, or auditing instrumentation quality. Do NOT use for statistical inference or experiment design (route to data-scientist), data pipeline implementation (route to data-engineering), data architecture decisions (route to data-architect), observability or infrastructure monitoring (route to site-reliability-engineering), or general business intelligence and dashboard building (route to BI tooling).
postgres/README.md
+52
View File
@@ -0,0 +1,52 @@
# PostgreSQL — Operational Skill for PostgreSQL
Operate PostgreSQL safely: configuration review, index and query-plan analysis, vacuum and bloat management, WAL archiving and point-in-time recovery, replication and failover, extensions, major-version upgrades, and diagnostics with evidence.
## Why Install This Skill
Your agent can run PostgreSQL operations instead of guessing: review configuration against the workload, find the index and query-plan evidence behind a slow query, verify that autovacuum is keeping up, check that WAL archiving is actually working (not just configured), measure replication lag, plan a failover or a major-version upgrade, and diagnose incidents in a fixed evidence order.
It ships a read-only diagnostic script (`pgdiag`) that collects the operator-critical evidence in one bounded JSON payload — server version, configuration values, connection pressure, index usage, bloat signals, archiver health, replication state, extensions, and database sizes. Every session opens read-only (`default_transaction_read_only=on`), so the tool cannot mutate anything even by mistake, and `--help` works with no cluster and no psql installed.
The references are distilled from the official PostgreSQL documentation with dated sources and verification-first guidance. Schema design, application-level data access, and cross-engine methodology deliberately route to the skills that own them (`data-architect`, `backend-engineering`, `data-engineering`); this skill owns the day-to-day operation of PostgreSQL itself.
## What You Get
| Directory | Purpose |
|---|---|
| `SKILL.md` | Agent-facing operating loop, mutation gates, and verification boundaries |
| `references/` | Nine dated references: configuration, indexes/plans, vacuum/bloat, backups/WAL/PITR, replication/failover, extensions, upgrades, diagnostics, source index |
| `scripts/pgdiag` | Read-only diagnostic collector: stdlib-only Python, `--json`, `--check` subsets, `--plan-for` (EXPLAIN JSON), `--help` without a cluster |
| `tests/` | Deterministic tests against a fake psql stub, including the read-only contract |
| `evals/evals.json` | Six output-quality evaluation cases for agent runs |
## Quick Start
```bash
# Help works with no cluster and no psql installed
scripts/pgdiag --help
# Full read-only diagnostics, machine-readable
scripts/pgdiag --json
# Against a specific instance
scripts/pgdiag --host db1.example.com --dbname app --user ops --json
# Targeted checks
scripts/pgdiag --check identity --check wal_archive --check replication --json
# Add an EXPLAIN plan for one read-only statement
scripts/pgdiag --plan-for "SELECT * FROM orders WHERE id = 42" --json
```
The script shells out to `psql` (override the binary with `--psql /path/to/psql`; connection defaults come from the usual `PGHOST`/`PGPORT`/`PGUSER`/`PGDATABASE` environment variables or the flags above). Exit codes: 0 ok, 1 runtime/collection error, 2 usage error, 127 psql not found, 124 timeout.
## Triggers
Load this skill for `postgres`/`PostgreSQL`/`psql` operations: configuration review and `postgresql.conf` tuning, slow queries and `EXPLAIN` plan analysis, index usage measurement, vacuum and bloat, WAL archiving and point-in-time recovery, backups and restore drills, replication and standby lag, failover planning, extension installs and upgrades, minor or major version upgrades (`pg_upgrade`), or any PostgreSQL incident that needs evidence-first diagnosis. Do not load it for application data-access code (that's `backend-engineering`), schema design (that's `data-architect`/`data-engineering`), Supabase platform administration (that's `supabase`), or other database engines (those stay in `data-engineering`).
## Requirements
- Python 3.9+ for the `pgdiag` script (`--help` needs nothing else).
- The `psql` client (PostgreSQL 10+ server) for live diagnostics; it must be on `PATH` or passed with `--psql`.
- Socket or network access to the target instance and, for read-only diagnostics, a role that can read the system catalogs and statistics views.
postgres/SKILL.md
+168
View File
@@ -0,0 +1,168 @@
---
name: postgres
description: >-
Operate PostgreSQL instances safely: configuration review, index and
query-plan analysis, vacuum and bloat management, WAL archiving and
point-in-time recovery, replication and failover, extensions, major-version
upgrades, and evidence-based diagnostics with the bundled read-only pgdiag
script. Use when running or inspecting a PostgreSQL server, diagnosing
performance or backup health, or planning an upgrade or failover. Do not use
for application-level data access patterns (that's backend-engineering) or
schema design (that's data-architect/data-engineering).
license: MIT
compatibility: >-
The bundled pgdiag script runs on Python 3.9+ and needs no PostgreSQL server
for --help. Live diagnostics require the psql client (PostgreSQL 10+ servers)
and socket or network access to the instance.
metadata:
source: https://www.postgresql.org/docs/current/
research_checked: "2026-08-03"
---
# PostgreSQL Operations
Use this skill to operate PostgreSQL safely as the database engine it is: review configuration against workload, find index and query-plan problems, manage vacuum and bloat, set up and verify WAL archiving and point-in-time recovery, run replication with a defensible failover plan, handle extensions, plan major-version upgrades, and diagnose incidents with evidence. This is a **tool skill** for one named tool (PostgreSQL). Database *methodology* — backup strategy across engines, migration patterns, SQL analytical patterns — lives in [data-engineering](../data-engineering/SKILL.md); application-level data access patterns belong to [backend-engineering](../backend-engineering/SKILL.md); schema and data modeling belong to [data-architect](../data-architect/SKILL.md) and `data-engineering`. Supabase platform administration is [supabase](../supabase/SKILL.md).
## Operating contract
1. **Read-only discovery before any mutation.** Inspect configuration, catalog statistics, and logs first. The bundled `pgdiag` script collects read-only evidence and opens every session with `default_transaction_read_only=on`.
2. **Confirm the target, scope, and rollback path before acting.** Read-only discovery may proceed without confirmation. Mutations — a `pg_ctl` stop, a promotion, an extension install, a `pg_upgrade` run — require an explicit human directive naming the instance.
3. **A backup is not recovery evidence.** Verify restore on a scratch instance on a schedule; never claim recoverability from a backup log alone.
4. **Keep evidence bounded.** Summarize catalog queries and log excerpts; never dump full logs, `postgresql.conf`, or connection strings with passwords into chat.
5. **Verify at the delivery boundary.** A `SELECT 1` answer proves connectivity, not health; a replayed `pg_basebackup` proves recoverability, not that today's WAL is being archived.
## The pgdiag script
`scripts/pgdiag` is an agent-first, read-only diagnostic collector. It shells out to `psql`, opens every session with `default_transaction_read_only=on`, and emits bounded JSON. `--help` works with no server and no psql installed.
```bash
scripts/pgdiag --help # no cluster needed
scripts/pgdiag --json # all checks, machine-readable
scripts/pgdiag --host db1 --dbname app --json
scripts/pgdiag --check identity --check wal_archive --json
scripts/pgdiag --plan-for "SELECT * FROM orders WHERE id = 42" --json
```
Exit codes: 0 ok, 1 runtime/collection error, 2 usage error, 127 psql binary not found, 124 timeout. `--check` runs a named subset; `--plan-for` adds an `EXPLAIN (FORMAT JSON)` plan for one read-only statement. The script never issues data-changing statements, and the server-side read-only session setting rejects any that slip through.
## Operating loop
1. **Identify the instance**: version, recovery state, configuration file locations, connection string shape, and whether this is primary or standby.
2. **Collect evidence**: `pgdiag --json` for config, connections, index usage, bloat signals, WAL archiving, recovery, replication, extensions, and database sizes.
3. **Triage against the symptom**: map the reported problem to the evidence (slow queries → plans and index usage; stalled backups → archiver; drift → replication lag).
4. **Act with confirmation**: bounded, scoped mutations after a human directive, with a rollback path named first.
5. **Verify**: re-run the relevant check and confirm the observable at the delivery boundary.
## Configuration
- The runtime source of truth is `pg_settings`, not the file: `SHOW`/`current_setting()` reflect reloads and overrides (ALTER SYSTEM, command-line `-c`, env `PGOPTIONS`). `pgdiag`'s `config` check lists the operator-critical values.
- Know which changes need a reload (`pg_ctl reload` / `SELECT pg_reload_conf()`) versus a restart: memory (shared_buffers, max_connections, wal_level, max_wal_senders) requires restart; most tuning and logging parameters reload.
- Check `log_destination`, `logging_collector`, and `log_min_duration_statement` so slow-query evidence exists before you need it; `track_io_timing=on` makes `pg_stat_database` I/O timing meaningful.
- Connection pressure: compare `pg_stat_activity` state counts against `max_connections`; a connection pooler is an application-architecture decision for [backend-engineering](../backend-engineering/SKILL.md).
- GUC rationale, reload-versus-restart tables, and parameter-change review patterns: `references/01-configuration.md`.
## Indexes and query plans
- Evidence first: `pg_stat_user_indexes` shows `idx_scan`/`idx_tup_read`/`idx_tup_fetch`; a table scanned sequentially with a large `seq_tup_read` while a filter exists is a candidate for a missing index.
- `EXPLAIN (ANALYZE, BUFFERS)` on the real workload query beats guessing; compare estimated to actual rows — a large mismatch points at stale planner statistics (a `pg_statistic` freshness problem) or a bad parameter (random_page_cost, effective_cache_size).
- Unused indexes (`idx_scan = 0` over a long window) cost writes and maintenance; invalid indexes (`pg_index.indisvalid = false`) are dropped on next vacuum and should be repaired or removed deliberately.
- Index choices (BRIN vs btree, partial indexes, covering indexes) are schema design and belong to [data-architect](../data-architect/SKILL.md); this skill owns measuring and operating what exists.
- Query-plan reading, index-usage SQL probes, and plan-review checklists: `references/02-indexes-and-query-plans.md`.
## Vacuum and bloat
- Vacuum reclaims dead tuples and refreshes planner statistics; autovacuum should do this on its own. Verify it is actually running: `autovacuum=on`, worker count, and per-table `relfrozenxid`/`n_dead_tup` trends.
- Bloat is the gap between table file size and live data: `pg_stat_user_tables.n_dead_tup` rising faster than vacuum runs is the leading signal; heap bloat from failed or skipped vacuum shows as large `relpages` with low live tuples.
- If autovacuum lags, the response is a targeted, confirmed maintenance window (`VACUUM` on specific tables, not a firehose), then a check of why autovacuum fell behind (long transactions, connection saturation, worker starvation).
- Never treat `VACUUM FULL` as routine: it rewrites the table, takes locks, and needs a maintenance window plus a verified backup path.
- Bloat measurement probes and autovacuum tuning patterns: `references/03-vacuum-and-bloat.md`.
## Backups: WAL archiving and point-in-time recovery
- The recovery model: a base backup plus a continuous WAL archive gives point-in-time recovery (PITR) — restore the base, replay archived WAL up to the target time.
- Recovery targets come in two families: time-based (the `pitr|point.in.time` pattern is the shorthand for this family — a wall-clock target such as "yesterday 02:00") and position-based (a specific LSN or timeline marker). Both are valid `recovery_target` inputs.
- WAL archiving readiness is `archive_mode=on` with a working `archive_command` and a healthy archiver: `pg_stat_archiver` must show `archived_count` growing, `failed_count` stable, and `last_failed_wal` empty or old.
- `wal_level` must be `replica` (or higher) for both archiving and streaming replication; changing it requires a restart.
- Back up with `pg_basebackup` (or a dedicated tool) consistently with WAL: label each backup, record its `pg_stop_backup()` LSN / timeline, and test restore with the archive before trusting it.
- PITR procedure, `recovery_target` options, restore-to-point-in-time steps, and RPO/RTO framing (methodology in [data-engineering](../data-engineering/SKILL.md)): `references/04-backups-wal-pitr.md`.
## Replication and failover
- Streaming replication: standby connects with a replication slot, receives WAL continuously; verify with `pg_stat_replication` (`state=streaming`, `replay_lsn` keeping up, small `replay_lag`) and the standby's recovery state.
- Decide synchronous vs asynchronous deliberately: synchronous (`synchronous_standby_names`) trades commit latency for a durability guarantee; asynchronous risks losing the last commits on failover.
- A failover plan is more than a `pg_ctl promote`: it names who promotes, how clients are redirected, what happens to the old primary on return, and how to verify data (lag at promotion, timeline divergence).
- Promotion is a mutation — confirm the target and scope first. With a replication-manager tool (Patroni, repmgr), use its switchover command instead of manual promotion; rejoin the old primary as a standby, never let two primaries write.
- Streaming setup, slot management, lag measurement, and failover runbooks: `references/05-replication-and-failover.md`.
## Extensions
- Inventory first: `pg_extension` (installed) and `pg_available_extensions` (available) tell you what exists and what versions are on disk; `pgdiag`'s `extensions` check does this.
- Extension installs change the shared catalog and some extensions change the database in ways that are hard to reverse — an install is a mutation with a rollback path, not a `CREATE EXTENSION` reflex.
- Major-version upgrades usually require re-installing or re-building extensions (e.g., PostGIS, pgvector) on the new binaries; check each extension's upgrade notes before `pg_upgrade`.
- Trusted extensions can be installed by non-superusers into their own databases; extension policy and shared-library availability are infrastructure decisions for [platform-engineering](../platform-engineering/SKILL.md).
- Common extensions, lifecycle, and version-upgrade gotchas: `references/06-extensions.md`.
## Upgrades
- Minor upgrades are in-place binary swaps (restart); major upgrades (e.g., 15 → 16) change on-disk format and need `pg_upgrade` or a logical dump/restore.
- Plan the path first: read the release notes and upgrade guide for the full version span, check extensions and unsupported features, pick the method (`pg_upgrade` with link mode, or logical), and rehearse in a scratch environment with the real data shape.
- `pg_upgrade` is a mutation requiring downtime and a verified backup: stop writes, run the upgrade with the `--old`/`--new` binaries, run `analyze` on the new cluster, and verify at the application boundary before decommissioning the old.
- Logical replication (publisher/subscriber) can serve as a near-zero-downtime major-upgrade path; it is also a migration pattern whose methodology lives in [data-engineering](../data-engineering/SKILL.md).
- Version matrices, upgrade runbooks, and rollback decisions: `references/07-upgrades.md`.
## Diagnostics with evidence
Diagnose in evidence order: identity/version → configuration → connections → index usage and plans → vacuum/bloat → WAL archiving → replication → extensions.
- `pgdiag --json` gathers the first evidence layer in one bounded payload; re-run the affected check after any change.
- Slow query → `EXPLAIN (ANALYZE, BUFFERS)` plus `pg_stat_user_indexes`/`seq_tup_read`; check planner statistics freshness before touching `random_page_cost`.
- Backup stalled → `pg_stat_archiver`: `failed_count`, `last_failed_wal`, and the archive target's disk/network.
- Standby falling behind → `pg_stat_replication` lag columns, slot retention (`pg_replication_slots`), and network saturation between primary and standby.
- Never present correlation as cause: a slow query and a high `n_dead_tup` are evidence, not a diagnosis — state what was measured, what changed, and what was verified.
- Failure-mode routing and symptom→probe→fix tables: `references/08-diagnostics.md`.
## Reference routing
| Load when | Reference |
|---|---|
| Tuning, GUC review, reload vs restart | `references/01-configuration.md` |
| Slow queries, index usage, plan review | `references/02-indexes-and-query-plans.md` |
| Autovacuum, dead tuples, bloat measurement | `references/03-vacuum-and-bloat.md` |
| WAL archiving, base backups, PITR, restore drills | `references/04-backups-wal-pitr.md` |
| Streaming setup, slots, lag, failover runbooks | `references/05-replication-and-failover.md` |
| Extension inventory and upgrade gotchas | `references/06-extensions.md` |
| Minor and major upgrades, pg_upgrade, rollback | `references/07-upgrades.md` |
| Symptom-to-probe diagnosis tables | `references/08-diagnostics.md` |
| Sources, version observations, refresh procedure | `references/00-source-index.md` |
## Included artifacts
- `scripts/pgdiag`: read-only diagnostic collector (stdlib-only, `--json`, `--check`, `--plan-for`, `--help` without a cluster).
- `tests/test_pgdiag.py`: deterministic tests against a fake psql stub, including the read-only contract.
- `references/`: nine dated, source-indexed references covering the operational topics above.
## Verification boundary
| Claim | Minimum evidence |
|---|---|
| Instance is reachable and versioned | `pgdiag --check identity --json` parses and reports version and recovery state |
| Configuration is known | `pgdiag` `config` check lists the operator-critical GUCs |
| Archiving is healthy | `pg_stat_archiver`: `archived_count` increasing, `failed_count` not climbing, `last_failed_wal` stale |
| Replication is current | `pg_stat_replication`: `state=streaming` and lag within the agreed bound |
| Backups support recovery | A restore of a base backup + WAL replayed to a target time on a scratch instance |
| A diagnosis is sound | Evidence was collected before the claim, and the fix was verified by re-running the check |
## Hard boundaries
- Never run a mutation (`pg_ctl stop`, promote, `pg_upgrade`, extension install, maintenance `VACUUM`) without an explicit human directive naming the target and a stated rollback path. Read-only discovery may proceed freely.
- Never present unverified claims as evidence: state what was measured, when, and how.
- Never expose full logs, `postgresql.conf` contents, or connection strings containing passwords.
- Never run `pgdiag` with a write-capable session; the tool itself is read-only by design.
## When not to use
- **Application-level data access patterns** (connection pooling in app code, ORM usage, query construction, transactions in services) — that is [backend-engineering](../backend-engineering/SKILL.md).
- **Schema design and data modeling** (tables, keys, normalization, dimensional models) — that is [data-architect](../data-architect/SKILL.md) and [data-engineering](../data-engineering/SKILL.md).
- **Database methodology across engines** (backup strategy, migration patterns, analytical SQL) — that is `data-engineering`.
- **Supabase platform administration** (managed projects, CLI stack, the self-hosted Supabase stack) — that is [supabase](../supabase/SKILL.md); plain PostgreSQL operations without Supabase conventions belong here.
- **Other database engines** (Redis, MongoDB, Elasticsearch, vector stores) — those stay in `data-engineering` references; this skill owns PostgreSQL only.
postgres/evals/evals.json
+78
View File
@@ -0,0 +1,78 @@
{
"schema_version": 1,
"skill_name": "postgres",
"evals": [
{
"id": "config-review-for-new-instance",
"prompt": "We just provisioned a PostgreSQL 16 instance for a read-heavy reporting workload and left every setting at default. What configuration should we review before it goes live, and which changes require a restart versus a reload?",
"expected_output": "A configuration review that names the operator-critical parameters and classifies each change by when it takes effect: shared_buffers, effective_cache_size, work_mem, and maintenance_work_mem for cache and memory sizing; wal_level (must be replica or higher for archiving and streaming replication), max_wal_senders, and max_replication_slots; archive_mode and archive_command for WAL archiving readiness; autovacuum settings for maintenance; log_min_duration_statement and track_io_timing so slow-query and I/O evidence exist before they are needed. The response states that memory and process-shape parameters (shared_buffers, max_connections, wal_level, max_wal_senders, max_replication_slots, shared_preload_libraries) require a restart while most tuning and logging parameters reload with pg_reload_conf(), and that the runtime source of truth is pg_settings, not the file. It verifies each change with SHOW or current_setting() after applying, and routes connection-pooling and application access patterns to backend-engineering and schema decisions to data-architect.",
"assertions": [
"The review names shared_buffers, effective_cache_size, work_mem, maintenance_work_mem, wal_level, archive_mode, and logging parameters",
"Changes are classified as restart-required (memory and process shape) versus reload-only (pg_reload_conf)",
"pg_settings is identified as the runtime source of truth rather than the configuration file",
"WAL archiving readiness (wal_level replica, archive_mode, archive_command) is included for a production instance",
"Application access patterns route to backend-engineering and schema design routes to data-architect"
]
},
{
"id": "backup-restore-plan-wal-pitr",
"prompt": "Our PostgreSQL 15 instance has archive_mode off and we only take nightly pg_dump snapshots. A user accidentally deleted a row at 14:32 and we need to restore it. What is wrong with the current setup and what should the backup and recovery plan look like?",
"expected_output": "A diagnosis that the current setup cannot recover the specific row: a nightly dump alone recovers only to the last dump, and archive_mode off means there is no WAL archive to replay. The plan prescribes enabling wal_level replica and archive_mode on with a working archive_command, verifying pg_stat_archiver shows archived_count growing and failed_count flat, taking a base backup with pg_basebackup -X stream, and testing a point-in-time recovery to a recovery_target_time on a scratch instance as the only proof the backup supports recovery. The response explains that a base backup plus continuous WAL gives point-in-time recovery (PITR): restore the base, replay archived WAL up to the target time, verify the row exists at the boundary, then promote. It frames RPO/RTO trade-offs and notes that backup strategy methodology routes to data-engineering while the PostgreSQL mechanics live in this skill, and that a restore is a mutation requiring an explicit human directive.",
"assertions": [
"The current setup is diagnosed as unable to recover the deleted row because archive_mode is off and there is no WAL archive",
"The plan enables wal_level replica and archive_mode with a working archive_command",
"Archiver health is verified via pg_stat_archiver with archived_count growing and failed_count flat",
"A base backup with pg_basebackup plus WAL replay to recovery_target_time is described, with a restore drill as the evidence",
"Backup strategy methodology routes to data-engineering and a restore is treated as a mutation requiring confirmation"
]
},
{
"id": "performance-diagnosis-slow-query",
"prompt": "A reporting query over the orders table (about 5 million rows) started taking 40 seconds this week. The table has an index on order_date. How do I diagnose this with evidence before changing anything?",
"expected_output": "An evidence-first diagnosis: run the read-only diagnostic to capture identity, config, connection, and index-usage context, then EXPLAIN (ANALYZE, BUFFERS) the actual query and compare estimated to actual rows. The response checks pg_stat_user_indexes for idx_scan on the order_date index and pg_stat_user_tables for seq_scan and seq_tup_read to see whether the planner chose a sequential scan; checks whether planner statistics are stale by looking at n_dead_tup and whether autovacuum has run recently; and only then considers planner parameters such as random_page_cost or effective_cache_size. It treats a plan change over time as a clue (new data volume, new statistics, changed parameter) and verifies any fix by re-running EXPLAIN and re-checking index scan counts. The response explicitly routes adding a new index for the workload to data-architect and keeps this skill's role as measurement and operation, and notes that correlation between a slow query and high dead tuples is not automatically causation.",
"assertions": [
"Diagnosis starts with read-only evidence collection and EXPLAIN (ANALYZE, BUFFERS) rather than a config guess",
"Index usage and seq scan statistics (pg_stat_user_indexes, pg_stat_user_tables) are checked",
"Stale planner statistics are considered via dead-tuple trends and autovacuum recency",
"Planner parameters are changed one at a time and verified by re-running the plan",
"Schema decisions such as adding a new index route to data-architect and correlation is not presented as causation"
]
},
{
"id": "replication-and-failover-plan",
"prompt": "We want a standby for our primary PostgreSQL 16 instance so we can survive a server loss. What do we need to configure, how do we verify the standby is healthy, and what should the failover procedure include?",
"expected_output": "A replication setup and failover plan: wal_level must be replica or higher, max_wal_senders and max_replication_slots sized for the standby, a physical replication slot created per standby, and the standby built with pg_basebackup -X stream with primary_conninfo and standby.signal. Health is verified on the primary via pg_stat_replication showing state streaming with replay_lsn close to the primary and replay_lag within the agreed bound, and on the standby via pg_is_in_recovery with pg_last_wal_receive_lsn and pg_last_wal_replay_lsn advancing. The failover procedure names who promotes (pg_ctl promote or SELECT pg_promote()), how clients are redirected and verified with a fresh connection, and how the old primary is rejoined with pg_rewind so two primaries never accept writes. The response distinguishes synchronous from asynchronous replication and states the durability trade-off, and treats promotion as a mutation requiring an explicit human directive and a rollback path.",
"assertions": [
"Setup covers wal_level, max_wal_senders, max_replication_slots, a replication slot, and pg_basebackup -X stream",
"Health is verified with pg_stat_replication state streaming and lag within the agreed bound",
"The failover procedure names promotion, client redirection, and rejoining the old primary with pg_rewind",
"Synchronous versus asynchronous durability trade-offs are stated explicitly",
"Promotion is treated as a mutation requiring a human directive and a rollback path"
]
},
{
"id": "vacuum-and-bloat-investigation",
"prompt": "Our pg_stat_user_tables shows n_dead_tup growing on several large tables and some tables' file sizes look much bigger than their live data. What does this mean and what should we do, and what must we not do?",
"expected_output": "An explanation that rising n_dead_tup means the vacuum loop is falling behind: autovacuum is not reclaiming dead tuples fast enough, and the file-size-to-live-data gap is the bloat signal. The response first looks for why autovacuum lags — long-running transactions pinning old snapshots, not enough autovacuum_max_workers, connection saturation, or tables larger than the autovacuum nap cycle — using pg_stat_progress_vacuum and age(datfrozenxid) to check wraparound risk, then prescribes a targeted, confirmed maintenance vacuum (VACUUM ANALYZE on the specific tables) in a maintenance window. It states the hard boundaries: VACUUM FULL is a last resort that rewrites the table and takes exclusive locks and must never be routine, autovacuum must never be disabled to save load, and a high dead-tuple count is evidence of a lagging maintenance loop, not a diagnosis by itself. Verification is re-running the bloat probe and confirming the trend reversed and autovacuum is keeping up.",
"assertions": [
"Rising dead tuples are interpreted as the vacuum loop falling behind, with bloat as the file-size-to-live-data signal",
"Root causes of autovacuum lag are checked: long transactions, worker starvation, connection saturation, table size",
"The fix is a targeted confirmed maintenance vacuum, not a blanket or routine VACUUM FULL",
"VACUUM FULL's lock and rewrite cost and the never-disable-autovacuum boundary are stated",
"Verification re-runs the bloat probe and confirms the trend reversed"
]
},
{
"id": "major-version-upgrade-plan",
"prompt": "We are on PostgreSQL 15 with the PostGIS and pgvector extensions and want to move to PostgreSQL 16 with minimal downtime. What is the safe plan and what are the traps?",
"expected_output": "A sequenced major-upgrade plan: read the release notes and upgrade guide for the full version span first, inventory extensions and check each one's major-upgrade notes (PostGIS and pgvector are not binary-compatible across major versions and need reinstall or rebuild on the new binaries), run pg_upgrade --check as a no-change preflight, and rehearse on a scratch environment with the real data shape. The response distinguishes pg_upgrade link mode (fast, but the old cluster shares inodes and is not a clean rollback) from copy mode or logical dump/restore (slower, but keeps a rollback path), prescribes running analyze on the new cluster before debugging any 'slow queries', and requires verifying at the application boundary before decommissioning the old cluster. Logical replication is named as the near-zero-downtime alternative whose migration methodology routes to data-engineering. The upgrade is a mutation requiring a maintenance window, a verified backup, and an explicit human directive.",
"assertions": [
"Release notes and upgrade guides for the full version span are read before anything runs",
"Extension major-upgrade gotchas (PostGIS, pgvector) are checked and handled per extension",
"pg_upgrade --check is used as a no-change preflight and link mode versus copy mode trade-offs are stated",
"analyze runs on the new cluster before slow-query debugging, and the old cluster is kept until boundary verification",
"Logical replication is named as the near-zero-downtime path with migration methodology routed to data-engineering"
]
}
]
}
postgres/references/00-source-index.md
+55
View File
@@ -0,0 +1,55 @@
# PostgreSQL Operations — Source Index
> **Last Updated:** 2026-08-03
This index tracks the authoritative sources behind the PostgreSQL operational
skill and the refresh procedure for keeping it current.
## Canonical sources
| Topic | Source |
|---|---|
| Core documentation (current release) | https://www.postgresql.org/docs/current/ |
| Configuration reference | https://www.postgresql.org/docs/current/runtime-config.html |
| Server administration (backup, replication, upgrades) | https://www.postgresql.org/docs/current/admin.html |
| System catalogs and statistics views | https://www.postgresql.org/docs/current/catalogs.html and .../monitoring-stats.html |
| `pg_upgrade` | https://www.postgresql.org/docs/current/pgupgrade.html |
| `pg_basebackup` | https://www.postgresql.org/docs/current/app-pgbasebackup.html |
| Streaming replication | https://www.postgresql.org/docs/current/warm-standby.html and .../streaming-replication.html |
| Continuous archiving and PITR | https://www.postgresql.org/docs/current/continuous-archiving.html |
| `pg_stat_archiver` and statistics views | https://www.postgresql.org/docs/current/monitoring-stats.html |
## Version observations (as of this refresh)
- `pg_stat_replication` exposes `sent_lsn`/`write_lsn`/`flush_lsn`/`replay_lsn`
and the `*_lag` columns on PostgreSQL 10 and later. Pre-10 servers expose
`*_location` columns instead; this skill targets PostgreSQL 10+.
- `pg_backup_start()`/`pg_backup_stop()` replaced `pg_start_backup()`/
`pg_stop_backup()` in PostgreSQL 15. `pg_basebackup` remains the supported
base-backup path on every supported release.
- Recovery configuration moved from `recovery.conf` to `recovery.signal`/
`standby.signal` in PostgreSQL 12. `recovery_target_*` parameters are set in
`postgresql.conf` (or via `ALTER SYSTEM`) and take effect at start time.
- `VACUUM` progress is observable through `pg_stat_progress_vacuum`
(PostgreSQL 9.6+) and `pg_stat_progress_cluster` (12+).
- `pg_promote()` (PostgreSQL 12+) promotes a standby without shelling out.
## Refresh procedure
1. Re-check the sources above for a new minor or major release.
2. Update the version observations that changed (column renames, renamed
functions, moved configuration files).
3. Re-run the bundled diagnostic script against a test instance and confirm
every check still parses: `scripts/pgdiag --psql /path/to/psql --json`.
4. Re-verify the SKILL.md keyword sweep from the validation contract and the
routing links to `backend-engineering`, `data-architect`, and
`data-engineering`.
## Related skill sources
- `data-engineering` owns cross-engine methodology: backup strategy, migration
patterns, and analytical SQL. Its references are the source for engine-spanning
decisions; this skill covers PostgreSQL operation itself.
- `data-architect` owns schema and data modeling; index *choice* at design time
lives there, while index *operation and measurement* live in
`02-indexes-and-query-plans.md`.
postgres/references/01-configuration.md
+80
View File
@@ -0,0 +1,80 @@
# PostgreSQL Configuration Operations
> **Last Updated:** 2026-08-03
This reference covers reading, changing, and verifying PostgreSQL configuration
safely. The runtime source of truth is `pg_settings`, not the configuration
file: values shown by `SHOW`/`current_setting()` reflect reloads, `ALTER
SYSTEM` overrides, and command-line `-c` options.
## Reading configuration
```sql
SHOW shared_buffers;
SELECT name, setting, unit, context, pending_restart
FROM pg_settings
WHERE name IN ('shared_buffers', 'max_connections', 'wal_level', 'archive_mode');
```
The `context` column says when a change takes effect:
- `postmaster` — requires a restart (`pending_restart` becomes true).
- `sighup` — takes effect on reload (`pg_ctl reload` or
`SELECT pg_reload_conf();`).
- `user`/`superuser`/`backend` — settable per-session or per-role.
## Reload versus restart
Requires restart (memory and process-shape parameters):
- `shared_buffers`, `max_connections`, `wal_level`, `max_wal_senders`,
`max_replication_slots`, `max_prepared_transactions`,
`shared_preload_libraries`, `huge_pages`, `dynamic_shared_memory_type`.
Reloads safely (most tuning, logging, and vacuum parameters):
- `work_mem`, `maintenance_work_mem`, `effective_cache_size`,
`random_page_cost`, `seq_page_cost`, `checkpoint_timeout`,
`max_wal_size`, `archive_command`, `autovacuum` and its workers,
`log_min_duration_statement`, `log_statement`, `track_io_timing`.
The bundled `pgdiag` `config` check reports the operator-critical values in
one bounded payload:
```bash
scripts/pgdiag --check config --json
```
## Operator-critical values and what they do
| Parameter | What it controls | Typical signal of trouble |
|---|---|---|
| `shared_buffers` | Postgres's own cache | Too small: heavy `pg_stat_database` read activity while OS cache is idle |
| `effective_cache_size` | Planner's estimate of OS+PG cache | Too small: planner favors index scans that are slower than seq scans in reality |
| `work_mem` | Per-sort/hash memory | Too small: temp-file sorts (`pg_stat_database.temp_files`) |
| `maintenance_work_mem` | Vacuum/index build memory | Too small: slow index builds and slow vacuum |
| `max_connections` | Hard connection cap | Saturation: `pg_stat_activity` near cap, `FATAL: sorry, too many clients` |
| `wal_level` | What WAL carries | Must be `replica` (or higher) for archiving and streaming replication |
| `archive_mode` / `archive_command` | WAL archiving | See `04-backups-wal-pitr.md` |
| `autovacuum` / `autovacuum_max_workers` | Background maintenance | Rising `n_dead_tup`, XID wraparound risk; see `03-vacuum-and-bloat.md` |
| `log_min_duration_statement` | Slow-query logging threshold | 0 if you need every statement; 250-1000ms is a common operations default |
| `track_io_timing` | I/O timing in statistics | Off makes `pg_stat_database` I/O columns meaningless |
| `random_page_cost` | Planner cost of random reads | Overstated on SSD causes seq-scan preference for large tables |
## Changing configuration safely
1. Change in the file (or `ALTER SYSTEM SET ...`) on one named parameter.
2. Determine whether a reload or restart is required from `pg_settings.context`.
3. Prefer reload; schedule restarts with a maintenance window and a verified
rollback (the previous value).
4. Verify with `SHOW`/`current_setting()` and re-run the affected diagnostic.
Hard boundaries:
- Never change `shared_preload_libraries` without a restart plan — a typo can
make the server fail to start.
- Never raise `max_connections` without accounting for backend memory per
connection (`work_mem` is per operation, but each backend holds base memory).
- Connection pooling for application workloads is an application-architecture
decision that routes to `backend-engineering`; `max_connections` tuning here
is server capacity, not app design.
postgres/references/02-indexes-and-query-plans.md
+78
View File
@@ -0,0 +1,78 @@
# Indexes and Query Plans
> **Last Updated:** 2026-08-03
This reference covers measuring and operating indexes and diagnosing query
plans. Designing which index to add for a new workload is schema design and
routes to `data-architect`; this skill owns reading the evidence and operating
what exists.
## Evidence first: statistics views
```sql
-- Which indexes are actually used, most used first
SELECT schemaname, relname, indexrelname, idx_scan, idx_tup_read, idx_tup_fetch
FROM pg_stat_user_indexes
ORDER BY idx_scan DESC LIMIT 10;
-- Indexes with no recorded scans (write cost with no read benefit)
SELECT schemaname, relname, indexrelname, idx_scan
FROM pg_stat_user_indexes
WHERE idx_scan = 0
ORDER BY relname, indexrelname;
-- Tables scanned sequentially a lot (candidate for a missing index)
SELECT schemaname, relname, seq_scan, seq_tup_read, idx_scan
FROM pg_stat_user_tables
WHERE seq_scan > 0
ORDER BY seq_tup_read DESC LIMIT 10;
-- Indexes marked unusable (dropped on next vacuum)
SELECT c.relname AS index_name, i.indrelid::regclass AS table_name
FROM pg_index i
JOIN pg_class c ON c.oid = i.indexrelid
WHERE NOT i.indisvalid;
```
The bundled `pgdiag` script runs these probes in its `index_usage`,
`unused_indexes`, `invalid_indexes`, and `seq_scan_heavy` checks.
## Reading a plan
- `EXPLAIN` shows the planner's estimate; `EXPLAIN (ANALYZE, BUFFERS)` executes
the statement and reports actual rows, timing, and buffer usage. Read the
estimated-versus-actual row counts: a large mismatch means the planner is
working from stale or missing statistics.
- `EXPLAIN (FORMAT JSON)` is the machine-readable form:
`scripts/pgdiag --plan-for "SELECT ..." --json`.
- Node shapes to recognize: `Seq Scan` (whole-table read), `Index Scan` /
`Index Only Scan` (index seek), `Bitmap Heap Scan` (index + heap filter),
`Nested Loop`/`Hash Join`/`Merge Join`, `Sort`, `Materialize`.
## Common findings and next steps
| Finding | Likely cause | Check before acting |
|---|---|---|
| `Seq Scan` on a filtered large table | Missing index or planner cost settings | `pg_stat_user_tables.seq_tup_read`; actual row estimate vs `random_page_cost`/`effective_cache_size` |
| Estimated rows far from actual | Stale `pg_statistic` | Has `autovacuum`/autoanalyze run since the last big change? |
| Index with `idx_scan = 0` for weeks | Unused index (write overhead) | Confirm the access path is genuinely gone, then a deliberate drop in a maintenance window |
| Invalid index | Interrupted build / catalog issue | Repair deliberately; never assume it is serving queries |
| `Index Only Scan` not used | Visibility map not set | Check `pg_class.relallvisible`; the heap pages are probably dirty |
## Query-plan hygiene
- Test plans on the real workload shape, not on tiny samples: statistics scale
with data, and a plan that is right on a 1k-row table is often wrong at 10M.
- Change planner parameters one at a time and re-measure with
`EXPLAIN (ANALYZE, BUFFERS)` before and after.
- `pg_stat_statements` (extension) is the best aggregate evidence for which
statements deserve plan work. See `06-extensions.md` for enabling it.
- Never "fix" a slow query by disabling index scans globally
(`enable_indexscan=off`): treat that as a diagnostic probe, not a fix.
## Verification
After any index change: re-run the affected query's `EXPLAIN (ANALYZE,
BUFFERS)` and compare total time and rows, and re-check `idx_scan` trends on
the next reporting window. A plan change is verified by measurement, not by
assertion.
postgres/references/03-vacuum-and-bloat.md
+77
View File
@@ -0,0 +1,77 @@
# Vacuum and Bloat Management
> **Last Updated:** 2026-08-03
This reference covers the maintenance loop that keeps PostgreSQL tables
healthy: autovacuum supervision, dead-tuple and bloat measurement, and the
disciplined use of manual vacuum operations.
## Why vacuum matters
- Vacuum reclaims dead tuple space and refreshes the visibility map and
planner statistics. Without it, tables bloat and transaction ID wraparound
becomes an emergency.
- `autovacuum` is on by default; the operational task is *supervision*: is it
keeping up with the workload?
## Measuring the signals
```sql
-- Dead tuples per table, worst first
SELECT schemaname, relname, n_live_tup, n_dead_tup,
round(100.0 * n_dead_tup / NULLIF(n_live_tup + n_dead_tup, 0), 1) AS dead_pct
FROM pg_stat_user_tables
WHERE n_dead_tup > 0
ORDER BY n_dead_tup DESC LIMIT 10;
-- How far tables are from forced anti-wraparound vacuum
SELECT datname, age(datfrozenxid) AS xid_age
FROM pg_database ORDER BY xid_age DESC;
-- Is a vacuum running right now, and where?
SELECT * FROM pg_stat_progress_vacuum;
```
The `pgdiag` `bloat` check reports the dead-tuple signal in one bounded payload.
## Interpreting the evidence
- `n_dead_tup` rising faster than vacuum runs complete → autovacuum is falling
behind. Common causes: not enough `autovacuum_max_workers`, long-running
transactions pinning old snapshots, connection saturation, or very large
tables where a full pass takes longer than `autovacuum_naptime`.
- Table file size far above live-data size → heap bloat. Check `pg_class`
`relpages` against `n_live_tup` × average row width; a bloat estimate query
can quantify it.
- Index bloat shows as large `relpages` on indexes relative to key count.
- `age(datfrozenxid)` approaching `autovacuum_freeze_max_age` (default 200M) is
the wraparound warning; below ~50M it is time to act deliberately.
## Acting deliberately
1. Confirm the target and maintenance window with the human before any
maintenance vacuum; a manual `VACUUM` on a big table is a bounded mutation
with a measurable cost.
2. Prefer `VACUUM (ANALYZE)` on the specific tables showing the problem over a
blanket full-cluster pass.
3. `VACUUM FULL` rewrites the table and takes an exclusive lock; it is a
last resort for severe bloat, never routine. It requires a maintenance
window and a verified backup path.
4. After acting, re-run the `bloat`/dead-tuple probe and confirm the trend
direction changed; also confirm autovacuum is keeping up afterwards.
## Config levers (all reload-safe)
- `autovacuum_max_workers`, `autovacuum_naptime`, `autovacuum_vacuum_cost_limit`
— worker capacity and pacing.
- Per-table overrides via storage parameters (`autovacuum_vacuum_scale_factor`,
`autovacuum_vacuum_threshold`) for tables that need faster or slower cycles.
## Hard boundaries
- Never run `VACUUM FULL` without a maintenance window and a confirmed backup.
- Never stop autovacuum to "save load" on a production instance — the deferred
work comes back as bloat and wraparound risk.
- Never present a high `n_dead_tup` as a diagnosis by itself; it is evidence
that the vacuum loop needs attention, and the cause (long transactions,
worker starvation, pacing) must be identified before acting.
postgres/references/04-backups-wal-pitr.md
+95
View File
@@ -0,0 +1,95 @@
# Backups: WAL Archiving and Point-in-Time Recovery
> **Last Updated:** 2026-08-03
This reference covers the operational side of backups for PostgreSQL: WAL
archiving readiness, base backups, and point-in-time recovery (PITR). The
cross-engine backup *strategy* (RPO/RTO targets, retention policy, off-site
copies) belongs to `data-engineering`; this skill owns the PostgreSQL
mechanics and their verification.
## The recovery model
A usable recovery point is a **base backup plus the WAL archive that follows
it**: restore the base, then replay archived WAL up to the target time or LSN.
Without continuous archiving you can only recover to the moment of the last
base backup.
## WAL archiving readiness
```sql
-- The three settings that gate archiving (wal_level change needs restart)
SELECT name, setting FROM pg_settings
WHERE name IN ('wal_level', 'archive_mode', 'archive_command', 'archive_timeout');
-- Archiver health: archived_count must grow, failed_count must stay flat
SELECT archived_count, failed_count, last_archived_wal, last_archived_time,
last_failed_wal, last_failed_time
FROM pg_stat_archiver;
```
The `pgdiag` `wal_archive` check reports the archiver row directly.
Readiness checklist:
- `wal_level = replica` (or higher) and `archive_mode = on`.
- `archive_command` succeeds for every segment; it must be idempotent and
return zero only on success (a failing command makes the server retry and
log failures).
- `pg_stat_archiver.failed_count` is not climbing and `last_failed_wal` is
stale. A rising `failed_count` means archiving is broken *right now*, and
every segment since then widens the PITR gap.
## Taking a base backup
```bash
# Consistent base backup with WAL included (the supported path)
pg_basebackup -h primary -D /backup/base-2026-08-03 -X stream -c fast -P
# Label and record it
echo "backup of primary at $(date -Is)" > /backup/base-2026-08-03/BACKUP_LABEL
```
- Use `-X stream` (or `-X fetch`) so WAL segments produced during the backup
are included — a base backup without its WAL is not restorable.
- Record the label and timestamp; a restore target is only as good as the
backup's metadata.
- Test restore: on a scratch instance, restore the base, point
`restore_command` at the archive, and verify the server reaches the expected
recovery point. This is the only evidence that the backup works.
## Restoring to a point in time
1. Restore the base backup to the target data directory.
2. Set `restore_command` (how to fetch archived WAL) and a recovery target —
`recovery_target_time` (e.g., `'2026-08-03 02:00:00'`),
`recovery_target_lsn`, or `recovery_target_xid`.
3. Start the server; it replays WAL to the target and stops (or enters
recovery if `recovery_target_action = promote` on 12+).
4. Verify the data at the boundary (the row/table the incident was about),
then promote when confident.
Recovery-target kinds: time-based (the `pitr|point.in.time` family — a
wall-clock moment) or position-based (an LSN or timeline marker). Choose the
kind that matches how you know *when things went wrong*; a wall-clock target
is the common case.
## Verification boundaries
| Claim | Minimum evidence |
|---|---|
| Archiving is healthy | `archived_count` increasing and `failed_count` flat over a window |
| A backup exists | Labeled base backup directory with matching WAL |
| Backups support recovery | A full restore-to-time drill on a scratch instance, logged |
| PITR target is reachable | Server reaches the target and the expected rows exist |
## Hard boundaries
- Never claim recoverability from a backup log alone; only a restore drill is
evidence.
- Never run a restore against the production data directory without an
explicit human directive; a restore is a mutation.
- Never let `archive_command` return success on failure — silent archiving
failure widens the recovery gap undetected.
- Never keep only base backups: without WAL there is no point-in-time
recovery, only point-of-backup recovery.
postgres/references/05-replication-and-failover.md
+86
View File
@@ -0,0 +1,86 @@
# Replication and Failover
> **Last Updated:** 2026-08-03
This reference covers streaming replication setup, monitoring, and the
failover runbook. Replication topology design (sync vs async, quorum, cluster
managers) is an architecture decision; this skill owns the PostgreSQL
mechanics and their verification.
## Streaming replication model
A standby connects to the primary with a replication slot, receives WAL
segments as they are produced, and replays them. `wal_level = replica` (or
higher) and `max_wal_senders`/`max_replication_slots` must be sized for the
number of standbys.
Setup essentials:
- Create a physical replication slot per standby
(`SELECT pg_create_physical_replication_slot('standby-1');`).
- Build the standby with `pg_basebackup -X stream` and configure
`primary_conninfo` in `postgresql.conf` (12+) with `standby.signal`.
- Confirm the standby is actually streaming: it is `in_recovery` and appears
in `pg_stat_replication` on the primary.
## Measuring replication health
```sql
-- On the primary: who is streaming and how far behind
SELECT application_name, state, sync_state, client_addr,
sent_lsn, write_lsn, flush_lsn, replay_lsn, replay_lag
FROM pg_stat_replication;
-- On the standby: is it receiving and replaying?
SELECT pg_is_in_recovery(), pg_last_wal_receive_lsn(), pg_last_wal_replay_lsn();
```
The `pgdiag` `recovery` and `replication` checks report both views.
Health rules of thumb:
- `state = streaming` for every standby; anything else (`startup`,
`catchup`, `backup`) is transitional or stuck.
- Lag should stay within the agreed bound. `replay_lag` measures the gap
between the primary's current WAL and what the standby has replayed.
- A slot that is far behind but still `streaming` means the standby cannot
keep up or the network is saturated — the slot's retained WAL grows on the
primary until it does.
## Synchronous versus asynchronous
- Asynchronous (default): the primary commits without waiting; failover can
lose the most recent commits.
- Synchronous (`synchronous_standby_names = 'standby-1'`): the primary waits
for that standby's flush before acknowledging commits. Trade-off: commit
latency for a durability guarantee.
- Choose deliberately and document the choice; the failover runbook must
state what durability was promised.
## Failover runbook
A failover plan names: who promotes, how clients are redirected, what happens
to the old primary when it returns, and how the result is verified.
1. **Confirm the directive**: failover is a mutation; it requires an explicit
human decision naming the target standby.
2. **Check lag and timeline first**: how much data is at risk, and has the
standby been applying continuously? Promotion with a lagging standby is a
deliberate data-loss decision, not an accident.
3. **Promote**: `pg_ctl promote` or `SELECT pg_promote();` on the chosen
standby. With a cluster manager (Patroni, repmgr) use its switchover
command instead of manual promotion.
4. **Redirect clients**: DNS, connection strings, or the pooler — verify a
fresh connection lands on the new primary and writes succeed.
5. **Rejoin the old primary** as a standby with `pg_rewind` (it is now
diverged from the new primary's timeline) — never let two primaries accept
writes.
## Hard boundaries
- Never promote without a human directive and a stated rollback path.
- Never let two primaries run: the old primary must be fenced or rejoined
before it can accept writes again.
- Never fail over to a standby with unknown lag and call it "the same data".
- Never disable `archive_mode`/WAL sending to "simplify" replication — the
archive and the stream are both part of the recovery story.
postgres/references/06-extensions.md
+66
View File
@@ -0,0 +1,66 @@
# Extensions
> **Last Updated:** 2026-08-03
This reference covers inventorying, installing, and upgrading PostgreSQL
extensions, with attention to the upgrade and security gotchas that bite
operators.
## Inventory first
```sql
-- Installed extensions and versions
SELECT extname, extversion FROM pg_extension ORDER BY extname;
-- Available on this installation (name, default version, requires)
SELECT name, default_version, installed_version FROM pg_available_extensions
ORDER BY name;
```
The `pgdiag` `extensions` check reports the installed set.
## Installing an extension
- An extension install is a mutation: it changes the shared catalog and, for
many extensions, the database schema and behavior. Confirm the target and
rollback path before running `CREATE EXTENSION`.
- Some extensions must be loaded at server start via
`shared_preload_libraries` (e.g., `pg_stat_statements`, `timescaledb`,
`citus`) — a restart is required and a misconfigured library can prevent
startup.
- Trusted extensions (`pg_available_extensions.trusted = true`) can be
installed by non-superusers into their own databases; anything else needs
superuser.
- Extension *choice* and lifecycle policy are infrastructure decisions;
shared-library availability and packaging are platform concerns that route
to `platform-engineering`.
## Operating notes for common extensions
| Extension | Operational notes |
|---|---|
| `pg_stat_statements` | Best aggregate query evidence; needs `shared_preload_libraries` + restart; `pg_stat_statements_reset()` to reset |
| `postgis` | Large library; major-version upgrades need a reinstall or upgrade script per database |
| `pgvector` | Indexes are not binary-compatible across major versions — rebuild after a major upgrade |
| `pgcrypto`, `uuid-ossp` | Stable and small; rarely the source of upgrade pain |
| `hstore`, `citext` | Plain catalog extensions; safe through `pg_upgrade` with the `--no-...` flags checked |
## Upgrading extensions
- `ALTER EXTENSION name UPDATE TO 'newversion';` upgrades an installed
extension when the extension's script provides the path.
- A major PostgreSQL upgrade commonly requires new extension binaries: run
`pg_upgrade` and then install/re-install the extension versions matching the
new server, or upgrade each database's extension before decommissioning the
old cluster.
- Check each extension's release notes for the target version span *before*
the upgrade window; PostGIS and pgvector in particular publish explicit
major-upgrade procedures.
## Hard boundaries
- Never install an extension into production without a human directive, a
named rollback (drop or version pin), and a verified backup path.
- Never load an extension via `shared_preload_libraries` without a restart
plan and a check that the library exists on disk.
- Never assume an extension survives a major upgrade; verify per extension.
postgres/references/07-upgrades.md
+78
View File
@@ -0,0 +1,78 @@
# Upgrades
> **Last Updated:** 2026-08-03
This reference covers upgrading PostgreSQL safely, distinguishing minor
in-place upgrades from major-version upgrades, and the rollback decision at
each step.
## Minor versus major
- **Minor upgrades** (e.g., 16.4 → 16.5) are binary swaps: stop, replace
binaries, start. No data-format change; a restart is the whole procedure.
- **Major upgrades** (e.g., 15 → 16) change the on-disk format. Options:
`pg_upgrade` (in-place, fast, link or copy mode) or logical dump/restore
(`pg_dump`/`pg_dumpall`). Both are mutations requiring downtime and a
verified backup.
## Before any major upgrade
1. Read the release notes and the upgrade guide for **the full version span**
— skipping intermediate major versions compounds risks and deprecations.
2. Check extensions (see `06-extensions.md`): which have new binaries, which
need reinstall, which block the move.
3. Check `pg_upgrade` prerequisites: `--check` mode reports problems without
changing anything — run it first.
4. Rehearse in a scratch environment with the real data shape, including
representative table sizes and the workload's slowest queries.
## pg_upgrade flow
```bash
# Pre-flight only — no changes
/path/to/new/bin/pg_upgrade --old-bindir /path/to/old/bin \
--new-bindir /path/to/new/bin --old-datadir /var/lib/pg15 \
--new-datadir /var/lib/pg16 --check
# Real run (after stop-writes confirmation and a verified backup)
/path/to/new/bin/pg_upgrade --old-bindir /path/to/old/bin \
--new-bindir /path/to/new/bin --old-datadir /var/lib/pg15 \
--new-datadir /var/lib/pg16
```
- Stop writes on the old cluster first; the upgrade moves data between the
two directories.
- `--link` mode is fast but makes the old cluster unusable until the new one
works (shared inodes); `--copy` is slower but keeps the old cluster intact
as a rollback path.
- After the upgrade, run `analyze` on the new cluster (fresh planner
statistics are mandatory) and re-install/upgrade extensions per their notes.
- Verify at the application boundary — the workflow the database serves —
before decommissioning the old cluster.
## Logical upgrade paths
- `pg_dumpall`/`pg_dump` + restore into a fresh cluster: universal, slow for
large data, but the safest for unusual setups.
- Logical replication (publisher/subscriber) as a near-zero-downtime major
upgrade: run the new major as a subscriber, catch up, switch. This is also a
migration pattern whose methodology lives in `data-engineering`.
## Rollback decision
- With `--copy` mode (or dump/restore), rollback is "start the old cluster
again" — but WAL divergence since the upgrade start means the old cluster's
data reflects the stop point, not anything written to the new one.
- Decide the rollback trigger *before* the window: how much new-write loss is
acceptable, and who calls the rollback.
- Never delete the old cluster or its backup until the new cluster has
survived the verification boundary.
## Hard boundaries
- Never run a major upgrade without a human directive, a maintenance window, a
verified backup, and a rehearsed run.
- Never skip `analyze` on the new cluster and then debug "slow queries" as if
they were config problems.
- Never decommission the old cluster until the new one is verified at the
application boundary.
postgres/references/08-diagnostics.md
+68
View File
@@ -0,0 +1,68 @@
# Diagnostics with Evidence
> **Last Updated:** 2026-08-03
This reference maps symptoms to probes and fixes for PostgreSQL incidents,
with the evidence discipline that separates a real diagnosis from a guess.
## Evidence order
Diagnose in this order — each layer is cheaper than the next and rules out
whole classes of cause:
1. **Identity and version** — what are we actually looking at (primary or
standby, which version)?
2. **Configuration** — are the operator-critical settings what the workload
assumes?
3. **Connections** — is the instance saturated, or is the app doing something
odd?
4. **Indexes and plans** — is the query the problem, or the data shape?
5. **Vacuum/bloat** — is maintenance keeping up?
6. **WAL archiving** — is the recovery story intact?
7. **Replication** — is the standby current and is failover defensible?
8. **Extensions** — did a library or extension change break something?
The bundled `pgdiag` script collects layers 1–8 in one bounded JSON payload:
```bash
scripts/pgdiag --json
scripts/pgdiag --check connections --check wal_archive --json # targeted
scripts/pgdiag --plan-for "SELECT ..." --json # layer 4 probe
```
## Symptom-to-probe table
| Symptom | First probes | Likely next step |
|---|---|---|
| Queries suddenly slow | `pgdiag --json`; `EXPLAIN (ANALYZE, BUFFERS)` on the slow statement | Check plan row estimates vs actual; check `n_dead_tup` trend and planner stats freshness |
| Connections rejected | `pgdiag` `connections`; `pg_stat_activity` state counts | Compare against `max_connections`; look for stuck/idle-in-transaction backends; pooler sizing is `backend-engineering` |
| Backups stop completing | `pgdiag` `wal_archive`; archive destination disk/network | `failed_count`/`last_failed_wal`; fix `archive_command` or storage |
| Standby falls behind | `pgdiag` `replication` + `recovery`; `pg_replication_slots` | Lag columns, slot retention, network saturation; consider sync config |
| Instance is slow overall | `pg_stat_database` I/O + `pg_stat_bgwriter` | Check `track_io_timing` is on; look for checkpoint storms, heavy seq scans |
| Autovacuum stuck | `pg_stat_progress_vacuum`; long-running transactions | Find the snapshot-pinning transaction; tune workers if genuinely starved |
| After an upgrade, "everything is slow" | Planner statistics; extension reinstall | Run `analyze`; verify extensions were upgraded per `06-extensions.md` |
## Evidence discipline
- **Measure before claiming.** "The query is slow" is a symptom; "the plan
shows a seq scan with 1.5M rows read while the index has zero scans" is
evidence.
- **Correlation is not cause.** A high `n_dead_tup` and a slow query in the
same window are correlated, not necessarily causal. State what was measured,
what changed, and what was verified.
- **Re-run after acting.** A fix is verified when the relevant probe returns
the expected value, not when the symptom seems quieter.
- **Keep evidence bounded.** Summarize statistics rows and log excerpts;
never dump full logs or connection strings with passwords into chat.
- **When to stop.** Stop after three non-converging diagnostic passes and
report the evidence gathered so far, the hypotheses ruled out, and the
remaining candidates — rather than escalating into unconfirmed changes.
## Log sources
- Server log (wherever `logging_collector` writes) for errors, checkpoints,
and slow statements when `log_min_duration_statement` is set.
- `pg_stat_activity` for live state; `pg_stat_archiver` and
`pg_stat_replication` for the recovery story.
- Never parse the full server log into a response; extract the bounded window
relevant to the incident.
postgres/scripts/pgdiag
Executable
+411
View File
@@ -0,0 +1,411 @@
#!/usr/bin/env python3
"""pgdiag - read-only PostgreSQL operational diagnostics with bounded evidence.
Collects operational evidence from a live PostgreSQL instance through the
psql client without mutating anything: server identity and version,
configuration values, connection pressure, index usage, bloat signals, WAL
archiving health, recovery and replication state, installed extensions, and
database sizes. Every psql session opens with default_transaction_read_only=on
so the server itself rejects any write attempt; the tool only ever issues
read-only statements.
The script uses the Python standard library only, and --help works with no
PostgreSQL server and no psql binary installed. Diagnostic data is emitted as
bounded JSON with --json, or as human-readable text otherwise.
Exit codes: 0 ok, 1 runtime/collection error, 2 usage error,
127 psql binary not found, 124 timeout.
"""
import argparse
import json
import os
import subprocess
import sys
from datetime import datetime, timezone
from typing import Any, Dict, List, Optional
READ_ONLY_SQL = "SET default_transaction_read_only = on"
READ_ONLY_PREFIXES = ("SELECT", "SHOW", "WITH", "EXPLAIN")
DEFAULT_TIMEOUT = 30
CHECKS = [
{
"name": "identity",
"title": "Server identity and version",
"sql": (
"SELECT current_database(), current_user, "
"current_setting('server_version'), "
"current_setting('server_version_num'), "
"pg_is_in_recovery(), pg_postmaster_start_time()"
),
},
{
"name": "config",
"title": "Configuration values",
"sql": (
"SELECT 'max_connections', current_setting('max_connections') "
"UNION ALL SELECT 'shared_buffers', current_setting('shared_buffers') "
"UNION ALL SELECT 'effective_cache_size', current_setting('effective_cache_size') "
"UNION ALL SELECT 'work_mem', current_setting('work_mem') "
"UNION ALL SELECT 'maintenance_work_mem', current_setting('maintenance_work_mem') "
"UNION ALL SELECT 'wal_level', current_setting('wal_level') "
"UNION ALL SELECT 'max_wal_size', current_setting('max_wal_size') "
"UNION ALL SELECT 'checkpoint_timeout', current_setting('checkpoint_timeout') "
"UNION ALL SELECT 'archive_mode', current_setting('archive_mode') "
"UNION ALL SELECT 'archive_command', current_setting('archive_command') "
"UNION ALL SELECT 'archive_timeout', current_setting('archive_timeout') "
"UNION ALL SELECT 'max_wal_senders', current_setting('max_wal_senders') "
"UNION ALL SELECT 'max_replication_slots', current_setting('max_replication_slots') "
"UNION ALL SELECT 'autovacuum', current_setting('autovacuum') "
"UNION ALL SELECT 'autovacuum_max_workers', current_setting('autovacuum_max_workers') "
"UNION ALL SELECT 'autovacuum_naptime', current_setting('autovacuum_naptime') "
"UNION ALL SELECT 'log_min_duration_statement', current_setting('log_min_duration_statement') "
"UNION ALL SELECT 'log_statement', current_setting('log_statement') "
"UNION ALL SELECT 'ssl', current_setting('ssl') "
"UNION ALL SELECT 'track_io_timing', current_setting('track_io_timing') "
"UNION ALL SELECT 'random_page_cost', current_setting('random_page_cost') "
"UNION ALL SELECT 'seq_page_cost', current_setting('seq_page_cost')"
),
},
{
"name": "connections",
"title": "Connection activity by state",
"sql": (
"SELECT state, count(*) FROM pg_stat_activity "
"GROUP BY state ORDER BY count(*) DESC"
),
},
{
"name": "index_usage",
"title": "Most-used indexes",
"sql": (
"SELECT schemaname, relname, indexrelname, idx_scan, idx_tup_read, idx_tup_fetch "
"FROM pg_stat_user_indexes ORDER BY idx_scan DESC LIMIT 10"
),
},
{
"name": "unused_indexes",
"title": "Indexes with no recorded scans",
"sql": (
"SELECT schemaname, relname, indexrelname, idx_scan "
"FROM pg_stat_user_indexes WHERE idx_scan = 0 "
"ORDER BY relname, indexrelname LIMIT 10"
),
},
{
"name": "invalid_indexes",
"title": "Indexes marked unusable",
"sql": (
"SELECT c.relname AS index_name, i.indrelid::regclass AS table_name "
"FROM pg_index i JOIN pg_class c ON c.oid = i.indexrelid "
"WHERE NOT i.indisvalid"
),
},
{
"name": "seq_scan_heavy",
"title": "Tables with heavy sequential scans",
"sql": (
"SELECT schemaname, relname, seq_scan, seq_tup_read, idx_scan "
"FROM pg_stat_user_tables WHERE seq_scan > 0 "
"ORDER BY seq_tup_read DESC LIMIT 10"
),
},
{
"name": "bloat",
"title": "Dead-tuple accumulation signals",
"sql": (
"SELECT schemaname, relname, n_live_tup, n_dead_tup, "
"round(100.0 * n_dead_tup / NULLIF(n_live_tup + n_dead_tup, 0), 1) AS dead_pct "
"FROM pg_stat_user_tables WHERE n_dead_tup > 0 "
"ORDER BY n_dead_tup DESC LIMIT 10"
),
},
{
"name": "wal_archive",
"title": "WAL archiving health",
"sql": (
"SELECT archived_count, failed_count, last_archived_wal, "
"last_archived_time, last_failed_wal, last_failed_time "
"FROM pg_stat_archiver"
),
},
{
"name": "recovery",
"title": "Recovery state",
"sql": (
"SELECT pg_is_in_recovery(), pg_last_wal_receive_lsn(), pg_last_wal_replay_lsn()"
),
},
{
"name": "replication",
"title": "Replication state",
"sql": (
"SELECT application_name, state, sync_state, client_addr, "
"sent_lsn, write_lsn, flush_lsn, replay_lsn "
"FROM pg_stat_replication"
),
},
{
"name": "extensions",
"title": "Installed extensions",
"sql": "SELECT extname, extversion FROM pg_extension ORDER BY extname",
},
{
"name": "databases",
"title": "Database sizes",
"sql": (
"SELECT datname, pg_size_pretty(pg_database_size(datname)) AS size "
"FROM pg_database ORDER BY pg_database_size(datname) DESC"
),
},
]
CHECKS_BY_NAME = {check["name"]: check for check in CHECKS}
def emit(payload: Dict[str, Any], as_json: bool) -> None:
"""Write a payload to stdout; JSON when --json, readable text otherwise."""
if as_json:
print(json.dumps(payload, indent=2, sort_keys=True))
return
lines: List[str] = []
for check in payload.get("checks", []):
lines.append(f"== {check['name']}: {check['title']} ==")
if check["status"] == "error":
lines.append(f" error: {check['error']}")
continue
for row in check.get("rows", []):
lines.append(" " + " | ".join(str(value) for value in row))
if "error" in payload:
lines.append(f"error: {payload['error']}")
if not lines:
lines.append("no diagnostics collected")
print("\n".join(lines))
def locate_psql(explicit: Optional[str]) -> Optional[str]:
"""Resolve the psql binary: --psql wins, otherwise search the PATH."""
if explicit:
if os.path.isfile(explicit) and os.access(explicit, os.X_OK):
return explicit
return None
for directory in os.environ.get("PATH", "").split(os.pathsep):
for name in ("psql", "psql.exe"):
candidate = os.path.join(directory, name)
if os.path.isfile(candidate) and os.access(candidate, os.X_OK):
return candidate
return None
def build_command(binary: str, args: argparse.Namespace, sql: str) -> List[str]:
"""Assemble a single-session psql command that is read-only by default."""
command = [binary, "-X", "-q", "-A", "-t", "-F", "|"]
if args.host:
command += ["-h", args.host]
if args.port:
command += ["-p", str(args.port)]
if args.user:
command += ["-U", args.user]
if args.dbname:
command += ["-d", args.dbname]
command += ["-c", READ_ONLY_SQL, "-c", sql]
return command
def validate_plan_query(query: str) -> Optional[str]:
"""Return a normalized single read-only query for EXPLAIN, or None if unsafe.
Accepts only a single statement (allowing one trailing semicolon) whose
first keyword is a read-only prefix. The read-only session setting remains
the authoritative guard: EXPLAIN only plans, it never executes, and any
server-side write rejection still applies to the session.
"""
stripped = query.strip()
if not stripped:
return None
without_semicolon = stripped[:-1] if stripped.endswith(";") else stripped
if ";" in without_semicolon:
return None
first_word = without_semicolon.lstrip().split(None, 1)[0].upper()
if first_word not in READ_ONLY_PREFIXES:
return None
return without_semicolon
def run_check(binary: str, args: argparse.Namespace, check: Dict[str, Any]) -> Dict[str, Any]:
"""Run one diagnostic check; never allow a failure to stop the others."""
result: Dict[str, Any] = {"name": check["name"], "title": check["title"]}
command = build_command(binary, args, check["sql"])
try:
proc = subprocess.run(
command,
capture_output=True,
text=True,
timeout=args.timeout,
)
except subprocess.TimeoutExpired:
result["status"] = "error"
result["error"] = f"psql timed out after {args.timeout}s"
return result
if proc.returncode != 0:
result["status"] = "error"
result["error"] = proc.stderr.strip() or f"psql exited {proc.returncode}"
return result
rows = [line.split("|") for line in proc.stdout.splitlines() if line.strip()]
result["status"] = "ok"
result["rows"] = rows
return result
def run_plan(binary: str, args: argparse.Namespace, query: str) -> Dict[str, Any]:
"""EXPLAIN (FORMAT JSON) a caller-supplied read-only query."""
result: Dict[str, Any] = {"name": "plan", "title": "Query plan", "status": "ok"}
command = build_command(binary, args, f"EXPLAIN (FORMAT JSON) {query}")
try:
proc = subprocess.run(
command,
capture_output=True,
text=True,
timeout=args.timeout,
)
except subprocess.TimeoutExpired:
result["status"] = "error"
result["error"] = f"psql timed out after {args.timeout}s"
return result
if proc.returncode != 0:
result["status"] = "error"
result["error"] = proc.stderr.strip() or f"psql exited {proc.returncode}"
return result
output = proc.stdout.strip()
try:
result["plan"] = json.loads(output)
except json.JSONDecodeError:
result["plan"] = output
return result
def main(argv: Optional[List[str]] = None) -> int:
parser = argparse.ArgumentParser(
prog="pgdiag",
description=(
"Collect read-only PostgreSQL operational diagnostics through psql. "
"Every session runs with default_transaction_read_only=on, so the "
"server rejects any write attempt."
),
epilog=(
"Exit codes: 0 ok, 1 runtime/collection error, 2 usage error, "
"127 psql binary not found, 124 timeout."
),
)
parser.add_argument("--host", help="server host (default: local socket or PGHOST)")
parser.add_argument("--port", type=int, help="server port (default: 5432 or PGPORT)")
parser.add_argument("--user", help="role name (default: current OS user or PGUSER)")
parser.add_argument("--dbname", help="database to connect to (default: user name)")
parser.add_argument(
"--psql",
help="path to the psql binary (default: first psql on PATH)",
)
parser.add_argument(
"--json",
action="store_true",
help="emit machine-readable JSON on stdout",
)
parser.add_argument(
"--plan-for",
metavar="QUERY",
help=(
"run EXPLAIN (FORMAT JSON) on a single read-only statement "
"(SELECT, SHOW, WITH, or EXPLAIN) and add it as the plan check"
),
)
parser.add_argument(
"--check",
action="append",
default=[],
metavar="NAME",
help=(
"run only the named check; repeatable. Names: "
+ ", ".join(check["name"] for check in CHECKS)
),
)
parser.add_argument(
"--timeout",
type=int,
default=DEFAULT_TIMEOUT,
help="seconds to wait per psql invocation (default: %(default)s)",
)
parser.add_argument(
"--version",
action="version",
version="pgdiag 1.0.0",
)
args = parser.parse_args(argv)
binary = locate_psql(args.psql)
if binary is None:
emit(
{
"ok": False,
"error": (
"psql binary not found; install the PostgreSQL client or pass "
"--psql with an explicit path"
),
},
args.json,
)
return 127
if args.check:
unknown = [name for name in args.check if name not in CHECKS_BY_NAME]
if unknown:
parser.error(f"unknown check(s): {', '.join(unknown)}")
selected = [CHECKS_BY_NAME[name] for name in args.check]
else:
selected = list(CHECKS)
results = [run_check(binary, args, check) for check in selected]
if args.plan_for:
query = validate_plan_query(args.plan_for)
if query is None:
emit(
{
"ok": False,
"error": (
"--plan-for requires a single read-only statement whose first "
"keyword is SELECT, SHOW, WITH, or EXPLAIN"
),
},
args.json,
)
return 2
results.append(run_plan(binary, args, query))
server: Dict[str, Any] = {}
for result in results:
if result["name"] == "identity" and result["status"] == "ok" and result.get("rows"):
row = result["rows"][0]
server = {
"database": row[0],
"user": row[1],
"version": row[2],
"version_num": row[3],
"in_recovery": row[4],
"started_at": row[5],
}
break
ok = any(result["status"] == "ok" for result in results)
payload: Dict[str, Any] = {
"ok": ok,
"psql": binary,
"server": server,
"checks": results,
"generated_at": datetime.now(timezone.utc).isoformat(),
}
emit(payload, args.json)
return 0 if ok else 1
if __name__ == "__main__":
sys.exit(main())
postgres/tests/test_pgdiag.py
+215
View File
@@ -0,0 +1,215 @@
#!/usr/bin/env python3
"""Deterministic tests for the postgres/scripts/pgdiag diagnostic tool.
Runs the script as a subprocess so the tests exercise the real CLI surface
(--help, --json, --check selection, --plan-for, exit codes, JSON payloads).
No PostgreSQL server is required: a fake psql stub is written to a temp
directory at test time and pointed at with --psql. The stub also enforces
the read-only contract by failing any invocation that does not carry the
default_transaction_read_only session setting.
"""
import json
import os
import stat
import subprocess
import sys
import tempfile
import unittest
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent
SCRIPT = ROOT / "scripts" / "pgdiag"
STUB_TEMPLATE = """\
#!/usr/bin/env python3
import os
import sys
args = sys.argv[1:]
sqls = []
index = 0
while index < len(args):
if args[index] == "-c":
sqls.append(args[index + 1])
index += 2
else:
index += 1
if not any("default_transaction_read_only" in item for item in sqls):
print("read-only session setting missing", file=sys.stderr)
sys.exit(1)
sql = sqls[-1]
if os.environ.get("PGDIAG_STUB_FAIL") == "1":
print("injected psql failure", file=sys.stderr)
sys.exit(1)
if "server_version" in sql:
print("mydb|dba|16.4|160004|f|2026-07-01 08:00:00+00")
elif "pg_stat_archiver" in sql:
print("42|0|00000001000000000000002A|2026-07-01 00:00:00+00||")
elif "application_name" in sql:
print("standby-1|streaming|sync|10.0.0.5|0/2A000000|0/2A000000|0/2A000000|0/2A000000")
elif "pg_last_wal_receive_lsn" in sql:
print("f|0/2A000000|0/2A000000")
elif "pg_extension" in sql:
print("plpgsql|1.0")
elif "idx_scan DESC" in sql:
print("public|orders|orders_pkey|12345|24680|12000")
elif "idx_scan = 0" in sql:
print("public|events|events_created_at_idx|0")
elif "pg_index" in sql:
pass
elif "seq_tup_read DESC" in sql:
print("public|orders|5000|1500000|120")
elif "n_dead_tup > 0" in sql:
print("public|orders|100000|5000|4.8")
elif "pg_stat_activity" in sql:
print("active|12")
print("idle|38")
elif "current_setting" in sql:
print("max_connections|100")
print("wal_level|replica")
print("archive_mode|on")
elif "pg_database_size" in sql:
print("mydb|128 MB")
elif "EXPLAIN" in sql:
print('[{"Plan": {"Node Type": "Seq Scan", "Plan Rows": 1000}}]')
elif "current_database()" in sql:
print("mydb|dba|16.4|160004|f|2026-07-01 08:00:00+00")
else:
pass
"""
def write_stub(directory: Path) -> Path:
stub = directory / "fake-psql"
stub.write_text(STUB_TEMPLATE, encoding="utf-8")
stub.chmod(stub.stat().st_mode | stat.S_IXUSR)
return stub
def run_script(*args: str, fail: bool = False) -> subprocess.CompletedProcess:
env = os.environ.copy()
if fail:
env["PGDIAG_STUB_FAIL"] = "1"
with tempfile.TemporaryDirectory() as tmp:
stub = write_stub(Path(tmp))
proc = subprocess.run(
[sys.executable, str(SCRIPT), "--psql", str(stub), *args],
capture_output=True,
text=True,
env=env,
timeout=30,
)
return proc
class HelpTests(unittest.TestCase):
def test_help_exits_zero_without_cluster(self):
proc = subprocess.run(
[sys.executable, str(SCRIPT), "--help"],
capture_output=True,
text=True,
timeout=30,
)
self.assertEqual(proc.returncode, 0)
self.assertIn("--json", proc.stdout)
self.assertIn("read-only", proc.stdout.lower())
self.assertIn("default_transaction_read_only", proc.stdout)
def test_version_flag(self):
proc = subprocess.run(
[sys.executable, str(SCRIPT), "--version"],
capture_output=True,
text=True,
timeout=30,
)
self.assertEqual(proc.returncode, 0)
self.assertIn("pgdiag", proc.stdout)
class JsonRunTests(unittest.TestCase):
def test_full_json_run_is_parseable(self):
proc = run_script("--json")
self.assertEqual(proc.returncode, 0, proc.stderr)
payload = json.loads(proc.stdout)
self.assertTrue(payload["ok"])
self.assertEqual(payload["server"]["version"], "16.4")
self.assertEqual(payload["server"]["database"], "mydb")
names = [check["name"] for check in payload["checks"]]
self.assertIn("identity", names)
self.assertIn("config", names)
self.assertIn("index_usage", names)
self.assertIn("bloat", names)
self.assertIn("wal_archive", names)
self.assertIn("replication", names)
self.assertIn("extensions", names)
self.assertIn("databases", names)
for check in payload["checks"]:
self.assertEqual(check["status"], "ok", check["name"])
def test_text_output_mentions_server_and_checks(self):
proc = run_script()
self.assertEqual(proc.returncode, 0, proc.stderr)
self.assertIn("Server identity and version", proc.stdout)
self.assertIn("WAL archiving health", proc.stdout)
def test_check_subset_selection(self):
proc = run_script("--check", "identity", "--check", "config", "--json")
self.assertEqual(proc.returncode, 0, proc.stderr)
payload = json.loads(proc.stdout)
self.assertEqual([check["name"] for check in payload["checks"]], ["identity", "config"])
def test_unknown_check_is_usage_error(self):
proc = run_script("--check", "bogus", "--json")
self.assertEqual(proc.returncode, 2)
self.assertIn("unknown check", proc.stderr)
def test_psql_error_recorded_per_check_but_run_continues(self):
proc = run_script("--json", fail=True)
self.assertEqual(proc.returncode, 1)
payload = json.loads(proc.stdout)
self.assertFalse(payload["ok"])
names = [check["name"] for check in payload["checks"]]
self.assertEqual(len(names), len(set(names)))
self.assertTrue(all(check["status"] == "error" for check in payload["checks"]))
class BinaryAvailabilityTests(unittest.TestCase):
def test_missing_psql_returns_127_with_json_error(self):
proc = subprocess.run(
[sys.executable, str(SCRIPT), "--psql", "/nonexistent/psql", "--json"],
capture_output=True,
text=True,
timeout=30,
)
self.assertEqual(proc.returncode, 127)
payload = json.loads(proc.stdout)
self.assertFalse(payload["ok"])
self.assertIn("psql binary not found", payload["error"])
class PlanForTests(unittest.TestCase):
def test_plan_for_single_select_included(self):
proc = run_script("--plan-for", "SELECT 1", "--json")
self.assertEqual(proc.returncode, 0, proc.stderr)
payload = json.loads(proc.stdout)
plan_check = payload["checks"][-1]
self.assertEqual(plan_check["name"], "plan")
self.assertEqual(plan_check["status"], "ok")
self.assertIsInstance(plan_check["plan"], list)
def test_plan_for_rejects_write_statement(self):
proc = run_script("--plan-for", "INSERT INTO t VALUES (1)", "--json")
self.assertEqual(proc.returncode, 2)
self.assertIn("read-only statement", proc.stdout)
def test_plan_for_rejects_multiple_statements(self):
proc = run_script("--plan-for", "SELECT 1; SELECT 2", "--json")
self.assertEqual(proc.returncode, 2)
if __name__ == "__main__":
unittest.main()