Evidence-first troubleshooting

Universal loop

State the symptom and boundary: API, object admission, scheduling, startup, readiness, service, external route, storage, node, or control plane.
Capture context, versions, namespace, labels, owner references, conditions, events, and recent changes.
Form competing hypotheses.
Run the least invasive discriminating check.
Change one layer at a time.
Verify both Kubernetes state and the user-visible boundary.
Preserve a bounded report with commands, timestamps, exit codes, and redacted output.

Common symptoms

ImagePullBackOff: inspect Events, image name/tag, registry auth, node egress, architecture, and image policy.
CrashLoopBackOff: inspect previous logs, termination reason, probes, config/Secret mounts, limits, and recent rollout.
Pending: follow scheduling/resource/taint/quota evidence; do not simply increase replicas or delete Pods.
Running but unavailable: inspect readiness, Service selectors, EndpointSlices, DNS, NetworkPolicy, route/controller status, and external load balancer.
Terminating: inspect finalizers, owner/controller health, admission/webhook behavior, and whether the resource is still in use. Removing finalizers is destructive and requires explicit justification.
Upgrade blocked: inspect deprecated API use, PDBs, webhooks, node readiness, quotas, provider/distribution upgrade rules, and version skew.

Output discipline

Use -o json or -o jsonpath for machine parsing. Bound events and logs by namespace, selector, tail lines, and time. Never present an aggregator, cached page, or intermediate tool result as proof that a live endpoint is healthy.

2.0 KiB Raw Blame History

Evidence-first troubleshooting

Universal loop

Common symptoms

Output discipline

Sources

2.0 KiB

Raw Blame History