diff --git a/platform-engineering/README.md b/platform-engineering/README.md index 028ea2c..6319c10 100644 --- a/platform-engineering/README.md +++ b/platform-engineering/README.md @@ -12,6 +12,8 @@ Your agent gets a structured loading order and dense reference patterns for IaC, |-----------|---------| | `SKILL.md` | Core methodology, trigger conditions, reference index | | `references/` | Deep-dive reference files loaded on demand | +| `templates/` | Fillable records: golden-path/self-service portal design, IaC review record, observability contract | +| `evals/` | Output-quality eval manifest for the skill's methodology cases | ## Triggers diff --git a/platform-engineering/SKILL.md b/platform-engineering/SKILL.md index 9225528..05c688a 100644 --- a/platform-engineering/SKILL.md +++ b/platform-engineering/SKILL.md @@ -67,6 +67,14 @@ skill_view('platform-engineering', file_path='references/infrastructure-as-code. | `references/automation-languages.md` | Go CLI patterns, Python SDK integration, Bash bootstrap/conventions for platform tooling | | `references/release-engineering.md` | Container image lifecycle, artifact versioning strategies, release gate checklists, Helm chart promotion | +## Templates + +| Template | When to Use | +|-----------|-------------| +| `templates/golden-path-self-service-portal.md` | Designing a golden path or self-service portal for a developer workflow (scoping, journey, guardrails, escape hatch, metrics) | +| `templates/iac-review-record.md` | Recording a structured review of Terraform/OpenTofu/Pulumi/Ansible modules before they ship | +| `templates/observability-contract.md` | Declaring the metrics/logs/traces contract a service must meet before production traffic | + ## Output Contract The profile using this skill produces artifact pyramids. The response to any caller is the absolute path to `00-index.md`. See `artifact-pyramids` skill for the specification. diff --git a/platform-engineering/evals/evals.json b/platform-engineering/evals/evals.json new file mode 100644 index 0000000..80dd6fd --- /dev/null +++ b/platform-engineering/evals/evals.json @@ -0,0 +1,78 @@ +{ + "schema_version": 1, + "skill_name": "platform-engineering", + "evals": [ + { + "id": "ci-cd-pipeline-design", + "prompt": "We are migrating a monolith to microservices and I need to design the CI/CD pipeline for one of the new services. Requirements: every merge to main must produce a deployable artifact, staging should look like production, and we want to move to GitOps-based promotion instead of pushing from a laptop. What should the pipeline look like and where do the approval gates belong?", + "expected_output": "A CI/CD pipeline design with distinct stages: build and test on every pull request, artifact production and registry push on merge to main, environment promotion via GitOps (Argo CD or Flux) where the cluster reconciles to Git as the source of truth, and approval gates placed at the environment boundary (staging to production) rather than inside build. The design covers OIDC-based cloud authentication instead of static keys, image signing and provenance, semantic versioning of artifacts, and a rollback path that reverts the Git manifest rather than redeploying a build.", + "assertions": [ + "The response defines a multi-stage pipeline where build and test run on pull requests and artifact production runs on merge to main", + "The response uses GitOps promotion with Git as the source of truth for environment state", + "The response places manual or protected approval gates at the staging-to-production boundary, not inside the build stage", + "The response replaces static cloud credentials with OIDC workload identity for pipeline authentication", + "The response covers artifact signing, provenance, and a Git-revert-based rollback path" + ] + }, + { + "id": "iac-review-module-structure", + "prompt": "A new platform team member wrote Terraform modules for our AWS landing zone and I need to review them before they are used across the organization. The modules combine VPC, EKS, and IAM in one root module, store state locally, and hardcode account IDs in the code. What should the review focus on and what structural changes should I request?", + "expected_output": "An infrastructure-as-code review record that flags the root module as too broad and requests composition from smaller, single-purpose modules; requires a remote state backend with locking (e.g., S3 with DynamoDB or Terraform Cloud) instead of local state; removes hardcoded account IDs in favor of variables, data sources, or provider-level configuration; checks for state exposure of secrets and requires secret material to come from a secret manager or dynamic credentials rather than plaintext variables; validates that for_each and module composition are used instead of copy-pasted resource blocks, and that outputs expose the minimal surface area consumers need.", + "assertions": [ + "The review flags monolith root modules and recommends composing smaller single-purpose modules", + "The review requires a remote state backend with locking instead of local state", + "The review flags hardcoded account IDs and requires parameterization via variables or data sources", + "The review addresses secrets in state and requires secret material to come from a secret manager or dynamic credentials", + "The review recommends for_each-driven composition and a minimal output surface area" + ] + }, + { + "id": "observability-strategy-design", + "prompt": "Our new checkout service goes live next month and we currently have no monitoring, no dashboards, and no alerting. I want an observability strategy that tells us when the service is broken and lets us debug latency regressions after release. Where do I start and what should be the default contract for every service we build?", + "expected_output": "An observability strategy organized around the three signals: metrics (RED and USE patterns via Prometheus counters, gauges, histograms), logs (structured JSON shipped to a Loki-style store), and traces (OpenTelemetry spans with W3C trace context propagated across service boundaries). The strategy defines an observability contract for every service: a /metrics endpoint, structured logging, at least one RED dashboard, recording rules for latency and error rate, burn-rate-based alerting tied to an error budget rather than threshold guessing, and a golden signal dashboard as code in Git so dashboards are reviewable and reproducible. It also specifies the cardinality and label hygiene rules that keep the metrics usable at scale.", + "assertions": [ + "The response structures the strategy around metrics, logs, and traces with named tooling for each signal", + "The response defines a default observability contract every service must meet, including a metrics endpoint and structured logging", + "The response uses RED and USE patterns and burn-rate alerting tied to an error budget", + "The response requires dashboards-as-code stored in Git so they are reviewable and reproducible", + "The response covers OpenTelemetry tracing with W3C context propagation and metrics cardinality hygiene" + ] + }, + { + "id": "secret-management-design", + "prompt": "We run Kubernetes with GitOps and currently store database credentials in plaintext Kubernetes Secrets committed to a private Git repository. I want a secret-management design that stops putting credentials in Git, handles rotation, and works for both static config secrets and dynamically generated credentials. Which approach fits and what is the migration path?", + "expected_output": "A secret-management design that separates the problem into static configuration secrets and dynamic credentials. For static secrets in GitOps, the design recommends SOPS-encrypted files with age or KMS keys for small configs, or Sealed Secrets for cluster-bound secrets, with External Secrets Operator to sync from a central store such as HashiCorp Vault or a cloud secret manager. For dynamic credentials (database passwords, cloud access keys), it recommends Vault dynamic secrets with short TTLs and automatic revocation rather than long-lived static secrets. The design covers the migration path: inventory current plaintext secrets, encrypt at rest in Git, rotate existing credentials during the cutover, and wire renewal/rotation into the workload lifecycle. It also calls out that SOPS and Sealed Secrets are static-only and do not provide rotation or audit, which is why a dynamic store is needed for anything that changes.", + "assertions": [ + "The response separates static configuration secrets from dynamically generated credentials and recommends a different mechanism for each", + "The response recommends SOPS or Sealed Secrets for static secrets in Git and External Secrets Operator for syncing from a central store", + "The response recommends Vault dynamic secrets with short TTLs and revocation for database and cloud credentials", + "The response includes a migration path that inventories plaintext secrets and rotates credentials at cutover", + "The response identifies that SOPS and Sealed Secrets lack rotation and audit, motivating a dynamic store" + ] + }, + { + "id": "cloud-architecture-assessment", + "prompt": "Our leadership wants to move the entire platform to a second cloud provider in parallel with AWS to reduce vendor lock-in and cut costs. I have been asked to assess whether this is a good idea before we commit. What factors should the assessment weigh, and what should the recommendation look like?", + "expected_output": "A cloud architecture assessment that evaluates the multi-cloud proposal against decision criteria rather than assuming multi-cloud is inherently beneficial. It weighs regulatory data-residency requirements, the small set of services where lock-in actually matters (object storage, Kubernetes), operational cost of duplicated IAM, networking, and skills across providers, and calls out that multi-cloud is not a cost-savings strategy because egress charges and duplicated operational overhead usually outweigh rate differences. The assessment recommends single-cloud for the default path with provider-agnostic abstractions (Terraform/OpenTofu providers, Kubernetes) used as escape hatches rather than abstraction layers for everything, and it covers cost governance: tagging, budget alerts, right-sizing, and committed-use discounts applied to the primary provider before adding a second one.", + "assertions": [ + "The response evaluates multi-cloud against decision criteria including data residency and lock-in, not as a blanket strategy", + "The response states that multi-cloud is not a cost-savings strategy and explains egress and duplicated-operations costs", + "The response limits provider-agnostic abstraction to escape-hatch patterns such as Terraform providers and Kubernetes", + "The response recommends applying cost governance (tagging, budgets, right-sizing, committed use) before expanding providers", + "The response treats single-cloud as the default and multi-cloud as justified only by specific regulatory or availability requirements" + ] + }, + { + "id": "golden-path-self-service-portal", + "prompt": "Our developers keep opening tickets to get a database, a namespace, and a CI pipeline for each new service, and the platform team is the bottleneck. I want to design a golden path with a self-service portal so developers can provision their own stack. How do I scope the first golden path and what guardrails should the portal enforce?", + "expected_output": "A golden-path and self-service portal design that treats the platform as a product with developers as customers. The first golden path is scoped to the most common request (a new service: repository, CI pipeline, namespace, database, observability defaults) and implemented as a repeatable template with Terraform modules, a pipeline template, and an API-first portal that calls those templates behind the scenes. The design enforces guardrails as policy rather than documentation: least-privilege IAM generated from the request, budget and quota limits, mandatory observability and security baselines, and an escape hatch that lets developers leave the golden path with an exception review instead of forking it. It prioritizes self-service over tickets, measures cognitive load and time-to-first-deploy as the primary success metrics, and keeps every scaffolded artifact in Git for review.", + "assertions": [ + "The response scopes the first golden path to a single high-frequency request and implements it as a repeatable template", + "The response uses an API-first self-service portal that provisions through templates rather than tickets", + "The response enforces guardrails as policy, including least-privilege IAM, budget limits, and observability baselines", + "The response includes an escape hatch with exception review so the golden path is a paved road, not a cage", + "The response measures success by developer cognitive load and time-to-first-deploy and keeps scaffolded artifacts in Git" + ] + } + ] +} diff --git a/platform-engineering/references/cloud-platforms.md b/platform-engineering/references/cloud-platforms.md index 0f81bb3..5987b08 100644 --- a/platform-engineering/references/cloud-platforms.md +++ b/platform-engineering/references/cloud-platforms.md @@ -1,23 +1,57 @@ # Cloud Platforms — Reference -## AWS +> **Last Updated:** 2026-08-03 +> Patterns and decision guidance for cloud platform architecture. Operational configuration belongs to the tool skills (`terraform`, `kubernetes`, `docker-compose`, `traefik`); this file carries judgment frameworks. + +## Provider Selection — Decision Guidance + +### AWS - **Core services:** VPC (subnets, route tables, NAT, security groups, NACLs, VPC peering, Transit Gateway), EC2 (instances, AMIs, auto-scaling, launch templates, spot), EKS (managed K8s, node groups, Fargate, IRSA), S3 (buckets, versioning, lifecycle, replication, presigned URLs), IAM (users, roles, policies, instance profiles, OIDC), Route53 (DNS, alias records, health checks, routing policies) - **Common patterns:** Shared VPC (central networking team), multi-account (Control Tower, Organization, SCPs), IRSA for EKS pod IAM, S3 backend for Terraform state (bucket + DynamoDB lock), CodeBuild/CodePipeline for CI, CloudFront for CDN +- **When AWS fits:** Broadest service catalog, deepest managed-K8s and IAM maturity, most mature IaC ecosystem and third-party tooling. Strong default when the team already has AWS skills or needs services no other provider matches. -## GCP +### GCP - **Core services:** VPC (subnets, firewall rules, Cloud NAT, VPC peering, Shared VPC), GKE (K8s, node auto-repair/auto-upgrade, Workload Identity for pod IAM), Cloud Storage (buckets, nearline/archive, object lifecycle), IAM (roles, custom roles, service accounts, Workload Identity Federation), Cloud DNS (managed zones, DNS forwarding, policy-based routing) - **Common patterns:** Shared VPC (host project + service projects), workload identity federation (no static keys), Artifact Registry, Cloud Build CI, Terraform state via Cloud Storage +- **When GCP fits:** Kubernetes-first workloads (GKE is the closest managed-K8s experience), data/ML platform strengths, most aggressive committed-use discounts, clean identity-federation story for keyless workloads. -## Azure +### Azure - **Core services:** VNet (subnets, NSGs, Azure Bastion, VPN Gateway, VNet peering), AKS (K8s, node pools, managed identity, Azure AD integration), Blob Storage (containers, tiers, lifecycle, Azure Files), RBAC (roles, custom roles, managed identities, service principals), DNS (public/private zones, alias records, Azure DNS Private Resolver) -- **Common patterns:** Hub-and-spoke networking (central firewall), managed identity for pod IAM (AKS with aad-pod-identity), Terraform state via Azure Storage, Azure DevOps pipelines +- **Common patterns:** Hub-and-spoke networking (central firewall), managed identity for pod IAM (AKS with workload identity), Terraform state via Azure Storage, Azure DevOps pipelines +- **When Azure fits:** Windows/.NET/Active Directory shops, enterprise compliance and procurement (existing Microsoft agreements), hybrid on-prem connectivity, regulated industries where Azure's compliance footprint is a sales advantage. ## Multi-Cloud and Abstraction - **Abstraction layers:** Terraform/OpenTofu providers — write once, target any cloud (with provider-specific variance). Pulumi similarly abstracts. Crossplane for K8s-native cloud resource provisioning - **Governance cost:** State isolation per cloud, IAM duplication per provider, network egress charges (Free Tier per cloud but real cost at scale), skills distribution across cloud teams - **When multi-cloud is worth it:** Regulatory (data residency), avoiding single-vendor lock-in for critical few services (object storage, K8s), acquisition integration. It is NOT a cost-savings strategy. -- **Cost governance:** Budget alerts (each cloud), tagging policies (`CostCenter`, `Environment`, `Owner`), right-sizing, reserved instances/committed use discounts, spot/preemptible for batch, storage tier policies +- **When multi-cloud is a trap:** Teams assume abstraction layers erase provider differences; they do not. Each provider's IAM model, quota semantics, and operational behavior leak through. A second provider doubles the platform surface for zero resilience unless workloads are actually replicated (active-active or active-passive with real failover testing). +- **Decision rule:** Start single-cloud. Add a second provider only for a named, measurable requirement (residency, availability, acquisition). If the goal is resilience, prove failover works before committing to the second provider. + +## Cost Governance Patterns + +- **Budget alerts** (each cloud): per-account/project budget with alert thresholds at 50/80/100%, billing exports to a data warehouse for cost analytics +- **Tagging policies:** `CostCenter`, `Environment`, `Owner`, `Service` — enforced at provisioning time (guardrails/Terraform validators), not retroactively +- **Right-sizing:** instance/container resource analysis against utilization, right-size before scaling out +- **Committed use:** reserved instances / committed use discounts / savings plans for steady-state baseline; spot/preemptible for batch and stateless workloads +- **Storage tier policies:** lifecycle rules moving cold data to archive tiers; know the retrieval cost before designing hot paths +- **Egress awareness:** egress charges dominate surprise bills; keep data transfer within a region/zone where possible, and route cross-provider traffic deliberately +- **FinOps cadence:** monthly cost review with owners, anomaly detection on the billing feed, unit-economics per service (see `capacity-and-cost-engineering` for the methodology) + +## Security and Identity Patterns + +- **Workload identity over static keys:** OIDC federation (IRSA, Workload Identity Federation, managed identity) so pods and CI never hold long-lived cloud keys +- **Multi-account/project structure as the security boundary:** control plane (org/root) separate from workload accounts, SCPs as policy guardrails, audit account for centralized logs +- **Shared responsibility model:** the provider secures the fabric; the platform team owns IAM, network boundaries, data encryption at rest/in transit, and image/artifact supply chain +- **Audit logging:** enable cloud trail/audit logs centrally with retention and alerting on privileged-role usage + +## Sources and Dated References + +- AWS Well-Architected Framework: https://aws.amazon.com/architecture/well-architected/ (accessed 2026-08-03) +- AWS Organizations multi-account best practices: https://docs.aws.amazon.com/whitepapers/latest/organizing-your-aws-environment/organizing-your-aws-environment.html (accessed 2026-08-03) +- GCP resource hierarchy and IAM: https://cloud.google.com/docs/overview (accessed 2026-08-03) +- Azure cloud adoption framework / landing zones: https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/ (accessed 2026-08-03) +- FinOps framework and cost optimization: https://www.finops.org/framework/ (accessed 2026-08-03) diff --git a/platform-engineering/references/infrastructure-as-code.md b/platform-engineering/references/infrastructure-as-code.md index 1f61897..6be1c1a 100644 --- a/platform-engineering/references/infrastructure-as-code.md +++ b/platform-engineering/references/infrastructure-as-code.md @@ -1,30 +1,71 @@ # Infrastructure as Code — Reference -## Terraform / OpenTofu +> **Last Updated:** 2026-08-03 +> Patterns and decision guidance for infrastructure-as-code. Operational commands and runbooks belong to the tool skills (`terraform`, `kubernetes`, `docker-compose`); this file carries the judgment frameworks for choosing, structuring, and reviewing IaC. + +## Tool Selection — Decision Guidance + +### Terraform / OpenTofu - **Core concepts:** Resources, data sources, providers, state (local, remote backends), modules, variables, outputs, lifecycle rules (`create_before_destroy`, `prevent_destroy`) - **State management:** Remote backends (S3 + DynamoDB, GCS, Azure Storage, Terraform Cloud), state locking, state migration, workspaces for env separation, `terraform state` subcommands (mv, rm, pull, push) - **Module design:** Composition (call smaller modules), version pinning, registry conventions (hashicorp/terraform-google-modules), output minimal surface area, internal vs published modules - **Advanced patterns:** `for_each`/`count` for dynamic resources, `templatefile` for config injection, file/external data sources for bridge to external systems, provisioners as last resort (remote-exec/local-exec) - **OpenTofu specifics:** Drop-in Terraform replacement, same HCL syntax, OSS license (no BSL change), `tofu` CLI, supports encryption at rest in state natively, enhanced provider signing +- **When to choose:** The default for cloud resource provisioning. Largest provider ecosystem, most transferable skills, works across all three major clouds. Choose OpenTofu when license/BSL or state encryption is a hard requirement. -## Pulumi +### Pulumi - **Core model:** Infrastructure as real code — Go, Python, TypeScript, .NET, Java, YAML - **Key concepts:** Programs (stack definitions), stacks (env instances), resources, components (custom abstractions), providers (Pulumi-native, TF bridge), outputs, config/secret management - **State:** Pulumi Cloud (managed), self-managed backends (S3, GCS, Azure Blob S3-compatible), state encryption - **Automation API:** Embed Pulumi in applications (CI/CD, self-service platforms), inline updates, preview + deploy in code - **Bridge to Terraform:** TF bridge adapter wraps existing TF providers as native Pulumi providers — convenient but adds a layer +- **When to choose:** Teams that need real programming-language logic (loops, conditionals, tests) inside the IaC layer, or who are building self-service automation and want the Automation API. -## Ansible +### Ansible - **Core model:** Agentless — SSH/WinRM transport, push-based, YAML playbooks, Jinja2 templating - **Key concepts:** Inventory (static, dynamic from cloud APIs), modules (idempotent operations), roles (reusable content packages), playbooks (execution order), variables and facts, handlers (notify-based triggers) - **Best practices:** Role-based layout, vault for secrets, molecule for testing, ansible-lint, `--check --diff` for dry-run, `--limit` for targeted execution - **Use case in platform engineering:** Day-2 configuration (post-provisioning), OS hardening, agent installation, but generally less suited than Terraform for cloud resource provisioning +- **When to choose:** Configuration management of existing servers and day-2 operations; not the right tool for the initial cloud resource graph. -## CloudFormation / CDK +### CloudFormation / CDK - **CloudFormation:** Native AWS IaC — JSON/YAML templates, stacks, nested stacks, change sets, drift detection, stack sets (multi-account, multi-region) - **CDK (Cloud Development Kit):** CloudFormation as real code (TypeScript, Python, Go, Java, C#) — constructs (L1/L2/L3 abstraction), `cdk synth` → CloudFormation template, `cdk deploy` / `cdk diff`, context, aspects, permissions boundaries - **CDKTF (CDK for Terraform):** Bridge for Terraform providers in CDK languages — cross-platform between AWS and non-AWS providers +- **When to choose:** AWS-only shops that want native drift detection and change-set review, or teams already writing real code who prefer CDK's type safety over HCL. + +## State and Drift Governance + +- **State is a source of truth, not a database:** treat state as a serialized representation of the resource graph, never edit it directly; all changes go through `plan`/`apply` (or the equivalent) +- **Remote backend with locking is non-negotiable:** local state is a team-of-one anti-pattern; choose S3+DynamoDB, GCS, Azure Storage, or Terraform Cloud/Pulumi Cloud and make locking explicit +- **Drift detection cadence:** run periodic plans (`plan` on a schedule, `drift detect` in Terraform Cloud, or cloud-native drift tools) and review unintended diffs before they become incidents +- **Workspaces vs directories:** prefer directory-per-environment with shared modules over workspaces when environments differ materially; use workspaces only for near-identical instances +- **Import-before-manage:** adopt pre-existing resources with `terraform import`/state moves rather than `delete + recreate`; plan for state surgery (`state mv`, `state rm`) only with a locked state and a reviewed plan + +## Secrets in IaC + +- **Never commit secrets in plaintext:** secrets in HCL/JSON/YAML drift into state and logs; use provider-native secret references (`data "aws_secretsmanager_secret_version"`), Vault dynamic credentials, or SOPS/age for encrypted-at-rest config +- **Prefer dynamic credentials:** database and cloud keys should come from Vault dynamic secrets or managed identity (IRSA, Workload Identity) rather than static long-lived keys +- **State encryption:** OpenTofu encrypts state natively; on Terraform, encrypt the state backend at rest and restrict state read access (state holds secrets) +- **Mark sensitive outputs:** `sensitive = true` so values are redacted in logs and plan output; keep the full secret in the manager, only a reference in IaC + +## Review Checklist Patterns + +- **Composition over monoliths:** root modules should call child modules; a module that owns VPC + cluster + IAM + app is a candidate for splitting +- **Parameterize environment specifics:** no hardcoded account IDs, regions, or names; variables + data sources + consistent naming convention +- **Minimal outputs:** expose only what consumers need; every output is API surface +- **Plan review before apply:** review the plan for unintended replaces/deletes, not just additions; enforce a human gate on destructive changes +- **Tagging and cost attribution:** consistent tags (`CostCenter`, `Environment`, `Owner`, `Service`) enforced at plan time by guardrails/validators +- **Lifecycle rules:** `prevent_destroy` on irreplaceable resources (databases, state backends); `create_before_destroy` where downtime matters + +## Sources and Dated References + +- OpenTofu documentation and state encryption: https://opentofu.org/docs/ (accessed 2026-08-03) +- Terraform module composition best practices: https://developer.hashicorp.com/terraform/tutorials/modules (accessed 2026-08-03) +- Pulumi documentation (stacks, state, Automation API): https://www.pulumi.com/docs/ (accessed 2026-08-03) +- Ansible best practices (roles, vault, molecule): https://docs.ansible.com/ansible/latest/tips_tricks/sample_setup.html (accessed 2026-08-03) +- AWS CDK reference: https://docs.aws.amazon.com/cdk/v2/guide/home.html (accessed 2026-08-03) diff --git a/platform-engineering/templates/golden-path-self-service-portal.md b/platform-engineering/templates/golden-path-self-service-portal.md new file mode 100644 index 0000000..46ae674 --- /dev/null +++ b/platform-engineering/templates/golden-path-self-service-portal.md @@ -0,0 +1,92 @@ +--- +title: "Golden Path / Self-Service Portal: [Capability Name]" +doc_id: GP-[CAPABILITY-CODE]-[VERSION] +status: draft | proposed | approved | superseded +created: [YYYY-MM-DD] +last_modified: [YYYY-MM-DD] +owner: "[Platform Team / Individual]" +approver: "[Platform Lead]" +--- + +# Golden Path / Self-Service Portal — [Capability Name] + +## 1. Purpose and Scope + +| Field | Value | +|---|---| +| **Capability** | [What developers can obtain, e.g., a new service with CI pipeline, namespace, and database] | +| **Developer need** | [The workflow this removes from a ticket queue, e.g., provision a Postgres database for a new microservice] | +| **In scope** | [List what the portal provisions automatically] | +| **Out of scope** | [List what still requires a ticket or manual review, e.g., production firewall changes] | +| **Request frequency** | [e.g., 12 requests/week — evidence that this is the highest-friction path] | +| **Current cycle time** | [e.g., 3 days from ticket to working environment] | + +## 2. Developer Journey + +| Step | Actor | Action | System Response | Time | +|---|---|---|---|---| +| 1 | [Developer] | [Submit request with service name, team, environment] | [Validate naming and quota] | _[fill: seconds]_ | +| 2 | [System] | [Run scaffold from template] | [Create repo, pipeline, namespace, DB via IaC] | _[fill: minutes]_ | +| 3 | [Developer] | [Approve generated PR] | [Apply to Git, reconcile via GitOps] | _[fill: minutes]_ | +| 4 | [Developer] | [First deploy] | [Verify observability baseline is live] | _[fill: minutes]_ | + +- **Time-to-first-deploy target:** _[fill: e.g., under 30 minutes from request]_ +- **Cognitive load target:** _[fill: e.g., no more than N decisions required from the developer]_ + +## 3. Template Design + +### 3.1 Provisioning Template + +- **IaC module used:** _[fill: e.g., terraform module for service scaffolding, version pinned]_ +- **Resources created:** _[fill: repository, CI workflow, namespace, database, secrets placeholder, dashboards]_ +- **Input parameters:** _[fill: name, team, environment, size limits — every input validated]_ +- **Default values:** _[fill: what the template assumes when the developer leaves a field blank]_ + +### 3.2 Pipeline Template + +- **Stages:** _[fill: build, test, artifact, deploy — mirror the platform CI/CD reference]_ +- **Gates:** _[fill: where approvals sit and who can override]_ +- **Artifact handling:** _[fill: registry, signing, provenance, versioning scheme]_ + +## 4. Guardrails and Policies + +| Guardrail | Enforcement Mechanism | Escalation / Override | +|---|---|---| +| Least-privilege IAM | _[fill: generated from request scope, not admin defaults]_ | _[fill: role/person with authority]_ | +| Budget and quota limits | _[fill: tag-based budget alert, quota per namespace]_ | _[fill: cost owner approval]_ | +| Observability baseline | _[fill: mandatory dashboard + alert rules on scaffold]_ | _[fill: SRE review]_ | +| Security baseline | _[fill: secret scanning, image scanning, network policy default deny]_ | _[fill: security review]_ | +| Naming and ownership | _[fill: validated naming convention, required owner field]_ | _[fill: platform team]_ | + +- **Policy-as-code location:** _[fill: where policies live in Git, e.g., OPA/kyverno rules, Terraform guardrail module]_ + +## 5. Escape Hatch + +- **Escape hatch path:** _[fill: what a developer does when the golden path does not fit — e.g., exception request, custom module review]_ +- **Exception review criteria:** _[fill: what justifies leaving the paved road and who reviews]_ +- **Bounded by:** _[fill: golden paths are paved roads, not cages — the exception keeps the platform from blocking delivery]_ + +## 6. API-First Design + +- **Portal entry points:** _[fill: CLI command, web UI, API endpoint — each invokes the same scaffold service]_ +- **Request/response contract:** _[fill: schema of the request and the status response]_ +- **Audit trail:** _[fill: every provisioned change is a Git commit/PR with actor and timestamp]_ +- **Idempotency:** _[fill: what happens when the same request is submitted twice]_ + +## 7. Success Metrics + +| Metric | Target | Measurement Source | +|---|---|---| +| Time-to-first-deploy | _[fill: target]_ | _[fill: portal telemetry]_ | +| Ticket volume for this capability | _[fill: target decrease]_ | _[fill: ticketing system]_ | +| Developer satisfaction / cognitive load | _[fill: survey score]_ | _[fill: survey]_ | +| Guardrail violations | _[fill: target]_ | _[fill: policy engine logs]_ | + +## 8. Version History + +| Version | Date | Author | Changes | +|---|---|---|---| +| 1.0 | [YYYY-MM-DD] | [Author] | Initial golden path design | +| 1.1 | [YYYY-MM-DD] | [Author] | [Summary of changes] | + +*Keep this record in Git next to the portal implementation so the design and the code stay in sync.* diff --git a/platform-engineering/templates/iac-review-record.md b/platform-engineering/templates/iac-review-record.md new file mode 100644 index 0000000..e94eaa5 --- /dev/null +++ b/platform-engineering/templates/iac-review-record.md @@ -0,0 +1,87 @@ +--- +title: "Infrastructure as Code Review Record: [Module / Project Name]" +doc_id: IACR-[MODULE-CODE]-[VERSION] +status: draft | in-review | approved | changes-requested +created: [YYYY-MM-DD] +last_modified: [YYYY-MM-DD] +reviewer: "[Reviewer Name]" +author: "[Module Author Name]" +--- + +# Infrastructure as Code Review Record — [Module / Project Name] + +## 1. Review Metadata + +| Field | Value | +|---|---| +| **Module / project** | [Name and path in Git] | +| **IaC tooling** | [e.g., Terraform, OpenTofu, Pulumi, Ansible, CloudFormation] | +| **Provider(s)** | [e.g., AWS, GCP, Azure, on-prem] | +| **Review scope** | [Full module / resource block / state change] | +| **Plan applied?** | [Yes/No — if yes, plan ID and date] | +| **Drift baseline** | [State of the environment before the change] | + +## 2. Module Structure + +| Check | Verdict | Notes | +|---|---|---| +| Single-purpose composition (no monolith root) | _[fill: pass / fail / n/a]_ | _[fill: what should be split into child modules]_ | +| Variables and defaults parameterize env specifics | _[fill: pass / fail / n/a]_ | _[fill: hardcoded IDs, account numbers, regions]_ | +| `for_each`/`count` used instead of copy-pasted blocks | _[fill: pass / fail / n/a]_ | _[fill: specific resources to convert]_ | +| Minimal output surface area | _[fill: pass / fail / n/a]_ | _[fill: outputs consumers actually need]_ | +| Version pinning of modules and providers | _[fill: pass / fail / n/a]_ | _[fill: constraints and locked versions]_ | + +## 3. State and Drift + +| Check | Verdict | Notes | +|---|---|---| +| Remote backend with locking configured | _[fill: pass / fail / n/a]_ | _[fill: backend type and lock mechanism]_ | +| No secrets material in state | _[fill: pass / fail / n/a]_ | _[fill: which attributes are sensitive and how they are handled]_ | +| Workspaces/environments isolated | _[fill: pass / fail / n/a]_ | _[fill: env separation approach]_ | +| Drift detection cadence defined | _[fill: pass / fail / n/a]_ | _[fill: scheduled plan or drift tooling]_ | +| State operations documented (`state mv`, `rm`, imports) | _[fill: pass / fail / n/a]_ | _[fill: any state surgery required]_ | + +## 4. Security and Secrets + +| Check | Verdict | Notes | +|---|---|---| +| Secrets come from a secret manager, not plaintext vars | _[fill: pass / fail / n/a]_ | _[fill: Vault / SOPS / cloud secret store reference]_ | +| Least-privilege IAM on created resources | _[fill: pass / fail / n/a]_ | _[fill: overly broad policies to tighten]_ | +| Network boundaries default to deny | _[fill: pass / fail / n/a]_ | _[fill: security groups, firewalls, network policies]_ | +| Sensitive outputs marked `sensitive = true` | _[fill: pass / fail / n/a]_ | _[fill: which outputs]_ | +| Resource naming and tagging consistent | _[fill: pass / fail / n/a]_ | _[fill: tag keys, cost center, owner, environment]_ | + +## 5. Operational Readiness + +| Check | Verdict | Notes | +|---|---|---| +| `plan` output reviewed for unintended changes | _[fill: pass / fail / n/a]_ | _[fill: resources that will be replaced vs updated]_ | +| `prevent_destroy` on irreplaceable resources | _[fill: pass / fail / n/a]_ | _[fill: database, state bucket, registry]_ | +| Lifecycle rules match intent (`create_before_destroy`) | _[fill: pass / fail / n/a]_ | _[fill: where ordering matters]_ | +| Rollback path defined | _[fill: pass / fail / n/a]_ | _[fill: revert commit, previous state, or forward fix]_ | + +## 6. Findings + +### Blocking Findings + +| # | Severity | Finding | Location | Suggested Fix | Owner | Fixed? | +|---|---|---|---|---|---|---| +| 1 | [critical/high] | _[fill: what is wrong and why it blocks]_ | _[fill: file:line]_ | _[fill: concrete change]_ | _[fill: name]_ | _[fill: yes/no]_ | + +### Non-Blocking Findings + +| # | Severity | Finding | Location | Suggested Fix | Owner | Fixed? | +|---|---|---|---|---|---|---| +| 1 | [low/medium] | _[fill: what is suboptimal]_ | _[fill: file:line]_ | _[fill: concrete change]_ | _[fill: name]_ | _[fill: yes/no]_ | + +## 7. Verdict + +| Field | Value | +|---|---| +| **Verdict** | [approved / changes-requested] | +| **Blocking findings resolved** | [all / list of remaining] | +| **Re-review required** | [yes/no — and by when] | +| **Reviewer sign-off** | [Name, date] | +| **Author sign-off** | [Name, date] | + +*File this record alongside the module and the applied plan output so the review is auditable.* diff --git a/platform-engineering/templates/observability-contract.md b/platform-engineering/templates/observability-contract.md new file mode 100644 index 0000000..37c6fb5 --- /dev/null +++ b/platform-engineering/templates/observability-contract.md @@ -0,0 +1,98 @@ +--- +title: "Observability Contract: [Service Name]" +doc_id: OBC-[SERVICE-CODE]-[VERSION] +status: draft | reviewed | approved | superseded +created: [YYYY-MM-DD] +last_modified: [YYYY-MM-DD] +owner: "[Service Owner / Team]" +approver: "[SRE / Platform Lead]" +--- + +# Observability Contract — [Service Name] + +## 1. Service Context + +| Field | Value | +|---|---| +| **Service Name** | [Service Name] | +| **Owner** | [Team / Individual] | +| **Environments** | [dev, staging, prod] | +| **Dependencies** | [Upstream/downstream services, data stores] | +| **SLO reference** | [Link to SLO declaration or error budget policy] | + +## 2. Signals Required + +Every service must emit all three signals before production traffic is accepted. + +### 2.1 Metrics + +| Metric | Type | Name | Definition | +|---|---|---|---| +| Request rate | Counter | _[fill: e.g., svc_http_requests_total]_ | _[fill: label set, status split]_ | +| Error rate | Counter | _[fill: e.g., svc_http_errors_total]_ | _[fill: which statuses count as errors]_ | +| Latency | Histogram | _[fill: e.g., svc_http_request_duration_seconds]_ | _[fill: buckets, percentiles consumed]_ | +| Saturation | Gauge | _[fill: e.g., svc_queue_depth]_ | _[fill: what resource is near exhaustion]_ | + +- **Scrape endpoint:** _[fill: e.g., /metrics on :9090]_ — must be reachable by the platform scraper. + +### 2.2 Logs + +- **Format:** _[fill: structured JSON with timestamp, level, service, trace_id, span_id]_ +- **Shipping:** _[fill: agent/target — e.g., Promtail/Alloy/Fluent Bit]_ +- **Retention requirement:** _[fill: hot/warm/cold tiers and durations]_ +- **Sensitive data:** _[fill: what must never be logged — tokens, PII, full payloads]_ + +### 2.3 Traces + +- **Instrumentation:** _[fill: OpenTelemetry SDK, auto-instrumentation, or manual spans]_ +- **Context propagation:** _[fill: W3C TraceContext across all outbound calls]_ +- **Sampling:** _[fill: head/tail sampling strategy and rate]_ +- **Key spans:** _[fill: entry, external calls, DB queries, background jobs]_ + +## 3. Dashboards and Recording Rules + +| Artifact | Name / Path in Git | Content | +|---|---|---| +| Service dashboard | _[fill: provisioning path]_ | _[fill: RED panels, per row: traffic, errors, latency, saturation]_ | +| Recording rules | _[fill: rules file path]_ | _[fill: rate/error-duration derivations, error budget expressions]_ | +| Dashboard links | _[fill: links to related platform dashboards]_ | _[fill: cross-service dependency view]_ | + +- **Dashboard-as-code requirement:** _[fill: dashboards live in Git and change via review, not ad-hoc UI edits]_ + +## 4. Alerting and Error Budgets + +| Alert | Condition (query) | Severity | Routing | Action | +|---|---|---|---|---| +| _[fill: High error rate]_ | _[fill: PromQL expression]_ | _[fill: critical/warning]_ | _[fill: page/Slack]_ | _[fill: incident response, freeze, rollback]_ | +| _[fill: Latency p99 breach]_ | _[fill: PromQL expression]_ | _[fill: severity]_ | _[fill: routing]_ | _[fill: action]_ | +| _[fill: Budget burn rate]_ | _[fill: multi-window burn rate expression]_ | _[fill: severity]_ | _[fill: routing]_ | _[fill: action]_ | + +- **Error budget policy applied:** _[fill: link or reference to the team error budget policy]_ +- **Noise control:** _[fill: `for:` durations, deduplication, silenced maintenance windows]_ + +## 5. Release and Verification Gate + +| Gate | Requirement | +|---|---| +| Pre-release | _[fill: dashboards live, alerts firing correctly, metrics scraping, traces flowing]_ | +| Canary verification | _[fill: what SLIs are compared between canary and control and at what divergence]_ | +| Post-release | _[fill: regression check against baseline within N minutes, on-call notified]_ | + +- **Verification evidence:** _[fill: where the evidence (dashboards, alert receipts, trace samples) is recorded]_ + +## 6. Ownership and Review + +| Item | Value | +|---|---| +| **Observability owner** | [Team / Individual] | +| **Review cadence** | [Quarterly or on architecture change] | +| **Next review date** | [YYYY-MM-DD] | + +### Sign-off + +| Role | Name | Date | +|---|---|---| +| Service Owner | [Name] | [YYYY-MM-DD] | +| SRE / Platform Lead | [Name] | [YYYY-MM-DD] | + +*This contract is part of the service's production readiness review and lives in Git next to the dashboards and rules it describes.*