diff --git a/github-runner/SKILL.md b/github-runner/SKILL.md index 23bdb1f..ff10da7 100644 --- a/github-runner/SKILL.md +++ b/github-runner/SKILL.md @@ -1,6 +1,6 @@ --- name: github-runner -version: 1.0.1 +version: 1.0.2 description: >- Deploy, manage, and troubleshoot self-hosted GitHub Actions runners. Covers systemd service, Docker containers, Kubernetes (Actions Runner Controller), diff --git a/github-runner/references/deployment.md b/github-runner/references/deployment.md index 82106c4..5ad6db6 100644 --- a/github-runner/references/deployment.md +++ b/github-runner/references/deployment.md @@ -128,6 +128,21 @@ Standalone Go module for building custom autoscaling outside Kubernetes. Handles **Repository:** `actions/scaleset` on GitHub +## Decision Matrix: Which Approach to Choose + +| Scenario | Recommended | Why | +|----------|-------------|-----| +| Single machine, simple CI, <10 runs/day | systemd | Minimal dependencies, full filesystem access, no Docker needed | +| Small team (3-10), multiple servers, ~10-100 runs/day | Docker | Container isolation, easy management, restart-on-failure, DinD for builds | +| 10+ devs, growing CI volume, existing K8s cluster | ARC | Production autoscaling, K8s-native, JIT tokens, clean per-job isolation | +| Platform team, non-K8s infra, custom provisioning | Scale Set Client | Full control over provisioning, VM/hybrid/multi-platform, customized scaling logic | + +**Decision flow:** +1. Do you have a Kubernetes cluster you already maintain? → **ARC** +2. No K8s but need isolation and container management? → **Docker** +3. Only 1 machine, low volume, no containers needed? → **systemd** +4. Platform team with custom infrastructure and non-K8s scale requirements? → **Scale Set Client** + ## Deployment Comparison | Approach | Complexity | Autoscaling | Security | Best For | diff --git a/github-runner/references/management.md b/github-runner/references/management.md index b8ebd25..9a06fa6 100644 --- a/github-runner/references/management.md +++ b/github-runner/references/management.md @@ -80,6 +80,37 @@ gh api orgs//actions/runner-groups --jq '.runner_groups[].name' ``` Tests each required endpoint (github.com, api.github.com, *.actions.githubusercontent.com, etc.) and outputs PASS/FAIL per endpoint. Logs in `_diag/`. +## Runner Offline Diagnostic Procedure + +When a runner shows "offline" in GitHub UI, follow these steps in order: + +**1. Check container/service logs** +```bash +docker logs --tail 50 # Docker runner +sudo journalctl -u actions.runner.* -f # systemd runner +``` +Look for: connection failures, auth errors, or the runner starting then immediately losing connection. + +**2. Test network connectivity from the runner** +```bash +./config.sh --check --url --pat +``` +This tests all required endpoints (github.com, api.github.com, *.actions.githubusercontent.com, etc.) and reports PASS/FAIL per endpoint. Logs in `_diag/`. + +**3. Verify runner registration with gh CLI** +```bash +gh api repos///actions/runners --jq '.runners[] | "\(.name) (\(.status))"' +``` +If the runner name doesn't appear, it was auto-removed (offline >14 days). With ACCESS_TOKEN set, the container should re-register on restart. + +**4. Check IP allow lists** +If your org uses IP allow lists, the runner's outbound IP must be added. Find it via: +```bash +curl -s ifconfig.me +``` + +**Most likely cause for "worked yesterday, offline today":** Network/firewall change or expired ACCESS_TOKEN. + ## Troubleshooting | Issue | Likely Cause | Fix | @@ -116,6 +147,38 @@ Tests each required endpoint (github.com, api.github.com, *.actions.githubuserco Check latest release: https://github.com/actions/runner/releases +## Runner Lifecycle Hooks + +Runner lifecycle hooks let you run scripts before and after each job using environment variables. + +| Hook | When it runs | Use cases | +|------|-------------|----------| +| `ACTIONS_RUNNER_HOOK_JOB_STARTED` | After job assignment, before any workflow steps | Workspace cleanup, secret injection, telemetry start | +| `ACTIONS_RUNNER_HOOK_JOB_COMPLETED` | After all workflow steps, before job completion | Teardown, log shipping, metric collection | + +**Setup:** +1. Create a script (`.sh` or `.ps1`) anywhere on the runner machine +2. Make it executable: `chmod +x /path/to/script.sh` +3. Set the env var in the runner's `.env` file, systemd unit, or Docker environment + +**Docker example:** +```yaml +environment: + - ACTIONS_RUNNER_HOOK_JOB_STARTED=/opt/runner/workspace-cleanup.sh +``` + +**Cleanup script example:** +```bash +#!/bin/bash +# Cleans workspace before each job +WORKSPACE="${RUNNER_WORKSPACE:-${GITHUB_WORKSPACE:-/home/runner/_work}}" +rm -rf "${WORKSPACE:?}"/* || true +``` + +**Testing:** Run a job and check workflow logs for a "Set up runner" section containing "Hook output:". Non-zero exit from JOB_STARTED prevents the job from running. + +**Note:** JOB_COMPLETED runs synchronously during job cleanup — it cannot be used to destroy the runner itself. Use `--ephemeral` for that. + ## Common Pitfalls 1. **ACCESS_TOKEN vs RUNNER_TOKEN** — most common failure mode diff --git a/github-runner/references/network.md b/github-runner/references/network.md index b887223..1654202 100644 --- a/github-runner/references/network.md +++ b/github-runner/references/network.md @@ -59,6 +59,14 @@ github-cloud.s3.amazonaws.com dependabot-actions.githubapp.com ``` +**Runner group access for Dependabot:** If Dependabot PRs aren't triggering CI on self-hosted runners, check two settings: +1. **Org/repo setting:** Settings → Actions → General → "Allow Dependabot to use self-hosted runners" +2. **Runner group API:** If using groups, enable via: + ```bash + gh api --method PATCH orgs//actions/runner-groups/ \ + -f allows_dependabot=true + ``` + ### Release Assets ``` release-assets.githubusercontent.com