From dfd7f9636d17f1be94bcba157334e5b27da09cad Mon Sep 17 00:00:00 2001 From: Paul Bakaus Date: Sat, 25 Jul 2026 19:37:01 -0700 Subject: [PATCH 1/3] Retry the generated-output push when main advances mid-sync The sync workflow built once from the checked-out main and aborted when a human commit landed during the ~30s build window (about 10% of runs per the evidence in issue #388), leaving generated provider output stale until the next unrelated push re-triggered it. The commit step now loops up to five times: on a lost race it resets hard to the fresh origin/main (source included), re-installs and rebuilds, and pushes again with linear backoff. Every attempt therefore builds from the main it will land on, which is the invariant the old abort guard protected; the merge-base check stays inside the loop as the pre-push verification. When the rebuilt output shows no drift (the racing commit was another sync, or the new source produces identical output) the step exits cleanly instead of committing an empty sync. Validated by yaml-lint, bash -n, and a local three-repo simulation (bare origin + worker + racer) confirming the lost race rebuilds against the racer's source and lands matching output on attempt two. Retry design proposed by @mktdgtbrz in #388; implemented from the description with the no-drift early exit added. Prepared with AI assistance (Claude Code), directed by @pbakaus. Co-Authored-By: Claude Code --- .github/workflows/sync-generated-output.yml | 40 ++++++++++++++++----- 1 file changed, 32 insertions(+), 8 deletions(-) diff --git a/.github/workflows/sync-generated-output.yml b/.github/workflows/sync-generated-output.yml index 76741b03e..41f564232 100644 --- a/.github/workflows/sync-generated-output.yml +++ b/.github/workflows/sync-generated-output.yml @@ -80,18 +80,42 @@ jobs: - name: Commit generated output if: steps.drift.outputs.changed == 'true' run: | + set -euo pipefail git config user.name "github-actions[bot]" git config user.email "41898282+github-actions[bot]@users.noreply.github.com" - git add $GENERATED_PATHS - git commit -m "Sync generated provider output" - git fetch origin main - if ! git merge-base --is-ancestor origin/main HEAD; then - echo "::error::main advanced while generated output was building; rerun this workflow on the latest main." - exit 1 - fi + # A push can lose the race against a human commit landing on main + # during the ~30s build window (issue #388: ~10% of runs). Instead + # of aborting and waiting for an unrelated push to re-trigger the + # sync, re-sync to the latest main, rebuild against the current + # source, and push again, with backoff. Every attempt builds from + # a fresh origin/main, so pushed output always matches the source + # state it lands on. + for attempt in 1 2 3 4 5; do + git add $GENERATED_PATHS + if git diff --cached --quiet; then + # The race that beat us was another sync (or the rebuilt output + # matches the new main); nothing left to push. + echo "No generated output drift after rebuild (attempt $attempt); nothing to push." + exit 0 + fi + git commit -m "Sync generated provider output" - git push origin HEAD:main + git fetch origin main + if git merge-base --is-ancestor origin/main HEAD && git push origin HEAD:main; then + echo "Pushed generated output on attempt $attempt." + exit 0 + fi + + echo "main advanced during attempt $attempt; re-syncing and rebuilding." + git reset --hard origin/main + bun install --frozen-lockfile + bun run build:release + sleep $((attempt * 5)) + done + + echo "::error::main kept advancing through 5 sync attempts; rerun this workflow on the latest main." + exit 1 - name: Summarize generated output commit if: steps.drift.outputs.changed == 'true' From bea601ac767176657be5759d9e4603b1c727c622 Mon Sep 17 00:00:00 2001 From: Paul Bakaus Date: Sat, 25 Jul 2026 19:43:03 -0700 Subject: [PATCH 2/3] Skip the pointless final-attempt rebuild; stop misattributing push failures Copilot's two review points: the fifth attempt performed a full reset + install + rebuild + 25s backoff that nothing would ever consume before the job failed, and the retry message blamed "main advanced" when the combined condition also fails on push errors (network, auth). The loop now breaks before recovery on the final attempt, and both the retry and terminal messages name the two possible causes. Prepared with AI assistance (Claude Code), directed by @pbakaus. Co-Authored-By: Claude Code --- .github/workflows/sync-generated-output.yml | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/.github/workflows/sync-generated-output.yml b/.github/workflows/sync-generated-output.yml index 41f564232..f77998922 100644 --- a/.github/workflows/sync-generated-output.yml +++ b/.github/workflows/sync-generated-output.yml @@ -107,14 +107,19 @@ jobs: exit 0 fi - echo "main advanced during attempt $attempt; re-syncing and rebuilding." + # The recovery work is pointless on the final attempt: nothing + # would consume the rebuild, and the backoff would only delay + # the failure. + if [ "$attempt" = 5 ]; then break; fi + + echo "Push did not land on attempt $attempt (main advanced, or the push itself failed); re-syncing and rebuilding." git reset --hard origin/main bun install --frozen-lockfile bun run build:release sleep $((attempt * 5)) done - echo "::error::main kept advancing through 5 sync attempts; rerun this workflow on the latest main." + echo "::error::Could not push generated output after 5 attempts (main kept advancing, or pushes kept failing); rerun this workflow on the latest main." exit 1 - name: Summarize generated output commit From 166ec9a51e8ed7e52a532b17a7429db3531d507d Mon Sep 17 00:00:00 2001 From: Paul Bakaus Date: Sat, 25 Jul 2026 19:53:39 -0700 Subject: [PATCH 3/3] Make the job summary reflect whether a sync commit actually pushed Both bots caught the same false report: the summarize step ran off the initial drift flag, so the no-drift-after-rebuild exit still claimed a commit landed on main. The commit step now records pushed=true/false in its step output and the summary reads it. Prepared with AI assistance (Claude Code), directed by @pbakaus. Co-Authored-By: Claude Code --- .github/workflows/sync-generated-output.yml | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/.github/workflows/sync-generated-output.yml b/.github/workflows/sync-generated-output.yml index f77998922..9e5acbdfa 100644 --- a/.github/workflows/sync-generated-output.yml +++ b/.github/workflows/sync-generated-output.yml @@ -78,6 +78,7 @@ jobs: fi - name: Commit generated output + id: commit if: steps.drift.outputs.changed == 'true' run: | set -euo pipefail @@ -97,6 +98,7 @@ jobs: # The race that beat us was another sync (or the rebuilt output # matches the new main); nothing left to push. echo "No generated output drift after rebuild (attempt $attempt); nothing to push." + echo "pushed=false" >> "$GITHUB_OUTPUT" exit 0 fi git commit -m "Sync generated provider output" @@ -104,6 +106,7 @@ jobs: git fetch origin main if git merge-base --is-ancestor origin/main HEAD && git push origin HEAD:main; then echo "Pushed generated output on attempt $attempt." + echo "pushed=true" >> "$GITHUB_OUTPUT" exit 0 fi @@ -125,4 +128,8 @@ jobs: - name: Summarize generated output commit if: steps.drift.outputs.changed == 'true' run: | - echo "Committed generated provider output directly to main." >> "$GITHUB_STEP_SUMMARY" + if [ "${{ steps.commit.outputs.pushed }}" = "true" ]; then + echo "Committed generated provider output directly to main." >> "$GITHUB_STEP_SUMMARY" + else + echo "Generated output drift resolved itself after a mid-run rebuild; nothing was pushed." >> "$GITHUB_STEP_SUMMARY" + fi