Six schema-v1 eval cases grounded in real skill behavior: council
invocation and bootstrapping, convergence-diagnostic reading,
quick-mode selection, claims-verification discipline, profile-library
fallback, plus one should-not-trigger case for simple factual lookups.
Satisfies the eval-coverage ratchet for this modified skill.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
2026-08-22 23:15:23 -04:00
Magnus Hedemarkandfactory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Add an Available Scripts table for scripts/bootstrap.py with invocation
and run-when guidance; add Prerequisites and Limitations covering the
single-model debate design and claims-verification guardrail. Lead the
frontmatter description with the imperative verb Run and add an explicit
negative boundary so the changed skill passes the CI quality gate; all
trigger keywords are preserved.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
RiskVector severity was hardcoded to 'medium' for all risks
regardless of how many agents flagged them. Now uses a simple
heuristic: 3+ agents = high, 2 agents = medium, 1 agent = low.
Also deduplicates similar risks and tracks which agents flagged
each one.
Signed-off-by: Magnus Hedemark <magnus919@pm.me>
PydanticAI reads API keys from provider-specific env vars
(OPENAI_API_KEY, DEEPSEEK_API_KEY, etc.) at Agent creation time.
agent-council was reading AGENT_COUNCIL_API_KEY into a config dict
but never setting the env var PydanticAI actually looks for.
If a user had OPENAI_API_KEY set for something else, agent-council
silently used the wrong key for debate agents.
Fix: load_config() now maps AGENT_COUNCIL_API_KEY to the correct
env var based on the model prefix (openai:, deepseek:, anthropic:,
google:, etc.) and also maps AGENT_COUNCIL_BASE_URL to OPENAI_BASE_URL.
Only sets if not already set, so explicit env vars take precedence.
Signed-off-by: Magnus Hedemark <magnus919@pm.me>
- Updates default model from gpt-4o-mini to gpt-5.6-luna across all
documentation, config, and CLI help text
- Fixes model string format from provider/model to provider:model
(PydanticAI convention) in README, SKILL.md, references, and examples
- Fixes anthropic, deepseek, and google example model strings to use
colon format consistently
Signed-off-by: Magnus Hedemark <magnus919@pm.me>
Adds retry tolerance to structured output parsing across all
debate phases (premortem, position, cross-examine, synthesis,
compose, and verification guardrails). Prevents 'Exceeded
maximum output retries' failures when models produce
near-valid structured output on first attempt.
Signed-off-by: Magnus Hedemark <magnus919@pm.me>
Instead of the compose phase fabricating personas with fake backgrounds,
the council now draws from 39 real professional profiles via a git
submodule (https://github.com/magnus919/hermes-profiles).
Key changes:
- New select phase reads SOUL.md + profile.yaml from profiles submodule
- Auto-updates submodule before selection via git submodule update --remote
- --profiles flag for explicit selection (comma-separated names)
- Auto-selection by keyword overlap with profile descriptions when omitted
- Each agent's identity is their real SOUL.md — actual methodology,
values, and operating principles, not invented backgrounds
- Falls back to composed personas if profile library is unavailable
- Real profiles produce genuine methodological disagreement (debugger
said 'unanswerable without a verified process' to naming question)
Signed-off-by: Magnus Hedemark <magnus919@pm.me>
Two-layer defense against fabricated factual claims:
1. Prompt-level guardrail (FACTUAL_CLAIM_GUARDRAIL) injected into every
debate agent's system prompt — tells agents not to assert specific
verifiable facts they cannot know (domain availability, package
namespace status, pricing, etc.)
2. Post-synthesis verification eval — an LLM-based QA pass that scans
the narrative synthesis for unsubstantiated factual claims and
appends a 'Claims Not Verified' section to the output with specific
quotes, claim types, and explanations.
Both layers are self-contained — no search or external tools required.
Signed-off-by: Magnus Hedemark <magnus919@pm.me>
- Each phase now prints completion status to stdout immediately
- Phase output files written to /tmp/agent-council/<ts>/ as they complete
- Users see real-time progress: persona count, phase timing,
cross-examination metrics per round, convergence verdict
- Synthesis still printed at end as primary deliverable
- Full intermediate output available in /tmp/agent-council/ for inspection
Signed-off-by: Magnus Hedemark <magnus919@pm.me>
- Adds stdlib-only .env loader (no python-dotenv dependency)
- Env vars always take precedence over .env values
- Updates SKILL.md with .env usage example
- Updates configuration reference
Signed-off-by: Magnus Hedemark <magnus919@pm.me>
Magnus Hedemarkandfactory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Jasper (AI Assistant)