skills/magnus919_agent-skills
1
0
Fork 0
mirror of https://github.com/magnus919/agent-skills.git synced 2026-09-11 19:47:12 +03:00
Code Issues Packages Projects Releases Wiki Activity

fix: SkillOpt optimize AI operating economics (#369)

Browse Source 
* fix: SkillOpt epoch 1 for AI operating economics

Promote cold-load entry points, quick-start reference routing, the minimum decision-record contract, and trigger-oriented progressive disclosure.

Signed-off-by: Magnus Hedemark <magnus919@pm.me>

* fix: SkillOpt epoch 2 for AI operating economics

Add review-depth selection, evidence-to-disposition guidance, and scenario-led routing across adjacent skills.

Signed-off-by: Magnus Hedemark <magnus919@pm.me>

* fix: SkillOpt epoch 3 for AI operating economics

Expose a minimum claim ledger and explicit closure conditions for every bounded disposition.

Signed-off-by: Magnus Hedemark <magnus919@pm.me>

* fix: resolve SkillOpt review consistency findings

Align entry-point paths, canonical step routing, claim-ledger fields, and triage disposition wording.

Signed-off-by: Magnus Hedemark <magnus919@pm.me>

* fix: resolve final SkillOpt disposition wording

Keep review-depth outputs inside the canonical disposition set and distinguish supported claims from permitted language.

Signed-off-by: Magnus Hedemark <magnus919@pm.me>

* fix: complete SkillOpt routing correction

Route triage through the outcome-map step and identify the evidence-classification step explicitly.

Signed-off-by: Magnus Hedemark <magnus919@pm.me>

* fix: complete AI economics review template

Add the minimum decision-record fields required by the optimized skill routing contract.

Signed-off-by: Magnus Hedemark <magnus919@pm.me>

---------

Signed-off-by: Magnus Hedemark <magnus919@pm.me>
This commit is contained in:
Magnus Hedemark
2026-08-21 16:21:23 -04:00
committed by GitHub
parent 94b7231147
commit 1291f9576e
4 changed files with 92 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ai-operating-economics/SKILL.md
+72 -15
View File
@@ -25,6 +25,16 @@ This skill provides the cross-domain decision spine for evaluating an AI-enabled
The core question is not “Did the model make people faster?” It is: “What changed in this workflow, for whom, at what full cost, with what outcome and countermetric evidence, and what authority should the organization grant next?” The core question is not “Did the model make people faster?” It is: “What changed in this workflow, for whom, at what full cost, with what outcome and countermetric evidence, and what authority should the organization grant next?”
## Entry Points
| Starting state | Start with | Primary artifact or route |
|---|---|---|
| Idea or proposed AI workflow | Steps 1–2 | `templates/ai-initiative-evidence-record.md` |
| Existing pilot or outcome data | Steps 3–7 | `references/evidence-method.md` plus the evidence record |
| Request for broader population or side-effect authority | Steps 7–8; load `references/evidence-method.md` section 7a for the governance packet | Governance evidence packet plus the evidence record |
| Executive, portfolio, launch, or lifecycle review | Steps 8–9 | `templates/ai-economics-review.md`; route launch/runtime details onward |
| Standalone financial, statistical, telemetry, runtime, or governance implementation task | When Not to Use | Named adjacent specialist skill |
## When to Use ## When to Use
Load this skill when the user needs to: Load this skill when the user needs to:
@@ -39,15 +49,17 @@ Load this skill when the user needs to:
## When Not to Use ## When Not to Use
- **Standalone financial statements, pricing, CAC/LTV, runway, or SaaS metrics:** use [financial-modeling](../financial-modeling/SKILL.md). | If the task is primarily... | Route to | This skill still contributes... |
- **Token, infrastructure, quota, capacity, or SLO-cost modeling:** use [capacity-and-cost-engineering](../capacity-and-cost-engineering/SKILL.md). |---|---|---|
- **Metric trees, event schemas, instrumentation QA, or product dashboards:** use [product-analytics-and-measurement](../product-analytics-and-measurement/SKILL.md). | Financial statements, pricing, CAC/LTV, runway, or SaaS metrics | [financial-modeling](../financial-modeling/SKILL.md) | The AI workflow's outcome and cost evidence can feed the model |
- **Experimental design, causal inference, statistical testing, or power analysis:** use [data-scientist](../data-scientist/SKILL.md). | Token, infrastructure, quota, capacity, or SLO-cost modeling | [capacity-and-cost-engineering](../capacity-and-cost-engineering/SKILL.md) | The economic decision can consume the resulting cost boundary |
- **Agent datasets, graders, traces, regression analysis, or telemetry implementation:** use [agent-evals-and-observability](../agent-evals-and-observability/SKILL.md). | Metric trees, event schemas, instrumentation QA, or product dashboards | [product-analytics-and-measurement](../product-analytics-and-measurement/SKILL.md) | The decision defines which outcome and countermetric evidence matters |
- **Production rollout, runtime budgets, authority, fallback, escalation, or disablement:** use [agent-production-operations](../agent-production-operations/SKILL.md). | Experimental design, causal inference, statistical testing, or power analysis | [data-scientist](../data-scientist/SKILL.md) | The decision specifies the claim and comparison it must support |
- **Organization-wide AI risk, policy, compliance, or governance operating models:** use [ai-governance](../ai-governance/SKILL.md). | Agent datasets, graders, traces, regression analysis, or telemetry implementation | [agent-evals-and-observability](../agent-evals-and-observability/SKILL.md) | The decision consumes verified evaluation and telemetry evidence |
- **A launch-readiness packet or production go/no-go decision:** use [production-readiness](../production-readiness/SKILL.md). | Production rollout, runtime budgets, authority, fallback, escalation, or disablement | [agent-production-operations](../agent-production-operations/SKILL.md) | The decision sets the evidence and authority boundary |
- **General product governance cadence without an AI-specific value question:** use [product-operations-and-governance](../product-operations-and-governance/SKILL.md). | Organization-wide AI risk, policy, compliance, or governance operating models | [ai-governance](../ai-governance/SKILL.md) | The initiative record supplies an operating case and unresolved gaps |
| Launch-readiness packet or production go/no-go decision | [production-readiness](../production-readiness/SKILL.md) | The initiative disposition becomes one readiness input |
| General product governance cadence without an AI-specific value question | [product-operations-and-governance](../product-operations-and-governance/SKILL.md) | Use this skill only for the AI-specific value and operating-economics question |
## Non-Negotiable Reasoning Rules ## Non-Negotiable Reasoning Rules
@@ -66,6 +78,23 @@ Load this skill when the user needs to:
Use this sequence for an AI initiative review. Load the detailed method and the evidence-record template when the task requires a durable artifact. Use this sequence for an AI initiative review. Load the detailed method and the evidence-record template when the task requires a durable artifact.
### Quick Start by Need
| Need | First action | Load next |
|---|---|---|
| Triage a claim | Name the workflow, decision, and evidence class | Steps 1–3; evidence classes are defined in Step 7 |
| Build a durable record | Copy the initiative evidence record and complete the header first | `templates/ai-initiative-evidence-record.md` |
| Investigate uncertain evidence | Freeze the claim table before drafting conclusions | `references/evidence-method.md` |
| Prepare a review | Assemble evidence, slices, cost, gaps, and disposition | `templates/ai-economics-review.md` |
### Choose Review Depth
| Mode | Use when | Minimum evidence | Output |
|---|---|---|---|
| Triage | A claim or opportunity needs a bounded first decision | Workflow, value hypothesis, one outcome, one countermetric, known gaps | Hold, with a routing/evidence plan |
| Standard | A pilot or workflow decision can change population or investment | Comparison, outcome/countermetrics, slices, cost boundary, owner, reversal path | Scale, constrain, redesign, or hold |
| High-assurance | Authority, sensitive data, material user impact, or irreversible change is involved | Standard evidence plus governance packet, human oversight, incident/revalidation, and decommissioning evidence | Scale only within an explicit authority boundary, or Hold |
### 1. Define the intervention and decision ### 1. Define the intervention and decision
Name the workflow, population, task boundary, intervention mode, baseline, decision sought, and decision owner. State whether the AI assists, recommends, routes, executes, or replaces/removes work. Define what remains human-controlled. Name the workflow, population, task boundary, intervention mode, baseline, decision sought, and decision owner. State whether the AI assists, recommends, routes, executes, or replaces/removes work. Define what remains human-controlled.
@@ -143,6 +172,25 @@ For every material claim, label it:
Keep the source, access date, scope, version, caveat, and permitted interpretation with each claim. Load `references/source-index.md` for the research basis and evidence boundaries. Keep the source, access date, scope, version, caveat, and permitted interpretation with each claim. Load `references/source-index.md` for the research basis and evidence boundaries.
### Minimum Claim Ledger
For each material claim, record: **claim**, **evidence class**, **source and scope**, **what it supports**, **what it does not support**, **open challenge**, and **permitted language**. Keep unknown claims visible; do not let a source URL or vendor report stand in for direct workflow evidence.
### Minimum Decision Record
Every completed review must expose, in one durable artifact: the intervention and population, value hypothesis, primary outcome, countermetrics, comparison and limitations, cost boundary, relevant slices, evidence classes, missing evidence with owner, disposition, authority limit, reversal path, and review trigger.
### Disposition Quick Pick
| Evidence state | Default disposition | Next control |
|---|---|---|
| Outcome and countermetrics support a bounded expansion; cost and slices are understood | Scale | Name the next population and authority slice |
| Value is plausible but a cost, quality, subgroup, or authority boundary remains unresolved | Constrain | Limit population, task, quota, or human review |
| The mechanism creates avoidable failure or burden | Redesign | Change the workflow or control and rerun the comparison |
| Required evidence is missing or conflicting | Hold | Assign the evidence owner and review trigger |
| Value is absent or countermetrics exceed bounds | Retire | Protect affected people, migrate, and record learning |
| A material gap is accepted temporarily by a named human | Exception | Set expiry, containment, approver, and revisit condition |
### 8. Produce a bounded decision ### 8. Produce a bounded decision
Choose exactly one primary disposition: Choose exactly one primary disposition:
@@ -154,6 +202,15 @@ Choose exactly one primary disposition:
- **Retire:** observed value is absent or countermetrics exceed acceptable bounds, with a transition path. - **Retire:** observed value is absent or countermetrics exceed acceptable bounds, with a transition path.
- **Exception:** proceed despite a named gap only with an accountable human approver, expiry or revisit trigger, and containment plan. - **Exception:** proceed despite a named gap only with an accountable human approver, expiry or revisit trigger, and containment plan.
### Closure Conditions
- **Scale:** next population, authority slice, owner, and review trigger are recorded.
- **Constrain:** the boundary, quota, human-review rule, and condition for expansion are recorded.
- **Redesign:** the changed mechanism, rerun comparison, and new acceptance boundary are recorded.
- **Hold:** the missing evidence, owner, method, and due trigger are recorded.
- **Retire:** transition, affected-person protection, decommissioning, and retained learning are recorded.
- **Exception:** named human approver, scope, expiry, containment, and revisit condition are recorded.
A decision is incomplete without an owner, review date or trigger, evidence gaps, and reversal path. Route launch or runtime consequences to the appropriate specialist skill. A decision is incomplete without an owner, review date or trigger, evidence gaps, and reversal path. Route launch or runtime consequences to the appropriate specialist skill.
### 9. Close the learning loop ### 9. Close the learning loop
@@ -162,12 +219,12 @@ At the review date, compare expected versus observed outcomes, cost, quality, wo
## Load-on-Demand References ## Load-on-Demand References
| Need | Load | | Need | Load when | File |
|---|---| |---|---|---|
| Apply the full research and decision method, including comparison design and uncertainty | [references/evidence-method.md](references/evidence-method.md) | | Apply the full research and decision method, including comparison design and uncertainty | Evidence is incomplete, contested, or consequential | [references/evidence-method.md](references/evidence-method.md) |
| Review the sources and their permitted interpretations | [references/source-index.md](references/source-index.md) | | Review sources and permitted interpretations | A claim needs provenance or a source boundary | [references/source-index.md](references/source-index.md) |
| Fill a durable initiative record before a review | [templates/ai-initiative-evidence-record.md](templates/ai-initiative-evidence-record.md) | | Fill a durable initiative record | Starting a new workflow review or pilot assessment | [templates/ai-initiative-evidence-record.md](templates/ai-initiative-evidence-record.md) |
| Prepare an executive or lifecycle review from one or more initiative records | [templates/ai-economics-review.md](templates/ai-economics-review.md) | | Prepare an executive or lifecycle review | Combining one or more initiative records for a decision | [templates/ai-economics-review.md](templates/ai-economics-review.md) |
## Common Pitfalls ## Common Pitfalls
ai-operating-economics/references/evidence-method.md
+3 -3
View File
@@ -35,9 +35,9 @@ Benefits-realization planning can borrow from public-sector benefits-management
Build a claim table before writing the conclusion: Build a claim table before writing the conclusion:
| Claim | Evidence class | Source and scope | What it supports | What it does not support | Open challenge | | Claim | Evidence class | Source and scope | What it supports | What it does not support | Open challenge | Permitted interpretation |
|---|---|---|---|---|---| |---|---|---|---|---|---|---|
| [claim] | observed / causal / inferred / vendor-reported / asserted / normative | [citation, date, version] | [permitted interpretation] | [boundary] | [test or missing evidence] | | [claim] | observed / causal / inferred / vendor-reported / asserted / normative | [citation, date, version] | [supported claim] | [boundary] | [test or missing evidence] | [calibrated language] |
Use the strongest source appropriate to the claim: Use the strongest source appropriate to the claim:
ai-operating-economics/templates/ai-economics-review.md
+14
View File
@@ -11,6 +11,20 @@ Use this template for an executive, portfolio, experiment, launch, or lifecycle
- Records reviewed: - Records reviewed:
- Decision deadline or trigger: - Decision deadline or trigger:
## Minimum decision record
- Intervention and population:
- Value hypothesis:
- Primary outcome:
- Countermetrics:
- Comparison and limitations:
- Cost boundary:
- Relevant slices:
- Evidence classes:
- Missing evidence, owner, and next trigger:
- Authority limit:
- Reversal path:
## Executive decision ## Executive decision
- Disposition: scale / constrain / redesign / hold / retire / exception - Disposition: scale / constrain / redesign / hold / retire / exception
ai-operating-economics/templates/ai-initiative-evidence-record.md
+3 -3
View File
@@ -36,9 +36,9 @@
- Quality measurement limitations: - Quality measurement limitations:
- Statistical or causal analysis owner: - Statistical or causal analysis owner:
| Claim | Class | Source | Result | Caveat | Permitted interpretation | | Claim | Evidence class | Source and scope | What it supports | What it does not support | Open challenge | Permitted interpretation |
|---|---|---|---|---|---| |---|---|---|---|---|---|---|
| | observed / causal / inferred / vendor-reported / asserted / normative | | | | | | | observed / causal / inferred / vendor-reported / asserted / normative | | | | | |
## Outcome and countermetrics ## Outcome and countermetrics