mirror of
https://github.com/magnus919/agent-skills.git
synced 2026-09-20 16:16:25 +03:00
Magnus HedemarkandGitHub
f53bddcd7b
* feat(constrained-optimization): add formulation and audit methodology * docs(constrained-optimization): register skill and align trigger metadata * docs(constrained-optimization): route templates and define completion * docs(constrained-optimization): cite solver evidence and applicability
14 lines
4.0 KiB
JSON
14 lines
4.0 KiB
JSON
{
|
|
"schema_version": 1,
|
|
"skill_name": "constrained-optimization",
|
|
"evals": [
|
|
{"id":"hard-constraint-violation","prompt":"Candidate A has objective 10 but exceeds a hard safety limit; candidate B has objective 12 and is feasible. Decide.","expected_output":"Reject A and compare feasible candidates only, reporting the violation and independent feasibility check.","assertions":["The response rejects a candidate that violates a hard constraint","The response does not hide the violation in a penalty score","The response compares feasible-best results separately","The response requires independent constraint recomputation"]},
|
|
{"id":"objective-units-direction","prompt":"A team says optimize cost, quality, and latency but gives no units or direction. What must be clarified?","expected_output":"Require explicit objective direction, units, aggregation, baseline, thresholds, and decision priorities before solver selection.","assertions":["The response requires units for every objective","The response requires minimize or maximize direction","The response requires aggregation and baseline","The response requires explicit priorities or tradeoff rule"]},
|
|
{"id":"pareto-tradeoff","prompt":"Two feasible solutions trade lower cost for higher quality and lower latency. There is no agreed weighting. Recommend a decision process.","expected_output":"Identify nondominated candidates, report units and residuals, then obtain weights, lexicographic priorities, or epsilon limits from the decision maker.","assertions":["The response identifies Pareto or nondominated candidates","The response does not invent a scalar weighting","The response reports constraint status and objective units","The response requires a decision-maker tradeoff rule"]},
|
|
{"id":"stochastic-equal-budget","prompt":"A heuristic beats the baseline once with a lucky seed but uses twice the compute budget. How should it be evaluated?","expected_output":"Repeat under equal budgets with declared or paired seeds, report distribution and failure rate, and avoid selecting on the best run alone.","assertions":["The response enforces equal evaluation budgets","The response requires declared seeds or repeat protocol","The response reports spread or distribution and failure rate","The response rejects best-run-only evidence"]},
|
|
{"id":"unknown-vs-infeasible","prompt":"A CP solver stops at its time limit without a solution and reports UNKNOWN. Can we claim the constraints are infeasible?","expected_output":"No. Preserve UNKNOWN as unresolved; claim infeasibility only with a proof or explicit infeasible status, and record the limit and next action.","assertions":["The response does not call UNKNOWN infeasible","The response distinguishes timeout from proven infeasibility","The response records solver status and limit","The response proposes a bounded follow-up or labels the result unresolved"]},
|
|
{"id":"bound-gap-interpretation","prompt":"An exact solver returns a feasible incumbent and a nonzero optimality gap at timeout. What can be claimed?","expected_output":"Report the feasible incumbent and bound/gap, but do not claim global optimality; include tolerance, budget, and unresolved optimality.","assertions":["The response reports the feasible incumbent separately from the bound","The response interprets the nonzero gap as lack of proof of optimality","The response includes timeout/budget and tolerance","The response avoids claiming global optimum"]},
|
|
{"id":"independent-feasibility-audit","prompt":"The solver says success, but its objective and constraints were computed by the same callback code. What release evidence is missing?","expected_output":"Recompute objectives, domains, residuals, and feasibility with an independent checker and retain both raw solver and audit outputs.","assertions":["The response requires an independent feasibility implementation","The response recomputes objective and every hard constraint","The response checks domains and tolerances","The response retains raw and audit evidence"]}
|
|
]
|
|
}
|