mirror of
https://github.com/magnus919/agent-skills.git
synced 2026-09-22 00:56:35 +03:00
Magnus HedemarkandGitHub
3c0a21493a
* feat(skill): add spring ai diagnostics * fix(spring-ai): harden project compatibility diagnostics * Register Spring AI skill and normalize metadata * fix(spring-ai): scope Maven versions to dependency elements * fix(spring-ai): validate the shipped diagnostic fixture * docs(spring-ai): state the limits of static diagnostics * docs(spring-ai): clarify warning exit semantics * test(spring-ai): load the sibling diagnostic module explicitly * fix(spring-ai): ignore Java block comments in diagnostics * fix(spring-ai): preserve Java text block contents
14 lines
3.9 KiB
JSON
14 lines
3.9 KiB
JSON
{
|
||
"schema_version": 1,
|
||
"skill_name": "spring-ai",
|
||
"evals": [
|
||
{"id":"diagnose-maven-config","prompt":"Inspect this Maven Spring AI project and report versions, configuration risks, and whether prompt logging is enabled.","expected_output":"Runs the read-only checker and returns structured facts plus severity-coded findings without exposing secrets.","assertions":["Maven build facts are reported","configuration findings have severity and paths","secret values are not printed","exit/status semantics are explained"]},
|
||
{"id":"memory-isolation","prompt":"A Spring AI app uses MessageChatMemoryAdvisor but follow-up requests sometimes see another user’s conversation. Diagnose the likely boundary and propose checks.","expected_output":"Checks explicit conversation IDs, authenticated scope, persistence/eviction, concurrent access, and tests cross-conversation isolation.","assertions":["conversation ID is explicit","cross-user memory leakage is tested","the response separates static diagnosis from live proof","the fix is routed to application ownership"]},
|
||
{"id":"rag-authorization","prompt":"Review a Spring AI RAG design where a VectorStore feeds context into ChatClient. What must be verified before retrieval results enter a prompt?","expected_output":"Verifies document ownership/authorization, tenant filtering, freshness, empty/conflicting results, and prompt-context limits.","assertions":["authorization precedes prompt construction","empty and stale retrieval are distinct cases","source ownership and freshness are recorded","answer quality is not conflated with access control"]},
|
||
{"id":"tool-mcp-boundary","prompt":"A model can call tools and an MCP server in a Spring AI application. Give a safe troubleshooting checklist for a tool timeout and schema drift.","expected_output":"Treats model calls as proposals; checks tool identity, schema, authorization, timeout, retries, side effects, transport/capabilities, and audit evidence.","assertions":["application owns authorization and execution","timeout is not rendered as success","MCP transport and schema are checked against installed versions","side effects and retry/deduplication are addressed"]},
|
||
{"id":"stream-partial","prompt":"Our Spring AI streaming endpoint persists an answer when the first chunks arrive, even if the provider later fails. What should change?","expected_output":"Models partial output as provisional, commits only after terminal completion and validation, and defines disconnect, cancellation, timeout, and recovery behavior.","assertions":["partial output is distinguished from completion","commit occurs only after terminal success","provider failure and disconnect have recovery paths","the answer does not claim provider integration was tested"]},
|
||
{"id":"observability-redaction","prompt":"We need to debug Spring AI prompts in production. What observability checks and rollback safeguards should we establish?","expected_output":"Keeps prompt/completion logging disabled by default, defines redaction/access/retention and a temporary enablement rollback, and identifies low/high-cardinality observation fields.","assertions":["prompt and completion logging risk is explicit","redaction access and retention are specified","temporary debugging has rollback","conversation and tool metadata are handled carefully"]},
|
||
{"id":"version-upgrade","prompt":"A project is upgrading from Spring AI 1.x to 2.x. How should the skill keep its compatibility claims honest?","expected_output":"Reports observed project versions, consults current official release and upgrade notes, calls out API differences such as tool-loop/advisor behavior and required conversation IDs, and refuses to infer compatibility from a compile alone.","assertions":["version facts are separated from compatibility claims","official upgrade documentation is the authority","tool-loop and memory-ID differences are checked","a successful compile is not treated as production readiness"]}
|
||
]
|
||
}
|