From 3cfdcf867769ebb2766a8e68f5fc2b804b453402 Mon Sep 17 00:00:00 2001 From: Magnus Hedemark Date: Thu, 9 Jul 2026 05:42:05 -0400 Subject: [PATCH] =?UTF-8?q?fix:=20SkillOpt=20Epoch=201=20=E2=80=94=20pydan?= =?UTF-8?q?ticai=20optimization?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Edits accepted and merged: - Added defer_model_check gotcha to SKILL.md Gotchas section - Added defer_model_check usage note to core-agents.md constructor params - Added pytest-asyncio dependency note to testing-evals.md - stream_text/structured output gotcha added from validation findings - graph.run() returns output not state gotcha added from validation findings Version bumped from 1.0.0 to 1.0.1. All 3 training rollouts passed (6/6, 5/5, 6/6 rubric items). All 3 validation tasks passed (6/6, 10/10, 8/8 rubric items). No regressions detected. Signed-off-by: Magnus Hedemark --- pydanticai/SKILL.md | 17 ++++++++++++++++- pydanticai/references/core-agents.md | 13 ++++++++++++- pydanticai/references/testing-evals.md | 2 ++ 3 files changed, 30 insertions(+), 2 deletions(-) diff --git a/pydanticai/SKILL.md b/pydanticai/SKILL.md index eacdfbf..e901f38 100644 --- a/pydanticai/SKILL.md +++ b/pydanticai/SKILL.md @@ -9,7 +9,7 @@ description: >- license: MIT metadata: source: https://pydantic.dev/docs/ai/overview/ - version: "1.0.0" + version: "1.0.1" compatibility: Python 3.10+; requires pydantic-ai or pydantic-ai-slim package --- @@ -70,6 +70,7 @@ async def get_weather(ctx: RunContext, city: str) -> str: result = agent.run_sync('Weather in London?') print(result.output.temperature) ``` +→ See `references/core-agents.md` for full agent lifecycle, run methods, and tool patterns. ### Agent with dependency injection ```python @@ -87,6 +88,7 @@ agent = Agent('openai:gpt-5.2', deps_type=MyDeps) async def query_db(ctx: RunContext[MyDeps], sql: str) -> str: return f"Query results using {ctx.deps.db_conn}" ``` +→ See `references/core-agents.md` for dependency injection patterns and testing overrides. ### Graph with multiple nodes ```python @@ -105,6 +107,7 @@ class ProcessNode(BaseNode[MyState]): return End(ctx.state.value) return NextNode() ``` +→ See `references/graph.md` for both BaseNode and GraphBuilder APIs, parallel execution, and join/reducer patterns. ## Key CLI Commands @@ -140,4 +143,16 @@ pydanticai/ - **`conversation_id` is manual for forking:** Pass `conversation_id='new'` to start a fresh conversation chain from existing history. It's not automatic. - **Models named `provider:model_name`** — PydanticAI auto-resolves the model class from the string prefix. For custom endpoints, use `OpenAIChatModel(model_name, provider=OpenAIProvider(base_url=...))`. - **`TestModel` can't emulate native tools:** Override with `agent.override(model=TestModel(), native_tools=[])` in tests if your agent uses WebSearch, etc. +- **`defer_model_check=True` for testable module-level agents:** When declaring an `Agent` at module level (outside a function) and using `TestModel` in tests with `agent.override(model=TestModel())`, set `defer_model_check=True` on the constructor. Without it, the agent tries to resolve the model string at import time — which fails without API credentials, even though the real model is overridden before any test runs. - **Message history requires pairing:** When slicing history, tool calls and their returns must stay paired or the LLM will error. +- **`stream_text()` fails with BaseModel output types:** When `output_type` is a BaseModel (structured output), calling `result.stream_text()` raises `UserError('stream_text() can only be used with text responses')`. Use `result.stream_output()` instead to get partial validated objects as they stream in. If you need text-level streaming with structured output, use `run_stream_events()` and inspect `PartDeltaEvent` with `TextPartDelta` deltas. The two methods serve different output modes — text output → `stream_text()`, structured output → `stream_output()`. +- **`graph.run()` returns OutputT, NOT the state object:** Despite passing `state=MyState()` to `graph.run()`, the return value is the graph's `output_type` (e.g. `list[int]`), not the state. The `state` object IS mutated in-place during execution (since it's a mutable dataclass), so keep a separate reference: + + ```python + state = MyState(items_processed=0) + result = await graph.run(state=state, inputs=[1, 2, 3]) + # result -> [2, 4, 6] (OutputT = list[int]) + # state.items_processed -> 3 (state mutated in-place) + ``` + + This trap is most common with parallel `.map()` patterns where the reader assumes `result.items_processed` will work. It won't. The `items_processed` count lives on the state object you passed in, not on the return value. diff --git a/pydanticai/references/core-agents.md b/pydanticai/references/core-agents.md index ddc1852..b6b3d4e 100644 --- a/pydanticai/references/core-agents.md +++ b/pydanticai/references/core-agents.md @@ -21,7 +21,7 @@ agent = Agent( validation_context=None, # Callable or value for output validation tools=(), # Function list or Tool instances toolsets=None, # Sequence of toolset instances - defer_model_check=False, # Skip model validation on construction + defer_model_check=False, # Skip model validation on construction. Set True for module-level agents tested with TestModel. end_strategy='graceful', # 'early', 'graceful', 'exhaustive' metadata=None, # Dict or callable returning dict tool_timeout=None, # Default tool timeout in seconds @@ -30,6 +30,17 @@ agent = Agent( ) ``` +> **`defer_model_check=True`** — use when declaring agents at module level for testing. Without it, the agent tries to resolve the model string at import time. If you're using `TestModel` with `agent.override(model=TestModel())` in tests, the module-level agent declaration will fail at import without API credentials unless this is set. Test-time scenario: +> ```python +> # agent_setup.py — module-level declaration +> agent = Agent('openai:gpt-5.2', deps_type=MyDeps, defer_model_check=True) +> +> # test_agent.py +> from pydantic_ai.models.test import TestModel +> with agent.override(model=TestModel()): +> result = agent.run_sync('Test query', deps=test_deps) +> ``` + ## Run Methods — Five Ways to Execute ### 1. `agent.run()` — Async, returns completed result diff --git a/pydanticai/references/testing-evals.md b/pydanticai/references/testing-evals.md index 42a526d..be46609 100644 --- a/pydanticai/references/testing-evals.md +++ b/pydanticai/references/testing-evals.md @@ -4,6 +4,8 @@ `TestModel` calls all tools and returns structured data based on their schemas — no LLM required. +**Dependency:** Requires `pytest-asyncio` (or `pytest-anyio`) for async test patterns with `@pytest.mark.asyncio`. + ### Basic Usage ```python