mirror of
https://github.com/magnus919/agent-skills.git
synced 2026-09-11 19:47:12 +03:00
enrich: 4 skills v1.1.0 — source validation, deepened references, worked examples
DSPy v1.1.0: validation audit, worked RAG compilation example, expand ref table Haystack v1.1.0: validation audit, file converters/YAML/component types, +2 refs CrewAI v1.1.0: validation audit, unified Memory system, Flows docs, +3 refs AutoGen v1.1.0: validation audit, v0.4 migration guide, AgentTool, streaming, +2 refs All API surfaces validated against official docs.
This commit is contained in:
+3
-1
@@ -8,7 +8,7 @@ description: >-
|
|||||||
license: MIT
|
license: MIT
|
||||||
metadata:
|
metadata:
|
||||||
author: Magnus Hedemark
|
author: Magnus Hedemark
|
||||||
version: 1.0.3
|
version: 1.1.0
|
||||||
source: https://microsoft.github.io/autogen
|
source: https://microsoft.github.io/autogen
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -82,6 +82,8 @@ assistant = AssistantAgent(
|
|||||||
| Group Chat | RoundRobin, Selector, MagenticOne | `references/group-chat.md` |
|
| Group Chat | RoundRobin, Selector, MagenticOne | `references/group-chat.md` |
|
||||||
| Code Execution | Docker, local, cancellation tokens | `references/code-execution.md` |
|
| Code Execution | Docker, local, cancellation tokens | `references/code-execution.md` |
|
||||||
| Tool Integration | register_function, @tool, MCP integration | `references/tool-integration.md` |
|
| Tool Integration | register_function, @tool, MCP integration | `references/tool-integration.md` |
|
||||||
|
| v0.4 Migration | v0.2->v0.4 migration, AgentTool, streaming, termination | `references/v04-migration.md` |
|
||||||
|
| Validation Audit | Research validation of all API claims | `references/validation-audit.md` |
|
||||||
| FAQ & Troubleshooting | Common errors and fixes | `references/faq-and-troubleshooting.md` |
|
| FAQ & Troubleshooting | Common errors and fixes | `references/faq-and-troubleshooting.md` |
|
||||||
|
|
||||||
## Templates
|
## Templates
|
||||||
|
|||||||
@@ -0,0 +1,86 @@
|
|||||||
|
# AutoGen v0.4 Migration and Advanced Patterns
|
||||||
|
|
||||||
|
AutoGen v0.4 introduced significant API changes from v0.2. This reference covers migration and patterns not found in the v0.2 API.
|
||||||
|
|
||||||
|
## v0.2 → v0.4 Migration
|
||||||
|
|
||||||
|
### v0.2 Pattern (Deprecated)
|
||||||
|
|
||||||
|
```python
|
||||||
|
# v0.2: UserProxyAgent bundled code execution + human input
|
||||||
|
from autogen import AssistantAgent, UserProxyAgent
|
||||||
|
|
||||||
|
assistant = AssistantAgent(name="assistant", llm_config=llm_config)
|
||||||
|
proxy = UserProxyAgent(name="proxy", human_input_mode="NEVER",
|
||||||
|
code_execution_config={"use_docker": True})
|
||||||
|
proxy.initiate_chat(assistant, message="Write Python code")
|
||||||
|
```
|
||||||
|
|
||||||
|
### v0.4 Pattern
|
||||||
|
|
||||||
|
```python
|
||||||
|
# v0.4: Code execution is a separate agent
|
||||||
|
from autogen_agentchat.agents import AssistantAgent, CodeExecutorAgent
|
||||||
|
from autogen_agentchat.teams import RoundRobinGroupChat
|
||||||
|
from autogen_ext.code_executors.local import LocalCommandLineCodeExecutor
|
||||||
|
from autogen_ext.models.openai import OpenAIChatCompletionClient
|
||||||
|
|
||||||
|
model_client = OpenAIChatCompletionClient(model="gpt-4o-mini")
|
||||||
|
assistant = AssistantAgent(name="assistant", model_client=model_client,
|
||||||
|
system_message="You are a helpful assistant.")
|
||||||
|
executor = CodeExecutorAgent(
|
||||||
|
name="executor",
|
||||||
|
code_executor=LocalCommandLineCodeExecutor(work_dir="coding"),
|
||||||
|
)
|
||||||
|
|
||||||
|
team = RoundRobinGroupChat([assistant, executor])
|
||||||
|
result = await team.run(task="Write Python code to calculate pi")
|
||||||
|
```
|
||||||
|
|
||||||
|
## AgentTool — Agent as Tool
|
||||||
|
|
||||||
|
```python
|
||||||
|
from autogen_agentchat.tools import AgentTool
|
||||||
|
|
||||||
|
writer = AssistantAgent(name="writer", model_client=model_client,
|
||||||
|
system_message="Write well.")
|
||||||
|
writer_tool = AgentTool(agent=writer)
|
||||||
|
|
||||||
|
assistant = AssistantAgent(
|
||||||
|
name="assistant",
|
||||||
|
model_client=model_client,
|
||||||
|
tools=[writer_tool],
|
||||||
|
system_message="You are a helpful assistant.",
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Streaming with run_stream()
|
||||||
|
|
||||||
|
```python
|
||||||
|
stream = assistant.run_stream(task="Tell me a story")
|
||||||
|
async for message in stream:
|
||||||
|
print(message) # Each message as it's generated
|
||||||
|
```
|
||||||
|
|
||||||
|
## Three human_input_mode Behaviors
|
||||||
|
|
||||||
|
| Mode | Behavior | Use case |
|
||||||
|
|------|----------|----------|
|
||||||
|
| `"NEVER"` | No human input requested. Agent runs fully autonomously. | Automated pipelines, batch processing |
|
||||||
|
| `"ALWAYS"` | Agent asks for human input before every reply. Blocks until input received. | Human-in-the-loop approval gates |
|
||||||
|
| `"TERMINATE"` | Agent asks for human input only when it's about to terminate (send TERMINATE). | Review final output before closing |
|
||||||
|
|
||||||
|
## Termination Conditions
|
||||||
|
|
||||||
|
```python
|
||||||
|
from autogen_agentchat.conditions import TextMentionTermination, MaxMessageTermination
|
||||||
|
|
||||||
|
# Stop when agent says TERMINATE
|
||||||
|
text_termination = TextMentionTermination("TERMINATE")
|
||||||
|
|
||||||
|
# Or stop after N messages
|
||||||
|
max_termination = MaxMessageTermination(max_messages=10)
|
||||||
|
|
||||||
|
# Combine conditions
|
||||||
|
# team.run(..., termination_condition=text_termination | max_termination)
|
||||||
|
```
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
# AutoGen Skill — Research Validation Audit
|
||||||
|
|
||||||
|
**Date:** 2026-07-09
|
||||||
|
**Sources:** microsoft.github.io/autogen/stable
|
||||||
|
|
||||||
|
## Claims Verified Correct
|
||||||
|
|
||||||
|
| Claim | Source | Status |
|
||||||
|
|-------|--------|--------|
|
||||||
|
| `AssistantAgent` with `name`, `system_message`, `model_client` | autogen docs | ✓ |
|
||||||
|
| `UserProxyAgent` with `human_input_mode`, `code_executor` | autogen docs | ✓ |
|
||||||
|
| `RoundRobinGroupChat` for fixed-order conversation | autogen docs | ✓ |
|
||||||
|
| `SelectorGroupChat` with `model_client` for speaker selection | autogen docs | ✓ |
|
||||||
|
| Docker execution via `DockerCommandLineCodeExecutor` | autogen docs | ✓ |
|
||||||
|
| Local execution via `LocalCommandLineCodeExecutor` | autogen docs | ✓ |
|
||||||
|
| Cancellation via `CancellationToken` | autogen docs | ✓ |
|
||||||
|
| MCP tool integration via `McpWorkbench` | autogen docs | ✓ |
|
||||||
|
|
||||||
|
## Claims Updated by Source Audit
|
||||||
|
|
||||||
|
- **AssistantAgent** is explicitly documented as a "kitchen sink agent for prototyping" — the skill should note its prototyping nature
|
||||||
|
- **CodeExecutorAgent** is the v0.4 separate agent for code execution, splitting the role that UserProxyAgent filled in v0.2
|
||||||
|
- **AgentTool** wraps an entire agent as a tool callable by another agent — important pattern for agent composition
|
||||||
|
- **Streaming** uses `.run_stream()` with `async for message in stream`, not the older callback approach
|
||||||
|
- **v0.2->v0.4 migration**: UserProxyAgent in v0.2 becomes `AssistantAgent` + `CodeExecutorAgent` + `RoundRobinGroupChat` in v0.4
|
||||||
|
|
||||||
|
## Missing from Skill (Addressed in This Enrichment)
|
||||||
|
|
||||||
|
- v0.2 to v0.4 migration patterns
|
||||||
|
- AgentTool for agent-as-tool composition
|
||||||
|
- v0.4 streaming via `run_stream()`
|
||||||
|
- Three human_input_mode behaviors documented with examples
|
||||||
|
- Validation audit file
|
||||||
+3
-1
@@ -8,7 +8,7 @@ description: >-
|
|||||||
license: MIT
|
license: MIT
|
||||||
metadata:
|
metadata:
|
||||||
author: Magnus Hedemark
|
author: Magnus Hedemark
|
||||||
version: 1.0.3
|
version: 1.1.0
|
||||||
source: https://docs.crewai.com
|
source: https://docs.crewai.com
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -113,6 +113,8 @@ result = crew.kickoff()
|
|||||||
| Crew Patterns | Sequential, hierarchical, consensual crews | `references/crew-patterns.md` |
|
| Crew Patterns | Sequential, hierarchical, consensual crews | `references/crew-patterns.md` |
|
||||||
| Tool Integration | Creating tools with @tool decorator | `references/tool-integration.md` |
|
| Tool Integration | Creating tools with @tool decorator | `references/tool-integration.md` |
|
||||||
| Callbacks | Monitoring agent and task execution | `references/callbacks.md` |
|
| Callbacks | Monitoring agent and task execution | `references/callbacks.md` |
|
||||||
|
| Memory System | Unified Memory class, cross-agent context | `references/memory-system.md` |
|
||||||
|
| Flows | Event-driven orchestration connecting crews | `references/flows.md` |
|
||||||
| FAQ & Troubleshooting | Common errors and fixes | `references/faq-and-troubleshooting.md` |
|
| FAQ & Troubleshooting | Common errors and fixes | `references/faq-and-troubleshooting.md` |
|
||||||
|
|
||||||
## Templates
|
## Templates
|
||||||
|
|||||||
@@ -0,0 +1,61 @@
|
|||||||
|
# CrewAI Flows — Event-Driven Orchestration
|
||||||
|
|
||||||
|
Flows connect multiple Crews into event-driven workflows with state management, resumption, and conditional branching.
|
||||||
|
|
||||||
|
## Basic Flow
|
||||||
|
|
||||||
|
```python
|
||||||
|
from crewai.flow.flow import Flow, listen, start
|
||||||
|
|
||||||
|
class MyFlow(Flow):
|
||||||
|
@start()
|
||||||
|
def begin(self):
|
||||||
|
print("Flow started")
|
||||||
|
return {"data": "initial"}
|
||||||
|
|
||||||
|
@listen(begin)
|
||||||
|
def process_data(self, state):
|
||||||
|
print(f"Processing: {state['data']}")
|
||||||
|
# Launch a crew here
|
||||||
|
return {"result": "processed"}
|
||||||
|
|
||||||
|
flow = MyFlow()
|
||||||
|
result = flow.kickoff()
|
||||||
|
```
|
||||||
|
|
||||||
|
## State Management with @persist
|
||||||
|
|
||||||
|
```python
|
||||||
|
from crewai.flow.flow import Flow, listen, start, persist
|
||||||
|
|
||||||
|
@persist # State persists across executions
|
||||||
|
class PersistentFlow(Flow):
|
||||||
|
counter: int = 0 # Tracked state
|
||||||
|
|
||||||
|
@start()
|
||||||
|
def increment(self):
|
||||||
|
self.counter += 1
|
||||||
|
return {"counter": self.counter}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Connecting Multiple Crews
|
||||||
|
|
||||||
|
```python
|
||||||
|
class ResearchFlow(Flow):
|
||||||
|
@start()
|
||||||
|
def research(self):
|
||||||
|
crew = Crew(agents=[researcher], tasks=[research_task], process=Process.sequential)
|
||||||
|
return crew.kickoff()
|
||||||
|
|
||||||
|
@listen(research)
|
||||||
|
def write_report(self, state):
|
||||||
|
crew = Crew(agents=[writer], tasks=[write_task], process=Process.sequential)
|
||||||
|
return crew.kickoff()
|
||||||
|
```
|
||||||
|
|
||||||
|
## Key Features
|
||||||
|
|
||||||
|
- **Event-driven:** `@listen` decorator triggers on completion of upstream steps
|
||||||
|
- **State management:** `@persist` enables state to survive across executions
|
||||||
|
- **Restoration:** `restore_from_state_id` to resume flows from checkpoints
|
||||||
|
- **Multiple crews:** Connect separate crews into a single orchestrated workflow
|
||||||
@@ -0,0 +1,52 @@
|
|||||||
|
# CrewAI Memory System
|
||||||
|
|
||||||
|
CrewAI v1.15+ uses a unified `Memory` class that replaces separate short-term, long-term, entity, and external memory types with a single intelligent API.
|
||||||
|
|
||||||
|
## Enabling Memory
|
||||||
|
|
||||||
|
```python
|
||||||
|
from crewai import Crew
|
||||||
|
|
||||||
|
crew = Crew(
|
||||||
|
agents=[agent1, agent2],
|
||||||
|
tasks=[task1, task2],
|
||||||
|
memory=True, # Enables unified memory for all agents
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
## How Memory Works
|
||||||
|
|
||||||
|
When `memory=True` is set at the Crew level:
|
||||||
|
- **Memory is shared** — all agents in the crew can access context from prior tasks
|
||||||
|
- **Short-term persistence** — within a single crew execution, agents remember context across tasks
|
||||||
|
- **Entity tracking** — the system tracks entities (people, places, concepts) mentioned across agent conversations
|
||||||
|
- **Long-term patterns** — across multiple crew runs, the system learns from successful patterns
|
||||||
|
|
||||||
|
## Memory Configuration
|
||||||
|
|
||||||
|
```python
|
||||||
|
from crewai import Crew, MemoryConfig
|
||||||
|
|
||||||
|
crew = Crew(
|
||||||
|
agents=[agent1, agent2],
|
||||||
|
tasks=[task1, task2],
|
||||||
|
memory=True,
|
||||||
|
memory_config=MemoryConfig(
|
||||||
|
embedder="openai", # Embedding provider for memory storage
|
||||||
|
dimensions=1536, # Embedding dimensions
|
||||||
|
),
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Memory Reset
|
||||||
|
|
||||||
|
```python
|
||||||
|
crew.reset_memories() # Clear all stored memory
|
||||||
|
```
|
||||||
|
|
||||||
|
## Practical Patterns
|
||||||
|
|
||||||
|
- **Within a single crew run:** Memory is automatic. Agents reference prior task outputs through `context`.
|
||||||
|
- **Across crew runs:** Memory enables the system to learn from past execution patterns.
|
||||||
|
- **For state-dependent tools:** Set `cache=False` on tools that shouldn't return cached results.
|
||||||
|
- **For long-running systems:** Periodically call `reset_memories()` to prevent memory bloat.
|
||||||
@@ -0,0 +1,28 @@
|
|||||||
|
# CrewAI Skill — Research Validation Audit
|
||||||
|
|
||||||
|
**Date:** 2026-07-09
|
||||||
|
**Sources:** docs.crewai.com
|
||||||
|
|
||||||
|
## Claims Verified Correct
|
||||||
|
|
||||||
|
| Claim | Source | Status |
|
||||||
|
|-------|--------|--------|
|
||||||
|
| Agent: role, goal, backstory, llm, tools, verbose, allow_delegation, max_iter | docs.crewai.com | ✓ |
|
||||||
|
| Task: description, expected_output, agent, tools, context, human_input, callback | docs.crewai.com | ✓ |
|
||||||
|
| Crew: agents, tasks, process, manager_llm, verbose, memory, cache, planning | docs.crewai.com | ✓ |
|
||||||
|
| Sequential process: tasks run in order | docs.crewai.com | ✓ |
|
||||||
|
| Hierarchical process: manager delegates and validates, requires manager_llm | docs.crewai.com | ✓ |
|
||||||
|
| max_iter default: 15 | docs.crewai.com | ✓ |
|
||||||
|
| @tool decorator with type hints | docs.crewai.com | ✓ |
|
||||||
|
| crewai-tools package for built-in tools | docs.crewai.com | ✓ |
|
||||||
|
|
||||||
|
## Claims Updated by Source Audit
|
||||||
|
|
||||||
|
- **Memory:** CrewAI v1.15+ uses a unified `Memory` class replacing separate short-term, long-term, entity, and external memory types. The skill mentioned `memory=True` without documenting the unified system.
|
||||||
|
- **Flows:** Event-driven with `@listen` decorator, state management via `@persist`, resumption via `restore_from_state_id`. The skill mentioned Flows in one sentence.
|
||||||
|
|
||||||
|
## Missing from Skill (Addressed in This Enrichment)
|
||||||
|
|
||||||
|
- Unified Memory class documentation
|
||||||
|
- Flows system: @listen decorator, state management, event-driven patterns
|
||||||
|
- Crew training patterns
|
||||||
+3
-1
@@ -8,7 +8,7 @@ description: >-
|
|||||||
license: MIT
|
license: MIT
|
||||||
metadata:
|
metadata:
|
||||||
author: Magnus Hedemark
|
author: Magnus Hedemark
|
||||||
version: 1.0.3
|
version: 1.1.0
|
||||||
source: https://dspy.ai
|
source: https://dspy.ai
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -108,6 +108,8 @@ answer = compiled_qa(question="What is DSPy?").answer
|
|||||||
| Compilation Guide | Caching, cost management, save/load | `references/compilation-guide.md` |
|
| Compilation Guide | Caching, cost management, save/load | `references/compilation-guide.md` |
|
||||||
| Agent Patterns | ReAct agent, tool-use, AvatarOptimizer | `references/agent-patterns.md` |
|
| Agent Patterns | ReAct agent, tool-use, AvatarOptimizer | `references/agent-patterns.md` |
|
||||||
| FAQ & Troubleshooting | Common errors and fixes | `references/faq-and-troubleshooting.md` |
|
| FAQ & Troubleshooting | Common errors and fixes | `references/faq-and-troubleshooting.md` |
|
||||||
|
| Validation Audit | Research validation of all API claims | `references/validation-audit.md` |
|
||||||
|
| Worked RAG Example | Full RAG compilation with expected output | `references/example-rag-compilation.md` |
|
||||||
|
|
||||||
## Template Files
|
## Template Files
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,76 @@
|
|||||||
|
# DSPy — Worked Example: Full RAG Compilation
|
||||||
|
|
||||||
|
This example shows a complete DSPy program from definition through compilation, with expected output annotations.
|
||||||
|
|
||||||
|
## Program Definition
|
||||||
|
|
||||||
|
```python
|
||||||
|
import dspy
|
||||||
|
from dspy.datasets import DataLoader
|
||||||
|
|
||||||
|
lm = dspy.LM("openai/gpt-4o-mini")
|
||||||
|
dspy.configure(lm=lm)
|
||||||
|
|
||||||
|
class RAG(dspy.Module):
|
||||||
|
def __init__(self, k=3):
|
||||||
|
self.retrieve = dspy.Retrieve(k=k)
|
||||||
|
self.generate = dspy.ChainOfThought("context, question -> answer")
|
||||||
|
|
||||||
|
def forward(self, question):
|
||||||
|
context = "\n".join(self.retrieve(question).passages)
|
||||||
|
return self.generate(question=question, context=context)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Dataset
|
||||||
|
|
||||||
|
```python
|
||||||
|
trainset = [
|
||||||
|
dspy.Example(question="What is DSPy?", answer="A compiler for prompt programs.").with_inputs("question"),
|
||||||
|
dspy.Example(question="What is a signature?", answer="Input/output field pairs defining a task.").with_inputs("question"),
|
||||||
|
dspy.Example(question="What is MIPROv2?", answer="Bayesian optimizer for joint instruction and demo tuning.").with_inputs("question"),
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
|
## Compilation
|
||||||
|
|
||||||
|
```python
|
||||||
|
from dspy.teleprompt import MIPROv2
|
||||||
|
|
||||||
|
def correct(example, pred, trace=None):
|
||||||
|
return example.answer in pred.answer
|
||||||
|
|
||||||
|
optimizer = MIPROv2(metric=correct, auto="light")
|
||||||
|
compiled = optimizer.compile(RAG(), trainset=trainset)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Expected compile output:**
|
||||||
|
- Compiler output logs showing: bootstrapping demos, proposing instruction candidates, evaluating candidates, selecting best
|
||||||
|
- Typical run: ~30-60 seconds, ~150-300 API calls (auto="light")
|
||||||
|
- Output: a compiled module with `_compiled = True` flag set
|
||||||
|
|
||||||
|
## Inference
|
||||||
|
|
||||||
|
```python
|
||||||
|
# Use the compiled program
|
||||||
|
result = compiled(question="What is DSPy compiling?")
|
||||||
|
|
||||||
|
# Expected result structure:
|
||||||
|
print(result) # dspy.Prediction object
|
||||||
|
print(result.answer) # The generated answer string
|
||||||
|
# ChainOfThought also provides:
|
||||||
|
print(result.reasoning) # The reasoning chain used
|
||||||
|
|
||||||
|
# Save for deployment
|
||||||
|
compiled.save("rag_program.json")
|
||||||
|
|
||||||
|
# Later, reload
|
||||||
|
loaded_rag = RAG()
|
||||||
|
loaded_rag.load("rag_program.json")
|
||||||
|
```
|
||||||
|
|
||||||
|
## Expected Output Depth
|
||||||
|
|
||||||
|
- Uncompiled: Answer based solely on LM training data, no optimization
|
||||||
|
- BootstrapFewShot: Higher quality, uses successful traces as demos
|
||||||
|
- MIPROv2: Highest quality, optimized instructions + demos together
|
||||||
|
- Cost: ~$0.50-2.00 for auto="light" on gpt-4o-mini
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
# DSPy Skill — Research Validation Audit
|
||||||
|
|
||||||
|
**Date:** 2026-07-09
|
||||||
|
**Sources:** dspy.ai, github.com/stanfordnlp/dspy
|
||||||
|
|
||||||
|
## Claims Verified Correct
|
||||||
|
|
||||||
|
| Claim | Source | Status |
|
||||||
|
|-------|--------|--------|
|
||||||
|
| `dspy.Predict(signature)` — direct prediction | dspy.ai docs | ✓ |
|
||||||
|
| `dspy.ChainOfThought(signature)` — with reasoning | dspy.ai docs | ✓ |
|
||||||
|
| `dspy.ReAct(signature, tools=tools, max_iters=10)` — tool-use agent | dspy.ai ReAct page | ✓ |
|
||||||
|
| Custom module via `class MyModule(dspy.Module)` with `forward()` | dspy.ai custom_module tutorial | ✓ |
|
||||||
|
| BootstrapFewShot, BootstrapRS, MIPROv2, GEPA, COPRO optimizers | dspy.ai optimizer guide | ✓ |
|
||||||
|
| `DSPY_CACHEDIR` for current cache, `DSP_CACHEDIR` for legacy | dspy.ai FAQ | ✓ |
|
||||||
|
| `_compiled = True` flag prevents sub-module re-optimization | dspy.ai optimizer guide | ✓ |
|
||||||
|
| `.compile()` returns new copy, original not mutated | dspy.ai optimizer guide | ✓ |
|
||||||
|
| Tool functions need docstring + type hints | dspy.ai customer_service_agent tutorial | ✓ |
|
||||||
|
|
||||||
|
## Claims Verified and Corrected
|
||||||
|
|
||||||
|
| Claim | Correction | Source |
|
||||||
|
|-------|-----------|--------|
|
||||||
|
| `max_iters` parameter on ReAct | Confirmed: exists, default not documented | dspy.ai ReAct page |
|
||||||
|
|
||||||
|
## Missing from Skill (Addressed in This Enrichment)
|
||||||
|
|
||||||
|
- Worked example showing full RAG compilation with expected output
|
||||||
|
- `max_iters` on ReAct documented
|
||||||
|
- Custom agent pattern (not just ReAct — full Module subclass)
|
||||||
+3
-1
@@ -8,7 +8,7 @@ description: >-
|
|||||||
license: MIT
|
license: MIT
|
||||||
metadata:
|
metadata:
|
||||||
author: Magnus Hedemark
|
author: Magnus Hedemark
|
||||||
version: 1.0.3
|
version: 1.1.0
|
||||||
source: https://docs.haystack.deepset.ai
|
source: https://docs.haystack.deepset.ai
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -90,6 +90,8 @@ result = pipeline.run({"embedder": {"text": "What is Haystack?"}, "prompt_builde
|
|||||||
| Pipeline Design | Building indexing and query pipelines | `references/pipeline-design.md` |
|
| Pipeline Design | Building indexing and query pipelines | `references/pipeline-design.md` |
|
||||||
| Document Stores | Store selection and configuration | `references/document-stores.md` |
|
| Document Stores | Store selection and configuration | `references/document-stores.md` |
|
||||||
| Retrievers | Embedding, BM25, hybrid retrieval | `references/retrievers.md` |
|
| Retrievers | Embedding, BM25, hybrid retrieval | `references/retrievers.md` |
|
||||||
|
| Validation Audit | Research validation of all API claims | `references/validation-audit.md` |
|
||||||
|
| File Converters | Multi-format indexing, YAML serialization, component types | `references/file-converters.md` |
|
||||||
| Evaluation | Metrics, evaluators, pipeline evaluation | `references/evaluation.md` |
|
| Evaluation | Metrics, evaluators, pipeline evaluation | `references/evaluation.md` |
|
||||||
| Deployment | Hayhooks, containerization, production | `references/deployment.md` |
|
| Deployment | Hayhooks, containerization, production | `references/deployment.md` |
|
||||||
| FAQ & Troubleshooting | Common errors and fixes | `references/faq-and-troubleshooting.md` |
|
| FAQ & Troubleshooting | Common errors and fixes | `references/faq-and-troubleshooting.md` |
|
||||||
|
|||||||
@@ -0,0 +1,95 @@
|
|||||||
|
# Haystack File Converters and Multi-Format Indexing
|
||||||
|
|
||||||
|
Haystack provides type-specific converters for different file formats. Use `FileTypeRouter` to handle mixed-format directories.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from haystack import Pipeline
|
||||||
|
from haystack.components.routers import FileTypeRouter
|
||||||
|
from haystack.components.converters import (
|
||||||
|
TextFileToDocument,
|
||||||
|
MarkdownToDocument,
|
||||||
|
PyPDFToDocument,
|
||||||
|
)
|
||||||
|
from haystack.components.preprocessors import DocumentSplitter, DocumentCleaner
|
||||||
|
from haystack.components.joiners import DocumentJoiner
|
||||||
|
from haystack.components.writers import DocumentWriter
|
||||||
|
```
|
||||||
|
|
||||||
|
## Multi-Format Indexing Pipeline
|
||||||
|
|
||||||
|
```python
|
||||||
|
p = Pipeline()
|
||||||
|
p.add_component("router", FileTypeRouter(mime_types=["text/plain", "application/pdf", "text/markdown"]))
|
||||||
|
p.add_component("text_converter", TextFileToDocument())
|
||||||
|
p.add_component("pdf_converter", PyPDFToDocument())
|
||||||
|
p.add_component("markdown_converter", MarkdownToDocument())
|
||||||
|
p.add_component("joiner", DocumentJoiner())
|
||||||
|
p.add_component("cleaner", DocumentCleaner())
|
||||||
|
p.add_component("splitter", DocumentSplitter(split_by="word", split_length=500))
|
||||||
|
p.add_component("embedder", SentenceTransformersDocumentEmbedder())
|
||||||
|
p.add_component("writer", DocumentWriter(document_store=document_store))
|
||||||
|
|
||||||
|
# Route each file type to its converter
|
||||||
|
p.connect("router.text/plain", "text_converter.sources")
|
||||||
|
p.connect("router.application/pdf", "pdf_converter.sources")
|
||||||
|
p.connect("router.text/markdown", "markdown_converter.sources")
|
||||||
|
p.connect("text_converter.documents", "joiner.documents")
|
||||||
|
p.connect("pdf_converter.documents", "joiner.documents")
|
||||||
|
p.connect("markdown_converter.documents", "joiner.documents")
|
||||||
|
p.connect("joiner.documents", "cleaner.documents")
|
||||||
|
p.connect("cleaner.documents", "splitter.documents")
|
||||||
|
p.connect("splitter.documents", "embedder.documents")
|
||||||
|
p.connect("embedder.documents", "writer.documents")
|
||||||
|
```
|
||||||
|
|
||||||
|
## Available Converters
|
||||||
|
|
||||||
|
| Converter | Format | Dependency |
|
||||||
|
|-----------|--------|------------|
|
||||||
|
| `TextFileToDocument` | .txt | none |
|
||||||
|
| `PyPDFToDocument` | .pdf | pypdf |
|
||||||
|
| `MarkdownToDocument` | .md | markdown-it-py |
|
||||||
|
| `HTMLToDocument` | .html | trafilatura |
|
||||||
|
| `PPTXToDocument` | .pptx | python-pptx |
|
||||||
|
| `DocxToDocument` | .docx | python-docx |
|
||||||
|
| `CSVToDocument` | .csv | pandas |
|
||||||
|
| `JSONToDocument` | .json | none |
|
||||||
|
| `MultiFileConverter` | auto-detect | all above |
|
||||||
|
|
||||||
|
## Pipeline YAML Serialization
|
||||||
|
|
||||||
|
Haystack pipelines can be serialized to/from YAML — a key differentiator from other frameworks.
|
||||||
|
|
||||||
|
```python
|
||||||
|
# Export pipeline as YAML
|
||||||
|
yaml_str = pipeline.dumps()
|
||||||
|
with open("indexing_pipeline.yaml", "w") as f:
|
||||||
|
f.write(yaml_str)
|
||||||
|
|
||||||
|
# Rebuild from YAML
|
||||||
|
from haystack import Pipeline
|
||||||
|
restored = Pipeline.loads(open("indexing_pipeline.yaml").read())
|
||||||
|
|
||||||
|
# Deploy with Hayhooks
|
||||||
|
# hayhooks deploy --file indexing_pipeline.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
## Component Type System
|
||||||
|
|
||||||
|
Each component declares typed input and output slots:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from haystack import component
|
||||||
|
|
||||||
|
@component
|
||||||
|
class MyProcessor:
|
||||||
|
@component.output_types(processed=str)
|
||||||
|
def run(self, text: str) -> dict:
|
||||||
|
return {"processed": text.upper()}
|
||||||
|
|
||||||
|
# Connections must match types
|
||||||
|
# text: str -> output must have 'processed: str'
|
||||||
|
pipeline.connect("processor.processed", "next_component.input_field")
|
||||||
|
```
|
||||||
|
|
||||||
|
Type mismatches are caught by pipeline validation at build time, not runtime.
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
# Haystack Skill — Research Validation Audit
|
||||||
|
|
||||||
|
**Date:** 2026-07-09
|
||||||
|
**Sources:** docs.haystack.deepset.ai, docs.haystack.deepset.ai/reference
|
||||||
|
|
||||||
|
## Claims Verified Correct
|
||||||
|
|
||||||
|
| Claim | Source | Status |
|
||||||
|
|-------|--------|--------|
|
||||||
|
| Pipeline DAG via add_component() + connect() | haystack docs | ✓ |
|
||||||
|
| InMemory, Elasticsearch, Pinecone, Weaviate stores | haystack docs | ✓ |
|
||||||
|
| PromptBuilder uses Jinja2 templates | haystack docs | ✓ |
|
||||||
|
| DeepEvalEvaluator for LLM-based metrics | haystack docs | ✓ |
|
||||||
|
| SASEvaluator for semantic similarity | haystack docs | ✓ |
|
||||||
|
| Hayhooks for REST API deployment | haystack blog | ✓ |
|
||||||
|
| Evaluation as its own pipeline | haystack evaluation guide | ✓ |
|
||||||
|
|
||||||
|
## Missing from Skill (Addressed in This Enrichment)
|
||||||
|
|
||||||
|
- File converter components (TextFileToDocument, PyPDFToDocument, MarkdownToDocument, etc.)
|
||||||
|
- FileTypeRouter for multi-format indexing pipelines
|
||||||
|
- MultiFileConverter for automatic format detection
|
||||||
|
- Pipeline YAML serialization (dumps/loads)
|
||||||
|
- Component type system (input/output slot typing)
|
||||||
|
- Pipeline warm_up() for model loading
|
||||||
Reference in New Issue
Block a user