diff --git a/autogen/SKILL.md b/autogen/SKILL.md index dca4c10..99993c7 100644 --- a/autogen/SKILL.md +++ b/autogen/SKILL.md @@ -8,7 +8,7 @@ description: >- license: MIT metadata: author: Magnus Hedemark - version: 1.0.3 + version: 1.1.0 source: https://microsoft.github.io/autogen --- @@ -82,6 +82,8 @@ assistant = AssistantAgent( | Group Chat | RoundRobin, Selector, MagenticOne | `references/group-chat.md` | | Code Execution | Docker, local, cancellation tokens | `references/code-execution.md` | | Tool Integration | register_function, @tool, MCP integration | `references/tool-integration.md` | +| v0.4 Migration | v0.2->v0.4 migration, AgentTool, streaming, termination | `references/v04-migration.md` | +| Validation Audit | Research validation of all API claims | `references/validation-audit.md` | | FAQ & Troubleshooting | Common errors and fixes | `references/faq-and-troubleshooting.md` | ## Templates diff --git a/autogen/references/v04-migration.md b/autogen/references/v04-migration.md new file mode 100644 index 0000000..82f6cfe --- /dev/null +++ b/autogen/references/v04-migration.md @@ -0,0 +1,86 @@ +# AutoGen v0.4 Migration and Advanced Patterns + +AutoGen v0.4 introduced significant API changes from v0.2. This reference covers migration and patterns not found in the v0.2 API. + +## v0.2 → v0.4 Migration + +### v0.2 Pattern (Deprecated) + +```python +# v0.2: UserProxyAgent bundled code execution + human input +from autogen import AssistantAgent, UserProxyAgent + +assistant = AssistantAgent(name="assistant", llm_config=llm_config) +proxy = UserProxyAgent(name="proxy", human_input_mode="NEVER", + code_execution_config={"use_docker": True}) +proxy.initiate_chat(assistant, message="Write Python code") +``` + +### v0.4 Pattern + +```python +# v0.4: Code execution is a separate agent +from autogen_agentchat.agents import AssistantAgent, CodeExecutorAgent +from autogen_agentchat.teams import RoundRobinGroupChat +from autogen_ext.code_executors.local import LocalCommandLineCodeExecutor +from autogen_ext.models.openai import OpenAIChatCompletionClient + +model_client = OpenAIChatCompletionClient(model="gpt-4o-mini") +assistant = AssistantAgent(name="assistant", model_client=model_client, + system_message="You are a helpful assistant.") +executor = CodeExecutorAgent( + name="executor", + code_executor=LocalCommandLineCodeExecutor(work_dir="coding"), +) + +team = RoundRobinGroupChat([assistant, executor]) +result = await team.run(task="Write Python code to calculate pi") +``` + +## AgentTool — Agent as Tool + +```python +from autogen_agentchat.tools import AgentTool + +writer = AssistantAgent(name="writer", model_client=model_client, + system_message="Write well.") +writer_tool = AgentTool(agent=writer) + +assistant = AssistantAgent( + name="assistant", + model_client=model_client, + tools=[writer_tool], + system_message="You are a helpful assistant.", +) +``` + +## Streaming with run_stream() + +```python +stream = assistant.run_stream(task="Tell me a story") +async for message in stream: + print(message) # Each message as it's generated +``` + +## Three human_input_mode Behaviors + +| Mode | Behavior | Use case | +|------|----------|----------| +| `"NEVER"` | No human input requested. Agent runs fully autonomously. | Automated pipelines, batch processing | +| `"ALWAYS"` | Agent asks for human input before every reply. Blocks until input received. | Human-in-the-loop approval gates | +| `"TERMINATE"` | Agent asks for human input only when it's about to terminate (send TERMINATE). | Review final output before closing | + +## Termination Conditions + +```python +from autogen_agentchat.conditions import TextMentionTermination, MaxMessageTermination + +# Stop when agent says TERMINATE +text_termination = TextMentionTermination("TERMINATE") + +# Or stop after N messages +max_termination = MaxMessageTermination(max_messages=10) + +# Combine conditions +# team.run(..., termination_condition=text_termination | max_termination) +``` diff --git a/autogen/references/validation-audit.md b/autogen/references/validation-audit.md new file mode 100644 index 0000000..83de802 --- /dev/null +++ b/autogen/references/validation-audit.md @@ -0,0 +1,33 @@ +# AutoGen Skill — Research Validation Audit + +**Date:** 2026-07-09 +**Sources:** microsoft.github.io/autogen/stable + +## Claims Verified Correct + +| Claim | Source | Status | +|-------|--------|--------| +| `AssistantAgent` with `name`, `system_message`, `model_client` | autogen docs | ✓ | +| `UserProxyAgent` with `human_input_mode`, `code_executor` | autogen docs | ✓ | +| `RoundRobinGroupChat` for fixed-order conversation | autogen docs | ✓ | +| `SelectorGroupChat` with `model_client` for speaker selection | autogen docs | ✓ | +| Docker execution via `DockerCommandLineCodeExecutor` | autogen docs | ✓ | +| Local execution via `LocalCommandLineCodeExecutor` | autogen docs | ✓ | +| Cancellation via `CancellationToken` | autogen docs | ✓ | +| MCP tool integration via `McpWorkbench` | autogen docs | ✓ | + +## Claims Updated by Source Audit + +- **AssistantAgent** is explicitly documented as a "kitchen sink agent for prototyping" — the skill should note its prototyping nature +- **CodeExecutorAgent** is the v0.4 separate agent for code execution, splitting the role that UserProxyAgent filled in v0.2 +- **AgentTool** wraps an entire agent as a tool callable by another agent — important pattern for agent composition +- **Streaming** uses `.run_stream()` with `async for message in stream`, not the older callback approach +- **v0.2->v0.4 migration**: UserProxyAgent in v0.2 becomes `AssistantAgent` + `CodeExecutorAgent` + `RoundRobinGroupChat` in v0.4 + +## Missing from Skill (Addressed in This Enrichment) + +- v0.2 to v0.4 migration patterns +- AgentTool for agent-as-tool composition +- v0.4 streaming via `run_stream()` +- Three human_input_mode behaviors documented with examples +- Validation audit file diff --git a/crewai/SKILL.md b/crewai/SKILL.md index 7061a8c..fb40ed0 100644 --- a/crewai/SKILL.md +++ b/crewai/SKILL.md @@ -8,7 +8,7 @@ description: >- license: MIT metadata: author: Magnus Hedemark - version: 1.0.3 + version: 1.1.0 source: https://docs.crewai.com --- @@ -113,6 +113,8 @@ result = crew.kickoff() | Crew Patterns | Sequential, hierarchical, consensual crews | `references/crew-patterns.md` | | Tool Integration | Creating tools with @tool decorator | `references/tool-integration.md` | | Callbacks | Monitoring agent and task execution | `references/callbacks.md` | +| Memory System | Unified Memory class, cross-agent context | `references/memory-system.md` | +| Flows | Event-driven orchestration connecting crews | `references/flows.md` | | FAQ & Troubleshooting | Common errors and fixes | `references/faq-and-troubleshooting.md` | ## Templates diff --git a/crewai/references/flows.md b/crewai/references/flows.md new file mode 100644 index 0000000..5f81902 --- /dev/null +++ b/crewai/references/flows.md @@ -0,0 +1,61 @@ +# CrewAI Flows — Event-Driven Orchestration + +Flows connect multiple Crews into event-driven workflows with state management, resumption, and conditional branching. + +## Basic Flow + +```python +from crewai.flow.flow import Flow, listen, start + +class MyFlow(Flow): + @start() + def begin(self): + print("Flow started") + return {"data": "initial"} + + @listen(begin) + def process_data(self, state): + print(f"Processing: {state['data']}") + # Launch a crew here + return {"result": "processed"} + +flow = MyFlow() +result = flow.kickoff() +``` + +## State Management with @persist + +```python +from crewai.flow.flow import Flow, listen, start, persist + +@persist # State persists across executions +class PersistentFlow(Flow): + counter: int = 0 # Tracked state + + @start() + def increment(self): + self.counter += 1 + return {"counter": self.counter} +``` + +## Connecting Multiple Crews + +```python +class ResearchFlow(Flow): + @start() + def research(self): + crew = Crew(agents=[researcher], tasks=[research_task], process=Process.sequential) + return crew.kickoff() + + @listen(research) + def write_report(self, state): + crew = Crew(agents=[writer], tasks=[write_task], process=Process.sequential) + return crew.kickoff() +``` + +## Key Features + +- **Event-driven:** `@listen` decorator triggers on completion of upstream steps +- **State management:** `@persist` enables state to survive across executions +- **Restoration:** `restore_from_state_id` to resume flows from checkpoints +- **Multiple crews:** Connect separate crews into a single orchestrated workflow diff --git a/crewai/references/memory-system.md b/crewai/references/memory-system.md new file mode 100644 index 0000000..be169b5 --- /dev/null +++ b/crewai/references/memory-system.md @@ -0,0 +1,52 @@ +# CrewAI Memory System + +CrewAI v1.15+ uses a unified `Memory` class that replaces separate short-term, long-term, entity, and external memory types with a single intelligent API. + +## Enabling Memory + +```python +from crewai import Crew + +crew = Crew( + agents=[agent1, agent2], + tasks=[task1, task2], + memory=True, # Enables unified memory for all agents +) +``` + +## How Memory Works + +When `memory=True` is set at the Crew level: +- **Memory is shared** — all agents in the crew can access context from prior tasks +- **Short-term persistence** — within a single crew execution, agents remember context across tasks +- **Entity tracking** — the system tracks entities (people, places, concepts) mentioned across agent conversations +- **Long-term patterns** — across multiple crew runs, the system learns from successful patterns + +## Memory Configuration + +```python +from crewai import Crew, MemoryConfig + +crew = Crew( + agents=[agent1, agent2], + tasks=[task1, task2], + memory=True, + memory_config=MemoryConfig( + embedder="openai", # Embedding provider for memory storage + dimensions=1536, # Embedding dimensions + ), +) +``` + +## Memory Reset + +```python +crew.reset_memories() # Clear all stored memory +``` + +## Practical Patterns + +- **Within a single crew run:** Memory is automatic. Agents reference prior task outputs through `context`. +- **Across crew runs:** Memory enables the system to learn from past execution patterns. +- **For state-dependent tools:** Set `cache=False` on tools that shouldn't return cached results. +- **For long-running systems:** Periodically call `reset_memories()` to prevent memory bloat. diff --git a/crewai/references/validation-audit.md b/crewai/references/validation-audit.md new file mode 100644 index 0000000..5116205 --- /dev/null +++ b/crewai/references/validation-audit.md @@ -0,0 +1,28 @@ +# CrewAI Skill — Research Validation Audit + +**Date:** 2026-07-09 +**Sources:** docs.crewai.com + +## Claims Verified Correct + +| Claim | Source | Status | +|-------|--------|--------| +| Agent: role, goal, backstory, llm, tools, verbose, allow_delegation, max_iter | docs.crewai.com | ✓ | +| Task: description, expected_output, agent, tools, context, human_input, callback | docs.crewai.com | ✓ | +| Crew: agents, tasks, process, manager_llm, verbose, memory, cache, planning | docs.crewai.com | ✓ | +| Sequential process: tasks run in order | docs.crewai.com | ✓ | +| Hierarchical process: manager delegates and validates, requires manager_llm | docs.crewai.com | ✓ | +| max_iter default: 15 | docs.crewai.com | ✓ | +| @tool decorator with type hints | docs.crewai.com | ✓ | +| crewai-tools package for built-in tools | docs.crewai.com | ✓ | + +## Claims Updated by Source Audit + +- **Memory:** CrewAI v1.15+ uses a unified `Memory` class replacing separate short-term, long-term, entity, and external memory types. The skill mentioned `memory=True` without documenting the unified system. +- **Flows:** Event-driven with `@listen` decorator, state management via `@persist`, resumption via `restore_from_state_id`. The skill mentioned Flows in one sentence. + +## Missing from Skill (Addressed in This Enrichment) + +- Unified Memory class documentation +- Flows system: @listen decorator, state management, event-driven patterns +- Crew training patterns diff --git a/dspy/SKILL.md b/dspy/SKILL.md index 05b53d2..49d8f05 100644 --- a/dspy/SKILL.md +++ b/dspy/SKILL.md @@ -8,7 +8,7 @@ description: >- license: MIT metadata: author: Magnus Hedemark - version: 1.0.3 + version: 1.1.0 source: https://dspy.ai --- @@ -108,6 +108,8 @@ answer = compiled_qa(question="What is DSPy?").answer | Compilation Guide | Caching, cost management, save/load | `references/compilation-guide.md` | | Agent Patterns | ReAct agent, tool-use, AvatarOptimizer | `references/agent-patterns.md` | | FAQ & Troubleshooting | Common errors and fixes | `references/faq-and-troubleshooting.md` | +| Validation Audit | Research validation of all API claims | `references/validation-audit.md` | +| Worked RAG Example | Full RAG compilation with expected output | `references/example-rag-compilation.md` | ## Template Files diff --git a/dspy/references/example-rag-compilation.md b/dspy/references/example-rag-compilation.md new file mode 100644 index 0000000..07c35c8 --- /dev/null +++ b/dspy/references/example-rag-compilation.md @@ -0,0 +1,76 @@ +# DSPy — Worked Example: Full RAG Compilation + +This example shows a complete DSPy program from definition through compilation, with expected output annotations. + +## Program Definition + +```python +import dspy +from dspy.datasets import DataLoader + +lm = dspy.LM("openai/gpt-4o-mini") +dspy.configure(lm=lm) + +class RAG(dspy.Module): + def __init__(self, k=3): + self.retrieve = dspy.Retrieve(k=k) + self.generate = dspy.ChainOfThought("context, question -> answer") + + def forward(self, question): + context = "\n".join(self.retrieve(question).passages) + return self.generate(question=question, context=context) +``` + +## Dataset + +```python +trainset = [ + dspy.Example(question="What is DSPy?", answer="A compiler for prompt programs.").with_inputs("question"), + dspy.Example(question="What is a signature?", answer="Input/output field pairs defining a task.").with_inputs("question"), + dspy.Example(question="What is MIPROv2?", answer="Bayesian optimizer for joint instruction and demo tuning.").with_inputs("question"), +] +``` + +## Compilation + +```python +from dspy.teleprompt import MIPROv2 + +def correct(example, pred, trace=None): + return example.answer in pred.answer + +optimizer = MIPROv2(metric=correct, auto="light") +compiled = optimizer.compile(RAG(), trainset=trainset) +``` + +**Expected compile output:** +- Compiler output logs showing: bootstrapping demos, proposing instruction candidates, evaluating candidates, selecting best +- Typical run: ~30-60 seconds, ~150-300 API calls (auto="light") +- Output: a compiled module with `_compiled = True` flag set + +## Inference + +```python +# Use the compiled program +result = compiled(question="What is DSPy compiling?") + +# Expected result structure: +print(result) # dspy.Prediction object +print(result.answer) # The generated answer string +# ChainOfThought also provides: +print(result.reasoning) # The reasoning chain used + +# Save for deployment +compiled.save("rag_program.json") + +# Later, reload +loaded_rag = RAG() +loaded_rag.load("rag_program.json") +``` + +## Expected Output Depth + +- Uncompiled: Answer based solely on LM training data, no optimization +- BootstrapFewShot: Higher quality, uses successful traces as demos +- MIPROv2: Highest quality, optimized instructions + demos together +- Cost: ~$0.50-2.00 for auto="light" on gpt-4o-mini diff --git a/dspy/references/validation-audit.md b/dspy/references/validation-audit.md new file mode 100644 index 0000000..803c0ad --- /dev/null +++ b/dspy/references/validation-audit.md @@ -0,0 +1,30 @@ +# DSPy Skill — Research Validation Audit + +**Date:** 2026-07-09 +**Sources:** dspy.ai, github.com/stanfordnlp/dspy + +## Claims Verified Correct + +| Claim | Source | Status | +|-------|--------|--------| +| `dspy.Predict(signature)` — direct prediction | dspy.ai docs | ✓ | +| `dspy.ChainOfThought(signature)` — with reasoning | dspy.ai docs | ✓ | +| `dspy.ReAct(signature, tools=tools, max_iters=10)` — tool-use agent | dspy.ai ReAct page | ✓ | +| Custom module via `class MyModule(dspy.Module)` with `forward()` | dspy.ai custom_module tutorial | ✓ | +| BootstrapFewShot, BootstrapRS, MIPROv2, GEPA, COPRO optimizers | dspy.ai optimizer guide | ✓ | +| `DSPY_CACHEDIR` for current cache, `DSP_CACHEDIR` for legacy | dspy.ai FAQ | ✓ | +| `_compiled = True` flag prevents sub-module re-optimization | dspy.ai optimizer guide | ✓ | +| `.compile()` returns new copy, original not mutated | dspy.ai optimizer guide | ✓ | +| Tool functions need docstring + type hints | dspy.ai customer_service_agent tutorial | ✓ | + +## Claims Verified and Corrected + +| Claim | Correction | Source | +|-------|-----------|--------| +| `max_iters` parameter on ReAct | Confirmed: exists, default not documented | dspy.ai ReAct page | + +## Missing from Skill (Addressed in This Enrichment) + +- Worked example showing full RAG compilation with expected output +- `max_iters` on ReAct documented +- Custom agent pattern (not just ReAct — full Module subclass) diff --git a/haystack/SKILL.md b/haystack/SKILL.md index d02c5f7..f71909e 100644 --- a/haystack/SKILL.md +++ b/haystack/SKILL.md @@ -8,7 +8,7 @@ description: >- license: MIT metadata: author: Magnus Hedemark - version: 1.0.3 + version: 1.1.0 source: https://docs.haystack.deepset.ai --- @@ -90,6 +90,8 @@ result = pipeline.run({"embedder": {"text": "What is Haystack?"}, "prompt_builde | Pipeline Design | Building indexing and query pipelines | `references/pipeline-design.md` | | Document Stores | Store selection and configuration | `references/document-stores.md` | | Retrievers | Embedding, BM25, hybrid retrieval | `references/retrievers.md` | +| Validation Audit | Research validation of all API claims | `references/validation-audit.md` | +| File Converters | Multi-format indexing, YAML serialization, component types | `references/file-converters.md` | | Evaluation | Metrics, evaluators, pipeline evaluation | `references/evaluation.md` | | Deployment | Hayhooks, containerization, production | `references/deployment.md` | | FAQ & Troubleshooting | Common errors and fixes | `references/faq-and-troubleshooting.md` | diff --git a/haystack/references/file-converters.md b/haystack/references/file-converters.md new file mode 100644 index 0000000..00dbaee --- /dev/null +++ b/haystack/references/file-converters.md @@ -0,0 +1,95 @@ +# Haystack File Converters and Multi-Format Indexing + +Haystack provides type-specific converters for different file formats. Use `FileTypeRouter` to handle mixed-format directories. + +```python +from haystack import Pipeline +from haystack.components.routers import FileTypeRouter +from haystack.components.converters import ( + TextFileToDocument, + MarkdownToDocument, + PyPDFToDocument, +) +from haystack.components.preprocessors import DocumentSplitter, DocumentCleaner +from haystack.components.joiners import DocumentJoiner +from haystack.components.writers import DocumentWriter +``` + +## Multi-Format Indexing Pipeline + +```python +p = Pipeline() +p.add_component("router", FileTypeRouter(mime_types=["text/plain", "application/pdf", "text/markdown"])) +p.add_component("text_converter", TextFileToDocument()) +p.add_component("pdf_converter", PyPDFToDocument()) +p.add_component("markdown_converter", MarkdownToDocument()) +p.add_component("joiner", DocumentJoiner()) +p.add_component("cleaner", DocumentCleaner()) +p.add_component("splitter", DocumentSplitter(split_by="word", split_length=500)) +p.add_component("embedder", SentenceTransformersDocumentEmbedder()) +p.add_component("writer", DocumentWriter(document_store=document_store)) + +# Route each file type to its converter +p.connect("router.text/plain", "text_converter.sources") +p.connect("router.application/pdf", "pdf_converter.sources") +p.connect("router.text/markdown", "markdown_converter.sources") +p.connect("text_converter.documents", "joiner.documents") +p.connect("pdf_converter.documents", "joiner.documents") +p.connect("markdown_converter.documents", "joiner.documents") +p.connect("joiner.documents", "cleaner.documents") +p.connect("cleaner.documents", "splitter.documents") +p.connect("splitter.documents", "embedder.documents") +p.connect("embedder.documents", "writer.documents") +``` + +## Available Converters + +| Converter | Format | Dependency | +|-----------|--------|------------| +| `TextFileToDocument` | .txt | none | +| `PyPDFToDocument` | .pdf | pypdf | +| `MarkdownToDocument` | .md | markdown-it-py | +| `HTMLToDocument` | .html | trafilatura | +| `PPTXToDocument` | .pptx | python-pptx | +| `DocxToDocument` | .docx | python-docx | +| `CSVToDocument` | .csv | pandas | +| `JSONToDocument` | .json | none | +| `MultiFileConverter` | auto-detect | all above | + +## Pipeline YAML Serialization + +Haystack pipelines can be serialized to/from YAML — a key differentiator from other frameworks. + +```python +# Export pipeline as YAML +yaml_str = pipeline.dumps() +with open("indexing_pipeline.yaml", "w") as f: + f.write(yaml_str) + +# Rebuild from YAML +from haystack import Pipeline +restored = Pipeline.loads(open("indexing_pipeline.yaml").read()) + +# Deploy with Hayhooks +# hayhooks deploy --file indexing_pipeline.yaml +``` + +## Component Type System + +Each component declares typed input and output slots: + +```python +from haystack import component + +@component +class MyProcessor: + @component.output_types(processed=str) + def run(self, text: str) -> dict: + return {"processed": text.upper()} + +# Connections must match types +# text: str -> output must have 'processed: str' +pipeline.connect("processor.processed", "next_component.input_field") +``` + +Type mismatches are caught by pipeline validation at build time, not runtime. diff --git a/haystack/references/validation-audit.md b/haystack/references/validation-audit.md new file mode 100644 index 0000000..1238f5d --- /dev/null +++ b/haystack/references/validation-audit.md @@ -0,0 +1,25 @@ +# Haystack Skill — Research Validation Audit + +**Date:** 2026-07-09 +**Sources:** docs.haystack.deepset.ai, docs.haystack.deepset.ai/reference + +## Claims Verified Correct + +| Claim | Source | Status | +|-------|--------|--------| +| Pipeline DAG via add_component() + connect() | haystack docs | ✓ | +| InMemory, Elasticsearch, Pinecone, Weaviate stores | haystack docs | ✓ | +| PromptBuilder uses Jinja2 templates | haystack docs | ✓ | +| DeepEvalEvaluator for LLM-based metrics | haystack docs | ✓ | +| SASEvaluator for semantic similarity | haystack docs | ✓ | +| Hayhooks for REST API deployment | haystack blog | ✓ | +| Evaluation as its own pipeline | haystack evaluation guide | ✓ | + +## Missing from Skill (Addressed in This Enrichment) + +- File converter components (TextFileToDocument, PyPDFToDocument, MarkdownToDocument, etc.) +- FileTypeRouter for multi-format indexing pipelines +- MultiFileConverter for automatic format detection +- Pipeline YAML serialization (dumps/loads) +- Component type system (input/output slot typing) +- Pipeline warm_up() for model loading