skills/magnus919_agent-skills
1
0
Fork 0
mirror of https://github.com/magnus919/agent-skills.git synced 2026-09-11 19:47:12 +03:00
Code Issues Packages Projects Releases Wiki Activity

feat: add haystack — expert skill for production search pipelines

Browse Source 
Greenfield SkillOpt: 3 epochs for deepset Haystack skill.
Pipeline DAG model, document stores, retrievers, evaluation, deployment.

Epoch 1 — Prominence: Hard-gate on Pipeline DAG vs LCEL pipe model
Epoch 2 — Decision Guidance: Where to Start, Framework Routing Guide
Epoch 3 — Pattern Expansion: Hybrid RAG pattern, evaluation pipeline, deployment

11 files: SKILL.md, 6 references, 3 templates, 1 script.
This commit is contained in:
Magnus Hedemark
2026-07-09 14:53:43 -04:00
parent 43bed4d72b
commit fe5b275d00
11 changed files with 567 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
haystack/SKILL.md
+114
View File
@@ -0,0 +1,114 @@
---
name: haystack
description: >-
Expert skill for production search and NLP pipelines with Haystack (deepset).
Pipeline DAG composition, document stores, retrievers, PromptBuilder (Jinja2),
generators, evaluation, Hayhooks deployment. Use when building search pipelines
or comparing NLP application frameworks.
license: MIT
metadata:
author: Magnus Hedemark
version: 1.0.3
source: https://docs.haystack.deepset.ai
---
# Haystack Expert Skill
Haystack (by deepset) is a production-oriented framework for building search and NLP pipelines. Its core abstraction is the **Pipeline** — a directed acyclic graph of typed components with explicit connections. Unlike LangChain's LCEL (pipe operator) or LlamaIndex's query engines, Haystack pipelines are **declared upfront with add_component and connect**, giving validated, debuggable DAGs.
## Core Paradigm
```python
from haystack import Pipeline
from haystack.components.embedders import SentenceTransformersTextEmbedder
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
from haystack.components.builders import PromptBuilder
from haystack.components.generators import OpenAIGenerator
from haystack.document_stores.in_memory import InMemoryDocumentStore
# Build a pipeline
document_store = InMemoryDocumentStore()
pipeline = Pipeline()
pipeline.add_component("embedder", SentenceTransformersTextEmbedder())
pipeline.add_component("retriever", InMemoryEmbeddingRetriever(document_store=document_store))
pipeline.add_component("prompt_builder", PromptBuilder(template="Answer using: {{documents}}\n\nQuestion: {{question}}"))
pipeline.add_component("generator", OpenAIGenerator())
# Connect components
pipeline.connect("embedder.embedding", "retriever.query_embedding")
pipeline.connect("retriever.documents", "prompt_builder.documents")
pipeline.connect("prompt_builder", "generator")
# Run
result = pipeline.run({"embedder": {"text": "What is Haystack?"}, "prompt_builder": {"question": "What is Haystack?"}})
```
## Core Principles
1. **Pipelines are validated DAGs.** add_component + connect. Pipeline validation catches errors BEFORE execution — leverage this during development.
2. **Components are typed.** Each component has input/output slots. Connections must match types. This prevents runtime errors.
3. **PromptBuilder uses Jinja2.** Templates are Jinja2 strings, not f-strings. `{{documents}}`, `{{query}}`, `{{question}}` are variable placeholders.
4. **Indexing and query are separate pipelines.** One pipeline loads/cleans/embeds/writes documents. Another retrieves/generates answers. They share the DocumentStore.
5. **Evaluation is a pipeline too.** Add evaluator components to measure faithfulness, relevancy, or custom metrics.
## Where to Start
| You already have... | Start here |
|---|---|
| Nothing — exploring Haystack | Build a basic indexing + query pipeline |
| Documents to index | Build an indexing pipeline (converters, splitter, embedder, writer) |
| A search use case | Build a query pipeline (embedder, retriever, prompt, generator) |
| A production deployment | Add Hayhooks + evaluation pipeline |
## Quick Reference
| Task | Approach | Reference |
|------|----------|-----------|
| Build indexing pipeline | add_component -> connect -> run | `references/pipeline-design.md` |
| Build query pipeline | retriever -> prompt_builder -> generator | `references/pipeline-design.md` |
| Choose document store | InMemory (dev), Elasticsearch/Pinecone (prod) | `references/document-stores.md` |
| Embedding retrieval | SentenceTransformersTextEmbedder + EmbeddingRetriever | `references/retrievers.md` |
| Hybrid retrieval | BM25 + Embedding in parallel, DocumentJoiner | `references/retrievers.md` |
| Prompt templates | Jinja2 in PromptBuilder | `references/pipeline-design.md` |
| Evaluation | DeepEvalEvaluator, SASEvaluator | `references/evaluation.md` |
| Deploy | Hayhooks REST API | `references/deployment.md` |
## Framework Routing Guide
| Scenario | Reach for | Why |
|----------|-----------|-----|
| Search / NLP pipelines | **Haystack** | Pipeline DAG model is most mature for retrieval-heavy workloads |
| Documents to query / RAG | **LlamaIndex** | Data ingestion is the primary primitive |
| Chain/agent composition | **LangChain** | LCEL pipe operator for general chain building |
| Compiled prompt programs | **DSPy** | Auto-optimizes prompts against a metric |
| Role-based multi-agent | **CrewAI** | Higher-level agent abstraction |
## Reference Files
| Reference | Load when | File |
|-----------|-----------|------|
| Pipeline Design | Building indexing and query pipelines | `references/pipeline-design.md` |
| Document Stores | Store selection and configuration | `references/document-stores.md` |
| Retrievers | Embedding, BM25, hybrid retrieval | `references/retrievers.md` |
| Evaluation | Metrics, evaluators, pipeline evaluation | `references/evaluation.md` |
| Deployment | Hayhooks, containerization, production | `references/deployment.md` |
| FAQ & Troubleshooting | Common errors and fixes | `references/faq-and-troubleshooting.md` |
## Templates
| Template | When to use | File |
|----------|-------------|------|
| Indexing Pipeline | Load, split, embed, write to store | `templates/indexing-pipeline.py` |
| Query Pipeline | Retrieve, prompt, generate answer | `templates/query-pipeline.py` |
| Hybrid RAG | BM25 + embedding in parallel | `templates/hybrid-rag.py` |
## Troubleshooting
| Symptom | Likely cause | Fix | Reference |
|---------|-------------|-----|-----------|
| Pipeline run errors | Component connection mismatch | Check component input/output slot types | `references/pipeline-design.md` |
| No documents retrieved | Empty document store | Run indexing pipeline first | `references/pipeline-design.md` |
| Prompt not rendering | Wrong variable name in Jinja2 template | Check {{variables}} match pipeline input | `references/pipeline-design.md` |
| Slow retrieval | Full scan instead of ANN | Configure approximate nearest neighbor index | `references/retrievers.md` |
| Embedding mismatch | Different models for indexing vs query | Use same model in both pipelines | `references/retrievers.md` |
| Hayhooks not starting | Port conflict or missing config | Check port, run with --help for options | `references/deployment.md` |
haystack/references/deployment.md
+45
View File
@@ -0,0 +1,45 @@
# Haystack Deployment
## Hayhooks
Hayhooks turns Haystack pipelines into REST APIs:
```bash
pip install hayhooks
hayhooks run # Starts server on port 1416
```
Deploy a pipeline:
```python
# deploy.py
from hayhooks import deploy
deploy("my_pipeline.yaml") # Serialized pipeline YAML
# Then use curl:
# curl -X POST http://localhost:1416/my_pipeline \
# -H "Content-Type: application/json" \
# -d '{"text_embedder": {"text": "query"}}'
```
## MCP Server
Hayhooks also exposes pipelines as MCP servers, enabling any MCP client to use your Haystack pipeline as a tool.
## Containerization
```dockerfile
FROM python:3.11-slim
RUN pip install haystack hayhooks
COPY pipelines/ /app/pipelines/
CMD ["hayhooks", "run", "--host", "0.0.0.0"]
```
## Production Checklist
- [ ] Use a production document store (not InMemory)
- [ ] Separate indexing and query pipelines
- [ ] Set up Hayhooks for REST API access
- [ ] Add evaluation pipeline for monitoring
- [ ] Containerize with Docker
- [ ] Configure logging and error tracking
- [ ] Set up model caching to avoid reloading on every request
haystack/references/document-stores.md
+51
View File
@@ -0,0 +1,51 @@
# Haystack Document Stores
Document stores are the persistence layer. All share the same write/query interface.
## Available Stores
| Store | Production | Setup |
|-------|-----------|-------|
| `InMemoryDocumentStore` | Dev only | Built-in, no setup |
| `ElasticsearchDocumentStore` | Yes | `pip install elasticsearch-haystack`, running ES cluster |
| `PineconeDocumentStore` | Yes | `pip install pinecone-haystack`, API key |
| `WeaviateDocumentStore` | Yes | `pip install weaviate-haystack`, running Weaviate |
| `PGVectorStore` | Yes | `pip install pgvector-haystack`, PostgreSQL instance |
| `ChromaDocumentStore` | Dev | `pip install chroma-haystack` |
## Common Operations
```python
# Write documents
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack import Document
doc_store = InMemoryDocumentStore()
doc_store.write_documents([
Document(content="Haystack is a framework for building search systems."),
Document(content="It uses pipeline-based architecture.")
])
# Query (BM25 by default)
results = doc_store.query("What is Haystack?", top_k=3)
```
## Metadata Filtering
```python
from haystack.document_stores.filters import document_store_filter
filtered = doc_store.filter_documents({
"field": "meta.source",
"operator": "==",
"value": "internal"
})
```
## Store Selection Guide
- **InMemoryDocumentStore** — prototyping, testing, small datasets
- **ElasticsearchDocumentStore** — production search at scale, full-text + vector
- **PineconeDocumentStore** — serverless vector search, large-scale embedding retrieval
- **WeaviateDocumentStore** — hybrid search with built-in vectorization
- **PGVectorStore** — if you already use PostgreSQL, minimal infrastructure overhead
haystack/references/evaluation.md
+53
View File
@@ -0,0 +1,53 @@
# Haystack Evaluation
## Evaluation Pipeline
Evaluation in Haystack is a pipeline itself — add evaluator components to measure your pipeline's outputs.
```python
from haystack import Pipeline
from haystack.components.evaluators import DeepEvalEvaluator, DeepEvalMetric, SASEvaluator
eval_pipeline = Pipeline()
eval_pipeline.add_component("faithfulness", DeepEvalEvaluator(
metric=DeepEvalMetric.FAITHFULNESS,
metric_params={"model": "gpt-4o-mini"}
))
```
## Available Evaluators
| Evaluator | What it measures | Type |
|-----------|-----------------|------|
| `DeepEvalEvaluator` | Faithfulness, relevancy, context recall | LLM-as-judge |
| `SASEvaluator` | Semantic answer similarity | Embedding-based |
| `LLMEvaluator` | Custom criteria via instruction + examples | LLM-as-judge |
| `DocumentMAPEvaluator` | Mean average precision for retrieval | Statistical |
## Evaluation Workflow
```python
from haystack import Pipeline
from haystack.components.evaluators import SASEvaluator
# Run your query pipeline
results = query_pipeline.run(...)
# Build evaluation pipeline
eval_pipeline = Pipeline()
eval_pipeline.add_component("sa_eval", SASEvaluator())
eval_result = eval_pipeline.run({
"sa_eval": {
"predicted_answers": [results["generator"]["replies"][0]],
"golden_answers": ["Expected answer text"]
}
})
print(eval_result["sa_eval"]["score"])
```
## Best Practices
- Evaluate on a held-out golden dataset (not your training queries)
- Use multiple metrics — faithfulness catches hallucinations, relevancy catches retrieval misses
- Build evaluation into CI/CD for regression detection
- For production, schedule periodic evaluation runs against new data
haystack/references/faq-and-troubleshooting.md
+39
View File
@@ -0,0 +1,39 @@
# Haystack FAQ and Troubleshooting
## Installation
**Q: Installation fails?**
A: `pip install haystack-ai` (not `haystack` — that's an older, deprecated package).
**Q: Module not found for integration?**
A: Install integration packages separately: `pip install elasticsearch-haystack pinecone-haystack weaviate-haystack chroma-haystack`.
## Common Errors
**Q: Pipeline.run() returns empty results?**
A: Check that your indexing pipeline actually ran and wrote documents. Verify with `document_store.count_documents()`.
**Q: "Component X has no output slot Y"?**
A: Connection mismatch. Each component has typed input/output slots. Check the component's documentation for slot names.
**Q: Prompt rendering issues?**
A: PromptBuilder uses Jinja2. Variable names must match what you pass in `pipeline.run()`. `{{documents}}` vs `{{docs}}` is a common error.
**Q: Embedding mismatch between indexing and query?**
A: Use the same model in both `SentenceTransformersDocumentEmbedder` and `SentenceTransformersTextEmbedder`. Different models produce incompatible embeddings.
## Performance
**Q: Retrieval too slow?**
A: For production, use a vector database with ANN indexing (Elasticsearch, Pinecone, Weaviate). InMemory scales poorly beyond ~100K documents.
**Q: Pipeline warm-up too slow?**
A: Model loading happens on `warm_up()`. For production, warm up once and reuse the pipeline instance.
## Deployment
**Q: How to deploy Haystack?**
A: Use Hayhooks. Serialize your pipeline to YAML, deploy via Hayhooks REST API.
**Q: Can I use multiple pipelines?**
A: Yes — run separate Hayhooks instances or use a proxy to route requests.
haystack/references/pipeline-design.md
+89
View File
@@ -0,0 +1,89 @@
# Haystack Pipeline Design
Haystack uses a **Pipeline** abstraction — a validated directed acyclic graph (DAG) of typed components.
## Basic Structure
```python
from haystack import Pipeline
pipeline = Pipeline()
pipeline.add_component("name", SomeComponent())
pipeline.connect("source_component.output_slot", "target_component.input_slot")
result = pipeline.run({"source_component": {"input_param": value}})
```
## Indexing Pipeline
```python
from haystack import Pipeline
from haystack.components.converters import TextFileToDocument
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.embedders import SentenceTransformersDocumentEmbedder
from haystack.components.writers import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore
document_store = InMemoryDocumentStore()
indexing = Pipeline()
indexing.add_component("converter", TextFileToDocument())
indexing.add_component("splitter", DocumentSplitter(split_by="word", split_length=500))
indexing.add_component("embedder", SentenceTransformersDocumentEmbedder())
indexing.add_component("writer", DocumentWriter(document_store=document_store))
indexing.connect("converter.documents", "splitter.documents")
indexing.connect("splitter.documents", "embedder.documents")
indexing.connect("embedder.documents", "writer.documents")
indexing.run({"converter": {"sources": ["docs.txt"]}})
```
## Query Pipeline
```python
query = Pipeline()
query.add_component("text_embedder", SentenceTransformersTextEmbedder())
query.add_component("retriever", InMemoryEmbeddingRetriever(document_store=document_store))
query.add_component("prompt_builder", PromptBuilder(template="Context: {{documents}}\nQ: {{question}}\nA:"))
query.add_component("generator", OpenAIGenerator())
query.connect("text_embedder.embedding", "retriever.query_embedding")
query.connect("retriever.documents", "prompt_builder.documents")
query.connect("prompt_builder", "generator")
result = query.run({
"text_embedder": {"text": "What is Haystack?"},
"prompt_builder": {"question": "What is Haystack?"}
})
```
## Pipeline Validation
Haystack validates the pipeline at build time:
```python
pipeline.warm_up() # Load models, validate connections
pipeline.run(...) # Execute
```
Validation catches: missing connections, type mismatches, required inputs not provided.
## Custom Components
```python
from haystack import component
@component
class MyProcessor:
@component.output_types(processed=str)
def run(self, text: str):
return {"processed": text.upper()}
```
## Pipeline YAML Serialization
Pipelines can be serialized to/from YAML:
```python
pipeline.dumps() # to YAML string
Pipeline.loads(yaml_string) # from YAML string
```
haystack/references/retrievers.md
+58
View File
@@ -0,0 +1,58 @@
# Haystack Retrievers
## Embedding Retrieval
```python
from haystack.components.embedders import SentenceTransformersTextEmbedder
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
# Indexing pipeline uses SentenceTransformersDocumentEmbedder
# Query pipeline uses:
text_embedder = SentenceTransformersTextEmbedder(model="sentence-transformers/all-MiniLM-L6-v2")
retriever = InMemoryEmbeddingRetriever(document_store=document_store, top_k=5)
```
## BM25 Retrieval (Keyword)
```python
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
bm25_retriever = InMemoryBM25Retriever(document_store=document_store, top_k=5)
```
## Hybrid Retrieval
Run BM25 and embedding retrieval in parallel, merge results:
```python
from haystack.components.joiners import DocumentJoiner
pipeline.add_component("bm25_retriever", InMemoryBM25Retriever(document_store=doc_store))
pipeline.add_component("embedding_retriever", InMemoryEmbeddingRetriever(document_store=doc_store))
pipeline.add_component("joiner", DocumentJoiner(join_mode="concatenate")) # or "merge"
pipeline.connect("text_embedder.embedding", "embedding_retriever.query_embedding")
pipeline.connect("bm25_retriever.documents", "joiner.documents")
pipeline.connect("embedding_retriever.documents", "joiner.documents")
```
## Reranking
Add a ranker after retrieval:
```python
from haystack_integrations.components.rankers.cohere import CohereRanker
pipeline.add_component("ranker", CohereRanker(model="rerank-english-v3.0", top_k=3))
pipeline.connect("joiner.documents", "ranker.documents")
pipeline.connect("ranker.documents", "prompt_builder.documents")
```
## Retriever Selection Guide
| Retriever | When to use |
|-----------|-------------|
| EmbeddingRetriever | Semantic search, conceptual queries |
| BM25Retriever | Keyword search, exact phrase matching |
| Hybrid (both + joiner) | Production RAG — best of both worlds |
| + Ranker after hybrid | Highest quality, adds latency |
haystack/scripts/check-setup.py
+29
View File
@@ -0,0 +1,29 @@
#!/usr/bin/env python3
"""Verify Haystack installation."""
import sys
REQUIRED = ["haystack", "haystack_components"]
OPTIONAL = ["hayhooks"]
for pkg in REQUIRED:
try:
__import__(pkg.replace("-", "_"))
print(f" [OK] {pkg}")
except ImportError:
print(f" [FAIL] {pkg} — install with pip install {pkg}")
sys.exit(1)
for pkg in OPTIONAL:
try:
__import__(pkg.replace("-", "_"))
print(f" [OK] {pkg} (optional)")
except ImportError:
print(f" [—] {pkg} (optional, not installed)")
# Test basic pipeline creation
from haystack import Pipeline
p = Pipeline()
print(" [OK] Pipeline creation works")
print("\nHaystack setup check: ALL REQUIRED PACKAGES OK")
haystack/templates/hybrid-rag.py
+35
View File
@@ -0,0 +1,35 @@
#!/usr/bin/env python3
"""Hybrid RAG pipeline — BM25 + embedding in parallel."""
from haystack import Pipeline
from haystack.components.embedders import SentenceTransformersTextEmbedder
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever, InMemoryBM25Retriever
from haystack.components.joiners import DocumentJoiner
from haystack.components.builders import PromptBuilder
from haystack.components.generators import OpenAIGenerator
from haystack.document_stores.in_memory import InMemoryDocumentStore
document_store = InMemoryDocumentStore()
pipeline = Pipeline()
pipeline.add_component("text_embedder", SentenceTransformersTextEmbedder())
pipeline.add_component("bm25_retriever", InMemoryBM25Retriever(document_store=document_store, top_k=5))
pipeline.add_component("embedding_retriever", InMemoryEmbeddingRetriever(document_store=document_store, top_k=5))
pipeline.add_component("joiner", DocumentJoiner(join_mode="merge"))
pipeline.add_component("prompt_builder", PromptBuilder(
template="Context:\n{{documents}}\n\nQuestion: {{question}}\nAnswer:"
))
pipeline.add_component("generator", OpenAIGenerator())
pipeline.connect("text_embedder.embedding", "embedding_retriever.query_embedding")
pipeline.connect("bm25_retriever.documents", "joiner.documents")
pipeline.connect("embedding_retriever.documents", "joiner.documents")
pipeline.connect("joiner.documents", "prompt_builder.documents")
pipeline.connect("prompt_builder", "generator")
result = pipeline.run({
"text_embedder": {"text": "hybrid search query"},
"bm25_retriever": {"query": "hybrid search query"},
"prompt_builder": {"question": "hybrid search query"}
})
print(result["generator"]["replies"][0])
haystack/templates/indexing-pipeline.py
+24
View File
@@ -0,0 +1,24 @@
#!/usr/bin/env python3
"""Haystack indexing pipeline — load, split, embed, write."""
from haystack import Pipeline
from haystack.components.converters import TextFileToDocument
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.embedders import SentenceTransformersDocumentEmbedder
from haystack.components.writers import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore
document_store = InMemoryDocumentStore()
pipeline = Pipeline()
pipeline.add_component("converter", TextFileToDocument())
pipeline.add_component("splitter", DocumentSplitter(split_by="word", split_length=500, split_overlap=50))
pipeline.add_component("embedder", SentenceTransformersDocumentEmbedder())
pipeline.add_component("writer", DocumentWriter(document_store=document_store))
pipeline.connect("converter.documents", "splitter.documents")
pipeline.connect("splitter.documents", "embedder.documents")
pipeline.connect("embedder.documents", "writer.documents")
result = pipeline.run({"converter": {"sources": ["docs.txt"]}})
print(f"Indexed {document_store.count_documents()} documents")
haystack/templates/query-pipeline.py
+30
View File
@@ -0,0 +1,30 @@
#!/usr/bin/env python3
"""Haystack query pipeline — retrieve, prompt, generate."""
from haystack import Pipeline
from haystack.components.embedders import SentenceTransformersTextEmbedder
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
from haystack.components.builders import PromptBuilder
from haystack.components.generators import OpenAIGenerator
from haystack.document_stores.in_memory import InMemoryDocumentStore
# Assume document_store already has documents
document_store = InMemoryDocumentStore()
pipeline = Pipeline()
pipeline.add_component("text_embedder", SentenceTransformersTextEmbedder())
pipeline.add_component("retriever", InMemoryEmbeddingRetriever(document_store=document_store, top_k=5))
pipeline.add_component("prompt_builder", PromptBuilder(
template="Answer based on the context.\n\nContext: {{documents}}\n\nQuestion: {{question}}\nAnswer:"
))
pipeline.add_component("generator", OpenAIGenerator())
pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
pipeline.connect("retriever.documents", "prompt_builder.documents")
pipeline.connect("prompt_builder", "generator")
result = pipeline.run({
"text_embedder": {"text": "What is Haystack?"},
"prompt_builder": {"question": "What is Haystack?"}
})
print(result["generator"]["replies"][0])