mirror of
https://github.com/magnus919/agent-skills.git
synced 2026-09-18 23:16:38 +03:00
Magnus HedemarkandGitHub
e863aab851
* feat: add research-methodology skill Signed-off-by: Magnus Hedemark <magnus919@pm.me> * fix: remove nonportable source assumptions Signed-off-by: Magnus Hedemark <magnus919@pm.me> --------- Signed-off-by: Magnus Hedemark <magnus919@pm.me>
62 lines
3.6 KiB
Markdown
62 lines
3.6 KiB
Markdown
# Source Evaluation
|
|
|
|
How to judge whether a source is credible, relevant, and useful before committing to deeper work with it.
|
|
|
|
## The CRAAP Test
|
|
|
|
The most widely used framework for evaluating source credibility. Each dimension scored 1 (poor) to 5 (excellent).
|
|
|
|
| Dimension | What to ask | Low (1-2) | High (4-5) |
|
|
|-----------|-------------|-----------|------------|
|
|
| **Currency** | When was this published? Has it been updated? Is the field moving fast enough that age matters? | >5 years old in a fast-moving field | Recent, or classic/stable reference |
|
|
| **Relevance** | Does this directly address the research question? Who is the intended audience? | Tangential, or wrong audience | Directly addresses the brief |
|
|
| **Authority** | Who wrote this? What are their credentials? Is the publisher reputable? | Unknown author, non-reputable publisher | Known expert, peer-reviewed, institutional affiliation |
|
|
| **Accuracy** | Is the information supported by evidence? Can it be verified elsewhere? | No sources, claims unsupported | Cited, verifiable, consistent with other sources |
|
|
| **Purpose** | Why does this source exist? To inform, persuade, sell, entertain? Is there bias? | Clear bias, advocacy, or commercial intent | Educational or informational, bias acknowledged |
|
|
|
|
**Thresholds:**
|
|
- **18-25:** High-quality source, use confidently
|
|
- **13-17:** Adequate, corroborate with another source
|
|
- **Below 13:** Low quality, use with caution or discard
|
|
|
|
## Triangulation
|
|
|
|
A single source is never enough. Confidence comes from convergence across multiple independent sources.
|
|
|
|
### Types of Triangulation
|
|
|
|
| Type | What it means | How to use |
|
|
|------|--------------|------------|
|
|
| **Source triangulation** | Multiple sources saying the same thing | If 3+ independent sources agree, confidence is high |
|
|
| **Methodological triangulation** | Different methods producing the same conclusion | A paper + a benchmark + a case study all pointing the same way |
|
|
| **Investigator triangulation** | Different researchers/teams reaching the same conclusion | Multiple labs replicating a finding |
|
|
| **Theory triangulation** | Different theoretical frameworks predicting the same outcome | Multiple lenses converging |
|
|
|
|
### The Triangulation Rule
|
|
|
|
- **3+ independent sources agree:** Treat as high confidence
|
|
- **2 sources agree, 1 contradicts:** Investigate the contradiction — it may be the more interesting finding
|
|
- **Sources disagree without clear resolution:** Report the disagreement, don't force consensus
|
|
- **Single source on a key claim:** Flag as "single source" in the brief, not as established fact
|
|
|
|
## Red Flags That Warrant Caution
|
|
|
|
| Red flag | What to do |
|
|
|----------|------------|
|
|
| Source is the sole origin of a claim | Corroborate before trusting |
|
|
| Source has a clear commercial interest | Discount claims that align with their product |
|
|
| Source cites itself or a closed loop | Low credibility for external claims |
|
|
| Source makes extraordinary claims without extraordinary evidence | Require higher corroboration standard |
|
|
| Source is anonymous or unverifiable | Use only as directional signal |
|
|
| Source is from a known advocacy organization | Understand the bias, still use if evidence is sound |
|
|
|
|
## Source Types by Reliability
|
|
|
|
| Tier | Source type | Trust |
|
|
|------|-------------|-------|
|
|
| 1 | Peer-reviewed research, official documentation, primary sources | High |
|
|
| 2 | Industry reports, technical blogs by known practitioners, conference talks | Medium-High |
|
|
| 3 | News articles, community discussions, forum posts | Medium |
|
|
| 4 | Social media, anonymous posts, marketing content | Low-Directional |
|
|
| 5 | Opinion without evidence, satire, known unreliable sources | Discard |
|