mirror of
https://github.com/magnus919/agent-skills.git
synced 2026-09-16 22:16:52 +03:00
Magnus HedemarkandGitHub
e863aab851
* feat: add research-methodology skill Signed-off-by: Magnus Hedemark <magnus919@pm.me> * fix: remove nonportable source assumptions Signed-off-by: Magnus Hedemark <magnus919@pm.me> --------- Signed-off-by: Magnus Hedemark <magnus919@pm.me>
3.6 KiB
3.6 KiB
Source Evaluation
How to judge whether a source is credible, relevant, and useful before committing to deeper work with it.
The CRAAP Test
The most widely used framework for evaluating source credibility. Each dimension scored 1 (poor) to 5 (excellent).
| Dimension | What to ask | Low (1-2) | High (4-5) |
|---|---|---|---|
| Currency | When was this published? Has it been updated? Is the field moving fast enough that age matters? | >5 years old in a fast-moving field | Recent, or classic/stable reference |
| Relevance | Does this directly address the research question? Who is the intended audience? | Tangential, or wrong audience | Directly addresses the brief |
| Authority | Who wrote this? What are their credentials? Is the publisher reputable? | Unknown author, non-reputable publisher | Known expert, peer-reviewed, institutional affiliation |
| Accuracy | Is the information supported by evidence? Can it be verified elsewhere? | No sources, claims unsupported | Cited, verifiable, consistent with other sources |
| Purpose | Why does this source exist? To inform, persuade, sell, entertain? Is there bias? | Clear bias, advocacy, or commercial intent | Educational or informational, bias acknowledged |
Thresholds:
- 18-25: High-quality source, use confidently
- 13-17: Adequate, corroborate with another source
- Below 13: Low quality, use with caution or discard
Triangulation
A single source is never enough. Confidence comes from convergence across multiple independent sources.
Types of Triangulation
| Type | What it means | How to use |
|---|---|---|
| Source triangulation | Multiple sources saying the same thing | If 3+ independent sources agree, confidence is high |
| Methodological triangulation | Different methods producing the same conclusion | A paper + a benchmark + a case study all pointing the same way |
| Investigator triangulation | Different researchers/teams reaching the same conclusion | Multiple labs replicating a finding |
| Theory triangulation | Different theoretical frameworks predicting the same outcome | Multiple lenses converging |
The Triangulation Rule
- 3+ independent sources agree: Treat as high confidence
- 2 sources agree, 1 contradicts: Investigate the contradiction — it may be the more interesting finding
- Sources disagree without clear resolution: Report the disagreement, don't force consensus
- Single source on a key claim: Flag as "single source" in the brief, not as established fact
Red Flags That Warrant Caution
| Red flag | What to do |
|---|---|
| Source is the sole origin of a claim | Corroborate before trusting |
| Source has a clear commercial interest | Discount claims that align with their product |
| Source cites itself or a closed loop | Low credibility for external claims |
| Source makes extraordinary claims without extraordinary evidence | Require higher corroboration standard |
| Source is anonymous or unverifiable | Use only as directional signal |
| Source is from a known advocacy organization | Understand the bias, still use if evidence is sound |
Source Types by Reliability
| Tier | Source type | Trust |
|---|---|---|
| 1 | Peer-reviewed research, official documentation, primary sources | High |
| 2 | Industry reports, technical blogs by known practitioners, conference talks | Medium-High |
| 3 | News articles, community discussions, forum posts | Medium |
| 4 | Social media, anonymous posts, marketing content | Low-Directional |
| 5 | Opinion without evidence, satire, known unreliable sources | Discard |