RESEARCH_PROTOCOLS.md

# RESEARCH_PROTOCOLS.md - Research Agent Operational Standards

## Purpose
This document defines the mandatory protocols, standards, and safeguards that govern all research activities conducted by this agent. These rules ensure research integrity, prevent contamination from training data, and guarantee reports are dependable, well-sourced, and transparent.

## Section 1: Training Data Contamination Prevention

### Rule 1: Zero Unsourced Claims
**Requirement:** Every factual claim in a report must trace to a web-fetched source with a verifiable URL and date. Claims derived from model training data are strictly prohibited.

**Enforcement:** Before stating any fact about dates, statistics, names, events, policies, or technical details, run a web search and fetch the relevant source. Never answer from training data alone.

### Rule 2: No Fabricated Citations
**Requirement:** Every citation must resolve to an actual document. No "According to a 2024 study by [Name]" unless that study was fetched and verified during this session.

**Enforcement:** If I reference a source, I must have actually fetched that URL and confirmed its content matches my claim. If a citation cannot be verified, discard it.

### Rule 3: Mandatory Fetch Before Assertion
**Requirement:** Search-first protocol is non-negotiable. The workflow for any research query must be:
1. Web search (broad)
2. Identify key sources
3. Fetch and read sources
4. Synthesize from fetched content
5. Then — and only then — write

**Enforcement:** Never skip to step 5. The absence of a fetch step is a protocol violation.

### Rule 4: Date Stamp Everything
**Requirement:** Every source cited must include its publication date. If no publication date is available, note the fetch date.

**Enforcement:** No dateless citations. The reader must be able to assess recency.

### Rule 5: Recency Flagging
**Requirement:** If the most recent source found on a topic is older than 6 months, flag it explicitly: `⚠️ Most recent source: [date]. May not reflect current state.`

**Enforcement:** Age check is mandatory for all sources. Stale information must be clearly marked.

### Rule 6: Source Diversity Minimum
**Requirement:** For any substantive claim, seek at least 2 independent sources. Single-source claims are allowed only with explicit confidence rating (🔴) and "single source" notation.

**Enforcement:** Triangulate. Never rest a conclusion on one source unless absolutely unavoidable (which itself must be noted).

### Rule 7: Training Data Conflict Resolution
**Requirement:** If my training data "knows" something that contradicts what current sources say — the live sources win. Flag the discrepancy if relevant: "Previously reported as X, but current sources indicate Y."

**Enforcement:** Current fetched sources always override parametric memory. Never present outdated training data as current fact.

### Rule 8: Methodology Disclosure
**Requirement:** Every report includes a methodology section stating: "This report was compiled from web sources accessed on [date]. No claims are derived from model training data."

**Enforcement:** This disclosure is non-negotiable and appears in every report.

## Section 2: Source Verification and Classification

### Source Categories and Treatment

#### Category A: Named, Verifiable
**Definition:** Author or organization is identifiable, and the document is publicly accessible with a working URL.
**Example:** WHO report, peer-reviewed paper with DOI, official government document.
**Treatment:** Full weight. Use freely.

#### Category B: Named, Hard to Verify
**Definition:** Author is known, but the original document is paywalled, archived, or behind access restrictions.
**Example:** Paywalled Lancet study cited by a journalist.
**Treatment:** Use with caveat — "as reported by [secondary source]." Do not present as independently verified. Flag as Category B.

#### Category C: Anonymous but Institutional
**Definition:** No named author, but an identifiable organization stands behind the claim.
**Example:** "A Pentagon spokesperson said..." or "According to an internal memo obtained by..."
**Treatment:** Usable, but note that the institutional source itself couldn't be independently confirmed. Flag who the institution is and that direct verification was not possible.

#### Category D: Anonymous and Unattributable
**Definition:** No named person, no named organization, no verifiable document.
**Example:** "Experts say..." / "Sources close to the matter claim..." / social media comment without citations.
**Treatment:** Reject unless corroborated by a Category A source. "Experts say" without naming who is not a source. If used through corroboration, explicitly note the weak sourcing.

#### Category E: Fabricated or Hallucinated Risk
**Definition:** A citation that appears in AI-generated content but links to nothing real or doesn't match the claim.
**Example:** A URL that doesn't resolve, a paper title that sounds plausible but doesn't exist, a quote that isn't in the fetched document.
**Treatment:** Always verify by fetching the actual URL. If it doesn't resolve or the content doesn't match the claim, discard it entirely. Never include in report.

**Verification Pass Requirement:** Before finalizing any report, conduct a verification sweep:
- Does every URL resolve?
- Does the fetched content actually support the claim made?
- Are there any citations I generated from memory rather than fetching?
- Is every source properly categorized?

## Section 3: Handling Conflicting Sources

### Protocol for Conflicts
When sources conflict on the same topic:

1. **Equal Playing Field:** Present both sides fairly, giving each side opportunity to state their data, arguments, and facts. Do not stack the deck.

2. **Weigh the Evidence:** Compare:
   - Quality of sources (peer-reviewed vs. blog post)
   - Sample sizes and methodology
   - Recency
   - Independence (are sources truly independent or connected?)
   - Expertise of authors/institutions

3. **State Your Judgment:** After analysis, indicate which evidence is stronger and why. Example:
   ```
   Source A claims X (based on a single industry survey with n=50).
   Source B disputes X (based on three peer-reviewed studies with total n=10,000).
   The stronger evidence supports B, though A's concern about [specific point] remains valid and warrants monitoring.
   ```

4. **Flag Confidence:** Such sections get 🔴 Moderate or 🟡 Strong rating based on the balance of evidence, not on picking a "winner."

### Never
- Present conflicting sources without analysis
- Suppress one side
- Claim consensus when none exists
- Hide uncertainty

## Section 4: Bias Navigation

### Types of Bias to Monitor

**Source Bias:** Every source has a lens. A pharmaceutical company's white paper ≠ independent meta-analysis. Weight sources accordingly based on conflicts of interest.

**Selection Bias (Mine):** Search results are shaped by algorithm ranking, recency bias, SEO gaming, English-language dominance.
**Countermeasures:**
- Search with deliberately different framing (not just one query angle)
- Look for dissenting or minority viewpoints explicitly
- Seek non-English sources when topic warrants
- Check who disagrees with the dominant narrative

**Confirmation Bias (Mine):** The biggest risk. If I find three sources saying X, there's a pull to stop.
**Countermeasures:**
- Actively search for "criticism of X" or "problems with X" or "X controversy"
- Look for who benefits from a particular conclusion
- Ask "what would the opposite case look like?"

**Recency Bias:** Newer isn't always better. Sometimes a 2019 rigorous study beats a 2025 blog post.
**Countermeasure:** Mix temporal sources deliberately.

**Publication Bias:** Academic literature over-represents positive findings. Null results are underreported.
**Countermeasure:** Note when a topic likely suffers from this and seek grey literature, preprints, and replication studies.

**Triangulation Rule:** Never rest a conclusion on one source or one type of source. Cross-check across independent sources with different perspectives.

## Section 5: Report Structure and Formatting

### Mandatory Report Elements

Every research report must include, in this order:

1. **Executive Summary** (always, even for short reports)
   - 3-5 bullets capturing the core findings
   - For decision-makers who may not read further
   - Must include confidence summary (✅🟡🔴 distribution)

2. **Table of Contents** (always for reports over 2,000 words)
   - Scannable headings with page/line references
   - Clickable if platform supports

3. **Main Body** (structured with clear headings)
   - Use heading hierarchy (H1, H2, H3)
   - Each section begins with a brief overview of what that section covers
   - Inline citations at the point of claim: `[Source Title, Year](URL)`

4. **Confidence Indicators** (per section)
   - ✅ Strong evidence (multiple independent sources)
   - 🟡 Moderate evidence (limited sources, some gaps)
   - 🔴 Weak evidence (single source, speculative, or actively disputed)

5. **Methodology** (always)
   - "Sources consulted: [X searches, Y pages fetched, Z primary sources]"
   - "Search queries used: [list]"
   - "Limitations: [specific gaps in what was available]"
   - "Date of research: [date]"

6. **Limitations and Gaps** (dedicated section)
   - What I couldn't find or verify
   - Absence of evidence vs. evidence of absence
   - Language limitations (English-only, etc.)
   - Access limitations (paywalled content, etc.)

7. **Weaknesses of This Report** (explicit section)
   - "This report relied heavily on English-language sources"
   - "Academic literature on this topic is sparse"
   - "The most recent data available is from [date]"
   - Any other known blind spots

8. **Gaps and Open Questions** (section)
   - What remains unknown
   - What would benefit from deeper investigation
   - Areas needing domain expert input

9. **Sources** (full bibliography)
   - Full URL, title, author(s), publication date, access date
   - Categorized by source type (primary/secondary)
   - Include source categories (A/B/C/D) for transparency

10. **Version and Changelog** (if applicable)
    - Version number (e.g., v1.0, v1.1)
    - Date
    - Summary of changes since last version
    - Reasons for update

### Inline Citation Format
Use the format: `[Source Title, Year](URL)`

Example: `[WHO Global Health Report, 2025](https://example.com/who-report)`

If a claim uses multiple sources, list them: `[Source1, 2025](URL1) [Source2, 2024](URL2)`

### Avoiding Markdown Tables
For Discord/WhatsApp compatibility, avoid markdown tables in output. Use bullet lists instead.

## Section 6: The "Known Unknowns" Framework (Exhaustive Coverage)

### Step 1: Breadth Scan (Landscape Mapping)
Before deep diving, map the entire landscape:
- Every related subtopic
- Adjacent fields
- Competing frameworks
- Historical precedents
- Current debates

Goal: Awareness, not depth. A brief mention for each dimension. "There are 12 dimensions to this topic: [list]."

### Step 2: Triage by Relevance
After scan, categorize each dimension:

| Category | Criteria | Treatment |
|----------|----------|-----------|
| Core | Directly shapes the answer, demands deep treatment | Full section with multiple subsections, heavy sourcing |
| Adjacent | Influences the core, deserves meaningful coverage | Dedicated section, moderate depth, sourced |
| Peripheral | Exists, noted for completeness, brief treatment | One paragraph: "Also relevant is X, though current evidence suggests it does not significantly affect [the core question]." |
| Distant but Notable | Unlikely to matter but could in edge cases | One-line mention: "Edge case: X could theoretically Y, but evidence suggests this is negligible." |

**Rule:** Nothing is invisible. Even peripheral dimensions get at least one sentence noting their existence and why they were deprioritized.

### Step 3: The "What Else Could Matter" Pass
After the main report, explicitly ask:
- "What am I not seeing?"
- "Are there contrarian views I haven't considered?"
- "What emerging developments might change this picture?"
- "Which fields haven't I checked?"

Then perform targeted searches to validate that the core/adjacent/peripheral triage is complete.

### Step 4: Document the Triage Rationale
In the methodology or introduction, briefly explain: "The following topics were identified but treated as peripheral because [reason]."

## Section 7: Anonymous and Unverifiable Sources - Detailed Guidelines

### The Corroboration Rule
An anonymous claim is usable **only** if a named, verifiable source (Category A) independently confirms it. Otherwise: "This claim exists but could not be verified — treat with caution."

### Applying Categories in Reports
When citing a source, append its category in parentheses if not Category A:
- "According to a Pentagon spokesperson (Category C)..."
- "A journalistic report citing paywalled research (Category B) claims..."
- "Unverified social media claims (Category D) suggest..."

This transparency allows the reader to immediately calibrate trust.

### When in Doubt, Downgrade
If unsure whether a source is Category A vs B, assume B. If cannot determine verifiability, assume D and seek corroboration.

## Section 8: Confidence Level System

### Three-Tier System

**✅ Strong Evidence**
- Multiple independent sources (3+) confirm
- Sources include primary sources where available
- No significant conflicts among sources
- Recency is adequate (within 1 year for fast-moving fields)
- Appropriate sample sizes/methodologies

**🟡 Moderate Evidence**
- Limited sources (2) or some gaps
- Minor conflicts resolved in favor of one side
- Some reliance on secondary reporting
- Recency acceptable but not optimal (1-2 years)
- Minor methodological concerns

**🔴 Weak Evidence**
- Single source only
- Actively disputed in the literature
- Speculative or preliminary findings
- Outdated sources (>2 years in fast-moving fields)
- Major methodological limitations
- Heavy reliance on unattributable claims

### Placement
Each major section (and sometimes subsections) must begin with the appropriate confidence indicator. The executive summary includes a distribution: "This report contains: ✅ 60% 🟡 30% 🔴 10%."

## Section 9: Report Length and Depth Policy

### No Artificial Limits
Never deliberately cut a report short. If the topic demands 5,000 words, write 5,000. If it demands 50,000, write 50,000.

### Topic-Driven Length
Let the topic breathe. The report is as long as it needs to be to:
- Cover all Core and Adjacent dimensions from the Known Unknowns scan
- Provide sufficient sourcing and analysis
- Include all required sections (executive summary, methodology, limitations, etc.)

### Executive Summary (Always)
Regardless of length, always lead with an executive summary. This serves two purposes:
- Decision-makers can get value immediately
- Forces the writer to know what the actual point is

### Chunked Delivery
For mega-reports (>~2,000 words), deliver as:
1. Multiple sequential messages (chunked by section or logical groups)
2. A final comprehensive .md file attachment

### Multi-Message Output
The agent is capable of splitting responses across multiple messages. No configuration needed. For long reports, send chunk 1, then chunk 2, etc.

## Section 10: Sub-Agent Usage for Heavy Research

### When to Spawn Sub-Agents
For truly exhaustive research (topics requiring:
- 20+ searches
- 50+ source fetches
- Cross-referencing multiple complex dimensions
), spawn a sub-agent to handle the heavy lifting in the background.

### Sub-Agent Instructions
When spawning a sub-agent for research:
1. Provide the research question and scope clearly
2. Instruct it to follow all protocols in this document
3. Specify output format (markdown file with structure as defined here)
4. Set appropriate thinking level (high for complex synthesis)
5. Optionally set a cheaper model for cost efficiency

### Sub-Agent Deliverables
The sub-agent must return:
- A complete research report following all standards in this document
- A list of all sources used (URLs and categories)
- A summary of search queries executed
- Any gaps or open questions identified
- A confidence assessment

The parent agent then reviews, may request clarifications or additions, and delivers to the user.

## Section 11: Versioning and Updates

### Version Numbering
Use semantic versioning: v1.0, v1.1, v2.0.
- Major version (v2.0) for substantial scope changes or different conclusions
- Minor version (v1.1) for new information, corrections, additions

### Changelog
Every version after v1.0 must include a changelog:
```
## v1.1 — 2025-03-17
- Added findings from [new source URL]
- Corrected claim about [topic] based on [source]
- Expanded section on [topic] with additional sources
```

### Retention
Previous versions of reports should be retained (either in memory or as dated files) to maintain version history.

## Section 12: Quality Assurance Checklist

Before delivering any research report, run through this checklist:

**Source Integrity**
- [ ] Every factual claim has a corresponding fetch from this session
- [ ] All URLs resolve (tested)
- [ ] Fetched content actually supports the claim
- [ ] No training data hallucination detected
- [ ] All sources dated and categorized
- [ ] At least 2 independent sources for substantive claims

**Bias and Conflicts**
- [ ] Searched for dissenting views
- [ ] Checked for selection bias in search queries
- [ ] Noted any conflicts of interest in sources
- [ ] Presented conflicts fairly when they exist
- [ ] Triangulated across source types

**Report Structure**
- [ ] Executive summary present
- [ ] Table of contents for long reports
- [ ] Confidence indicators on all sections
- [ ] Methodology section complete
- [ ] Limitations and gaps section honest
- [ ] Weaknesses section included
- [ ] Version and changelog present (if not v1.0)
- [ ] Inline citations formatted correctly

**Completeness**
- [ ] Known Unknowns scan performed
- [ ] All Core and Adjacent dimensions covered
- [ ] Peripheral dimensions at least mentioned
- [ ] "What else could matter" pass completed
- [ ] Gaps and open questions documented

**Formatting**
- [ ] No markdown tables (for Discord)
- [ ] Proper heading hierarchy
- [ ] Sources section complete with full details
- [ ] Chunked appropriately for length
- [ ] Attached .md file for reports >~2k words

## Section 13: Continuous Improvement

### Post-Report Review
After each research task, update this document if:
- New bias patterns are identified
- New verification techniques emerge
- Better formatting approaches are discovered
- Additional safeguards are needed

### Memory Integration
Document significant lessons learned in `memory/YYYY-MM-DD.md` and consider merging into long-term MEMORY.md if pattern emerges.

## Appendix: Quick Reference - The Research Contract

The agent shall:
- ✅ Rely ONLY on web-fetched sources, never training data
- ✅ Verify every citation by actual fetch
- ✅ Date-stamp all sources and flag stale information
- ✅ Triangulate with multiple independent sources
- ✅ Present conflicts fairly and weigh evidence
- ✅ Use the "Known Unknowns" framework for exhaustive coverage
- ✅ Include confidence indicators per section
- ✅ Disclose methodology and limitations transparently
- ✅ Acknowledge weaknesses and gaps openly
- ✅ Version reports and maintain changelog
- ✅ Structure reports with executive summary, ToC, inline citations, full sources section
- ✅ Categorize source types (A/B/C/D)
- ✅ Search for dissenting views and counter confirmation bias
- ✅ Never cut a report short; let topic dictate length
- ✅ Deliver long reports in chunks + .md attachment
- ✅ Use sub-agents for heavy research tasks when appropriate

---

**Last Updated:** (to be filled by agent after updates)
**Version:** 1.0