PRISM v2.2 - Protocol for Rigorous Investigation of Scientific Mechanisms

Author: Dr. Aneesh Joseph
Implementation: Claude (Anthropic)
Version: 2.2 | December 2025

---

When to Use This Skill

Use PRISM when the user asks to:

• Evaluate scientific hypotheses quantitatively
• Compare multiple hypotheses using Bayesian methods
• Perform rigorous evidence synthesis with meta-analysis
• Calculate posterior probabilities from heterogeneous evidence
• Research and prioritize scientific questions
• Assess treatment options, drug efficacy, or technology predictions

Trigger phrases:

• "Find the best hypothesis for..."
• "Compare these hypotheses..."
• "What's the probability that..."
• "Analyze the evidence for..."
• "Which treatment is most likely to work..."

---

Core Philosophy

Traditional hypothesis testing asks: "Is this result statistically significant?"

PRISM asks: "Given all available evidence, what is the probability this hypothesis is true?"

This shift from binary significance to continuous credence enables:

• Comparison across different evidence types
• Principled evidence accumulation
• Explicit uncertainty quantification
• Optimizer's curse correction for multiple comparisons

---

Mathematical Framework

Bayesian Foundation

PRISM uses Bayes' theorem with log-odds for numerical stability:

log-odds(H|E) = log-odds(H) + log(LR)

Reference Class Priors

Instead of arbitrary priors, PRISM uses empirical base rates with Beta distributions:

• phase2_clinical: 15.3% [9.0%, 23.0%] (based on FDA 2000-2020)
• phase3_clinical: 35.1% [26.2%, 44.7%]
• replication: 40.1% [30.8%, 49.8%] (based on OSF 2015)
• general: 50.0% (uninformative)

Hierarchical Correlation Correction

Critical: Naive multiplication of likelihood ratios causes exponential overconfidence.

PRISM uses hierarchical correlation:

• Within-cluster ρ = 0.6 (same lab/method)
• Between-cluster ρ = 0.2 (independent labs)

Effective N = Σ(cluster_size / DEFF_within) / √DEFF_between

Key Statistical Methods

1. REML Meta-Analysis with Hartung-Knapp adjustment
2. P-Curve Analysis for publication bias detection
3. Kalman Filtering for temporal evidence integration
4. Optimizer's Curse Correction when comparing multiple hypotheses
5. Sobol Sensitivity Analysis to identify critical evidence

---

Implementation Guide

File Structure

/home/claude/
├── prism_v2_2.py          # Core PRISM engine (Dr. Joseph's implementation)
├── prism_session.py       # Session management with checkpointing
└── example_*.py           # Example analyses

/mnt/user-data/outputs/prism_{project}/
├── state.json             # Project state + resume instructions
├── RESUME.md              # Human-readable resume point
├── hypotheses/            # Hypothesis data files
│   ├── h1_*.json
│   └── h2_*.json
└── results/
    ├── comparison.json    # Comparative analysis
    ├── FINAL_REPORT.md    # Complete report
    └── *_results.json     # Individual hypothesis results

Basic Usage Pattern

from prism_session import PRISMSession
from prism_v2_2 import Evidence, Domain

# 1. Create session
session = PRISMSession("project_name")

# 2. Add hypotheses
h1 = session.add_hypothesis(
    hypothesis_id="h1_treatment_a",
    title="Treatment A reduces symptoms by >20%",
    domain=Domain.MEDICAL,
    reference_class="phase2_clinical"
)

# 3. Add evidence (from web_search, papers, etc.)
h1.add_evidence(Evidence(
    id="study1",
    content="RCT shows 25% reduction",
    source="NEJM 2024",
    domain=Domain.MEDICAL,
    study_design="rct",
    sample_size=200,
    supports=True,
    p_value=0.01,
    effect_size=-0.45,
    effect_var=0.0144,  # SE^2
    authors=["Smith"]
))

# Update the saved hypothesis
session._save_hypothesis(h1, 
    session.hypotheses_dir / "h1_treatment_a.json",
    "h1_treatment_a")

# 4. Analyze all hypotheses
session.analyze_all(set_n_compared=True)  # Applies optimizer's curse

# 5. Generate report
session.generate_report()

Workflow for Real Research Questions

When a user asks: "Find the best hypothesis for treating osteoarthritis"

1. Search for hypotheses:

# Use web_search to find treatment options
web_search("osteoarthritis treatment options 2024 clinical trials")
web_search("osteoarthritis systematic reviews meta-analysis")

2. Create session and define hypotheses:

session = PRISMSession("osteoarthritis_treatment_2025")

# Add each candidate hypothesis
h1 = session.add_hypothesis(
    "h1_weight_loss",
    "Weight loss reduces knee OA pain",
    Domain.MEDICAL,
    "replication"
)
# ... add more hypotheses

3. Extract evidence from search results:

# For each relevant study found:
h1.add_evidence(Evidence(
    id="messier_2013",
    content="18-month RCT: 10% weight loss reduced pain 50%",
    source="JAMA 2013;310(12):1263",
    study_design="rct",
    sample_size=454,
    supports=True,
    p_value=0.0001,
    effect_size=-0.48,
    effect_var=0.0144,
    authors=["Messier", "Mihalko"]
))

4. Run complete analysis:

session.analyze_all(set_n_compared=True)

5. Present results:

session.generate_report()
present_files([
    session.results_dir / "FINAL_REPORT.md",
    session.results_dir / "comparison.json"
])

---

Checkpointing and Resumability

Design Principle

"Build for one Claude, checkpoint for many"

The system is designed to complete in ONE session but checkpoints after each hypothesis for safety.

Automatic Checkpointing

After analyzing each hypothesis, PRISM automatically:

1. Saves results to JSON
2. Updates state.json
3. Writes RESUME.md with instructions
4. Estimates tokens used

Resume Protocol

If analysis is interrupted:

from prism_session import PRISMSession

# Load existing project
session = PRISMSession("project_name")  # Automatically loads state

# Continue analysis
session.resume()  # Or session.analyze_all()

The RESUME.md file tells you exactly where you left off:

• Which hypotheses are completed
• Which are pending
• Python code to continue

---

Critical Implementation Rules

1. ALWAYS use web_search when user asks about real-world hypotheses

   • Search for recent studies, clinical trials, systematic reviews
   • Extract evidence from authoritative sources
   • Don't fabricate studies or data

2. ALWAYS checkpoint after each hypothesis

   • Use session._checkpoint_hypothesis() automatically called
   • Saves to /mnt/user-data/outputs/prism_{project}/

3. ALWAYS apply optimizer's curse correction when comparing >1 hypothesis

   • Set n_compared on each hypothesis
   • Or use session.analyze_all(set_n_compared=True)

4. ALWAYS use hierarchical correlation

   • Evidence is automatically clustered by (author, study_design, source)
   • within_rho=0.6, between_rho=0.2

5. ALWAYS decompose uncertainty

   • Statistical (from CI width)
   • Prior (from reference class uncertainty)
   • Model (from correlation assumptions)

6. Token awareness:

   • Typical analysis: 5-10K tokens per hypothesis
   • 8 hypotheses ≈ 40-80K tokens
   • Well within 190K budget for most analyses

---

Study Design Strength

Evidence strength by study design (likelihood ratios):

Study Type	LR+	Interpretation
Meta-analysis	5.4	Very strong evidence
Systematic review	5.4	Very strong evidence
RCT	2.3	Strong evidence
Cohort	1.9	Moderate evidence
Case-control	1.5	Weak evidence
Observational	1.2	Minimal evidence
Expert opinion	1.1	Very weak evidence

---

Interpretation Guidelines

Posterior Probability Scale

Posterior	Interpretation	Action
< 10%	Unlikely	Deprioritize
10-30%	Possible	Gather more evidence
30-70%	Uncertain	Key decision point
70-90%	Probable	Consider acting
> 90%	Highly likely	Act with monitoring

Warning Signs

⚠️ High model uncertainty (>25%): Evidence may be highly correlated
⚠️ P-hacking detected: Effect sizes may be inflated
⚠️ High I² (>75%): Studies measuring different things
⚠️ Extreme posterior (>95%): Likely overconfident
⚠️ Wide credible intervals: Need more evidence

---

Example Output Format

When presenting results to users:

🏆 Best Hypothesis: h1_weight_loss
   Weight loss reduces knee OA pain and improves function
   Posterior (corrected): 88.8%

📋 Full Ranking:
   🥇 h1_weight_loss: 88.8%
   🥈 h2_exercise: 88.5%
   🥉 h3_combination: 67.2%
   ...

📄 Full report available: [link to FINAL_REPORT.md]

---

Limitations

What PRISM Cannot Do

1. Replace domain expertise - requires judgment for prior selection
2. Detect fraud - assumes evidence is honestly reported
3. Handle unknown unknowns - only evaluates provided evidence
4. Guarantee calibration - model assumptions may be wrong

Appropriate Use

✅ Exploratory analysis and hypothesis generation
✅ Research prioritization and resource allocation
✅ Structured evidence synthesis
✅ Treatment comparison and decision support
✅ Teaching Bayesian reasoning

Inappropriate Use

❌ Regulatory approval decisions (use established methods)
❌ Legal proceedings (requires validated forensic tools)
❌ Fully automated decision-making
❌ Single-study evaluation

---

Quick Reference

Available Reference Classes

'phase2_clinical'  # ~15% prior (early drug development)
'phase3_clinical'  # ~35% prior (late drug development)
'drug_approval'    # ~10% prior (FDA approval standard)
'replication'      # ~40% prior (scientific replication)
'general'          # ~50% prior (uninformative)

Key Functions

# Session management
session = PRISMSession("project_name")
h = session.add_hypothesis(id, title, domain, ref_class)
session.analyze_all()
session.generate_report()

# Evidence creation
e = Evidence(id, content, source, domain, study_design,
             sample_size, supports, p_value, effect_size, 
             effect_var, authors)

---

References

1. Gelman et al. (2013) Bayesian Data Analysis
2. Patterson & Thompson (1971) Biometrika - REML
3. Hartung & Knapp (2001) Statistics in Medicine
4. Simonsohn et al. (2014) J Exp Psych: General - P-curve
5. Kahneman & Tversky (1979) - Reference class forecasting
6. Dr. Aneesh Joseph (2025) PRISM v2.2 Scientific Guide

---

PRISM v2.2 - Rigorous hypothesis evaluation for evidence-based science