validate-predictions
Use Cautionnpx machina-cli add skill hugoguerrap/crypto-claude-desk/validate-predictions --openclawValidate Predictions
Review all pending predictions and check them against current market data.
Workflow
Step 0: Auto-Find Expired Predictions
Delegate using the Task tool with subagent_type: general-purpose and model: opus:
"You are the learning-agent. Read agents/learning-agent.md for your analysis framework. First, use get_crypto_prices() from crypto-data MCP to get current prices for major coins (bitcoin, ethereum, solana, etc.). Then call find_expired_predictions(current_prices='{"BTC/USDT": ..., "ETH/USDT": ...}') from crypto-learning-db to discover predictions whose timeframe has passed. For each expired prediction, reason about how close it was and validate with an NL evaluation using validate_prediction(). Do NOT use the Edit tool."
Step 1: Check Remaining Pending Predictions
Delegate using the Task tool with subagent_type: general-purpose and model: opus:
"You are the learning-agent. Call query_predictions(status='pending') from crypto-learning-db for predictions still within their timeframe. For each prediction:
- Use get_exchange_prices(symbol=...) from crypto-exchange MCP to check current price
- Compare current price against the prediction's target_value
- Report current progress toward or away from target Do NOT use the Edit tool."
Step 2: Present Results
Show a summary table:
## Prediction Validation Report
### Resolved This Check
| ID | Agent | Prediction | Target | Actual | Result |
|----|-------|-----------|--------|--------|--------|
### Still Pending
| ID | Agent | Prediction | Target | Current | Progress | Expires |
|----|-------|-----------|--------|---------|----------|---------|
### Overall Accuracy
- Total predictions: X
- Correct: X (X%)
- Incorrect: X (X%)
- Pending: X
### Track Record by Setup Type
| Setup Type | Total | Correct | Accuracy | Trend |
|-----------|-------|---------|----------|-------|
Source
git clone https://github.com/hugoguerrap/crypto-claude-desk/blob/main/skills/validate-predictions/SKILL.mdView on GitHub Overview
Validate Predictions automates review of pending crypto predictions by comparing them to current market data. It auto-finds expired predictions, checks remaining pending items, and outputs a structured Validation Report to gauge accuracy and progress.
How This Skill Works
Step 0 auto-finds expired predictions by delegating to a general-purpose subagent, retrieves current prices with get_crypto_prices, and identifies expired items via find_expired_predictions. Step 1 queries remaining pending predictions from crypto-learning-db, fetches current prices with get_exchange_prices, and compares them to each target_value to compute progress. Step 2 presents a Markdown-style Prediction Validation Report detailing resolved predictions, still-pending items, overall accuracy, and setup-type trends.
When to Use It
- You need to clear a backlog of pending predictions after a market move or time window closes.
- You want to measure prediction accuracy across multiple setups over a period.
- You are preparing a backtesting or stakeholder-ready report with a standardized format.
- Predictions expired and you need an auditable assessment of how close they were to targets.
- You require a consolidated view of progress and accuracy before finalizing forecasts.
Quick Start
- Step 1: Auto-Find Expired Predictions – use get_crypto_prices to fetch current prices and find_expired_predictions to identify expired items.
- Step 2: Check Remaining Pending Predictions – query crypto-learning-db for status='pending', then compare current prices (get_exchange_prices) to targets.
- Step 3: Present Results – output the Prediction Validation Report with Resolved, Still Pending, Overall Accuracy, and Track Record sections.
Best Practices
- Run auto-find expired predictions first to ensure data freshness before evaluating pending items.
- Use get_crypto_prices for a reliable macro view and get_exchange_prices for symbol-specific checks.
- Compare actual versus target with precise numeric checks and log progress toward or away from targets.
- Publish results using the exact Prediction Validation Report template for consistency.
- Maintain a running track record by setup type to observe trends and improve future predictions.
Example Use Cases
- After a sudden BTC rally, validate all pending BTC/USDT predictions and record how close they were to the new price.
- During a DeFi rally, review pending ETH and SOL predictions against live prices to gauge forecast accuracy.
- Close a prediction window and generate an auditable report showing which predictions hit targets and which missed.
- Build a setup-type performance table to compare forecasting accuracy across different trading strategies.
- Clear a backlog of predictors by producing a comprehensive Validation Report for stakeholders.
