This folder contains example scripts that integrate third-party tools and SDKs with the Metaculus forecasting bot template.
Integration dependencies are in an optional poetry group. Install them with:
poetry install --with integrationsmain_lightningrod_eval.py - Uses the LightningRod SDK to generate forecasting questions from news.
The Lightning Rod SDK is a Python library for generating custom forecasting datasets. It transforms real-world data sources into labeled forecasting samples automatically, using built-in integrations like Google News, or your own documents.
The output is an exportable dataset you can use to benchmark LLMs, or train on to improve calibration and sharpen reasoning. The SDK covers the full pipeline: ingesting sources, generating questions, labeling questions, and scoring against real outcomes.
Data generation pipelines are fully customizable: including date ranges, question format (e.g. binary, multiple choice), and custom instructions to shape the output.
The Metaculus community gets $100 in free credits with code METACULUS100. Create an account at lightningrod.ai and explore the SDK examples to get started.
- Get a
LIGHTNINGROD_API_KEYat dashboard.lightningrod.ai - Add it to your
.envfile - Run:
poetry run python integrations/main_lightningrod_eval.pyHave a tool or SDK that could help with forecasting? Add it here and open a PR!