A foundation for building, evaluating, and comparing forecasting systems — conventional numerical models, LLM Processes, and agentic forecasters — on real economic, financial, and event-prediction tasks.
The repo has two layers. A small core library (aieng.forecasting) owns cutoff-safe data handling, a shared Predictor interface, reusable methods, and the backtest/evaluation harness. Self-contained reference implementations under implementations/ apply those methods to real forecasting problems — pick the one closest to what you want to build; each directory has its own README.
| # | Implementation | Use case | Methods |
|---|---|---|---|
| 0 | Getting started | Canada CPI gasoline, one month ahead — the smallest end-to-end loop | Naive last-value, AutoARIMA; CRPS via backtest() / evaluate() |
| 1 | S&P 500 | Daily index returns under a leak-safe macro/market covariate panel (1 / 5 / 21 business-day horizons) | Naive, ETS, Kalman, AutoARIMA, linear regression, LightGBM; covariate-aware LLM-Process |
| 2 | Food price forecasting | Multivariate Canadian food CPI in the style of Canada's Food Price Report (nine sub-indices, 12-month trajectory, avg/avg YoY) | Naive last-value, AutoARIMA; report-grounded LLM-Process (quantile grid and sampled trajectory) |
| 3 | Energy / oil | Daily WTI crude under regime-breaking news (continuous trajectory, binary up-shock, scenario analysis) | Prophet, LLM-Process, news-grounded agent, code-executing agent, adaptive (curriculum-trained) agent |
| 4 | BoC rate decisions | Will the Bank of Canada cut, hold, or hike at its next meeting? (ordered categorical; binary cut-vs-not special case) | Climatological frequency, multinomial logistic, categorical LLM-Process, analyst agent; LLM-as-judge reasoning alignment |
Also in this README: Setup · Core concepts · Repository layout · Documentation
👉 First time here? Run the environment check. After
uv sync(see Setup), openimplementations/getting_started/00_environment_check.ipynband run it top to bottom. It's a self-guided preflight that verifies every capability — proxy LLM inference, Langfuse, E2B code execution, StatCan/FRED data access, and an end-to-end mini backtest — and tells you exactly what to fix when something isn't set up. Do this before anything else.
- Core library —
aieng-forecasting(aieng.forecasting): data services, cutoff enforcement, forecasting tasks, prediction payloads, backtesting, evaluation, and artifacts. - Reusable methods —
aieng.forecasting.methods:Predictorimplementations including naive baselines (continuous, binary, and categorical), Darts numerical predictors, LLM-process predictors (continuous, binary-probability, and categorical-probability), and ADK-based agentic infrastructure (build_adk_agent,AdkTextRunner,AgentPredictor). - Reference implementations —
implementations/<use-case>/: notebooks, helper modules, task-specific configuration, and co-located YAML specs. - Tracing — Langfuse / OpenTelemetry bootstrap (
aieng.forecasting.langfuse_tracing) for LiteLLM and Google ADK. - Data scripts —
scripts/: one fetch script per data source, plusbuild_e2b_template.pyfor the agentic code-execution sandbox.
Every method can be used in one of two modes, and the distinction runs through the library:
- Track 1 — evaluated prediction. Numerical methods, LLM Processes, and agentic forecasters emit standardized
Predictionobjects and are compared head-to-head with the evaluation harness (CRPS, Brier, RPS, calibration). - Track 2 — interactive analysis. The same agents can do scenario analysis, monitoring, open-ended Q&A, code-backed analysis, and reasoning over evidence — useful work that isn't reduced to a single score.
Use cases, methods, and links are in the contents table above. Each implementation is independent — pick the problem you care about and read that directory's README.md for the full walkthrough. They are numbered in a recommended order that mirrors the bootcamp progression — conventional numerical methods → LLM Processes → agents → agentic evaluation — but any one stands on its own.
Start here → #0 getting_started/ if the evaluation loop is new to you. That directory also includes 99_repo_concierge.ipynb — a lite-model repo guide for “how does this codebase work?” questions (uv run adk run implementations/getting_started/concierge_agent from the repo root).
Not sure where to start building? Each of the four domain implementations (#1–#4) ends with a 99_starter_agent.ipynb — a fresh, hackable starter agent (a starter_agent/ module) with toggleable news search and code execution, two lightweight tool-usage skills, an interactive cell, and one scored forecast. It's the consistent "continue from here" entry point for taking any reference use case in an agentic direction, and a quick end-to-end test of that use case's agent stack.
- StatCan — Canadian CPI and related macroeconomic series.
- FRED — macroeconomic and commodity series.
- yfinance — equities, indices, and commodity futures.
Historical data is cached locally under data/ and is not committed. Each implementation's README names the fetch script(s) it needs.
Several reference implementations (S&P 500, BoC rate decisions) fetch data from the Federal Reserve Economic Data (FRED) API, which requires a free personal API key. We cannot provide this key for you — each participant must request their own at:
FRED keys are free and approval is typically quick, but it can occasionally take some time, so request yours early. When asked for a use-case description, something extended from the following works well:
"Requesting an API key to explore the effectiveness of various forecasting techniques on economic data."
Once you have the key, add it to your repo-root .env:
FRED_API_KEY=your_fred_api_key
On Coder workspaces, bootcamp keys (OPENAI_*, E2B_*, LANGFUSE_*) live in your shell environment — not in repo .env. See Bootcamp environment.
aieng-forecasting/ # Installable library: import as aieng.forecasting
implementations/ # Self-contained reference implementations + co-located specs
guides/ # Step-by-step strategy guides for common build-phase tasks
scripts/ # Data-fetch scripts + E2B template builder
tests/ # Onboarding integration tests (not run in CI)
planning-docs/ # Architecture notes and the extension/roadmap catalog
playground/ # Exploration and archived demos (not reference implementations)
Install dependencies from the repo root:
git clone <repo-url>. # If running locally. Coder environment setup clones repo automatically.
cd agentic-forecasting
uv sync --devmacOS — LightGBM and OpenMP. The library depends on LightGBM (used by DartsLightGBMPredictor and some notebooks). The PyPI wheel expects OpenMP at runtime. If you see Library not loaded: @rpath/libomp.dylib when importing or training, install Homebrew's OpenMP once and restart your shell or Jupyter kernel:
brew install libompOn Apple Silicon the dylib is typically under /opt/homebrew/opt/libomp/lib/; on Intel Homebrew, /usr/local/opt/libomp/lib/.
When you open a Coder workspace, startup runs automatically in the background. By the time you connect you should have:
- The repo cloned, a Python venv, and dependencies installed
- Bootcamp API keys (
OPENAI_*,E2B_*,LANGFUSE_*) available in your shell (not in.env) - A shell that opens in the repo with the venv activated
Your next step: run 00_environment_check.ipynb top to bottom. That notebook will confirm that startup succeeded.
On first boot, keys are verified against live services and your onboarding status is recorded. Workspace restarts reload keys without re-running the full test suite.
Local machine or troubleshooting — fetch and verify keys manually:
eval "$(onboard --bootcamp-name agentic-forecasting --test-script tests/test_integration.py)"Reload keys in a new shell without re-testing:
eval "$(onboard --bootcamp-name agentic-forecasting --skip-test)"Headless verification (same checks as first-boot onboarding):
uv sync --all-extras --dev --all-packages
uv run pytest tests/test_integration.py -vCredential model: bootcamp keys live in your shell environment. Optional personal keys (e.g. FRED_API_KEY) go in a .env only — see .env.example.
New to the project? Open implementations/getting_started/00_environment_check.ipynb and run it top to bottom. It's a self-guided preflight that checks every major capability — proxy LLM inference, Langfuse, E2B code execution, StatCan/FRED data access, and a full end-to-end mini backtest — one cell at a time, and tells you exactly what to fix when something isn't set up (most often a missing or placeholder key in your .env). It's the fastest way to confirm setup before working through the reference implementations.
Data is fetched once and cached locally (gitignored). Each implementation names the fetch script(s) it needs in its own README.md — for example scripts/fetch_cpi.py (getting started), scripts/fetch_sp500_market.py + scripts/fetch_fred.py (S&P 500), scripts/fetch_wti.py (energy), and scripts/fetch_boc.py and scripts/fetch_boc_press_releases.py (BoC). Run the relevant one before opening that implementation's notebooks:
uv run python scripts/fetch_cpi.pyAgentic forecasters can run code in an E2B cloud sandbox. Credentials for e2b should be automatically injected into the environment for bootcamp participants, and you can confirm successful setup by running 00_environment_check.ipynb.
If this was unsuccessful, or if you prefer to run with E2B in an alternative environment, do this once before enabling code execution in build_adk_agent:
- Create a free account at e2b.dev and copy your API key.
- Add it to your
.envfile alongside the other keys (see.env.example):
E2B_API_KEY=your_e2b_api_key
- Build the template (takes a few minutes on first run):
uv run --env-file .env scripts/build_e2b_template.pyThe template name is the default in CodeExecutionConfig.template_name, so notebooks pick it up automatically.
Predictor is the interface every forecasting method implements:
class MyPredictor(Predictor):
@property
def predictor_id(self) -> str:
return "my_predictor"
def predict(self, task: ForecastingTask, context: ForecastContext) -> list[Prediction]:
series = context.get_series(task.target_series_id)
...
return [Prediction(...)]ForecastContext is cutoff-scoped. Predictors only see observations available as of the forecast origin, which keeps backtests honest.
backtest() is the open iteration loop against historical data. evaluate() is the budgeted protected-window loop.
This repo is a starting point, not a finished product. The shape of a new forecaster is always the same: implement Predictor, declare a spec, and run backtest() / evaluate() to compare it against the baselines. Each reference implementation's README ends with concrete extension ideas; planning-docs/roadmap.md collects the cross-cutting ones (new data sources, additional methods, live forecasting, deeper agent work).
make lint
make formatmake lint runs the expected pre-push quality checks. Git commits do not run hooks locally. To mirror the full pre-commit suite, run:
uv run pre-commit run --all-files- Per-implementation READMEs under
implementations/— the primary user surface. guides/— self-contained, step-by-step strategy guides for the most common build-phase tasks: onboarding a dataset, creating an experiment, customizing an agent's strategy, and auditing a result before you believe it.- Architecture atlas (source) — a self-contained visual atlas of the system architecture: the loop, the temporal fence, predictor families, the harness, agent anatomy, and how each reference implementation instantiates them.
aieng-forecasting/README.mdandaieng-forecasting/aieng/forecasting/methods/README.md— the library and the method catalog.planning-docs/roadmap.md— architecture principles and extension ideas.
Keep code, notebooks, specs, and these docs in sync when you change behavior, setup, layout, or datasets.