Forecast Backtest
Score past forecasts against what actually happened — accuracy over time.
Scores a ForecastSim in this application against what actually happened since it last ran.
It reads the forecasts the named module already produced — lookbackRuns of them — and joins them to actuals, so accuracy comes from the runs that really happened rather than a re-simulation. Durable and long-running, because it walks a history of runs.
At a glance
Module type
ForecastBacktest
Category
Intelligence
Default tier
durable
Runs on
reactive, durable
Starts a flow
no
Holds a body
no
Keeps state between runs
no
Available in
every application type
Parameters
forecastModuleId
string · required · defaults to ""
The ForecastSim in this application whose past runs are scored.
lookbackRuns
number · required · defaults to 12
How many of that module's runs to go back over.
horizons
number[] · optional
1-based lead times to score; omit for adaptive defaults from forecastH + cadence.
primaryHorizon
number · optional
KPI / trend horizon; must be in horizons (or adaptive list).
metrics
string[] · optional · defaults to ["mape","wmape","mae","mse","rmse","bias","coverage"]
Metrics to compute (mape, wmape, mae, mse, rmse, bias, coverage).
primaryMetric
string · optional · defaults to "mape"
KPI / trend metric; must be in metrics.
Inputs
on
event
no
Implicit on every module that does not itself start a flow: wire an event here to make this module the start of what that event runs.
Outputs
comparison
dataframe
dataframe, signal
One row per forecast point beside the actual it is now scored against.
metrics
dataframe
dataframe, signal
One row per scored run: every metric in metrics, for the run as a whole.
byHorizon
dataframe
dataframe, signal
The same metrics split by lead time, so a 1-month error is not averaged with a 12-month one.
summary
dataframe
dataframe, signal
The latest run's accuracy alone, for a KPI tile.
Last updated