nanoTSFM

Gray points and dashed lines mark published models on the GIFT-Eval leaderboard; the diamond is STRIDE w/ Synapse, ranked first on the leaderboard, an agentic system. In the two panels by metric, each small gray dot is one verified run and the orange line is the record, the mean of its runs. Records are decided on CRPS.

The task

nanoTSFM is an open benchmark for training small time-series foundation models on a fixed budget. A run trains on GIFT-Eval Pretrain and is then scored on GIFT-Eval datasets it has never seen; the best verified score holds the record. Each try takes minutes and ends in one verified number, so nanoTSFM is also a small environment for recursive self-improvement, where an AI agent runs the loop.

Model: 3.3M parameters in the baseline; free to change
Data: GIFT-Eval Pretrain
Budget: $\leq 3600\,\text{s}$ of training on $1 \times$ A100 80GB
Score: CRPS on GIFT-Eval, zero-shot; lower is better
Submission: $\geq 3$ repeated runs at one commit; a record needs $\geq 0.013$ below the last (rule)

Records

Each record is the mean of its runs, retrained by the maintainers before it counts.

#	Date	Change	GIFT-Eval CRPS	MASE	Runs	Contributors
1	2026-09-29	Simplified Toto 2.0, 5,000 steps on GEP-M	0.6699 ± 0.0073	0.9670	3	@Shu-Wan
2	2026-09-29	Weight every series equally	0.6316 ± 0.0045	0.9088	3	@Shu-Wan

Record details

Record 2: Weight every series equally (0.6316)

The record is the mean of the maintainers' retrains, with new seeds: CRPS 0.6267, 0.6355 and 0.6326. The team's runs:

Seed	GIFT-Eval CRPS	MASE	GEP-Val	GEP-Test	Training	GPU
7	0.6352	0.9044	0.6113	0.6032	91 s	NVIDIA A100-SXM4-80GB
1	0.6296	0.9070	0.6118	0.6078	85 s	NVIDIA A100-SXM4-80GB
2	0.6354	0.9079	0.6160	0.6123	82 s	NVIDIA A100-SXM4-80GB

Change from the previous record

training.source_power: — → 1

Training loss

Report · Commit 399d636 · @Shu-Wan

Record 1: Simplified Toto 2.0, 5,000 steps on GEP-M (0.6699)

Seed	GIFT-Eval CRPS	MASE	GEP-Val	GEP-Test	Training	GPU
7	0.6620	0.9625	0.6276	0.6225	113 s	NVIDIA A100-SXM4-80GB
1	0.6763	0.9714	0.6301	0.6177	107 s	NVIDIA A100-SXM4-80GB
2	0.6715	0.9671	0.6276	0.6287	106 s	NVIDIA A100-SXM4-80GB

Change from the previous record

The starting point: configs/baseline.yaml.

Training loss

Report · Commit e9df886 · @Shu-Wan