Skip to content

feat(examples): add reproducible SLAC-FEL tutorial - #147

Open
pmoriano wants to merge 3 commits into
mainfrom
pmoriano/fel-tutorial
Open

pmoriano wants to merge 3 commits into
mainfrom
pmoriano/fel-tutorial

Conversation

@pmoriano

@pmoriano pmoriano commented Nov 11, 2026 •

Copy link
Copy Markdown

Summary

Adds a reproducible SLAC-FEL scientific example to current APEIRON, including a focused drift-adaptation tutorial, the existing pretrained model and scalers, a paper-style replay figure, documentation, and automated tests.

The tutorial uses 14 windows from the December 2025 FEL dataset and accepts the dataset location through data.path, allowing the permanent hosting decision to be incorporated later without changing the workflow.

Motivation & Context

The SLAC-FEL workflow was previously available on a historical development branch and was used for the original paper-reproduction work. This PR ports that workflow to current main and provides a smaller, documented tutorial suitable for new users.

The permanent public dataset host is still being evaluated through AmSC. The implementation therefore uses a configurable local data path and does not commit dataset files.

Approach

  • Port the existing FEL model harness, feature configuration, pretrained model, and scalers from commit 5542efbb47fdfddbb0b3202beaa2ad4c68cc3bd2.
  • Register slac-fel through the existing example factory.
  • Add a focused configuration for data_730.pkl through data_743.pkl.
  • Use Page-Hinkley detection and 200 online-EWC adaptation updates.
  • Add deterministic startup seeding for Python, NumPy, Torch, and CUDA.
  • Add a plotting utility and a checked-in paper-style segmented-replay figure.
  • Retain a configuration for the complete December 2025 reproduction.
  • Add tests for configuration, model assets, window ordering, trailing-window handling, and figure generation.
  • Update the root and examples documentation and the changelog.

The README-driven command-line workflow is canonical. A notebook can be added later as an optional presentation layer without duplicating the implementation.

Screenshots / Logs

The focused replay figure is included at:

examples/slac_fel/figures/focused_drift_adaptation.png

Validated focused-run result:

  • 14 files and 70,000 observations
  • one drift event at data_740.pkl, pipeline step 31
  • one completed 200-update adaptation
  • static MAE: 3.600666 mJ
  • paper-style segmented-replay MAE: 0.282357 mJ
  • improvement: 12.75x
  • CPU runtime: under one minute on the validation system

Two independent seeded runs produced identical scientific metrics and bitwise-identical adapted checkpoint tensors.

API / CLI Changes

  • Adds slac-fel as a supported data.name in the example factory.
  • Adds examples/slac_fel/slac-fel-tutorial.toml.
  • Adds examples/slac_fel/test_tomls/dec2025.toml.
  • Adds direct dependencies on botorch and pyyaml, plus the types-pyyaml development dependency.
  • Applies the existing top-level seed configuration at application startup.
  • No new CLI flags are introduced.

Breaking Changes

  • None.

Security & Privacy

  • No secrets committed
  • Input validation added where needed

No FEL dataset files are included in the repository. The model and scaler artifacts are retained from the existing historical APEIRON commit and were verified by Git object identity before the compatibility changes.

Users should run the example only with trusted model and dataset artifacts because the existing FEL workflow loads PyTorch and pandas serialized files.

Dependencies

Added:

  • botorch>=0.18.1
  • pyyaml>=6.0.3
  • types-pyyaml>=6.0.12.20261103 in the development dependency group

uv.lock was regenerated and validated with the repository-compatible uv version.

Testing Plan

  • Unit tests
  • Integration tests
  • e2e / smoke test
  • Manual steps:
uv run python -m src.main \
  --config examples/slac_fel/slac-fel-tutorial.toml \
  --set data.path=/path/to/fel-tutorial-data

Validation completed:

  • full non-slow test suite: 228 passed, 11 skipped, 1 deselected
  • focused FEL and configuration tests: 39 passed
  • Ruff lint: passed
  • Ruff format check: passed
  • mypy: passed
  • uv lock --check: passed
  • two-run deterministic metrics and checkpoint comparison: passed
  • complete 14-window CPU smoke test: passed without stream wraparound
  • historical-versus-current checkpoint predictions agreed within approximately 1.4e-7 mJ

Documentation

  • Docstrings updated
  • User docs / README updated
  • CHANGELOG entry

Documentation includes dataset expectations, configurable paths, execution instructions, expected results, figure reproduction, scientific interpretation, and implementation provenance.

AI / LLM Assistance

  • AI/LLM tools were primarily used to generate code or artifacts in this PR
  • I have reviewed all such output line-by-line and take full responsibility for its correctness, security, and licensing

AI assistance was used during implementation, documentation, testing, validation design, and review. The contributor remains responsible for the submitted work. The second checkbox will be completed after the final human review.

Checklist

  • Lint and format pass → make lint
  • Types pass (mypy) → make type-check
  • Tests pass (pytest) → make test
  • Backward compatibility considered
  • Adequate comments for tricky parts
  • CI green

Risk & Rollback Plan

Risk is medium. Most changes are isolated to a new scientific example, but the PR also restores application-wide deterministic seeding and adds dependencies.

Rollback consists of reverting the PR. The FEL dataset is external and no data migration or persistent service state is introduced.

Notes for Reviewers

Suggested review focus:

  • Rafael: tutorial structure and consistency with the other APEIRON scientific examples.
  • Sara: FEL scientific workflow, model assets, configuration, and interpretation.
  • Andrew: framework integration and the configurable data-hosting boundary.

The figure reports a paper-style offline segmented replay. It does not claim strict causal availability of the adapted checkpoint at the beginning of the trigger window.

The permanent public dataset URL is intentionally not hard-coded. Once the AmSC hosting option is clarified, the README can be updated without changing the tutorial implementation.

@pmoriano
pmoriano marked this pull request as ready for review November 11, 2026 16:12
@pmoriano
pmoriano requested review from andrewfayres and rz4 November 11, 2026 16:15

@anagainaru anagainaru left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add Sarah or Auralee to review this. Also add Sarah as co-author since this is based on her PR and she deserves the credit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants