Skip to content

feat(tutorials): ship tutorial data + downloadable example systems #706

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
FBumann merged 3 commits into from
Jun 16, 2026
Merged

feat(tutorials): ship tutorial data + downloadable example systems #706

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
df34a1a
feat(tutorials): ship tutorial data + downloadable example systems
FBumann Jun 16, 2026
f95c86b
ci(tutorials): disable checkout credential persistence
FBumann Jun 16, 2026
4e477d4
Merge remote-tracking branch 'origin/main' into feat/tutorial-data-fetch
FBumann Jun 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions .github/workflows/tutorial-data.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
name: Tutorial data

# Builds the pre-built example FlowSystems and uploads them (plus registry.txt) as
# assets to the GitHub release that `flixopt.tutorials.load_example` downloads from.
# Run manually whenever the example systems change. The release tag must match
# `flixopt.tutorials._examples.DATA_RELEASE` (default: tutorial-data-v1).

on:
workflow_dispatch:
inputs:
release_tag:
description: "Release tag to (re)upload the data assets to (must match DATA_RELEASE)."
required: true
default: "tutorial-data-v1"

env:
PYTHON_VERSION: "3.11"

jobs:
build-and-upload:
name: Build and upload example systems
runs-on: ubuntu-24.04
permissions:
contents: write
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0
persist-credentials: false

- uses: astral-sh/setup-uv@v7
with:
version: "0.10.9"
enable-cache: true

- uses: actions/setup-python@v6
with:
python-version: ${{ env.PYTHON_VERSION }}
Comment on lines +26 to +38

@coderabbitai coderabbitai Bot Jun 16, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "Mutable action refs (should be SHA-pinned):"
rg -n '^\s*-\s*uses:\s*.+@v[0-9]+' .github/workflows/tutorial-data.yaml || true

echo
echo "Checkout hardening block:"
rg -n -A5 -B1 'actions/checkout@' .github/workflows/tutorial-data.yaml

echo
echo "Expected: no `@v`* refs, and persist-credentials: false present under checkout."

Repository: flixOpt/flixopt

Length of output: 482

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Get the full workflow file to check permissions
cat -n .github/workflows/tutorial-data.yaml | head -50

Repository: flixOpt/flixopt

Length of output: 2057

Pin action revisions to commit SHAs and disable checkout credential persistence in this write-scoped job.

This workflow has permissions: contents: write at the job level. The three action references use mutable @v* version tags rather than pinned commit SHAs:

  • actions/checkout@v6 (line 26)
  • astral-sh/setup-uv@v7 (line 30)
  • actions/setup-python@v6 (line 35)

Additionally, the checkout action does not set persist-credentials: false. For a write-scoped job, pin each action to a full commit SHA and disable credential persistence to reduce supply-chain and token exposure risk.

🧰 Tools
🪛 zizmor (1.25.2)

[warning] 26-28: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false

(artipacked)

[error] 26-26: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)

(unpinned-uses)

[error] 30-30: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)

(unpinned-uses)

[error] 35-35: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)

(unpinned-uses)

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/tutorial-data.yaml around lines 26 - 37, In the
tutorial-data.yaml workflow file, replace the three mutable action version tags
with pinned commit SHAs for security hardening: change actions/checkout@v6 to
use its full commit SHA, astral-sh/setup-uv@v7 to use its full commit SHA, and
actions/setup-python@v6 to use its full commit SHA. Additionally, add
persist-credentials: false as a parameter to the actions/checkout step to
disable credential persistence and reduce token exposure risk for this
write-scoped job.

Source: Linters/SAST tools


# docs extra provides demandlib/pvlib/holidays used by the example generators.
- name: Install build dependencies
run: uv pip install --system -e ".[docs,full]"

- name: Verify DATA_RELEASE matches the requested tag
run: |
EXPECTED=$(python -c "from flixopt.tutorials._examples import DATA_RELEASE; print(DATA_RELEASE)")
if [[ "$EXPECTED" != "$TAG" ]]; then
echo "::error::DATA_RELEASE is '$EXPECTED' but the workflow was asked to upload to '$TAG'."
echo "Update flixopt/tutorials/_examples.py or pass the matching tag."
exit 1
fi
env:
TAG: ${{ inputs.release_tag }}

- name: Build example systems
run: python scripts/build_tutorial_datasets.py --out-dir dist/tutorial_datasets

- name: Create the data release if it does not exist
run: |
if ! gh release view "$TAG" >/dev/null 2>&1; then
gh release create "$TAG" \
--title "Tutorial data ($TAG)" \
--notes "Pre-built example FlowSystems downloaded by flixopt.tutorials.load_example." \
--prerelease
fi
env:
GH_TOKEN: ${{ github.token }}
TAG: ${{ inputs.release_tag }}

- name: Upload data assets
run: gh release upload "$TAG" dist/tutorial_datasets/* --clobber
env:
GH_TOKEN: ${{ github.token }}
TAG: ${{ inputs.release_tag }}
4 changes: 1 addition & 3 deletions docs/notebooks/02-heat-system.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,9 +57,7 @@
"metadata": {},
"outputs": [],
"source": [
"from data.tutorial_data import get_heat_system_data\n",
"\n",
"data = get_heat_system_data()\n",
"data = fx.tutorials.get_data('heat_system')\n",
"timesteps = data['timesteps']\n",
"heat_demand = data['heat_demand']\n",
"gas_price = data['gas_price']"
Expand Down
4 changes: 1 addition & 3 deletions docs/notebooks/03-investment-optimization.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,9 +82,7 @@
"metadata": {},
"outputs": [],
"source": [
"from data.tutorial_data import get_investment_data\n",
"\n",
"data = get_investment_data()\n",
"data = fx.tutorials.get_data('investment')\n",
"timesteps = data['timesteps']\n",
"solar_profile = data['solar_profile']\n",
"pool_demand = data['pool_demand']\n",
Expand Down
4 changes: 1 addition & 3 deletions docs/notebooks/04-operational-constraints.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,9 +71,7 @@
"metadata": {},
"outputs": [],
"source": [
"from data.tutorial_data import get_constraints_data\n",
"\n",
"data = get_constraints_data()\n",
"data = fx.tutorials.get_data('constraints')\n",
"timesteps = data['timesteps']\n",
"steam_demand = data['steam_demand']"
]
Expand Down
4 changes: 1 addition & 3 deletions docs/notebooks/05-multi-carrier-system.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,9 +83,7 @@
"metadata": {},
"outputs": [],
"source": [
"from data.tutorial_data import get_multicarrier_data\n",
"\n",
"data = get_multicarrier_data()\n",
"data = fx.tutorials.get_data('multicarrier')\n",
"timesteps = data['timesteps']\n",
"electricity_demand = data['electricity_demand']\n",
"heat_demand = data['heat_demand']\n",
Expand Down
4 changes: 1 addition & 3 deletions docs/notebooks/06a-time-varying-parameters.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,9 +77,7 @@
"metadata": {},
"outputs": [],
"source": [
"from data.tutorial_data import get_time_varying_data\n",
"\n",
"data = get_time_varying_data()\n",
"data = fx.tutorials.get_data('time_varying')\n",
"timesteps = data['timesteps']\n",
"outdoor_temp = data['outdoor_temp']\n",
"heat_demand = data['heat_demand']\n",
Expand Down
4 changes: 1 addition & 3 deletions docs/notebooks/07-scenarios-and-periods.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,9 +71,7 @@
"metadata": {},
"outputs": [],
"source": [
"from data.tutorial_data import get_scenarios_data\n",
"\n",
"data = get_scenarios_data()\n",
"data = fx.tutorials.get_data('scenarios')\n",
"timesteps = data['timesteps']\n",
"periods = data['periods']\n",
"scenarios = data['scenarios']\n",
Expand Down
4 changes: 1 addition & 3 deletions docs/notebooks/08a-aggregation.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,9 +59,7 @@
"metadata": {},
"outputs": [],
"source": [
"from data.generate_example_systems import create_district_heating_system\n",
"\n",
"flow_system = create_district_heating_system()\n",
"flow_system = fx.tutorials.load_example('district_heating')\n",

@coderabbitai coderabbitai Bot Jun 16, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

BASE_URL="https://github.com/flixOpt/flixopt/releases/download/tutorial-data-v1"
files=(
  "registry.txt"
  "district_heating.nc"
  "operational.nc"
  "simple.nc"
  "complex.nc"
  "seasonal_storage.nc"
  "multiperiod.nc"
)

for f in "${files[@]}"; do
  code=$(curl -s -o /dev/null -w "%{http_code}" -L "${BASE_URL}/${f}")
  echo "${f} -> HTTP ${code}"
done

Repository: flixOpt/flixopt

Length of output: 247

Tutorial release assets are missing; publish tutorial-data-v1 release before merging.

Line 63 uses fx.tutorials.load_example('district_heating'), which requires district_heating.nc and registry.txt from the tutorial-data-v1 release. Verification confirms all tutorial data files (registry.txt, district_heating.nc, operational.nc, simple.nc, complex.nc, seasonal_storage.nc, multiperiod.nc) are currently unavailable (HTTP 404). The notebook will fail at runtime if this release is not published before merge.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/notebooks/08a-aggregation.ipynb` at line 63, The notebook at line 63
calls fx.tutorials.load_example('district_heating'), which depends on data files
(district_heating.nc, registry.txt, and other referenced tutorial files) that
must be available through the tutorial-data-v1 release. Currently, these files
are returning HTTP 404 errors, meaning the release has not been published.
Before merging this PR, you must publish the tutorial-data-v1 release containing
all required tutorial data files (registry.txt, district_heating.nc,
operational.nc, simple.nc, complex.nc, seasonal_storage.nc, and multiperiod.nc)
so that the load_example function can successfully retrieve the data at runtime.

"flow_system.connect_and_transform() # Align all data as xarray\n",
"\n",
"timesteps = flow_system.timesteps\n",
Expand Down
4 changes: 1 addition & 3 deletions docs/notebooks/08b-rolling-horizon.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,9 +63,7 @@
"metadata": {},
"outputs": [],
"source": [
"from data.generate_example_systems import create_operational_system\n",
"\n",
"flow_system = create_operational_system().transform.resample('1h')\n",
"flow_system = fx.tutorials.load_example('operational').transform.resample('1h')\n",
"flow_system.connect_and_transform() # Align all data as xarray\n",
"\n",
"timesteps = flow_system.timesteps\n",
Expand Down
4 changes: 1 addition & 3 deletions docs/notebooks/08c-clustering.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,9 +54,7 @@
"metadata": {},
"outputs": [],
"source": [
"from data.generate_example_systems import create_district_heating_system\n",
"\n",
"flow_system = create_district_heating_system()\n",
"flow_system = fx.tutorials.load_example('district_heating')\n",
"flow_system.connect_and_transform()\n",
"\n",
"timesteps = flow_system.timesteps\n",
Expand Down
4 changes: 1 addition & 3 deletions docs/notebooks/08c2-clustering-storage-modes.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,9 +63,7 @@
"metadata": {},
"outputs": [],
"source": [
"from data.generate_example_systems import create_seasonal_storage_system\n",
"\n",
"flow_system = create_seasonal_storage_system()\n",
"flow_system = fx.tutorials.load_example('seasonal_storage')\n",
"flow_system.connect_and_transform() # Align all data as xarray\n",
"\n",
"timesteps = flow_system.timesteps\n",
Expand Down
4 changes: 1 addition & 3 deletions docs/notebooks/08d-clustering-multiperiod.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,9 +62,7 @@
"metadata": {},
"outputs": [],
"source": [
"from data.generate_example_systems import create_multiperiod_system\n",
"\n",
"flow_system = create_multiperiod_system()\n",
"flow_system = fx.tutorials.load_example('multiperiod')\n",
"\n",
"print(f'Timesteps: {len(flow_system.timesteps)} ({len(flow_system.timesteps) // 24} days)')\n",
"print(f'Periods: {list(flow_system.periods.values)}')\n",
Expand Down
4 changes: 1 addition & 3 deletions docs/notebooks/08e-clustering-internals.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,11 @@
"metadata": {},
"outputs": [],
"source": [
"from data.generate_example_systems import create_district_heating_system\n",
"\n",
"import flixopt as fx\n",
"\n",
"fx.CONFIG.notebook()\n",
"\n",
"flow_system = create_district_heating_system()\n",
"flow_system = fx.tutorials.load_example('district_heating')\n",
"flow_system.connect_and_transform()"
]
},
Expand Down
8 changes: 2 additions & 6 deletions docs/notebooks/08f-clustering-segmentation.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,9 +80,7 @@
"metadata": {},
"outputs": [],
"source": [
"from data.generate_example_systems import create_district_heating_system\n",
"\n",
"flow_system = create_district_heating_system()\n",
"flow_system = fx.tutorials.load_example('district_heating')\n",
"flow_system.connect_and_transform()\n",
"\n",
"print(f'Timesteps: {len(flow_system.timesteps)}')\n",
Expand Down Expand Up @@ -466,9 +464,7 @@
"metadata": {},
"outputs": [],
"source": [
"from data.generate_example_systems import create_multiperiod_system\n",
"\n",
"fs_multi = create_multiperiod_system()\n",
"fs_multi = fx.tutorials.load_example('multiperiod')\n",
"# Use first week only for faster demo\n",
"fs_multi = fs_multi.transform.isel(time=slice(0, 168))\n",
"\n",
Expand Down
8 changes: 3 additions & 5 deletions docs/notebooks/09-plotting-and-data-access.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,6 @@
"metadata": {},
"outputs": [],
"source": [
"from data.generate_example_systems import create_complex_system, create_multiperiod_system, create_simple_system\n",
"\n",
"import flixopt as fx\n",
"\n",
"fx.CONFIG.notebook()"
Expand All @@ -62,14 +60,14 @@
"# Create and optimize the example systems\n",
"solver = fx.solvers.HighsSolver(mip_gap=0.01, log_to_console=False)\n",
"\n",
"simple = create_simple_system()\n",
"simple = fx.tutorials.load_example('simple')\n",
"\n",
"simple.optimize(solver)\n",
"\n",
"complex_sys = create_complex_system()\n",
"complex_sys = fx.tutorials.load_example('complex')\n",
"complex_sys.optimize(solver)\n",
"\n",
"multiperiod = create_multiperiod_system()\n",
"multiperiod = fx.tutorials.load_example('multiperiod')\n",
"multiperiod.optimize(solver)\n",
"\n",
"print('Created systems:')\n",
Expand Down
3 changes: 2 additions & 1 deletion flixopt/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
# - xr.Dataset.fxstats (from stats_accessor)
import xarray_plotly as _xpx # noqa: F401

from . import clustering, linear_converters, plotting, results, solvers
from . import clustering, linear_converters, plotting, results, solvers, tutorials
from . import stats_accessor as _fxstats # noqa: F401
from .carrier import Carrier, CarrierContainer
from .comparison import Comparison
Expand Down Expand Up @@ -69,6 +69,7 @@
'results',
'linear_converters',
'solvers',
'tutorials',
]

# Initialize logger with default configuration (silent: WARNING level, NullHandler).
Expand Down
24 changes: 24 additions & 0 deletions flixopt/tutorials/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
"""Datasets and example systems for the flixopt tutorials and notebooks.

Two tiers, so every notebook is standalone after ``pip install flixopt`` - no need
to clone the repository or copy files out of GitHub:

* **Synthetic data** (notebooks 01-07) - generated on the fly from numpy/pandas,
no files and no network. Access by name with :func:`get_data`; see :func:`list_data`.

* **Pre-built example systems** (notebooks 08-09) - downloaded (and cached) from the
project's GitHub releases with :func:`load_example`; see :func:`list_examples`.
Needs ``pooch`` (``pip install flixopt[tutorials]``).
"""

from ._examples import ExampleName, list_examples, load_example
from ._tutorial_data import DataName, get_data, list_data

__all__ = [
'DataName',
'get_data',
'list_data',
'ExampleName',
'load_example',
'list_examples',
]
Loading
Loading