add statistics and remove the old ones

Author: piebro

.github/workflows/monthly_data_release.yml

@@ -13,10 +13,11 @@ jobs:
     steps:
       - uses: actions/checkout@v3
 
-      - name: Setup uv
-        run: |
-          curl -LsSf https://astral.sh/uv/install.sh | sh
-          uv sync --python 3.13
+      - name: Install uv
+        uses: astral-sh/setup-uv@v7
+
+      - name: Install the project
+        run: uv sync --python 3.13
 
       - name: Download raw data from Hugging Face
         env:

.github/workflows/update_statistics.yml

@@ -0,0 +1,52 @@
+name: Update Statistics
+
+on:
+  schedule:
+    # Runs at 12:00 UTC on the 3rd day of every month
+    - cron: '0 12 3 * *'
+  workflow_dispatch:
+
+jobs:
+  update-statistics:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout Code
+        uses: actions/checkout@v4
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v7
+
+      - name: Install the project
+        run: uv sync --python 3.13
+
+      - name: Download Monthly Processed Data (Last 3 Months)
+        run: |
+          REPO_ID="piebro/deutsche-bahn-data"
+
+          # Calculate the last 3 months
+          MONTH_1=$(date -d "last month" +"%Y-%m")
+          MONTH_2=$(date -d "2 months ago" +"%Y-%m")
+          MONTH_3=$(date -d "3 months ago" +"%Y-%m")
+
+          echo "Downloading monthly processed data for: $MONTH_3, $MONTH_2, $MONTH_1"
+
+          uv run --with huggingface_hub hf download "$REPO_ID" \
+            --repo-type=dataset \
+            --include "monthly_processed_data/data-$MONTH_1.parquet" \
+            --include "monthly_processed_data/data-$MONTH_2.parquet" \
+            --include "monthly_processed_data/data-$MONTH_3.parquet" \
+            --local-dir .
+
+          echo "Download complete!"
+          ls -la monthly_processed_data/
+
+      - name: Run Notebooks and Generate HTML
+        run: uv run python notebooks/src/nb_to_html.py --run
+
+      - name: Commit and Push
+        run: |
+          git config --local user.email "noreply@github.com"
+          git config --local user.name "GitHub Actions Bot"
+          git add stats/ notebooks/
+          git diff --staged --quiet || git commit -m "Monthly statistics update $(date +'%Y-%m')"
+          git push

README.md

@@ -1,7 +1,9 @@
 # Deutsche Bahn Data
 
 This project saves public historical data from "Deutsche Bahn", the biggest german train company and makes it [accessible](https://huggingface.co/datasets/piebro/deutsche-bahn-data) for everyone to use.
-It includes train schedules, delays, and cancellations from stations across Germany. 
+It includes train schedules, delays, and cancellations from stations across Germany.
+
+There is also a small website to show same stat about the Deutsche Bahn: [piebro.github.io/deutsche-bahn-data](https://piebro.github.io/deutsche-bahn-data/stats/allgemein.html)
 
 The data can be used to validate the [official statistics](https://www.deutschebahn.com/de/konzern/konzernprofil/zahlen_fakten/puenktlichkeitswerte-6878476) and to create many other statistics.
 
@@ -84,14 +86,38 @@ uv sync --python 3.13
 uv run pre-commit install
 ```
 
+## Generating HTML from Notebooks
+
+```bash
+uv run python notebooks/src/nb_to_html.py                 # Convert all notebooks (no execution)
+uv run python notebooks/src/nb_to_html.py --run           # Run and convert all notebooks
+uv run python notebooks/src/nb_to_html.py --run allgemein # Run only allgemein, convert all
+```
+
 ## Contributing
 
 Contributions are welcome. Open an Issue if you want to report a bug, have an idea or want to propose a change.
 
+## Related Deutsche Bahn and Open Data Websites
+
+There are a few other projects that look at similar data.
+- [Video](https://www.youtube.com/watch?v=0rb9CfOvojk): BahnMining - Pünktlichkeit ist eine Zier (David Kriesel) [2019]
+- [www.deutschebahn.com](https://www.deutschebahn.com/de/konzern/konzernprofil/zahlen_fakten/puenktlichkeitswerte-6878476#): official statistics from Deutsche Bahn
+- [bahn.expert](https://bahn.expert): look at the departure monitor of train stations in real time
+- [next.bahnvorhersage.de](https://next.bahnvorhersage.de): a tool to calculate the probability that a train connection works using historical data
+- [www.zugfinder.net](https://www.zugfinder.net/de/start): multiple maps of current train positions and statistics for long-distance trains in Germany, Austria, BeNeLux, Denmark, Italy and Slovenia
+- [strecken-info.de](https://strecken-info.de/): a map of the German railroads with current construction sites and disruptions on the routes
+- [openrailwaymap.org](https://openrailwaymap.org/): a worldwide map with railway infrastructure using OpenStreetMap Data
+- [zugspaet.de](https://zugspaet.de): a website, where you can then enter your train and see how often it was late or on time in the past
+
 ## License
 
 All code in this project is licensed under the MIT License. The [data](https://huggingface.co/datasets/piebro/deutsche-bahn-data) is licensed under [Attribution 4.0 International (CC BY 4.0)](https://creativecommons.org/licenses/by/4.0/) by Deutsche Bahn.
 
+## Disclaimer
+
+This website is developed by Piet Brömmel. It has no affiliation with Deutsche Bahn or any other transportation company. This website is my personal project and everything stated here is provided without warranty, but is maintained by me to the best of my ability.
+
 ## Acknowledgments
 
 Data sourced from Deutsche Bahn's public APIs. Special thanks to Deutsche Bahn for providing open access to this data.

index.html

@@ -1,7 +1,7 @@
 <!DOCTYPE html>
 <html>
 <head>
-    <meta http-equiv="refresh" content="0; url=stats/übersicht.html">
+    <meta http-equiv="refresh" content="0; url=stats/allgemein.html">
 </head>
 <body>
 </body>