Skip to content

Add a new DAG for SFT post-training from Jupyter notebooks#1164

Merged
alfredyu-cienet merged 1 commit intoGoogleCloudPlatform:masterfrom
CIeNET-International:maxtext/release/pr-193
Jan 29, 2026
Merged

Add a new DAG for SFT post-training from Jupyter notebooks#1164
alfredyu-cienet merged 1 commit intoGoogleCloudPlatform:masterfrom
CIeNET-International:maxtext/release/pr-193

Conversation

@ooops678
Copy link
Contributor

Description

Airflow DAG for automating Llama3.1-8B SFT training from Jupyter notebook.
This DAG automates the sft_llama3_demo.ipynb notebook, executing SFT
training on single-host TPU VMs.

Tests

Airflow/Composer

  • GCP Composer name: camilo-orbax-dev (under GCP project: cloud-ml-auto-solutions)
  • GCP Composer version: 2.13.1

Bucket with HNS (Hierarchical Name Space)

  • Bucket Name: gs://mtc-automation-bucket

Checklist

Before submitting this PR, please make sure (put X in square brackets):

  • I have performed a self-review of my code.
  • I have necessary comments in my code, particularly in hard-to-understand areas.
  • I have run one-shot tests and provided workload links above if applicable.
  • I have made or will make corresponding changes to the doc if needed.

Add an Airflow DAG for automating Llama3.1-8B SFT training from Jupyter notebook.
This DAG automates the sft_llama3_demo.ipynb notebook, executing SFT
training on single-host TPU VMs.
@alfredyu-cienet alfredyu-cienet merged commit 6036dea into GoogleCloudPlatform:master Jan 29, 2026
7 checks passed
@alfredyu-cienet alfredyu-cienet deleted the maxtext/release/pr-193 branch January 29, 2026 09:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants