-
Notifications
You must be signed in to change notification settings - Fork 16.3k
Closed
Description
Currently, models.BaseOperator and models.DAG use the Task SDK classes.
This was done to maintain backwards-compat and to get 3.0 in a timely manner.
However, we don't want to depend on Task SDK in the Scheduler or the API-server (airflow-core package) - at least not a required dependency.
We only need to ensure we have enough information for the Scheduler & API-server for a DAG & the Operator. This is done via the SerializedDAG & SerializedBaseOp.
So this task involves: figuring out how we can consolidate: SerializedDAG, models.DAG by not inheriting Task SDK; and same for SerializedBaseOp and models.BaseOperator
To do (not ordered by dependency):
- Timetables
- Assets - Split SDK and serialized asset classes #58993
- DAGNode - Split DAGNode in Core and SDK #59708
- NOTSET and ArgNotSet
- EdgeInfoType Duplicate EdgeInfoType in Core #59774
- DeadlineAlert and related - On hold, tracked in Separate DeadlineAlert implementations in SDK and Core #59303
- BaseOperatorLink
- ExpandInput Clean up SDK references in airflow.models.expandinput #59815
- PriorityWeightStrategy
- First attempt (reverted) Remove PriorityWeightStrategy reference in SDK #59780
- Second attempt Re-apply PriorityWeightStrategy SDK work #60112
- Partitions - Going to figure it as we implement them - Tracked in asset partition - core / task SDK separation #59318
- Clean up remaining SDK references in Core Clean up top-level SDK imports in Core #59817
- Remove
ti.render_templates() - Take
airflow.serialization.serialized_objectsapart- SerializedDAG Split serde logic from SerializedDAG #59596
- SerializedBaseOperator Split SerializedBaseOperator from serde logic #59627
- XComOperatorLink Move XComOperatorLink to separate module #59776
- General cleanups
- Move MappedOperator to
airflow.serialization.definitionsMove MappedOperator to serialization #59628 - Move SchedulerXComArg to
airflow.serialization.definitionsMove SchedulerXComArg to serialization #59777 - Ensure all things moved have back compat
Metadata
Metadata
Labels
Type
Projects
Status
Done