Skip to content

Commit 3ff4f2d

Browse files
authored
Removing feature: send context in venv operators (using use_airflow_context) (apache#46306)
1 parent 40fd35c commit 3ff4f2d

File tree

7 files changed

+0
-511
lines changed

7 files changed

+0
-511
lines changed

airflow/decorators/__init__.pyi

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -126,7 +126,6 @@ class TaskDecoratorCollection:
126126
show_return_value_in_logs: bool = True,
127127
env_vars: dict[str, str] | None = None,
128128
inherit_env: bool = True,
129-
use_airflow_context: bool = False,
130129
**kwargs,
131130
) -> TaskDecorator:
132131
"""Create a decorator to convert the decorated callable to a virtual environment task.
@@ -172,7 +171,6 @@ class TaskDecoratorCollection:
172171
environment. If set to ``True``, the virtual environment will inherit the environment variables
173172
of the parent process (``os.environ``). If set to ``False``, the virtual environment will be
174173
executed with a clean environment.
175-
:param use_airflow_context: Whether to provide ``get_current_context()`` to the python_callable.
176174
"""
177175
@overload
178176
def virtualenv(self, python_callable: Callable[FParams, FReturn]) -> Task[FParams, FReturn]: ...
@@ -188,7 +186,6 @@ class TaskDecoratorCollection:
188186
show_return_value_in_logs: bool = True,
189187
env_vars: dict[str, str] | None = None,
190188
inherit_env: bool = True,
191-
use_airflow_context: bool = False,
192189
**kwargs,
193190
) -> TaskDecorator:
194191
"""Create a decorator to convert the decorated callable to a virtual environment task.
@@ -219,7 +216,6 @@ class TaskDecoratorCollection:
219216
environment. If set to ``True``, the virtual environment will inherit the environment variables
220217
of the parent process (``os.environ``). If set to ``False``, the virtual environment will be
221218
executed with a clean environment.
222-
:param use_airflow_context: Whether to provide ``get_current_context()`` to the python_callable.
223219
"""
224220
@overload
225221
def branch( # type: ignore[misc]
@@ -252,7 +248,6 @@ class TaskDecoratorCollection:
252248
index_urls: None | Collection[str] | str = None,
253249
venv_cache_path: None | str = None,
254250
show_return_value_in_logs: bool = True,
255-
use_airflow_context: bool = False,
256251
**kwargs,
257252
) -> TaskDecorator:
258253
"""Create a decorator to wrap the decorated callable into a BranchPythonVirtualenvOperator.
@@ -291,7 +286,6 @@ class TaskDecoratorCollection:
291286
logs. Defaults to True, which allows return value log output.
292287
It can be set to False to prevent log output of return value when you return huge data
293288
such as transmission a large amount of XCom to TaskAPI.
294-
:param use_airflow_context: Whether to provide ``get_current_context()`` to the python_callable.
295289
"""
296290
@overload
297291
def branch_virtualenv(self, python_callable: Callable[FParams, FReturn]) -> Task[FParams, FReturn]: ...

airflow/example_dags/example_python_context_decorator.py

Lines changed: 0 additions & 92 deletions
This file was deleted.

airflow/example_dags/example_python_context_operator.py

Lines changed: 0 additions & 95 deletions
This file was deleted.

providers/standard/docs/operators/python.rst

Lines changed: 0 additions & 81 deletions
Original file line numberDiff line numberDiff line change
@@ -113,25 +113,6 @@ It can be used implicitly, such as with ``**kwargs``,
113113
but can also be used explicitly with ``get_current_context()``.
114114
In this case, the type hint can be used for static analysis.
115115

116-
.. tab-set::
117-
118-
.. tab-item:: @task
119-
:sync: taskflow
120-
121-
.. exampleinclude:: /../../airflow/example_dags/example_python_context_decorator.py
122-
:language: python
123-
:dedent: 4
124-
:start-after: [START get_current_context]
125-
:end-before: [END get_current_context]
126-
127-
.. tab-item:: PythonOperator
128-
:sync: operator
129-
130-
.. exampleinclude:: /../../airflow/example_dags/example_python_context_operator.py
131-
:language: python
132-
:dedent: 4
133-
:start-after: [START get_current_context]
134-
:end-before: [END get_current_context]
135116

136117
.. _howto/operator:PythonVirtualenvOperator:
137118

@@ -240,44 +221,6 @@ In case you have problems during runtime with broken cached virtual environments
240221
Note that any modification of a cached virtual environment (like temp files in binary path, post-installing further requirements) might pollute a cached virtual environment and the
241222
operator is not maintaining or cleaning the cache path.
242223

243-
Context
244-
^^^^^^^
245-
246-
With some limitations, you can also use ``Context`` in virtual environments.
247-
248-
.. important::
249-
Using ``Context`` in a virtual environment is a bit of a challenge
250-
because it involves library dependencies and serialization issues.
251-
252-
You can bypass this to some extent by using :ref:`Jinja template variables <templates:variables>` and explicitly passing it as a parameter.
253-
254-
You can also use ``get_current_context()`` in the same way as before, but with some limitations.
255-
256-
* Requires ``apache-airflow>=3.0.0``.
257-
258-
* Set ``use_airflow_context`` to ``True`` to call ``get_current_context()`` in the virtual environment.
259-
260-
* Set ``system_site_packages`` to ``True`` or set ``expect_airflow`` to ``True``
261-
262-
.. tab-set::
263-
264-
.. tab-item:: @task.virtualenv
265-
:sync: taskflow
266-
267-
.. exampleinclude:: /../../airflow/example_dags/example_python_context_decorator.py
268-
:language: python
269-
:dedent: 4
270-
:start-after: [START get_current_context_venv]
271-
:end-before: [END get_current_context_venv]
272-
273-
.. tab-item:: PythonVirtualenvOperator
274-
:sync: operator
275-
276-
.. exampleinclude:: /../../airflow/example_dags/example_python_context_operator.py
277-
:language: python
278-
:dedent: 4
279-
:start-after: [START get_current_context_venv]
280-
:end-before: [END get_current_context_venv]
281224

282225
.. _howto/operator:ExternalPythonOperator:
283226

@@ -347,30 +290,6 @@ Templating
347290

348291
Jinja templating can be used in same way as described for the :ref:`howto/operator:PythonOperator`.
349292

350-
Context
351-
^^^^^^^
352-
353-
You can use ``Context`` under the same conditions as ``PythonVirtualenvOperator``.
354-
355-
.. tab-set::
356-
357-
.. tab-item:: @task.external_python
358-
:sync: taskflow
359-
360-
.. exampleinclude:: /../../airflow/example_dags/example_python_context_decorator.py
361-
:language: python
362-
:dedent: 4
363-
:start-after: [START get_current_context_external]
364-
:end-before: [END get_current_context_external]
365-
366-
.. tab-item:: ExternalPythonOperator
367-
:sync: operator
368-
369-
.. exampleinclude:: /../../airflow/example_dags/example_python_context_operator.py
370-
:language: python
371-
:dedent: 4
372-
:start-after: [START get_current_context_external]
373-
:end-before: [END get_current_context_external]
374293

375294
.. _howto/operator:PythonBranchOperator:
376295

0 commit comments

Comments
 (0)