[Python] Deprecate --enable_streaming_engine flag since it is always enabled.#35917
[Python] Deprecate --enable_streaming_engine flag since it is always enabled.#35917scwhittle merged 2 commits intoapache:masterfrom
Conversation
…enabled. Remove weird verification if experiment is set.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Summary of Changes
Hello @scwhittle, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request deprecates the --enable_streaming_engine flag within the Apache Beam Python SDK. As Python pipelines now inherently utilize the Streaming Engine, this flag's default behavior is updated to True, and a previously problematic validation check that caused failures when the flag was unset but experiments were present has been removed. This change simplifies pipeline option handling and removes unnecessary complexity.
Highlights
- Flag Deprecation: The --enable_streaming_engine flag's default value is changed from False to True, and its help text is updated to indicate its deprecated status, clarifying that all Python pipelines now use Streaming Engine.
- Validation Logic Removal: Redundant and error-prone validation logic in dataflow_runner.py has been removed. This previously caused failures if the --enable_streaming_engine flag was not explicitly set but related streaming engine experiments were enabled.
- Test Case Update: A relevant test case in dataflow_runner_test.py has been updated to reflect the deprecation, no longer explicitly setting the --enable_streaming_engine flag.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
|
Assigning reviewers: R: @tvalentyn for label python. Note: If you would like to opt out of this review, comment Available commands:
The PR bot will only process comments in the main thread (not review comments). |
| default=True, | ||
| action='store_true', | ||
| help='Enable Windmill Service for this Dataflow job. ') | ||
| help='Deprecated. All Python pipelines use Streaming Engine. ') |
There was a problem hiding this comment.
nit: All Python pipelines on Dataflow?
scwhittle
force-pushed
the
python_streaming_engine
branch
2 times, most recently
from
9a183b9 to
b1b050c
Compare
scwhittle
force-pushed
the
python_streaming_engine
branch
from
b1b050c to
528ba91
Compare
|
No longer formatting errors, the test failure appears unrelated. |
3 participants
Remove weird verification if flag was unset but experiment was set, which caused failure even if we otherwise ignore the flag by overwriting it.
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, commentfixes #<ISSUE NUMBER>instead.CHANGES.mdwith noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.