Merge branch 'main' into martani/no-sibling-import

Author: amartani

CHANGELOG.md

@@ -91,6 +91,10 @@ END_UNRELEASED_TEMPLATE
 * Multi-line python imports (e.g. with escaped newlines) are now correctly processed by Gazelle.
 * (toolchains) `local_runtime_repo` works with multiarch Debian with Python 3.8
   ([#3099](https://github.com/bazel-contrib/rules_python/issues/3099)).
+* (pypi) Expose pypi packages only common to all Python versions in `all_requirements`
+  ([#2921](https://github.com/bazel-contrib/rules_python/issues/2921)).
+* (repl) Normalize the path for the `REPL` stub to make it possible to use the
+  default stub template from outside `rules_python` ({gh-issue}`3101`).
 
 {#v0-0-0-added}
 ### Added
@@ -116,6 +120,8 @@ END_UNRELEASED_TEMPLATE
   dep is not added to the {obj}`py_test` target.
 * (gazelle) New directive `gazelle:python_generate_proto`; when `true`,
   Gazelle generates `py_proto_library` rules for `proto_library`. `false` by default.
+* (gazelle) New directive `gazelle:python_proto_naming_convention`; controls
+  naming of `py_proto_library` rules.
 * (gazelle) Added directive `# gazelle:python_resolve_sibling_imports` (default
   `enabled` for backwards-compatibility). When enabled, gazelle will allow absolute
   imports to be resolved to sibling modules (Python 2's behavior without

docs/requirements.txt

@@ -350,9 +350,9 @@ sphinxcontrib-serializinghtml==2.0.0 \
     --hash=sha256:6e2cb0eef194e10c27ec0023bfeb25badbbb5868244cf5bc5bdc04e4464bf331 \
     --hash=sha256:e9d912827f872c029017a53f0ef2180b327c3f7fd23c87229f7a8e8b70031d4d
     # via sphinx
-typing-extensions==4.13.2 \
-    --hash=sha256:a439e7c04b49fec3e5d3e2beaa21755cadbbdc391694e28ccdd36ca4a1408f8c \
-    --hash=sha256:e6c81219bd689f51865d9e372991c540bda33a0379d5573cddb9a3a23f7caaef
+typing-extensions==4.14.1 \
+    --hash=sha256:38b39f4aeeab64884ce9f74c94263ef78f3c22467c8724005483154c26648d36 \
+    --hash=sha256:d1e1e3b58374dc93031d6eda2420a48ea44a36c2b4766a4fdeb3710755731d76
     # via
     #   rules-python-docs (docs/pyproject.toml)
     #   sphinx-autodoc2

gazelle/README.md

@@ -208,6 +208,8 @@ Python-specific directives are as follows:
 | Controls the `py_binary` naming convention. Follows the same interpolation rules as `python_library_naming_convention`.                                                                                                                                                                         | |
 | `# gazelle:python_test_naming_convention`                                                                                                                                                                                                                                                       | `$package_name$_test` |
 | Controls the `py_test` naming convention. Follows the same interpolation rules as `python_library_naming_convention`.                                                                                                                                                                           | |
+| [`# gazelle:python_proto_naming_convention`](#directive-python_proto_naming_convention)                                                                                                                                                                                                         | `$proto_name$_py_pb2` |
+| Controls the `py_proto_library` naming convention. It interpolates `$proto_name$` with the proto_library rule name, minus any trailing _proto. E.g. if the proto_library name is `foo_proto`, setting this to `$proto_name$_my_lib` would render to `foo_my_lib`.                               | |
 | `# gazelle:resolve py ...`                                                                                                                                                                                                                                                                      | n/a |
 | Instructs the plugin what target to add as a dependency to satisfy a given import statement. The syntax is `# gazelle:resolve py import-string label` where `import-string` is the symbol in the python `import` statement, and `label` is the Bazel label that Gazelle should write in `deps`. | |
 | [`# gazelle:python_default_visibility labels`](#directive-python_default_visibility)                                                                                                                                                                                                            | |
@@ -264,6 +266,31 @@ py_libary(
 
 [python-packaging-user-guide]: https://github.com/pypa/packaging.python.org/blob/4c86169a/source/tutorials/packaging-projects.rst
 
+#### Directive: `python_proto_naming_convention`:
+
+Set this directive to a string pattern to control how the generated `py_proto_library` targets are named. When generating new `py_proto_library` rules, Gazelle will replace `$proto_name$` in the pattern with the name of the `proto_library` rule, stripping out a trailing `_proto`. For example:
+
+```starlark
+# gazelle:python_generate_proto true
+# gazelle:python_proto_naming_convention my_custom_$proto_name$_pattern
+
+proto_library(
+    name = "foo_proto",
+    srcs = ["foo.proto"],
+)
+```
+
+produces the following `py_proto_library` rule:
+```starlark
+py_proto_library(
+    name = "my_custom_foo_pattern",
+    deps = [":foo_proto"],
+)
+```
+
+The default naming convention is `$proto_name$_pb2_py`, so by default in the above example Gazelle would generate `foo_pb2_py`. Any pre-existing rules are left in place and not renamed.
+
+Note that the Python library will always be imported as `foo_pb2` in Python code, regardless of the naming convention. Also note that Gazelle is currently not able to map said imports, e.g. `import foo_pb2`, to fill in `py_proto_library` targets as dependencies of other rules. See [this issue](https://github.com/bazel-contrib/rules_python/issues/1703).
 
 #### Directive: `python_default_visibility`:
 

gazelle/python/configure.go

@@ -63,6 +63,7 @@ func (py *Configurer) KnownDirectives() []string {
 		pythonconfig.LibraryNamingConvention,
 		pythonconfig.BinaryNamingConvention,
 		pythonconfig.TestNamingConvention,
+		pythonconfig.ProtoNamingConvention,
 		pythonconfig.DefaultVisibilty,
 		pythonconfig.Visibility,
 		pythonconfig.TestFilePattern,
@@ -180,6 +181,8 @@ func (py *Configurer) Configure(c *config.Config, rel string, f *rule.File) {
 			config.SetBinaryNamingConvention(strings.TrimSpace(d.Value))
 		case pythonconfig.TestNamingConvention:
 			config.SetTestNamingConvention(strings.TrimSpace(d.Value))
+		case pythonconfig.ProtoNamingConvention:
+			config.SetProtoNamingConvention(strings.TrimSpace(d.Value))
 		case pythonconfig.DefaultVisibilty:
 			switch directiveArg := strings.TrimSpace(d.Value); directiveArg {
 			case "NONE":

gazelle/python/generate.go

@@ -227,7 +227,7 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes
 	result.Gen = make([]*rule.Rule, 0)
 
 	if cfg.GenerateProto() {
-		generateProtoLibraries(args, pythonProjectRoot, visibility, &result)
+		generateProtoLibraries(args, cfg, pythonProjectRoot, visibility, &result)
 	}
 
 	collisionErrors := singlylinkedlist.New()
@@ -569,7 +569,7 @@ func ensureNoCollision(file *rule.File, targetName, kind string) error {
 	return nil
 }
 
-func generateProtoLibraries(args language.GenerateArgs, pythonProjectRoot string, visibility []string, res *language.GenerateResult) {
+func generateProtoLibraries(args language.GenerateArgs, cfg *pythonconfig.Config, pythonProjectRoot string, visibility []string, res *language.GenerateResult) {
 	// First, enumerate all the proto_library in this package.
 	var protoRuleNames []string
 	for _, r := range args.OtherGen {
@@ -582,18 +582,29 @@ func generateProtoLibraries(args language.GenerateArgs, pythonProjectRoot string
 
 	// Next, enumerate all the pre-existing py_proto_library in this package, so we can delete unnecessary rules later.
 	pyProtoRules := map[string]bool{}
+	pyProtoRulesForProto := map[string]string{}
 	if args.File != nil {
 		for _, r := range args.File.Rules {
 			if r.Kind() == "py_proto_library" {
 				pyProtoRules[r.Name()] = false
+
+				protos := r.AttrStrings("deps")
+				for _, proto := range protos {
+					pyProtoRulesForProto[strings.TrimPrefix(proto, ":")] = r.Name()
+				}
 			}
 		}
 	}
 
 	emptySiblings := treeset.Set{}
 	// Generate a py_proto_library for each proto_library.
 	for _, protoRuleName := range protoRuleNames {
-		pyProtoLibraryName := strings.TrimSuffix(protoRuleName, "_proto") + "_py_pb2"
+		pyProtoLibraryName := cfg.RenderProtoName(protoRuleName)
+		if ruleName, ok := pyProtoRulesForProto[protoRuleName]; ok {
+			// There exists a pre-existing py_proto_library for this proto. Keep this name.
+			pyProtoLibraryName = ruleName
+		}
+
 		pyProtoLibrary := newTargetBuilder(pyProtoLibraryKind, pyProtoLibraryName, pythonProjectRoot, args.Rel, &emptySiblings, false).
 			addVisibility(visibility).
 			addResolvedDependency(":" + protoRuleName).

gazelle/python/testdata/directive_python_proto_naming_convention/README.md

@@ -0,0 +1,9 @@
+# Directive: `python_proto_naming_convention`
+
+This test case asserts that the `# gazelle:python_proto_naming_convention` directive
+correctly:
+
+1.  Has no effect on pre-existing `py_proto_library` when `gazelle:python_generate_proto` is disabled.
+2.  Uses the default value when proto generation is on and `python_proto_naming_convention` is not set.
+3.  Uses the provided naming convention when proto generation is on and `python_proto_naming_convention` is set.
+4.  With a pre-existing `py_proto_library` not following a given naming convention, keeps it intact and does not rename it.

gazelle/python/testdata/directive_python_proto_naming_convention/WORKSPACE

@@ -0,0 +1 @@
+# This is a Bazel workspace for the Gazelle test data.

gazelle/python/testdata/directive_python_proto_naming_convention/test.yaml

@@ -0,0 +1,3 @@
+---
+expect:
+  exit_code: 0

gazelle/python/testdata/directive_python_proto_naming_convention/test1_python_generation_disabled_does_nothing/BUILD.in

@@ -0,0 +1,17 @@
+load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library")
+load("@rules_proto//proto:defs.bzl", "proto_library")
+
+# gazelle:python_generate_proto false
+# gazelle:python_proto_naming_convention some_$proto_name$_value
+
+proto_library(
+    name = "foo_proto",
+    srcs = ["foo.proto"],
+    visibility = ["//:__subpackages__"],
+)
+
+py_proto_library(
+    name = "foo_proto_custom_name",
+    visibility = ["//:__subpackages__"],
+    deps = [":foo_proto"],
+)

gazelle/python/testdata/directive_python_proto_naming_convention/test1_python_generation_disabled_does_nothing/BUILD.out

@@ -0,0 +1,17 @@
+load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library")
+load("@rules_proto//proto:defs.bzl", "proto_library")
+
+# gazelle:python_generate_proto false
+# gazelle:python_proto_naming_convention some_$proto_name$_value
+
+proto_library(
+    name = "foo_proto",
+    srcs = ["foo.proto"],
+    visibility = ["//:__subpackages__"],
+)
+
+py_proto_library(
+    name = "foo_proto_custom_name",
+    visibility = ["//:__subpackages__"],
+    deps = [":foo_proto"],
+)