Skip to content

feat(gazelle): add Python tags directives #3146

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,7 @@ END_UNRELEASED_TEMPLATE

{#v0-0-0-added}
### Added
* (gazelle) Added Python tags directives: `python_tags`, `python_library_tags`, `python_binary_tags`, and `python_test_tags`. These directives allow adding Bazel tags to generated Python targets for better build control and test categorization. Tags from general and specific directives are combined and sorted alphabetically.
* (repl) Default stub now has tab completion, where `readline` support is available,
see ([#3114](https://github.com/bazel-contrib/rules_python/pull/3114)).
([#3114](https://github.com/bazel-contrib/rules_python/pull/3114)).
Expand Down
96 changes: 96 additions & 0 deletions gazelle/docs/directives.md
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,26 @@ The Python-specific directives are:
* Allowed Values: `true`, `false`
* Allows absolute imports to be resolved to sibling modules (Python 2's
behavior without `absolute_import`).
* [`# gazelle:python_tags`](#directive-python_tags)
* Default: n/a
* Allowed Values: A comma-separated list of tags
* Adds tags to all generated Python targets. Multiple tags can be specified
as a comma-separated list.
* [`# gazelle:python_library_tags`](#directive-python_tags)
* Default: n/a
* Allowed Values: A comma-separated list of tags
* Adds tags specific to `py_library` targets. Multiple tags can be specified
as a comma-separated list.
* [`# gazelle:python_binary_tags`](#directive-python_tags)
* Default: n/a
* Allowed Values: A comma-separated list of tags
* Adds tags specific to `py_binary` targets. Multiple tags can be specified
as a comma-separated list.
* [`# gazelle:python_test_tags`](#directive-python_tags)
* Default: n/a
* Allowed Values: A comma-separated list of tags
* Adds tags specific to `py_test` targets. Multiple tags can be specified
as a comma-separated list.


## `python_extension`
Expand Down Expand Up @@ -645,3 +665,79 @@ previously-generated or hand-created rules.
:::{error}
Detailed docs are not yet written.
:::


## `python_tags` {#directive-python_tags}

The tag directives allow you to add [Bazel tags](https://bazel.build/reference/be/common-definitions#common-attributes) to generated Python targets. Tags are metadata labels that can be used to control test execution, categorize targets, or influence build behavior.

There are four tag directives available:

- `# gazelle:python_tags` - Adds tags to all generated Python targets (`py_library`, `py_binary`, `py_test`)
- `# gazelle:python_library_tags` - Adds tags specifically to `py_library` targets
- `# gazelle:python_binary_tags` - Adds tags specifically to `py_binary` targets
- `# gazelle:python_test_tags` - Adds tags specifically to `py_test` targets

Tags from general (`python_tags`) and specific directives are combined and sorted alphabetically.

**Usage:**

```starlark
# Add tags to all Python targets
# gazelle:python_tags manual,integration

# Add specific tags to different target types
# gazelle:python_library_tags reusable,shared
# gazelle:python_binary_tags deploy,production
# gazelle:python_test_tags unit,fast
```

This generates targets like:

```starlark
py_library(
name = "mylib",
srcs = ["mylib.py"],
tags = [
"integration",
"manual",
"reusable",
"shared",
],
)

py_binary(
name = "myapp_bin",
srcs = ["__main__.py"],
main = "__main__.py",
tags = [
"deploy",
"integration",
"manual",
"production",
],
)

py_test(
name = "mylib_test",
srcs = ["__test__.py"],
main = "__test__.py",
tags = [
"fast",
"integration",
"manual",
"unit",
],
)
```

**Common tag examples:**

- `manual` - Prevents the target from being built by `bazel build //...`
- `integration` - Marks integration tests that may be run separately
- `unit` - Marks unit tests for selective execution
- `exclusive` - Indicates tests that need exclusive resource access
- `deploy` - Marks binaries used for deployment
- `fast` - Indicates fast-running tests

Multiple tags can be specified as a comma-separated list. Whitespace around commas is automatically trimmed.
48 changes: 48 additions & 0 deletions gazelle/python/configure.go
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,10 @@ func (py *Configurer) KnownDirectives() []string {
pythonconfig.TestFilePattern,
pythonconfig.LabelConvention,
pythonconfig.LabelNormalization,
pythonconfig.Tags,
pythonconfig.LibraryTags,
pythonconfig.BinaryTags,
pythonconfig.TestTags,
pythonconfig.GeneratePyiDeps,
pythonconfig.ExperimentalAllowRelativeImports,
pythonconfig.GenerateProto,
Expand Down Expand Up @@ -254,6 +258,50 @@ func (py *Configurer) Configure(c *config.Config, rel string, f *rule.File) {
log.Fatal(err)
}
config.SetResolveSiblingImports(v)
case pythonconfig.Tags:
value := strings.TrimSpace(d.Value)
if value == "" {
config.SetTags([]string{})
} else {
tags := strings.Split(value, ",")
for i, tag := range tags {
tags[i] = strings.TrimSpace(tag)
}
config.SetTags(tags)
}
case pythonconfig.LibraryTags:
value := strings.TrimSpace(d.Value)
if value == "" {
config.SetLibraryTags([]string{})
} else {
tags := strings.Split(value, ",")
for i, tag := range tags {
tags[i] = strings.TrimSpace(tag)
}
config.SetLibraryTags(tags)
}
case pythonconfig.BinaryTags:
value := strings.TrimSpace(d.Value)
if value == "" {
config.SetBinaryTags([]string{})
} else {
tags := strings.Split(value, ",")
for i, tag := range tags {
tags[i] = strings.TrimSpace(tag)
}
config.SetBinaryTags(tags)
}
case pythonconfig.TestTags:
value := strings.TrimSpace(d.Value)
if value == "" {
config.SetTestTags([]string{})
} else {
tags := strings.Split(value, ",")
for i, tag := range tags {
tags[i] = strings.TrimSpace(tag)
}
config.SetTestTags(tags)
}
}
}

Expand Down
13 changes: 13 additions & 0 deletions gazelle/python/generate.go
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,14 @@ func matchesAnyGlob(s string, globs []string) bool {
return false
}

// combineTags combines general tags with specific target-type tags
func combineTags(generalTags, specificTags []string) []string {
combined := make([]string, 0, len(generalTags)+len(specificTags))
combined = append(combined, generalTags...)
combined = append(combined, specificTags...)
return combined
}

// GenerateRules extracts build metadata from source files in a directory.
// GenerateRules is called in each directory where an update is requested
// in depth-first post-order.
Expand Down Expand Up @@ -261,6 +269,7 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes
}
pyBinary := newTargetBuilder(pyBinaryKind, pyBinaryTargetName, pythonProjectRoot, args.Rel, pyFileNames, cfg.ResolveSiblingImports()).
addVisibility(visibility).
addTags(combineTags(cfg.Tags(), cfg.BinaryTags())).
addSrc(filename).
addModuleDependencies(mainModules[filename]).
addResolvedDependencies(annotations.includeDeps).
Expand Down Expand Up @@ -303,6 +312,7 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes

pyLibrary := newTargetBuilder(pyLibraryKind, pyLibraryTargetName, pythonProjectRoot, args.Rel, pyFileNames, cfg.ResolveSiblingImports()).
addVisibility(visibility).
addTags(combineTags(cfg.Tags(), cfg.LibraryTags())).
addSrcs(srcs).
addModuleDependencies(allDeps).
addResolvedDependencies(annotations.includeDeps).
Expand Down Expand Up @@ -357,6 +367,7 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes
pyBinaryTarget := newTargetBuilder(pyBinaryKind, pyBinaryTargetName, pythonProjectRoot, args.Rel, pyFileNames, cfg.ResolveSiblingImports()).
setMain(pyBinaryEntrypointFilename).
addVisibility(visibility).
addTags(combineTags(cfg.Tags(), cfg.BinaryTags())).
addSrc(pyBinaryEntrypointFilename).
addModuleDependencies(deps).
addResolvedDependencies(annotations.includeDeps).
Expand Down Expand Up @@ -393,6 +404,7 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes
addResolvedDependencies(annotations.includeDeps).
setAnnotations(*annotations).
addVisibility(visibility).
addTags(combineTags(cfg.Tags(), cfg.LibraryTags())).
setTestonly().
generateImportsAttribute()

Expand Down Expand Up @@ -423,6 +435,7 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes
addSrcs(srcs).
addModuleDependencies(deps).
addResolvedDependencies(annotations.includeDeps).
addTags(combineTags(cfg.Tags(), cfg.TestTags())).
setAnnotations(*annotations).
generateImportsAttribute()
}
Expand Down
16 changes: 16 additions & 0 deletions gazelle/python/target.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ package python

import (
"path/filepath"
"strings"

"github.com/bazelbuild/bazel-gazelle/config"
"github.com/bazelbuild/bazel-gazelle/rule"
Expand All @@ -37,6 +38,7 @@ type targetBuilder struct {
main *string
imports []string
testonly bool
tags *treeset.Set
annotations *annotations
resolveSiblingImports bool
}
Expand All @@ -53,6 +55,7 @@ func newTargetBuilder(kind, name, pythonProjectRoot, bzlPackage string, siblingS
deps: treeset.NewWith(moduleComparator),
resolvedDeps: treeset.NewWith(godsutils.StringComparator),
visibility: treeset.NewWith(godsutils.StringComparator),
tags: treeset.NewWith(godsutils.StringComparator),
annotations: new(annotations),
resolveSiblingImports: resolveSiblingImports,
}
Expand Down Expand Up @@ -122,6 +125,16 @@ func (t *targetBuilder) addVisibility(visibility []string) *targetBuilder {
return t
}

// addTags adds tags to the target.
func (t *targetBuilder) addTags(tags []string) *targetBuilder {
for _, tag := range tags {
if strings.TrimSpace(tag) != "" {
t.tags.Add(strings.TrimSpace(tag))
}
}
return t
}

// setMain sets the main file to the target.
func (t *targetBuilder) setMain(main string) *targetBuilder {
t.main = &main
Expand Down Expand Up @@ -168,6 +181,9 @@ func (t *targetBuilder) build() *rule.Rule {
if !t.visibility.Empty() {
r.SetAttr("visibility", t.visibility.Values())
}
if !t.tags.Empty() {
r.SetAttr("tags", t.tags.Values())
}
if t.main != nil {
r.SetAttr("main", *t.main)
}
Expand Down
11 changes: 11 additions & 0 deletions gazelle/python/testdata/directive_python_tags/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Python tags directive test

Tests the python_tags, python_library_tags, python_binary_tags, and python_test_tags directives.

These directives allow adding tags to generated Python targets:
- `python_tags` - Tags added to all Python targets
- `python_library_tags` - Tags specific to py_library targets
- `python_binary_tags` - Tags specific to py_binary targets
- `python_test_tags` - Tags specific to py_test targets

Tags from general and specific directives are combined.
1 change: 1 addition & 0 deletions gazelle/python/testdata/directive_python_tags/WORKSPACE
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
workspace(name = "directive_python_tags")
17 changes: 17 additions & 0 deletions gazelle/python/testdata/directive_python_tags/test.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Copyright 2024 The Bazel Authors. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

---
expect:
exit_code: 0
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Test general python_tags directive
# gazelle:python_tags manual,integration
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test")

# Test general python_tags directive
# gazelle:python_tags manual,integration

py_library(
name = "test1_general_tags",
srcs = ["__init__.py"],
tags = [
"integration",
"manual",
],
visibility = ["//:__subpackages__"],
)

py_binary(
name = "test1_general_tags_bin",
srcs = ["__main__.py"],
main = "__main__.py",
tags = [
"integration",
"manual",
],
visibility = ["//:__subpackages__"],
)

py_test(
name = "test1_general_tags_test",
srcs = ["__test__.py"],
main = "__test__.py",
tags = [
"integration",
"manual",
],
)
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# Empty library file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
print("Hello from test1")
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
assert True, "Test passed"
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Test specific target type tags
# gazelle:python_library_tags reusable,shared
# gazelle:python_binary_tags deploy,production
# gazelle:python_test_tags unit,fast
Loading