Skip to content

Commit 45a30f3

Browse files
committed
docs: add comments to fix_proto_imports.sh script
1 parent da646fd commit 45a30f3

File tree

1 file changed

+46
-0
lines changed

1 file changed

+46
-0
lines changed

python/datafed_pkg/scripts/fix_proto_imports.sh

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,52 @@
11
#!/bin/sh
22
set -e
33

4+
# What this script does
5+
#
6+
# protoc --python_out
7+
#
8+
# generates _pb2.py files with absolute imports based on the proto import
9+
# paths. For example, if envelope.proto imports anon/auth_by_token.proto, the
10+
# generated envelope_pb2.py will contain:
11+
#
12+
# python from anon import auth_by_token_pb2
13+
#
14+
15+
# This works if you run Python from the exact output directory, but breaks when
16+
# the generated code is consumed as a Python package (which is how DataFed uses
17+
# it). Python's package system requires relative imports for intra-package
18+
# references:
19+
#
20+
# File at package level
21+
#
22+
# from .anon import auth_by_token_pb2
23+
#
24+
# File at root
25+
#
26+
# from ..anon import auth_by_token_pb2
27+
#
28+
# file in a subdirectory protoc
29+
# has no option to emit relative imports. This is a well-known, long-standing
30+
# limitation (protocolbuffers/protobuf#1491). The script does three things:
31+
#
32+
# 1. Rewrites imports to be relative. It finds every _pb2.py file, determines
33+
# whether it lives at the package root or in a subdirectory (e.g., anon/,
34+
# auth/), and rewrites bare absolute imports (from anon import ...) to the
35+
# correct relative form (.anon for root-level files, ..anon for files one level
36+
# deep).
37+
# 2. Creates __init__.py files in each subdirectory (anon/, auth/, enums/,
38+
# messages/) so Python recognizes them as subpackages. Appends re-exports to
39+
# envelope_pb2.py for backward compatibility. The existing Python client
40+
# (Connection.py) uses getattr(envelope_module, ClassName) to dynamically look
41+
# up message classes by name on the envelope module.
42+
#
43+
# Under the old single-file
44+
# proto2 layout, all message classes lived directly in envelope_pb2.py. Now
45+
# that messages are split across subpackages, this dynamic lookup would break.
46+
# The wildcard re-exports (from .anon.auth_by_token_pb2 import *, etc.) restore
47+
# the flat namespace on envelope_pb2 so existing code continues to work without
48+
# modification.
49+
450
PROTO_DIR="$1"
551
ROOT_DIR="${2:-$1}"
652

0 commit comments

Comments
 (0)