[BUG] ModuleNotFoundError: No module named 'buf' when importing protovalidate

[jkruzek]

Description

After pip installing protovalidate==0.3.1 and trying to import it I get the error ModuleNotFoundError: No module named 'buf'. This was using Python 3.11

Steps to Reproduce

1. pip install protovalidate
2. In Python REPL for 3.11 run import protovalidate
3. The error will show up on that command

Expected Behavior

Be able to import with out error. The reference to buf is internal to protovalidate but it doesn't seem to be in the package

Actual Behavior

Import errors out with ModuleNotFoundError: No module named 'buf'

Screenshots/Logs

>>> import protovalidate
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/venv/lib/python3.11/site-packages/protovalidate/__init__.py", line 15, in <module>
    from protovalidate import validator
  File "/venv/lib/python3.11/site-packages/protovalidate/validator.py", line 19, in <module>
    from buf.validate import expression_pb2  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'buf'

Environment

• Operating System: macOS M2 chip
• Version: Sonoma 14.3.1
• Compiler/Toolchain:
• Protobuf Compiler & Version:
• Protoc-gen-validate Version: 0.4.2
• Protovalidate Version: 0.3.1

Possible Solution

Additional Context

[dobet]

What should I do if I am using python -m grpcio_tools.protoc instead of buf to compile the proto file?

[TakumaKurosawa]

I faced same problem 🙋

[joshuaostrom-cb]

Seeing the same

[emcfarlane]

This is an issue with the documentation. The wheel file doesn't contain the dependency for the provalidate source files and must be generated. Ensure you follow the guide here to generate with buf.

You should have the following source files locally:

gen
└── buf
    └── validate
        ├── expression_pb2.py
        ├── priv
        │   └── private_pb2.py
        └── validate_pb2.py

Now for python to find this package as buf.validate.expression_pb2 the following env variable is required to be set:

PYTHONPATH=gen

[jkruzek]

Thanks @emcfarlane . I found that running buf generate --include-imports does not generate the buf folder as expected, however, running buf generate buf.build/bufbuild/protovalidate does generate the proper folder structure. Any ideas why that might be?

[emcfarlane]

Glad you fixed it. It could be due to not having any imports for the validation library. Check your source files have import "buf/validate/validate.proto"; and use the import to validate. Run buf dep update and check that a buf.lock file is generated with a dep on buf.build/bufbuild/protovalidate. Once you start to use the validation library --inlcude-imports should work.

[jkruzek]

@emcfarlane Thanks! That indeed did fix it. Appreciate the help

[jmspereira]

Hey @emcfarlane, sorry for reopening this issue, but I think that I missing something when using protoc instead of buf.
I have a project with a set of protobuf files, I am compiling them with:

protoc <protobuf-files> 
-I/usr/local/include/protovalidate  
--fatal_warnings 
--python_out=/<outpath> 
--pyi_out=/<outpath>  
--include_imports 
--descriptor_set_out=<outpath>/gen

Where the include_imports the descriptor_set_out was set as documented. However, even when I define the PYTHONPATH to the gen file, still does not work. This is likely because this is not generating the validate_pb2.py but a binary file. So, what is the correct thing to do in order use protovalidate with protoc?

Thanks!