substrait-validator-py build fails when using pip (possibly old pip version)

[westonpace]

(conbench3) pace@pace-desktop:~/dev/substrait-validator/py$ pip wheel .
Processing /home/pace/dev/substrait-validator/py
  DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.
   pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555.
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... error
    ERROR: Command errored out with exit status 1:
     command: /home/pace/miniconda3/envs/conbench3/bin/python3.9 /home/pace/miniconda3/envs/conbench3/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /tmp/tmprj1aou9e
         cwd: /tmp/pip-req-build-exis0cph
    Complete output (18 lines):
    Traceback (most recent call last):
      File "/home/pace/miniconda3/envs/conbench3/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 349, in <module>
        main()
      File "/home/pace/miniconda3/envs/conbench3/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 331, in main
        json_out['return_val'] = hook(**hook_input['kwargs'])
      File "/home/pace/miniconda3/envs/conbench3/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 151, in prepare_metadata_for_build_wheel
        return hook(metadata_directory, config_settings)
      File "/tmp/pip-req-build-exis0cph/substrait_validator_build/__init__.py", line 59, in prepare_metadata_for_build_wheel
        _prepare()
      File "/tmp/pip-req-build-exis0cph/substrait_validator_build/__init__.py", line 50, in _prepare
        populate()
      File "/tmp/pip-req-build-exis0cph/substrait_validator_build/__init__.py", line 42, in populate
        _copytree(source, dest)
      File "/tmp/pip-req-build-exis0cph/substrait_validator_build/__init__.py", line 35, in _copytree
        shutil.copyfile(source, dest)
      File "/home/pace/miniconda3/envs/conbench3/lib/python3.9/shutil.py", line 264, in copyfile
        with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
    FileNotFoundError: [Errno 2] No such file or directory: '../proto'

The deprecation warning in this case seems significant. If I supply the recommended argument (--use-feature=in-tree-build) then the build works correctly. I'm not sure if this is something that needs to be fixed but if there is a minimum pip version it could be flagged and reported as a violation.

[jvanstraten]

Hm. I thought I'd tested everything with pip's build-in-tmp-dir logic but I guess not. Thanks for the report; I'll take another look when able but I'm about to go on two weeks PTO so I don't know when that will be.

[westonpace]

This is not at all a priority. Mostly me just jotting down any warts I encountered as I got the validator installed (I needed it installed to use the consumer testing PR).

[jvanstraten]

I managed to reproduce this with older pip versions indeed. Unfortunately I'm not sure how this can be fixed; even if I make the script that's crashing there more robust, maturin would fail, because the crates that substrait-validator-py depend on would no longer be where they should be according to Cargo.toml. Detecting out-of-tree builds and printing a clearer error message is the best I could come up with (#38).

If you want to build wheels though, like you're doing there, you can just use maturin instead of pip. I've added the docs for that to the readme in #38 as well.