Conversation
Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
There was a problem hiding this comment.
Code Review
This pull request introduces a new parser module for pip lock files using pip_requirements_parser, which is a good addition for managing dependencies in raydepsets. The overall structure, including the new module, tests, and Bazel configuration, is well-organized. However, I've identified a significant correctness issue in the parser implementation where it loses information like comments and options during a read-write cycle. I've provided a suggestion to make the parser lossless. Additionally, I've recommended strengthening the tests to better ensure the correctness of the parsing and writing logic. The dependency updates are appropriate for the new functionality.
ci/raydepsets/tests/test_parser.py
Outdated
| assert len(reqs) == len(reqs2) | ||
| assert [r.name for r in reqs] == [r.name for r in reqs2] |
There was a problem hiding this comment.
The assertions in this roundtrip test are not very strong as they only check the number of packages and their names. This might not catch issues with version specifiers, hashes, or other requirement details being lost or modified during the write-read cycle.
A stronger assertion would be to compare the full requirement lines. For example:
assert [r.requirement_line.line for r in reqs] == [r.requirement_line.line for r in reqs2]If you adopt my suggestion for parser.py, you could even compare the full serialized output, which would be an even better test of correctness:
# With suggested changes to parser.py
rf = parse_lock_file(str(lock_file))
output_file = Path(tmpdir) / "output.txt"
write_lock_file(rf, str(output_file))
rf2 = parse_lock_file(str(output_file))
assert rf.dumps() == rf2.dumps()Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
pip_requirements_parser to read and write pip lock files,
providing parse_lock_file() and write_lock_file() utilities
release/requirements_py310.in and recompile the lock file
with Python 3.10