feat: Convert legacy verifiers to mellea reqlib #8
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Converts the imperative legacy verifiers into declarative mellea Requirements.
Precursor
The feat/llm-sandbox-execution branch that I already pushed to the main mellea repo provides the safe/unsafe execution structure with llm-sandbox.
Implementation
7 auto-fixing requirements:
python_files_accessible- Creates missing data/image/text files automaticallypython_imports_resolved- Adds missing import statements with nickname supportpython_columns_accessible- Adds missing DataFrame columns with dummy datapython_code_formatted- Fixes indentation and formatting with autopep8python_packages_installed- Auto-installs missing packages with correct mappingpython_paths_fixed- Fixes file path issues (./prefixes, missingdata/)python_auto_fix- Iterative pipeline combining all fixesThree utility modules support these requirements:
data_generators.py- Random data generation for all data typesfile_utils.py- File type predicates / I/O operationsmetadata_utils.py- Directory structure conversion and reconstructionAPI
Testing