Skip to content

Conversation

@0xCUB3
Copy link

@0xCUB3 0xCUB3 commented Oct 25, 2025

Flexible Python code execution validation system with three execution modes:

  • Safe mode (default): Syntax validation only
  • Unsafe execution: Direct subprocess execution with warnings
  • Sandbox execution: Secure Docker-based execution via llm-sandbox

Elements

  • Abstract backend architecture with pluggable execution strategies
  • Import restrictions as an add'l security layer (AST-based analysis)
  • Configurable timeouts for both unsafe and sandbox execution
  • Safety warnings when using unsafe execution mode
  • Fallbacks when llm-sandbox is not available

API

# Safe mode (default) - validation only
req = PythonExecutesWithoutError()

# Unsafe execution with warning
req = PythonExecutesWithoutError(allow_unsafe_execution=True, timeout=10)

# Secure sandbox execution
req = PythonExecutesWithoutError(use_sandbox=True, timeout=10)

# With import restrictions
req = PythonExecutesWithoutError(
    use_sandbox=True,
    allowed_imports=["os", "sys", "json"],
    timeout=10
)

Dependencies

  • Adds llm-sandbox[docker]
  • Requires Docker for sandbox functionality

Testing

# Run core tests (no Docker required)
python -m pytest test/stdlib_basics/test_reqlib_python.py -k "not sandbox"

# Run all tests including sandbox (requires Docker)
python -m pytest test/stdlib_basics/test_reqlib_python.py

No breaking changes. Existing code using PythonExecutesWithoutError() continues to work with safe validation mode.

TODO: Documentation, perhaps more rigorous testing on larger code chunks

- Add PythonExecutesWithoutError requirement with three execution backends:
  - SafeBackend: Validates syntax and imports without execution (default)
  - UnsafeBackend: Direct subprocess execution with warnings
  - LLMSandboxBackend: Docker-based execution using llm-sandbox
- Implement allow_unsafe_execution flag with explicit opt-in and warnings
- Add import restriction support for defense-in-depth security
- Support use_sandbox flag for secure Docker-based execution
- Include comprehensive test suite with 21 test cases
- Maintain backward compatibility while defaulting to safe mode
- Add llm-sandbox[docker] dependency for optional sandbox functionality
Improves code formatting and readability in python.py by splitting long lines, adding whitespace, and updating argument formatting. Also updates test import order in test_reqlib_python.py for consistency.
@mergify
Copy link

mergify bot commented Oct 25, 2025

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert|release)(?:\(.+\))?:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant