Add targeted device selection for CUDA and 128GB M3 Max MPS #5
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR adds intelligent device selection for PyTorch operations, prioritizing CUDA while restricting MPS (Metal Performance Shaders) usage to Apple M3 Max machines with 128GB memory to ensure optimal performance and compatibility.
- Implements device detection utilities for Apple Silicon Macs including memory and chip identification
- Adds a centralized
select_device()function with CUDA priority and restricted MPS support - Updates model loading to be compatible across different device types
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| if platform.system() != "Darwin": | ||
| return None | ||
| try: | ||
| output = subprocess.check_output(["sysctl", "-n", name]) |
There was a problem hiding this comment.
Using subprocess.check_output with user-controlled input could be vulnerable to command injection. Consider validating the name parameter against an allowlist of known sysctl keys before passing it to the subprocess call.
| print( | ||
| "Warning: MPS backend detected but restricted to Apple M3 Max systems " | ||
| "with 128GB memory. " | ||
| f"Detected chip '{chip_name}' with {mem_gb:.1f}GB. Falling back to CPU." | ||
| ) |
There was a problem hiding this comment.
[nitpick] Consider using the logging module instead of print statements for warnings. This would allow better control over log levels and output formatting in production environments.
| print( | ||
| "Warning: MPS backend detected but unable to verify Apple M3 Max 128GB " | ||
| "requirement. Falling back to CPU." | ||
| ) |
There was a problem hiding this comment.
[nitpick] Consider using the logging module instead of print statements for warnings. This would allow better control over log levels and output formatting in production environments.
Summary
select_devicehelper, logging the device choicedevice_map="auto", keeping compatibility across CUDA, MPS, and CPUTesting
https://chatgpt.com/codex/tasks/task_e_68c879619398832c81d48af0ee412241