Consider upgrading pytorch version

[planetf1]

When adding integration tests for #212 I noted I was unable to run alora training on a macbook due to:

ValueError: Grad Scaler with MPS device requires a Pytorch >= 2.8.0

We are currently using 2.7.0 ( 2025-04-23)

2.10.0 -- 2026-01-21
2.9.1  -- 2025-11-20 
2.9.0  -- 2025-10-15 
2.8.0  -- 2025-07-30

Obviously a pytorch upgrade is significant given our usage

We use transformers 4.57.6 as mentioned in #367. This appears to have been validated against recent pytorch versions including 2.10.0

[planetf1]

Summary of changes 2.8 through to 2.10

1. Major Deprecations & Removals
TorchScript Deprecation (2.10): TorchScript is officially deprecated in 2.10.0. The ecosystem is moving towards torch.export as the primary way to capture and export models.

TorchAudio Refactoring (2.8/2.9): In 2.8.0, many TorchAudio APIs were deprecated as the library transitioned into a maintenance phase. Many of these were removed in 2.9.0, with audio/video I/O capabilities consolidating into TorchCodec.

Legacy Profiler (2.8): The old profiler is being deprecated in favor of the newer Memory Snapshot API (torch.cuda.memory._record_memory_history).

2. Hardware & Driver Gotchas
GPU Architecture Support (2.8/2.9): Support for older architectures (Maxwell, Pascal, and Volta) has been dropped in CUDA 12.8+ builds. If you are using these older GPUs, you must stick to the CUDA 12.6 builds of PyTorch to maintain compatibility.

macOS Ventura (2.9): Support for macOS Ventura (13) ended with the 2.9.0 release.

Python 3.14 (2.10): Support for Python 3.14 and the experimental "freethreaded" (no-GIL) build was introduced in 2.10.0.

3. Changes in Compilation (torch.compile)
Graph Break Management (2.9): A new context manager, torch._dynamo.error_on_graph_break(), allows you to force an error if a graph break occurs in a specific region, which is more flexible than the older fullgraph=True flag.

Deterministic Mode (2.10): torch.compile now respects use_deterministic_algorithms(True), ensuring that the compilation process itself is reproducible across runs.

Hierarchical Compilation (2.8): You can now mark regions with nested_compile_region to reduce compile times for repeated structures like transformer layers.

4. Breaking API Changes
ONNX Export (2.10): The default for torch.onnx.export changed to fallback=False and dynamo=True. This is the new recommended path, but it may break legacy export scripts that rely on the old symbolic tracer.

Distributed Slicing (2.10): Slicing a flattened dimension from a DeviceMesh now triggers a warning as the API moves toward more explicit mesh definitions.

It doesn't appear as if we use these but specific hw environments may have additional constraints.

Support for Python 3.14 is useful to reduce friction.