You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[SPARK-46079][INFRA] Install torch nightly only at Python 3.12 in Infra docker image
### What changes were proposed in this pull request?
This PR aims to install `torch` nightly distribution and `torcheval` for Apache Spark 4.0.0 preparation.
### Why are the changes needed?
Currently, `pytorch` is not available in Python 3.12 environment.
- pytorch/pytorch#110436
We can start to use `nightly` build and switch to the official one via SPARK-46078 when it's released.
### Does this PR introduce _any_ user-facing change?
No. This is an infra only update.
### How was this patch tested?
1. Passed the image building.

2. Verified manually
```
$ docker run -it --rm ghcr.io/dongjoon-hyun/apache-spark-ci-image:master-6975332773 python3.12 -m pip freeze | grep torch
torch==2.2.0.dev20231123+cpu
torcheval==0.0.7
```
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes#43988 from dongjoon-hyun/SPARK-46079.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
0 commit comments