Add support for custom Volumes/Volume Mounts

[Bobbins228]

Issue link

What changes have been made

Verification steps

Setup

Notebook server ODH/RHOAI/Local

• Clone this repository with git clone https://github.com/project-codeflare/codeflare-sdk.git
• Checkout this PR's branch
• Run poetry build - install if needed (pip install poetry)
• Run pip install --force-reinstall dist/codeflare_sdk-0.0.0.dev0-py3-none-any.whl
• Restart your notebook kernel

Testing

The volumes and volume_mounts config parameters are both lists that take the V1Volume and V1VolumeMount objects respectively.
Below I have given some examples for testing purposes.

Volumes/Volume Mounts configuration

from kubernetes.client import V1Volume, V1VolumeMount, V1EmptyDirVolumeSource, V1ConfigMapVolumeSource, V1KeyToPath, V1SecretVolumeSource
volume_mounts = [
    V1VolumeMount(
        mount_path="/home/ray/test1",
        name = "test"
    ),
    V1VolumeMount(
        mount_path = "/home/ray/test2",
        name = "test2",
    ),
    V1VolumeMount(
        mount_path = "/home/ray/test3",
        name = "test3",
    )
]

volumes = [
    V1Volume(
        name="test",
        empty_dir=V1EmptyDirVolumeSource(size_limit="2Gi"),
    ),
    V1Volume(
        name="test2",
        config_map=V1ConfigMapVolumeSource(
            name="test-config-map",
            items=[V1KeyToPath(key="test", path="data.txt")]
        )
    ),
    V1Volume(
        name="test3",
        secret=V1SecretVolumeSource(
            secret_name="test-secret"
    )
)
]

ClusterConfiguration

cluster = Cluster(ClusterConfiguration(
    name='raytest', 
    head_cpu_requests='500m',
    head_cpu_limits='500m',
    head_memory_requests=2,
    head_memory_limits=2,
    head_extended_resource_requests={'nvidia.com/gpu':0}, 
    worker_extended_resource_requests={'nvidia.com/gpu':0},
    num_workers=2,
    worker_cpu_requests='250m',
    worker_cpu_limits=1,
    worker_memory_requests=4,
    worker_memory_limits=4,
    appwrapper=True,
    # image="", # Optional Field 
    write_to_file=True, # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources 
    # local_queue="local-queue-name" # Specify the local queue manually
    volumes=volumes,
    volume_mounts=volume_mounts,
))

Test Config Map

kind: ConfigMap
apiVersion: v1
metadata:
  name: test-config-map
  namespace: YOUR_NAMESPACE
immutable: false
data:
  test: pass

Test Secret

• Create a secret on OpenShift by going to Workloads -> Secrets -> Create -> key/vaule secret
• Set the name to test-secret
• The key/value can be whatever you like

Verify

The volumes & volume mounts should be added to your worker and head pods.
You can exec into your pod and check that the emptyDir, Config Map volume and Secret Volume were mounted correctly

Checks

• I've made sure the tests are passing.
• Testing Strategy
  • Unit tests
  • Manual tests
  • Testing is not required for this change

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 90.40%. Comparing base (6b0a3cc) to head (2d479cf).
Report is 3 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #772      +/-   ##
==========================================
+ Coverage   90.33%   90.40%   +0.07%     
==========================================
  Files          23       23              
  Lines        1345     1355      +10     
==========================================
+ Hits         1215     1225      +10     
  Misses        130      130              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Bobbins228]

Target 2.17

[KPostOffice]

Once small comment. Should simplify the code a bit, otherwise this LGTM

[astefanutti]

@cmisale FYI that should fix #774.

[ChristianZaccaria]

/lgtm !

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: astefanutti, ChristianZaccaria

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [ChristianZaccaria,astefanutti]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment