Feature Requests

[giovtorres]

Use this discussion to suggest improvements and other ideas to the project.

[kinow]

Not sure if that'd be feasible, but have you considered a single lite container, that contains everything needed? We use that for pytest integration tests using TestContainer to launch a small slurm server -- https://github.com/BSC-ES/autosubmit/blob/2ce789ecbe1d646ad5cdfa23256bb2f14b6bc5bf/test/integration/test_utils/docker.py#L184-L227

Our goal is not to test any Slurm features. What we need the most in our cause (an HPC workflow manager) is to validate the slurm directives and commands we use, and that our Shell scripts (that contain a header/tailer plus the user shell script) runs correctly. We don't run anything complicated in our tests, and that's enough to cover a lot of our code related to Slurm.

We stated using this image, but realized that it didn't have SSH (I think it's being added now) which is needed as we submit jobs via SSH to HPC clusters, and it would require launching and managing more containers on GH Actions.

Thanks!

[kinow]

I wasn't aware of that one! Do you know if it'd work with Slurm 25.05? If that image works with this version of Slurm, and if it supports SSH, I'd prefer to use this image as there will be other users/community using/reporting issues/etc.

I am participating in some tasks in another project, Toil, that also uses Slurm for tests: https://github.com/DataBiosphere/toil/blob/master/contrib/slurm-test/docker-compose.yml

I'll share the link of this image with one of the maintainers of that tool that I work with. Maybe that would be 2 users already :)

Thanks

[giovtorres]

Oof, i'm not sure about the version support. I haven't touched that project in a while. I'll have to take a look. What were your requirements, an all-in-one container with slurm, ssh, and one or more python versions?

[kinow]

• Slurm 25.05, although an earlier version could be OK, the two main HPCs we support have 24.05.8 and 23.02.7
• Bash 4.4 or superior, or 5.1 or superior
• SSH like OpenSSH (other versions could work, as long as Paramiko supports)

We run only Shell in the Slurm jobs. But having a version of Python there is OK too. In the real world most of our workflows load lua modules, but we are not doing that in our tests (trying to keep the focus on the workflow manager itself).

If you accept suggestions of other features... maybe these could be useful for the other images too:

• SSH key and password auth (with EuroHPC in EU, projects are being done more with consortiums sharing their HPCs... some require password sometimes 🤷)
• SSH 2FA (ditto the above)
• Option to enable/disable SSH settings (e.g. everything related to throttling, anti-attack, etc., so it doesn't break our CICD pipelines)
• Option to install extra packages on the server & maybe enable/disable Slurm plugins (e.g. energy, this one is important for us, but we are not using yet)

Thanks!

[giovtorres]

check out: https://github.com/giovtorres/slurm-docker

[kinow]

That was quick, @giovtorres !

# CI/CD friendly defaults (no throttling)
MaxStartups 100:30:200
LoginGraceTime 120
MaxAuthTries 20
UseDNS no

Thanks a lot!

I had to make a few adjustments as we leaked some of our code requirements into the container (like creating an Autosubmit folder). For others, this is the command I used to test SSH:

docker run --rm -ti --init --privileged --hostname=slurmctl --env MYSQL_USER=autosubmit --env MYSQL_PASSWORD=autosubmit --env ROOT_PASSWORD=autosubmit --env SSH_PORT=2222 --env SSH_PERMIT_ROOT_LOGIN=yes --env SSH_PASSWORD_AUTH=yes -p 2222:2222 giovtorres/slurm-docker:25.11.2-v0.1.5

With that I could successfully use SSH and sftp. Then it was just a matter of adjusting the SSH key, the log message expected on the output by TestContainers, and I got my first successful Slurm test locally with pytest. I've pushed to GH Actions now and am waiting for the first build to see if I have to fix anything else.

I saw the option to include more packages, but I think we got everything we need from that image for now.

Thanks @giovtorres !!! ❤️