How to specify memory to an allocation?

[jemunro]

I'm trying:

hq alloc add slurm --time-limit 1h --cpus 2 --resource 'mem=2GB' --max-worker-count 4 -- --ntasks 2 --cpus-per-task 1 --mem 2G

but I get this error:

error: Invalid value "mem=2GB" for '--resource <RESOURCE>': Unexpected token found while attempting to parse resource kind, expected _ or x:
  mem=2GB
       |
       --- Unexpected token `G`


For more information try --help

[spirali]

Hello,

In the current version, resources does not support suffixes in definitions. In your case, you have to put number of bytes, i.e. 2147483648 (= 2 * 1024 ^ 3).

I am now working on new internals for resource management, so I will try to improve this in the next release.

[jemunro]

Hi,

Thanks for replying. Suffixes support would be great!

I've tried with your suggestion but it still gives me an error:

hq alloc add slurm --time-limit 1h --cpus 2 --resource mem=2147483648 --max-worker-count 4

error: Invalid value "mem=2147483648" for '--resource <RESOURCE>': Unexpected end of input found while attempting to parse resource kind, expected _ or x:
  mem=2147483648
                |
                --- Unexpected end of input


For more information try --help

[jemunro]

Have you thought about allowing specifying memory in the same way as CPUs?
e.g.

hq alloc add slurm --time-limit 1h --cpus 8 --mem 16G

[jemunro]

The reason I'm asking about this is that, at least on my system, hyperqueue isn't able to autodetect the amount of memory available within the slurm job. It seems to detect the total memory on the node where the job is running.

Another option, at least on SLURM, would be to use the SLURM environment variables SLURM_MEM_PER_NODE/SLURM_MEM_PER_CPU/SLURM_MEM_PER_GPU to autodetect the memory. E.g.

MEM=$((SLURM_MEM_PER_NODE * SLURM_JOB_NUM_NODES))
MEM=$((MEM + SLURM_MEM_PER_CPU * SLURM_CPUS_ON_NODE))
MEM=$((MEM + SLURM_MEM_PER_GPU * SLURM_GPUS_ON_NODE))

[spirali]

Ah, I am sorry. I have overlooked that you did not specified the resource type. The right argument is:

--resource mem=sum(2147483648)

What the "sum" means is defined here: https://it4innovations.github.io/hyperqueue/stable/jobs/gresources/

[spirali]

--mem seems like a good idea. If we do not find the way how to slightly generalize it for user resources, it is a good fallback

[spirali]

We will fix the detection. Thank you for finding the problem.

Btw: If you do not submit tasks that asks for particular size of memory (e.g. hq submit --resource mem=1234) than it is not a problem that detection fails.

[Kobzol]

HQ now detects available memory from Slurm: #987, so I think that this is resolved. You can follow #496 for the --mem 2GB prefix feature. Let us know if you have any further questions!