Process bottlenecking in docker

[ardydavari]

Bug report

Expected behavior and actual behavior

The process that I am running should be saturating CPU use in all specified threads. Instead it bottlenecks at around 8-12% use per thread.

Steps to reproduce the problem

process multiphylowgs {
    container 'blcdsdockerregistry/docker-phylowgs:2205be1'
    publishDir "./demo/results" + "/chains"

    input:
        path ssmdata
        path cnvdata
        val numchains
        val chaininclusionfactor

    script:
    """
    python2 /phylowgs/multievolve.py \
    --num-chains $numchains \
    --ssms $ssmdata \
    --cnvs $cnvdata
    """
}

Program output

Sep-10 05:47:12.870 [Task submitter] INFO nextflow.Session - [29/158b6f] Submitted process > multiphylowgs

Environment

• Nextflow version: version 21.04.3 build 5560
• Java version: openjdk 11.0.12 2021-07-20
• Operating system: Linux
• Bash version: zsh 5.8

Additional context

If I obtain the docker run command using "docker run --rm -v /var/run/docker.sock:/var/run/docker.sock:ro assaflavie/runlike containername", the command below works appropriately and saturates the cpu use per thread.

docker run --name=nxf-8zfDSQRtiABQ6f33pniSOJNzrp --rm --hostname=5c6c30a285df --mac-address=02:42:ac:11:00:02 --env=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin --volume=/scratch:/tmp --volume=/home/ardy/temp/pipeline/work:/home/ardy/temp/pipeline/work --volume=/home/ardy/temp/pipeline/work/29/158b6f83f065c6f12b22171cb085b7:/home/ardy/temp/pipeline/work/29/158b6f83f065c6f12b22171cb085b7 --workdir=/home/ardy/temp/pipeline/work/29/158b6f83f065c6f12b22171cb085b7 --restart=no --runtime=runc blcdsdockerregistry/docker-phylowgs:2205be1 /bin/bash -ue /home/ardy/temp/pipeline/work/29/158b6f83f065c6f12b22171cb085b7/.command.sh

nextflow config

process {
    executor = 'local'
}


docker {
    enabled = true
    temp = "/scratch"
}

[ardydavari]

I was able to fix the problem by changing

process {
    executor = 'local'
}

to

process {
    cpus = SysHelper.getAvailCpus()
    executor = 'local'
}