sh fails with process apparently never started on kubernetes when using dir #523
Description
Jenkins and plugins versions report
Environment
Jenkins: 2.541
OS: Linux - 6.1.158
Java: 21.0.9 - Eclipse Adoptium (OpenJDK 64-Bit Server VM)
---
active-directory:2.41
allure-jenkins-plugin:2.34.0
amazon-ecr:1.161.v1a_1e8df852d6
analysis-model-api:13.17.0-931.v730d8659ea_9c
ansicolor:1.0.6
ant:520.vd082ecfb_16a_9
antisamy-markup-formatter:173.v680e3a_b_69ff3
apache-httpcomponents-client-4-api:4.5.14-269.vfa_2321039a_83
apache-httpcomponents-client-5-api:5.5-170.v023de017ccd7
artifact-manager-s3:962.v15f7000205fa_
asm-api:9.9-185.va_6c6b_3348b_c3
atlassian-jira-software-cloud:2.0.15
audit-trail:436.vc0d1e79fc5a_3
authentication-tokens:1.144.v5ff4a_5ec5c33
authorize-project:2.0.0
aws-credentials:254.v978a_5e206a_d7
aws-global-configuration:152.ve13a_2a_c319a_9
aws-java-sdk:1.12.780-480.v4a_0819121a_9e
aws-java-sdk-api-gateway:1.12.780-480.v4a_0819121a_9e
aws-java-sdk-autoscaling:1.12.780-480.v4a_0819121a_9e
aws-java-sdk-cloudformation:1.12.780-480.v4a_0819121a_9e
aws-java-sdk-cloudfront:1.12.780-480.v4a_0819121a_9e
aws-java-sdk-cloudwatch:1.12.780-480.v4a_0819121a_9e
aws-java-sdk-codebuild:1.12.780-480.v4a_0819121a_9e
aws-java-sdk-codedeploy:1.12.780-480.v4a_0819121a_9e
aws-java-sdk-ec2:1.12.780-480.v4a_0819121a_9e
aws-java-sdk-ecr:1.12.780-480.v4a_0819121a_9e
aws-java-sdk-ecs:1.12.780-480.v4a_0819121a_9e
aws-java-sdk-efs:1.12.780-480.v4a_0819121a_9e
aws-java-sdk-elasticbeanstalk:1.12.780-480.v4a_0819121a_9e
aws-java-sdk-elasticloadbalancingv2:1.12.780-480.v4a_0819121a_9e
aws-java-sdk-iam:1.12.780-480.v4a_0819121a_9e
aws-java-sdk-kinesis:1.12.780-480.v4a_0819121a_9e
aws-java-sdk-lambda:1.12.780-480.v4a_0819121a_9e
aws-java-sdk-logs:1.12.780-480.v4a_0819121a_9e
aws-java-sdk-minimal:1.12.780-480.v4a_0819121a_9e
aws-java-sdk-organizations:1.12.780-480.v4a_0819121a_9e
aws-java-sdk-secretsmanager:1.12.780-480.v4a_0819121a_9e
aws-java-sdk-sns:1.12.780-480.v4a_0819121a_9e
aws-java-sdk-sqs:1.12.780-480.v4a_0819121a_9e
aws-java-sdk-ssm:1.12.780-480.v4a_0819121a_9e
aws-java-sdk2-core:2.33.4-62.vc1a_8df64b_4c9
aws-java-sdk2-ec2:2.33.4-62.vc1a_8df64b_4c9
aws-java-sdk2-ecr:2.33.4-62.vc1a_8df64b_4c9
aws-java-sdk2-s3:2.33.4-62.vc1a_8df64b_4c9
aws-lambda:0.5.10
aws-parameter-store:1.2.2
aws-secrets-manager-credentials-provider:1.214.va_0a_d8268d068
badge:2.560.v5135054c2e76
basic-branch-build-strategies:275.vde2351b_4a_58b_
bootstrap5-api:5.3.8-895.v4d0d8e47fea_d
bouncycastle-api:2.30.1.82-277.v70ca_0b_877184
branch-api:2.1268.v044a_87612da_8
build-discarder:158.vce570d01ce4c
build-name-setter:2.5.1
build-timeout:1.38
build-token-trigger:1.0.0
build-user-vars-plugin:195.v8c35f9d5c3dc
build-with-parameters:81.ve4a_9c2499d9a
built-on-column:1.5
caffeine-api:3.2.3-194.v31a_b_f7a_b_5a_81
changelog-history:1.7
checks-api:373.vfe7645102093
cloudbees-folder:6.1073.va_7888eb_dd514
cobertura:1.17
code-coverage-api:4.99.0
command-launcher:123.v37cfdc92ef67
commons-compress-api:1.28.0-2
commons-lang3-api:3.20.0-109.ve43756e2d2b_4
commons-text-api:1.15.0-210.v7480a_da_70b_9e
conditional-buildstep:1.5.0
config-file-provider:1006.vc7366c201f57
copyartifact:770.va_6c69e063442
coverage:2.3060.v035a_5557cdb_c
credentials:1453.v9b_a_29777a_b_fd
credentials-binding:702.vfe613e537e88
cron_column:83.vd52c9b_5c758c
custom-folder-icon:2.938.vb_42c40c33ea_5
dark-theme:574.va_19f05d54df5
dashboard-view:2.543.vca_9da_3cb_9c60
data-tables-api:2.3.5-1482.vb_47a_3c19f78c
datadog:9.1.8
delivery-pipeline-plugin:1.4.2
description-setter:264.v1957f215dcd5
devops-portal:415.vaf57f86bb4d1
disable-github-multibranch-status:1.2
display-url-api:2.217.va_6b_de84cc74b_
docker-commons:457.v0f62a_94f11a_3
docker-workflow:634.vedc7242b_eda_7
dockerhub-notification:222.v483263f10718
downstream-build-cache:1.7
dtkit-api:3.0.3
durable-task:639.vefb_3d8372f6d
ec2:2043.v483cc2854116
echarts-api:6.0.0-1165.vd1283a_3e37d4
eddsa-api:0.3.0.1-19.vc432d923e5ee
email-ext:1933.v45cec755423f
emailext-template:233.v1eb_88fc160b_5
embeddable-build-status:627.v0249270b_488e
emoji-symbols-api:17.0-57.v8d44b_9a_b_d5ea_
envinject:2.926.v69c9b_3896a_96
envinject-api:1.236.v35fd4d7eb_515
extended-read-permission:68.vd270568a_7520
extended-security-settings:1.3
extensible-choice-parameter:250.va_1cf60782b_1a_
external-monitor-job:223.vb_fddcf42c9b_3
file-parameters:412.vf26e5b_6e7431
folder-properties:62.v1636b_4a_84608
font-awesome-api:7.1.0-882.v1dfb_771e3278
forensics-api:3.1772.v99ca_3d83b_9fa_
generic-webhook-trigger:2.4.1
git:5.8.1
git-client:6.4.1
git-forensics:3.2138.vf25ea_d549e33
git-parameter:460.v71e7583a_c099
github:1.45.0
github-api:1.330-492.v3941a_032db_2a_
github-autostatus:3.6.2
github-branch-source:1925.v62fb_7ffb_08ce
github-checks:634.v371dc6d978a_3
github-label-filter:1.0.0
github-pr-comment-build:134.ve7ff0b_719821
github-scm-filter-aged-refs:59.vc6b_39c26988c
github-scm-trait-notification-context:45.v8ef831829589
global-slack-notifier:1.5
golang:1.4
google-oauth-plugin:1.335.ve6de40e2db_18
gradle:2.16.1149.v711b_998b_0532
groovy-postbuild:272.v52a_03efb_8653
gson-api:2.13.2-173.va_a_092315913c
handy-uri-templates-2-api:2.1.8-38.vcea_5d521d5f3
hidden-parameter:504.v610a_b_b_c05003
htmlpublisher:427
http_request:1.23
instance-identity:203.v15e81a_1b_7a_38
ionicons-api:94.vcc3065403257
jackson2-api:2.20.1-423.v13951f6b_6532
jacoco:3.3.7
jakarta-activation-api:2.1.4-1
jakarta-mail-api:2.1.5-1
jakarta-xml-bind-api:4.0.6-10.v9b_7e1d1fc40b_
javadoc:354.vee1a_660b_4990
javax-activation-api:1.2.0-8
javax-mail-api:1.6.2-11
jaxb:2.3.9-133.vb_ec76a_73f706
jdk-tool:83.v417146707a_3d
jenkinslint:0.14.0
jfrog:1.5.10
jira-steps:2.0.180.vccfe35b_5910d
jjwt-api:0.11.5-120.v0268cf544b_89
jnr-posix-api:3.1.21-186.vb_7ec9b_23ce83
job-dsl:1.93
job-parameter-summary:0.5
jobConfigHistory:1356.ve360da_6c523a_
jobcacher:764.v098c635e4315
joda-time-api:2.14.0-149.v1c3ce991d1b_9
jquery:1.12.4-3
jquery3-api:3.7.1-619.vdb_10e002501a_
jsch:0.2.16-95.v3eecb_55fa_b_78
json-api:20250517-173.v596efb_962a_31
json-path-api:2.10.0-202.va_9cc16c1e476
jsoup:1.21.2-66.v6ea_38164b_8a_2
junit:1380.v491ff054cd35
kubernetes:4398.vb_b_33d9e7fe23
kubernetes-cli:1.364.vadef8cb8b823
kubernetes-client-api:7.3.1-256.v788a_0b_787114
kubernetes-credentials:206.vde31a_b_0f71a_c
kubernetes-pipeline-devops-steps:1.6
ldap:793.v754d6b_41b_ea_4
list-git-branches-parameter:0.0.13
locale:614.va_6a_5a_1a_f2b_38
lockable-resources:1438.v3c0f8c9e2060
login-theme:284.v95044396a_484
mailer:525.v2458b_d8a_1a_71
mapdb-api:1.0.9-44.va_1e1310c9118
mask-passwords:212.v4967a_a_73b_506
matrix-auth:3.2.9
matrix-project:870.v9db_fcfc2f45b_
maven-plugin:3.27
mercurial:1323.ve69d2a_db_8a_b_d
metrics:4.2.37-489.vb_6db_69b_ce753
mina-sshd-api-common:2.16.0-167.va_269f38cc024
mina-sshd-api-core:2.16.0-167.va_269f38cc024
mina-sshd-api-scp:2.16.0-167.va_269f38cc024
multibranch-action-triggers:1.8.10
node-iterator-api:72.vc90e81737df1
nodejs:1.6.6
oauth-credentials:0.657.v7d8dd90b_0382
okhttp-api:4.12.0-195.vc02552c04ffd
oss-symbols-api:424.ved751e062911
ownership:0.13.0
parameter-separator:296.v9b_a_90c81160d
parameterized-scheduler:285.ve611986d4c48
parameterized-trigger:873.v8b_e37dd8418f
pipeline-aggregator-view:119.v4b_ec11953552
pipeline-aws:1.45
pipeline-build-step:571.v08a_fffd4b_0ce
pipeline-config-history:1.6
pipeline-github:2.8-162.382498405fdc
pipeline-github-lib:65.v203688e7727e
pipeline-githubnotify-step:49.vf37bf92d2bc8
pipeline-gitstatuswrapper:1.2.0
pipeline-graph-analysis:245.v88f03631a_b_21
pipeline-graph-view:724.v41764e7b_3176
pipeline-groovy-lib:787.ve2fef0efdca_6
pipeline-input-step:540.v14b_100d754dd
pipeline-milestone-step:138.v78ca_76831a_43
pipeline-model-api:2.2277.v00573e73ddf1
pipeline-model-definition:2.2277.v00573e73ddf1
pipeline-model-extensions:2.2277.v00573e73ddf1
pipeline-multibranch-defaults:2.1
pipeline-rest-api:2.38
pipeline-stage-step:322.vecffa_99f371c
pipeline-stage-tags-metadata:2.2277.v00573e73ddf1
pipeline-stage-view:2.38
pipeline-timeline:1.0.3
pipeline-utility-steps:2.20.0
plain-credentials:199.v9f8e1f741799
platformlabeler:2734.v604521b_646b_b_
plugin-usage-plugin:4.10
plugin-util-api:6.1192.v30fe6e2837ff
postbuildscript:3.4.1-695.vf6b_0b_8053979
prism-api:1.30.0-630.va_e19d17f83b_0
prometheus:819.v50953a_c560dd
resource-disposer:0.25
role-strategy:848.va_a_ea_673cf0b_c
run-condition:243.v3c3f94e46a_8b_
s3-jobcacher-storage:62.v260cb_8db_5b_fa_
saml:4.590.v6170b_1013da_f
scm-api:724.v7d839074eb_5c
scm-filter-branch-pr:222.v72301ecb_a_ee7
script-security:1385.v7d2d9ec4d909
simple-theme-plugin:211.v5424a_5510e47
skip-notifications-trait:576.v7a_67396e2e0b_
slack:795.v4b_9705b_e6d47
snakeyaml-api:2.5-143.v93b_c004f89de
solarized-theme:28.vfe25223f14fe
splunk-devops:1.11.1
splunk-devops-extend:1.11.1
ssh-agent:386.v36cc0c7582f0
ssh-credentials:361.vb_f6760818e8c
ssh-slaves:3.1085.vc64d040efa_85
sshd:3.374.v19b_d59ce6610
structs:362.va_b_695ef4fdf9
subversion:1292.ve8cf25770ee3
templating-engine:2.5.5
test-results-aggregator:3.2
theme-manager:327.v780d7096ec29
throttle-concurrents:2.18
timestamper:1.30
token-macro:477.vd4f0dc3cb_cf1
trilead-api:2.284.v1974ea_324382
uno-choice:2.8.8
variant:70.va_d9f17f859e0
versioncolumn:320.v6b_b_814ca_01f7
warnings-ng:12.9848.v5603977006f2
workflow-aggregator:608.v67378e9d3db_1
workflow-api:1398.v67030756d3fb_
workflow-basic-steps:1098.v808b_fd7f8cf4
workflow-cps:4238.va_6fb_65c1f699
workflow-durable-task-step:1464.v2d3f5c68f84c
workflow-job:1559.va_a_533730b_ea_d
workflow-multibranch:821.vc3b_4ea_780798
workflow-scm-step:466.va_d69e602552b_
workflow-step-api:710.v3e456cc85233
workflow-support:1010.vb_b_39488a_9841
ws-cleanup:0.49
xcode-plugin:2.0.17-565.v1c48051d46ef
xunit:3.1.6
yet-another-build-visualizer:1.17
What Operating System are you using (both controller, and any agents involved in the problem)?
linux
Reproduction steps
stage('static-code-analysis') {
agent {
kubernetes {
defaultContainer 'php'
yaml """
apiVersion: v1
kind: Pod
spec:
securityContext:
runAsGroup: 0
runAsUser: 0
containers:
- name: php
image: ${imagePhp}:${imageTag}
command:
- cat
tty: true
"""
}
}
steps {
dir('/srv/app') {
sh 'pwd'
}
}Expected Results
I would expect the sh command to run from /srv/app
Actual Results
the dreaded process apparently never started error appears:
22:50:22 cp: can't stat '/srv/app@tmp/durable-9d83efe6/script.sh': No such file or directory
22:50:22 sh: can't create /srv/app@tmp/durable-9d83efe6/jenkins-log.txt: nonexistent directory
22:50:22 sh: can't create /srv/app@tmp/durable-9d83efe6/jenkins-result.txt.tmp: nonexistent directory
22:50:22 mv: can't rename '/srv/app@tmp/durable-9d83efe6/jenkins-result.txt.tmp': No such file or directory
22:50:22 [Pipeline] sh
22:50:23 cp: can't stat '/srv/app@tmp/durable-030df48e/script.sh': No such file or directory
22:50:23 sh: can't create /srv/app@tmp/durable-030df48e/jenkins-log.txt: nonexistent directory
22:50:23 sh: can't create /srv/app@tmp/durable-030df48e/jenkins-result.txt.tmp: nonexistent directory
22:50:23 mv: can't rename '/srv/app@tmp/durable-030df48e/jenkins-result.txt.tmp': No such file or directory
23:08:37 process apparently never started in /srv/app@tmp/durable-030df48e
23:08:37 (running Jenkins temporarily with -Dorg.jenkinsci.plugins.durabletask.BourneShellScript.LAUNCH_DIAGNOSTICS=true might make the problem clearer)
Anything else?
I've looked into related issues but couldn't find anything that really matches.. it doesn't seem to be a permission error, since we're running as uid 0 in that pod:
11:09:43 + whoami
11:09:43 root
11:09:44 [Pipeline] sh
11:09:45 + ls -lA /srv/
11:09:45 total 0
11:09:45 drwxr-xr-x 1 root root 37 Dec 12 09:41 app
11:09:46 [Pipeline] sh
11:09:47 + ls -lA /srv/app/
11:09:47 total 448
11:09:47 -rw-rw-r-- 1 root root 1713 Dec 12 09:36 .env
11:09:47 -rw-rw-r-- 1 root root 47 Dec 12 09:36 .env.testing
11:09:47 drwxr-xr-x 11 root root 177 Dec 12 09:36 app
11:09:47 -rwxrwxr-x 1 root root 425 Dec 12 09:36 artisan
11:09:47 drwxr-xr-x 1 root root 19 Dec 12 09:36 bootstrap
11:09:47 drwxr-xr-x 2 root root 4096 Dec 12 09:36 config
11:09:47 drwxr-xr-x 5 root root 74 Dec 12 09:36 database
11:09:47 drwxr-xr-x 2 root root 103 Dec 12 09:36 public
11:09:47 drwxr-xr-x 5 root root 51 Dec 12 09:36 resources
11:09:47 drwxr-xr-x 2 root root 55 Dec 12 09:36 routes
11:09:47 drwxr-xr-x 5 root root 46 Dec 12 09:36 storage
11:09:47 drwxr-xr-x 6 root root 84 Dec 12 09:36 tests
11:09:47 drwxr-xr-x 1 root root 22 Dec 12 09:41 vendor
11:09:48 [Pipeline] dir
11:09:48 Running in /srv/app
11:09:48 [Pipeline] {
11:09:49 [Pipeline] sh
11:14:56 process apparently never started in /srv/app@tmp/durable-fb2090b2
11:14:56 (running Jenkins temporarily with -Dorg.jenkinsci.plugins.durabletask.BourneShellScript.LAUNCH_DIAGNOSTICS=true might make the problem clearer)
I've been trying to find any cause but I'm stumped. We don't seem to have this issue on other builds, so I assumed it must be related to the image somehow, but I can't find an explanation and would appreciate any help you can give. Thanks!
Are you interested in contributing a fix?
I'd love to but I don't even know where to start.