awsbsub ignores -w option when string command is provided

[mattwelborn]

Environment:

• AWS ParallelCluster / CfnCluster version: aws-parallelcluster-2.3.1
• OS: alinux
• Scheduler: awsbatch
• Master instance type: t2.nano
• Compute instance type: determined by awsbatch

Bug description and how to reproduce:
The -w flag is ignored by awsbsub when a string command is provided as an argument. Looking at cli/awsbatch/awsbsub.py, it can be seen that args.working_dir is only used in _compose_bash_command which is only called if args.command_file or not sys.stdin.isatty() or args.env.

Incorrect behavior example:
Input: awsbsub -w /home/ec2-user pwd
Output:

2019-05-19T21:35:15+00:00: Job id: d519dc44-72e9-45d0-aae9-6eb5e502fafa
2019-05-19T21:35:15+00:00: Initializing the environment...
2019-05-19T21:35:15+00:00: Starting ssh agents...
2019-05-19T21:35:15+00:00: Agent pid 7
2019-05-19T21:35:15+00:00: Identity added: /root/.ssh/id_rsa (/root/.ssh/id_rsa)
2019-05-19T21:35:16+00:00: Mounting /home...
2019-05-19T21:35:16+00:00: Mounting shared file system...
2019-05-19T21:35:16+00:00: Starting the job...
2019-05-19T21:35:16+00:00: /

Correct behavior example:
Input: echo pwd | awsbsub -w /home/ec2-user
Output:

2019-05-19T21:35:18+00:00: Job id: 033c9be8-6964-4a67-86bd-eea069c55ba1
2019-05-19T21:35:18+00:00: Initializing the environment...
2019-05-19T21:35:18+00:00: Starting ssh agents...
2019-05-19T21:35:18+00:00: Agent pid 9
2019-05-19T21:35:18+00:00: Identity added: /root/.ssh/id_rsa (/root/.ssh/id_rsa)
2019-05-19T21:35:19+00:00: Mounting /home...
2019-05-19T21:35:19+00:00: Mounting shared file system...
2019-05-19T21:35:19+00:00: Starting the job...
2019-05-19T21:35:20+00:00: /home/ec2-user

[lukeseawalker]

Hi @mattwelborn,
good catch. The parsing of the -w flag was not implemented when a string command is provided as an argument. This because in this case the command is sent directly to AWS Batch instead of composing a wrapper script (which is in charge to switch the working directory). I'm marking this as bug, in order to have the user experience consistent.
Just for curiosity what's the use case of using a simple command provided as argument?

[mattwelborn]

I was using it in a bash loop in order to run a computationally intensive code on each of a few thousand input files.

[gmarciani]

ParallelCluster v3.15.0 is the last version to support AWS Batch.
Starting with v3.16.0, ParallelCluster will no longer support AWS Batch as a scheduler.
Given this deprecation, there is no plan to address this issue.