Sending email with attachment is failing when using awsbatch executor

[markpanganiban]

I've tested a couple of different versions of sending email but usually still ends up having an error when using awsbatch as executor for both DSL1 and DSL2. Below is the debug and trace logs of the most recent testing using the latest nextflow version. This works when tested with Slurm.

EDIT: Updated the logs from today's run using 22.10.0


Oct.-31 18:15:33.186 [Actor Thread 9] ERROR nextflow.extension.OperatorImpl - @unknown
javax.mail.MessagingException: The following attachment file does not exist: /nf-mpanganiban/scratch/f0/4564d7c7a4c32bc379973160d719fb/ASD_output.txt
	at nextflow.mail.Mailer.createAttachment(Mailer.groovy:339)
	at nextflow.mail.Mailer.createMimeMessage(Mailer.groovy:328)
	at nextflow.mail.Mailer.send(Mailer.groovy:421)
	at nextflow.mail.Mailer.send(Mailer.groovy:496)

Replication:

#!/usr/bin/env nextflow

nextflow.enable.dsl=2

process generate_email {
    debug true
    input:
        val(input_file_content)

    output:
        path "*_output.txt"

    script:
        """
        echo "${input_file_content}" > sample_output.txt
        """
}

workflow { 
    main: 
        channel.from("Here is some text")
        | generate_email
        | flatten
        | map { file_attachment ->
             sendMail {
                 to '[email protected]'
                 attach "${file_attachment}"
                 subject "TEST EMAIL"
                 """
                 TEST CONTENT
                 """
             }
        }
}

Full Logs: nextflow_s3.log

[pditommaso]

please include the full .nextflow.log file

[pditommaso]

and @robsyme knows the cause of this

[markpanganiban]

Full Logs: nextflow_s3.log

[pditommaso]

The problem is that the attachment file path is wrapped into a string

     attach "${file_attachment}"

it should be

   attach file_attachment

Files are not strings!

[pditommaso]

Now it makes sense. Essentially S3 files are remote objects, they cannot used for email attachments

[robsyme]

Thanks Paolo, Mark.
Can I confirm that (at least for the moment), sending emails with attachments that are S3 objects is not supported? Is this something that we'd like to make available as a feature?

[pditommaso]

in principle, it should not be to too hard. It should be checked if it's a remote object and copy to a tmp File. But I'm already expecting users to create several gigas attachments 😄

[robsyme]

From what I've seen, people are being very sensible with their email attachment sizes. I'd hope that the email server will bounce anything unreasonable.
Would it be helpful if I wrote this up as an issue with the kind/feature tag?

[pditommaso]

agreed

[robsyme]

Feature request: #3349