EntityStreamSizeException with a URL input file

[sulsj]

Hi,
We are trying to use a URI input like

{
  "fq_count.fastq_file": "https://portal.nersc.gov/project/m3408/test_data/Ecoli_10x-int.fastq.gz"
}

, and we are seeing this exception from Cromwell.

023-01-11 23:00:02 cromwell-system-akka.stream.default-blocking-io-dispatcher-91 ERROR - Tearing down FileSink(/.../xxx
 249 akka.http.scaladsl.model.EntityStreamSizeException: EntityStreamSizeException: incoming entity size (43033910) exceeded size limit (8388608 by
 250         at akka.http.scaladsl.model.HttpEntity$Limitable$$anon$1.preStart(HttpEntity.scala:631)

It worked fine when we tried with a smaller file (smaller than 8388608).

Does anyone have any idea?

Thansk,
Seung

[sulsj]

OK. Found we need to set akka.http.client.parsing.max-content-length like the below.

  1 include required(classpath("application"))
  2 webservice
  3 {
  4   port = 50103
  5   interface = 127.0.0.1
  6 }
  7
  8 akka.http.server.parsing.max-content-length = 1024m
  9 akka.http.client.parsing.max-content-length = 1024m
 10
 11 system
 12 {
 13   abort-jobs-on-terminate = true
 14   graceful-server-shutdown = true
 15   workflow-restart = false
 16   max-concurrent-workflows = 100000
 17   max-workflow-launch-count = 100000
 18   new-workflow-poll-rate = 1
 19   number-of-workflow-log-copy-workers = 20
 20   number-of-cache-read-workers = 50
 21   job-rate-control
 22   {
 23     jobs = 1
 24     per = 1 second
 25   }
 26 }