-
|
We have come across an interesting scenario, but please let us know if we simply have the wrong architecture. We are trying to copy a file from one location in our application to another. The way we approached this was using an Activity Trigger to download the file and return a byte array to the parent orchestrator. With large files this becomes an issue as the memory consumption can get out of hand during the replays and deserialization. What do you recommend in these scenarios? Should we try to do everything in the Activity Trigger instead of returning the file? |
Beta Was this translation helpful? Give feedback.
Replies: 3 comments 1 reply
-
|
Yes, we strongly recommend doing the large file copy directly in the activity function. Otherwise you're unnecessarily copying the file contents multiple times needlessly (copying it into activity function output, orchestration state, and again into activity input) plus the significant memory overhead that you already mentioned. Was there anything you were concerned about that made you consider passing the file through the orchestration initially? Just want to make sure there aren't any other considerations we should be aware of for your scenario. |
Beta Was this translation helpful? Give feedback.
-
|
@cgillum The original intent was to create microservices, but in this case it got us into trouble as we tried to break things in too small of pieces. That answers my question. While we don't have the issue now, we do commonly work with documents in our application. I could see a use case where we want to pass the document through multiple processes in the application. In these scenarios we would either need to get the document in each activity trigger or perform all the processes in the same activity trigger. You can close, thank you for the prompt response. |
Beta Was this translation helpful? Give feedback.
-
|
No problem. In the case where multiple activities need to operate on the same document, another option you have is to pass some kind of document reference (URL, etc.) to each activity from the orchestrator and let each activity fetch the document using that reference rather than having it provided explicitly as an input. |
Beta Was this translation helpful? Give feedback.
-
|
@cgillum Got it, makes sense. Will let you know if we run into anything else. |
Beta Was this translation helpful? Give feedback.
Yes, we strongly recommend doing the large file copy directly in the activity function. Otherwise you're unnecessarily copying the file contents multiple times needlessly (copying it into activity function output, orchestration state, and again into activity input) plus the significant memory overhead that you already mentioned.
Was there anything you were concerned about that made you consider passing the file through the orchestration initially? Just want to make sure there aren't any other considerations we should be aware of for your scenario.