Is it possible to send binary files as task parameters?

[yigitest]

Hi,
I am developing a wrapper project around this library using fastapi.

1. Is there a way to use a binary file as a task parameter? for example, I want to send a zip file to a remote worker and process it there. And get the results back (again as a zip file).

2. I guess this will not be possible as the task/job parameters are stored as jsonb fields in the DB.
   I'm thinking of creating a table myself in the same database as procrastinate and uploading my input and output files there.

   Can I access the psycopg2 connection used by procrastinate from my task/job? So I can access the files without opening another connection pool to the DB.

3. Or do you have another suggestion? :)

[ewjoachim]

I'd say there are 2 ways:

• Either integrate your binary file in the payload, for example encode your payload as base64 and consider it as a string. This works if files are small, because you'll be storing them directly in the database
• Or store the file somewhere and send an identifier to the file (e.g. store the file in S3 and send the s3:// url). This is a generalization over your suggestion of creating another table. Given that it will be kept outside of procrastinate anyway, a table may not be the best way, depending on your file size (postgresql is not a filesystem).

Also, keep in mind that the way it's done today, procrastinate doesn't provide a way of getting results back. Tasks are responsible for storing their results somewhere.

[ewjoachim]

Does this answer your question ? If so, feel free to mark it as answered :)