How to get file name when loading data from S3

[jvyoralek]

I was using the SLING_STREAM_URL_COLUMN property to obtain the original filename for the row being processed from S3, but it appears the behavior has changed. Currently, I am receiving only the path stored in the _sling_stream_url column.

Example value in the target table (the target is a PostgreSQL database):

• before - s3://finance/test/file1.csv
• now - s3://finance/test/

I am unsure when this change occurred, as I might have missed a few versions. The current behavior is observed with Sling version 1.2.5.

Is there another way to store the filename in the target? It was useful in some cases for me.

[flarco]

Thanks, yes there was a process optimization done last release when ingesting folders, but the stream name lost granularity. Should be fixed next release.

…

On Sun, Apr 21, 2024, 11:40 AM Jiri Vyoralek ***@***.***> wrote: I was using the SLING_STREAM_URL_COLUMN property to obtain the original filename for the row being processed from *S3*, but it appears the behavior has changed. Currently, I am receiving only the path stored in the _sling_stream_url column. *Example value in the target table* (the target is a PostgreSQL database): - before - s3://finance/test/file1.csv - now - s3://finance/test/ I am unsure when this change occurred, as I might have missed a few versions. The current behavior is observed with Sling version 1.2.5. Is there another way to store the filename in the *target*? It was useful in some cases for me. — Reply to this email directly, view it on GitHub <#270>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB2QZYV6Z7B6AIUORLJG3KTY6PFWLAVCNFSM6AAAAABGRN4HI2VHI2DSMVQWIX3LMV43ERDJONRXK43TNFXW4OZWGUZTMNJQGY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>