What is the upper limit of parquet file sizes? #30488
alberttwong
started this conversation in
General
Replies: 3 comments
-
|
the file I used was nyc yellow taxi https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
it worked with the smaller nyc green taxi dataset |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
related to #27169 |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Connector Name
source-file
Connector Version
0.3.4
What step the error happened?
Other
Revelant information
I got an out of memory error for a parquet file that 47 megs. However the 11 megs file didn't give me an OOM error. https://docs.airbyte.com/integrations/sources/file/
Relevant log output
No response
Contribute
Beta Was this translation helpful? Give feedback.
All reactions