-
Notifications
You must be signed in to change notification settings - Fork 1k
Description
Description
In the Parquet reader/writer, we may be using the _mr (the memory resource with which the output device memory should be allocated) to also allocate internal memory/vectors and (highly unlikely) vice versa with the cudf::get_current_device_resource_ref() for output memory.
Proposed fix
Create a new member variable called internal_mr or temp_mr initialized with cudf::get_current_device_resource_ref() or user provided temp_mr (as per proposed libcudf API design) and use that to allocate all internal memory (strictly no use of cudf::get_current_device_resource_ref() or not-providing an mr). xref #20780
Additional context
Hmm, since this memory is not for the output table, we should actually use cudf::get_current_device_resource_ref() for all these. That said, I think this is likely a parquet-wide issue that we need to fix (in a separate PR of course).
Originally posted by @mhaseeb123 in #21139 (comment)