Skip to content

Commit 7135199

Browse files
GH-47009: [C#] ExportedAllocationOwner should use 64-bit integer to track total allocated memory. (#47011)
### Rationale for this change Fixes #47009 ### What changes are included in this PR? `ExportedAllocationOwner` now uses a 64-bit (instead of 32-bit) variable to track total allocated memory. ### Are these changes tested? yes ### Are there any user-facing changes? **This PR contains a "Critical Fix".** Previously, it wasn't possible to export Record Batches larger than 2GB, it resulted in overflowing the integer variable that was used to track allocated memory. * GitHub Issue: #47009 Authored-by: Marcin Krystianc <[email protected]> Signed-off-by: Curt Hagenlocher <[email protected]>
1 parent 73472a9 commit 7135199

File tree

2 files changed

+30
-1
lines changed

2 files changed

+30
-1
lines changed

csharp/src/Apache.Arrow/Memory/ExportedAllocationOwner.cs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ namespace Apache.Arrow.Memory
2424
internal sealed class ExportedAllocationOwner : INativeAllocationOwner, IDisposable
2525
{
2626
private readonly List<IntPtr> _pointers = new List<IntPtr>();
27-
private int _allocationSize;
27+
private long _allocationSize;
2828
private long _referenceCount;
2929
private bool _disposed;
3030

csharp/test/Apache.Arrow.Tests/CDataInterfaceDataTests.cs

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -110,5 +110,34 @@ public unsafe void RoundTripInt32ArrayWithOffset()
110110
}
111111
CArrowArray.Free(cArray);
112112
}
113+
114+
[Fact]
115+
public unsafe void ExportRecordBatch_LargerThan2GB_Succeeds()
116+
{
117+
RecordBatch GetTestRecordBatch()
118+
{
119+
const int rows = 50_000;
120+
var doubles = new double[rows];
121+
for (var i = 0; i < rows; ++i)
122+
{
123+
doubles[i] = i * 1.1;
124+
}
125+
126+
var batchBuilder = new RecordBatch.Builder();
127+
for (var i = 0; i < 10_000; i++)
128+
{
129+
batchBuilder.Append($"doubles{i}", true, ab => ab.Double(b => b.Append(doubles)));
130+
}
131+
132+
return batchBuilder.Build();
133+
}
134+
135+
RecordBatch batch = GetTestRecordBatch();
136+
137+
CArrowArray* cArray = CArrowArray.Create();
138+
CArrowArrayExporter.ExportRecordBatch(batch, cArray);
139+
140+
CArrowArray.Free(cArray);
141+
}
113142
}
114143
}

0 commit comments

Comments
 (0)