Skip to content

BUG: fix memory leak in JSON datetime serialization #62217

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
+11 −1
Open

BUG: fix memory leak in JSON datetime serialization #62217

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
791bb41
fix: memory leak in JSON datetime serialization
Alvaro-Kothe Aug 29, 2025
f7b2f5b
fix: flag to free `cStr` in `Object_getBigNumStringValue`
Alvaro-Kothe Aug 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion pandas/_libs/src/vendored/ujson/python/objToJSON.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,7 @@ typedef struct __TypeContext {
JSINT64 longValue;

const char *cStr;
int freeCStr;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally the problem with the JSON code is that over time we have incrementally added many different state management operations, and it is not always clear how they should work in tandem. So I am a little hesitant to add more state to solve the current issue. Is it possible to solve the issue without changing this struct?

Copy link
Contributor

@swt2c swt2c Sep 2, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any suggestions on how to do that? Revert the changes in fb6c4e3 that make some of the uses of cStr stack allocated? It doesn't seem possible to make all uses of cStr stack allocated, at least at first glance.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure that might be a reasonable approach

NpyArrContext *npyarr;
PdBlockContext *pdblock;
int transpose;
Expand Down Expand Up @@ -162,6 +163,7 @@ static TypeContext *createTypeContext(void) {
pc->longValue = 0;
pc->doubleValue = 0.0;
pc->cStr = NULL;
pc->freeCStr = 0;
pc->npyarr = NULL;
pc->pdblock = NULL;
pc->rowLabels = NULL;
Expand Down Expand Up @@ -327,13 +329,15 @@ static const char *NpyDateTimeToIsoCallback(JSOBJ Py_UNUSED(unused),
NPY_DATETIMEUNIT base = ((PyObjectEncoder *)tc->encoder)->datetimeUnit;
NPY_DATETIMEUNIT valueUnit = ((PyObjectEncoder *)tc->encoder)->valueUnit;
GET_TC(tc)->cStr = int64ToIso(GET_TC(tc)->longValue, valueUnit, base, len);
GET_TC(tc)->freeCStr = 1;
return GET_TC(tc)->cStr;
}

/* JSON callback. returns a char* and mutates the pointer to *len */
static const char *NpyTimeDeltaToIsoCallback(JSOBJ Py_UNUSED(unused),
JSONTypeContext *tc, size_t *len) {
GET_TC(tc)->cStr = int64ToIsoDuration(GET_TC(tc)->longValue, len);
GET_TC(tc)->freeCStr = 1;
return GET_TC(tc)->cStr;
}

Expand All @@ -347,7 +351,9 @@ static const char *PyDateTimeToIsoCallback(JSOBJ obj, JSONTypeContext *tc,
}

NPY_DATETIMEUNIT base = ((PyObjectEncoder *)tc->encoder)->datetimeUnit;
return PyDateTimeToIso(obj, base, len);
GET_TC(tc)->cStr = PyDateTimeToIso(obj, base, len);
GET_TC(tc)->freeCStr = 1;
return GET_TC(tc)->cStr;
}

static const char *PyTimeToJSON(JSOBJ _obj, JSONTypeContext *tc,
Expand Down Expand Up @@ -1880,6 +1886,9 @@ static void Object_endTypeContext(JSOBJ Py_UNUSED(obj), JSONTypeContext *tc) {
GET_TC(tc)->rowLabels = NULL;
NpyArr_freeLabels(GET_TC(tc)->columnLabels, GET_TC(tc)->columnLabelsLen);
GET_TC(tc)->columnLabels = NULL;
if (GET_TC(tc)->freeCStr) {
PyObject_Free((void *)GET_TC(tc)->cStr);
}
GET_TC(tc)->cStr = NULL;
PyObject_Free(tc->prv);
tc->prv = NULL;
Expand Down
Loading