Stack Overflow in libplist node_estimate_size Function Due to Circular References

[LkkkLxy]

Summary

A stack overflow vulnerability exists in the node_estimate_size function in src/xplist.c:457 when processing plist data structures containing circular references. The function enters infinite recursion, leading to stack exhaustion and application crash.

Affected Version

This issue affects the current version of libplist where serialization functions (plist_to_xml, plist_to_json, plist_to_openstep) rely on the node_estimate_size function for size estimation.

Root Cause

The node_estimate_size function assumes plist structures are non-circular trees and lacks cycle detection. When two or more plist nodes reference each other (creating circular references), the recursive traversal never terminates, causing infinite recursion and stack overflow.

Reproduction Steps

Minimal Test Case

#include <plist/plist.h>
#include <stdlib.h>

int main() {
    plist_t dict1 = plist_new_dict();
    plist_t dict2 = plist_new_dict();
    
    // Create circular references
    plist_dict_set_item(dict1, "reference", dict2);
    plist_dict_set_item(dict2, "reference", dict1);
    
    // Trigger stack overflow
    char* xml_out = NULL;
    uint32_t xml_len = 0;
    plist_to_xml(dict1, &xml_out, &xml_len);  // This will cause infinite recursion
    
    return 0;
}

Build and Run

cd /src
clang test_circular.c -o test_circular -fsanitize=address \
  -I/src/libplist-install/include \
  /src/libplist-install/lib/libplist-2.0.a \
  /src/libplist/libcnary/.libs/libcnary.a -lpthread
./test_circular

Expected vs Actual Behavior

Expected: The serialization function should either detect the circular reference and handle it gracefully (e.g., return an error or use a reference mechanism) or limit recursion depth.

Actual: The application crashes with a stack overflow due to infinite recursion in node_estimate_size.

Stack Trace

ERROR: AddressSanitizer: stack-overflow on address 0x7ffcef107ff8
SCARINESS: 10 (stack-overflow)
    #0 0x55bd144502e4 in node_estimate_size /src/libplist/src/xplist.c:453:12
    #1 0x55bd1445032d in node_estimate_size /src/libplist/src/xplist.c:457:13
    #2 0x55bd1445032d in node_estimate_size /src/libplist/src/xplist.c:457:13
    ... (repeats over 120+ times)

Discussion: A Fuzzing Worker's Perspective on Circular Reference Issues

This bug was discovered through fuzzing. As a fuzzing researcher, I noticed a similar issue previously reported in the cJSON library (see: DaveGamble/cJSON#880). Since libplist shares certain similarities with cJSON, this case may be relevant as a reference. In cJSON, the developers introduced a depth check (DaveGamble/cJSON#888) to mitigate part of the problem. However, I do not consider this a fundamentally effective solution.

Have you been aware of this issue? Do you have any suggestions for better solutions? I would be glad to discuss this with you.