Skip to content

Conversation

@redox
Copy link
Contributor

@redox redox commented Sep 22, 2025

This ensures we rely on the right CURLOPT_WRITEFUNCTION depending on whether we write to a string or to a file.

* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
  * frame #0: 0x000000010180bc04 libclang_rt.asan_osx_dynamic.dylib`__sanitizer_internal_memmove + 132
    frame #1: 0x000000010183da70 libclang_rt.asan_osx_dynamic.dylib`wrap_memcpy + 148
    frame #2: 0x0000000193c34208 libc++.1.dylib`std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>::append(char const*, unsigned long) + 144
    frame #3: 0x0000000122c24cac libduckdb.1.4.dylib`duckdb::netquack::WriteCallback(contents=0x0000629009808229, size=1, nmemb=1360, userp=0x0000000201ca69d8) at utils.cpp:87:37
    frame #4: 0x00000001ad832298 libcurl.4.dylib`cw_out_ptr_flush + 184
    frame #5: 0x00000001ad831f5c libcurl.4.dylib`cw_out_do_write + 168
    frame #6: 0x00000001ad831d64 libcurl.4.dylib`cw_out_write + 84
    frame #7: 0x00000001ad809ed8 libcurl.4.dylib`cw_download_write + 524
    frame #8: 0x00000001ad8195e4 libcurl.4.dylib`Curl_xfer_write_resp + 68
    frame #9: 0x00000001ad7f1004 libcurl.4.dylib`on_data_chunk_recv + 108
    frame #10: 0x00000001a215b800 libapple_nghttp2.dylib`nghttp2_session_mem_recv2 + 8316
    frame #11: 0x00000001ad7f1dc4 libcurl.4.dylib`h2_process_pending_input + 88
    frame #12: 0x00000001ad7f012c libcurl.4.dylib`h2_progress_ingress + 360
    frame #13: 0x00000001ad7eeffc libcurl.4.dylib`cf_h2_recv + 192
    frame #14: 0x00000001ad7cc3b0 libcurl.4.dylib`Curl_conn_recv + 48
    frame #15: 0x00000001ad8185c0 libcurl.4.dylib`Curl_readwrite + 448
    frame #16: 0x00000001ad8016bc libcurl.4.dylib`multi_runsingle + 1884
    frame #17: 0x00000001ad800eb4 libcurl.4.dylib`curl_multi_perform + 204
    frame #18: 0x00000001ad7db1b0 libcurl.4.dylib`curl_easy_perform + 272
    frame #19: 0x0000000122c0b338 libduckdb.1.4.dylib`duckdb::netquack::LoadTrancoList(db=0x000061b000004d98, force=true) at get_tranco.cpp:95:23

@redox redox requested a review from hatamiarash7 as a code owner September 22, 2025 08:11
@hatamiarash7
Copy link
Owner

In the code, since we already have:

curl_easy_setopt(curl, CURLOPT_WRITEDATA, file);

And we haven’t set a custom write function; libcurl is already using that default fwrite-based behavior.
So adding CURLOPT_WRITEFUNCTION, NULL is redundant, and it doesn’t change anything unless we want to make it explicit.


If you create this PR to solve the problem stated in this comment, I must say that it is not necessary. The problem here was that you had not updated the submodules to the latest version, v1.4.0, in your PR; as a result, the ExtensionLoader was not recognized (you can check here).

@redox
Copy link
Contributor Author

redox commented Sep 22, 2025

Ah great to know, I wasn't entirely sure! Might still be worth adding a update_tranco test

@redox redox closed this Sep 22, 2025
@redox
Copy link
Contributor Author

redox commented Sep 22, 2025

I'll reopen this with an up-to-date main target - but I still reproduce the segfault

@redox redox reopened this Sep 22, 2025
@redox redox closed this Sep 22, 2025
@redox redox reopened this Sep 22, 2025
@redox
Copy link
Contributor Author

redox commented Sep 22, 2025

OK, I think I understand now. I'll propose a nicer patch

Ensure the `CURLOPT_WRITEFUNCTION` knows how to write to `string` or
`File`.
@redox
Copy link
Contributor Author

redox commented Sep 22, 2025

@hatamiarash7 Let me know how this patch looks to you - not sure I get how it was working before 😅

@redox redox changed the title Fix Curl segfault Fix cURL's WRITEFUNCTION undefined behavior leading to segfault Sep 22, 2025
@hatamiarash7 hatamiarash7 merged commit 813c32c into hatamiarash7:main Sep 28, 2025
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants