[ABLD-267] Create bazel rules for curl #42546

aiuto · 2025-10-30T21:35:27Z

Bazel rules for curl

This is mostly generated with configure2bazel.
- The contents of overlay/lib_content.bzl is not designed to be human readable.
- Fix IDN configure flags and update overlay
- Add macos and linux arm overlays.
generates working curl library of the right size for macos

Verification on linux

Pull from a recent .deb file

$ ls -l $(find opt -name 'lib*3.5.4') $(find opt -name 'libcurl.so.4.8.0')
-rw-r--r--  1 tony.aiuto  staff  5804632 Jan 28 15:26 opt/datadog-agent/embedded/lib/libcrypto.so.3.5.4
-rwxr-xr-x  1 tony.aiuto  staff   744888 Jan 28 15:26 opt/datadog-agent/embedded/lib/libcurl.so.4.8.0
-rw-r--r--  1 tony.aiuto  staff  1085144 Jan 28 15:26 opt/datadog-agent/embedded/lib/libssl.so.3.5.4

vs.

$ bazel run -- @curl//:install --destdir=/tmp/opt/da
$ bazel run -- @openssl//:install --destdir=/tmp/opt/da/embedded
$ strip /tmp/opt/da/embedded/lib/lib*
$ ls -o $(find /tmp/opt/da -type f -name 'lib*')
-rw-r--r-- 1 root 5590024 Feb  3 17:46 /tmp/opt/da/embedded/lib/libcrypto.so.3.5.5
-rw-r--r-- 1 root  831416 Feb  3 17:46 /tmp/opt/da/embedded/lib/libcurl.so.4.8.0
-rw-r--r-- 1 root  998504 Feb  3 17:46 /tmp/opt/da/embedded/lib/libssl.so.3.5.5

Verification on macos

Look at /opt/datadog-agent on local laptop

$ ls -o /opt/datadog-agent/embedded/lib/libcrypto.3.5.4.dylib \
        /opt/datadog-agent/embedded/lib/libssl.3.5.4.dylib  \
        /opt/datadog-agent/embedded/lib/python3.13/site-packages/confluent_kafka/.dylibs/libcurl.4.dylib
-rw-r--r--  1 root  4847872 Jan 10 04:39 /opt/datadog-agent/embedded/lib/libcrypto.3.5.4.dylib
-rw-r--r--  1 root   965376 Jan 10 04:39 /opt/datadog-agent/embedded/lib/libssl.3.5.4.dylib
-rwxr-xr-x  1 root   731904 Jan 10 04:39 /opt/datadog-agent/embedded/lib/python3.13/site-packages/confluent_kafka/.dylibs/libcurl.4.dylib

vs.

$ bazel run -- @curl//:install --destdir=/tmp/opt/da
$ bazel run -- @openssl//:install --destdir=/tmp/opt/da/embedded
$ ls -o $(find /tmp/opt/da -type f -name 'lib*')
-rw-r--r--  1 tony.aiuto  4828368 Feb  3 12:35 /tmp/opt/da/embedded/lib/libcrypto.3.5.5.dylib
-rw-r--r--  1 tony.aiuto   720560 Feb  3 12:35 /tmp/opt/da/embedded/lib/libcurl.4.8.0.dylib
-rw-r--r--  1 tony.aiuto   981616 Feb  3 12:35 /tmp/opt/da/embedded/lib/libssl.3.5.5.dylib
-rw-r--r--  1 tony.aiuto      343 Feb  3 12:35 /tmp/opt/da/embedded/lib/pkgconfig/libcrypto.pc

Note the similar sizes. The drift in libssl is from other changes over the last month.

Looking at the linking

$ otool -L /opt/datadog-agent/embedded/lib/python3.13/site-packages/confluent_kafka/.dylibs/libcurl.4.dylib
/opt/datadog-agent/embedded/lib/python3.13/site-packages/confluent_kafka/.dylibs/libcurl.4.dylib:
        /DLC/confluent_kafka/.dylibs/libcurl.4.dylib (compatibility version 13.0.0, current version 13.0.0)
        @loader_path/libssl.3.dylib (compatibility version 3.0.0, current version 3.0.0)
        @loader_path/libcrypto.3.dylib (compatibility version 3.0.0, current version 3.0.0)
        @loader_path/libz.1.3.1.dylib (compatibility version 1.0.0, current version 1.3.1)
        /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation ...
        /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices ...
        /System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration ...
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1345.120.2)
$ otool -L bazel-bin/external/+_repo_rules+curl/libcurl.dylib
bazel-bin/external/+_repo_rules+curl/libcurl.dylib:
	@rpath/libcurl.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/nghttp2.so.14.25.1 (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libssl.dylib (compatibility version 3.0.0, current version 3.0.0)
	@rpath/libcrypto.dylib (compatibility version 3.0.0, current version 3.0.0)
	@rpath/libz.dylib (compatibility version 0.0.0, current version 0.0.0)
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 4201.0.0)
	/System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration (compatibility version 1.0.0, current version 1385.60.3)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1356.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 2000.67.0)

obviously, the paths need rewriting, but that is for when we are closer to install time
The extra CoreServices in the other version is unexpected, but may be a fault in the way Kafka builds it.

agent-platform-auto-pr · 2025-10-30T22:17:18Z

Static quality checks

✅ Please find below the results from static quality gates
Comparison made with ancestor 197c8df
📊 Static Quality Gates Dashboard

31 successful checks with minimal change (< 2 KiB)

	Quality gate	Current Size
✅	agent_deb_amd64	746.903 MiB
✅	agent_deb_amd64_fips	706.908 MiB
✅	agent_heroku_amd64	324.292 MiB
✅	agent_msi	659.182 MiB
✅	agent_rpm_amd64	746.886 MiB
✅	agent_rpm_amd64_fips	706.891 MiB
✅	agent_rpm_arm64	726.122 MiB
✅	agent_rpm_arm64_fips	688.561 MiB
✅	agent_suse_amd64	746.886 MiB
✅	agent_suse_amd64_fips	706.891 MiB
✅	agent_suse_arm64	726.122 MiB
✅	agent_suse_arm64_fips	688.561 MiB
✅	docker_agent_amd64	809.309 MiB
✅	docker_agent_arm64	813.092 MiB
✅	docker_agent_jmx_amd64	1000.220 MiB
✅	docker_agent_jmx_arm64	992.786 MiB
✅	docker_cluster_agent_amd64	180.927 MiB
✅	docker_cluster_agent_arm64	196.745 MiB
✅	docker_cws_instrumentation_amd64	7.135 MiB
✅	docker_cws_instrumentation_arm64	6.689 MiB
✅	docker_dogstatsd_amd64	38.449 MiB
✅	docker_dogstatsd_arm64	36.749 MiB
✅	dogstatsd_deb_amd64	29.669 MiB
✅	dogstatsd_deb_arm64	27.833 MiB
✅	dogstatsd_rpm_amd64	29.669 MiB
✅	dogstatsd_suse_amd64	29.669 MiB
✅	iot_agent_deb_amd64	42.782 MiB
✅	iot_agent_deb_arm64	39.899 MiB
✅	iot_agent_deb_armhf	40.470 MiB
✅	iot_agent_rpm_amd64	42.783 MiB
✅	iot_agent_suse_amd64	42.783 MiB

On-wire sizes (compressed)

	Quality gate	Change	Size (prev → curr → max)
✅	agent_deb_amd64	-39.31 KiB (0.02% reduction)	182.653 → 182.615 → 184.810
✅	agent_deb_amd64_fips	+14.91 KiB (0.01% increase)	174.047 → 174.062 → 177.560
✅	agent_heroku_amd64	-4.21 KiB (0.00% reduction)	87.088 → 87.084 → 88.450
✅	agent_msi	+28.0 KiB (0.02% increase)	142.605 → 142.633 → 143.300
✅	agent_rpm_amd64	+23.27 KiB (0.01% increase)	185.682 → 185.705 → 188.160
✅	agent_rpm_amd64_fips	-61.63 KiB (0.03% reduction)	176.462 → 176.402 → 178.900
✅	agent_rpm_arm64	+25.82 KiB (0.01% increase)	168.286 → 168.311 → 169.930
✅	agent_rpm_arm64_fips	-47.23 KiB (0.03% reduction)	160.585 → 160.539 → 163.120
✅	agent_suse_amd64	+23.27 KiB (0.01% increase)	185.682 → 185.705 → 188.160
✅	agent_suse_amd64_fips	-61.63 KiB (0.03% reduction)	176.462 → 176.402 → 178.900
✅	agent_suse_arm64	+25.82 KiB (0.01% increase)	168.286 → 168.311 → 169.930
✅	agent_suse_arm64_fips	-47.23 KiB (0.03% reduction)	160.585 → 160.539 → 163.120
✅	docker_agent_amd64	neutral	274.688 MiB → 277.400
✅	docker_agent_arm64	+6.27 KiB (0.00% increase)	262.235 → 262.241 → 266.040
✅	docker_agent_jmx_amd64	neutral	343.338 MiB → 346.020
✅	docker_agent_jmx_arm64	+2.82 KiB (0.00% increase)	326.866 → 326.869 → 330.660
✅	docker_cluster_agent_amd64	neutral	63.890 MiB → 64.510
✅	docker_cluster_agent_arm64	neutral	60.162 MiB → 61.170
✅	docker_cws_instrumentation_amd64	neutral	2.994 MiB → 3.330
✅	docker_cws_instrumentation_arm64	neutral	2.726 MiB → 3.090
✅	docker_dogstatsd_amd64	neutral	14.876 MiB → 15.820
✅	docker_dogstatsd_arm64	neutral	14.218 MiB → 14.830
✅	dogstatsd_deb_amd64	neutral	7.837 MiB → 8.790
✅	dogstatsd_deb_arm64	neutral	6.725 MiB → 7.710
✅	dogstatsd_rpm_amd64	neutral	7.850 MiB → 8.800
✅	dogstatsd_suse_amd64	neutral	7.850 MiB → 8.800
✅	iot_agent_deb_amd64	neutral	11.221 MiB → 12.040
✅	iot_agent_deb_arm64	neutral	9.591 MiB → 10.450
✅	iot_agent_deb_armhf	neutral	9.791 MiB → 10.620
✅	iot_agent_rpm_amd64	neutral	11.236 MiB → 12.060
✅	iot_agent_suse_amd64	neutral	11.236 MiB → 12.060

cit-pr-commenter · 2025-10-30T22:31:21Z

Regression Detector

Regression Detector Results

Metrics dashboard
Target profiles
Run ID: cd3dd45c-389a-4151-a116-26728daf2020

Baseline: 0134072
Comparison: 59ec14b
Diff

Optimization Goals: ✅ No significant changes detected

Experiments ignored for regressions

Regressions in experiments with settings containing erratic: true are ignored.

perf	experiment	goal	Δ mean %	Δ mean % CI	trials	links
➖	docker_containers_cpu	% cpu utilization	-0.71	[-3.78, +2.37]	1	Logs

Fine details of change detection per experiment

perf	experiment	goal	Δ mean %	Δ mean % CI	trials	links
➖	quality_gate_logs	% cpu utilization	+0.86	[-0.66, +2.39]	1	Logs bounds checks dashboard
➖	ddot_metrics_sum_cumulative	memory utilization	+0.59	[+0.43, +0.75]	1	Logs
➖	quality_gate_metrics_logs	memory utilization	+0.51	[+0.30, +0.72]	1	Logs bounds checks dashboard
➖	docker_containers_memory	memory utilization	+0.34	[+0.26, +0.41]	1	Logs
➖	ddot_metrics_sum_cumulativetodelta_exporter	memory utilization	+0.15	[-0.08, +0.39]	1	Logs
➖	quality_gate_idle	memory utilization	+0.15	[+0.10, +0.19]	1	Logs bounds checks dashboard
➖	tcp_syslog_to_blackhole	ingress throughput	+0.09	[-0.02, +0.20]	1	Logs
➖	file_to_blackhole_500ms_latency	egress throughput	+0.07	[-0.30, +0.44]	1	Logs
➖	tcp_dd_logs_filter_exclude	ingress throughput	+0.01	[-0.08, +0.11]	1	Logs
➖	file_to_blackhole_0ms_latency	egress throughput	+0.01	[-0.47, +0.49]	1	Logs
➖	file_to_blackhole_100ms_latency	egress throughput	+0.00	[-0.04, +0.05]	1	Logs
➖	uds_dogstatsd_to_api	ingress throughput	+0.00	[-0.12, +0.13]	1	Logs
➖	uds_dogstatsd_to_api_v3	ingress throughput	-0.01	[-0.14, +0.12]	1	Logs
➖	file_to_blackhole_1000ms_latency	egress throughput	-0.02	[-0.45, +0.41]	1	Logs
➖	quality_gate_idle_all_features	memory utilization	-0.04	[-0.07, +0.00]	1	Logs bounds checks dashboard
➖	file_tree	memory utilization	-0.04	[-0.09, +0.01]	1	Logs
➖	ddot_logs	memory utilization	-0.05	[-0.11, +0.01]	1	Logs
➖	ddot_metrics	memory utilization	-0.07	[-0.29, +0.15]	1	Logs
➖	otlp_ingest_metrics	memory utilization	-0.14	[-0.29, +0.00]	1	Logs
➖	otlp_ingest_logs	memory utilization	-0.16	[-0.25, -0.07]	1	Logs
➖	uds_dogstatsd_20mb_12k_contexts_20_senders	memory utilization	-0.61	[-0.66, -0.56]	1	Logs
➖	ddot_metrics_sum_delta	memory utilization	-0.65	[-0.84, -0.46]	1	Logs
➖	docker_containers_cpu	% cpu utilization	-0.71	[-3.78, +2.37]	1	Logs

Bounds Checks: ✅ Passed

perf	experiment	bounds_check_name	replicates_passed	links
✅	docker_containers_cpu	simple_check_run	10/10
✅	docker_containers_memory	memory_usage	10/10
✅	docker_containers_memory	simple_check_run	10/10
✅	file_to_blackhole_0ms_latency	lost_bytes	10/10
✅	file_to_blackhole_0ms_latency	memory_usage	10/10
✅	file_to_blackhole_1000ms_latency	lost_bytes	10/10
✅	file_to_blackhole_1000ms_latency	memory_usage	10/10
✅	file_to_blackhole_100ms_latency	lost_bytes	10/10
✅	file_to_blackhole_100ms_latency	memory_usage	10/10
✅	file_to_blackhole_500ms_latency	lost_bytes	10/10
✅	file_to_blackhole_500ms_latency	memory_usage	10/10
✅	quality_gate_idle	intake_connections	10/10	bounds checks dashboard
✅	quality_gate_idle	memory_usage	10/10	bounds checks dashboard
✅	quality_gate_idle_all_features	intake_connections	10/10	bounds checks dashboard
✅	quality_gate_idle_all_features	memory_usage	10/10	bounds checks dashboard
✅	quality_gate_logs	intake_connections	10/10	bounds checks dashboard
✅	quality_gate_logs	lost_bytes	10/10	bounds checks dashboard
✅	quality_gate_logs	memory_usage	10/10	bounds checks dashboard
✅	quality_gate_metrics_logs	cpu_usage	10/10	bounds checks dashboard
✅	quality_gate_metrics_logs	intake_connections	10/10	bounds checks dashboard
✅	quality_gate_metrics_logs	lost_bytes	10/10	bounds checks dashboard
✅	quality_gate_metrics_logs	memory_usage	10/10	bounds checks dashboard

Explanation

Confidence level: 90.00%
Effect size tolerance: |Δ mean %| ≥ 5.00%

Performance changes are noted in the perf column of each table:

✅ = significantly better comparison variant performance
❌ = significantly worse comparison variant performance
➖ = no significant change in performance

A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".

For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:

Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.
Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that if our statistical model is accurate, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.
Its configuration does not mark it "erratic".

CI Pass/Fail Decision

✅ Passed. All Quality Gates passed.

quality_gate_metrics_logs, bounds check lost_bytes: 10/10 replicas passed. Gate passed.
quality_gate_metrics_logs, bounds check cpu_usage: 10/10 replicas passed. Gate passed.
quality_gate_metrics_logs, bounds check memory_usage: 10/10 replicas passed. Gate passed.
quality_gate_metrics_logs, bounds check intake_connections: 10/10 replicas passed. Gate passed.
quality_gate_idle, bounds check memory_usage: 10/10 replicas passed. Gate passed.
quality_gate_idle, bounds check intake_connections: 10/10 replicas passed. Gate passed.
quality_gate_idle_all_features, bounds check intake_connections: 10/10 replicas passed. Gate passed.
quality_gate_idle_all_features, bounds check memory_usage: 10/10 replicas passed. Gate passed.
quality_gate_logs, bounds check memory_usage: 10/10 replicas passed. Gate passed.
quality_gate_logs, bounds check lost_bytes: 10/10 replicas passed. Gate passed.
quality_gate_logs, bounds check intake_connections: 10/10 replicas passed. Gate passed.

…zma (#42545) It turns out the things that depend on openssl can not build with the :lzma usege. It needs more investigation, but I think the culprit is using the output_shared_library feature of rules-foreign-cc. We probably should just build a regulary library and then add a cc_shared_library on top of that. Doing this now, even without complete understanding, allows us to add curl on top of openssl, and then test a real dependency stack to a binary. Needed by #42546 Co-authored-by: tony.aiuto <tony.aiuto@datadoghq.com>

… binary dynamically against it. - Use case in the curl PR. #42546 After building curl, we see this. ``` - ldd bazel-bin/external/+_repo_rules+curl/curl_bin linux-vdso.so.1 (0x0000ffffa8c25000) libnghttp2.so.14 => /opt/datadog-agent/embedded/lib/libnghttp2.so.14 (0x0000ffff90240000) libz.so => /opt/datadog-agent/embedded/lib/libz.so (0x0000ffff90210000) libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000ffff901e0000) libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000ffff90130000) libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000ffff90100000) librt.so.1 => /lib/aarch64-linux-gnu/librt.so.1 (0x0000ffff900d0000) libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000ffff8ff10000) /lib/ld-linux-aarch64.so.1 (0x0000ffff9028e000) ```

… binary dynamically against it. (#43778) Allow nghttp2 to export a shared library so that we can link the curl binary dynamically against it. - Use case in the curl PR. #42546 After building curl, we see this. ``` $ ldd bazel-bin/external/+_repo_rules+curl/curl_bin linux-vdso.so.1 (0x0000ffffa8c25000) libnghttp2.so.14 => /opt/datadog-agent/embedded/lib/libnghttp2.so.14 (0x0000ffff90240000) libz.so => /opt/datadog-agent/embedded/lib/libz.so (0x0000ffff90210000) libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000ffff901e0000) libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000ffff90130000) libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000ffff90100000) librt.so.1 => /lib/aarch64-linux-gnu/librt.so.1 (0x0000ffff900d0000) libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000ffff8ff10000) /lib/ld-linux-aarch64.so.1 (0x0000ffff9028e000) ``` Co-authored-by: tony.aiuto <tony.aiuto@datadoghq.com>

deps/curl/Makefile

deps/curl/config_opts.txt

# This is the 1st commit message: Start to bazel rules for curl - mostly generated with configure2bazel - Fix IDN configure flags and update overlay - Add macos and linux arm overlays. - generates working curl library of the right size for macos cc toolchain is injectined a link arg that doesn't work. ``` /root/.cache/bazel/_bazel_root/81a15fa9a2846e82038a778136785275/external/gcc_toolchain++gcc_toolchains+gcc_toolchain_aarch64/bin/aarch64-unknown-linux-gnu-ld.bfd: unrecognized option '--start-lib' ``` There is some real ugly stuff going on to try to use the output of configure_make as shared libarries. - copying the ssl headers to a new place so we can make a cc_library This is needed because the `includes` attribute on cc_import doesn't work. We need to create a real cc_library to depend on to get that right. - the cc_library / cc_shared_library pair for everything Again, cc_import doesn't seem to work Look at /opt/datadog-agent on local laptop ``` -rwxr-xr-x 1 root wheel 4815040 Dec 9 14:57 /opt/datadog-agent/embedded/lib/libcrypto.3.dylib -rwxr-xr-x 1 root wheel 4857760 Dec 9 14:57 /opt/datadog-agent/embedded/lib/python3.13/site-packages/confluent_kafka/.dylibs/libcrypto.3.dylib -rwxr-xr-x 1 root wheel 4857760 Dec 9 14:57 /opt/datadog-agent/embedded/lib/python3.13/site-packages/psycopg_c/.dylibs/libcrypto.3.dylib -rwxr-xr-x 1 root wheel 965408 Dec 9 14:57 /opt/datadog-agent/embedded/lib/libssl.3.dylib -rwxr-xr-x 1 root wheel 946896 Dec 9 14:57 /opt/datadog-agent/embedded/lib/python3.13/site-packages/confluent_kafka/.dylibs/libssl.3.dylib -rwxr-xr-x 1 root wheel 946896 Dec 9 14:57 /opt/datadog-agent/embedded/lib/python3.13/site-packages/psycopg_c/.dylibs/libssl.3.dylib -rwxr-xr-x 1 root wheel 731904 Dec 9 14:57 /opt/datadog-agent/embedded/lib/python3.13/site-packages/confluent_kafka/.dylibs/libcurl.4.dylib ``` ``` ls -l bazel-bin/external/+_repo_rules+curl/libcurl.dylib bazel-bin/external/+_repo_rules+openssl/libimported_*dylib -r-xr-xr-x 1 tony.aiuto wheel 803968 Jan 13 23:02 bazel-bin/external/+_repo_rules+curl/libcurl.dylib -r-xr-xr-x 1 tony.aiuto wheel 4839472 Jan 13 16:54 bazel-bin/external/+_repo_rules+openssl/libimported_crypto_shared.dylib -r-xr-xr-x 1 tony.aiuto wheel 949008 Jan 13 22:06 bazel-bin/external/+_repo_rules+openssl/libimported_ssl_shared.dylib Note the similar sizes. Drift is probably because of nghttp2. Looking at the linking ``` $ otool -L /opt/datadog-agent/embedded/lib/python3.13/site-packages/confluent_kafka/.dylibs/libcurl.4.dylib /opt/datadog-agent/embedded/lib/python3.13/site-packages/confluent_kafka/.dylibs/libcurl.4.dylib: /DLC/confluent_kafka/.dylibs/libcurl.4.dylib (compatibility version 13.0.0, current version 13.0.0) @loader_path/libssl.3.dylib (compatibility version 3.0.0, current version 3.0.0) @loader_path/libcrypto.3.dylib (compatibility version 3.0.0, current version 3.0.0) @loader_path/libz.1.3.1.dylib (compatibility version 1.0.0, current version 1.3.1) /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation ... /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices ... /System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration ... /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1345.120.2) $ otool -L bazel-bin/external/+_repo_rules+curl/libcurl.dylib bazel-bin/external/+_repo_rules+curl/libcurl.dylib: @rpath/libcurl.dylib (compatibility version 0.0.0, current version 0.0.0) @rpath/nghttp2.so.14 (compatibility version 0.0.0, current version 0.0.0) @rpath/libimported_crypto_shared.dylib (compatibility version 0.0.0, current version 0.0.0) @rpath/libimported_ssl_shared.dylib (compatibility version 0.0.0, current version 0.0.0) @rpath/libz.dylib (compatibility version 0.0.0, current version 0.0.0) /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation ... /System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration ... /usr/lib/libc++.1.dylib ... /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1356.0.0) ``` The extra CoreServices in the other version is unexpected, but may be a fault in the way Kafka builds it. checkpoint curl got it back to linking no time for jokes maybe readme readme tmp finally # This is the commit message #2: delete some apis we do not need # This is the commit message #3: comments # This is the commit message #4: comments

- mostly generated with configure2bazel - Fix IDN configure flags and update overlay - Add macos and linux arm overlays. - generates working curl library of the right size for macos cc toolchain is injectined a link arg that doesn't work. ``` /root/.cache/bazel/_bazel_root/81a15fa9a2846e82038a778136785275/external/gcc_toolchain++gcc_toolchains+gcc_toolchain_aarch64/bin/aarch64-unknown-linux-gnu-ld.bfd: unrecognized option '--start-lib' ``` There is some real ugly stuff going on to try to use the output of configure_make as shared libarries. - copying the ssl headers to a new place so we can make a cc_library This is needed because the `includes` attribute on cc_import doesn't work. We need to create a real cc_library to depend on to get that right. - the cc_library / cc_shared_library pair for everything Again, cc_import doesn't seem to work Look at /opt/datadog-agent on local laptop ``` -rwxr-xr-x 1 root wheel 4815040 Dec 9 14:57 /opt/datadog-agent/embedded/lib/libcrypto.3.dylib -rwxr-xr-x 1 root wheel 4857760 Dec 9 14:57 /opt/datadog-agent/embedded/lib/python3.13/site-packages/confluent_kafka/.dylibs/libcrypto.3.dylib -rwxr-xr-x 1 root wheel 4857760 Dec 9 14:57 /opt/datadog-agent/embedded/lib/python3.13/site-packages/psycopg_c/.dylibs/libcrypto.3.dylib -rwxr-xr-x 1 root wheel 965408 Dec 9 14:57 /opt/datadog-agent/embedded/lib/libssl.3.dylib -rwxr-xr-x 1 root wheel 946896 Dec 9 14:57 /opt/datadog-agent/embedded/lib/python3.13/site-packages/confluent_kafka/.dylibs/libssl.3.dylib -rwxr-xr-x 1 root wheel 946896 Dec 9 14:57 /opt/datadog-agent/embedded/lib/python3.13/site-packages/psycopg_c/.dylibs/libssl.3.dylib -rwxr-xr-x 1 root wheel 731904 Dec 9 14:57 /opt/datadog-agent/embedded/lib/python3.13/site-packages/confluent_kafka/.dylibs/libcurl.4.dylib ``` ``` ls -l bazel-bin/external/+_repo_rules+curl/libcurl.dylib bazel-bin/external/+_repo_rules+openssl/libimported_*dylib -r-xr-xr-x 1 tony.aiuto wheel 803968 Jan 13 23:02 bazel-bin/external/+_repo_rules+curl/libcurl.dylib -r-xr-xr-x 1 tony.aiuto wheel 4839472 Jan 13 16:54 bazel-bin/external/+_repo_rules+openssl/libimported_crypto_shared.dylib -r-xr-xr-x 1 tony.aiuto wheel 949008 Jan 13 22:06 bazel-bin/external/+_repo_rules+openssl/libimported_ssl_shared.dylib Note the similar sizes. Drift is probably because of nghttp2. Looking at the linking ``` $ otool -L /opt/datadog-agent/embedded/lib/python3.13/site-packages/confluent_kafka/.dylibs/libcurl.4.dylib /opt/datadog-agent/embedded/lib/python3.13/site-packages/confluent_kafka/.dylibs/libcurl.4.dylib: /DLC/confluent_kafka/.dylibs/libcurl.4.dylib (compatibility version 13.0.0, current version 13.0.0) @loader_path/libssl.3.dylib (compatibility version 3.0.0, current version 3.0.0) @loader_path/libcrypto.3.dylib (compatibility version 3.0.0, current version 3.0.0) @loader_path/libz.1.3.1.dylib (compatibility version 1.0.0, current version 1.3.1) /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation ... /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices ... /System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration ... /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1345.120.2) $ otool -L bazel-bin/external/+_repo_rules+curl/libcurl.dylib bazel-bin/external/+_repo_rules+curl/libcurl.dylib: @rpath/libcurl.dylib (compatibility version 0.0.0, current version 0.0.0) @rpath/nghttp2.so.14 (compatibility version 0.0.0, current version 0.0.0) @rpath/libimported_crypto_shared.dylib (compatibility version 0.0.0, current version 0.0.0) @rpath/libimported_ssl_shared.dylib (compatibility version 0.0.0, current version 0.0.0) @rpath/libz.dylib (compatibility version 0.0.0, current version 0.0.0) /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation ... /System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration ... /usr/lib/libc++.1.dylib ... /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1356.0.0) ``` The extra CoreServices in the other version is unexpected, but may be a fault in the way Kafka builds it. checkpoint curl got it back to linking no time for jokes maybe readme readme tmp finally delete some apis we do not need comments comments allow haxx.se in adms config log the generated curl_config.h split curl / curlu - does not help

github-actions bot added long review PR is complex, plan time to review it team/agent-build labels Oct 30, 2025

aiuto changed the title ~~Create bazel rules for curl~~ [ABLD-158] Create bazel rules for curl Oct 30, 2025

aiuto mentioned this pull request Oct 30, 2025

[ABLD-158] Switch @openssl// to depend on @xz//:liblzma instead of :lzma #42545

Merged

aiuto added changelog/no-changelog qa/no-code-change No code change in Agent code requiring validation labels Oct 31, 2025

aiuto force-pushed the aiuto/curl branch from cf05f24 to 34b431c Compare December 3, 2025 04:41

aiuto mentioned this pull request Dec 3, 2025

Allow nghttp2 to export a shared library so that we can link the curl binary dynamically against it. #43778

Merged

aiuto force-pushed the aiuto/curl branch 2 times, most recently from 9d26984 to dce83a7 Compare January 14, 2026 04:04

aiuto marked this pull request as ready for review January 14, 2026 04:13

aiuto requested a review from a team as a code owner January 14, 2026 04:13

aiuto force-pushed the aiuto/curl branch from dce83a7 to 3f7aa9e Compare January 14, 2026 04:17

aiuto added the ask-review Ask required teams to review this PR label Jan 14, 2026

rdesgroppes reviewed Jan 14, 2026

View reviewed changes

deps/curl/Makefile Outdated Show resolved Hide resolved

rdesgroppes reviewed Jan 14, 2026

View reviewed changes

deps/curl/config_opts.txt Show resolved Hide resolved

aiuto requested a review from rdesgroppes January 14, 2026 15:38

aiuto force-pushed the aiuto/curl branch from 8186c28 to f7e2573 Compare January 31, 2026 05:13

aiuto marked this pull request as draft February 3, 2026 03:19

aiuto changed the title ~~[ABLD-158] Create bazel rules for curl~~ [ABLD-267] Create bazel rules for curl Feb 3, 2026

aiuto force-pushed the aiuto/curl branch from 72c80da to 12cc469 Compare February 3, 2026 17:02

aiuto added 2 commits February 10, 2026 16:26

aiuto force-pushed the aiuto/curl branch from 765e2d3 to 59ec14b Compare February 10, 2026 21:30

dd-octo-sts bot added the internal Identify a non-fork PR label Feb 10, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ABLD-267] Create bazel rules for curl #42546

[ABLD-267] Create bazel rules for curl #42546

aiuto commented Oct 30, 2025 •

edited

Loading

Uh oh!

agent-platform-auto-pr bot commented Oct 30, 2025 •

edited

Loading

Uh oh!

cit-pr-commenter bot commented Oct 30, 2025 •

edited by cit-pr-commenter-54b7da bot

Loading

Experiments ignored for regressions

Fine details of change detection per experiment

Bounds Checks: ✅ Passed

Explanation

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

[ABLD-267] Create bazel rules for curl #42546

Are you sure you want to change the base?

[ABLD-267] Create bazel rules for curl #42546

Conversation

aiuto commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Verification on linux

Verification on macos

Uh oh!

agent-platform-auto-pr bot commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Static quality checks

Uh oh!

cit-pr-commenter bot commented Oct 30, 2025 • edited by cit-pr-commenter-54b7da bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Regression Detector

Regression Detector Results

Optimization Goals: ✅ No significant changes detected

Experiments ignored for regressions

Fine details of change detection per experiment

Bounds Checks: ✅ Passed

Explanation

CI Pass/Fail Decision

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

aiuto commented Oct 30, 2025 •

edited

Loading

agent-platform-auto-pr bot commented Oct 30, 2025 •

edited

Loading

cit-pr-commenter bot commented Oct 30, 2025 •

edited by cit-pr-commenter-54b7da bot

Loading