Please add number version in SONAMEs.

[santiagorr]

Dear OpenTelemetry Team,

While packaging opentelemetry-cpp for Debian, I noticed the SDK's SONAME lacks a numerical version. While I understand the library has been carefully designed and considering ABI stability, IMO, it would be useful to add a numerical version in case you need to introduce breaking changes in the future. Bumping the SONAME would help to have smooth library transitions.

Could you please consider this?

Than you

[marcalff]

Thanks for the report.

Agreed, opentelemetry-cpp needs to have this.

In more details:

• OPENTELEMETRY_ABI_VERSION_NO 1 --> SONAME 1
• OPENTELEMETRY_ABI_VERSION_NO 2 --> SONAME 2

[sjinks]

In more details:

Unfortunately, it could be more complicated than that 😞

Whenever we break the ABI, the SOVERSION has to change.

version-info is current:revision:age
SOVERSION is (current-age), VERSION is (current-age).revision.age

• Always increase the revision value.
• Increase the current value whenever an interface has been added, removed or changed.
• Increase the age value only if the changes made to the ABI are backward compatible.

More formal rules are available here.

Say, we had current=1, revision=0, age=0 in the very beginning. The SOVERSION is 1.
Now, #3147 breaks the ABI: current=2, revision=0, age=0, SOVERSION becomes 2. This makes sense—if the ABI is broken, it is unsafe for consumers to use the new version of the library.

But since the code breaks the ABI of libopentelemetry-logs only, the SOVERSION for other libraries remains the same.

[sjinks]

A test script to check for ABI changes between different releases (adapt as needed to add more tags or to build various exporters):

#!/bin/sh

mkdir -p dumps
first=v1.10.0
tags="v1.11.0 v1.12.0 v1.13.0 v1.14.0 v1.14.1 v1.14.2 v1.15.0 v1.16.0 v1.16.1 v1.17.0"
for tag in ${first} ${tags}; do
	rm -rf build
	git checkout "${tag}"
	cmake -B build -DBUILD_TESTING=OFF -DWITH_BENCHMARK=OFF -DCMAKE_CXX_FLAGS="-Og -g" -DBUILD_SHARED_LIBS=ON -DWITH_EXAMPLES=OFF
	cmake --build build -j
	cmake --install build --prefix "dist/${tag}"
	for i in "dist/${tag}/lib/"*.so; do abi-dumper "${i}" -o "dumps/${tag}-$(basename "${i}").dump" -lver "${tag}"; done
	git reset --hard "${tag}"
done

prev="${first}"
for tag in ${tags}; do
	for lib in "dist/${prev}/lib/"*.so; do
		name="$(basename "${lib}" .so)"
		base="$(basename "${lib}")"
		abi-compliance-checker -l "${name}" -old "dumps/${prev}-${base}.dump" -new dumps/${tag}-${base}.dump --extended
	done
	prev="${tag}"
done

[sjinks]

The ABI has changed several times:

libopentelemetry_metrics

Binary compatibility: 99.8%
Source compatibility: 99.96%
Total binary compatibility problems: 10, warnings: 4
Total source compatibility problems: 8, warnings: 0
Report: compat_reports/libopentelemetry_metrics/v1.10.0_to_v1.11.0/compat_report.html

Binary compatibility: 99.8%
Source compatibility: 99.8%
Total binary compatibility problems: 3, warnings: 14
Total source compatibility problems: 10, warnings: 1
Report: compat_reports/libopentelemetry_metrics/v1.11.0_to_v1.12.0/compat_report.html

Binary compatibility: 99.5%
Source compatibility: 99.7%
Total binary compatibility problems: 17, warnings: 2
Total source compatibility problems: 16, warnings: 0
Report: compat_reports/libopentelemetry_metrics/v1.12.0_to_v1.13.0/compat_report.html

Binary compatibility: 98.9%
Source compatibility: 99.6%
Total binary compatibility problems: 42, warnings: 1
Total source compatibility problems: 31, warnings: 1
Report: compat_reports/libopentelemetry_metrics/v1.13.0_to_v1.14.0/compat_report.html

Binary compatibility: 99.9%
Source compatibility: 100%
Total binary compatibility problems: 2, warnings: 0
Total source compatibility problems: 0, warnings: 0
Report: compat_reports/libopentelemetry_metrics/v1.14.1_to_v1.14.2/compat_report.html

Binary compatibility: 97.2%
Source compatibility: 98.2%
Total binary compatibility problems: 83, warnings: 4
Total source compatibility problems: 82, warnings: 0
Report: compat_reports/libopentelemetry_metrics/v1.15.0_to_v1.16.0/compat_report.html

Binary compatibility: 99.3%
Source compatibility: 99.8%
Total binary compatibility problems: 40, warnings: 0
Total source compatibility problems: 36, warnings: 0
Report: compat_reports/libopentelemetry_metrics/v1.16.0_to_v1.16.1/compat_report.html

Binary compatibility: 99.7%
Source compatibility: 99.9%
Total binary compatibility problems: 5, warnings: 4
Total source compatibility problems: 4, warnings: 7
Report: compat_reports/libopentelemetry_metrics/v1.16.1_to_v1.17.0/compat_report.html

libopentelemetry_exporter_in_memory

Binary compatibility: 97.7%
Source compatibility: 98.7%
Total binary compatibility problems: 1, warnings: 0
Total source compatibility problems: 1, warnings: 0
Report: compat_reports/libopentelemetry_exporter_in_memory/v1.13.0_to_v1.14.0/compat_report.html

libopentelemetry_exporter_ostream_span

Binary compatibility: 97.9%
Source compatibility: 98.8%
Total binary compatibility problems: 1, warnings: 0
Total source compatibility problems: 1, warnings: 0
Report: compat_reports/libopentelemetry_exporter_ostream_span/v1.13.0_to_v1.14.0/compat_report.html

Binary compatibility: 99.6%
Source compatibility: 100%
Total binary compatibility problems: 4, warnings: 1
Total source compatibility problems: 0, warnings: 1
Report: compat_reports/libopentelemetry_exporter_ostream_span/v1.16.1_to_v1.17.0/compat_report.html

libopentelemetry_trace

Binary compatibility: 98%
Source compatibility: 99.1%
Total binary compatibility problems: 4, warnings: 2
Total source compatibility problems: 1, warnings: 0
Report: compat_reports/libopentelemetry_trace/v1.13.0_to_v1.14.0/compat_report.html

Binary compatibility: 99.5%
Source compatibility: 99.6%
Total binary compatibility problems: 14, warnings: 3
Total source compatibility problems: 4, warnings: 3
Report: compat_reports/libopentelemetry_trace/v1.15.0_to_v1.16.0/compat_report.html

Binary compatibility: 99.3%
Source compatibility: 99.8%
Total binary compatibility problems: 18, warnings: 0
Total source compatibility problems: 18, warnings: 0
Report: compat_reports/libopentelemetry_trace/v1.16.0_to_v1.16.1/compat_report.html

Binary compatibility: 99.6%
Source compatibility: 100%
Total binary compatibility problems: 7, warnings: 10
Total source compatibility problems: 0, warnings: 10
Report: compat_reports/libopentelemetry_trace/v1.16.1_to_v1.17.0/compat_report.html

libopentelemetry_common

Binary compatibility: 99.1%
Source compatibility: 100%
Total binary compatibility problems: 5, warnings: 0
Total source compatibility problems: 0, warnings: 0
Report: compat_reports/libopentelemetry_common/v1.14.2_to_v1.15.0/compat_report.html

Binary compatibility: 99%
Source compatibility: 100%
Total binary compatibility problems: 4, warnings: 1
Total source compatibility problems: 0, warnings: 1
Report: compat_reports/libopentelemetry_common/v1.16.1_to_v1.17.0/compat_report.html

libopentelemetry_exporter_ostream_metrics

Binary compatibility: 98.3%
Source compatibility: 100%
Total binary compatibility problems: 1, warnings: 2
Total source compatibility problems: 0, warnings: 0
Report: compat_reports/libopentelemetry_exporter_ostream_metrics/v1.15.0_to_v1.16.0/compat_report.html

Binary compatibility: 99.2%
Source compatibility: 100%
Total binary compatibility problems: 6, warnings: 1
Total source compatibility problems: 0, warnings: 1
Report: compat_reports/libopentelemetry_exporter_ostream_metrics/v1.16.1_to_v1.17.0/compat_report.html

libopentelemetry_logs

Binary compatibility: 99.5%
Source compatibility: 99.6%
Total binary compatibility problems: 14, warnings: 3
Total source compatibility problems: 4, warnings: 3
Report: compat_reports/libopentelemetry_logs/v1.15.0_to_v1.16.0/compat_report.html

Binary compatibility: 99.5%
Source compatibility: 99.9%
Total binary compatibility problems: 12, warnings: 0
Total source compatibility problems: 12, warnings: 0
Report: compat_reports/libopentelemetry_logs/v1.16.0_to_v1.16.1/compat_report.html

libopentelemetry_exporter_ostream_logs

Binary compatibility: 99%
Source compatibility: 100%
Total binary compatibility problems: 4, warnings: 1
Total source compatibility problems: 0, warnings: 1
Report: compat_reports/libopentelemetry_exporter_ostream_logs/v1.16.1_to_v1.17.0/compat_report.html

libopentelemetry_resources

Binary compatibility: 98.4%
Source compatibility: 100%
Total binary compatibility problems: 8, warnings: 1
Total source compatibility problems: 0, warnings: 1
Report: compat_reports/libopentelemetry_resources/v1.16.1_to_v1.17.0/compat_report.html

compat_reports.zip

[marcalff]

@sjinks Thanks for commenting on this.

To clarify a few expectations, see details below.

Typical library

In a typical library, there are:

• public API headers
• a library (binary)

and the term ABI compatibility refers to compatibility of:

• header files version X
• library version Y

so an application compiled against header X can link (dynamically) with version Y.

ABI compatibility in opentelemetry-cpp

Opentelemetry-cpp consist of:

• public API header files (located under api/include)
• public SDK header files (located under sdk/include)
• libraries for the components of the SDK (traces, metrics, logs, resource, etc)
• public exporter header files (for example, OTLP is under exporter/otlp/include)
• libraries for exporters

Typical application code using opentelemetry-cpp falls into several categories:

• Instrumented code
• Application main() function
• Custom exporter
• Custom SDK

Expectations are as follows:

Instrumented code

Instrumented code is only allowed to use the public API headers, and nothing else.

Some libraries, say libA, libB, libC, can be compiled against opentelemetry-cpp API headers versions 1.12, 1.15, 1.17 respectively, and are all expected to work properly when linked inside an application that uses version 1.18 for example.

This is what we call abi compatibility for short: ABI compatibility of the API.

Application main() function

In an application that deploys opentelemetry-cpp, the main function initializes the SDK and add exporters, thereby initializing the global singleton provider objects defined in the API, to point to an actual (not noop) implementation.

To do this, the main() needs to use header files from the SDK and exporters.

Here we do NOT provide ABI compatibility, so a change in SDK or exporter public headers is acceptable.

Custom exporter

Someone may want to implement an exporter for a custom protocol, or to a different backend.

To do this, the exporter implementation needs to use header files from the SDK, hopefully only using SDK header files considered public.

Here we do NOT provide ABI compatibility, so a change in SDK or exporter public headers is acceptable.

Custom SDK

Someone may want to extend or change the SDK implementation itself.

To do this, the custom sdk implementation needs to use the full header files from the SDK, including headers considered private.

Here we do provide compatibility of any kind.

---

It is correct to say the ABI has changed several times.

To be more clear, the SDK ABI has changed several times, but this is not an issue.

What we aim to maintain is the API ABI.

In other words:

• once instrumented (aka, only using the API), an application is expected to ship a binary package (i.e., the application library itself) that will work with any opentelemetry-cpp version.
• an application that deploys opentelemetry-cpp in the main() function is expected to compile and link with the same opentelemetry-cpp version (aka, link statically).

Dynamic linking was added for windows, but is experimental and unstable, given that there is no stable SDK or exporter api.

---

I tried to implement some tooling, but never got time to finish it.

See repository:

• https://github.com/marcalff/opentelemetry-cpp-abi

[marcalff]

Orthogonal discussion:

Currently we ship different libraries for traces, metrics, logs, common libraries, etc for the SDK.

These libraries have inter dependencies (error handler), and cross signal dependencies (metrics depends on traces, because of examplars).

A preliminary refactoring before using SONAME is to ship only one library for the entire SDK, to reconcile this.

[sjinks]

once instrumented (aka, only using the API), an application is expected to ship a binary package (i.e., the application library itself) that will work with any opentelemetry-cpp version.

Because there is no libopentelemetry-api.so, it is hard to verify :-(

For example, between v1.16.1 and v1.17.0, the size of absl::otel_v1::variant_internal::VariantStateBase<bool, int> has increased from 24 to 48 bytes. As far as I can tell, this is the internal part of nostd::variant and is the part of the API. However, because API is the header-only library (or so it seems on Linux), a nostd::variant initialized in, say, v1.16.0 will have a different layout when passed to v1.17.0. Which may (or may not) result in crashes or interesting bugs.

Also, abseil says:

the ABI is subject to change without notice

Thus,

the parts of the API that depend on abseil may break :-(

Dynamic linking was added for windows, but is experimental and unstable, given that there is no stable SDK or exporter api.

Linux distros start to adopt OpenTelemetry, which is great. However, they prefer to ship dynamic libraries, not static ones. Which means that the applications using OpenTelemetry will use dynamic linking.

To be more clear, the SDK ABI has changed several times, but this is not an issue.

My point was that

OPENTELEMETRY_ABI_VERSION_NO 1 --> SONAME 1
OPENTELEMETRY_ABI_VERSION_NO 2 --> SONAME 2

won't probably work :-( Every library will have to use its own SONAME, and even for OPENTELEMETRY_ABI_VERSION_NO=1, there will be more than one SONAME.

For example, libopentelemetry_metrics will be smth like libopentelemetry_metrics.so.8, and libopentelemetry_common will be smth like libopentelemetry_common.so.2.

[marcalff]

My point was that

OPENTELEMETRY_ABI_VERSION_NO 1 --> SONAME 1
OPENTELEMETRY_ABI_VERSION_NO 2 --> SONAME 2

won't probably work :-( Every library will have to use its own SONAME, and even for OPENTELEMETRY_ABI_VERSION_NO=1, there will be more than one SONAME.

Agreed, this is a good discussion.

So, to summarize, there are 2 ABI compatibility issues that are important to address:

• compatibility of user code with the API only, currently covered by OPENTELEMETRY_ABI_VERSION_NO
  • this is assuming it really is stable, which is at risk (see ABSEIL)
• compatibility of user code (in main()) with the SDK and exporters, currently not covered at all.
  • Some people will use shared libraries for the SDK anyway, whether documented as stable or not, so opentelemetry-cpp should better be prepared to support it.

[marcalff]

For example, between v1.16.1 and v1.17.0, the size of absl::otel_v1::variant_internal::VariantStateBase<bool, int> has increased from 24 to 48 bytes.

@sjinks Do you have more details on this ?

I am looking at the git history, but can not find when/why this happened.

As far as I can tell, this is the internal part of nostd::variant and is the part of the API. However, because API is the header-only library (or so it seems on Linux), a nostd::variant initialized in, say, v1.16.0 will have a different layout when passed to v1.17.0. Which may (or may not) result in crashes or interesting bugs.

Yes, nostd::variant is part of the API, so this is indeed an issue.

[sjinks]

Do you have more details on this ?

I used this script:

#!/bin/sh

mkdir -p dumps
first="v1.16.1"
tags="v1.17.0"
for tag in ${first} ${tags}; do
        rm -rf build
        git checkout "${tag}"
        cmake -B build -DBUILD_TESTING=OFF -DWITH_BENCHMARK=OFF -DCMAKE_CXX_FLAGS="-Og -g" -DBUILD_SHARED_LIBS=ON -DWITH_EXAMPLES=OFF
        cmake --build build -j
        cmake --install build --prefix "dist/${tag}"
        for i in "dist/${tag}/lib/"*.so; do abi-dumper "${i}" -o "dumps/${tag}-$(basename "${i}").dump" -lver "${tag}"; done
        git reset --hard "${tag}"
done

prev="${first}"
for tag in ${tags}; do
        for lib in "dist/${prev}/lib/"*.so; do
                name="$(basename "${lib}" .so)"
                base="$(basename "${lib}")"
                abi-compliance-checker -l "${name}" -old "dumps/${prev}-${base}.dump" -new dumps/${tag}-${base}.dump --extended
        done
        prev="${tag}"
done

and then checked the report for libopentelemetry_common:

[marcalff]

Thanks for the scripts.

I tried to reproduce locally, and got this:

The following command:

abi-compliance-checker -l libopentelemetry_common -old dumps/v1.16.1-libopentelemetry_common.so.dump -new dumps/v1.17.0-libopentelemetry_common.so.dump --extended

allocated up to 110 Gb of memory (I have a big machine), and is then killed.

When trying without the --extended flag, the report is generated immediately, and says 100 percent compatible.

I am using:

Version #1	v1.16.1
Version #2	v1.17.0
Arch	x86_64
GCC Version	11.5.0