-
Notifications
You must be signed in to change notification settings - Fork 653
api: Versioned namespace to preserve ABI compatibility between minor releases #4869
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
api: Versioned namespace to preserve ABI compatibility between minor releases #4869
Conversation
2c3e7c9 to
32d11c9
Compare
|
I have revised and replaced the contents of this PR and its description. When I first posted, it tentatively prototyped the scheme for just libOpenImageIO_Util. The new revision implements the full scheme for the entire OIIO codebase, and it works! The code in this PR is in the 3.2 branch, but is 100% ABI and link compatible with OIIO 3.1. |
|
As this PR stands,
There is not currently (in this PR) a way for application source to lock down which API/ABI you want, for example to peg to 3.1 even if you are using the 3.3 headers. But I believe this should be straightforward, and after this PR is merged, I will try that and submit a subsequent patch for consideration. I suspect that such a thing would look like: |
jfpanisset
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I imagine you've looked for this, but are there any frameworks for managing API versioning that would reduce / eliminate the need to hard code specific API versions (here 3.1) in the code and manage that automatically? Or perhaps the overhead of such a framework is not worth it.
Not having to rebuild / relink apps for compatible upgrades in OIIO is a big win.
|
I haven't looked for it, it never occurred to me that such a thing would be readily available. Did you have a specific framework in mind, or do you even know that it exists? Where I'd like to end up is to have such a painless and risk-free upgrade path that we only need to support the current release version and never backport or need to support older releases. |
|
I don't know of any such framework, I was wishfully thinking that it feels like this is not a unique problem for any C++ project that has to maintain API compatibility, so somewhere someone must have scratched this itch in a general way? But of course even if such a framework does exist, that doesn't mean it can be (easily) adopted by an existing project. It does seem like a documented approach to API (and ABI?) compatibility that could be adhered to across ASWF projects would be a win. |
|
I believe OpenVDB also maintains several versions of ABI that are selectable, but I negligently have not actually looked at their code to see exactly how they do it. I've also seen guides explaining how glibc does it, for example. It's pretty cool but I feel like the symbol aliasing is both tricky and not likely to be portable across all compilers and platforms. |
|
It will, honestly, be at least a year before we really know the results of this experiment. I think I can backport these changes to the 3.1 tree compatibly, but it's not until 3.2 is released that we'll see if it really works in practice to allow 100% ABI compatible upgrades. (I wish I'd gotten the 2-level namespacing prerequisite into OIIO in time for 3.0, then this 3.1 release could be the real-world test for compatibility with 3.0.) And maybe it's not until 3.3 or 3.4, when we've had a couple releases to accumulate a growing number of ABI divergences, that we'll know if it turns into a big ball of mud or not. |
|
Reminder: This is 2 weeks old now (ok, 1 week since I expanded its scope a lot, but still the same general idea as the original take). I don't want to let it fester forever. Let's say, give it until the end of next week at the latest, and then I will merge if nobody has reason to caution me away from it or suggest a different approach? Though I would prefer to merge after people give it a positive approval than just default into merging because nobody ever says yes or no. This is a change that will have real strategic consequences for how we organize the code to manage version compatibility. |
|
Reminder: I intend to merge this on Friday night if no objections are raised by then. |
This patch implements the new scheme that we hope will preserve ABI compatibility across minor (yearly) releases. Brief summary: * We will strive to have annual minor releases (e.g., 3.1.x -> 3.2.x) be ABI/link backwards-compatible, as strongly as monthly patch releases (3.1.5 -> 3.1.6) always have been. Every-several-years major releases (3.x -> 4.x) are still a potential full compatibility break. * Consumers of OpenImageIO don't need to change their code, and can just continue to refer to `OIIO::Foo` or `OIIO::Bar()` as always. * Internally, all items in the public OIIO APIs will be in a versioned inner namespace, and will continue to live in the versioned namespace where they were first introduced, forever. If they need to change in an ABI-breaking way, they will be *duplicated* in the newer namespaces, so the old version continues to be available in the old namespace. * The new header nsversions.h contains a long and detailed comment explaining how all the new declarations work. --- Detailed explanation: "Major" or "first digit" releases, which only happen once every several years for us, are full ABI+API breaks. Previously, "minor" or "second digit" releases, which are annual, have fully incompatible ABIs, with separate namespaces to enforce it. Here, we are striving to allow our annual/minor releases to no longer break ABI, to make upgrading easier for downstream users. The basics are that once a symbol is introduced, it will forever live in the namespace of that release. Subsequent release years will have their own namespaces, but will either alias or duplicate the original symbols. This is enabled by our recent (in the lead-up to 3.1) introduction of a split 2-part namespacing scheme. So 3.2 is the first release that has the potential to perserve compatibility with 3.1 in this way. In this PR, I have done this for the whole codebase. It works! -- as measured by being able to have our automated ABI checker show 100% back compatibility with 3.1. So currently, even though this is the nascent 3.2 tree, everything in the public APIs live in the 3.1 ABI namespace. And so they will remain, except for changes that cannot be done without breaking ABI back-compatibility, which will then end up with multiple versions in different versioned namespaces. Let's look this over and decide if we like it. It does involve our developers to do some hoop jumping and take extra care as we develop, to keep things associated with the right namespaces. Users of OpenImageIO shouldn't need to change anything on their end, nor to be aware of this at all. Signed-off-by: Larry Gritz <[email protected]>
32d11c9 to
c59a39e
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This pull request implements a versioned namespace scheme to preserve ABI compatibility between minor releases. The purpose is to allow annual minor releases (e.g., 3.1.x -> 3.2.x) to maintain ABI/link backwards-compatibility while continuing to provide the same public API to consumers.
Key changes:
- Introduction of versioned inner namespaces (OIIO_NAMESPACE_3_1_BEGIN/END) for all public API items
- Addition of compatibility aliases in the main OIIO namespace to maintain existing API usage patterns
- Updates to CI configuration to test ABI compatibility with the reference version
Reviewed Changes
Copilot reviewed 114 out of 114 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/include/OpenImageIO/*.h | Updated all public headers to use versioned namespaces with compatibility aliases |
| src/doc/Doxyfile | Added OIIO_NAMESPACE_3_2_* macros for documentation generation |
| .github/workflows/ci.yml | Updated ABI check reference commit to match the new versioned implementation |
| .github/workflows/build-steps.yml | Enhanced artifact upload to include ABI compatibility reports |
| src/build-scripts/ci-abicheck.bash | Improved ABI check script to always save compatibility reports |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
…releases (AcademySoftwareFoundation#4869) This patch implements the new scheme that we hope will preserve ABI compatibility across minor (yearly) releases. Brief summary: * We will strive to have annual minor releases (e.g., 3.1.x -> 3.2.x) be ABI/link backwards-compatible, as strongly as monthly patch releases (3.1.5 -> 3.1.6) always have been. Every-several-years major releases (3.x -> 4.x) are still a potential full compatibility break. * Consumers of OpenImageIO don't need to change their code, and can just continue to refer to `OIIO::Foo` or `OIIO::Bar()` as always. * Internally, all items in the public OIIO APIs will be in a versioned inner namespace, and will continue to live in the versioned namespace where they were first introduced, forever. (Well, until the next first-digit-changing major release, at which point we will clean up the old cruft and version everything up.) If anything needs to change in an ABI-breaking way, it will be *duplicated* in the newer namespaces, so the old version continues to be available in the old namespace. * The new header nsversions.h contains a long and detailed comment explaining how all the new declarations work. --- Detailed explanation: "Major" or "first digit" releases, which only happen once every several years for us, are full ABI+API breaks. Previously, "minor" or "second digit" releases, which are annual, have fully incompatible ABIs, with separate namespaces to enforce it. Here, we are striving to allow our annual/minor releases to no longer break ABI, to make upgrading easier for downstream users. The basics are that once a symbol is introduced, it will forever live in the namespace of that release. Subsequent release years will have their own namespaces, but will either alias or duplicate the original symbols. This is enabled by our recent (in the lead-up to 3.1) introduction of a split 2-part namespacing scheme. So 3.2 is the first release that has the potential to perserve compatibility with 3.1 in this way. In this PR, I have done this for the whole codebase. It works! -- as measured by being able to have our automated ABI checker show 100% back compatibility with 3.1. So currently, even though this is the nascent 3.2 tree, everything in the public APIs live in the 3.1 ABI namespace. And so they will remain, except for changes that cannot be done without breaking ABI back-compatibility, which will then end up with multiple versions in different versioned namespaces. Let's look this over and decide if we like it. It does involve our developers to do some hoop jumping and take extra care as we develop, to keep things associated with the right namespaces. Users of OpenImageIO shouldn't need to change anything on their end, nor to be aware of this at all. Signed-off-by: Larry Gritz <[email protected]>
…releases (AcademySoftwareFoundation#4869) This is a backported version of PR AcademySoftwareFoundation#4869 to minimize the textual differences between 3.1 and 3.2 branches. In 3.1, the main OIIO_NAMESPACE and the OIIO_3_1_NAMESPACE are the same. --- This patch implements the new scheme that we hope will preserve ABI compatibility across minor (yearly) releases. Brief summary: * We will strive to have annual minor releases (e.g., 3.1.x -> 3.2.x) be ABI/link backwards-compatible, as strongly as monthly patch releases (3.1.5 -> 3.1.6) always have been. Every-several-years major releases (3.x -> 4.x) are still a potential full compatibility break. * Consumers of OpenImageIO don't need to change their code, and can just continue to refer to `OIIO::Foo` or `OIIO::Bar()` as always. * Internally, all items in the public OIIO APIs will be in a versioned inner namespace, and will continue to live in the versioned namespace where they were first introduced, forever. (Well, until the next first-digit-changing major release, at which point we will clean up the old cruft and version everything up.) If anything needs to change in an ABI-breaking way, it will be *duplicated* in the newer namespaces, so the old version continues to be available in the old namespace. * The new header nsversions.h contains a long and detailed comment explaining how all the new declarations work. --- Detailed explanation: "Major" or "first digit" releases, which only happen once every several years for us, are full ABI+API breaks. Previously, "minor" or "second digit" releases, which are annual, have fully incompatible ABIs, with separate namespaces to enforce it. Here, we are striving to allow our annual/minor releases to no longer break ABI, to make upgrading easier for downstream users. The basics are that once a symbol is introduced, it will forever live in the namespace of that release. Subsequent release years will have their own namespaces, but will either alias or duplicate the original symbols. This is enabled by our recent (in the lead-up to 3.1) introduction of a split 2-part namespacing scheme. So 3.2 is the first release that has the potential to perserve compatibility with 3.1 in this way. In this PR, I have done this for the whole codebase. It works! -- as measured by being able to have our automated ABI checker show 100% back compatibility with 3.1. So currently, even though this is the nascent 3.2 tree, everything in the public APIs live in the 3.1 ABI namespace. And so they will remain, except for changes that cannot be done without breaking ABI back-compatibility, which will then end up with multiple versions in different versioned namespaces. Let's look this over and decide if we like it. It does involve our developers to do some hoop jumping and take extra care as we develop, to keep things associated with the right namespaces. Users of OpenImageIO shouldn't need to change anything on their end, nor to be aware of this at all. Signed-off-by: Larry Gritz <[email protected]>
…releases (AcademySoftwareFoundation#4869) This is a backported version of PR AcademySoftwareFoundation#4869 to minimize the textual differences between 3.1 and 3.2 branches. In 3.1, the main OIIO_NAMESPACE and the OIIO_3_1_NAMESPACE are the same. --- This patch implements the new scheme that we hope will preserve ABI compatibility across minor (yearly) releases. Brief summary: * We will strive to have annual minor releases (e.g., 3.1.x -> 3.2.x) be ABI/link backwards-compatible, as strongly as monthly patch releases (3.1.5 -> 3.1.6) always have been. Every-several-years major releases (3.x -> 4.x) are still a potential full compatibility break. * Consumers of OpenImageIO don't need to change their code, and can just continue to refer to `OIIO::Foo` or `OIIO::Bar()` as always. * Internally, all items in the public OIIO APIs will be in a versioned inner namespace, and will continue to live in the versioned namespace where they were first introduced, forever. (Well, until the next first-digit-changing major release, at which point we will clean up the old cruft and version everything up.) If anything needs to change in an ABI-breaking way, it will be *duplicated* in the newer namespaces, so the old version continues to be available in the old namespace. * The new header nsversions.h contains a long and detailed comment explaining how all the new declarations work. --- Detailed explanation: "Major" or "first digit" releases, which only happen once every several years for us, are full ABI+API breaks. Previously, "minor" or "second digit" releases, which are annual, have fully incompatible ABIs, with separate namespaces to enforce it. Here, we are striving to allow our annual/minor releases to no longer break ABI, to make upgrading easier for downstream users. The basics are that once a symbol is introduced, it will forever live in the namespace of that release. Subsequent release years will have their own namespaces, but will either alias or duplicate the original symbols. This is enabled by our recent (in the lead-up to 3.1) introduction of a split 2-part namespacing scheme. So 3.2 is the first release that has the potential to perserve compatibility with 3.1 in this way. In this PR, I have done this for the whole codebase. It works! -- as measured by being able to have our automated ABI checker show 100% back compatibility with 3.1. So currently, even though this is the nascent 3.2 tree, everything in the public APIs live in the 3.1 ABI namespace. And so they will remain, except for changes that cannot be done without breaking ABI back-compatibility, which will then end up with multiple versions in different versioned namespaces. Let's look this over and decide if we like it. It does involve our developers to do some hoop jumping and take extra care as we develop, to keep things associated with the right namespaces. Users of OpenImageIO shouldn't need to change anything on their end, nor to be aware of this at all. Signed-off-by: Larry Gritz <[email protected]>
…releases (AcademySoftwareFoundation#4869) This patch implements the new scheme that we hope will preserve ABI compatibility across minor (yearly) releases. Brief summary: * We will strive to have annual minor releases (e.g., 3.1.x -> 3.2.x) be ABI/link backwards-compatible, as strongly as monthly patch releases (3.1.5 -> 3.1.6) always have been. Every-several-years major releases (3.x -> 4.x) are still a potential full compatibility break. * Consumers of OpenImageIO don't need to change their code, and can just continue to refer to `OIIO::Foo` or `OIIO::Bar()` as always. * Internally, all items in the public OIIO APIs will be in a versioned inner namespace, and will continue to live in the versioned namespace where they were first introduced, forever. (Well, until the next first-digit-changing major release, at which point we will clean up the old cruft and version everything up.) If anything needs to change in an ABI-breaking way, it will be *duplicated* in the newer namespaces, so the old version continues to be available in the old namespace. * The new header nsversions.h contains a long and detailed comment explaining how all the new declarations work. --- Detailed explanation: "Major" or "first digit" releases, which only happen once every several years for us, are full ABI+API breaks. Previously, "minor" or "second digit" releases, which are annual, have fully incompatible ABIs, with separate namespaces to enforce it. Here, we are striving to allow our annual/minor releases to no longer break ABI, to make upgrading easier for downstream users. The basics are that once a symbol is introduced, it will forever live in the namespace of that release. Subsequent release years will have their own namespaces, but will either alias or duplicate the original symbols. This is enabled by our recent (in the lead-up to 3.1) introduction of a split 2-part namespacing scheme. So 3.2 is the first release that has the potential to perserve compatibility with 3.1 in this way. In this PR, I have done this for the whole codebase. It works! -- as measured by being able to have our automated ABI checker show 100% back compatibility with 3.1. So currently, even though this is the nascent 3.2 tree, everything in the public APIs live in the 3.1 ABI namespace. And so they will remain, except for changes that cannot be done without breaking ABI back-compatibility, which will then end up with multiple versions in different versioned namespaces. Let's look this over and decide if we like it. It does involve our developers to do some hoop jumping and take extra care as we develop, to keep things associated with the right namespaces. Users of OpenImageIO shouldn't need to change anything on their end, nor to be aware of this at all. Signed-off-by: Larry Gritz <[email protected]> Signed-off-by: Zach Lewis <[email protected]>
This patch implements the new scheme that we hope will preserve ABI
compatibility across minor (yearly) releases.
Brief summary:
We will strive to have annual minor releases (e.g., 3.1.x -> 3.2.x)
be ABI/link backwards-compatible, as strongly as monthly patch
releases (3.1.5 -> 3.1.6) always have been. Every-several-years
major releases (3.x -> 4.x) are still a potential full compatibility
break.
Consumers of OpenImageIO don't need to change their code, and can
just continue to refer to
OIIO::FooorOIIO::Bar()as always.Internally, all items in the public OIIO APIs will be in a versioned
inner namespace, and will continue to live in the versioned
namespace where they were first introduced, forever. (Well, until the
next first-digit-changing major release, at which point we will clean
up the old cruft and version everything up.) If anything needs to
change in an ABI-breaking way, it will be duplicated in the newer
namespaces, so the old version continues to be available in the old
namespace.
The new header nsversions.h contains a long and detailed comment
explaining how all the new declarations work.
Detailed explanation:
"Major" or "first digit" releases, which only happen once every
several years for us, are full ABI+API breaks. Previously, "minor" or
"second digit" releases, which are annual, have fully incompatible
ABIs, with separate namespaces to enforce it. Here, we are striving to
allow our annual/minor releases to no longer break ABI, to make upgrading
easier for downstream users.
The basics are that once a symbol is introduced, it will forever live
in the namespace of that release. Subsequent release years will have
their own namespaces, but will either alias or duplicate the original
symbols. This is enabled by our recent (in the lead-up to 3.1)
introduction of a split 2-part namespacing scheme. So 3.2 is the first
release that has the potential to perserve compatibility with 3.1 in
this way.
In this PR, I have done this for the whole codebase. It works! -- as
measured by being able to have our automated ABI checker show 100%
back compatibility with 3.1. So currently, even though this is the
nascent 3.2 tree, everything in the public APIs live in the 3.1 ABI
namespace. And so they will remain, except for changes that cannot be
done without breaking ABI back-compatibility, which will then end up
with multiple versions in different versioned namespaces.
Let's look this over and decide if we like it. It does involve our
developers to do some hoop jumping and take extra care as we develop,
to keep things associated with the right namespaces.
Users of OpenImageIO shouldn't need to change anything on their end,
nor to be aware of this at all.