|
| 1 | ++++ |
| 2 | +title = "News Announcements for Version 23.x" |
| 3 | +linkTitle = "Version 23.x" |
| 4 | +toc_hide = "true" |
| 5 | +description = "Changes announced for Protocol Buffers version 23.x." |
| 6 | +type = "docs" |
| 7 | ++++ |
| 8 | + |
| 9 | +The following announcements are specific to Version 23.x. For information |
| 10 | +presented chronologically, see [News](/news). |
| 11 | + |
| 12 | +## Changes to Ruby Generator {#ruby} |
| 13 | + |
| 14 | +[This GitHub PR](https://github.com/protocolbuffers/protobuf/pull/12319), which |
| 15 | +will appear in the 23.x release, changes the Ruby code generator to emit a |
| 16 | +serialized proto instead of the DSL. |
| 17 | + |
| 18 | +It removes the DSL from the code generator in anticipation of splitting the DSL |
| 19 | +out into a separate package. |
| 20 | + |
| 21 | +Given a .proto file like: |
| 22 | + |
| 23 | +```proto |
| 24 | +syntax = "proto3"; |
| 25 | +
|
| 26 | +package pkg; |
| 27 | +
|
| 28 | +message TestMessage { |
| 29 | + optional int32 i32 = 1; |
| 30 | + optional TestMessage msg = 2; |
| 31 | +} |
| 32 | +``` |
| 33 | + |
| 34 | +Generated code before: |
| 35 | + |
| 36 | +```ruby |
| 37 | +# Generated by the protocol buffer compiler. DO NOT EDIT! |
| 38 | +# source: protoc_explorer/main.proto |
| 39 | + |
| 40 | +require 'google/protobuf' |
| 41 | + |
| 42 | +Google::Protobuf::DescriptorPool.generated_pool.build do |
| 43 | + add_file("test.proto", :syntax => :proto3) do |
| 44 | + add_message "pkg.TestMessage" do |
| 45 | + proto3_optional :i32, :int32, 1 |
| 46 | + proto3_optional :msg, :message, 2, "pkg.TestMessage" |
| 47 | + end |
| 48 | + end |
| 49 | +end |
| 50 | + |
| 51 | +module Pkg |
| 52 | + TestMessage = ::Google::Protobuf::DescriptorPool.generated_pool.lookup("pkg.TestMessage").msgclass |
| 53 | +end |
| 54 | +``` |
| 55 | + |
| 56 | +Generated code after: |
| 57 | + |
| 58 | +```ruby |
| 59 | +# frozen_string_literal: true |
| 60 | +# Generated by the protocol buffer compiler. DO NOT EDIT! |
| 61 | +# source: test.proto |
| 62 | + |
| 63 | +require 'google/protobuf' |
| 64 | + |
| 65 | +descriptor_data = "\n\ntest.proto\x12\x03pkg\"S\n\x0bTestMessage\x12\x10\n\x03i32\x18\x01 \x01(\x05H\x00\x88\x01\x01\x12\"\n\x03msg\x18\x02 \x01(\x0b\x32\x10.pkg.TestMessageH\x01\x88\x01\x01\x42\x06\n\x04_i32B\x06\n\x04_msgb\x06proto3" |
| 66 | +begin |
| 67 | + Google::Protobuf::DescriptorPool.generated_pool.add_serialized_file(descriptor_data) |
| 68 | +rescue TypeError => e |
| 69 | + # <compatibility code, see below> |
| 70 | +end |
| 71 | + |
| 72 | +module Pkg |
| 73 | + TestMessage = ::Google::Protobuf::DescriptorPool.generated_pool.lookup("pkg.TestMessage").msgclass |
| 74 | +end |
| 75 | +``` |
| 76 | + |
| 77 | +This change fixes nearly all remaining conformance problems that existed |
| 78 | +previously. This is a side effect of moving from the DSL (which is lossy) to a |
| 79 | +serialized descriptor (which preserves all information). |
| 80 | + |
| 81 | +### Backward Compatibility {#backward} |
| 82 | + |
| 83 | +This change should be 100% compatible with Ruby Protobuf >= 3.18.0, released in |
| 84 | +Sept 2021. Additionally, it should be compatible with all existing users and |
| 85 | +deployments. |
| 86 | + |
| 87 | +There **is** some special compatibility code inserted to achieve this level of |
| 88 | +backward compatibility that you should be aware of. Without the compatibility |
| 89 | +code, there is an edge case that could break backward compatibility. The |
| 90 | +previous code is lax in a way that the new code will be more strict. |
| 91 | + |
| 92 | +When using a full serialized descriptor, it contains a list of all `.proto` |
| 93 | +files imported by this file (whereas the DSL never added dependencies properly). |
| 94 | +See the code in |
| 95 | +[`descriptor.proto`](https://github.com/protocolbuffers/protobuf/blob/dfb71558a2226718dc3bcf5df27cbc11c1f72382/src/google/protobuf/descriptor.proto#L65-L66). |
| 96 | + |
| 97 | +`add_serialized_file` verifies that all dependencies listed in the descriptor |
| 98 | +were previously added with `add_serialized_file`. Generally that should be fine, |
| 99 | +because the generated code will contain Ruby `require` statements for all |
| 100 | +dependencies, and the descriptor will fail to load anyway if the types depended |
| 101 | +on were not previously defined in the `DescriptorPool`. |
| 102 | + |
| 103 | +But there is a potential for problems if there are ambiguities around file |
| 104 | +paths. For example, consider the following scenario: |
| 105 | + |
| 106 | +```proto |
| 107 | +// foo/bar.proto |
| 108 | +
|
| 109 | +syntax = "proto2"; |
| 110 | +
|
| 111 | +message Bar {} |
| 112 | +``` |
| 113 | + |
| 114 | +```proto |
| 115 | +// foo/baz.proto |
| 116 | +
|
| 117 | +syntax = "proto2"; |
| 118 | +
|
| 119 | +import "bar.proto"; |
| 120 | +
|
| 121 | +message Baz { |
| 122 | + optional Bar bar = 1; |
| 123 | +} |
| 124 | +``` |
| 125 | + |
| 126 | +If you invoke `protoc` like so, it will work correctly: |
| 127 | + |
| 128 | +``` |
| 129 | +$ protoc --ruby_out=. -Ifoo foo/bar.proto foo/baz.proto |
| 130 | +$ RUBYLIB=. ruby baz_pb.rb |
| 131 | +``` |
| 132 | + |
| 133 | +However if you invoke `protoc` like so, and didn't have any compatibility code, |
| 134 | +it would fail to load: |
| 135 | + |
| 136 | +``` |
| 137 | +$ protoc --ruby_out=. -I. -Ifoo foo/baz.proto |
| 138 | +$ protoc --ruby_out=. -I. -Ifoo foo/bar.proto |
| 139 | +$ RUBYLIB=foo ruby foo/baz_pb.rb |
| 140 | +foo/baz_pb.rb:10:in `add_serialized_file': Unable to build file to DescriptorPool: Depends on file 'bar.proto', but it has not been loaded (Google::Protobuf::TypeError) |
| 141 | + from foo/baz_pb.rb:10:in `<main>' |
| 142 | +``` |
| 143 | + |
| 144 | +The problem is that `bar.proto` is being referred to by two different canonical |
| 145 | +names: `bar.proto` and `foo/bar.proto`. This is a user error: each import should |
| 146 | +always be referred to by a consistent full path. Hopefully user errors of this |
| 147 | +sort will be rare, but it is hard to know without trying. |
| 148 | + |
| 149 | +The code in this change prints a warning using `warn` if we detect that this |
| 150 | +edge case has occurred: |
| 151 | + |
| 152 | +``` |
| 153 | +$ RUBYLIB=foo ruby foo/baz_pb.rb |
| 154 | +Warning: Protobuf detected an import path issue while loading generated file foo/baz_pb.rb |
| 155 | +- foo/baz.proto imports bar.proto, but that import was loaded as foo/bar.proto |
| 156 | +Each proto file must use a consistent fully-qualified name. |
| 157 | +This will become an error in the next major version. |
| 158 | +``` |
| 159 | + |
| 160 | +There are two possible fixes in this case. One is to consistently use the name |
| 161 | +`bar.proto` for the import (removing `-I.`). The other is to consistently use |
| 162 | +the name `foo/bar.proto` for the import (changing the import line to `import |
| 163 | +"foo/bar.proto"` and removing `-Ifoo`). |
| 164 | + |
| 165 | +We plan to remove this compatibility code in the next major version. |
| 166 | + |
| 167 | +## Dropping Support for Bazel <5.3 {#bazel} |
| 168 | + |
| 169 | +v23 will drop support for Bazel 4. Protobuf will continue to support the Bazel 5 |
| 170 | +LTS with Bazel 5.3 as the minimum required version. This is per the build |
| 171 | +support policy described in |
| 172 | +[Foundational C++ Support Policy](https://opensource.google/documentation/policies/cplusplus-support#build_systems) |
| 173 | +and as reflected in the versions in |
| 174 | +[Foundational C++ Support](https://github.com/google/oss-policies-info/blob/main/foundational-cxx-support-matrix.md). |
| 175 | + |
| 176 | +## Syntax Reflection Deprecation {#deprecation} |
| 177 | + |
| 178 | +v23 will deprecate the ability to check syntax version using reflection. The |
| 179 | +deprecation will be included as warnings at build time. The capability will be |
| 180 | +removed in a future release. |
0 commit comments