|
| 1 | ++++ |
| 2 | +title = "Extension Declarations" |
| 3 | +weight = 83 |
| 4 | +description = "This topic describes in detail what extension declarations are, why we need them, and how we use them." |
| 5 | +type = "docs" |
| 6 | ++++ |
| 7 | + |
| 8 | +<!--* |
| 9 | +# Document freshness: For more information, see go/fresh-source. |
| 10 | +freshness: { owner: 'shaod' reviewed: '2023-09-06' } |
| 11 | +*--> |
| 12 | + |
| 13 | +## Introduction {#intro} |
| 14 | + |
| 15 | +This page describes in detail what extension declarations are, why we need them, |
| 16 | +and how we use them. |
| 17 | + |
| 18 | +**NOTE:** Extension declarations are used in proto2 only, as proto3 does not |
| 19 | +support extensions at this time. |
| 20 | + |
| 21 | +If you need an introduction to extensions, read this |
| 22 | +[extensions guide](https://protobuf.dev/programming-guides/proto2/#extensions) |
| 23 | + |
| 24 | +## Motivation {#motivation} |
| 25 | + |
| 26 | +Extension declarations aim to strike a happy medium between regular fields and |
| 27 | +extensions. Like extensions, they avoid creating a dependency on the message |
| 28 | +type of the field, which therefore results in a leaner build graph and smaller |
| 29 | +binaries in environments where unused messages are difficult or impossible to |
| 30 | +strip. Like regular fields, the field name/number appear in the enclosing |
| 31 | +message, which makes it easier to avoid conflicts and see a convenient listing |
| 32 | +of what fields are declared. |
| 33 | + |
| 34 | +Listing the occupied extension numbers with extension declarations makes it |
| 35 | +easier for users to pick an available extension number and to avoid conflicts. |
| 36 | + |
| 37 | +## Usage {#usage} |
| 38 | + |
| 39 | +Extension declarations are an option of extension ranges. Like forward |
| 40 | +declarations in C++, you can declare the field type, field name, and cardinality |
| 41 | +(singular or repeated) of an extension field without importing the `.proto` file |
| 42 | +containing the full extension definition: |
| 43 | + |
| 44 | +```proto |
| 45 | +syntax = "proto2"; |
| 46 | +
|
| 47 | +message Foo { |
| 48 | + extensions 4 to 1000 [ |
| 49 | + declaration = { |
| 50 | + number: 4, |
| 51 | + full_name: ".my.package.event_annotations", |
| 52 | + type: ".logs.proto.ValidationAnnotations", |
| 53 | + repeated: true }, |
| 54 | + declaration = { |
| 55 | + number: 999, |
| 56 | + full_name: ".foo.package.bar", |
| 57 | + type: "int32"}]; |
| 58 | +} |
| 59 | +``` |
| 60 | + |
| 61 | +This syntax has the following semantics: |
| 62 | + |
| 63 | +* Multiple `declaration`s with distinct extension numbers can be defined in a |
| 64 | + single extension range if the size of the range allows. |
| 65 | +* If there is any declaration for the extension range, *all* extensions of the |
| 66 | + range must also be declared. This prevents non-declared extensions from |
| 67 | + being added, and enforces that any new extensions use declarations for the |
| 68 | + range. |
| 69 | +* The given message type (`.logs.proto.ValidationAnnotations`) does not need |
| 70 | + to have been previously defined or imported. We check only that it is a |
| 71 | + valid name that could potentially be defined in another `.proto` file. |
| 72 | +* When this or another `.proto` file defines an extension of this message |
| 73 | + (`Foo`) with this name or number, we enforce that the number, type, and full |
| 74 | + name of the extension match what is forward-declared here. |
| 75 | + |
| 76 | +**WARNING:** Avoid using declarations for extension range groups such as |
| 77 | +`extensions 4, 999`. It is unclear which extension range the declarations apply |
| 78 | +to, and it is currently unsupported. |
| 79 | + |
| 80 | +The extension declarations expect two extension fields with different packages: |
| 81 | + |
| 82 | +```proto |
| 83 | +package my.package; |
| 84 | +extend Foo { |
| 85 | + repeated logs.proto.ValidationAnnotations event_annotations = 4; |
| 86 | +} |
| 87 | +``` |
| 88 | + |
| 89 | +```proto |
| 90 | +package foo.package; |
| 91 | +extend Foo { |
| 92 | + optional int32 bar = 999; |
| 93 | +} |
| 94 | +``` |
| 95 | + |
| 96 | +### Reserved Declarations {#reserved} |
| 97 | + |
| 98 | +An extension declaration can be marked `reserved: true` to indicate that it is |
| 99 | +no longer actively used and the extension definition has been deleted. **Do not |
| 100 | +delete the extension declaration or edit its `type` or `full_name` value**. |
| 101 | + |
| 102 | +This `reserved` tag is separate from the reserved keyword for regular fields and |
| 103 | +**does not require breaking up the extension range**. |
| 104 | + |
| 105 | +```proto {highlight="context:reserved"} |
| 106 | +syntax = "proto2"; |
| 107 | +
|
| 108 | +message Foo { |
| 109 | + extensions 4 to 1000 [ |
| 110 | + declaration = { |
| 111 | + number: 500, |
| 112 | + full_name: ".my.package.event_annotations", |
| 113 | + type: ".logs.proto.ValidationAnnotations", |
| 114 | + reserved: true }]; |
| 115 | +} |
| 116 | +``` |
| 117 | + |
| 118 | +An extension field definition using a number that is `reserved` in the |
| 119 | +declaration will fail to compile. |
| 120 | + |
| 121 | +## Representation in descriptor.proto {#representation} |
| 122 | + |
| 123 | +Extension declaration is represented in descriptor.proto as fields in |
| 124 | +`proto2.ExtensionRangeOptions`: |
| 125 | + |
| 126 | +```proto |
| 127 | +message ExtensionRangeOptions { |
| 128 | + message Declaration { |
| 129 | + optional int32 number = 1; |
| 130 | + optional string full_name = 2; |
| 131 | + optional string type = 3; |
| 132 | + optional bool reserved = 5; |
| 133 | + optional bool repeated = 6; |
| 134 | + } |
| 135 | + repeated Declaration declaration = 2; |
| 136 | +} |
| 137 | +``` |
| 138 | + |
| 139 | +## Reflection Field Lookup {#reflection} |
| 140 | + |
| 141 | +Extension declarations are *not* returned from the normal field lookup functions |
| 142 | +like `Descriptor::FindFieldByName()` or `Descriptor::FindFieldByNumber()`. Like |
| 143 | +extensions, they are discoverable by extension lookup routines like |
| 144 | +`DescriptorPool::FindExtensionByName()`. This is an explicit choice that |
| 145 | +reflects the fact that declarations are not definitions and do not have enough |
| 146 | +information to return a full `FieldDescriptor`. |
| 147 | + |
| 148 | +Declared extensions still behave like regular extensions from the perspective of |
| 149 | +TextFormat and JSON. It also means that migrating an existing field to a |
| 150 | +declared extension will require first migrating any reflective use of that |
| 151 | +field. |
| 152 | + |
| 153 | +## Use Extension Declarations to Allocate Numbers {#recommendation} |
| 154 | + |
| 155 | +Extensions use field numbers just like ordinary fields do, so it is important |
| 156 | +for each extension to be assigned a number that is unique within the parent |
| 157 | +message. We recommend using extension |
| 158 | +declarations to declare the field |
| 159 | +number and type for each extension in the parent message. The extension |
| 160 | +declarations serve as a registry of all the parent message's extensions, and |
| 161 | +protoc will enforce that there are no field number conflicts. When you add a new |
| 162 | +extension, choose the number by just incrementing by one the previously added |
| 163 | +extension number. Whenever you delete an extension, make sure to mark the field |
| 164 | +number `reserved` to eliminate the risk of accidentally reusing |
| 165 | +it. |
| 166 | + |
| 167 | +This convention is only a recommendation--the protobuf team does not have the |
| 168 | +ability or desire to force anyone to adhere to it for every extendable message. |
| 169 | +If you as the owner of an extendable proto do not want to coordinate extension |
| 170 | +numbers through extension declarations, you can choose to provide coordination |
| 171 | +through other means. Be very careful, though, |
| 172 | +because accidental reuse of an extension number can cause serious problems. |
| 173 | + |
| 174 | +One way to sidestep the issue would be to avoid extensions entirely and use |
| 175 | +[`google.protobuf.Any`](/programming-guides/proto3/#any) |
| 176 | +instead. This could be a good choice for APIs that front storage or for |
| 177 | +pass-through systems where the client cares about the contents of the proto but |
| 178 | +the system receiving it does not. |
| 179 | + |
| 180 | +### Consequences of Reusing an Extension Number {#reusing} |
| 181 | + |
| 182 | +An extension is a field defined outside the container message; usually in a |
| 183 | +separate .proto file. This distribution of definitions makes it easy for two |
| 184 | +developers to accidentally create different definitions for the same extension |
| 185 | +field number. |
| 186 | + |
| 187 | +The consequences of changing an extension definition are the same for extensions |
| 188 | +and standard fields. Reusing a field number introduces an ambiguity in how a |
| 189 | +proto should be decoded from the wire format. The protobuf wire format is lean |
| 190 | +and doesn’t provide a good way to detect fields encoded using one definition and |
| 191 | +decoded using another. |
| 192 | + |
| 193 | +This ambiguity can manifest in a short time frame, such as a client using one |
| 194 | +extension definition and a server using another communicating |
| 195 | +. |
| 196 | + |
| 197 | +This ambiguity can also manifest over a longer time frame, such as storing data |
| 198 | +encoded using one extension definition and later retrieving and decoding using |
| 199 | +the second extension definition. This long-term case can be difficult to |
| 200 | +diagnose if the first extension definition was deleted after the data was |
| 201 | +encoded and stored. |
| 202 | + |
| 203 | +The outcome of this can be: |
| 204 | + |
| 205 | +1. A parse error (best case scenario). |
| 206 | +2. Leaked PII / SPII – if PII or SPII is written using one extension definition |
| 207 | + and read using another extension definition. |
| 208 | +3. Data Corruption – if data is read using the “wrong” definition, modified and |
| 209 | + rewritten. |
| 210 | + |
| 211 | +Data definition ambiguity is almost certainly going to cost someone time for |
| 212 | +debugging at a minimum. It could also cause data leaks or corruption that takes |
| 213 | +months to clean up. |
| 214 | + |
| 215 | +## Usage Tips |
| 216 | + |
| 217 | +### Never Delete an Extension Declaration {#never-delete} |
| 218 | + |
| 219 | +Deleting an extension declaration opens the door to accidental reuse in the |
| 220 | +future. If the extension is no longer processed and the definition is deleted, |
| 221 | +the extension declaration can be [marked reserved](#reserved). |
| 222 | + |
| 223 | +### Never Use a Field Number from the `reserved` List for a New Extension Declaration {#never-reuse-reserved} |
| 224 | + |
| 225 | +Reserved numbers may have been used for fields or other extensions in the past. |
| 226 | + |
| 227 | +### Never change the type of an existing extension declaration {#never-change-type} |
| 228 | + |
| 229 | +Changing the extension field's type can result in data corruption. |
| 230 | + |
| 231 | +If the extension field is of an enum or message type, and that enum or message |
| 232 | +type is being renamed, updating the declaration name is required and safe. To |
| 233 | +avoid breakages, the update of the type, the extension field definition, and |
| 234 | +extension declaration should all happen in a single |
| 235 | +commit. |
| 236 | + |
| 237 | +### Use Caution When Renaming an Extension Field {#caution-renaming} |
| 238 | + |
| 239 | +While renaming an extension field is fine for the wire format, it can break JSON |
| 240 | +and TextFormat parsing. |
0 commit comments