Skip to content

Commit e240485

Browse files
Protocol Buffer TeamLogofile
authored andcommitted
Project import generated by Copybara.
PiperOrigin-RevId: 548150989 Change-Id: I612f699f5476149a5a8e738d3fe302a67afc1b4f
1 parent 4f13dbb commit e240485

File tree

10 files changed

+107
-32
lines changed

10 files changed

+107
-32
lines changed

content/programming-guides/encoding.md

Lines changed: 14 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -81,15 +81,15 @@ And here is 150, encoded as `` `9601` `` -- this is a bit more complicated:
8181
How do you figure out that this is 150? First you drop the MSB from each byte,
8282
as this is just there to tell us whether we've reached the end of the number (as
8383
you can see, it's set in the first byte as there is more than one byte in the
84-
varint). Then we concatenate the 7-bit payloads, and interpret it as a
85-
little-endian, 64-bit unsigned integer:
84+
varint). These 7-bit payloads are in little-endian order. Convert to big-endian
85+
order, concatenate, and interpret as an unsigned 64-bit integer:
8686

8787
```proto
8888
10010110 00000001 // Original inputs.
8989
0010110 0000001 // Drop continuation bits.
90-
0000001 0010110 // Put into little-endian order.
91-
10010110 // Concatenate.
92-
128 + 16 + 4 + 2 = 150 // Interpret as integer.
90+
0000001 0010110 // Convert to big-endian.
91+
00000010010110 // Concatenate.
92+
128 + 16 + 4 + 2 = 150 // Interpret as an unsigned 64-bit integer.
9393
```
9494

9595
Because varints are so crucial to protocol buffers, in protoscope syntax, we
@@ -218,7 +218,7 @@ original, signed version.
218218
In protoscope, suffixing an integer with a `z` will make it encode as ZigZag.
219219
For example, `-500z` is the same as the varint `999`.
220220

221-
### Non-varint Numbers
221+
### Non-varint Numbers {#non-varints}
222222

223223
Non-varint numeric types are simple -- `double` and `fixed64` have wire type
224224
`I64`, which tells the parser to expect a fixed eight-byte lump of data. We can
@@ -331,7 +331,11 @@ respect to each other is preserved. Thus, this could also have been encoded as
331331
5: 3
332332
```
333333

334-
There is no special treatment for `oneof`s in the wire format.
334+
### Oneofs {#oneofs}
335+
336+
[`Oneof` fields](/programming-guides/proto2#oneof) are
337+
encoded the same as if the fields were not in a `oneof`. The rules that apply to
338+
`oneofs` are independent of how they are represented on the wire.
335339

336340
### Last One Wins {#last-one-wins}
337341

@@ -411,7 +415,7 @@ Protocol buffer parsers must be able to parse repeated fields that were compiled
411415
as `packed` as if they were not packed, and vice versa. This permits adding
412416
`[packed=true]` to existing fields in a forward- and backward-compatible way.
413417

414-
### Maps
418+
### Maps {#maps}
415419

416420
Map fields are just a shorthand for a special kind of repeated field. If we have
417421

@@ -436,7 +440,7 @@ message Test6 {
436440
Thus, maps are encoded exactly like a `repeated` message field: as a sequence of
437441
`LEN`-typed records, with two fields each.
438442

439-
## Groups
443+
## Groups {#groups}
440444

441445
Groups are a deprecated feature that should not be used, but they remain in the
442446
wire format, and deserve a passing mention.
@@ -484,7 +488,7 @@ be written. Serialization order is an implementation detail, and the details of
484488
any particular implementation may change in the future. Therefore, protocol
485489
buffer parsers must be able to parse fields in any order.
486490

487-
### Implications
491+
### Implications {#implications}
488492

489493
* Do not assume the byte output of a serialized message is stable. This is
490494
especially true for messages with transitive bytes fields representing other

content/programming-guides/proto2.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1189,6 +1189,8 @@ the oneof automatically clears all the other members. You can check which value
11891189
in a oneof is set (if any) using a special `case()` or `WhichOneof()` method,
11901190
depending on your chosen language.
11911191
1192+
Field numbers for oneof fields must be unique within the enclosing message.
1193+
11921194
### Using Oneof {#using-oneof}
11931195
11941196
To define a oneof in your `.proto` you use the `oneof` keyword followed by your

content/programming-guides/proto3.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1110,6 +1110,8 @@ depending on your chosen language.
11101110
Note that if *multiple values are set, the last set value as determined by the
11111111
order in the proto will overwrite all previous ones*.
11121112
1113+
Field numbers for oneof fields must be unique within the enclosing message.
1114+
11131115
### Using Oneof
11141116
11151117
To define a oneof in your `.proto` you use the `oneof` keyword followed by your

content/programming-guides/techniques.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,20 @@ You can also send design and usage questions to
99
the
1010
[Protocol Buffers discussion group](http://groups.google.com/group/protobuf).
1111

12+
## Common Filename Suffixes {#suffixes}
13+
14+
It is fairly common to write messages to files in several different formats. We
15+
recommend using the following file extensions for these files.
16+
17+
Content | Extension
18+
------------------------------------------------------------------------- | ---------
19+
[Text Format](/reference/protobuf/textformat-spec) | `.txtpb`
20+
[Wire Format](/programming-guides/encoding) | `.binpb`
21+
[JSON Format](/programming-guides/proto3#json) | `.json`
22+
23+
For Text Format specifically, `.textproto` is also fairly common, but we
24+
recommend `.txtpb` for its brevity.
25+
1226
## Streaming Multiple Messages {#streaming}
1327

1428
If you want to write multiple messages to a single file or stream, it is up to

content/reference/cpp/cpp-generated.md

Lines changed: 45 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -115,9 +115,13 @@ In addition to these methods, the `Foo` class defines the following methods:
115115
- `Foo& operator=(Foo&& other)`: Move-assignment operator.
116116
- `void Swap(Foo* other)`: Swap content with another message.
117117
- `const UnknownFieldSet& unknown_fields() const`: Returns the set of unknown
118-
fields encountered while parsing this message.
118+
fields encountered while parsing this message. If `option optimize_for =
119+
LITE_RUNTIME` is specified in the `.proto` file, then the return type
120+
changes to `std::string&`.
119121
- `UnknownFieldSet* mutable_unknown_fields()`: Returns a pointer to the
120-
mutable set of unknown fields encountered while parsing this message.
122+
mutable set of unknown fields encountered while parsing this message. If
123+
`option optimize_for = LITE_RUNTIME` is specified in the `.proto` file, then
124+
the return type changes to `std::string*`.
121125

122126
The class also defines the following static methods:
123127

@@ -132,6 +136,38 @@ The class also defines the following static methods:
132136
default instance of a message can be used as a factory by calling its
133137
`New()` method.
134138

139+
### Generated Filenames {#generated-filenames}
140+
141+
[Reserved keywords](https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/compiler/cpp/helpers.cc#L4)
142+
are appended with an underscore in the generated output.
143+
144+
For example, the following proto3 definition syntax:
145+
146+
```proto
147+
message MyMessage {
148+
string false = 1;
149+
string myFalse = 2;
150+
}
151+
```
152+
153+
generates the following partial output:
154+
155+
```cpp
156+
void clear_false_() ;
157+
const std::string& false_() const;
158+
void set_false_(Arg_&& arg, Args_... args);
159+
std::string* mutable_false_();
160+
PROTOBUF_NODISCARD std::string* release_false_();
161+
void set_allocated_false_(std::string* ptr);
162+
163+
void clear_myfalse() ;
164+
const std::string& myfalse() const;
165+
void set_myfalse(Arg_&& arg, Args_... args);
166+
std::string* mutable_myfalse();
167+
PROTOBUF_NODISCARD std::string* release_myfalse();
168+
void set_allocated_myfalse(std::string* ptr);
169+
```
170+
135171
### Nested Types
136172
137173
A message can be declared inside another message. For example:
@@ -184,7 +220,7 @@ any method inherited from `Message` or accessing the message through other ways
184220
Correspondingly, the value of the returned pointer is never guaranteed to be the
185221
same across two different invocations of the accessor.
186222

187-
### Singular Numeric Fields (proto2)
223+
### Optional Numeric Fields (proto2 and proto3)
188224

189225
For either of these field definitions:
190226

@@ -207,7 +243,7 @@ For other numeric field types (including `bool`), `int32` is replaced with the
207243
corresponding C++ type according to the
208244
[scalar value types table](/programming-guides/proto3#scalar).
209245

210-
### Singular Numeric Fields (proto3)
246+
### Implicit Presence Numeric Fields (proto3)
211247

212248
For these field definitions:
213249

@@ -229,7 +265,7 @@ For other numeric field types (including `bool`), `int32` is replaced with the
229265
corresponding C++ type according to the
230266
[scalar value types table](/programming-guides/proto3#scalar).
231267

232-
### Singular String/Bytes Fields (proto2) {#string}
268+
### Optional String/Bytes Fields (proto2 and proto3) {#string}
233269

234270
For any of these field definitions:
235271

@@ -278,7 +314,7 @@ The compiler will generate the following accessor methods:
278314
calling this, caller takes the ownership of the allocated `string` object,
279315
`has_foo()` will return `false`, and `foo()` will return the default value.
280316

281-
### Singular String/Bytes Fields (proto3) {#proto3_string}
317+
### Implicit Presence String/Bytes Fields (proto3) {#proto3_string}
282318

283319
For any of these field definitions:
284320

@@ -355,7 +391,7 @@ The compiler will generate the following accessor methods:
355391
`foo()` will return an empty `Cord` (proto3) or the default value (proto2).
356392
- `bool has_foo()`: Returns `true` if the field is set.
357393

358-
### Singular Enum Fields (proto2) {#enum_field}
394+
### Optional Enum Fields (proto2 and proto3) {#enum_field}
359395

360396
Given the enum type:
361397

@@ -386,7 +422,7 @@ The compiler will generate the following accessor methods:
386422
- `void clear_foo()`: Clears the value of the field. After calling this,
387423
`has_foo()` will return `false` and `foo()` will return the default value.
388424

389-
### Singular Enum Fields (proto3)
425+
### Implicit Presence Enum Fields (proto3)
390426

391427
Given the enum type:
392428

@@ -414,7 +450,7 @@ The compiler will generate the following accessor methods:
414450
- `void clear_foo()`: Clears the value of the field. After calling this,
415451
`foo()` will return the default value.
416452

417-
### Singular Embedded Message Fields {#embeddedmessage}
453+
### Optional Embedded Message Fields (proto2 and proto3) {#embeddedmessage}
418454

419455
Given the message type:
420456

content/reference/go/go-generated.md

Lines changed: 15 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -385,13 +385,13 @@ The following example shows how to set the field:
385385

386386
```go
387387
p1 := &account.Profile{
388-
Avatar: &account.Profile_ImageUrl{"http://example.com/image.png"},
388+
Avatar: &account.Profile_ImageUrl{ImageUrl: "http://example.com/image.png"},
389389
}
390390

391391
// imageData is []byte
392392
imageData := getImageData()
393393
p2 := &account.Profile{
394-
Avatar: &account.Profile_ImageData{imageData},
394+
Avatar: &account.Profile_ImageData{ImageData: imageData},
395395
}
396396
```
397397

@@ -429,15 +429,26 @@ message Venue {
429429
KIND_STADIUM = 2;
430430
KIND_BAR = 3;
431431
KIND_OPEN_AIR_FESTIVAL = 4;
432-
// ...
433432
}
434433
Kind kind = 1;
435434
// ...
436435
}
437436
```
438437

439438
the protocol buffer compiler generates a type and a series of constants with
440-
that type.
439+
that type:
440+
441+
```go
442+
type Venue_Kind int32
443+
444+
const (
445+
Venue_KIND_UNSPECIFIED Venue_Kind = 0
446+
Venue_KIND_CONCERT_HALL Venue_Kind = 1
447+
Venue_KIND_STADIUM Venue_Kind = 2
448+
Venue_KIND_BAR Venue_Kind = 3
449+
Venue_KIND_OPEN_AIR_FESTIVAL Venue_Kind = 4
450+
)
451+
```
441452

442453
For enums within a message (like the one above), the type name begins with the
443454
message name:

content/reference/kotlin/kotlin-generated.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -332,8 +332,8 @@ The protocol buffer compiler will add the following methods to `FooKt.Dsl`:
332332
- `operator fun <T : Comparable<T>> set(extension: ExtensionLite<Foo, T>)`:
333333
sets the current value of the extension field in the DSL (for `Comparable`
334334
field types)
335-
- `operator fun <T : MessageLite<T>> set(extension: ExtensionLite<Foo, T>)`:
336-
sets the current value of the extension field in the DSL (for message field
335+
- `operator fun <T : MessageLite> set(extension: ExtensionLite<Foo, T>)`: sets
336+
the current value of the extension field in the DSL (for message field
337337
types)
338338
- `operator fun set(extension: ExtensionLite<Foo, ByteString>)`: sets the
339339
current value of the extension field in the DSL (for `bytes` fields)
@@ -350,7 +350,7 @@ The protocol buffer compiler will add the following methods to `FooKt.Dsl`:
350350
alias for `addAll` using operator syntax
351351
- `operator fun <E> ExtensionList<Foo, E>.set(index: Int, value: E)`: sets the
352352
element of the repeated extension field at the specified index
353-
- `operator fun ExtensionList<Foo, *>.clear()`: clears the elements of the
353+
- `inline fun ExtensionList<Foo, *>.clear()`: clears the elements of the
354354
repeated extension field
355355

356356
The generics here are complex, but the effect is that `this[extension] = value`

content/reference/protobuf/textformat-spec.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -571,6 +571,8 @@ Both the `key` and `value` fields are optional and default to the zero value of
571571
their respective types if unspecified. If a key is duplicated, only the
572572
last-specified value will be retained in a parsed map.
573573

574+
The order of maps is not maintained in textprotos.
575+
574576
## `oneof` Fields {#oneof}
575577

576578
While there is no special syntax related to `oneof` fields in text format, only
@@ -657,7 +659,7 @@ the schema, so they may provide various features.
657659

658660
```textproto
659661
# proto-file: some/proto/my_file.proto
660-
# proto-message: some.package.MyMessage
662+
# proto-message: MyMessage
661663
```
662664

663665
## Working with the Format Programmatically

content/reference/python/python-generated.md

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -6,11 +6,11 @@ description = "This topic describes exactly what Python definitions the protocol
66
type = "docs"
77
+++
88

9-
Any differences
10-
between proto2 and proto3 generated code are highlighted - note that these
11-
differences are in the generated code as described in this document, not the
12-
base message classes/interfaces, which are the same in both versions. You should
13-
read the
9+
Any
10+
differences between proto2 and proto3 generated code are highlighted - note that
11+
these differences are in the generated code as described in this document, not
12+
the base message classes/interfaces, which are the same in both versions. You
13+
should read the
1414
[proto2 language guide](/programming-guides/proto) and/or
1515
[proto3 language guide](/programming-guides/proto3)
1616
before reading this document.
@@ -48,6 +48,8 @@ The compiler will automatically create the directory `build/gen/bar` if
4848
necessary, but it will *not* create `build` or `build/gen`; they must already
4949
exist.
5050

51+
Protoc can generate Python stubs (`.pyi`) using the `--pyi_out` parameter.
52+
5153
Note that if the `.proto` file or its path contains any characters which cannot
5254
be used in Python module names (for example, hyphens), they will be replaced
5355
with underscores. So, the file `foo-bar.proto` becomes the Python file

content/support/version-support.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -813,6 +813,8 @@ went out of support on July 1, 2023.
813813
Protobuf is committed to following the platform and library support policy
814814
described in
815815
[Python Support Policy](https://cloud.google.com/python/docs/supported-python-versions).
816+
For specific versions supported, see
817+
[Foundational Python Support Matrix](https://github.com/google/oss-policies-info/blob/main/foundational-python-support-matrix.md).
816818

817819
## Ruby {#ruby}
818820

0 commit comments

Comments
 (0)