Skip to content

Commit 2f2ea58

Browse files
committed
docs: ✏️ optimize README.md
1 parent fc767a5 commit 2f2ea58

File tree

4 files changed

+213
-57
lines changed

4 files changed

+213
-57
lines changed

README.md

Lines changed: 105 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -4,11 +4,19 @@ Little cli utility for lazy guy😉 ~ Transforming protobuf idl to thrift, and v
44
[![YYCoder](https://circleci.com/gh/YYCoder/protobuf-thrift.svg?style=svg)](https://app.circleci.com/pipelines/github/YYCoder/protobuf-thrift)
55
[![GoDoc](https://pkg.go.dev/badge/github.com/YYCoder/protobuf-thrift)](https://pkg.go.dev/github.com/YYCoder/protobuf-thrift)
66
[![goreportcard](https://goreportcard.com/badge/github.com/yycoder/protobuf-thrift)](https://goreportcard.com/report/github.com/yycoder/protobuf-thrift)
7+
[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](http://makeapullrequest.com)
78

89
> [IDL](https://en.wikipedia.org/wiki/IDL)(Interface description language), which is a descriptive language used to define data types and interfaces in a way that is independent of the programming language or operating system/processor platform.
910
1011
[中文文档](./docs/cn.md)
1112

13+
Feel free to try out our [web interface](https://pb-thrift.markeyyuan.monster/), and of course, both languages specification as below, if there are any questions, don't hesitate to open an issue, and PRs are welcome too.
14+
15+
* [thrift](https://thrift.apache.org/docs/idl.html)
16+
17+
* [protobuf](https://developers.google.com/protocol-buffers/docs/reference/proto3-spec)
18+
19+
1220

1321
## Install
1422
For folks don't have GO development environment, directly download corresponding platform binary from latest release is the best choice.
@@ -59,31 +67,103 @@ protobuf-thrift -t thrift2proto -i ./path/to/idl.thrift -o ./idl.proto -r 1
5967

6068
Since protobuf and thrift have many different syntaxes, we can only transform syntaxes that have same meaning, e.g. protobuf message => thrift struct, protobuf enum => thrift enum.
6169

62-
Here is a list of transformation rule, so we hope you don't have to worry about protobuf-thrift do sth unexpected.
63-
64-
|protobuf type|thrift type|field type|notice|
65-
|:--:|:--:|:--:|:--:|
66-
|message|struct|optional => optional; repeated T => list\<T\>|only protobuf 2 have optional field|
67-
|map<T1,T2>|map<T1,T2>||T1 only support int32/int64/string/float/double, due to thrift syntax|
68-
|enum|enum|||
69-
|int32|i32|||
70-
|int64|i64|||
71-
|float|double|||
72-
|double|double|||
73-
|bool|bool|||
74-
|string|string|||
75-
|bytes|binary|||
76-
|service|service|rpc => methods||
77-
|constant|const||not support currently|
78-
|package|namespace|||
79-
|import|include|||
80-
|syntax|||only supported in protobuf, so thrift will omit it|
81-
|option|||only supported in protobuf, so thrift will omit it|
82-
|extend|||only supported in protobuf, so thrift will omit it|
83-
|extension|||only supported in protobuf, so thrift will omit it|
84-
85-
### Nested Fields
86-
protobuf support nested field within message, but thrift does not, so protobuf-thrift will prefix nested field name with outer message name to work around this. for example:
70+
We hope you don't have to worry about protobuf-thrift do sth unexpected, so we **strongly recommend you to read the following document** to get a grasp of what it will do for specific syntaxes.
71+
72+
### Basic Types
73+
Here is a list of basic type conversion rules:
74+
75+
|[protobuf type](https://developers.google.com/protocol-buffers/docs/proto3#scalar)|[thrift type](https://thrift.apache.org/docs/types.html#base-types)|
76+
|:--:|:--:|
77+
|uint32|-|
78+
|uint64|-|
79+
|sint32|-|
80+
|sint64|-|
81+
|fixed32|-|
82+
|fixed64|-|
83+
|sfixed32|-|
84+
|sfixed64|-|
85+
|-|i16|
86+
|int32|i32|
87+
|int64|i64|
88+
|float|double|
89+
|double|double|
90+
|bool|bool|
91+
|string|string|
92+
|bytes|-|
93+
|-|byte|
94+
95+
### Enum
96+
Protobuf and thrift both have `enum` declaration syntax and basically same grammar, only to note that:
97+
98+
> **Proto3 enum declaration's first element must be zero, so if thrift enum with non-zero first element transform to protobuf, protobut-thrift will automatically generate a zero element for you.**
99+
100+
for example, if thrift enum like this:
101+
102+
```thrift
103+
enum Status {
104+
StatusUnreviewed = 1 // first non-zero element
105+
StatusOnline = 2
106+
StatusRejected = 3
107+
StatusOffline = 4
108+
}
109+
```
110+
111+
will be transformed to:
112+
113+
```protobuf
114+
enum Status {
115+
Status_Unknown = 0;
116+
Status_Unreviewed = 1; // first non-zero element
117+
Status_Online = 2;
118+
Status_Rejected = 3;
119+
Status_Offline = 4;
120+
}
121+
```
122+
123+
### Service
124+
Protobuf and thrift both have same `service` declaration syntax, but there are several differences:
125+
126+
1. **oneway**: only thrift support, which means function will not wait for response. so during thrift-to-pb transformation, this keyword will be ignored.
127+
128+
2. **throws**: only thrift support, which specified what kind of exceptions can be thrown by the function. this keyword will be ignored, too, in thrift-to-pb mode.
129+
130+
3. **arguments**:
131+
* thrift supports multiple arguments for one function, but protobuf only supports one, so it will ignore all the arguments other than the first one in thrift-to-pb transformation.
132+
133+
* thrift functions support `void` return type, but protobuf doesn't, so it will leave the return type blank in thrift-to-pb mode.
134+
135+
* currently, only support basic type and identifier for function/rpc request and response type, might be implemented in the future.
136+
137+
### Options || Annotation
138+
Both language support this feature, but they have different syntax to apply it, since the meaning for them are language-bound, we decide to ignore this between transformations.
139+
140+
### Message || Struct
141+
Thrift `struct` and protobuf `message` are very similar, but still have some differences:
142+
143+
1. **set type**: only thrift support, it will be transformed to `repeated` field in protobuf just like thrift `list`.
144+
145+
2. **optional**: thrift and proto2 support, it will be ignored in thrift-to-pb mode if protobuf syntax is proto3
146+
147+
3. **required**: thrift and proto2 support, since it's highly not recommend to mark field as `required`, currently it will be ignored, if you have any questions about this, please open an issue.
148+
149+
4. **map type**: as protobuf [language-specification](https://developers.google.com/protocol-buffers/docs/reference/proto3-spec#map_field) mentioned, protobuf only support basic type as key type, but thrift support any [FieldType](https://thrift.apache.org/docs/idl.html) as map key type, for simplicity, currently only support basic type and identifier as map key and value
150+
151+
152+
### Import || Include
153+
As [language-specification](https://developers.google.com/protocol-buffers/docs/proto#importing_definitions) mentioned, protobuf import paths are relative to protoc command's working directory or using -I/--proto_path specified path, and can not include relative paths prefix, such as `./XXX.proto`, we are not able to detect the correct path for current file both in thrift-to-pb mode and pb-to-thrift mode, since it's dynamic.
154+
155+
So, you have to manually check whether the generated path is correct.
156+
157+
### Constant || Const
158+
Currently not supported.
159+
160+
### Package || Namespace
161+
Thrift `namespace` value will be used for protobuf `package`, the NamespaceScope will be ignored in thrift-to-pb mode.
162+
163+
In pb-to-thrift mode, generated `namespace` will use `*` as NamespaceScope.
164+
165+
### Nested Types
166+
Protobuf supports [nested types](https://developers.google.com/protocol-buffers/docs/proto#nested) within message, but thrift does not, so protobuf-thrift will prefix nested field name with outer message name to work around this. for example:
87167

88168
```protobuf
89169
message GroupMsgTaskQueryExpress {

docs/cn.md

Lines changed: 101 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,16 @@
44
[![YYCoder](https://circleci.com/gh/YYCoder/protobuf-thrift.svg?style=svg)](https://app.circleci.com/pipelines/github/YYCoder/protobuf-thrift)
55
[![GoDoc](https://pkg.go.dev/badge/github.com/YYCoder/protobuf-thrift)](https://pkg.go.dev/github.com/YYCoder/protobuf-thrift)
66
[![goreportcard](https://goreportcard.com/badge/github.com/yycoder/protobuf-thrift)](https://goreportcard.com/report/github.com/yycoder/protobuf-thrift)
7+
[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](http://makeapullrequest.com)
78

89
> [IDL](https://en.wikipedia.org/wiki/IDL)(Interface description language)。是指一种用于定义数据类型以及接口的描述性语言,与编程语言以及平台无关,常用在微服务架构中。
910
11+
欢迎试用我们的 [web 界面](https://pb-thrift.markeyyuan.monster/),更简单直观地进行转换,以及,如下是两者的语言规范,如果有任何问题,欢迎提 issue 或 PR。
12+
13+
* [thrift](https://thrift.apache.org/docs/idl.html)
14+
15+
* [protobuf](https://developers.google.com/protocol-buffers/docs/reference/proto3-spec)
16+
1017
## 安装
1118
如果没有 go 开发环境,可以直接从 release 中下载最新版的对应平台的二进制文件。
1219

@@ -53,28 +60,100 @@ protobuf-thrift -t thrift2proto -i ./path/to/idl.thrift -o ./idl.proto -r 1
5360
## 注意事项
5461
由于 protobuf 与 thrift 有很多语法上的不同,我们不可能完全将一种 idl 转换成另一种,protobuf-thrift 也只是一个帮助我们摆脱复制粘贴的小工具,它所提供的功能能够满足 80% 的场景就足够了。因此,我们只会尽可能将有相同语义的语法进行转换,如 protobuf message => thrift struct,protobuf enum => thrift enum。
5562

56-
为了确保你能够明确的知道 protobuf-thrift 会如何转换,如下是目前的转换规则:
57-
58-
|protobuf type|thrift type|field type|notice|
59-
|:--:|:--:|:--:|:--:|
60-
|message|struct|optional => optional; repeated T => list\<T\>|only protobuf 2 have optional field|
61-
|map<T1,T2>|map<T1,T2>||T1 only support int32/int64/string/float/double, due to thrift syntax|
62-
|enum|enum|||
63-
|int32|i32|||
64-
|int64|i64|||
65-
|float|double|||
66-
|double|double|||
67-
|bool|bool|||
68-
|string|string|||
69-
|bytes|binary|||
70-
|service|service|rpc => methods||
71-
|constant|const||not support currently|
72-
|package|namespace|||
73-
|import|include|||
74-
|syntax|||only supported in protobuf, so thrift will omit it|
75-
|option|||only supported in protobuf, so thrift will omit it|
76-
|extend|||only supported in protobuf, so thrift will omit it|
77-
|extension|||only supported in protobuf, so thrift will omit it|
63+
为了确保你能够明确的知道 protobuf-thrift 会如何转换,我们**强烈建议你阅读下方的文档**,从而明确了解对于特定语法是如何做转换的。
64+
65+
### 基本类型
66+
如下是两种 idl 语言的基本类型转换规则:
67+
68+
|[protobuf type](https://developers.google.com/protocol-buffers/docs/proto3#scalar)|[thrift type](https://thrift.apache.org/docs/types.html#base-types)|
69+
|:--:|:--:|
70+
|uint32|-|
71+
|uint64|-|
72+
|sint32|-|
73+
|sint64|-|
74+
|fixed32|-|
75+
|fixed64|-|
76+
|sfixed32|-|
77+
|sfixed64|-|
78+
|-|i16|
79+
|int32|i32|
80+
|int64|i64|
81+
|float|double|
82+
|double|double|
83+
|bool|bool|
84+
|string|string|
85+
|bytes|-|
86+
|-|byte|
87+
88+
### Enum
89+
Protobuf 和 thrift 都有 `enum` 声明,并且语法基本一致,只有如下一点需要注意:
90+
91+
> **Proto3 的 enum 声明中第一个元素必须值为 0,因此在 thrift 转换 pb 的过程中,源 thrift 枚举中不包括值为 0 的元素,则 protobuf-thrift 会自动添加。**
92+
93+
如下例:
94+
95+
```thrift
96+
enum Status {
97+
StatusUnreviewed = 1 // first non-zero element
98+
StatusOnline = 2
99+
StatusRejected = 3
100+
StatusOffline = 4
101+
}
102+
```
103+
104+
会转换成:
105+
106+
```protobuf
107+
enum Status {
108+
Status_Unknown = 0;
109+
Status_Unreviewed = 1; // first non-zero element
110+
Status_Online = 2;
111+
Status_Rejected = 3;
112+
Status_Offline = 4;
113+
}
114+
```
115+
116+
### Service
117+
Protobuf 和 thrift 都有 `service` 作为顶级声明,但也有一些区别:
118+
119+
1. **oneway**: 只在 thrift 中支持,语义是该方法不会关心返回结果,在 thrift-to-pb 模式下该字段会被忽略
120+
121+
2. **throws**: 只在 thrift 中支持,语义是指定该函数可能抛出什么类型的异常,同上,在 thrift-to-pb 模式也会被忽略thrift-to-pb mode.
122+
123+
3. **函数参数**:
124+
* thrift 函数支持多个参数,但 pb 的 `rpc` 函数只支持一个参数,因此 thrift-to-pb 模式转换时会忽略除第一个参数以外的所有参数
125+
126+
* thrift 支持 `void` 返回类型,但 pb 不支持,在 thrift-to-pb 模式下会对返回 `void` 的 thrift 函数生成的 `rpc` 函数返回结果置空
127+
128+
* 目前函数参数和返回值都只支持基本类型和标识符,以后有需要可以在实现
129+
130+
### Options || Annotation
131+
两种语言都支持这个特性,但由于这种语法是跟语言强绑定的,强行搬到另一个语言中很难符合语义,因此目前在转换中都会忽略。
132+
133+
### Message || Struct
134+
Thrift `struct` 和 protobuf `message` 非常相似,但仍有些许不同:
135+
136+
1. **set type**: 只在 thrift 中支持,最终会被转成 protobuf 的 `repeated` 字段,thrift `list` 也一样
137+
138+
2. **optional**: thrift 和 proto2 支持,在 thrift-to-pb 模式下若选择的 `syntax` 是 proto3,则会忽略
139+
140+
3. **required**: thrift 和 proto2 支持,由于该字段标示为 required 在 pb 中是强烈不建议的,因此目前都会忽略,若有需求可以提 issue
141+
142+
4. **map type**: 正如 protobuf [语言规范](https://developers.google.com/protocol-buffers/docs/reference/proto3-spec#map_field) 中提到, protobuf 只支持基础类型作为 map 的 key,但 thrift 支持任意 [FieldType](https://thrift.apache.org/docs/idl.html),为了简洁性考虑,目前对于 map 的 key 和 value 都只支持基本类型和标识符
143+
144+
### Import || Include
145+
正如 [protobuf 语言规范](https://developers.google.com/protocol-buffers/docs/proto#importing_definitions) 中定义,protobuf `import` 路径是以 protoc 命令执行时的当前工作目录或 -I/--proto_path 指定的路径为基础路径的,并且也要求路径中不能包含相对路径前缀,如 `./XXX.proto`,因此我们无法在转换时得知正确的引用路径是什么。
146+
147+
因此,你需要在转换之后手动检查一下转换出来的路径是否正确,并自行修改。
148+
149+
### Constant || Const
150+
目前还不支持转换,若有需求欢迎提 issue 或 PR。
151+
152+
### Package || Namespace
153+
Thrift `namespace` 的 value 会被用作 `package` 的 value,但 NamespaceScope 在 thrift-to-pb 模式下会被忽略。
154+
155+
在 pb-to-thrift 模式下,生成的 `namespace` 会默认使用 `*` 作为 NamespaceScope。
156+
78157

79158
### 嵌套字段
80159
protobuf 支持在 message 结构体中嵌套字段(如 enum/message),但在 thrift 中不支持,因此 protobuf-thrift 会通过给嵌套字段的标识符使用外部 message 名称作为前缀的方式来实现相同命名空间的效果。如下例:

proto-2-thrift-gen.go

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -523,8 +523,8 @@ func (g *thriftGenerator) basicTypeConverter(t string) (res string, err error) {
523523
res = "double"
524524
case "bool":
525525
res = "bool"
526-
case "bytes":
527-
res = "binary"
526+
// case "bytes":
527+
// res = "byte"
528528
default:
529529
err = fmt.Errorf("Invalid basic type %s", t)
530530
}

thrift-2-proto-gen.go

Lines changed: 5 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -209,7 +209,6 @@ func (g *protoGenerator) handleIncludes(path string) (newFile FileInfo) {
209209
}
210210

211211
filePath := strings.ReplaceAll(path, ".thrift", ".proto")
212-
// TODO: update doc
213212
// ! NOTE: proto import paths are relative to protoc command's working directory or using
214213
// ! NOTE: -I/--proto_path specified path, and can not include relative paths prefix, such as `./XXX.proto`.
215214
// ! NOTE: so, user have to manually check the generated path is correct.
@@ -257,7 +256,7 @@ func (g *protoGenerator) handleService(s *thrifter.Service) {
257256
if !function.Void && function.FunctionType != nil {
258257
resName = utils.CaseConvert(g.conf.nameCase, function.FunctionType.Ident)
259258
}
260-
// TODO: update doc: oneway/throws/options will be ignored.
259+
// oneway/throws/options will be ignored.
261260
g.writeIndent()
262261
g.protoContent.WriteString(fmt.Sprintf("rpc %s(%s) returns (%s) {}", name, reqName, resName))
263262

@@ -364,10 +363,9 @@ func (g *protoGenerator) handleStruct(s *thrifter.Struct) {
364363
name := utils.CaseConvert(g.conf.nameCase, ele.Ident)
365364

366365
switch ele.FieldType.Type {
367-
// TODO: update doc, set would be list
366+
// set would be list
368367
case thrifter.FIELD_TYPE_LIST, thrifter.FIELD_TYPE_SET:
369368
// TODO: support nested list/set/map
370-
// TODO: update doc
371369
typeNameOrIdent := ""
372370
if ele.FieldType.List.Elem.Type == thrifter.FIELD_TYPE_BASE {
373371
typeNameOrIdent = ele.FieldType.List.Elem.BaseType
@@ -381,8 +379,7 @@ func (g *protoGenerator) handleStruct(s *thrifter.Struct) {
381379

382380
case thrifter.FIELD_TYPE_MAP:
383381
fieldType, keyType := "", ""
384-
// TODO: support nested types
385-
// TODO: update doc
382+
// TODO: support nested types for map value
386383
if ele.FieldType.Map.Value.Type == thrifter.FIELD_TYPE_BASE {
387384
fieldType, _ = g.typeConverter(ele.FieldType.Map.Value.BaseType)
388385
} else {
@@ -446,8 +443,8 @@ func (g *protoGenerator) basicTypeConverter(t string) (res string, err error) {
446443
res = "double"
447444
case "bool":
448445
res = "bool"
449-
case "binary":
450-
res = "bytes"
446+
// case "byte":
447+
// res = "bytes"
451448
default:
452449
err = fmt.Errorf("Invalid basic type %s", t)
453450
}

0 commit comments

Comments
 (0)