Missing import in generated .pb.gw.go file when rpc response message is imported in .proto

[theJenix]

🐛 Bug Report

protoc-gen-grpc-gateway fails to generate necessary imports for response types when a service method returns a message type from a different package/proto file and uses response_body in the gRPC API Configuration YAML.

To Reproduce

1. Create a proto file defining a service in package foo:

// foo.proto
syntax = "proto3";
package foo;

import "bar/bar.proto";

service FooService {
    rpc GetFoo(GetFooRequest) returns (bar.BarResponse);
}

message GetFooRequest {}

2. Create a proto file defining the response type in package bar:

 // bar/bar.proto
 syntax = "proto3";
 package bar;

 message BarResponse {
     repeated string items = 1;
 }

3. Create a gRPC API Configuration YAML file (api_config.yaml):

type: google.api.Service
config_version: 3

http:
  rules:
    - selector: foo.FooService.GetFoo
      get: /v1/foo
      response_body: items

4. Generate gateway code with protoc-gen-grpc-gateway using the API configuration:

protoc -I . \
    --grpc-gateway_out . \
    --grpc-gateway_opt logtostderr=true \
    --grpc-gateway_opt grpc_api_configuration=api_config.yaml \
    foo.proto bar/bar.proto

Expected behavior

The generated foo.pb.gw.go file should include an import for the bar package since bar.BarResponse is referenced in the response type assertion within the XXX_ResponseBody() method:

  import (
      "bar"  // This import should be generated
      // ... other imports
  )

  type response_FooService_GetFoo_0 struct {
      proto.Message
  }

  func (m response_FooService_GetFoo_0) XXX_ResponseBody() interface{} {
      response := m.Message.(*bar.BarResponse)  // Uses bar package
      return response.Items
  }

Actual Behavior

The generated code is missing the import for the bar package, resulting in a compilation error:

foo.pb.gw.go:168:26: undefined: bar

Your Environment

• protoc-gen-grpc-gateway version: v2.11.3 (also present in latest main branch)
• protoc version: libprotoc 3.x+
• Go version: 1.16+
• Operating system: macOS / Linux

[theJenix]

I did a bit of looking into this. The generator function in protoc-gen-grpc-gateway/internal/gengateway/generator.go only checks RequestType.File.GoPkg for imports but never checks ResponseType.File.GoPkg:

  for _, svc := range file.Services {
      for _, m := range svc.Methods {
          imports = append(imports, g.addEnumPathParamImports(file, m, pkgSeen)...)
          pkg := m.RequestType.File.GoPkg  // Only checks request type!
          if len(m.Bindings) == 0 ||
              pkg == file.GoPkg || pkgSeen[pkg.Path] {
              continue
          }
          pkgSeen[pkg.Path] = true
          imports = append(imports, pkg)
      }
  }

The fix requires:

1. Also checking m.ResponseType.File.GoPkg for imports
2. And either only adding response type imports when response_body is specified (since that's when the type is used at compile-time in the generated struct), or changing template.go to replace use of proto.Message with the specific method response type. Otherwise, 1. will generate a lot of unused imports.

I implemented and tested this fix with filtering on response_body in a fork: https://github.com/bommerio/grpc-gateway/tree/2.11.3-fixed-protoc-gen

Happy to submit a PR with this fix if you'd like. The changes are minimal and should be backward-compatible.

[johanbrandhorst]

Thanks for the detailed bug report, please do submit a PR with a fix. Ideally the PR should also include an example proto definition that would have triggered this failure before, as a test against future regressions. You can add it to one of the files in examples/internal/proto. Thanks!

[theJenix]

Sure thing! I will work on it this weekend! Do you have a preference between the localized fix of checking each Binding’s for a response body, or applying the response type more broadly in the generated code template? The former is easier, but the consequence of incompatible edge cases might be generating unused imports (and potentially causing a compiler error in the generated code).

[johanbrandhorst]

I don't have a preference, the proposed fix looks ok!