Merging swagger specs fails to use rpc comments

[lukasmalkmus]

I have two proto files:

a.proto:

syntax = "proto3";

package api.v1;

import "google/api/annotations.proto";

// A is a service for handling Foo stuff.
service A {
  // List all Foos
  //
  // Returns a (possibly paginated) list of all foo resources on success.
  rpc ListFoos (ListFoosRequest) returns (ListFoosResponse) {
    option (google.api.http) = { get: "/v1/foos" };
  }
}

// The ListFoos request message.
message ListFoosRequest {}

// The ListFoos response message.
message ListFoosResponse {}

b.proto:

syntax = "proto3";

package api.v1;

import "google/api/annotations.proto";

// B is a service for handling Bar stuff.
service B {
  // List all Bars
  //
  // Returns a (possibly paginated) list of all bar resources on success.
  rpc ListBars (ListBarsRequest) returns (ListBarsResponse) {
    option (google.api.http) = { get: "/v1/bars" };
  }
}

// The ListBars request message.
message ListBarsRequest {}

// The ListBars response message.
message ListBarsResponse {}

I tried to create a single swagger spec by using the allow_merge option. However, the generator removes some of the protobuf comments which get transformed into the endpoint description.

Steps you follow to reproduce the error:

1. Grab the provided protobuf file a.proto and b.proto
2. Generate the swagger spec without merging:

   protoc -I=. \
           -I=$GOPATH/src/github.com/grpc-ecosystem/grpc-gateway \
           -I=$GOPATH/src/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis \
           --swagger_out=logtostderr=true:. \
           a.proto b.proto

3. Generate the swagger spec with merging:

   protoc -I=. \
           -I=$GOPATH/src/github.com/grpc-ecosystem/grpc-gateway \
           -I=$GOPATH/src/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis \
           --swagger_out=logtostderr=true,allow_merge=true:. \
           a.proto b.proto

What did you expect to happen instead:

Take a look at the separate specifications, the protobuf comments got transformed into endpoint descriptions:

a.swagger.json:

{
  "swagger": "2.0",
  "info": {
    "title": "a.proto",
    "version": "version not set"
  },
  "schemes": [
    "http",
    "https"
  ],
  "consumes": [
    "application/json"
  ],
  "produces": [
    "application/json"
  ],
  "paths": {
    "/v1/foos": {
      "get": {
        "summary": "List all Foos",
        "description": "Returns a (possibly paginated) list of all foo resources on success.",
        "operationId": "ListFoos",
        "responses": {
          "200": {
            "description": "",
            "schema": {
              "$ref": "#/definitions/v1ListFoosResponse"
            }
          }
        },
        "tags": [
          "A"
        ]
      }
    }
  },
  "definitions": {
    "v1ListFoosResponse": {
      "type": "object",
      "description": "The ListFoos response message."
    }
  }
}

b.swagger.json:

{
  "swagger": "2.0",
  "info": {
    "title": "b.proto",
    "version": "version not set"
  },
  "schemes": [
    "http",
    "https"
  ],
  "consumes": [
    "application/json"
  ],
  "produces": [
    "application/json"
  ],
  "paths": {
    "/v1/bars": {
      "get": {
        "summary": "List all Bars",
        "description": "Returns a (possibly paginated) list of all bar resources on success.",
        "operationId": "ListBars",
        "responses": {
          "200": {
            "description": "",
            "schema": {
              "$ref": "#/definitions/v1ListBarsResponse"
            }
          }
        },
        "tags": [
          "B"
        ]
      }
    }
  },
  "definitions": {
    "v1ListBarsResponse": {
      "type": "object",
      "description": "The ListBars response message."
    }
  }
}

This isn't true for the merged specification:

apidocs.swagger.json:

{
  "swagger": "2.0",
  "info": {
    "title": "a.proto",
    "version": "version not set"
  },
  "schemes": [
    "http",
    "https"
  ],
  "consumes": [
    "application/json"
  ],
  "produces": [
    "application/json"
  ],
  "paths": {
    "/v1/bars": {
      "get": {
        "operationId": "ListBars",
        "responses": {
          "200": {
            "description": "",
            "schema": {
              "$ref": "#/definitions/v1ListBarsResponse"
            }
          }
        },
        "tags": [
          "B"
        ]
      }
    },
    "/v1/foos": {
      "get": {
        "summary": "List all Foos",
        "description": "Returns a (possibly paginated) list of all foo resources on success.",
        "operationId": "ListFoos",
        "responses": {
          "200": {
            "description": "",
            "schema": {
              "$ref": "#/definitions/v1ListFoosResponse"
            }
          }
        },
        "tags": [
          "A"
        ]
      }
    }
  },
  "definitions": {
    "v1ListBarsResponse": {
      "type": "object",
      "description": "The ListBars response message."
    },
    "v1ListFoosResponse": {
      "type": "object",
      "description": "The ListFoos response message."
    }
  }
}

This spec misses the description of the /v1/bars endpoint.

What's your theory on why it isn't working:

There seems to be a bug in the code which merges the two specs. I this its located around here:

grpc-gateway/protoc-gen-swagger/genswagger/generator.go

Lines 93 to 97 in a5b66c1

	} else {
	mergedTarget.Enums = append(mergedTarget.Enums, f.Enums...)
	mergedTarget.Messages = append(mergedTarget.Messages, f.Messages...)
	mergedTarget.Services = append(mergedTarget.Services, f.Services...)
	}

If merged to the leader, the code does not copy the comments.

[emcfarlane]

Have found the same issue.

[lukasmalkmus]

Looks like this was fixed in latest master with 11bef10. Can you confirm @afking ?

[emcfarlane]

@lukasmalkmus fixed, thanks!

[princejha95]

I have a proto file which contains two services. The comments of the rpc methods present in second service gets copied from first service on swagger UI. Can somebody please tell me how to fix it ?

[johanbrandhorst]

@princejha95 That sounds like a bug, have you got your protofile and generated swagger file so we can see if there's an error?

[princejha95]

syntax = "proto3";

import "google/api/annotations.proto";

package test;

service SampleService1{
    // Say Hello
    rpc SayHello (HelloRequest) returns (HelloResponse){
        option (google.api.http) = {
            get : "/api/hello"
        };
    }
}

message HelloRequest{
    // Hello message
    string msg = 1;
}

message HelloResponse{
    //Hello response
    string res = 1;
}

service SampleService2{
    //Say Bye
    rpc SayBye (ByeRequest) returns (ByeResponse){
        option (google.api.http) = {
            get : "/api/bye"
        };
    }
}

message ByeRequest{
    //Bye message
    string msg = 1;
}

message ByeResponse{
    //Bye response
    string res = 1;
}

This is the proto file.

[princejha95]

{
    "paths": {
        "/api/bye": {
            "get": {
                "tags": [
                    "SampleService2"
                ],
                "operationId": "SayBye",
                "responses": {
                    "200": {
                        "description": "",
                        "schema": {
                            "$ref": "#/definitions/testByeResponse"
                        }
                    },
                    "404": {
                        "description": "Not Found",
                        "schema": {}
                    }
                },
                "parameters": [
                    {
                        "required": false,
                        "type": "string",
                        "name": "msg",
                        "description": "Bye message."
                    }
                ],
                "summary": "Say Hello"
            }
        },
        "/api/hello": {
            "get": {
                "tags": [
                    "SampleService1"
                ],
                "operationId": "SayHello",
                "responses": {
                    "200": {
                        "description": "",
                        "schema": {
                            "$ref": "#/definitions/testHelloResponse"
                        }
                    },
                    "404": {
                        "description": "Not Found",
                        "schema": {}
                    }
                },
                "parameters": [
                    {
                        "required": false,
                        "type": "string",
                        "name": "msg",
                        "description": "Hello message."
                    }
                ],
                "summary": "Say Hello"
            }
        }
    },
    "schemes": [
        "http",
        "https"
    ],
    "tags": [
        {
            "name": "SampleService2",
            "description": "Description"
        },
        {
            "name": "SampleService1",
            "description": "Description"
        }
    ],
    "basePath": "/",
    "host": "localhost:3000",
    "definitions": {
        "testHelloResponse": {
            "type": "object",
            "properties": {
                "res": {
                    "type": "string",
                    "title": "Hello response"
                }
            }
        },
        "testByeResponse": {
            "type": "object",
            "properties": {
                "res": {
                    "type": "string",
                    "title": "Bye response"
                }
            }
        }
    },
    "swagger": "2.0"
}

This is the generated swagger json file.

[johanbrandhorst]

Yeah, that looks wrong. Could you open a separate issue with a minimal protofile which reproduces the error? I think something like the following should do:

syntax = "proto3";

import "google/api/annotations.proto";

service SampleService1{
    // Comment 1
    rpc Method (Empty) returns (Empty){
        option (google.api.http) = {
            get : "/api/method"
        };
    }
}

service SampleService2{
    // Comment 2
    rpc Method (Empty) returns (Empty){
        option (google.api.http) = {
            get : "/api/method"
        };
    }
}

message Empty {}

[princejha95]

sure

[princejha95]

I have raised a new issue. Here's the link:

#746

[bsakweson]

I ran into an issue similar to this, would love to open a new issue if needed but thought I should start here since they related.

I have a setup with multiple micro services each defined in their own .proto file. Code generation works well until I tried to combine my openAPI documentation using protoc-gen-swagger from grpc-gateway.

Using this command protoc --proto_path=api/v1 --proto_path=$GOPATH/src --proto_path=third_party --swagger_out=logtostderr=true,allow_merge=true:third_party/OpenAPI a/a.proto b/b.proto to generate a merged apidocs.swagger.json file containing all services defined in a and b respectively so that I can serve it as a single page throws this error --swagger_out: inconsistent package names: a b . Notice that a and b are all in different packages a and b.

service a:

syntax = "proto3";

package com.somecompany.a;

option go_package = "a";

import "google/api/annotations.proto";

// A is a service for handling Foo stuff.
service A {
  // List all Foos
  //
  // Returns a (possibly paginated) list of all foo resources on success.
  rpc ListFoos (ListFoosRequest) returns (ListFoosResponse) {
    option (google.api.http) = { get: "/v1/foos" };
  }
}

// The ListFoos request message.
message ListFoosRequest {}

// The ListFoos response message.
message ListFoosResponse {}

service b:

syntax = "proto3";

package com.somecompany.b;

option go_package = "b";
import "google/api/annotations.proto";

// B is a service for handling Bar stuff.
service B {
  // List all Bars
  //
  // Returns a (possibly paginated) list of all bar resources on success.
  rpc ListBars (ListBarsRequest) returns (ListBarsResponse) {
    option (google.api.http) = { get: "/v1/bars" };
  }
}

// The ListBars request message.
message ListBarsRequest {}

// The ListBars response message.
message ListBarsResponse {}

[johanbrandhorst]

Hi @bsakweson. Unfortunately, allow_merge will almost certainly only work within protofiles from the sake package. Yours would be a feature request, as I believe it is working as intended here. Would you like to open another issue to track this?