-
Notifications
You must be signed in to change notification settings - Fork 2.3k
[GSoD 2020] Added tutorials directory on gRPC-Gateway #1829
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 7 commits
Commits
Show all changes
19 commits
Select commit
Hold shift + click to select a range
d577143
added tutorials directory on grpc-gateway
iamrajiv 0179384
refactored badges and links
iamrajiv 0d16f4d
added requested changes
iamrajiv ed7913b
updated tutorials directory on grpc-gateway
iamrajiv 99c4371
updated tutorials directory on grpc-gateway
iamrajiv fa4d264
updated tutorials directory on grpc-gateway
iamrajiv 6b545a8
updated tutorials
iamrajiv 738ebf9
updated tutorials
iamrajiv 9ecaff0
updated tutorials
iamrajiv ccd3c4e
updated tutorials
iamrajiv d873ebe
updated headings and added next button for tutorials
iamrajiv 1e1558a
updated next buttons
iamrajiv 7005e3a
updated tutorials
iamrajiv 7411cca
updated tutorials
iamrajiv 2bf6b0d
updated tutorials
iamrajiv 340994f
updated tutorials
iamrajiv 0848bd0
updated tutorials
iamrajiv 961f085
updated tutorials
iamrajiv 449d352
added helloworld link
iamrajiv File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,7 +1,7 @@ | ||
--- | ||
layout: default | ||
title: FAQ | ||
nav_order: 7 | ||
nav_order: 8 | ||
--- | ||
|
||
# FAQ | ||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,123 @@ | ||
--- | ||
layout: default | ||
title: Adding the grpc-gateway annotations to an existing protobuf file | ||
parent: Tutorials | ||
nav_order: 5 | ||
--- | ||
|
||
## Adding the grpc-gateway annotations to an existing protobuf file | ||
|
||
Now that we've got a working Go gRPC server, we need to add the grpc-gateway annotations. | ||
|
||
The annotations define how gRPC services map to the JSON request and response. When using protocol buffers, each RPC must define the HTTP method and path using the `google.api.http` annotation. | ||
|
||
So you will need to add `import "google/api/http.proto";` to the gRPC proto file. | ||
iamrajiv marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
```proto | ||
syntax = "proto3"; | ||
|
||
package helloworld; | ||
|
||
import "google/api/annotations.proto"; | ||
|
||
// Here is the overall greeting service definition where we define all our endpoints | ||
service Greeter { | ||
// Sends a greeting | ||
rpc SayHello (HelloRequest) returns (HelloReply) { | ||
option (google.api.http) = { | ||
post: "/v1/example/echo" | ||
body: "*" | ||
}; | ||
} | ||
} | ||
|
||
// The request message containing the user's name. | ||
message HelloRequest { | ||
string name = 1; | ||
} | ||
|
||
// The response message containing the greetings | ||
message HelloReply { | ||
string message = 1; | ||
} | ||
``` | ||
|
||
johanbrandhorst marked this conversation as resolved.
Show resolved
Hide resolved
iamrajiv marked this conversation as resolved.
Show resolved
Hide resolved
|
||
Also, See [a_bit_of_everything.proto](https://github.com/grpc-ecosystem/grpc-gateway/blob/master/examples/internal/proto/examplepb/a_bit_of_everything.proto) for examples of more annotations you can add to customize gateway behavior and generated OpenAPI output. | ||
|
||
### Generating the grpc-gateway stubs | ||
|
||
Now that we've got the grpc-gateway annotations added to the proto file, we need to use the grpc-gateway generator to generate the stubs. | ||
|
||
Before we can do that, we need to copy some dependencies into our protofile structure. Copy the `third_party/googleapis` folder from the grpc-gateway repository to your local protofile structure. It should look like this afterwards: | ||
|
||
``` | ||
proto | ||
│ └── api | ||
│ ├── annotations.proto | ||
│ └── http.proto | ||
└── helloworld | ||
└── hello_world.proto | ||
iamrajiv marked this conversation as resolved.
Show resolved
Hide resolved
|
||
``` | ||
|
||
#### Using buf | ||
|
||
We'll need to add the grpc-gateway generator to the generation configuration: | ||
|
||
```yml | ||
plugins: | ||
- name: go | ||
out: proto | ||
opt: paths=source_relative | ||
- name: go-grpc | ||
out: proto | ||
opt: paths=source_relative | ||
- name: grpc-gateway | ||
out: proto | ||
opt: paths=source_relative | ||
``` | ||
|
||
And that's it! Now if you run: | ||
|
||
```sh | ||
buf generate | ||
``` | ||
|
||
It should produce a `*.gw.pb.go` file. | ||
|
||
#### Using protoc | ||
|
||
Now we need to add the grpc-gateway generator to the protoc invocation: | ||
|
||
```sh | ||
protoc -I ./proto \ | ||
--go_out ./proto --go_opt paths=source_relative \ | ||
--go-grpc_out ./proto --go-grpc_opt paths=source_relative \ | ||
--grpc-gateway_out ./proto --grpc-gateway_opt paths=source_relative \ | ||
./proto/helloworld/hello_world.proto | ||
``` | ||
|
||
This should generate a `*.gw.pb.go` file. | ||
|
||
johanbrandhorst marked this conversation as resolved.
Show resolved
Hide resolved
|
||
Usage examples can be found on this [Usage](https://github.com/grpc-ecosystem/grpc-gateway#usage) | ||
iamrajiv marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
For more refer to this boilerplate repository [grpc-gateway-boilerplate | ||
](https://github.com/johanbrandhorst/grpc-gateway-boilerplate) | ||
iamrajiv marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
### Testing the gRPC-Gateway | ||
|
||
Then we use curl to send HTTP requests: | ||
iamrajiv marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
```sh | ||
curl -X POST -k http://localhost:8080/v1/example/echo -d '{"name": " Hello"}' | ||
``` | ||
|
||
``` | ||
{"message":"Hello World"} | ||
``` | ||
|
||
The process is as follows: | ||
iamrajiv marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
`curl` sends a request to the gateway with the post, gateway as proxy forwards the request to greeter_server through grpc, greeter_server returns the result through grpc, the gateway receives the result, and JSON returns to the front end. | ||
|
||
In this way, the transformation process from HTTP JSON to internal grpc is completed through gRPC-Gateway. | ||
iamrajiv marked this conversation as resolved.
Show resolved
Hide resolved
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,56 @@ | ||
--- | ||
layout: default | ||
title: Creating main.go | ||
parent: Tutorials | ||
nav_order: 4 | ||
--- | ||
|
||
## Creating main.go | ||
|
||
Before creating `main.go` file we are assuming that the user has created a `go.mod` with the name `github.com/myuser/myrepo`. Then import here is using the path to the generated files in `proto/helloworld` relative to the root of the repository. | ||
iamrajiv marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
```go | ||
package main | ||
|
||
import ( | ||
"context" | ||
"io/ioutil" | ||
"net" | ||
"os" | ||
|
||
"google.golang.org/grpc" | ||
"google.golang.org/grpc/grpclog" | ||
|
||
helloworldpb "github.com/myuser/myrepo/proto/helloworld/" | ||
iamrajiv marked this conversation as resolved.
Show resolved
Hide resolved
|
||
) | ||
|
||
type server struct{} | ||
|
||
func NewServer() *server { | ||
return &server{} | ||
} | ||
|
||
func (s *server) SayHello(ctx context.Context, in *helloworldpb.HelloRequest) (*helloworldpb.HelloReply, error) { | ||
|
||
iamrajiv marked this conversation as resolved.
Show resolved
Hide resolved
|
||
return &helloworldpb.HelloReply{Message: in.Name + " World"}, nil | ||
} | ||
|
||
func main() { | ||
log := grpclog.NewLoggerV2(os.Stdout, ioutil.Discard, ioutil.Discard) | ||
grpclog.SetLoggerV2(log) | ||
iamrajiv marked this conversation as resolved.
Show resolved
Hide resolved
|
||
lis, err := net.Listen("tcp", ":8080") | ||
johanbrandhorst marked this conversation as resolved.
Show resolved
Hide resolved
|
||
if err != nil { | ||
log.Fatalln("Failed to listen:", err) | ||
} | ||
s := grpc.NewServer() | ||
helloworldpb.RegisterGreeterServer(s, &server{}) | ||
// Serve gRPC Server | ||
log.Info("Serving gRPC on 0.0.0.0:8080") | ||
log.Fatal(s.Serve(lis)) | ||
} | ||
|
||
``` | ||
|
||
### Read More | ||
|
||
For more refer to gRPC docs [https://grpc.io/docs/languages/go/](https://grpc.io/docs/languages/go/). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
--- | ||
layout: default | ||
title: Generating stubs | ||
parent: Tutorials | ||
nav_order: 3 | ||
has_children: true | ||
--- | ||
iamrajiv marked this conversation as resolved.
Show resolved
Hide resolved
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,46 @@ | ||
--- | ||
layout: default | ||
title: Generating stubs using buf | ||
parent: Generating stubs | ||
grand_parent: Tutorials | ||
nav_order: 1 | ||
--- | ||
|
||
## Generating stubs using buf | ||
|
||
[Buf](https://github.com/bufbuild/buf) is a tool that provides various protobuf utilities such as linting, breaking change detection and generation. Please find installation instructions on https://github.com/bufbuild/buf. | ||
|
||
It is configured through a `buf.yaml` file that should be checked in to the root of your repository. Buf will automatically read this file if present. Configuration can also be provided via the command-line flag `--config`, which accepts a path to a `.json` or `.yaml` file, or direct JSON or YAML data. | ||
|
||
All Buf operations that use your local `.proto` files as input rely on a valid build configuration. This configuration tells Buf where to search for `.proto` files, and how to handle imports. As opposed to protoc, where all `.proto` files are manually specified on the command-line, buf operates by recursively discovering all `.proto` files under configuration and building them. | ||
iamrajiv marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
The following is an example of a valid configuration, assuming you have your `.proto` files rooted in the `proto` folder relative to the root of your repository. | ||
|
||
```yml | ||
version: v1beta1 | ||
build: | ||
roots: | ||
- proto | ||
``` | ||
|
||
To generate type and gRPC stubs for Go, create the file `buf.gen.yaml` at the root of the repository: | ||
|
||
```yml | ||
plugins: | ||
- name: go | ||
out: proto | ||
opt: paths=source_relative | ||
- name: go-grpc | ||
out: proto | ||
opt: paths=source_relative | ||
``` | ||
|
||
We use the `go` and `go-grpc` plugins to generate Go types and gRPC service definitions. We're outputting the generated files relative to the `proto` folder, and we're using the `paths=source_relative` option, which means that the generated files will appear in the same directory as the source `.proto` file. | ||
|
||
Then run | ||
|
||
```sh | ||
buf generate | ||
``` | ||
|
||
This will have generated a `*.pb.go` and a `*_grpc.pb.go` file for each protobuf package in our `proto` file hierarchy. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
--- | ||
layout: default | ||
title: Generating stubs using protoc | ||
parent: Generating stubs | ||
grand_parent: Tutorials | ||
nav_order: 2 | ||
--- | ||
|
||
## Generating stubs using protoc | ||
|
||
Here's an example of what a `protoc` command might look like to generate Go stubs, assuming that you're at the root of your repository and you have your proto files in a directory called `proto`: | ||
|
||
```sh | ||
protoc -I ./proto \ | ||
--go_out ./proto --go_opt paths=source_relative \ | ||
--go-grpc_out ./proto --go-grpc_opt paths=source_relative \ | ||
./proto/helloworld/hello_world.proto | ||
``` | ||
|
||
We use the `go` and `go-grpc` plugins to generate Go types and gRPC service definitions. We're outputting the generated files relative to the `proto` folder, and we're using the `paths=source_relative` option, which means that the generated files will appear in the same directory as the source `.proto` file. | ||
|
||
This will have generated a `*.pb.go` and a `*_grpc.pb.go` file for each protobuf package in our `proto` file hierarchy. | ||
iamrajiv marked this conversation as resolved.
Show resolved
Hide resolved
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
--- | ||
layout: default | ||
title: Tutorials | ||
nav_order: 7 | ||
has_children: true | ||
--- |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
--- | ||
layout: default | ||
title: Introduction about gRPC-Gateway | ||
iamrajiv marked this conversation as resolved.
Show resolved
Hide resolved
|
||
parent: Tutorials | ||
nav_order: 1 | ||
--- | ||
|
||
## Introduction | ||
|
||
We all know that gRPC is not a tool for everything. There are cases where we still want to provide a traditional RESTful JSON API. The reasons can range from maintaining backwards-compatibility to supporting programming languages or clients not well supported by gRPC. But coding another API for REST is quite a time consuming and tedious. | ||
iamrajiv marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
So is there any way to code just once, but can provide APIs in both gRPC and REST at the same time? | ||
iamrajiv marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
The answer is Yes. | ||
|
||
The grpc-gateway is a plugin of the Google protocol buffers compiler [protoc](https://github.com/protocolbuffers/protobuf). It reads protobuf service definitions and generates a reverse-proxy server which translates a RESTful HTTP API into gRPC. This server is generated according to the [`google.api.http`](https://github.com/googleapis/googleapis/blob/master/google/api/http.proto#L46) annotations in your service definitions. | ||
|
||
This helps you provide your APIs in both gRPC and RESTful style at the same time. | ||
iamrajiv marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
 | ||
|
||
### Prerequisites | ||
|
||
Before we start coding, we have to install some tools and need to do a small amount of configuration in the service. | ||
iamrajiv marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
We will be using a Go gRPC server in the examples, so please install Go first from [https://golang.org/dl/](https://golang.org/dl/). | ||
|
||
After installing Go, use `go get` to download the following packages: | ||
|
||
```sh | ||
go get github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-grpc-gateway | ||
go get google.golang.org/protobuf/cmd/protoc-gen-go | ||
go get google.golang.org/grpc/cmd/protoc-gen-go-grpc | ||
``` | ||
|
||
This installs the protoc generator plugins we need to generate the stubs. Make sure to add `$GOPATH/bin` to your `$PATH` so that executables installed via `go get` are available on your `$PATH`. | ||
iamrajiv marked this conversation as resolved.
Show resolved
Hide resolved
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
--- | ||
layout: default | ||
title: Learn More | ||
parent: Tutorials | ||
nav_order: 6 | ||
--- | ||
|
||
## Learn More | ||
|
||
#### google.api.http | ||
|
||
GRPC transcoding is a conversion function between the gRPC method and one or more HTTP REST endpoints. This allows developers to create a single API service that supports both the gRPC API and the REST API. Many systems, including the API Google, Cloud Endpoints, gRPC Gateway, and the Envoy proxy server support this feature and use it for large-scale production services. | ||
|
||
The grcp-gateway the server is created according to the `google.api.http` annotations in your service definitions. | ||
|
||
HttpRule defines the gRPC / REST mapping scheme. The mapping defines how different parts of a gRPC request message are mapped to the URL path, URL request parameters, and HTTP request body. It also controls how the gRPC response message is displayed in the HTTP response body. HttpRule is usually specified as a `google.api.http` annotation in the gRPC method. | ||
|
||
Each mapping defines a URL path template and an HTTP method. A path template can refer to one or more fields in a gRPC request message if each field is a non-repeating field with a primitive type. The path template controls how the request message fields are mapped to the URL path. | ||
|
||
Read more about HTTP and gRPC Transcoding on https://google.aip.dev/127. | ||
iamrajiv marked this conversation as resolved.
Show resolved
Hide resolved
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
--- | ||
layout: default | ||
title: Creating a simple hello world with gRPC | ||
parent: Tutorials | ||
nav_order: 2 | ||
--- | ||
|
||
## Creating a simple hello world with gRPC | ||
|
||
To understand the gRPC-Gateway we are going to first make a hello world gRPC service. | ||
|
||
### Defining your gRPC service using protocol buffers | ||
|
||
Before we create a gRPC service, we should create a proto file to define what we need, here we create a file named `hello_world.proto` in the directory `proto/helloworld/hello_world.proto`. | ||
|
||
```proto | ||
syntax = "proto3"; | ||
|
||
package helloworld; | ||
|
||
johanbrandhorst marked this conversation as resolved.
Show resolved
Hide resolved
|
||
// The request message containing the user's name. | ||
message HelloRequest { | ||
string name = 1; | ||
} | ||
|
||
// The response message containing the greetings | ||
message HelloReply { | ||
string message = 1; | ||
} | ||
``` | ||
iamrajiv marked this conversation as resolved.
Show resolved
Hide resolved
iamrajiv marked this conversation as resolved.
Show resolved
Hide resolved
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.