Skip to content

Fail when expected files are not produced by protoc #4287

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 3, 2025

Conversation

linzhp
Copy link
Contributor

@linzhp linzhp commented Mar 2, 2025

What type of PR is this?
Bug fix

What does this PR do? Why is it needed?
This PR fail the action if a plugin does not produce a valid Go file that it is supposed to produce. This disallows people to add gRPC plugin to a go_proto_library target that has no service definition, but avoids the cache poisoning due to bugs in proto plugins. Because Gazelle is able to switch between proto and grpc plugins depending on the existence of service definition, existing repos managed by Gazelle will not be affected.

Which issues(s) does this PR fix?

Fixes #3949

Other notes for review
Credits to @r-hang for coming up with the patch

@linzhp linzhp requested a review from jayconrod March 2, 2025 16:42
@linzhp linzhp merged commit 077f15f into bazel-contrib:master Mar 3, 2025
1 check passed
@linzhp linzhp deleted the protoc branch March 3, 2025 18:34
sluongng added a commit to buildbuddy-io/buildbuddy that referenced this pull request Apr 16, 2025
rules_go upgraded org_golang_google_grpc_cmd_protoc_gen_go_grpc which
triggered a version bump on the rest of GCP SDK.

Because of bazel-contrib/rules_go#4287, trim
unused go proto compilers in proto/BUILD targets.
@sluongng
Copy link
Contributor

This is a bit too strict.

We have several use cases where a proto_library target would contain both message and service proto files inside srcs attribute. Then the go_proto_library target would wrap that proto_library target with both the regular compiler and the grpc compiler.

https://github.com/buildbuddy-io/buildbuddy/blob/87810234ab7ec1be168288d078a0d718d8e6ff25/proto/api/v1/BUILD#L36-L70

OR, a gazelle-generated BUILD like this in one of our dependencies

load("@io_bazel_rules_go//go:def.bzl", "go_library")
load("@io_bazel_rules_go//proto:def.bzl", "go_proto_library")
load("@rules_proto//proto:defs.bzl", "proto_library")

proto_library(
    name = "apis_proto",
    srcs = [
        "classification.proto",
        "get_model_metadata.proto",
        "inference.proto",
        "input.proto",
        "model.proto",
        "predict.proto",
        "prediction_service.proto", # service proto
        "regression.proto",
    ],
    visibility = ["//visibility:public"],
    deps = [
        "//tensorflow/core/example:example_proto",
        "//tensorflow/core/framework:framework_proto",
        "//tensorflow/core/protobuf:protobuf_proto",
        "@com_google_protobuf//:any_proto",
        "@com_google_protobuf//:wrappers_proto",
    ],
)

go_proto_library(
    name = "apis_go_proto",
    compilers = ["@io_bazel_rules_go//proto:go_grpc_v2"],
    importpath = "github.com/buildbuddy-io/tensorflow-proto/tensorflow_serving/apis",
    proto = ":apis_proto",
    visibility = ["//visibility:public"],
    deps = [
        "//tensorflow/core/example",
        "//tensorflow/core/framework",
        "//tensorflow/core/protobuf",
    ],
)

go_library(
    name = "apis",
    embed = [":apis_go_proto"],
    importpath = "github.com/buildbuddy-io/tensorflow-proto/tensorflow_serving/apis",
    visibility = ["//visibility:public"],
)

alias(
    name = "go_default_library",
    actual = ":apis",
    visibility = ["//visibility:public"],
)

@Moersity
Copy link

This is a bit too strict.

We have several use cases where a proto_library target would contain both message and service proto files inside srcs attribute. Then the go_proto_library target would wrap that proto_library target with both the regular compiler and the grpc compiler.

https://github.com/buildbuddy-io/buildbuddy/blob/87810234ab7ec1be168288d078a0d718d8e6ff25/proto/api/v1/BUILD#L36-L70

OR, a gazelle-generated BUILD like this in one of our dependencies

load("@io_bazel_rules_go//go:def.bzl", "go_library")
load("@io_bazel_rules_go//proto:def.bzl", "go_proto_library")
load("@rules_proto//proto:defs.bzl", "proto_library")

proto_library(
    name = "apis_proto",
    srcs = [
        "classification.proto",
        "get_model_metadata.proto",
        "inference.proto",
        "input.proto",
        "model.proto",
        "predict.proto",
        "prediction_service.proto", # service proto
        "regression.proto",
    ],
    visibility = ["//visibility:public"],
    deps = [
        "//tensorflow/core/example:example_proto",
        "//tensorflow/core/framework:framework_proto",
        "//tensorflow/core/protobuf:protobuf_proto",
        "@com_google_protobuf//:any_proto",
        "@com_google_protobuf//:wrappers_proto",
    ],
)

go_proto_library(
    name = "apis_go_proto",
    compilers = ["@io_bazel_rules_go//proto:go_grpc_v2"],
    importpath = "github.com/buildbuddy-io/tensorflow-proto/tensorflow_serving/apis",
    proto = ":apis_proto",
    visibility = ["//visibility:public"],
    deps = [
        "//tensorflow/core/example",
        "//tensorflow/core/framework",
        "//tensorflow/core/protobuf",
    ],
)

go_library(
    name = "apis",
    embed = [":apis_go_proto"],
    importpath = "github.com/buildbuddy-io/tensorflow-proto/tensorflow_serving/apis",
    visibility = ["//visibility:public"],
)

alias(
    name = "go_default_library",
    actual = ":apis",
    visibility = ["//visibility:public"],
)

We encountered the same problem, so what should we do now? downgrage rules_go version seem likes a bad idea

@linzhp
Copy link
Contributor Author

linzhp commented Apr 17, 2025

We should revert this PR

linzhp added a commit to linzhp/rules_go that referenced this pull request Apr 19, 2025
fmeum pushed a commit to linzhp/rules_go that referenced this pull request Apr 19, 2025
linzhp added a commit that referenced this pull request Apr 19, 2025
…4324)

**What type of PR is this?**
Bug fix


**What does this PR do? Why is it needed?**
It's common for a `go_proto_library` to include proto files with and
without service definitions at the same time. In this case, the gRPC
plugins are needed, but some gRPC plugins don't generate the grpc.pb.go
files if there is no service definition.

Partially reverting #4287 to restore the previous behavior of creating
an empty file

**Which issues(s) does this PR fix?**

Fixes #4317

**Other notes for review**

---------

Co-authored-by: Fabian Meumertzheim <[email protected]>
linzhp added a commit that referenced this pull request May 20, 2025
…4324)

**What type of PR is this?**
Bug fix


**What does this PR do? Why is it needed?**
It's common for a `go_proto_library` to include proto files with and
without service definitions at the same time. In this case, the gRPC
plugins are needed, but some gRPC plugins don't generate the grpc.pb.go
files if there is no service definition.

Partially reverting #4287 to restore the previous behavior of creating
an empty file

**Which issues(s) does this PR fix?**

Fixes #4317

**Other notes for review**

---------

Co-authored-by: Fabian Meumertzheim <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Proposal: Fail protoc code gen on missing expected output files
5 participants