addGrpcMetadata=true without NestJS

[jonaskello]

I would like to pass some meta data in my service calls but I am not using NestJS but instead building my own framework. Since the interfaces for service calls do not generate any parameter for metadata this is not really possible with the types generated today. I found the option addGrpcMetadata=true which is exactly what I want but I discovered it does not work unless using NestJS specific codegen. Is there anything stopping this option to be available without the NestJS generation being active? If not I would request it be added for other users too.

[stephenh]

Sure, I've moved the lines in config around so that these should work. Disclaimer I didn't add an integration test for your setup, but it should work. Let me know if not.

[zouyixiong]

Have you publish this feature in 1.76.0 version ? I got some error output.

service DemoService {
    rpc GetBasic (QueryById) returns (BasicInfo) {}
    rpc Insert (Demo) returns (Demo) {}
}

	protoc -I ./proto \
	--plugin=node_modules/ts-proto/protoc-gen-ts_proto \
	--ts_proto_opt=outputEncodeMethods=false,outputJsonMethods=false,outputClientImpl=false,lowerCaseServiceMethods=true,addGrpcMetadata=true, \
	--ts_proto_out=${OUT_DIR} \
    ./proto/*/*.proto

output [ without grpc import in the output file ]

import { util, configure } from 'protobufjs/minimal';
import * as Long from 'long';

export interface DemoService {
   getBasic(request: QueryById,metadata?: Metadata@grpc): Promise<BasicInfo>;
   insert(request: Demo,metadata?: Metadata@grpc): Promise<Demo>;
}

if more infos required, please tell me

[stephenh]

🎉 This issue has been resolved in version 1.78.1 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀

[stephenh]

@zouyixiong thanks for the report and reproduction; you're right the import wasn't correct. That's been fixed in the latest release.

[vojty]

Hello @stephenh , I'm trying to use addGrpcMetadata=true without NestJS as well it doesn't seem to be working. The client types are correct, for example:

export interface MyClient {
...
  ListRegions(
    request: ListRegionsRequest,
    metadata?: Metadata
  ): Promise<ListRegionsResponse>;
...
}

but the actual implementation looks like this:

export class MyClientImpl implements MyClient {
...
  ListRegions(request: ListRegionsRequest): Promise<ListRegionsResponse> {
                                        ^^ ----- metadata are missing here
    const data = ListRegionsRequest.encode(request).finish();
    const promise = this.rpc.request(
      "xxx",
      "ListRegions",
      data
    );
    return promise.then((data) =>
      ListRegionsResponse.decode(new _m0.Reader(data))
    );
  }
...

Shouldn't be the metadata passed to this.rpc.request somehow`?

[stephenh]

@vojty what are you using for ts_proto_out? We have a few different client outputs, so not sure which one to check...

[vojty]

@stephenh I'm using this script:

protoc \
    --plugin=./node_modules/.bin/protoc-gen-ts_proto \
    --proto_path=src \
    --ts_proto_opt=addGrpcMetadata=true \
    --ts_proto_opt=esModuleInterop=true \
    --ts_proto_out=. $(find src -name '*proto')

And this is the usage with @grpc/grpc-js:

import { UnaryCallback } from "@grpc/grpc-js/build/src/client";
import { Client, credentials } from "@grpc/grpc-js";
import {
  MainClientImpl,
} from "./generated";

type RpcImpl = (
  service: string,
  method: string,
  data: Uint8Array
) => Promise<Uint8Array>;

const address = "127.0.0.1:9090";

const c = new Client(address, credentials.createInsecure());

const sendRequest: RpcImpl = (service, method, data) => {
  const path = `${service}/${method}`;
  return new Promise((resolve, reject) => {
    const resultCallback: UnaryCallback<any> = (err, res) => {
      if (err) {
        return reject(err);
      }
      resolve(res);
    };

    const passThrough = (argument: any)  => argument;

    c.makeUnaryRequest(
      path,
      passThrough,
      passThrough,
      data,
      resultCallback
    );
  });
};

const client = new MainClientImpl({ request: sendRequest });
client.ListRegions() // this accepts only ListRegionsRequest

[patgod85]

@stephenh I'm using this script:

protoc \
    --plugin=./node_modules/.bin/protoc-gen-ts_proto \
    --proto_path=src \
    --ts_proto_opt=addGrpcMetadata=true \
    --ts_proto_opt=esModuleInterop=true \
    --ts_proto_out=. $(find src -name '*proto')

And this is the usage with @grpc/grpc-js:

import { UnaryCallback } from "@grpc/grpc-js/build/src/client";
import { Client, credentials } from "@grpc/grpc-js";
import {
  MainClientImpl,
} from "./generated";

type RpcImpl = (
  service: string,
  method: string,
  data: Uint8Array
) => Promise<Uint8Array>;

const address = "127.0.0.1:9090";

const c = new Client(address, credentials.createInsecure());

const sendRequest: RpcImpl = (service, method, data) => {
  const path = `${service}/${method}`;
  return new Promise((resolve, reject) => {
    const resultCallback: UnaryCallback<any> = (err, res) => {
      if (err) {
        return reject(err);
      }
      resolve(res);
    };

    const passThrough = (argument: any)  => argument;

    c.makeUnaryRequest(
      path,
      passThrough,
      passThrough,
      data,
      resultCallback
    );
  });
};

const client = new MainClientImpl({ request: sendRequest });
client.ListRegions() // this accepts only ListRegionsRequest

Have you found the solution for the problem?

[vojty]

@patgod85 rather a workaround than the solution but I'm wrapping the generated client with another object, which holds access to metadata (or rather to a function that generates them)

Boilerplate:

type MetadataGetter = () => Promise<Metadata>;
type SendRequest = (service: string, method: string, data: Uint8Array) => Promise<Uint8Array>;
type Rpc = {
  request: SendRequest;
  // We don't support streaming requests yet
  clientStreamingRequest(
    service: string,
    method: string,
    data: Observable<Uint8Array>,
  ): Promise<Uint8Array>;
  serverStreamingRequest(service: string, method: string, data: Uint8Array): Observable<Uint8Array>;
  bidirectionalStreamingRequest(
    service: string,
    method: string,
    data: Observable<Uint8Array>,
  ): Observable<Uint8Array>;

  setMetadataCallback: (metadataGetter: MetadataGetter) => void;
  getMetadataCallback: () => MetadataGetter;
};

export class GrpcError extends Error {
  constructor(
    message: string,
    public code: number,
    public details: string,
    public metadata: Metadata,
  ) {
    super(message);
    this.name = 'GrpcError';
  }
}

// based on https://github.com/stephenh/ts-proto#basic-grpc-implementation
// we need to expose setMetadata function as well so we won't have to create a new client for each request
const createRpc = (grpcClient: Client, initialMetadata: Metadata): Rpc => {
  let metadataCallback = () => Promise.resolve(initialMetadata);

  const sendRequest: SendRequest = async (service, method, data) => {
    // Conventionally in gRPC, the request path looks like
    // "package.names.ServiceName/MethodName"
    // Must start with a slash! see https://github.com/stephenh/ts-proto/pull/664
    const path = `/${service}/${method}`;
    const metadata = await metadataCallback();

    return new Promise((resolve, reject) => {
      // makeUnaryRequest transmits the result (and error) with a callback
      // transform this into a promise!
      const resultCallback: UnaryCallback<any> = (err, res) => {
        if (err) {
          return reject(new GrpcError(err.message, err.code, err.details, err.metadata));
        }
        resolve(res);
      };

      const passThrough = (value: any) => value;

      // Using passThrough as the serialize and deserialize functions
      grpcClient.makeUnaryRequest(path, passThrough, passThrough, data, metadata, resultCallback);
    });
  };

  return {
    request: sendRequest,
    bidirectionalStreamingRequest: () => {
      throw new Error('bidirectionalStreamingRequest is not supported yet');
    },
    serverStreamingRequest: () => {
      throw new Error('serverStreamingRequest is not supported yet');
    },
    clientStreamingRequest: () => {
      throw new Error('clientStreamingRequest is not supported yet');
    },

    setMetadataCallback: (callback: MetadataGetter) => {
      metadataCallback = callback;
    },
    getMetadataCallback: () => metadataCallback,
  };
};

const createGrpcClient = (address: string) =>
  createRpc(new Client(address, credentials.createInsecure()), new Metadata());

const createService = <T>(impl: { new (rpc: Rpc): T }, address: string) => {
  const rpc = createGrpcClient(address);
  return {
    client: new impl(rpc),
    rpc,
  };
};

Usage:

export const service = createService(MainClientImpl, 'localhost:9001');

// this is populated in an express middleware
service.rpc.setMetadataCallback(...)

// actual GRPC call
service.client.listRegions()

[patgod85]

@vojty I see. Thanks for quick answer!
Unfortunately, this is not available out of the box