Coder Social home page Coder Social logo

grpc-proxy's Introduction

gRPC Proxy

Travis Build Go Report Card Go Reference Apache 2.0 License

gRPC Go Proxy server

Project Goal

Build a transparent reverse proxy for gRPC targets that will make it easy to expose gRPC services over the internet. This includes:

  • no needed knowledge of the semantics of requests exchanged in the call (independent rollouts)
  • easy, declarative definition of backends and their mappings to frontends
  • simple round-robin load balancing of inbound requests from a single connection to multiple backends

The project now exists as a proof of concept, with the key piece being the proxy package that is a generic gRPC reverse proxy handler.

Proxy Handler

The package proxy contains a generic gRPC reverse proxy handler that allows a gRPC server to not know about registered handlers or their data types. Please consult the docs, here's an exaple usage.

You can call proxy.NewProxy to create a *grpc.Server that proxies requests.

proxy := proxy.NewProxy(clientConn)

More advanced users will want to define a StreamDirector that can make more complex decisions on what to do with the request.

director = func(ctx context.Context, fullMethodName string) (context.Context, *grpc.ClientConn, error) {
    md, _ := metadata.FromIncomingContext(ctx)
    outCtx = metadata.NewOutgoingContext(ctx, md.Copy())
    return outCtx, cc, nil
	
    // Make sure we never forward internal services.
    if strings.HasPrefix(fullMethodName, "/com.example.internal.") {
        return outCtx, nil, status.Errorf(codes.Unimplemented, "Unknown method")
    }
    
    if ok {
        // Decide on which backend to dial
        if val, exists := md[":authority"]; exists && val[0] == "staging.api.example.com" {
            // Make sure we use DialContext so the dialing can be cancelled/time out together with the context.
            return outCtx, grpc.DialContext(ctx, "api-service.staging.svc.local", grpc.WithCodec(proxy.Codec())), nil
        } else if val, exists := md[":authority"]; exists && val[0] == "api.example.com" {
            return outCtx, grpc.DialContext(ctx, "api-service.prod.svc.local", grpc.WithCodec(proxy.Codec())), nil
        }
    }
    return outCtx, nil, status.Errorf(codes.Unimplemented, "Unknown method")
}

Then you need to register it with a grpc.Server. The server may have other handlers that will be served locally.

server := grpc.NewServer(
    grpc.CustomCodec(proxy.Codec()),
    grpc.UnknownServiceHandler(proxy.TransparentHandler(director)))
pb_test.RegisterTestServiceServer(server, &testImpl{})

Testing

To make debugging a bit simpler, there are some helpers.

testservice contains a method TestTestServiceServerImpl which performs a complete test against the reference implementation of the TestServiceServer.

In proxy_test.go, the test framework spins up a TestServiceServer that it tests the proxy against. To make debugging a bit simpler (eg. if the developer needs to step into google.golang.org/grpc methods), this TestServiceServer can be provided by a server by passing -test-backend=addr to go test. A simple, local-only implementation of TestServiceServer exists in testservice/server.

License

grpc-proxy is released under the Apache 2.0 license. See LICENSE.txt.

grpc-proxy's People

Contributors

adamthesax avatar jackwink avatar marcusirgens avatar rubensf avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

grpc-proxy's Issues

Status update?

This project looks like it's gone dormant. Can we get some type of project status indicator?

Allow for Director to set OutgoingMetadata on proxied calls

As mentioned in
improbable-eng/grpc-web#91
and in discussion of #16 (@peteredge's comment) we need a way to add metadata to outgoing calls.

The improbable-eng/grpc-web#91 broke because of gRPC-Go changes in 1.5 that separated the inbound and output contexts. This meant that it no longer forwarded the metadata.

I propose the following change:

type StreamDirector func(ctx context.Context, fullMethodName string) (*grpc.ClientConn, error)

to

type StreamDirector func(ctx context.Context, fullMethodName string) (context.Context, *grpc.ClientConn, error)

Where the returned context will be used as a base for the call.

Thoughts? @yifanzz

grpc reverse proxy

Hi,

I would like to get more information about the grpc-proxy you built. can it also be used for Tensorflow serving (it uses grpc request / response)? I want to read the header and re-write it and forward it to to TensorFlow Serving and read the response and forward it to the grpc client.

grpc client <-> grpc proxy <-> TensorFlow Serving

Thanks.

Fields marked as `repeated` are truncated

While proxying messages that contain repeated fields, the values are truncated to only their last value. This happens both on incoming messages and the outgoing response. I created a simple test service that exhibits this behavior. Given a proto looks like:

service Greeter {
  rpc SayHelloAgain (HelloRequest) returns (RepeatedHelloReply) {}
}

// The request message containing the user's name.
message HelloRequest {
  string name = 1;
}

message RepeatedHelloReply {
  repeated string message = 1;
}

When I hit the gRPC service directly, I get an array of strings back:

$ grpcurl -plaintext -d '{"name": "Joe"}' localhost:50051 Greeter/SayHelloAgain
{
  "message": [
    "1: Hello Joe",
    "2: Hello Joe",
    "3: Hello Joe",
    "4: Hello Joe",
    "5: Hello Joe"
  ]
}

However making that same call via the proxy I get a truncated response:

grpcurl -plaintext -d '{"name": "Joe"}' localhost:50052 Greeter/SayHelloAgain
{
  "message": [
    "5: Hello Joe"
  ]
}

I've attached my example service that exhibits the behavior. The basic gRPC server runs on :50051 while the proxy runs on 50052. Use make run to start the service, and make test to hit the service to reproduce the issue.
grpc-proxy-test.zip

Thanks in advanced for your assistance!

middleware in grpc_proxy

hi,
I hava used "github.com/grpc-ecosystem/go-grpc-middleware" in my grpc-server,it's very good,when I want to use grpc_proxy,how can I add "github.com/grpc-ecosystem/go-grpc-middleware" to grpc_proxy.
thank you very much.

panic: interface conversion: *proxy.frame is not proto.Message: missing method ProtoMessage

I have a project relying on https://github.com/improbable-eng/grpc-web/, specifically:

https://github.com/improbable-eng/grpc-web/go/grpcwebproxy

As of today, I can no longer make calls through the proxy, I get this error stack:

proxy_1    | panic: interface conversion: *proxy.frame is not proto.Message: missing method ProtoMessage
proxy_1    | 
proxy_1    | goroutine 47 [running]:
proxy_1    | google.golang.org/grpc/encoding/proto.marshal(0x923e20, 0xc0001a8400, 0xc000145980, 0x0, 0x0, 0x40c100, 0xc000145950, 0x30)
proxy_1    | 	/go/pkg/mod/google.golang.org/[email protected]/encoding/proto/proto.go:54 +0x45
proxy_1    | google.golang.org/grpc/encoding/proto.codec.Marshal(0x923e20, 0xc0001a8400, 0x203000, 0x0, 0x0, 0x203000, 0x203000)
proxy_1    | 	/go/pkg/mod/google.golang.org/[email protected]/encoding/proto/proto.go:74 +0xad
proxy_1    | google.golang.org/grpc.encode(0x7fc8845365d8, 0xee0970, 0x923e20, 0xc0001a8400, 0x0, 0xec4b80, 0x7fc88679e008, 0x0, 0x0)
proxy_1    | 	/go/pkg/mod/google.golang.org/[email protected]/rpc_util.go:543 +0x52
proxy_1    | google.golang.org/grpc.prepareMsg(0x923e20, 0xc0001a8400, 0x7fc8845365d8, 0xee0970, 0x0, 0x0, 0x0, 0x0, 0x3, 0x40902b, ...)
proxy_1    | 	/go/pkg/mod/google.golang.org/[email protected]/stream.go:1519 +0x85
proxy_1    | google.golang.org/grpc.(*clientStream).SendMsg(0xc000089b00, 0x923e20, 0xc0001a8400, 0x0, 0x0)
proxy_1    | 	/go/pkg/mod/google.golang.org/[email protected]/stream.go:699 +0x169
proxy_1    | github.com/mwitkow/grpc-proxy/proxy.(*handler).forwardServerToClient.func1(0xafc6c0, 0xc0001a8220, 0xc0001884e0, 0xafc7e0, 0xc000089b00)
proxy_1    | 	/go/pkg/mod/github.com/mwitkow/[email protected]/proxy/handler.go:155 +0xa0
proxy_1    | created by github.com/mwitkow/grpc-proxy/proxy.(*handler).forwardServerToClient
proxy_1    | 	/go/pkg/mod/github.com/mwitkow/[email protected]/proxy/handler.go:148 +0x89

How can I go about debugging why this is happening?

I'm using this service / proto message:

service AuthService {
  rpc GetToken(AuthData) returns (Token) {
    option (google.api.http) = {
      post: "/v1/auth"
      body: "*"
    };
  }
}

message AuthData {
  string username = 1;
  string password = 2;
}

the Go output does have a method called ProtoMessage, the TypeScript output does not but even if I manually add this method to generated TypeScript the error is the exact same.

As far as I can tell no one is having this error but me, but I don't seem to have any ability to influence or debug it. Any sort of direction or suggestions are greatly appreciated thank you

proxy pass header metadata

hi,
Metadata can't pass client to server by grpc-proxy, what can I do to pass metadata , thanks a lot

ExampleStreamDirector: context.WithCancel() and no cancel?

go vet warns about ignoring the cancel function returned by context.WithCancel():

func ExampleStreamDirector() {
	director = func(ctx context.Context, fullMethodName string) (context.Context, *grpc.ClientConn, error) {
		// Make sure we never forward internal services.
		if strings.HasPrefix(fullMethodName, "/com.example.internal.") {
			return nil, nil, grpc.Errorf(codes.Unimplemented, "Unknown method")
		}
		md, ok := metadata.FromIncomingContext(ctx)
		// Copy the inbound metadata explicitly.
		outCtx, _ := context.WithCancel(ctx)
                outCtx = metadata.NewOutgoingContext(outCtx, md.Copy())

https://stackoverflow.com/questions/44393995/what-happens-if-i-dont-cancel-a-context explains that this leads to a resource leak.

What is the purpose of creating two contexts (first with context.WithCancel, then with metadata.NewOutgoingContext)? To me it looks like the first one is simply redundant and the code can be simplified to:

		// Copy the inbound metadata explicitly.
		outCtx := metadata.NewOutgoingContext(ctx, md.Copy())

to make proxy.StreamDirector return an array of grpcClientConnection

currently the proxy.StreamDirector returns single grpc.ClientConn.
what if i need to return multiple grpc ClientConn at once. this support needs to be added.

current supported:
type StreamDirector func(ctx context.Context, fullMethodName string) (context.Context, *grpc.ClientConn, error)

Required
type StreamDirector func(ctx context.Context, fullMethodName string) (context.Context, []*grpc.ClientConn, error)

Proxy Director

According to your comment

// Provide sa simple example of a director that shields internal services and dials a staging or production backend.
// This is a *very naive* implementation that creates a new connection on every request. Consider using pooling.

from https://github.com/mwitkow/grpc-proxy/blob/master/proxy/examples_test.go#L37

I've tried to implement pooling for my connections. Unfortunately connection pooling is not possible. *grpc.ClientConn is struct not interface hence I can't inject *grpc.ClientConn into composition with Close method that would return connection to the pool. That means I have to create connection every time it's requested.

Do you know any other way of optimization? Or maybe example from examples_test.go is not that scary?

`failed to unmarshal, message is *proxy.Frame, want proto.Message`

Hi!

When trying to use this proxy[0], I'm getting the following error: failed to unmarshal the received message: failed to unmarshal, message is *proxy.Frame, want proto.Message

Some more info in case that helps:

  • the backend service I'm using is proto3

What could I do to fix?

Thank you so much,
Philmod

[0]

director := func(ctx context.Context, fullMethodName string) (context.Context, *grpc.ClientConn, error) {
	md, _ := metadata.FromIncomingContext(ctx)
	outCtx := metadata.NewOutgoingContext(ctx, md.Copy())
	if err := CheckBearerToken(ctx); err != nil {
		return outCtx, nil, grpc.Errorf(codes.PermissionDenied, "unauthorized access: %v", err)
	}
	return outCtx, backendConn, nil
}

server := grpc.NewServer(grpc.UnknownServiceHandler(proxy.TransparentHandler(director)))

The sample in the README makes no sense

director = func(ctx context.Context, fullMethodName string) (context.Context, *grpc.ClientConn, error) {
    md, _ := metadata.FromIncomingContext(ctx)
    outCtx = metadata.NewOutgoingContext(ctx, md.Copy())
    // cc is probably intended to be the ClientConn?
    return outCtx, cc, nil
	
    // >>>> THIS CODE IS DEAD
    // Make sure we never forward internal services.
    if strings.HasPrefix(fullMethodName, "/com.example.internal.") {
        return outCtx, nil, status.Errorf(codes.Unimplemented, "Unknown method")
    }

Note that I stumbled onto this because some code I was writing that uses mwitkow/grpc-proxy stopped working after this change. (I spent several hours on it and still don't understand what went wrong. The code that is failing is https://github.com/observatorium/api/pull/199/files#diff-2873f79a86c0d8b3335cd7731b0ecf7dd4301eb19a82ef7a1cba7589b5252261R1049 )

Move to protobuf API v2.

We need a spike if grpc-proxy can easily work with API v2.

That might be an important step in knowing if we can push this project forward and how (:

How to use it ??

I cannot figure out how to use this library with these codes.

director := func(ctx context.Context) (*grpc.ClientConn, error) {
    if err := CheckBearerToken(ctx); err != nil {
        return nil, grpc.Errorf(codes.PermissionDenied, "unauthorized access: %v", err)
    }
    stream, _ := transport.StreamFromContext(ctx)
    backend, found := PreDialledBackends[stream.Method()];
    if !found {
        return nil, grpc.Errorf(codes.Unimplemented, "the service %v is not implemented", stream.Method)
    }
    return backend, nil
}

proxy := grpcproxy.NewProxy(director)
proxy.Server(boundListener)

Problem forwardServerToClient() since last month's modernization

I spent some time today trying to understand why my code fails with the recent modernization.

My change is related to switching out frame for anypb.Any in handler.go at https://github.com/mwitkow/grpc-proxy/pull/52/files#diff-e803501e5fa26d41cd2723669bd3e5c3df2252d9e8004cc8f42dbada07f80802R115

Before this change things behaved as I expected. With the use of Any to hold the data my code fails because my data doesn't want to go into an Any. It was probably happy going into a frame because that is just a byte array. I never make it past forwardServerToClient().

I am not an expert in the details of protobuf serialization. I have data that is already in a different protobuf format, a format that includes bytes, that somehow isn't compatible with anypb.Any. I will describe what I am doing, perhaps it will be clear which one of us is doing something wrong.

The data that I am trying to forward is Open Telemetry Protobufs, created for example using grpcurl -d @ -plaintext -import-path "$OTEL_DIR"/opentelemetry-proto/ -proto "$OTEL_DIR"/opentelemetry-proto/opentelemetry/proto/collector/trace/v1/trace_service.proto localhost:8317 opentelemetry.proto.collector.trace.v1.TraceService/Export where the data that I attempt to proxy that grpcurl encodes is

{"resourceSpans":[{"instrumentationLibrarySpans":[{"spans":[
  {
    "trace_id":"26/mx4ytaQfaV87uqKocZA==",
    "span_id": "wPE19XLMS30=",
    "name": "test"
  }
]}]}]}

(trace_id and span_id are bytes, encoded as base64 the way grpcurl likes it.). Source of the .proto is https://github.com/open-telemetry/opentelemetry-proto/blob/main/opentelemetry/proto/trace/v1/trace.proto

Your proxy/handler.go forwardClientToServer() calls grpc.RecvMsg(), which eventually tries to unmarshal using codec proxy>proto, bytes are [10 38 18 36 18 34 10 16 219 175 230 199 140 173 105 7 218 87 206 238 168 170 28 100 18 8 192 241 53 245 114 204 75 125 42 4 116 101 115 116]) into a *anypb.Any. This fails when it eventually gets to protobuf impl.consumeStringValidateUTF8() with some bytes that aren't a valid string. (I am not sure why anypb.Any thought there was a UTF-8-valid string in there.)

Any thoughts?

Connection leakage

Using grpc-proxy with a simple test server, I see every connection from the proxy to the endpoint leaking a file descriptor.

I've narrowed it down to 2 cases in handler.handler, both involve not closing backendConn.

  1. if the client goes away unexpectedly, grpc.NewClientStream can fail and the backendConn leaks. Need to close the backendConn in that case.

  2. seems that backendConn needs to be closed at some point. Experimentally, I found that closing backendConn after serverStream.SetTrailer eliminated the leaks I was seeing.

Without those two fixes, I see connections pile up within the proxy until it runs out of file descriptors and fails subsequent requests.

project status?

etcd team is working on etcd grpc layer proxy. For some rpc calls, we want to a go-through proxy just like what this project does. So I want to check the status of this project. Do you have plan to make this production ready? If yes, shall we collaborate on this one?

SSL Passthrough

Is there a way to proxy GRPC SSL connections using SSL passthrough right now if not is it something that is planned?

panic when proxying route_guide from grpc/examples

my proxy server code:

package main

import (
  "log"
  "net"

  "github.com/mwitkow/grpc-proxy/proxy"
  "golang.org/x/net/context"
  "google.golang.org/grpc"
)

func myDirector(ctx context.Context, fmn string) (*grpc.ClientConn, error) {
  return grpc.DialContext(ctx, "127.0.0.1:10001",
        grpc.WithCodec(proxy.Codec()),
        grpc.WithInsecure())
}

func main() {
  l, err := net.Listen("tcp", "127.0.0.1:10000")
  if err != nil {
    log.Fatalln(err)
  }
  s := grpc.NewServer(
        grpc.CustomCodec(proxy.Codec()),
        grpc.UnknownServiceHandler(proxy.TransparentHandler(myDirector)))
  if err := s.Serve(l); err != nil {
    log.Fatalln(err)
  }
}

i got the following error when proxy the route_guide example come with grpc:

panic: send on closed channel

goroutine 36 [running]:
github.com/mwitkow/grpc-proxy/proxy.(*handler).forwardServerToClient.func1(0x9f9e40, 0xc42007f900, 0xc420178180, 0x9f9de0, 0xc4200e1180)
/home/vagrant/src/ggrpc/src/github.com/mwitkow/grpc-proxy/proxy/handler.go:151 +0x12f
created by github.com/mwitkow/grpc-proxy/proxy.(*handler).forwardServerToClient
/home/vagrant/src/ggrpc/src/github.com/mwitkow/grpc-proxy/proxy/handler.go:159 +0x89
exit status 2

handling errors of StreamDirector: nil pointer dereference

The ExampleStreamDirector (in examples_test.go) my run:

    return nil, nil, grpc.Errorf(codes.Unimplemented, "Unknown method")

handler.go then uses the outgoingCtx without first checking for an error:

        outgoingCtx, backendConn, err := s.director(serverStream.Context(), fullMethodName)
	clientCtx, clientCancel := context.WithCancel(outgoingCtx)
	if err != nil {
		return err
        }

This leads to a nil pointer dereference if the directory really returns an error:

context.propagateCancel(0x0, 0x0, 0xa0ab80, 0xc4203be0c0)
	/usr/lib/go-1.10/src/context/context.go:243 +0x26
context.WithCancel(0x0, 0x0, 0xda1f90, 0x982680, 0xc4203ac3c0)
	/usr/lib/go-1.10/src/context/context.go:232 +0x111
github.com/intel/open-infrastructure-offload/vendor/golang.org/x/net/context.WithCancel(0x0, 0x0, 0xc420384040, 0x1c, 0x0)
	/fast/work/gopath/src/github.com/intel/open-infrastructure-offload/vendor/golang.org/x/net/context/go17.go:33 +0x35
github.com/intel/open-infrastructure-offload/vendor/github.com/mwitkow/grpc-proxy/proxy.(*handler).handler(0xc4202b8010, 0x0, 0x0, 0xa111c0, 0xc4203d40a0, 0x0, 0x0)
	/fast/work/gopath/src/github.com/intel/open-infrastructure-offload/vendor/github.com/mwitkow/grpc-proxy/proxy/handler.go:70 +0x11e
github.com/intel/open-infrastructure-offload/vendor/github.com/mwitkow/grpc-proxy/proxy.(*handler).(github.com/intel/open-infrastructure-offload/vendor/github.com/mwitkow/grpc-proxy/proxy.handler)-fm(0x0, 0x0, 0xa111c0, 0xc4203d40a0, 0x0, 0x0)
	/fast/work/gopath/src/github.com/intel/open-infrastructure-offload/vendor/github.com/mwitkow/grpc-proxy/proxy/handler.go:35 +0x52
github.com/intel/open-infrastructure-offload/vendor/google.golang.org/grpc.(*Server).processStreamingRPC(0xc4202e8000, 0xa11760, 0xc4202a6600, 0xc4203ac3c0, 0x0, 0xc4202e00e0, 0x0, 0x0, 0x0)
	/fast/work/gopath/src/github.com/intel/open-infrastructure-offload/vendor/google.golang.org/grpc/server.go:1052 +0x9b4

I suppose the err check should come before context.WithCancel(outgoingCtx)?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.