rookie-ninja / rk-grpc Goto Github PK

View Code? Open in Web Editor NEW

73.0 1.0 17.0 49.12 MB

Start gRPC microservice from YAML, plugin of rk-boot

Home Page: https://rkdev.info

License: Apache License 2.0

Go 93.09% HTML 0.37% Makefile 0.35% JavaScript 6.04% Dockerfile 0.14%

grpc golang interceptor swagger rk bootstrapper opentracing

rk-grpc's People

Contributors

Stargazers

Watchers

Forkers

fossabot wwek dongxuny ajunlonglive geomethu ludanfeng djun parasby tegarajipangestu averyyan vcnwafor igor-tsiglyar darevski agungcandra wanyuqin izouxv golinn

rk-grpc's Issues

panic: proto: file "error.proto" is already registered

Hi guys, I have an issue while working rk-grpc/v2 with google-pubsub

I got panic: proto: file "error.proto" is already registered when starting my services, below is the complete error

panic: proto: file "error.proto" is already registered


        previously from: "github.com/rookie-ninja/rk-grpc/v2/boot/error/gen"
        currently from:  "github.com/googleapis/gax-go/v2/apierror/internal/proto"
See https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict

After investigating, I found the issue is because the both rk-grpc and gax-go file descriptor is used error.proto as proto name, unlike the suggestion from google that we should also add the namespace before the filename

Below I also attach my go.mod file

module gitlab.com/tools/2sample

go 1.17

require (
	cloud.google.com/go/pubsub v1.17.0
	github.com/alicebob/miniredis/v2 v2.20.0
	github.com/go-redis/redis/v8 v8.11.5
	github.com/jackc/pgx/v4 v4.15.0
	github.com/joeshaw/envdecode v0.0.0-20200121155833-099f1fc765bd
	github.com/joho/godotenv v1.4.0
	github.com/pkg/errors v0.9.1
	github.com/rookie-ninja/rk-entry/v2 v2.1.4
	github.com/rookie-ninja/rk-grpc/v2 v2.1.2
	github.com/stretchr/testify v1.7.1
	google.golang.org/grpc v1.45.0
)

require (
	cloud.google.com/go v0.100.2 // indirect
	cloud.google.com/go/compute v1.5.0 // indirect
	cloud.google.com/go/iam v0.3.0 // indirect
	github.com/alicebob/gopher-json v0.0.0-20200520072559-a9ecdc9d1d3a // indirect
	github.com/andres-erbsen/clock v0.0.0-20160526145045-9e14626cd129 // indirect
	github.com/beorn7/perks v1.0.1 // indirect
	github.com/cespare/xxhash/v2 v2.1.2 // indirect
	github.com/davecgh/go-spew v1.1.1 // indirect
	github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
	github.com/envoyproxy/protoc-gen-validate v0.6.2 // indirect
	github.com/fsnotify/fsnotify v1.5.1 // indirect
	github.com/go-logr/logr v1.2.2 // indirect
	github.com/go-logr/stdr v1.2.2 // indirect
	github.com/golang-jwt/jwt/v4 v4.3.0 // indirect
	github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
	github.com/golang/protobuf v1.5.2 // indirect
	github.com/google/go-cmp v0.5.7 // indirect
	github.com/google/uuid v1.3.0 // indirect
	github.com/googleapis/gax-go/v2 v2.2.0 // indirect
	github.com/grpc-ecosystem/grpc-gateway/v2 v2.8.0 // indirect
	github.com/hashicorp/hcl v1.0.0 // indirect
	github.com/jackc/chunkreader/v2 v2.0.1 // indirect
	github.com/jackc/pgconn v1.11.0 // indirect
	github.com/jackc/pgio v1.0.0 // indirect
	github.com/jackc/pgpassfile v1.0.0 // indirect
	github.com/jackc/pgproto3/v2 v2.2.0 // indirect
	github.com/jackc/pgservicefile v0.0.0-20200714003250-2b9c44734f2b // indirect
	github.com/jackc/pgtype v1.10.0 // indirect
	github.com/jackc/puddle v1.2.1 // indirect
	github.com/magiconair/properties v1.8.5 // indirect
	github.com/matttproud/golang_protobuf_extensions v1.0.1 // indirect
	github.com/mitchellh/mapstructure v1.4.3 // indirect
	github.com/pelletier/go-toml v1.9.4 // indirect
	github.com/pmezard/go-difflib v1.0.0 // indirect
	github.com/prometheus/client_golang v1.12.1 // indirect
	github.com/prometheus/client_model v0.2.0 // indirect
	github.com/prometheus/common v0.32.1 // indirect
	github.com/prometheus/procfs v0.7.3 // indirect
	github.com/rookie-ninja/rk-logger v1.2.11 // indirect
	github.com/rookie-ninja/rk-query v1.2.14 // indirect
	github.com/soheilhy/cmux v0.1.5 // indirect
	github.com/spf13/afero v1.6.0 // indirect
	github.com/spf13/cast v1.4.1 // indirect
	github.com/spf13/jwalterweatherman v1.1.0 // indirect
	github.com/spf13/pflag v1.0.5 // indirect
	github.com/spf13/viper v1.10.1 // indirect
	github.com/subosito/gotenv v1.2.0 // indirect
	github.com/yuin/gopher-lua v0.0.0-20210529063254-f4c35e4016d9 // indirect
	go.opencensus.io v0.23.0 // indirect
	go.opentelemetry.io/contrib v1.4.0 // indirect
	go.opentelemetry.io/otel v1.5.0 // indirect
	go.opentelemetry.io/otel/exporters/jaeger v1.5.0 // indirect
	go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.5.0 // indirect
	go.opentelemetry.io/otel/sdk v1.5.0 // indirect
	go.opentelemetry.io/otel/trace v1.5.0 // indirect
	go.uber.org/atomic v1.7.0 // indirect
	go.uber.org/multierr v1.6.0 // indirect
	go.uber.org/ratelimit v0.2.0 // indirect
	go.uber.org/zap v1.21.0 // indirect
	golang.org/x/crypto v0.0.0-20210817164053-32db794688a5 // indirect
	golang.org/x/net v0.0.0-20220225172249-27dd8689420f // indirect
	golang.org/x/oauth2 v0.0.0-20220223155221-ee480838109b // indirect
	golang.org/x/sync v0.0.0-20210220032951-036812b2e83c // indirect
	golang.org/x/sys v0.0.0-20220227234510-4e6760a101f9 // indirect
	golang.org/x/text v0.3.7 // indirect
	google.golang.org/api v0.71.0 // indirect
	google.golang.org/appengine v1.6.7 // indirect
	google.golang.org/genproto v0.0.0-20220310185008-1973136f34c6 // indirect
	google.golang.org/protobuf v1.28.0 // indirect
	gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c // indirect
	gopkg.in/ini.v1 v1.66.2 // indirect
	gopkg.in/natefinch/lumberjack.v2 v2.0.0 // indirect
	gopkg.in/yaml.v2 v2.4.0 // indirect
	gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect
)

Add timeout interceptor

Implement ProxyEntry

Make gRPC server startup as a proxy server to backend gRPC server.

Allow to listen to localhost instead

I would like making my rkboot server listen to localhost rather than the default setting 0.0.0.0. It seems that the boot.yaml does not support such change for now.

Allow excluding label for prometheus middleware

Current prometheus middleware has restPath as one of its label. When the content is highly varied (e.g. /resource/id, where id can be anyting), it causes significant load on the prometheus due to high cardinality.

It would be nice if there's an option to remove labels on the configuration.

content type of http response is wrong when return grpc error through grpc gateway

How to find this issue:

use example in rk-boot: https://github.com/rookie-ninja/rk-boot/tree/main/example/web/grpc
enable grpc.enableRkGwOption in boot.yaml
return an error in Hello function in main.go
go run main.go
curl -v "http://0.0.0.0:8080/v1/hello?hi="

*   Trying 0.0.0.0:8080...
* Connected to 0.0.0.0 (127.0.0.1) port 8080 (#0)
> GET /v1/hello?hi= HTTP/1.1
> Host: 0.0.0.0:8080
> User-Agent: curl/7.77.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 400 Bad Request
< Content-Type: application/json
< Content-Type: application/grpc   <---  here
< X-Demo-App-Domain: *
< X-Demo-App-Name: 
< X-Demo-App-Unix-Time: 2022-04-04T20:33:49.565908+08:00
< X-Demo-App-Version: 
< X-Demo-Received-Time: 2022-04-04T20:33:49.565908+08:00
< X-Request-Id: 83731756-78c3-4723-9c9b-99f894b69408
< Date: Mon, 04 Apr 2022 12:33:49 GMT
< Content-Length: 74
< 
* Connection #0 to host 0.0.0.0 left intact
{"error":{"code":400,"status":"Bad Request","message":"空","details":[]}}%

PATCH request to endpoints served by grpc-gateway getting connection reset by peer

I am getting connection reset by peer when requesting to PATCH endpoints served by rk-grpc (with enableRkGwOption enabled)

 curl -vvv --location --request PATCH 'localhost:3000/v1/experiment/7' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--data-raw '{
  "name": "Experiment model 123"
}'
*   Trying 127.0.0.1:3000...
* Connected to localhost (127.0.0.1) port 3000 (#0)
> PATCH /v1/experiment/7 HTTP/1.1
> Host: localhost:3000
> User-Agent: curl/7.81.0
> Content-Type: application/json
> Accept: application/json
> Content-Length: 36
>
* Recv failure: Connection reset by peer
* Closing connection 0
curl: (56) Recv failure: Connection reset by peer

By default, enabling enableRkGwOption in your project will result in grpc endpoints and rest endpoints served in the same port using cmux as multiplexer. This is where the bug comes because cmux does not recognize PATCH HTTP methods in their default HTTP method. (https://github.com/soheilhy/cmux/blob/v0.1.5/matchers.go#L46-L55)

There are two possible solutions:

Serving different port for gRPC and REST API (configurable in yml file)
Adding PATCH method in boot/grpc_entry.go file so cmux recognize PATCH method:

httpL := tcpL.Match(cmux.HTTP1Fast("PATCH"))

Unable to set rateLimit per path

Hello team,

I tried to set rate limit per path but I always get 429 even though I have put reqPerSec > 0.

How to find the issue:

Use example in https://github.com/rookie-ninja/rk-grpc
Add rate limit per path in boot.yaml:

grpc:
  - name: greeter
    port: 8080
    enabled: true
    enableRkGwOption: true
    middleware:
      rateLimit:
        enabled: true
        paths:
          - path: "/v1/greeter"
            reqPerSec: 100

go run main.go
curl -v "localhost:8080/v1/greeter"

error":{"code":429,"status":"Too Many Requests","message":"slow down your request","details":[{"code":8,"status":"ResourceExhausted","message":"slow down your request"}]}}

Should I be able to call the endpoint right as I have set the reqPerSec to 100?

Add rate limit interceptor

Possible algorithm:

Leaky bucket
Token bucket

About the error i got.

Hello, I got an error msg when i purchase payment orders via alipay.
It says "Balance not enough".
Please can u fix these problems for me?Thank you.

How to gRPC Gateway keeps the field in the proto original style

1、Proto definition

Underline style
下划线

2、 GRPC Gateway

lowerCamelCase
小驼峰

I need to keep consistent with the original definition of proto
我需在gRPC Gateway中保持和Proto一致的字段命名风格（下划线）

Support interceptors

httpsign
cache
gzip
cors
session

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.