rookie-ninja / rk-grpc Goto Github PK
View Code? Open in Web Editor NEWStart gRPC microservice from YAML, plugin of rk-boot
Home Page: https://rkdev.info
License: Apache License 2.0
Start gRPC microservice from YAML, plugin of rk-boot
Home Page: https://rkdev.info
License: Apache License 2.0
Hi guys, I have an issue while working rk-grpc/v2
with google-pubsub
I got panic: proto: file "error.proto" is already registered
when starting my services, below is the complete error
panic: proto: file "error.proto" is already registered
previously from: "github.com/rookie-ninja/rk-grpc/v2/boot/error/gen"
currently from: "github.com/googleapis/gax-go/v2/apierror/internal/proto"
See https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
After investigating, I found the issue is because the both rk-grpc
and gax-go
file descriptor is used error.proto
as proto name, unlike the suggestion from google that we should also add the namespace before the filename
Below I also attach my go.mod file
module gitlab.com/tools/2sample
go 1.17
require (
cloud.google.com/go/pubsub v1.17.0
github.com/alicebob/miniredis/v2 v2.20.0
github.com/go-redis/redis/v8 v8.11.5
github.com/jackc/pgx/v4 v4.15.0
github.com/joeshaw/envdecode v0.0.0-20200121155833-099f1fc765bd
github.com/joho/godotenv v1.4.0
github.com/pkg/errors v0.9.1
github.com/rookie-ninja/rk-entry/v2 v2.1.4
github.com/rookie-ninja/rk-grpc/v2 v2.1.2
github.com/stretchr/testify v1.7.1
google.golang.org/grpc v1.45.0
)
require (
cloud.google.com/go v0.100.2 // indirect
cloud.google.com/go/compute v1.5.0 // indirect
cloud.google.com/go/iam v0.3.0 // indirect
github.com/alicebob/gopher-json v0.0.0-20200520072559-a9ecdc9d1d3a // indirect
github.com/andres-erbsen/clock v0.0.0-20160526145045-9e14626cd129 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.1.2 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
github.com/envoyproxy/protoc-gen-validate v0.6.2 // indirect
github.com/fsnotify/fsnotify v1.5.1 // indirect
github.com/go-logr/logr v1.2.2 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/golang-jwt/jwt/v4 v4.3.0 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.2 // indirect
github.com/google/go-cmp v0.5.7 // indirect
github.com/google/uuid v1.3.0 // indirect
github.com/googleapis/gax-go/v2 v2.2.0 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.8.0 // indirect
github.com/hashicorp/hcl v1.0.0 // indirect
github.com/jackc/chunkreader/v2 v2.0.1 // indirect
github.com/jackc/pgconn v1.11.0 // indirect
github.com/jackc/pgio v1.0.0 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgproto3/v2 v2.2.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20200714003250-2b9c44734f2b // indirect
github.com/jackc/pgtype v1.10.0 // indirect
github.com/jackc/puddle v1.2.1 // indirect
github.com/magiconair/properties v1.8.5 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.1 // indirect
github.com/mitchellh/mapstructure v1.4.3 // indirect
github.com/pelletier/go-toml v1.9.4 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/prometheus/client_golang v1.12.1 // indirect
github.com/prometheus/client_model v0.2.0 // indirect
github.com/prometheus/common v0.32.1 // indirect
github.com/prometheus/procfs v0.7.3 // indirect
github.com/rookie-ninja/rk-logger v1.2.11 // indirect
github.com/rookie-ninja/rk-query v1.2.14 // indirect
github.com/soheilhy/cmux v0.1.5 // indirect
github.com/spf13/afero v1.6.0 // indirect
github.com/spf13/cast v1.4.1 // indirect
github.com/spf13/jwalterweatherman v1.1.0 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/spf13/viper v1.10.1 // indirect
github.com/subosito/gotenv v1.2.0 // indirect
github.com/yuin/gopher-lua v0.0.0-20210529063254-f4c35e4016d9 // indirect
go.opencensus.io v0.23.0 // indirect
go.opentelemetry.io/contrib v1.4.0 // indirect
go.opentelemetry.io/otel v1.5.0 // indirect
go.opentelemetry.io/otel/exporters/jaeger v1.5.0 // indirect
go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.5.0 // indirect
go.opentelemetry.io/otel/sdk v1.5.0 // indirect
go.opentelemetry.io/otel/trace v1.5.0 // indirect
go.uber.org/atomic v1.7.0 // indirect
go.uber.org/multierr v1.6.0 // indirect
go.uber.org/ratelimit v0.2.0 // indirect
go.uber.org/zap v1.21.0 // indirect
golang.org/x/crypto v0.0.0-20210817164053-32db794688a5 // indirect
golang.org/x/net v0.0.0-20220225172249-27dd8689420f // indirect
golang.org/x/oauth2 v0.0.0-20220223155221-ee480838109b // indirect
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c // indirect
golang.org/x/sys v0.0.0-20220227234510-4e6760a101f9 // indirect
golang.org/x/text v0.3.7 // indirect
google.golang.org/api v0.71.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20220310185008-1973136f34c6 // indirect
google.golang.org/protobuf v1.28.0 // indirect
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c // indirect
gopkg.in/ini.v1 v1.66.2 // indirect
gopkg.in/natefinch/lumberjack.v2 v2.0.0 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect
)
Make gRPC server startup as a proxy server to backend gRPC server.
I would like making my rkboot server listen to localhost rather than the default setting 0.0.0.0. It seems that the boot.yaml does not support such change for now.
Current prometheus middleware has restPath
as one of its label. When the content is highly varied (e.g. /resource/id, where id can be anyting), it causes significant load on the prometheus due to high cardinality.
It would be nice if there's an option to remove labels on the configuration.
How to find this issue:
go run main.go
curl -v "http://0.0.0.0:8080/v1/hello?hi="
* Trying 0.0.0.0:8080...
* Connected to 0.0.0.0 (127.0.0.1) port 8080 (#0)
> GET /v1/hello?hi= HTTP/1.1
> Host: 0.0.0.0:8080
> User-Agent: curl/7.77.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 400 Bad Request
< Content-Type: application/json
< Content-Type: application/grpc <--- here
< X-Demo-App-Domain: *
< X-Demo-App-Name:
< X-Demo-App-Unix-Time: 2022-04-04T20:33:49.565908+08:00
< X-Demo-App-Version:
< X-Demo-Received-Time: 2022-04-04T20:33:49.565908+08:00
< X-Request-Id: 83731756-78c3-4723-9c9b-99f894b69408
< Date: Mon, 04 Apr 2022 12:33:49 GMT
< Content-Length: 74
<
* Connection #0 to host 0.0.0.0 left intact
{"error":{"code":400,"status":"Bad Request","message":"空","details":[]}}%
I am getting connection reset by peer when requesting to PATCH endpoints served by rk-grpc (with enableRkGwOption enabled)
curl -vvv --location --request PATCH 'localhost:3000/v1/experiment/7' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--data-raw '{
"name": "Experiment model 123"
}'
* Trying 127.0.0.1:3000...
* Connected to localhost (127.0.0.1) port 3000 (#0)
> PATCH /v1/experiment/7 HTTP/1.1
> Host: localhost:3000
> User-Agent: curl/7.81.0
> Content-Type: application/json
> Accept: application/json
> Content-Length: 36
>
* Recv failure: Connection reset by peer
* Closing connection 0
curl: (56) Recv failure: Connection reset by peer
By default, enabling enableRkGwOption in your project will result in grpc endpoints and rest endpoints served in the same port using cmux as multiplexer. This is where the bug comes because cmux does not recognize PATCH HTTP methods in their default HTTP method. (https://github.com/soheilhy/cmux/blob/v0.1.5/matchers.go#L46-L55)
There are two possible solutions:
httpL := tcpL.Match(cmux.HTTP1Fast("PATCH"))
Hello team,
I tried to set rate limit per path but I always get 429 even though I have put reqPerSec > 0.
How to find the issue:
grpc:
- name: greeter
port: 8080
enabled: true
enableRkGwOption: true
middleware:
rateLimit:
enabled: true
paths:
- path: "/v1/greeter"
reqPerSec: 100
error":{"code":429,"status":"Too Many Requests","message":"slow down your request","details":[{"code":8,"status":"ResourceExhausted","message":"slow down your request"}]}}
Should I be able to call the endpoint right as I have set the reqPerSec to 100?
Possible algorithm:
Hello, I got an error msg when i purchase payment orders via alipay.
It says "Balance not enough".
Please can u fix these problems for me?Thank you.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.