grpc-ecosystem / polyglot Goto Github PK
View Code? Open in Web Editor NEWA universal grpc command line client
License: BSD 3-Clause "New" or "Revised" License
A universal grpc command line client
License: BSD 3-Clause "New" or "Revised" License
Are there any parts of polyglot that heavily depend on Java 8?
I need Java 7 support for the project I am working on.
I think the addition of client cert support warrants a new release. Can you please cut a new tag and upload a new jar? Thanks!
please see here protocolbuffers/protobuf#4501
thanks!
You can use the CA and TLS certificates + loaders from upstream grpc-java
:
Certs in here:
https://github.com/grpc/grpc-java/tree/5c4817723e90b8adb8ed0cf923885df6301ed303/testing/src/main/resources/certs
The released jar for polyglot is not working with well known types for protos (i.e. timestamps).
protoc
run locally works fine and the well known types are present in /usr/local/include/google/protobuf/
Michaels-MacBook-Pro:idls sindelar$ java -jar polyglot.jar --command=list_services --proto_discovery_root=model/
[main] INFO me.dinowernli.grpc.polyglot.Main - Polyglot version: 1.3.0
[main] INFO me.dinowernli.grpc.polyglot.Main - Loaded configuration:
google/protobuf/timestamp.proto: File not found.
replica.proto: Import "google/protobuf/timestamp.proto" was not found or had errors.
replica.proto:120:3: "google.protobuf.Timestamp" is not defined.
replica.proto:122:3: "google.protobuf.Timestamp" is not defined.
replica.proto:145:3: "google.protobuf.Timestamp" is not defined.
replica.proto:146:3: "google.protobuf.Timestamp" is not defined.
[main] WARN me.dinowernli.grpc.polyglot.protobuf.ProtocInvoker - Protoc invocation failed with status: 1
[main] WARN me.dinowernli.grpc.polyglot.protobuf.ProtocInvoker - [Protoc log] protoc-jar: protoc version: 300, detected platform: mac os x/x86_64
[main] WARN me.dinowernli.grpc.polyglot.protobuf.ProtocInvoker - [Protoc log] protoc-jar: executing: [/var/folders/xz/ytdp_wr52cj2jhxz76_m6b6w0000gn/T/protoc4924784953496390184.exe, /Users/sindelar/sidewalk/idls/model/replica.proto, /Users/sindelar/sidewalk/idls/model/map_service.proto, /Users/sindelar/sidewalk/idls/model/persona.proto, -I/Users/sindelar/sidewalk/idls/model, --descriptor_set_out=/var/folders/xz/ytdp_wr52cj2jhxz76_m6b6w0000gn/T/descriptor904114720190676395.pb.bin, --include_imports]
Exception in thread "main" java.lang.RuntimeException: Failed to invoke the protoc binary
at me.dinowernli.grpc.polyglot.Main.getFileDescriptorSet(Main.java:96)
at me.dinowernli.grpc.polyglot.Main.main(Main.java:52)
Caused by: me.dinowernli.grpc.polyglot.protobuf.ProtocInvoker$ProtocInvocationException: Got exit code [1] from protoc with args [[/Users/sindelar/sidewalk/idls/model/replica.proto, /Users/sindelar/sidewalk/idls/model/map_service.proto, /Users/sindelar/sidewalk/idls/model/persona.proto, -I/Users/sindelar/sidewalk/idls/model, --descriptor_set_out=/var/folders/xz/ytdp_wr52cj2jhxz76_m6b6w0000gn/T/descriptor904114720190676395.pb.bin, --include_imports]]
at me.dinowernli.grpc.polyglot.protobuf.ProtocInvoker.invokeBinary(ProtocInvoker.java:132)
at me.dinowernli.grpc.polyglot.protobuf.ProtocInvoker.invoke(ProtocInvoker.java:82)
at me.dinowernli.grpc.polyglot.Main.getFileDescriptorSet(Main.java:94)
... 1 more
Seems related to this issue
protocolbuffers/protobuf#1889
If you try to connect with --tls_client_key_path
point to a valid private key in PKCS#1 format, Polyglot dies with:
java.lang.IllegalArgumentException: File does not contain valid private key: out/polyglot.key
at io.netty.handler.ssl.SslContextBuilder.keyManager(SslContextBuilder.java:267)
at io.netty.handler.ssl.SslContextBuilder.keyManager(SslContextBuilder.java:222)
at me.dinowernli.grpc.polyglot.grpc.ChannelFactory.createSslContext(ChannelFactory.java:105)
at me.dinowernli.grpc.polyglot.grpc.ChannelFactory.createChannelBuilder(ChannelFactory.java:71)
at me.dinowernli.grpc.polyglot.grpc.ChannelFactory.createChannel(ChannelFactory.java:51)
at me.dinowernli.grpc.polyglot.command.ServiceCall.callEndpoint(ServiceCall.java:68)
at me.dinowernli.grpc.polyglot.Main.main(Main.java:70)
Caused by: java.security.NoSuchAlgorithmException: EC KeyFactory not available
at java.security.KeyFactory.<init>(java.base@9-internal/KeyFactory.java:138)
at java.security.KeyFactory.getInstance(java.base@9-internal/KeyFactory.java:179)
at io.netty.handler.ssl.SslContext.getPrivateKeyFromByteBuffer(SslContext.java:1043)
at io.netty.handler.ssl.SslContext.toPrivateKey(SslContext.java:1014)
at io.netty.handler.ssl.SslContextBuilder.keyManager(SslContextBuilder.java:265)
... 6 more
Exception in thread "main" java.lang.RuntimeException: java.lang.IllegalArgumentException: File does not contain valid private key: out/polyglot.key
at me.dinowernli.grpc.polyglot.Main.main(Main.java:86)
Caused by: java.lang.IllegalArgumentException: File does not contain valid private key: out/polyglot.key
at io.netty.handler.ssl.SslContextBuilder.keyManager(SslContextBuilder.java:267)
at io.netty.handler.ssl.SslContextBuilder.keyManager(SslContextBuilder.java:222)
at me.dinowernli.grpc.polyglot.grpc.ChannelFactory.createSslContext(ChannelFactory.java:105)
at me.dinowernli.grpc.polyglot.grpc.ChannelFactory.createChannelBuilder(ChannelFactory.java:71)
at me.dinowernli.grpc.polyglot.grpc.ChannelFactory.createChannel(ChannelFactory.java:51)
at me.dinowernli.grpc.polyglot.command.ServiceCall.callEndpoint(ServiceCall.java:68)
at me.dinowernli.grpc.polyglot.Main.main(Main.java:70)
Caused by: java.security.NoSuchAlgorithmException: EC KeyFactory not available
at java.security.KeyFactory.<init>(java.base@9-internal/KeyFactory.java:138)
at java.security.KeyFactory.getInstance(java.base@9-internal/KeyFactory.java:179)
at io.netty.handler.ssl.SslContext.getPrivateKeyFromByteBuffer(SslContext.java:1043)
at io.netty.handler.ssl.SslContext.toPrivateKey(SslContext.java:1014)
at io.netty.handler.ssl.SslContextBuilder.keyManager(SslContextBuilder.java:265)
... 6 more
Converting the key with this command line makes it work:
openssl pkcs8 -topk8 -inform PEM -outform PEM -nocrypt -in polyglot.key -out polyglot-pkcs8.key
Would be nice if Polyglot figured this out on its own or at least told you what the issue is.
bazel test //src/test/java/me/dinowernli/grpc/polyglot/integration:tests
fails when there is a non-trivial ~/.polyglot/config.pb.json
Would be good to be able to cache some of the compiled protos or clients that are created so that they can be used again if the .proto
file has not been altered. This would hopefully speed up subsequent calls to polyglot
.
First call:
time echo {'name':'dom1'} | java -jar ~/polyglot.jar \
--command=call \
--endpoint=localhost:50051 \
--full_method=helloworld.Greeter/SayHello \
--proto_discovery_root=/Users/dominicgreen/go/src/google.golang.org/grpc/examples/helloworld/helloworld
[main] INFO me.dinowernli.grpc.polyglot.Main - Polyglot version: 1.6.0
[main] INFO me.dinowernli.grpc.polyglot.Main - Loaded configuration:
[main] INFO me.dinowernli.grpc.polyglot.command.ServiceCall - Creating channel to: localhost:50051
[main] INFO me.dinowernli.grpc.polyglot.command.ServiceCall - Using proto descriptors obtained from protoc
[main] INFO me.dinowernli.grpc.polyglot.command.ServiceCall - Creating dynamic grpc client
[main] INFO me.dinowernli.grpc.polyglot.command.ServiceCall - Making rpc with 1 request(s) to endpoint [localhost:50051]
[main] INFO me.dinowernli.grpc.polyglot.grpc.DynamicGrpcClient - Making unary call
[grpc-default-executor-0] INFO me.dinowernli.grpc.polyglot.io.LoggingStatsWriter - Got response message
{
"message": "Hello dom1"
}
[grpc-default-executor-0] INFO me.dinowernli.grpc.polyglot.io.LoggingStatsWriter - Completed rpc with 1 response(s)
echo {'name':'dom1'} 0.00s user 0.00s system 47% cpu 0.003 total
java -jar ~/polyglot.jar --command=call --endpoint=localhost:50051 1.74s user 0.18s system 30% cpu 6.403 total
Second call, calls protoc again and takes similar time:
time echo {'name':'dom2'} | java -jar ~/polyglot.jar \
--command=call \
--endpoint=localhost:50051 \
--full_method=helloworld.Greeter/SayHello \
--proto_discovery_root=/Users/dominicgreen/go/src/google.golang.org/grpc/examples/helloworld/helloworld
[main] INFO me.dinowernli.grpc.polyglot.Main - Polyglot version: 1.6.0
[main] INFO me.dinowernli.grpc.polyglot.Main - Loaded configuration:
[main] INFO me.dinowernli.grpc.polyglot.command.ServiceCall - Creating channel to: localhost:50051
[main] INFO me.dinowernli.grpc.polyglot.command.ServiceCall - Using proto descriptors obtained from protoc
[main] INFO me.dinowernli.grpc.polyglot.command.ServiceCall - Creating dynamic grpc client
[main] INFO me.dinowernli.grpc.polyglot.command.ServiceCall - Making rpc with 1 request(s) to endpoint [localhost:50051]
[main] INFO me.dinowernli.grpc.polyglot.grpc.DynamicGrpcClient - Making unary call
[grpc-default-executor-0] INFO me.dinowernli.grpc.polyglot.io.LoggingStatsWriter - Got response message
{
"message": "Hello dom2"
}
[grpc-default-executor-0] INFO me.dinowernli.grpc.polyglot.io.LoggingStatsWriter - Completed rpc with 1 response(s)
echo {'name':'dom2'} 0.00s user 0.00s system 39% cpu 0.004 total
java -jar ~/polyglot.jar --command=call --endpoint=localhost:50051 1.99s user 0.21s system 32% cpu 6.694 total
Had some problems running grpcc
(it fails only on some of our proto definitions, all of which have been successfully running in production for a while now), so I decided to try polyglot
, to no avail.
[main] INFO me.dinowernli.grpc.polyglot.Main - Polyglot version: 1.2.0+dev
[main] INFO me.dinowernli.grpc.polyglot.Main - Loaded configuration:
: File not found.
[main] WARN me.dinowernli.grpc.polyglot.protobuf.ProtocInvoker - Protoc invocation failed with status: 1
[main] WARN me.dinowernli.grpc.polyglot.protobuf.ProtocInvoker - [Protoc log] protoc-jar: protoc version: 300, detected platform: mac os x/x86_64
[main] WARN me.dinowernli.grpc.polyglot.protobuf.ProtocInvoker - [Protoc log] protoc-jar: executing: [/var/folders/nd/ygk5rs6x1v14q_p40027lmw80000gn/T/protoc2758170058438073348.exe, /Users/andrei/src/pc/proto/./portablecloud/cube/downloader/downloader.proto, -I/Users/andrei/src/pc/proto/./portablecloud/cube/downloader/downloader.proto, --descriptor_set_out=/var/folders/nd/ygk5rs6x1v14q_p40027lmw80000gn/T/descriptor2930471825732936678.pb.bin, --include_imports]
Exception in thread "main" java.lang.RuntimeException: Failed to invoke the protoc binary
at me.dinowernli.grpc.polyglot.Main.getFileDescriptorSet(Main.java:96)
at me.dinowernli.grpc.polyglot.Main.main(Main.java:52)
Caused by: me.dinowernli.grpc.polyglot.protobuf.ProtocInvoker$ProtocInvocationException: Got exit code [1] from protoc with args [[/Users/andrei/src/pc/proto/./portablecloud/cube/downloader/downloader.proto, -I/Users/andrei/src/pc/proto/./portablecloud/cube/downloader/downloader.proto, --descriptor_set_out=/var/folders/nd/ygk5rs6x1v14q_p40027lmw80000gn/T/descriptor2930471825732936678.pb.bin, --include_imports]]
at me.dinowernli.grpc.polyglot.protobuf.ProtocInvoker.invokeBinary(ProtocInvoker.java:132)
at me.dinowernli.grpc.polyglot.protobuf.ProtocInvoker.invoke(ProtocInvoker.java:82)
at me.dinowernli.grpc.polyglot.Main.getFileDescriptorSet(Main.java:94)
... 1 more
โ proto git:(master) polyglot --command=list_services --proto_discovery_root=./portablecloud/cube/downloader
[main] INFO me.dinowernli.grpc.polyglot.Main - Polyglot version: 1.2.0+dev
[main] INFO me.dinowernli.grpc.polyglot.Main - Loaded configuration:
gogoproto/gogo.proto: File not found.
downloader.proto: Import "gogoproto/gogo.proto" was not found or had errors.
[main] WARN me.dinowernli.grpc.polyglot.protobuf.ProtocInvoker - Protoc invocation failed with status: 1
[main] WARN me.dinowernli.grpc.polyglot.protobuf.ProtocInvoker - [Protoc log] protoc-jar: protoc version: 300, detected platform: mac os x/x86_64
[main] WARN me.dinowernli.grpc.polyglot.protobuf.ProtocInvoker - [Protoc log] protoc-jar: executing: [/var/folders/nd/ygk5rs6x1v14q_p40027lmw80000gn/T/protoc766653710895866770.exe, /Users/andrei/src/pc/proto/./portablecloud/cube/downloader/downloader.proto, -I/Users/andrei/src/pc/proto/./portablecloud/cube/downloader, --descriptor_set_out=/var/folders/nd/ygk5rs6x1v14q_p40027lmw80000gn/T/descriptor1304231409264632622.pb.bin, --include_imports]
Exception in thread "main" java.lang.RuntimeException: Failed to invoke the protoc binary
at me.dinowernli.grpc.polyglot.Main.getFileDescriptorSet(Main.java:96)
at me.dinowernli.grpc.polyglot.Main.main(Main.java:52)
Caused by: me.dinowernli.grpc.polyglot.protobuf.ProtocInvoker$ProtocInvocationException: Got exit code [1] from protoc with args [[/Users/andrei/src/pc/proto/./portablecloud/cube/downloader/downloader.proto, -I/Users/andrei/src/pc/proto/./portablecloud/cube/downloader, --descriptor_set_out=/var/folders/nd/ygk5rs6x1v14q_p40027lmw80000gn/T/descriptor1304231409264632622.pb.bin, --include_imports]]
at me.dinowernli.grpc.polyglot.protobuf.ProtocInvoker.invokeBinary(ProtocInvoker.java:132)
at me.dinowernli.grpc.polyglot.protobuf.ProtocInvoker.invoke(ProtocInvoker.java:82)
at me.dinowernli.grpc.polyglot.Main.getFileDescriptorSet(Main.java:94)
... 1 more```
โ ~ echo '{"id": "vikidia-en-all@2017-04"}' | polyglot --command=call --endpoint=localhost:7007 --full_method=portablecloud.cube.downloader.Downloader.Get --proto_discovery_root=./src/pc/proto/portablecloud/cube/downloader/downloader.proto
[main] INFO me.dinowernli.grpc.polyglot.Main - Polyglot version: 1.2.0+dev
[main] INFO me.dinowernli.grpc.polyglot.Main - Loaded configuration:
: File not found.
[main] WARN me.dinowernli.grpc.polyglot.protobuf.ProtocInvoker - Protoc invocation failed with status: 1
[main] WARN me.dinowernli.grpc.polyglot.protobuf.ProtocInvoker - [Protoc log] protoc-jar: protoc version: 300, detected platform: mac os x/x86_64
[main] WARN me.dinowernli.grpc.polyglot.protobuf.ProtocInvoker - [Protoc log] protoc-jar: executing: [/var/folders/nd/ygk5rs6x1v14q_p40027lmw80000gn/T/protoc5148216538954907554.exe, /Users/andrei/./src/pc/proto/portablecloud/cube/downloader/downloader.proto, -I/Users/andrei/./src/pc/proto/portablecloud/cube/downloader/downloader.proto, --descriptor_set_out=/var/folders/nd/ygk5rs6x1v14q_p40027lmw80000gn/T/descriptor777018564336507942.pb.bin, --include_imports]
Exception in thread "main" java.lang.RuntimeException: Failed to invoke the protoc binary
at me.dinowernli.grpc.polyglot.Main.getFileDescriptorSet(Main.java:96)
at me.dinowernli.grpc.polyglot.Main.main(Main.java:52)
Caused by: me.dinowernli.grpc.polyglot.protobuf.ProtocInvoker$ProtocInvocationException: Got exit code [1] from protoc with args [[/Users/andrei/./src/pc/proto/portablecloud/cube/downloader/downloader.proto, -I/Users/andrei/./src/pc/proto/portablecloud/cube/downloader/downloader.proto, --descriptor_set_out=/var/folders/nd/ygk5rs6x1v14q_p40027lmw80000gn/T/descriptor777018564336507942.pb.bin, --include_imports]]
at me.dinowernli.grpc.polyglot.protobuf.ProtocInvoker.invokeBinary(ProtocInvoker.java:132)
at me.dinowernli.grpc.polyglot.protobuf.ProtocInvoker.invoke(ProtocInvoker.java:82)
at me.dinowernli.grpc.polyglot.Main.getFileDescriptorSet(Main.java:94)
... 1 more```
I've run polyglot
a few times to list services or call procedures and every time it returns after a stack trace or some error the process always exists with status code 0. It would be useful if it could return with a non-zero value for the cases where it fails to perform the given operation. Of course there is also the case where we make a call to a procedure (command=call) and the RPC itself returns an error. I think in this last case it would be nice to have another flag added to polyglot
where we can specify what process exit code we want it to return when the RPC itself returns with error, e.g. --rpc_err_exit_code=1
would mean "have the polyglot
process exit with code 1 if RPC error != null`.
I have following (incomplete) protobuf definition
message Create {
bytes data = 1;
}
How do I create a json request with polyglot that sets the parameter.
data=$(base64 image.jpeg)
echo '{"data" : "'$data'" }' | call --full_method=Create
doens't seem to work.
[main] INFO me.dinowernli.grpc.polyglot.Main - Polyglot version: 1.2.0+dev
[main] INFO me.dinowernli.grpc.polyglot.Main - Loaded configuration:
[main] INFO me.dinowernli.grpc.polyglot.command.ServiceCall - Creating dynamic grpc client
[main] INFO io.grpc.internal.ManagedChannelImpl - [ManagedChannelImpl@448ff1a8] Created with target localhost:8182
Exception in thread "main" java.lang.RuntimeException: Caught exception during command execution
at me.dinowernli.grpc.polyglot.Main.main(Main.java:87)
Caused by: java.lang.IllegalArgumentException: Unable to read messages from: STDIN
at me.dinowernli.grpc.polyglot.io.MessageReader.read(MessageReader.java:85)
at me.dinowernli.grpc.polyglot.command.ServiceCall.callEndpoint(ServiceCall.java:75)
at me.dinowernli.grpc.polyglot.Main.main(Main.java:72)
Caused by: com.google.protobuf.InvalidProtocolBufferException: com.google.common.io.BaseEncoding$DecodingException: Unrecognized character: 0x20
at com.google.protobuf.util.JsonFormat$ParserImpl.merge(JsonFormat.java:992)
at com.google.protobuf.util.JsonFormat$Parser.merge(JsonFormat.java:231)
at me.dinowernli.grpc.polyglot.io.MessageReader.read(MessageReader.java:79)
Hi,
I tried to pass auth token in below format:
--metadata="Authorization:Bearer " into polyglot command line args, which results in error as space after "Bearer" is not read properly.
Is there any other way to achieve the same
Hi,
It would be great if you could add a paragraph about logging to the README.md file.
Something along the lines of this:
Polyglot uses the slf4j
logging framework with the org.slf4j.impl.SimpleLogger
implementation. It also redirects gRPC's JUL logs to slf4j. To change the default log level, specify the org.slf4j.simpleLogger.defaultLogLevel
JVM argument, e.g. -Dorg.slf4j.simpleLogger.defaultLogLevel=debug
.
Many thanks,
Feso
I could not resolve below issue. This is I am compiling polyglot from Virtual Machine where proxy is set.
No issue when I compiled in my laptop with public network.
root@BLR1000015302:/polyglot# bazel build src/main/java/me/dinowernli/grpc/polyglot/polyglot#
Unhandled exception thrown during build; message: Unrecoverable error while evaluating node 'REPOSITORY_DIRECTORY:@protoc_osx_x86_64' (requested by nodes 'REPOSITORY:@protoc_osx_x86_64')
INFO: Elapsed time: 12.518s
java.lang.RuntimeException: Unrecoverable error while evaluating node 'REPOSITORY_DIRECTORY:@protoc_osx_x86_64' (requested by nodes 'REPOSITORY:@protoc_osx_x86_64')
at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:447)
at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:496)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at com.sun.security.ntlm.Client.type3(Client.java:161)
at sun.net.www.protocol.http.ntlm.NTLMAuthentication.buildType3Msg(NTLMAuthentication.java:241)
at sun.net.www.protocol.http.ntlm.NTLMAuthentication.setHeaders(NTLMAuthentication.java:216)
at sun.net.www.protocol.http.HttpURLConnection.doTunneling(HttpURLConnection.java:2096)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:183)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153)
at com.google.devtools.build.lib.bazel.repository.downloader.HttpConnection.createAndConnectViaHttp(HttpConnection.java:144)
at com.google.devtools.build.lib.bazel.repository.downloader.HttpConnection.createAndConnect(HttpConnection.java:92)
at com.google.devtools.build.lib.bazel.repository.downloader.HttpDownloader.download(HttpDownloader.java:149)
at com.google.devtools.build.lib.bazel.repository.downloader.HttpDownloader.download(HttpDownloader.java:92)
at com.google.devtools.build.lib.bazel.repository.HttpArchiveFunction.fetch(HttpArchiveFunction.java:66)
at com.google.devtools.build.lib.rules.repository.RepositoryDelegatorFunction.compute(RepositoryDelegatorFunction.java:155)
at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:388)
... 4 more
java.lang.RuntimeException: Unrecoverable error while evaluating node 'REPOSITORY_DIRECTORY:@protoc_osx_x86_64' (requested by nodes 'REPOSITORY:@protoc_osx_x86_64')
at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:447)
at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:496)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at com.sun.security.ntlm.Client.type3(Client.java:161)
at sun.net.www.protocol.http.ntlm.NTLMAuthentication.buildType3Msg(NTLMAuthentication.java:241)
at sun.net.www.protocol.http.ntlm.NTLMAuthentication.setHeaders(NTLMAuthentication.java:216)
at sun.net.www.protocol.http.HttpURLConnection.doTunneling(HttpURLConnection.java:2096)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:183)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153)
at com.google.devtools.build.lib.bazel.repository.downloader.HttpConnection.createAndConnectViaHttp(HttpConnection.java:144)
at com.google.devtools.build.lib.bazel.repository.downloader.HttpConnection.createAndConnect(HttpConnection.java:92)
at com.google.devtools.build.lib.bazel.repository.downloader.HttpDownloader.download(HttpDownloader.java:149)
at com.google.devtools.build.lib.bazel.repository.downloader.HttpDownloader.download(HttpDownloader.java:92)
at com.google.devtools.build.lib.bazel.repository.HttpArchiveFunction.fetch(HttpArchiveFunction.java:66)
at com.google.devtools.build.lib.rules.repository.RepositoryDelegatorFunction.compute(RepositoryDelegatorFunction.java:155)
at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:388)
... 4 more
root@BLR1000015302:
Regards
-Mahesh
When using Polyglot 1.6.0
it doesnt seem to trigger server reflection and always falls back to protoc
this example is using the HelloWorld example from the go-grpc.
Server code with reflection: https://github.com/grpc/grpc-go/blob/master/examples/helloworld/greeter_server/main.go
Call:
echo {'name':'dom'} | java -jar ~/polyglot.jar \
--command=call \
--use_reflection=true --endpoint=localhost:50051 \
--full_method=helloworld.Greeter/SayHello \
--proto_discovery_root=.
Output:
[main] INFO me.dinowernli.grpc.polyglot.Main - Polyglot version: 1.6.0
[main] INFO me.dinowernli.grpc.polyglot.Main - Loaded configuration:
[main] INFO me.dinowernli.grpc.polyglot.command.ServiceCall - Creating channel to: localhost:50051
[main] INFO me.dinowernli.grpc.polyglot.command.ServiceCall - Using proto descriptors obtained from protoc
[main] INFO me.dinowernli.grpc.polyglot.command.ServiceCall - Creating dynamic grpc client
[main] INFO me.dinowernli.grpc.polyglot.command.ServiceCall - Making rpc with 1 request(s) to endpoint [localhost:50051]
[main] INFO me.dinowernli.grpc.polyglot.grpc.DynamicGrpcClient - Making unary call
[grpc-default-executor-0] INFO me.dinowernli.grpc.polyglot.io.LoggingStatsWriter - Got response message
{
"message": "Hello dom"
}
[grpc-default-executor-0] INFO me.dinowernli.grpc.polyglot.io.LoggingStatsWriter - Completed rpc with 1 response(s)
In 1.4.0
running the same command in previous versions we get a ERROR
logged with a stack trace when the server does not have reflection turned on. The call completes successfully but would be much nicer not to have the stacktrace.
Call log:
[main] INFO me.dinowernli.grpc.polyglot.Main - Polyglot version: 1.4.0
[main] INFO me.dinowernli.grpc.polyglot.Main - Loaded configuration: default
[main] INFO me.dinowernli.grpc.polyglot.command.ServiceCall - Creating channel to: 127.0.0.1:10103
[main] INFO me.dinowernli.grpc.polyglot.oauth2.OauthCredentialsFactory - Using access token credentials
[grpc-default-executor-0] ERROR me.dinowernli.grpc.polyglot.grpc.ServerReflectionClient - Error in the server reflection rpc
io.grpc.StatusRuntimeException: UNIMPLEMENTED: unknown service grpc.reflection.v1alpha.ServerReflection
at io.grpc.Status.asRuntimeException(Status.java:543)
at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:395)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:426)
at io.grpc.internal.ClientCallImpl.access$100(ClientCallImpl.java:76)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:512)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$700(ClientCallImpl.java:429)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:544)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:52)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:117)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[main] INFO me.dinowernli.grpc.polyglot.command.ServiceCall - Using proto descriptors obtained from protoc
[main] INFO me.dinowernli.grpc.polyglot.command.ServiceCall - Creating dynamic grpc client
[main] INFO me.dinowernli.grpc.polyglot.command.ServiceCall - Making rpc with 1 request(s) to endpoint [127.0.0.1:10103]
[main] INFO me.dinowernli.grpc.polyglot.grpc.DynamicGrpcClient - Making unary call
[grpc-default-executor-0] INFO me.dinowernli.grpc.polyglot.io.LoggingStatsWriter - Got response message
{
"isOk": true
}
[grpc-default-executor-0] INFO me.dinowernli.grpc.polyglot.io.LoggingStatsWriter - Completed rpc with 1 response(s)
Blocked on a Bazel Windows release.
Many people use \n
as a separator for multiple JSON objects in a single file. I think a \n\n
would work for TextFormat of proto2
.
Please support mulit-stream in and out for unary and streaming gRPC services. Basically:
\n
or \n\n
as separators)0
)response status: <grpc status>: <grpc message>
to stderr
if not OK
stderr
This will make it easy to use polyglot in bash scripts by cat
-ing output in, and processing output automatically
text format seems to be dead and for proto3
jsonpb
seems to be the human-readable encoding of choice.
Please support JSON for proto3.
After building the JAR via $ bazel build src/main/java/me/dinowernli/grpc/polyglot
and running via java -jar polyglot.jar ...
, I get this error:
no main manifest attribute, in polyglot.jar
This is after a fresh Bazel installation on Mac with latest Java.
The README says:
$ bazel test ./src/...
However, that command gives the following error (on OS X Sierra, using bazel 0.5.2):
$ bazel test ./src/...
ERROR: Invalid package name './src': package name component contains only '.' characters.
INFO: Elapsed time: 0.076s
ERROR: Couldn't start the build. Unable to run tests.
Getting rid of the leading '.' fixes the issue:
$ bazel test src/...
INFO: Found 27 targets and 8 test targets...
INFO: Elapsed time: 10.092s, Critical Path: 8.70s
//src/test/java/me/dinowernli/grpc/polyglot/command:tests PASSED in 0.7s
//src/test/java/me/dinowernli/grpc/polyglot/config:tests PASSED in 0.8s
//src/test/java/me/dinowernli/grpc/polyglot/grpc:tests PASSED in 0.7s
//src/test/java/me/dinowernli/grpc/polyglot/integration:plain PASSED in 5.8s
//src/test/java/me/dinowernli/grpc/polyglot/integration:tls PASSED in 3.0s
//src/test/java/me/dinowernli/grpc/polyglot/io:tests PASSED in 0.9s
//src/test/java/me/dinowernli/grpc/polyglot/oauth2:tests PASSED in 0.5s
//src/test/java/me/dinowernli/grpc/polyglot/protobuf:tests PASSED in 0.8s
Executed 8 out of 8 tests: 8 tests pass.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
I realize I could submit a PR, but for a README tweak it seems like overkill...
Use proto3
JSON mapping for a config file so that OAuth2 parameters (client id, client secret, refresh token location, default paths of all generated .proto
files) would be placed in there :)
When extending authentication with MetadataCredentials, polyglot should support add custom metadata.
Hello.
Which IDE do you use for working this project?
Because I'm trying to use intellij with bazel plugin, but it seems failing in loading project, so want to get more about it.
If exchanging the refresh token fails, Polyglot correctly prints the error, but then just hangs rather than existing.
bratch: master
system: Mac
build cmd:
python src/tools/generate-intellij.py
bazel build src/main/java/me/dinowernli/grpc/polyglot
bash src/tools/example/run-server.sh
bash src/tools/example/call-command-example.sh
INFO: Analysed target //src/main/java/me/dinowernli/grpc/polyglot:polyglot (0 packages loaded).
INFO: Found 1 target...
Target //src/main/java/me/dinowernli/grpc/polyglot:polyglot up-to-date:
bazel-bin/src/main/java/me/dinowernli/grpc/polyglot/polyglot.jar
bazel-bin/src/main/java/me/dinowernli/grpc/polyglot/polyglot
INFO: Elapsed time: 0.156s, Critical Path: 0.00s
INFO: Build completed successfully, 1 total action
[main] INFO me.dinowernli.grpc.polyglot.Main - Polyglot version: 1.7.0+dev
[main] INFO me.dinowernli.grpc.polyglot.Main - Loaded configuration: default
[main] INFO me.dinowernli.grpc.polyglot.command.ServiceCall - Creating channel to: localhost:12345
[main] INFO me.dinowernli.grpc.polyglot.command.ServiceCall - Using proto descriptors obtained from protoc
[main] INFO me.dinowernli.grpc.polyglot.command.ServiceCall - Creating dynamic grpc client
[main] INFO me.dinowernli.grpc.polyglot.command.ServiceCall - Making rpc with 1 request(s) to endpoint [localhost:12345]
[main] INFO me.dinowernli.grpc.polyglot.grpc.DynamicGrpcClient - Making unary call
[main] INFO me.dinowernli.grpc.polyglot.grpc.DynamicGrpcClient - TEST: callUnary
[grpc-default-executor-1] ERROR me.dinowernli.grpc.polyglot.io.LoggingStatsWriter - Aborted rpc due to error
io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: deadline exceeded after 2980246236ns
at io.grpc.Status.asRuntimeException(Status.java:526)
at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:385)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:422)
at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:61)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:504)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$600(ClientCallImpl.java:425)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:536)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:102)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[main] WARN me.dinowernli.grpc.polyglot.Main - Caught top-level exception during command execution
java.lang.RuntimeException: Caught exception while waiting for rpc
at me.dinowernli.grpc.polyglot.command.ServiceCall.callEndpoint(ServiceCall.java:113)
at me.dinowernli.grpc.polyglot.Main.main(Main.java:70)
Caused by: java.util.concurrent.ExecutionException: io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: deadline exceeded after 2980246236ns
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
at com.google.common.util.concurrent.AbstractFuture$TrustedFuture.get(AbstractFuture.java:79)
at me.dinowernli.grpc.polyglot.command.ServiceCall.callEndpoint(ServiceCall.java:111)
... 1 more
Caused by: io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: deadline exceeded after 2980246236ns
at io.grpc.Status.asRuntimeException(Status.java:526)
at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:385)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:422)
at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:61)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:504)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$600(ClientCallImpl.java:425)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:536)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:102)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Caught exception while waiting for rpc
at me.dinowernli.grpc.polyglot.Main.main(Main.java:86)
Caused by: java.lang.RuntimeException: Caught exception while waiting for rpc
at me.dinowernli.grpc.polyglot.command.ServiceCall.callEndpoint(ServiceCall.java:113)
at me.dinowernli.grpc.polyglot.Main.main(Main.java:70)
Caused by: java.util.concurrent.ExecutionException: io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: deadline exceeded after 2980246236ns
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:476)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:455)
at com.google.common.util.concurrent.AbstractFuture$TrustedFuture.get(AbstractFuture.java:79)
at me.dinowernli.grpc.polyglot.command.ServiceCall.callEndpoint(ServiceCall.java:111)
... 1 more
Caused by: io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: deadline exceeded after 2980246236ns
at io.grpc.Status.asRuntimeException(Status.java:526)
at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:385)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:422)
at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:61)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:504)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$600(ClientCallImpl.java:425)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:536)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:102)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
I'm trying to create a PR about supporting grpc metadata
. I changed the config.proto
file but I'm unable to generate the ConfigProto.class
from my change.
What's the bazel task to do it?
When using the Any type when making a polyglot server call I hit InvalidProtocolBufferException: Cannot resolve type: type.googleapis.com/my.thingie
Example, the file I pipe to polyglot contains:
{"anyTypefield":{"@type":"type.googleapis.com/my.thingie","value":"Ig5mb29BY2NvdW50TmFtZQ=="}}
my.thingie
is well defined and within the --add_protoc_includes
.
The proto spec is:
message thingie
google.protobuf.Any anyTypefield = 1;
}
The type URL "type.googleapis.com/my.proto.type" is the one produced by (jsonpb.Marshaler{}).MarshalToString() under golang, so expecting this to be the correct type reference.
When I use my.thingie as a normal type (not within an Any field), polyglot is able to see and use the type without issues.
I would like to know how the output response can be written to file to convert to string format which can be manipulated for further testing.
Currently i am using polyglot as library in my project.
It would nice to have that clear how to invoke different kind of grpc calls like streaming client (: An example here might be useful.
Hi,
I am getting this response (array has the right size, but no fields at all):
{
"okResponse": {
"items": [{
}, {
}, {
}]
}
}
Using a .proto file like this (shown only relevant parts):
message ItemRpcResponse {
int64 cat = 1;
string s = 2;
int32 qt = 3;
google.protobuf.Int64Value c = 4;
google.protobuf.Int64Value sz = 5;
google.protobuf.Int64Value eqsz = 6;
}
message ItemsRpcResponse {
repeated ItemRpcResponse items = 1;
}
message WrapperRpcResponse {
oneof reply {
ItemsRpcResponse ok_response = 1;
RpcErrorResponse error_response = 2;
}
}
gRPC server is returning this:
ok_response {
items {
id: "8c84d174-9f17-439f-af12-dbdd16e2fff3"
cat: 1
s: "favorite"
qt: 1
}
items {
id: "08420bd1-6f08-438c-b973-6bdaac8c6498"
cat: 2
s: "favorite"
qt: 1
}
items {
id: "ec3f774d-ea36-4da1-a8fe-d522770b39a7"
cat: 3
s: "favorite"
qt: 1
}
}
I am calling Polyglot v 1.6.0 Main class inside my own application, using something like this:
final InputStream stream = new ByteArrayInputStream(jsonRequest.getBytes(StandardCharsets.UTF_8));
System.setIn(stream);
final String[] polyArgs = new String[] { "--command=call", "--endpoint=" + serverAddress + ":" + port,
"--full_method=" + serviceClass + "/" + serviceMethod, "--proto_discovery_root=" + sources.get(0),
"--add_protoc_includes=.", "--use_tls=" + tls, "--use_reflection=false", "--deadline_ms=30000" };
me.dinowernli.grpc.polyglot.Main.main(polyArgs);
Any ideas?
This will allow you to drop the need for a .proto
reading locally, and just rely on the remote server.
Some caching of the results may be useful.
https://github.com/grpc/grpc-go/tree/master/reflection
Not yet available for Java
grpc/grpc-java#651
I would like to use polyglot as a library e.g. ideally pulling in a dependency from central that has all the transitive dependencies declared in a pom.xml.
Currently, I am using a local repo with a modified release JAR with the JVM dependencies ripped out and a manually crafted pom.xml used to pull in the JVM dependencies but this is not ideal.
What is the current suggested approach for this use case? Or is polyglot only intended to be used as a standalone binary?
Thanks!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.