spotify / async-google-pubsub-client Goto Github PK
View Code? Open in Web Editor NEW[SUNSET] Async Google Pubsub Client
License: Apache License 2.0
async-google-pubsub-client's People
Contributors
Stargazers
Watchers
Forkers
piyushnarang skinzer pettermahlen robin-anil rtvt123 reidwilbur telpr2 gamoshi-archives vad-babushkin sobolsigizmund lgustav walternate narape rwstephenson pinux-studio almanar alexgolubev badrock sderosiaux powerspace oraclewalid q225zhan leblonk bilisie olibye jwiklund dinduks sanjay007 rickardzwahlen pdeglopper pedro-carneiro morristech santhh tool-recommender-bot javifalces xugenfa zhishengzhangasync-google-pubsub-client's Issues
Metrics on bytes consumed/published
For the time being it seems like it's difficult or impossible to break apart Pub/Sub costs by stream or subscription and so it'd be useful to measure at least roughly client-side. While we can measure bytes written into the client, I don't see a way to measure bytes actually sent post-compression. Our streams vary greatly in compressibility, so this could lead to misleading results. It's also unclear how request overhead is factored in.
Ideally we'd like to measure the following:
- Total raw bytes sent/consumed
- Bytes sent/consumed, rounding batches up to the Pub/Sub minimum request size.
- Count of requests under/over minimum request size.
I can certainly understand reluctance to tie the project to any external metrics library, but perhaps exposing raw data via callbacks or metadata on the returned object would be possible? I'd be happy put together a PR for consideration if this seems like a possibility.
Thanks much, this lib has been a great tool.
Resource starvation at low message volumes
We've run into a bug where this client starves itself on resources. The puller threads never gives the acker thread a chance to run and vice versa. We've worked around it by setting the concurrency to 1. This only seems to be an issue when message throughput is very low.
I'm reporting this issue secondhand - I believe @lndbrg discovered it.
Error trying to wait for a completable future
Hi 👋 - thanks for a great project!
We have a use-case where we would like in some situations to wait for the CompletableFuture when publishing, so that we can be sure that pubsub is working as expected.
When we do this we get the following error:
java.util.concurrent.CompletionException: com.fasterxml.jackson.databind.JsonMappingException: Can not construct instance of com.spotify.google.cloud.pubsub.client.PublishResponse: abstract types either need to be mapped to concrete types, have custom deserializer, or contain additional type information
Here is some sample code to reproduce:
import com.spotify.google.cloud.pubsub.client.Message;
import com.spotify.google.cloud.pubsub.client.Publisher;
import com.spotify.google.cloud.pubsub.client.Pubsub;
import java.util.concurrent.CompletableFuture;
public class PubsubProblem {
public static void main(String[] args) {
Publisher publisher = Publisher.builder()
.pubsub(Pubsub.create())
.project("test-project")
.build();
Message msg = Message.ofEncoded("hello world");
CompletableFuture<Void> result = publisher.publish("topic", msg)
.thenAccept(x -> System.out.println(x))
.exceptionally(t -> {
t.printStackTrace();
return null;
});
}
}
Is there a way around this? We could be missing something obvious. We are using pubsub version 1.27.
Thanks in advance!
Full stack trace:
java.util.concurrent.CompletionException: com.fasterxml.jackson.databind.JsonMappingException: Can not construct instance of com.spotify.google.cloud.pubsub.client.PublishResponse: abstract types either need to be mapped to concrete types, have custom deserializer, or contain additional type information
at [Source: [B@6a7df875; line: 1, column: 1]
at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:647)
at java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:632)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
at com.spotify.google.cloud.pubsub.client.Publisher$TopicQueue.lambda$null$1(Publisher.java:405)
at java.util.ArrayList.forEach(ArrayList.java:1249)
at com.spotify.google.cloud.pubsub.client.Publisher$TopicQueue.lambda$sendBatch$3(Publisher.java:405)
at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
at com.spotify.google.cloud.pubsub.client.PubsubFuture.fail(PubsubFuture.java:291)
at com.spotify.google.cloud.pubsub.client.PubsubFuture.lambda$wrap$0(PubsubFuture.java:278)
at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
at com.spotify.google.cloud.pubsub.client.PubsubFuture.fail(PubsubFuture.java:291)
at com.spotify.google.cloud.pubsub.client.Pubsub$1.onCompleted(Pubsub.java:827)
at com.spotify.google.cloud.pubsub.client.Pubsub$1.onCompleted(Pubsub.java:776)
at com.ning.http.client.providers.netty.future.NettyResponseFuture.getContent(NettyResponseFuture.java:177)
at com.ning.http.client.providers.netty.future.NettyResponseFuture.done(NettyResponseFuture.java:214)
at com.ning.http.client.providers.netty.handler.HttpProtocol.finishUpdate(HttpProtocol.java:194)
at com.ning.http.client.providers.netty.handler.HttpProtocol.handleChunk(HttpProtocol.java:451)
at com.ning.http.client.providers.netty.handler.HttpProtocol.handle(HttpProtocol.java:474)
at com.ning.http.client.providers.netty.handler.Processor.messageReceived(Processor.java:88)
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleUpstream(ChunkedWriteHandler.java:142)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at org.jboss.netty.handler.codec.http.HttpContentDecoder.messageReceived(HttpContentDecoder.java:132)
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:459)
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:536)
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435)
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at org.jboss.netty.handler.codec.http.HttpClientCodec.handleUpstream(HttpClientCodec.java:92)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Can not construct instance of com.spotify.google.cloud.pubsub.client.PublishResponse: abstract types either need to be mapped to concrete types, have custom deserializer, or contain additional type information
at [Source: [B@6a7df875; line: 1, column: 1]
at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:261)
at com.fasterxml.jackson.databind.DeserializationContext.instantiationException(DeserializationContext.java:1420)
at com.fasterxml.jackson.databind.DeserializationContext.handleMissingInstantiator(DeserializationContext.java:1011)
at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserialize(AbstractDeserializer.java:149)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3789)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2920)
at com.spotify.google.cloud.pubsub.client.Json.read(Json.java:47)
at com.spotify.google.cloud.pubsub.client.Pubsub$1.onCompleted(Pubsub.java:825)
... 44 more
Puller JSON deserialization appears broken
ERROR com.spotify.google.cloud.pubsub.client.Puller - Pull failed
com.fasterxml.jackson.databind.JsonMappingException: Can not construct instance of com.spotify.google.cloud.pubsub.client.PullResponse: abstract types either need to be mapped to concrete types, have custom deserializer, or contain additional type information
at [Source: java.util.zip.GZIPInputStream@46a89674; line: 1, column: 1]
at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:270)
at com.fasterxml.jackson.databind.DeserializationContext.instantiationException(DeserializationContext.java:1456)
at com.fasterxml.jackson.databind.DeserializationContext.handleMissingInstantiator(DeserializationContext.java:1012)
at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserialize(AbstractDeserializer.java:149)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3798)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2908)
at com.spotify.google.cloud.pubsub.client.Json.read(Json.java:51)
at com.spotify.google.cloud.pubsub.client.Pubsub.lambda$requestJavaNet$8(Pubsub.java:908)
at com.spotify.google.cloud.pubsub.client.Pubsub$$Lambda$37/1565740893.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Connectivity Issue : Getting UnknownHostException: pubsub.googleapis.com
We are getting below exception while using the async-google-pubsub-client library to connect application with google cloud pub-sub.We have raised this issue to google cloud team and got a response to raise an issue in the libraray owners.
We are using compile group: 'com.spotify', name: 'async-google-pubsub-client', version: '1.31' .
Any lead from your end would be appreciated.
java.util.concurrent.ExecutionException: java.net.UnknownHostException: pubsub.googleapis.com
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
at com.google.common.util.concurrent.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:142)
at com.vuclip.vrss.queue.pubsub.service.impl.PubSubServiceImpl.publishXcodingMessage(PubSubServiceImpl.java:159)
at com.vuclip.vrss.service.impl.XcodingServiceImpl.pushDataIntoPubSub(XcodingServiceImpl.java:104)
at com.vuclip.vrss.service.impl.XcodingServiceImpl.lambda$postXcodingRequest$0(XcodingServiceImpl.java:61)
at java.util.ArrayList.forEach(ArrayList.java:1259)
at com.vuclip.vrss.service.impl.XcodingServiceImpl.postXcodingRequest(XcodingServiceImpl.java:48)
at com.vuclip.vrss.controller.v1.XapiController.postXcodingRequest(XapiController.java:42)
at sun.reflect.GeneratedMethodAccessor218.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:209)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:102)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:877)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:783)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:991)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:925)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:974)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:877)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:661)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:851)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:198)
at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:176)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.boot.actuate.web.trace.servlet.HttpTraceFilter.doFilterInternal(HttpTraceFilter.java:90)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:109)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:155)
at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:123)
at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:108)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:493)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:650)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:800)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:800)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1471)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.net.UnknownHostException: pubsub.googleapis.com
at java.net.InetAddress$CachedAddresses.get(InetAddress.java:764)
at java.net.InetAddress.getAllByName0(InetAddress.java:1291)
at java.net.InetAddress.getAllByName(InetAddress.java:1144)
at java.net.InetAddress.getAllByName(InetAddress.java:1065)
at java.net.InetAddress.getByName(InetAddress.java:1015)
at com.ning.http.client.NameResolver$JdkNameResolver.resolve(NameResolver.java:28)
at com.ning.http.client.providers.netty.request.NettyRequestSender.remoteAddress(NettyRequestSender.java:358)
at com.ning.http.client.providers.netty.request.NettyRequestSender.connect(NettyRequestSender.java:369)
at com.ning.http.client.providers.netty.request.NettyRequestSender.sendRequestWithNewChannel(NettyRequestSender.java:283)
at com.ning.http.client.providers.netty.request.NettyRequestSender.sendRequestWithCertainForceConnect(NettyRequestSender.java:142)
at com.ning.http.client.providers.netty.request.NettyRequestSender.sendRequest(NettyRequestSender.java:117)
at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.execute(NettyAsyncHttpProvider.java:87)
at com.ning.http.client.AsyncHttpClient.executeRequest(AsyncHttpClient.java:506)
at com.spotify.google.cloud.pubsub.client.Pubsub.request(Pubsub.java:785)
at com.spotify.google.cloud.pubsub.client.Pubsub.post(Pubsub.java:723)
at com.spotify.google.cloud.pubsub.client.Pubsub.publish0(Pubsub.java:525)
at com.spotify.google.cloud.pubsub.client.Pubsub.publish(Pubsub.java:496)
at com.spotify.google.cloud.pubsub.client.Pubsub.publish(Pubsub.java:483)
at com.vuclip.vrss.queue.pubsub.service.impl.PubSubServiceImpl.publishXcodingMessage(PubSubServiceImpl.java:157)
... 73 more
The type java.util.concurrent.CompletableFuture cannot be resolved. Java 1.7
The type java.util.concurrent.CompletableFuture cannot be resolved. Java 1.7
publisher.close()/publisher.closeFuture().get() should flush client
at least that's what i would expect :) i just modified testPullBatch in PubsubIT. should this test pass?
https://gist.github.com/bitkid/0ccc45d8d58dcf2b1ca88f9f9ed9dec6
Run integration tests using pubsub emulator
Future plans for async-google-pubsub-client?
Hey @danielnorberg or anyone else maintaining this library, do you have plans to continue to maintain it in the future?
Recently Google has made a pubsub client with async operations available in com.google.cloud:gcloud-java-pubsub.
This client has a Puller-like class in it's MessageConsumer concept, which will continuously pull messages from a subscription and update their ack timeouts until your code is finished processing the message, with some options to tune how many messages to queue up and how many threads to use in processing messages concurrently.
gcloud-java-pubsub uses the gRPC API for Cloud PubSub and also offers some async methods for each operation, for instance Future<List<String>> publishAsync(String topic, Iterable<Message> messages) or Future<Iterator<ReceivedMessage>> pullAsync(String subscription, int maxMessages).
(One annoyance is that the library returns naked Futures rather than ListenableFutures, so they have to be adapted if you want to use ListenableFuture or CompletableFuture)
One thing that seems to be lacking from gcloud-java-pubsub so far is an analogue to the Publisher class here that offers batching and internal queuing for publishing messages.
Given that Google has been advancing their client for pubsub, I am curious to see how much of a use case there is still for this custom client.
Enable access to create a custom subscription
I was having issues with subscriptions created with the createSubscription function, where messages weren't being acknowledged on time. I would like to be able to create a subscription with a custom acknowledgement deadline. I believe createSubscription, defined on line 399 of Pubsub.java should be made public to enable use of the subscription builder, allowing custom acknowledgement deadlines to be set.
Pubsub client breaks with Unrecognized field "publishTime"
java.util.concurrent.ExecutionException: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "publishTime" (class com.spotify.google.cloud.pubsub.client.MessageBuilder$Value), not marked as ignorable (3 known properties: "data", "messageId", "attributes"])
at [Source: [B@1b75edf; line: 9, column: 8] (through reference chain: com.spotify.google.cloud.pubsub.client.Value["receivedMessages"]->java.util.ArrayList[0]->com.spotify.google.cloud.pubsub.client.Value["message"]->com.spotify.google.cloud.pubsub.client.Value["publishTime"])
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
at com.tocktix.pubsub.GooglePubSub$TopicPoller.run(GooglePubSub.java:179)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "publishTime" (class com.spotify.google.cloud.pubsub.client.MessageBuilder$Value), not marked as ignorable (3 known properties: "data", "messageId", "attributes"])
at [Source: [B@1b75edf; line: 9, column: 8] (through reference chain: com.spotify.google.cloud.pubsub.client.Value["receivedMessages"]->java.util.ArrayList[0]->com.spotify.google.cloud.pubsub.client.Value["message"]->com.spotify.google.cloud.pubsub.client.Value["publishTime"])
at com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:51)
at com.fasterxml.jackson.databind.DeserializationContext.reportUnknownProperty(DeserializationContext.java:817)
at com.fasterxml.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:958)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1324)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperties(BeanDeserializerBase.java:1278)
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:452)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1073)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:295)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:142)
at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:523)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeWithErrorWrapping(BeanDeserializer.java:461)
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:381)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1073)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:295)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:142)
at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:245)
at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:25)
at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:523)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeWithErrorWrapping(BeanDeserializer.java:461)
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:381)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1073)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:295)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:142)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3564)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2673)
at com.spotify.google.cloud.pubsub.client.Json.read(Json.java:42)
at com.spotify.google.cloud.pubsub.client.Pubsub$1.onCompleted(Pubsub.java:782)
at com.spotify.google.cloud.pubsub.client.Pubsub$1.onCompleted(Pubsub.java:733)
at com.ning.http.client.providers.netty.future.NettyResponseFuture.getContent(NettyResponseFuture.java:177)
at com.ning.http.client.providers.netty.future.NettyResponseFuture.done(NettyResponseFuture.java:214)
at com.ning.http.client.providers.netty.handler.HttpProtocol.finishUpdate(HttpProtocol.java:194)
at com.ning.http.client.providers.netty.handler.HttpProtocol.handleChunk(HttpProtocol.java:451)
at com.ning.http.client.providers.netty.handler.HttpProtocol.handle(HttpProtocol.java:474)
at com.ning.http.client.providers.netty.handler.Processor.messageReceived(Processor.java:88)
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleUpstream(ChunkedWriteHandler.java:142)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at org.jboss.netty.handler.codec.http.HttpContentDecoder.messageReceived(HttpContentDecoder.java:132)
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:459)
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:536)
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435)
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at org.jboss.netty.handler.codec.http.HttpClientCodec.handleUpstream(HttpClientCodec.java:92)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
400 bad request
Hello: Thanks for making this project open for everyone! I am getting this error message when publishing to a topic. Payload is relatively big . Any idea what may be causing this?
Message Size: 1257
java.util.concurrent.ExecutionException: com.spotify.google.cloud.pubsub.client.RequestFailedException: 400 Bad Request
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
I am using this method below.
final List messageIds = pubsub.publish(PROJECT_ID, this.topic, messages).get();
Get access token error with appengine server
I am trying to use this library with google app engine server (java 8) , when I am running the code locally the code run normally but when I make deploy online I got this error :
Uncaught exception from servlet java.lang.RuntimeException: Failed to get access token at com.spotify.google.cloud.pubsub.client.Pubsub.<init>(Pubsub.java:160) at com.spotify.google.cloud.pubsub.client.Pubsub.<init>(Pubsub.java:82) at com.spotify.google.cloud.pubsub.client.Pubsub$Builder.build(Pubsub.java:975) at com.spotify.google.cloud.pubsub.client.Pubsub.create(Pubsub.java:938)
Create topic return error 400 Bad Request
Hi
i was trying version 1.28 of your library with a local env of pubsub
i started pubsub with command:
gcloud beta emulators pubsub start
And tried to create a topic with this code:
i put this code:
public class PubSubTest {
public static void main(String[] args) throws InterruptedException, ExecutionException, URISyntaxException, IOException{
Pubsub pubsub = Pubsub.builder().uri(new URI("http://localhost:8085/v1/")).credential(GoogleCredential.getApplicationDefault()).build();
Topic topic = pubsub.createTopic("project", "the-topic").join();
}
}
But it return this error:
Exception in thread "main" 11:24:31.932 [New I/O worker #1] DEBUG com.ning.http.client.providers.netty.channel.ChannelManager - Adding key: http://localhost:8085 for channel [id: 0x1cf05dc3, /127.0.0.1:59374 => localhost/127.0.0.1:8085]
java.util.concurrent.CompletionException: com.spotify.google.cloud.pubsub.client.RequestFailedException: 400 Bad Request
at java.util.concurrent.CompletableFuture.join(CompletableFuture.java:2303)
at com.miei.test.PubSubTest.main(PubSubTest.java:21)
Caused by: com.spotify.google.cloud.pubsub.client.RequestFailedException: 400 Bad Request
at com.spotify.google.cloud.pubsub.client.Pubsub$1.onStatusReceived(Pubsub.java:802)
at com.ning.http.client.providers.netty.handler.HttpProtocol.exitAfterHandlingStatus(HttpProtocol.java:372)
at com.ning.http.client.providers.netty.handler.HttpProtocol.handleHttpResponse(HttpProtocol.java:429)
at com.ning.http.client.providers.netty.handler.HttpProtocol.handle(HttpProtocol.java:470)
at com.ning.http.client.providers.netty.handler.Processor.messageReceived(Processor.java:88)
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleUpstream(ChunkedWriteHandler.java:142)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at org.jboss.netty.handler.codec.http.HttpContentDecoder.messageReceived(HttpContentDecoder.java:108)
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:459)
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:536)
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435)
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at org.jboss.netty.handler.codec.http.HttpClientCodec.handleUpstream(HttpClientCodec.java:92)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
On gcloud console i found this error:
[pubsub] This is the Google Pub/Sub fake.
[pubsub] Implementation may be incomplete or differ from the real system.
[pubsub] giu 09, 2017 11:22:38 AM com.google.cloud.pubsub.testing.v1.Main main
[pubsub] INFORMAZIONI: IAM integration is disabled. IAM policy methods and ACL checks are not supported
[pubsub] giu 09, 2017 11:22:38 AM io.grpc.internal.ManagedChannelImpl <init>
[pubsub] INFORMAZIONI: [ManagedChannelImpl@4a891e7e] Created with target localhost:8085
[pubsub] giu 09, 2017 11:22:38 AM io.gapi.emulators.netty.NettyUtil applyJava7LongHostnameWorkaround
[pubsub] INFORMAZIONI: Unable to apply Java 7 long hostname workaround.
[pubsub] giu 09, 2017 11:22:38 AM com.google.cloud.pubsub.testing.v1.Main main
[pubsub] INFORMAZIONI: Server started, listening on 8085
[pubsub] giu 09, 2017 11:24:06 AM io.gapi.emulators.grpc.GrpcServer$3 operationComplete
[pubsub] INFORMAZIONI: Adding handler(s) to newly registered Channel.
[pubsub] giu 09, 2017 11:24:06 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
[pubsub] INFORMAZIONI: Detected non-HTTP/2 connection.
[pubsub] giu 09, 2017 11:24:06 AM io.gapi.emulators.grpc.HttpAdapter$UnaryMethodHandler handle
[pubsub] AVVERTENZA: Failed to convert request to message: Field google.pubsub.v1.Topic.name has already been set.
[pubsub] com.google.protobuf.InvalidProtocolBufferException: Field google.pubsub.v1.Topic.name has already been set.
[pubsub] at io.gapi.emulators.grpc.JsonFormat$ParserImpl.mergeField(JsonFormat.java:1383)
[pubsub] at io.gapi.emulators.grpc.JsonFormat$ParserImpl.mergeMessage(JsonFormat.java:1239)
[pubsub] at io.gapi.emulators.grpc.JsonFormat$ParserImpl.merge(JsonFormat.java:1197)
[pubsub] at io.gapi.emulators.grpc.JsonFormat$ParserImpl.merge(JsonFormat.java:1079)
[pubsub] at io.gapi.emulators.grpc.JsonFormat$Parser.merge(JsonFormat.java:283)
[pubsub] at io.gapi.emulators.grpc.HttpJsonAdapter.merge(HttpJsonAdapter.java:61)
[pubsub] at io.gapi.emulators.grpc.HttpAdapter$UnaryMethodHandler.handle(HttpAdapter.java:466)
[pubsub] at io.gapi.emulators.grpc.HttpAdapter.handleRequest(HttpAdapter.java:165)
[pubsub] at io.gapi.emulators.netty.HttpHandler.channelRead(HttpHandler.java:52)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
[pubsub] at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
[pubsub] at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
[pubsub] at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:293)
[pubsub] at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:267)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:39)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:364)
[pubsub] at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
[pubsub] at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:418)
[pubsub] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
[pubsub] at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
[pubsub] at java.lang.Thread.run(Thread.java:745)
[pubsub]
[pubsub] giu 09, 2017 11:24:06 AM io.gapi.emulators.netty.HttpHandler$1 onError
[pubsub] INFORMAZIONI: Exception when handling request: INVALID_ARGUMENT: Payload isn't valid for request.
i've also tried to create the topic with a curl -XPUT http://localhost:8085/v1/projects/project/topics/the-topic and topic is correctly created
Debugging i've found that "put" request "com.spotify.google.cloud.pubsub.client.Pubsub.request(String, HttpMethod, String, Class, Object)" for a topic creation sends a payload that is different from "NO_PAYLOAD" , so i've tried to change runtime value of payload param to "NO_PAYLOAD" and with this configuration topic creation works perfectly
support NoCredentials for local testing
the google pubsub client does not need a valid account for local tests:
https://github.com/GoogleCloudPlatform/google-cloud-java/blob/master/TESTING.md#testing-code-that-uses-pubsub
it would be great if this would be possible with the spotify client too
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.