Comments (6)
Possibly of interest @gedw99 - ThreeDotsLabs/watermill-nats#13
I hope to have this passing tests in the not too distant future. But it is good enough to run the new example as is.
from watermill.
https://github.com/ThreeDotsLabs/watermill-nats/tree/master/pkg/nats Use jetstream but not the new public api
it has a stable api and also does things like ensure a topic / stream exists as well as better ordering guarantees
from watermill.
I have an issue open right now to support the new API. #373
Right now I am thinking it would make the most sense to introduce a whole new package in /pkg/jetstream
that uses the new API under the covers. This would be a good opportunity to simplify configuration etc as whats in the nats pkg is largely ported from the old STAN implementation.
Maybe it would be better to try and get the jetstream package added first and use upgrading the example as a test bed to suss out pain points etc....
from watermill.
hey @AlexCuse Your PR was merged and tagged at https://github.com/ThreeDotsLabs/watermill-nats/releases/tag/v2.0.2, so we can close this issue ?
from watermill.
No I don't think so @gedw99 - the example you mention is still on the old STAN-based v1.0.7 and would need some updates to the way publishers and subscribers are bootstrapped to use the v2 api (whether using nats or jetstream package).
Additionally its probably OK for use in an example but I would still consider the jetstream package experimental. There is a tricky race condition I have been chasing for awhile when I have time. It happens on almost every run in CI but is harder to replicate locally on decently spec'd hardware. I find if I run go clean -testcache && make jetstream_test_race
repeatedly I do get failures about on about 1/3 runs though. They are not consistent though, sometimes the run fails the race detector and other times its a panic for send on closed channel in the message handler. I've found a few issues that seem like they should fix it but I still end up with callbacks arriving late to the handler and making it through to the output channel send.
I am hoping the new Drain
functionality added to the nats client package in v1.32.0 will provide a path forward but I have not found it just yet. Sometimes I think it would be easier to just jump ahead to batching pull consumers since that would provide more control and that was what I really wanted the new package to support for maximal throughput, but the configuration changes a lot and I do think supporting Consume
well first is probably the right incremental step.
from watermill.
thanks for the explanation @AlexCuse
from watermill.
Related Issues (20)
- s
- MaxOutstandingMessages doesn't seem to be working
- follow-up handler will be broke while before msg handler error, what is the best practice for this situation?
- `msg.Nack` causes infinite loop with gochannel pubsub HOT 4
- Publish message processing delay in metrics Middleware?
- How to subscribe to all gochannel ? HOT 1
- Any plans to add support of RabbitMQ Streaming?
- Generated topic names for PubSub tests does not conform with Azure guidelines
- How to delete message after ack? HOT 1
- Context not being propagated through command bus HOT 2
- Is it possible to get google pubsub server generated unique message-id? HOT 3
- Data race on Publish for Google pubsub
- SQLite driver HOT 3
- Log custom fields inside message router
- Invalid publisher name in metrics
- Resume after shutdown always fetch last interrupted message again
- How to process a batch of received messages instead of a single piece?
- Can you output the message metadata to the logs?
- Only post messages after subscribing
- Watermill tests fail on windows
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from watermill.