Comments (11)
Implement CLUSTER SLOTS DENSE client capability.
CLUSTER SLOTS
is one of the mostly widely used command by clients for topology discovery. This would be a good feature addition and solve the large output response on fragmented slots. I think we can introduce the CLIENT CAPA <feature> <yes|no>
to enable/disable the feature rather than introducing another level of sub command.
from valkey.
I definitely feel that CLUSTER SLOTS had it almost right, and that CLUSTER SHARDS adds functionality that most clients don't need. Since most clients already use CLUSTER SLOTS, adopting this option should be relatively easy. However, note that we still need to sort the replicas in the current CLUSTER SLOTS implementation, not just compact the slots data.
from valkey.
... and if we do want to support DENSE, I think the slot ranges should be represented as a flat multi-range list [start1, end1, start2, end2, ...] rather than one list of starts and another list of ends. It's more intuitive (IMAO). For a single-range shard, it's identical to a non-DENSE reply.
The only reason I suggested the other approach is that it keeps the total number of arguments the same in all cases :) I'm not very convinced it's useful for them to be the same in specific edge cases.
Improve the docs. Let's show an example of CLUSTER SHARDS in RESP3, so you actually see the reply as a map, which is was designed for.
I honestly am not really convinced the map response is ideal for this case anymore. We are basically spending a bunch of bits in the network response to send information the client already knows. It's only useful if a human is reading it ad hoc.
Give it more time. CLUSTER SHARDS is still new. Clients want to support nodes running all still supported Redis OSS versions. The motivation for moving to CLUSTER SHARDS hasn't been strong enough, given they'll need to have a fallback anyway, but once all versions that don't support SHARDS are EoL, the situation will be different.
I don't agree because client developers (see Bar) aren't happy with it. I think we should be opinionated about the API. I think our original approach of moving to a new new command didn't really work as well as we thought it would.
from valkey.
I renamed it so it's easier to find when searching for it. :)
from valkey.
I think we should just do better marketing for CLUSTER SHARDS. It's very little extra information that the clients can simply ignore, so it's no real waste of bandwidth. It's also pretty trivial to parse, given the client can parse RESP, so that can hardly be a real problem.
Of the ideas suggested, I only support implementing defragmentation logic in valkey-cli, or at least making sure it's smart when rebalancing so that in minimizes creating fragmentation.
IMO we should instead focus on this:
- Improve the docs. Let's show an example of CLUSTER SHARDS in RESP3, so you actually see the reply as a map, which is was designed for.
- Explain the benefits, both in the docs of CLUSTER SHARDS and CLUSTER SLOTS.
- Give it more time. CLUSTER SHARDS is still new. Clients want to support nodes running all still supported Redis OSS versions. The motivation for moving to CLUSTER SHARDS hasn't been strong enough, given they'll need to have a fallback anyway, but once all versions that don't support SHARDS are EoL, the situation will be different.
- It may not be a frequent problem. To get very scattered slot distributions, you'd need to scale up multiple times (e.g. add one shard at a time) without paying attention to the slots. It's possible for cluster operators (not only valkey-cli) to be smarter and avoid creating fragmented slot ownershits. Always doubling or halving the number of shards is one way to completely avoid fragmentation, if done properly.
from valkey.
... and if we do want to support DENSE, I think the slot ranges should be represented as a flat multi-range list [start1, end1, start2, end2, ...]
rather than one list of starts and another list of ends. It's more intuitive (IMAO). For a single-range shard, it's identical to a non-DENSE reply.
from valkey.
OK, I'm convinced. 👍
from valkey.
I like CLUSTER SLOTS DENSE (or COMPACT) but I do think the slots should be a list of starts and end indices. That's the slot format used in CLUSTER SHARDS and CLUSTER ADDSLOTRANGE.
The idea about two lists that need to be iterated in parallel isn't great.
Another option we could add to CLUSTER SLOTS is NO-REPLICAS, that clients can use if they only care about primaries.
We should introduce these changes togerher with #298 and a possible format for push notifications id to use the same as one line of CLUSTER SLOTS DENSE. Clients can then implement both features at the same time.
from valkey.
@barshaul Also, the cluster slots output is sorted now, so it should be deterministic.
from valkey.
@barshaul Also, the cluster slots output is sorted now, so it should be deterministic.
Nice, thanks. Was sorting the replicas added in version 8.0?
However, it's still not ideal to use cluster slots because of the potential for very large outputs. While sorting the replicas reduces computation on the client side, it still exposes the client to delays due to large responses.
I believe we should establish guidelines on how clients should manage and update their topology, which will help us determine the best command strategy. Valkey can lead the way in standardizing best practices for OSS clients.
I can create a document outlining client design for handling cluster topology changes and share it with you for feedback. What do you think?
from valkey.
Nice, thanks. Was sorting the replicas added in version 8.0?
Yes, see #265. Replicas are ordered in CLUSTER SLOTS
.
However, it's still not ideal to use cluster slots because of the potential for very large outputs. While sorting the replicas reduces computation on the client side, it still exposes the client to delays due to large responses.
Yeah, that is sort of option three from the top of the issue. If the client sends CLIENT CAPA DENSE-SLOTS
, the cluster will send a more compact version of the cluster slots output, where the additional slots associated with the node are also sent. Clients needs to be smart enough to realize they were sent this dense output and handle it appropriately.
I can create a document outlining client design for handling cluster topology changes and share it with you for feedback. What do you think?
I think this would be great. You might consider starting it here, https://github.com/valkey-io/valkey-rfc, then we can merge it into the valkey doc repo when it's ready.
from valkey.
Related Issues (20)
- Blocking commands are marked as being nested
- [NEW] Enhanced Function API: off-thread JSON/MsgPack decoding of args and encoding of response HOT 1
- [Daily CI] Based line for Daily CI status (All test cases can passed)
- [BUG] `CLIENT CAPA redirect` may result in bouncing redirects during failover HOT 10
- [BUG]In big cluster Redis (Valkey) replica node is not able to sync since 6.2.11 HOT 2
- [NEW] Introduce server level network ingress / egress metrics for cluster bus. HOT 1
- [BUG] Replica can run cross-slot MULTI/EXEC, upon module hooks on primary HOT 2
- [Test failure] IO error with TLS
- [BUG] The result of the PING command does not match the documentation when using RESP3 HOT 1
- [BUG] A large negative number(e.g. -LONG_MAX) in HRANDFIELD/ZRANDMEMBER can easily cause Valkey to crash. HOT 2
- [NEW] Migrate from Redis 7.4.0 to Valkey 7.2.5 HOT 9
- [NEW] Introduce slot-level memory metrics
- [Test failure] Test issues with empty shard migrations
- [BUG] "CLIENT CAPA redirect" has no effect in scripts HOT 1
- [BUG] Cluster: Inconsistent MOVED/READONLY errors for read-only client
- [NEW] Introduce a new client capability for throttling
- [NEW] Add support for setting the group on a unix domain socket HOT 1
- Feature Request: Add Error Status Field for Diskless Syncs HOT 5
- [NEW] HTTP support for Valkey HOT 4
- [NEW] Support proxy protocol HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from valkey.