centrifugal / centrifuge Goto Github PK

I want to mock centrifugo for unit tests and i stucked on this error:
command ID required for commands with reply expected: map[client:7e9210d6-24ca-4508-b977-9c0a4fe0f61d user:]
Error can be reproduced with this server https://github.com/centrifugal/centrifuge/tree/master/examples/chat_json and this client https://github.com/centrifugal/centrifuge-go/blob/master/examples/private.go, with some correction of course. Dockerized centrifugo working perfectly. How to solve this error? Thanks.

Channels method of Engine (Broker part) returns a slice of all channels active throughout all Centrifugo nodes. This data is eventually consistent in case of Redis, not supported in Redis cluster case, not supported by most other PUB/SUB brokers, does not scale well for millions of channels thus can lead to error prone architectural decisions in applications that use Centrifuge or Centrifugo. I am considering to remove it in Centrifuge library Engine interface and in Centrifugo v3.

Maybe this sort of data should be extracted not via broker but over Centrifuge node cluster survey. But even in this case it won't scale well for business scenarios (should only be used as instrument of observability).

Hi,

I was wondering if you could help us with the following problem that we encounter when using Centrifuge classification: We wanted to classify our 16S rRNA reads sequence by Oxford Nanopore, and used Centrifuge and Kraken2 to do so while building a NCBI database based on all bacterial data and a 16S database based on the Silva database for both classification tools.
Whereas Kraken2 is able to classify most reads (>90%) with both bacterial NCBI and 16S database, Centrifuge only classifies comparably many reads if we take the bacterial NCBI database. In the case of the 16S database, very few reads (~10%) can be classified. I was wondering if you have any explanation for this effect?

Many thanks for your help!

Best regards,
Lara

This will let users to provide own token authentication and allow to remove hard dependency to github.com/dgrijalva/jwt-go. Also need to update current examples and include function that mimics current behaviour for simple migration.

And seems like it's time to add changelog file.

Let's close this when we reach at least 83%. Really appreciate any help with test coverage.

I have set history time limit to 1 hour in server.
When a client subscribes to channel, he gets the messages.
When he unsubscribes and again subscribes within 1 hour ,he will get some messages twice right.How to avoid reception of duplicate messages?

Sorry my bad english.
I am confused because poor of tutorial example in youtube.
I am confused between token for client and token for server centrifuge like this:

var centrifuge = new Centrifuge('ws://centrifuge.example.com/connection/websocket');

centrifuge.setToken(YOUR_TOKEN);

centrifuge.subscribe("news", function(message) {
    console.log(message);
});

centrifuge.connect();

How do i add token jwt in javascript client

var centrifuge = new Centrifuge('ws://centrifuge.example.com/connection/websocket');

centrifuge.subscribe("news", function(message) {
    console.log(message);
});

centrifuge.connect();

Does this support delivery ACK ?

My use case is that a user has many devices and I want to know that it was delivered to one of those devices AND cancel the delivery to their other devices.
So this use case is really only for notifications.

Can u abstract the method in /internal/auth/auth.go as an interface, so is it convenient for users to use this library to verify logic with their own auth sys? For example, access to the existing user system or redis's session.

In insecure mode it's possible to connect to centrifuge without specifying a token. But the same insecure option disables a lot of other security checks as well.

I suggest adding a separate flag for allowing connect without token (but still only allowing the connected user to log into anonym channels).

see:
https://github.com/centrifugal/centrifuge/blob/master/_examples/custom_broker/main.go#L137

I just get a browser list of files at localhost:8000

• sure nothing else is running :)

Reprod:

# nats
gnatsd

---

## redis
ln -sfv /usr/local/opt/redis/*.plist ~/Library/LaunchAgents  # Enable Redis autostart

launchctl load ~/Library/LaunchAgents/homebrew.mxcl.redis.plist  # Start Redis server via launchctl
# homebrew.mxcl.redis.plist contains reference to redis.conf file location: /usr/local/etc/redis.conf 

redis-server /usr/local/etc/redis.conf # Start Redis server using configuration file, Ctrl+C to stop
redis-cli ping # Check if the Redis server is running

launchctl unload ~/Library/LaunchAgents/homebrew.mxcl.redis.plist # Disable Redis autostart


---


# broker example
GO111MODULE=on
cd ./_examples/custom_broker && go run main.go

I have a few channels that my users join on login:

• lobby
• location:ABCD
• location:EFGH
• session:123

The only one that I can node.Publish() per the Redis example is "lobby" -> I believe because of the ":" in the name to namespace my channels.

Any chance I'm just doing this wrong?

is there possible to transfer binary file stream through ws/wss transport of centrifuge

At moment after Redis engine reconnects to Redis each shard resubscribes to all active channels on node (https://github.com/centrifugal/centrifuge/blob/master/engine_redis.go#L799). This must be changed in a way that each shard resubscribes only on its own subset of channels.

I'm curious if making the persistence backends swappable is on the roadmap for this library.

In my specific use case, it would be nice to use the lightweight Bolt for persistence while still utilizing centrifuge's pub sub system, unlike the way you have it set up with Redis now which seems to require using Redis's pub/sub system if you want persistence.

If there was an API for writing pluggable backends, I'd happily get started on one for bolt.

Hello ,How to pass authorisation headers from the client side when subscribing or publishing message?

Chat_json example doesn’t work, a0f7b99 commit.

# command-line-arguments cmd/main.go:94:3: unknown field 'HistoryMetaTTL' in struct literal of type centrifuge.MemoryEngineConfig cmd/main.go:151:92: e.Method undefined (type centrifuge.RPCEvent has no field or method Method) cmd/main.go:198:11: assignment mismatch: 2 variables but node.Publish returns 1 values

go.mod

`
go 1.13

require (
github.com/centrifugal/centrifuge v0.7.0
github.com/prometheus/client_golang v1.6.0
)
`

At the moment it is hard to run an example and then see whats going on.
So having the Admin GUI run at the same time is a good start.

Here is my suggestion for what we need to do:

1. Get Admin GUI running with an example.
2. Use chat-Protobuf as the example.

• BTW the Web gui does not work for the Chat_protobuf example.
• I tracked the issue down to caching. Had to turn off caching when dev tools is open just to keep working.

Optional but just putting it here for now :)
How to have all examples running at the same time:
https://github.com/pion/webrtc/tree/master/examples
Its just an idea. Maybe for the next iteration of this issue if we feel its helpful. Might not too.

I'm trying to implement WSS with https pages: https://stackoverflow.com/questions/9745249/html5-websocket-with-ssl

How can i do this?

To be consistent with client protocol.

Since private channel rules moved to Centrifugo we need a more generic way on client side to let client library know whether channel should be protected by token or not. I suppose a simple function that accepts channel and returns true/false. At moment client libraries operate with private_channel_prefix option.

Hi!

We have been using centrifuge in production for more than a year now, without any issues, so thanks for the awesome project. A couple weeks ago we started launching multiple websites using our api, and we need to separate them completely. Our api was desigend for this, so it was no issue, but we need to separate centrifuge channels as well.

My first idea was:

• user subscribes to some:channel
• in centrifuge, I add support for multiple site, and modify the channel name to some:<somesite>.channel (eg: based on a header, or some values encoded in the jwt token)
• when messages arrive on the some:<somesite>.channel, i send them out as some:channel

and although it would be best for us, I found no mechanism in centrifuge where a channel name is rewritten, so this approach would be very difficult for me to implement. If you think this is a good idea I would still love for it to happen, but until then I found a different approach.

My second idea:

• user subscibes to some:<somesite>.channel
• I validate that the user belongs to the site (eg: based on some values encoded in the jwt token)

I could almost implement this using centrifuge as a library (instead of using centrifugo server). I made a handler for the client.Subscribed event. The only thing I miss is to be able to access the JWT token info in the client.On().Subscribe handler. I made a fork exposing the client.info, with that it works perfectly.

Hello, I'm currently working on a use-case wherein:

1. a client should be able to publish to one or more channels (at the same time) and a subscriber should be able to subscribe to one or many channels (at the same time).
2. publish to all subscribers without a client publish. Something like a PUT call with the channel name and the message to publish.

Is this possible? Any directions for this use-case?

Also, I see two handers NewWebsocketHandler and NewSockjsHandler. is WebsocketHandler a pure websocket handler with no means to downgrade to other methods where WS is not supported? So, if thats the case do you suggest using SockjsHandler in these scenarios?

At moment Centrifuge does not allow to subscribe on the same channel twice inside single connection. This is ok for most use cases but leads to difficulties and non very clear API restrictions on client side (users can only create one subscription to the same channel).

I believe introducing incremental subscription ID should solve this.

Hello!
I implement the user authentication process completely with WebSockets.
As I understand it, changing Client.user now is possible only at node.On().ClientConnecting method (by return Credentials at ConnectReply struct).
But Singin reguest is possible only after the Connecting event. And for add UserID to Client.user i have emit Disconnect/Connect events. I would like to avoid this.
Is it possible to add Client.SetUserID method like this:

// SetUserID set user ID.
func (c *Client) SetUserID(userID string) {
	c.mu.Lock()
	c.user = userID
	c.mu.Unlock()
}

P.S.: I have studied the source code and did not see difficulty when applying sharding with Client.SetUserID method.

At moment if node unsubscribes from Redis channel (last client leaves channel) and an error happens during unsubscribe request to Redis then subscription to channel can stay alive. This does not affect general behaviour but can lead to stale subscriptions and receiving non-actual messages over PUB/SUB which are then dropped on node level. It would be nice to clean up such subscriptions. One idea is to have a sort of unsubscribe unbounded queue which is consumed by workers to reliably process unsubscriptions.

try with

git clone https://github.com/centrifugal/centrifuge.git
dep ensure 

with no luck(
it clones to centrifuge but depend from centrifugal/centrifuge
Must i clone all projects? Witch branches?

Hello,
Maybe will be better use https://github.com/gobwas/ws for websockets?

Should be protected by enabled level check to avoid unnecessary allocations.

I'm working on an application that provides updates to logged in users via web-socket. I don't even want the connection to be allowed for non-logged-in users and the way this was solved previously was to use the Sec-Websocket-Protocol header in the original request (setting that to a JWT).

I've added the same thing with centrifuge and added the UserID to centrifuge just fine, but the issue is that I can't make centrifuge reply with the protocol header set, which results in errors like the following:

WebSocket connection to 'ws://localhost:5000/v1/web-sockets-pubsub' failed: Error during WebSocket handshake: Sent non-empty 'Sec-WebSocket-Protocol' header but no response was received

Firefox seems fine with it and everything works well there.

As far as I know (from here) there's no other way to add auth, apart from adding the JWT to the URL but for some reason I'm trying to avoid it. I don't have a clear argument against it but something doesn't seem right to put tokens in the URL. Server and front-end are on different subdomains.

It would be great to have the option to add a customer gorilla *websocket.Upgrader to NewWebsocketHandler(), maybe by passing it as an option to WebsocketConfig. If it's null, the default is used, otherwise the given one.

What do you think?

Or am I thinking about this wrong, is there another way to handle authentication that I'm missing?

Currently Centrifuge/Centrifugo can show instant client presence information for channels. Though it's not effective for cases when client only subscribed on personal channel but still needs info about other users online status. Maybe it's reasonable to add new engine method:

UsersOnline(userIDs []string) (map[string]*UserOnlineInfo, error)

Where UserOnlineInfo is sth like this:

type UserOnlineInfo struct {
    Updated uint32
}

It will return approximation (in a configured window) of user state based on their presence in channels where new user_online option set (or maybe it should just be top-level option without any relation to channels but this will also require adding new method AddUserOnline to handle cases when client not subscribed to any channel which is pretty common situation in Centrifuge library case). Maybe this should be Credentials flag (or connection token claim in Centrifugo case).

Information should be sharded by userID, this is different from how other things sharded at moment (by channel).

This will allow apps to ask state of certain users. I think this can only be available as part of server API and not exposed to client protocol. Apps that use Centrifugo will be able to call it from backend, apps that use Centrifuge lib will be able to do whatever they want - for example make it one of RPC calls.

This is just an idea at moment, maybe it's not worth adding - but let's think about it.

With v0.8.0, if PublishOnHistoryAdd is set to false (it was set to false by mistake), i keep getting disconnected with reason insufficient state. I think this is because the offset magic:

if ARGV[4] ~= '' then
	local payload = "__" .. offset .. "__" .. ARGV[1]
	redis.call("publish", ARGV[4], payload)

is missing from engine_redis.Publish

Hello, is there a way to add data to client published message?

type Disconnect struct {
	// Code is disconnect code.
	Code int `json:"-"`
	// Reason is a short description of disconnect.
	Reason string `json:"reason"`
	// Reconnect gives client an advice to reconnect after disconnect or not.
	Reconnect bool `json:"reconnect"`
}

document, we can return a custom broken link error
https://centrifugal.github.io/centrifugo/server/proxy/#return-custom-disconnect

But the code that actually returns forever is 0 because JSON data is ignored here.
Why should our Code field ignored from json ?

Hi there,

a recent upgrade has broken my sockjs handler....

--

panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x190 pc=0x97f716]

goroutine 84 [running]:
github.com/centrifugal/centrifuge.(*Node).NotifyShutdown(...)
	[REMOVED]go/src/github.com/centrifugal/centrifuge/node.go:222
github.com/centrifugal/centrifuge.(*SockjsHandler).sockJSHandler.func1(0xc73d00, 0xc0005b66e0, 0xc0001fec40)
	[REMOVED]go/src/github.com/centrifugal/centrifuge/handler_sockjs.go:170 +0x186
created by github.com/centrifugal/centrifuge.(*SockjsHandler).sockJSHandler
	[REMOVED]go/src/github.com/centrifugal/centrifuge/handler_sockjs.go:163 +0xcc

At moment once set the sequence number will live forever in Redis. Looks like we can set a reasonable time when this key can safely expire if the channel have not been active for a long time - for example in a time that is much bigger than channel history retention period.

@FZambia Hi
i upgrade centrifuge to v0.9.0.
Why doesn't "redis presence data" renew automatically
I don't know what to do now.

Now when go 1.12 is out, I suppose we can experiment with go mod support for centrifuge package.

Hi,A client when subscribing to a channel needs to get last n messages published in that channel.
How can I handle this use case?Any reference for that?
centrifuge-js is the client library I am using.

Hello,

I'm wondering if you are planning (or maybe it's already implemented) to add a support for redis TLS. Looks like it's supported by redigo.

I think this would be really valuable feature.

I'm willing to make a PR for this feature.

After receiving credentials from client using node.ClientConnecting() handler,how to prevent an unauthorized client from getiting connected to the server node

According to specifications sub field is exactly for unique user identificator. Many cloud providers use this claim for this. Some examples:

• https://auth0.com/docs/tokens/id-token
• http://openid.net/specs/openid-connect-core-1_0.html#IDToken
• https://docs.aws.amazon.com/cognito/latest/developerguide/amazon-cognito-user-pools-using-tokens-with-identity-providers.html

The initial motivation described here.

This will reduce performance of Presence and History methods, but I believe will make library more safe and will incapsulate dependency to Protobuf types which is a good thing anyway.

At the moment we give access to string reason, but not to disconnect code. I suppose we should expose access to the code too.

At moment Centrifuge library follows Centrifugo in terms of subscribe and publish permissions. While this is fine for Centrifugo - having namespaces and predefined channel rules inside a library can be a bit confusing for users.

What I want to achieve with this library is make it only a reasonable wrapper over client-server protocol without many opinionated logic on top of it. I.e. give developers a possibility to define their own channel rules and permission checks.

Another confusing thing in my opinion is that if we don't define On().Subscribe() handler then client will be able to subscribe on all channels. I believe the default behaviour should be: not allow subscription at all if subscribe handler not set for connection.

That said I'd want to decouple namespace and channel permission logic from Centrifuge library core since it adds a very opinionated layer to library. Maybe this process will also require decoupling JWT parsing logic too.

Good time of day.
Is it possible to add StreamPosition to a PublishEvent?

client.On().Publish(func(e centrifuge.PublishEvent) centrifuge.PublishReply {
...
offset := e.StreamPosition.Offset
...
}

At moment Centrifuge library can recover missed messages automatically. Also it has API to get all publications in history limited by size and TTL.

I want to try extending history API to give a possibility to paginate over stream from certain position with limit.

Steps to make this possible:

1. Integrate with Redis Streams
2. Get rid of Seq and Gen uint32 fields and replace it with single uint64 Offset field. This should simlify working with API from outside. Javascript can't work with numbers greater than 2**53-1 but my calculations showed that if we publish 1000 messages per second to channel than we have about 285616 years until we hit that limit
3. I also want to fully get rid of Protobuf protocol aliases available as part of public API at moment in protocol.go

How is node health reported? Looked around in the code and couldn't see anything obvious.

centrifugo's api has no healthcheck endpoint, while investigating how to create one I noticed that health would need to be reported by each node. Is that correct?

To start simple we could just ping the configured engine, then report "ok" or "not ok".

I'm willing to send a PR, but would probably need a little guidance to get me started.

Great project btw.
Thanks!