We are facing an issue with pretty big payloads coming from published mutations through Absinthe (essentially nested Ecto schema structs from database query results pushed through pubsub). Recently, the issue forced us to increase default redis output buffer limits (32 MB per client/node) to avoid waves of dropped connections due to rapid spike of larger payloads filling them faster than the clients are able to empty them.

Although reducing the size of the query results is definitely something to address eventually, one thing that would be really helpful to us is being able to compress the payloads before submission. :erlang.term_to_binary/2 provides compressed option, which seems like a good fit for the purpose (per initial tests, level 1 compression allows for 10x payload size reduction with relatively low cost in CPU time).

Would it be deemed useful to add such option to this adapter? Here's a proposal of the change that could allow setting this (defaults to no compression):

master...sketch-hq:sketch-hq/add-compression

My configuration:

config :cogo, CogoWeb.Endpoint,
  pubsub: [
    name: Cogo.PubSub,
    adapter: Phoenix.PubSub.Redis,
    node_name: "dev_node"
  ]

It seems to connect correctly with redis (no errors)

when I try to broadcast it does not deliver the messages:

Interactive Elixir (1.5.0) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> Phoenix.PubSub.subscribe(Cogo.PubSub, "test")
:ok
iex(2)> Phoenix.PubSub.broadcast(Cogo.PubSub, "test", "bla")
:ok
iex(3)> flush
:ok

Is there any major issue with how I tested this? If this is the case, it might be helpful to raise an error somewhere.

...

mix.exs

defp deps do
[{:phoenix_pubsub_redis, "~> 2.0.0"}],
end
...

I am running 2.1.3 which includes the handle_info method that should take care of this but it's still showing up in our logs. I've double-checked that we are running 2.1.3.

The exception from Sentry is:

  File "lib/phoenix_pubsub_redis/redis_server.ex", line 62, in Phoenix.PubSub.RedisServer.handle_info({:redix_pubsub, #PID<0.303.0>, :disconnected, %{reason: :tcp_closed}}, %{namespace: "phx:Elixir.FontawesomeCdn.PubSub", node_ref: <<22, 140, 43, 140, 32, 47, 230, 136, 189, 12, 123, 20, 77, 8, 114, 84, 1, 252, 250,...
  File "gen_server.erl", line 601, in :gen_server.try_dispatch/4
  File "gen_server.erl", line 667, in :gen_server.handle_msg/5
  File "proc_lib.erl", line 247, in :proc_lib.init_p_do_apply/3

I wish that I had the entire exception because that would shed a little more light on why this is still occurring.

Redix has an ssl: true option so it can connect to a Redis via an encrypted connection. It looks like the Phoenix.PubSub.Redis adapter ignores that option right now :( via this take - https://github.com/phoenixframework/phoenix_pubsub_redis/blob/master/lib/phoenix_pubsub_redis/redis.ex#L58.

If I get some time this week I'll try to send a PR, but for now I'm just opening an issue so people are aware.

If you're just using Redis PubSub, why is unique nodename required to be set?

Seeing this error:

** (EXIT) shutdown: failed to start child: Phoenix.PubSub.Redis
        ** (EXIT) an exception was raised:
            ** (BadFunctionError) expected a function, got: [fastlane: Phoenix.Channel.Server, pool_size: 1, name: FontawesomeCdn.PubSub, adapter: Phoenix.PubSub.Redis, host: "localhost"]
                (elixir) lib/keyword.ex:608: Keyword.do_merge/5
                (phoenix_pubsub_redis) lib/phoenix_pubsub_redis/redis.ex:56: Phoenix.PubSub.Redis.init/1
                (stdlib) supervisor.erl:287: :supervisor.init/1
                (stdlib) gen_server.erl:328: :gen_server.init_it/6
                (stdlib) proc_lib.erl:240: :proc_lib.init_p_do_apply/3

Anyone else experiencing this?

Looks like it might be related to these lines in redis.ex

    opts = handle_url_opts(opts)
    |> Keyword.merge(@defaults, opts)

From what I have read merge/3 expects a function not a Keyword, correct?

https://github.com/whatyouhide/redix

Redix has a very similar API to Redo and is written natively in elixir. Also has the benefits of using fishcakez's excellent Connection abstraction.

I am planning on taking this on in the coming weeks but if anyone wants to jump on it sooner let me know.

Hi everyone,

I was thinking of giving a lightning talk at ElixirConf EU about a Ruby gem I wrote.
It enables using Phoenix Pubsub in e.g. Rails by broadcasting binary ETF messages to Redis which Phoenix then forwards.
https://github.com/frontrunnerio/phoenix_pubsub_rb

It worked really well in my POC and it would enable easy integration and later migration from Rails to Phoenix.

Would you rather discourage this approach for a reason I did not think of?
I am asking since I come from a Rails background not having so much exposure yet with Phoenix and I do not want to encourage people to do anything "wrong".

Would be great if you could help out here.

Thanks a lot.

Phoenix pubsub appears unable to connect to redis after upgrading to redix 1.4.0. We're getting spammed with the following error message, unfortunately without any more info:

unable to establish initial redis connection. Attempting to reconnect...

All has been working great until now. Let me know if there's anything I can do to help!

Discussed a bit on IRC, opening an issue so we can better document the issues.

Presence using Phoenix.PubSub.Redis has a few issues not present when using Phoenix.PubSub.PG2.

1. Join and leave events are sent N times, where N is the number of servers.
2. Zombie presences happen

Is it possible the first is causing the second?

To see the N join and leave events, set up N=2 nodes (simple chat app with presence):

PORT=4000 elixir --sname=n0 -S mix phoenix.server
PORT=4001 elixir --sname=n1 -S mix phoenix.server

Go to a browser and trigger a join event. You should see 2 identical join messages.

I think this first issue might be causing other issues so I'm hesitant to weigh in on why zombie processes are still happening (even with named nodes and homogenous hardware).

Hi there!

More a feature request i suppose...

With the upcoming release of phoenix_pubsub 1.2, would it be desirable to drop the dependency on phoenix and use phoenix_pubsub instead? This would make for a smaller package if you don't need phoenix but would like the nice abstraction of pubsub with different backends.

Thanks,
Annard

I try to understand pub sub and genserver etc by searching around, But I still get lost and don't know how to get started with this.

I thinks give some basic example on how to use this and when to use this will be good for newbie like me. I get some clues about using this as single model module but not very clear while I completely lose on how to use this inside controller. How I set value to redis? How to get value from redis in order to use it as variable to render page?

If you add

  {Phoenix.PubSub, adapter: Phoenix.PubSub.Redis}

to your list of child applications, and then try to start the application in development mode (for example with iex -S mix), you will get an error from this line:

    if node_name in [nil, :nonode@nohost] do
      raise ArgumentError, ":node_name is a required option for unnamed nodes"
    end

I hacked around this by doing the following:

  children = [
    {Phoenix.PubSub,
     [
       name: Philomena.PubSub,
       adapter: Phoenix.PubSub.Redis,
       node_name: valid_node_name(node())
     ]}
  ]

  # Redis adapter really really wants you to have a unique node name,
  # so just fake one if iex is being started
  defp valid_node_name(node) when node in [nil, :nonode@nohost],
    do: Base.encode16(:crypto.strong_rand_bytes(6))

  defp valid_node_name(node), do: node

Surely there must be a better way?

We have a link to the docs but they haven't been published yet.

Steps to reproduce:

1. Make a new Phoenix app
2. Use Phoenix.PubSub.Redis as the pubsub adapter (as per instructions)
3. Run mix phoenix.gen.presence and add the appropriate supervisor
4. Run mix phoenix.server
5. Turn off Redis locally

(Exact dependency versions: https://gist.github.com/shosti/46c73d522cc2c58997ccc5d28bdfa629 + Redis 3.2.4.)

Expected results

I was expecting it to attempt to reconnect until Redis comes back online (which IIUC is basically analogous to the behavior of the PG2 adapter).

Actual results

Once a heartbeat message is broadcast, a string of errors leads to total app shutdown: https://gist.github.com/shosti/8c82beee34fd994b6e9fa53d9625cf4d .

It looks like the problem is an unhandled function clause, but it may just be a matter of adjusting restart policies (let me know if that's the case).

Because of @default using 127.0.0.1 host and default port, when using sentinel it always barfs here:

redix/lib/redix/start_options.ex

Line 84 in 81a2e85
raise ArgumentError, ":host or :port can't be passed as option if :sentinel is used"

Need to be careful with

I set up the redis adapter according to the doc and deployed it to Heroku where we have 2 dynos running like:

elixir --sname server -S mix phx.server

I didn't set the node_name manually to let it pick the one of --sname:

{Phoenix.PubSub,
        adapter: Phoenix.PubSub.Redis,
        url: System.get_env("REDIS_URL"),
        name: WebInterface.PubSub},

But it didn't work: messages still not sync between nodes. So am i missing something?

Supervisor:

supervisor(Phoenix.PubSub.Redis, [:freeling, [host: "127.0.0.1", pool_size: 1, namespace: "freeling"]]),

Commands:

Phoenix.PubSub.subscribe :freeling, "user:123"
Phoenix.PubSub.broadcast :freeling, "user:123", {:user_update, %{id: 123, name: "Shane"}}

Result:

** (UndefinedFunctionError) function Phoenix.PubSub.Local.broadcast/5 is undefined or private. Did you mean one of:

      * broadcast/6

    (phoenix_pubsub) Phoenix.PubSub.Local.broadcast(:freeling, 1, :none, "user:123", {:user_update, %{id: 123, name: "Shane"}})

Enabling ssl is done by appending [ssl: true] to the config. But there is no further way of customizing this connection, and as Bram Verburg said at ElixirConfEU it’s important to be able to do so. Basically erlang by default accepts any certificates, not just the trusted ones. In addition to that AWS Redis requires some more adjustments to work.

Currently Redix configuration options are whitelisted, and then passed to Redix.PubSub.start_link/1. In Redix itself there is also a whitelist, but much richer.

I see two options to solve this:

1. Drop the @redis_opts alltogether, and pass everything from config
2. Expand accepted options to be able to properly configure tls connection

Which one do you think is better?

When version 3 was released, it added support for Phoenix.PubSub v2.0, but the configuration instructions in the readme and docs were not updated.

They still show code to configure the pubsub server within Phoenix's endpoint config, which shouldn't be correct for Phoenix 1.5.

I am attempting to disable certification "by passing verify: :verify_none in the socket options" as outlined in the Redix docs, however, specifying the following

{
        Phoenix.PubSub,
        socket_opts: [ verify: :verify_none ]
        name: Flow.PubSub,
        node_name: "something",
        adapter: Phoenix.PubSub.Redis,
        url: System.get_env("REDIS_URL") 
      }

in the Supervision tree results in

** (ArgumentError) argument error
    (kernel 6.5.1) gen_tcp.erl:168: :gen_tcp.connect/4
    (redix 1.1.5) lib/redix/connector.ex:33: Redix.Connector.connect_directly/3
    (redix 1.1.5) lib/redix/socket_owner.ex:43: Redix.SocketOwner.handle_info/2
    (stdlib 3.11) gen_server.erl:637: :gen_server.try_dispatch/4
    (stdlib 3.11) gen_server.erl:711: :gen_server.handle_msg/6
    (stdlib 3.11) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
Last message: :connect
State: %Redix.SocketOwner{conn: #PID<0.12800.0>, continuation: nil, opts: [sync_connect: false, backoff_initial: 500, backoff_max: 30000, exit_on_disconnection: false, timeout: 5000, ssl: false, socket_opts: [verify: :verify_none], password: "{REDACTED}", host: '{REDACTED]', port: 27899], queue_table: #Reference<0.3807523016.2488139780.171396>, socket: nil, transport: :gen_tcp}

Seems like socket_opts is passing through fine so I am not sure why it's blowing up.

Top of the stack trace is https://github.com/erlang/otp/blob/master/lib/kernel/src/gen_tcp.erl#L168

Any leads appreciated, thank you.

Would it be possible to publish a new release of the current master on hex.pm?

Hi

We needed to switch to TLS in our redis usage but unfortunately phoenix_pubsub_redis only started supporting ssl a few commits after updating its dependency (phoenix_pubsub) to v2.0.

Sadly the phoenix_pubsub update is a breaking change. We solved this issue by using our fork of phoenix_pubsub_redis which includes support for ssl without changing any dependencies. While this workaround works for us we feel that the elixir community would benefit from having an official hex release with version similar or identical to ours.

Would you consider making a hex release based on this version?
It's literally a one line change in the code but it could provide a lot of value to us and to other developers.
https://github.com/surgeventures/phoenix_pubsub_redis/tree/v2.1.7-redix-config