gar1t / erlang-czmq Goto Github PK

View Code? Open in Web Editor NEW

66.0 66.0 18.0 549 KB

License: GNU General Public License v3.0

Shell 4.45% C 36.29% Erlang 56.69% Makefile 1.80% C++ 0.39% M4 0.39%

erlang-czmq's People

Contributors

Stargazers

Watchers

Forkers

benoitc bjhaid sanmiguel doubleyou artemav sjmackenzie aseigo amclain prio kristinn cancoin iamd3vil fawkesley svdev jennhuang wayann shishirpy

erlang-czmq's Issues

idea: use a pipe / socket

I am not sure what are your plan to improve the performance in the lib, but I was thiking that one way to do it would be opening a pipe / socket with the followiing api:

czmq:socket_new/2 create a new Pid + pipe, The pipe is handled in the socket pipe and fill the process inbox while all control operations are sent to the socket.

what do you think about it?

The complexity with that design is that it will require a way to proxy socket operations to the pipe + an eventloop to manage events coming to the socket. (maybe using libuv?)

Such design may probably goes further than the initial intention so I don't know. Let me know.

Fails to compile with gcc version 7.3.0 (Ubuntu 7.3.0-16ubuntu3)

I am getting the following error when trying to compile on Ubuntu 18.04 with gcc 7.3.0

libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I../include -pedantic -Werror -Wall -D_GNU_SOURCE -DLINUX -D_REENTRANT -D_THREAD_SAFE -I/home/janabaran/src/erlang-czmq/c_src/.libs/libsodium/include -I/home/janabaran/src/erlang-czmq/c_src/.libs/libzmq/include -g -I/home/janabaran/src/erlang-czmq/c_src/.libs/libsodium/include -I/home/janabaran/src/erlang-czmq/c_src/.libs/libzmq/include -c zauth.c  -fPIC -DPIC -o .libs/zauth.o
In file included from /usr/include/ctype.h:25:0,
                 from ../include/czmq_prelude.h:203,
                 from ../include/czmq.h:19,
                 from zauth.c:25:
/usr/include/features.h:184:3: error: #warning "_BSD_SOURCE and _SVID_SOURCE are deprecated, use _DEFAULT_SOURCE" [-Werror=cpp]
 # warning "_BSD_SOURCE and _SVID_SOURCE are deprecated, use _DEFAULT_SOURCE"
   ^~~~~~~
cc1: all warnings being treated as errors
Makefile:531: recipe for target 'zauth.lo' failed
make[4]: *** [zauth.lo] Error 1
make[4]: Leaving directory '/home/janabaran/src/erlang-czmq/c_src/.libs/czmq-2.2.0/src'
Makefile:344: recipe for target 'all' failed
make[3]: *** [all] Error 2
make[3]: Leaving directory '/home/janabaran/src/erlang-czmq/c_src/.libs/czmq-2.2.0/src'
Makefile:326: recipe for target 'all-recursive' failed
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory '/home/janabaran/src/erlang-czmq/c_src/.libs/czmq-2.2.0'
Makefile:37: recipe for target 'buildcmq' failed
make[1]: *** [buildcmq] Error 1
make[1]: Leaving directory '/home/janabaran/src/erlang-czmq/c_src'
Makefile:7: recipe for target 'app' failed
make: *** [app] Error 2

compilation error: behaviour zmq_gen_benchmark undefined

When running the make I get the following command:

$ make
ERLC   czmq_poller.erl czmq_benchmark.erl czmq.erl czmq_reloader.erl  czmq_test.erl erlzmq_benchmark.erl zmq_gen_benchmark.erl
compile: warnings being treated as errors
src/czmq_benchmark.erl:12: behaviour zmq_gen_benchmark undefined
make: *** [ebin/czmq.app] Error 1

libsodium compilation issue on osx

I bumped libsodium to 1.0.15 locally to fix this problem for myself

./build_czmq.sh
==> build libsodium
checking build system type... x86_64-apple-darwin16.7.0
checking host system type... x86_64-apple-darwin16.7.0
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... ./install-sh -c -d
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether UID '502' is supported by ustar format... yes
checking whether GID '20' is supported by ustar format... yes
checking how to create a ustar tar archive... gnutar
checking whether make supports nested variables... (cached) yes
checking whether to enable maintainer-specific portions of Makefiles... no
checking for style of include used by make... GNU
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... configure: error: in `....<phx_project_path>.../deps/czmq/c_src/.libs/libsodium-1.0.0':
configure: error: cannot compute suffix of executables: cannot compile and link
See `config.log' for more details
make[1]: *** [buildcmq] Error 1
make: *** [app] Error 2
** (Mix) Could not compile dependency :czmq, "./configure; make" command failed. You can recompile this dependency with "mix deps.compile czmq", update it with "mix deps.update czmq" or clean it with "mix deps.clean czmq"

zmq_test error: zauth exception

tests fail with the following error:

1> czmq_test:test().
Testing erlang-czmq...
 * zstr_send_recv: ok
 * sendmem_framerecv: ok
 * zauth: ** exception error: no match of right hand side value true
     in function  czmq_test:zauth/1 (src/czmq_test.erl, line 173)
     in call from czmq_test:test/0 (src/czmq_test.erl, line 27)

=ERROR REPORT==== 1-Mar-2014::07:31:06 ===
** Generic server <0.35.0> terminating 
** Last message in was {'EXIT',<0.33.0>,
                           {{badmatch,true},
                            [{czmq_test,zauth,1,
                                 [{file,"src/czmq_test.erl"},{line,173}]},
                             {czmq_test,test,0,
                                 [{file,"src/czmq_test.erl"},{line,27}]},
                             {erl_eval,do_apply,6,
                                 [{file,"erl_eval.erl"},{line,573}]},
                             {shell,exprs,7,[{file,"shell.erl"},{line,674}]},
                             {shell,eval_exprs,7,
                                 [{file,"shell.erl"},{line,629}]},
                             {shell,eval_loop,3,
                                 [{file,"shell.erl"},{line,614}]}]}}
** When Server state == {state,#Port<0.726>}
** Reason for termination == 
** {{badmatch,true},
    [{czmq_test,zauth,1,[{file,"src/czmq_test.erl"},{line,173}]},
     {czmq_test,test,0,[{file,"src/czmq_test.erl"},{line,27}]},
     {erl_eval,do_apply,6,[{file,"erl_eval.erl"},{line,573}]},
     {shell,exprs,7,[{file,"shell.erl"},{line,674}]},
     {shell,eval_exprs,7,[{file,"shell.erl"},{line,629}]},
     {shell,eval_loop,3,[{file,"shell.erl"},{line,614}]}]}
2>

Compilation failed on macOS 10.12.4 (Sierra)

Hi,

I've got this error when compiling on OSX Sierra:

/tmp/erlang-czmq$ make
 APP    czmq.app.src
cd c_src; make
./build_czmq.sh
gcc -Wall -g2 -I/usr/local/Cellar/erlang/19.3/lib/erlang/lib/erl_interface-3.9.3/include -I/tmp/erlang-czmq/c_src/.libs/libsodium/include -I/tmp/erlang-czmq/c_src/.libs/libzmq/include -I/tmp/erlang-czmq/c_src/.libs/czmq/include -c czmq_benchmark.c -o czmq_benchmark.o
In file included from czmq_benchmark.c:41:
In file included from /tmp/erlang-czmq/c_src/.libs/czmq/include/czmq.h:19:
In file included from /tmp/erlang-czmq/c_src/.libs/czmq/include/czmq_prelude.h:210:
/usr/include/time.h:177:5: error: conflicting types for 'clock_gettime'
int clock_gettime(clockid_t __clock_id, struct timespec *__tp);
    ^
czmq_benchmark.c:26:5: note: previous definition is here
int clock_gettime(int clk_id, struct timespec *t){
    ^
1 error generated.
make[1]: *** [czmq_benchmark.o] Error 1
make: *** [app] Error 2

I tried some hacky solution but I think I'm stuck.

czmq: zstr_recv_nowait/1 is missing some messages

from time to time czmq:zstr_recv_nowait/1 is missing some messages even if you retry it in loop. Is this expected?

czmq:recv/{1,2} is concatenating multipart send

When you send a message with a dealer on which you set the identity, the message retrieved on a router socket with zmq:recv /{1,2} is concatenated. I have to use czmq:zframe_recv_all/1 to retrive correctly the multipart. Not sure if it's normal. Also this function doesn't provide any way to just wait on it. Maybe implementing czstr_recvx would do the trick?

Here is a small snippet to reproduce it:

Ex of dealer in python:

>>> import zmq
>>> ctx = zmq.Context()
>>> s = ctx.socket(zmq.DEALER)
>>> s.setsockopt(zmq.IDENTITY,b"a")
>>> s.connect("tcp://127.0.0.1:5555")
>>> s.send(b"hello")
>>> s.send(b"hello")

Example of router:

1> {ok, Ctx} = czmq:start_link().
{ok,<0.34.0>}
2>  Router = czmq:zsocket_new(Ctx, router).
{<0.34.0>,0}
3> czmq:zsocket_bind(Router, "tcp://127.0.0.1:5555").
{ok,5555}
5> czmq:zframe_recv_all(Router).
{ok,[<<"a">>,<<"hello">>]}
6> czmq:zstr_recv(Router).
{ok,"ahello"}

zeromq binding page

Are you OK to add erlang-czmq to the zeromq binding page [1] ?

[1] http://zeromq.org/bindings:erlang

Is there any way to receive messages as Erlang messages instead of doing recv manually?

Hi,
Is there any way to receive messages as Erlang messages instead of calling czmq:zframe_recv_all/1 manually, just like an active TCP socket?

Memory leak

I think that handle_zframe_recv_nowait has a memory leak. (At least, something does -- this is my best guess as to the problem.) The culprit appears to be in the freeing of terms at the end of the function. There is a call to "erl_free_term(more_boolean)", but "more_boolean" is a pointer to an atom that shouldn't be freed. But there is no free of the tuple, "data_more". I suspect swapping one for the other will fix this leak.

Missing a lot of messages in PubSub pattern

When I use a pubsub pattern, there are a lot of missing messages. It only displays one message per second. I am using this library through Elixir.
Here is my client code.

defmodule SubClient do
  def start_client do
    {:ok, ctx} = :czmq.start_link()
    socket = :czmq.zsocket_new(ctx, :czmq_const.zmq_sub)
    :ok = :czmq.zsocket_connect(socket, "tcp://127.0.0.1:5555")
    :czmq.zsocket_set_subscribe(socket, '')
    sub(socket)
  end

  defp sub(socket) do
    {:ok, msg} = :czmq.zstr_recv(socket)
    IO.puts msg
    sub(socket)
  end
end

I am using Erlang/OTP 18, erts-7.1

zbeacon api is missing

Would be interresting to cover the zbeacon API:

http://czmq.zeromq.org/manual:zbeacon

tag the current release

In my tests erlang-czmq is enough good to be used, would you mind to tag it? It would be easier to integrate :)

Doesn't link correctly on systems with clock_gettime

On Ubuntu, a program that uses clock_gettime (in both libsodium and zeromq) requires the "-rt" library. Because rebar.config.script hard-codes the libraries, czmq-port doesn't link correctly.

Missing erl_interface.h

When I try and build it, it's giving me a fatal error: erl_interface.h: No such file or directory
compilation terminated.

The v2.2.0 branch of `czmq` dose not build with gcc-4.9.2

I am building it in an updated archlinux box, and the error message says:

...
make[2]: Entering directory '/home/pank7/src/czmq/src'
  CC       zauth.lo
In file included from /usr/include/ctype.h:25:0,
                 from ../include/czmq_prelude.h:203,
                 from ../include/czmq.h:19,
                 from zauth.c:25:
/usr/include/features.h:148:3: error: #warning "_BSD_SOURCE and _SVID_SOURCE are deprecated, use _DEFAULT_SOURCE" [-Werror=cpp]
 # warning "_BSD_SOURCE and _SVID_SOURCE are deprecated, use _DEFAULT_SOURCE"
   ^
cc1: all warnings being treated as errors
Makefile:802: recipe for target 'zauth.lo' failed
make[2]: *** [zauth.lo] Error 1
...

I know the lastest master branch of czmq has already fixed this, but in my situation, I am trying to use it through erlang-czmq, which is using the v2.2.0 branch, so my question is are we planning on moving to a newer version of czmq?

czmq_port binary in c_src

Is this expected to have czmq_port as a binary in c_src?

Missing documentation on benchmark tool in README.md

The README.md contains this:

"### C Receiver / erlang-czmq Sender

This test measures the throughput of using an erlang-czmq sender with a C
receiver.

Start the receiver as with the C / C test above.

Next,"

Next ... what? :) It would be nice to know how the benchmark tool is intended to work without reading through the code itself.