thesamet / rpcz Goto Github PK

don't know why, zmq is fast,while...

really a question for suggestions than a bug, and much of it might be a zeromq question rather than rpcz proper, but anyway:

I plan to use rpcz in www.machinekit.io
right now we dont use any encryption/authentication ontop of zeromq, but eventually we'll have to, including role-based access of clients to methods

any suggestions how to do this?
anybody needing something along these lines so we can share the effort?

@thesamet Just out of curiosity, can gRPC perform as good as RPC over ZMQ? Is that why rpcz is not maintained any more? From my tests we get latency in order of 300us with ZMQ while gRPC is in order of milliseconds. I thought RPC over ZMQ should be a good option if you want to use ipc transport. Thoughts?

compile proto files("syntax=proto3") using protobuf3.1，server run ok and got client request，client crash with Application_error(0)

The README mentions that rpcz is tested on OS/X, but there are no installation instructions for OS/X.

I tried to improvise, but I didn't get very far...

I used Homebrew to install all dependencies (unfortunately I could not find a formula for installing rpcz using Homebrew - that would have been nice), and here's where I failed:

~/work/rpcz/build (master)>cmake .. -Drpcz_build_examples=0 -DCMAKE_INSTALL_PREFIX:PATH=/usr/local
-- The C compiler identification is AppleClang 6.1.0.6020053
-- The CXX compiler identification is AppleClang 6.1.0.6020053
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Build type: Release
-- Found PROTOBUF: /usr/local/lib/libprotobuf.dylib
-- Boost version: 1.58.0
-- Found the following Boost libraries:
--   thread
--   program_options
-- Found PkgConfig: /usr/local/bin/pkg-config (found version "0.28")
CMake Error at /usr/local/Cellar/cmake/3.3.1/share/cmake/Modules/FindPkgConfig.cmake:112 (elseif):
  given arguments:

    "VERSION_LESS" "3.1"

  Unknown arguments specified
Call Stack (most recent call first):
  /usr/local/Cellar/cmake/3.3.1/share/cmake/Modules/FindPkgConfig.cmake:501 (_pkgconfig_parse_options)
  cmake_modules/LibFindMacros.cmake:27 (pkg_check_modules)
  cmake_modules/FindZeroMQ.cmake:12 (libfind_pkg_check_modules)
  CMakeLists.txt:14 (find_package)


-- Configuring incomplete, errors occurred!
See also "/Users/itamar/work/rpcz/build/CMakeFiles/CMakeOutput.log".

hi~

I am a new guy.

My teacher asked me to add ssl to the messaging part of RPCZ, but i am confused about the messaging type that RPCZ uses. I mean , for example, it uses zeroMQ's pipeline mode, pub-sub mode, or uses router and dealer to build a broker for clients and services.

I'v found that it seems that RPCZ creates socket ZMQ_ROUTER and socket ZMQ_DEALER which I found in the file connection_manager.cc. But to add my own methods to those send and receive parts, I need more details about how does RPCZ use zeromq, and where I can find them in those files included in RPCZ .

Sorry about my poor English. I am still practicing.
hope that my words are clear enough for you to understand .

in the machinekit.io comms stack (zeromq + protobufs), beyond synchronous request-reply we use modified RPC-style patterns which look like so:

• certain requests might not need a reply at all, and hence are marked (a "no RSVP required" in the protobuf message)
• certain requests might cause a series of replies, like intermediate updates, followed by a final response.

I admit it doesnt sound very RPC-ish and maybe a bit whacky, and maybe the better solution is not to use RPCZ for such patterns at all (which we now do), but I would eventually like to funnel all such interactions through the same code path.

I have not yet dug into the code to explore what is needed, if anything.

is this doable? does it make sense to do in the rpcz context? any idea what the effort could be?

oh my, this one took some time to grok, but we finally figured that when we use deadline_ms=60000 when using zmq3 (under ubuntu 14.04), the actual deadline is 60,000 seconds... O_O

the obvious workaround is to use deadline_ms=60, but i was wondering if there's a cleaner fix... (that also preserves that sub-second timeout)

I have a program based on rpcz using the synchronous stub api. Sometimes (deterministically, but following no obvious pattern) the stub simply doesn't return. The stack is:

#0  pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00007ffff7bc941b in rpcz::sync_event::wait() ()
   from /usr/local/lib/librpcz.so
#2  0x00007ffff7bc6209 in rpcz::rpc::wait() () from /usr/local/lib/librpcz.so
#3  0x0000000000420a42 in TabletServerService_Stub::Query (
    this=0x7fffe400c480, request=..., response=0x7ffff3f568c0, 
    deadline_ms=1000) at common/gen/tabletserver.rpcz.cc:286

and then a lot of frames from my code

I'm pretty sure the server is calling reply.send (I put a print statement right before it, plus it works fine with other clients). Also, if the server weren't sending, it should still return when the timeout takes place.

I put a breakpoint on rpcz::rpc_channel_impl::handle_client_response and it didn't trigger for the problematic request (though it did for others).

The scenario that causes this is the third identical request in which the client and server are the same process. It doesn't matter whether the invocations use the same stub and rpc_channel object or create fresh ones. The requests two go through fine. I can run almost the same code with different processes (still on the same host) and that goes through fine. I haven't done really extensive tests, though, so I'm far from confident that this is the only scenario that triggers the bug.

I can post my code if you want. It's about a thousand lines and pretty messy, though. I tried to replicate the bug in a simple program, but couldn't quite get the same behavior.

Are there other tests I should be running?

Any clues on how to either debug or work around this would be greatly appreciated.

Hello,

I have an intermittent crash in rpcz::sync_event::signal(). Unfortunately, I'm not able right now to provide a way to reproduce the problem since it happens very rarely. Here is the backtrace:

what():  boost: mutex lock failed in pthread_mutex_lock: Invalid argument

Program received signal SIGABRT, Aborted.
[Switching to Thread 0xa8fffb40 (LWP 22450)]
0xb7fdd424 in __kernel_vsyscall ()
(gdb) bt
#0  0xb7fdd424 in __kernel_vsyscall ()
#1  0xb6655607 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#2  0xb6658a33 in __GI_abort () at abort.c:89
#3  0xb6885405 in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/i386-linux-gnu/libstdc++.so.6
#4  0xb6883063 in ?? () from /usr/lib/i386-linux-gnu/libstdc++.so.6
#5  0xb688309f in std::terminate() () from /usr/lib/i386-linux-gnu/libstdc++.so.6
#6  0xb6883306 in __cxa_throw () from /usr/lib/i386-linux-gnu/libstdc++.so.6
#7  0xb69ff97b in void boost::throw_exception<boost::lock_error>(boost::lock_error const&) () from /usr/lib/librpcz.so
#8  0xb6a08f62 in rpcz::sync_event::signal() () from /usr/lib/librpcz.so
#9  0xb6a06d79 in rpcz::rpc_channel_impl::handle_client_response(rpcz::rpc_response_context, rpcz::connection_manager::status, rpcz::message_iterator&) ()
   from /usr/lib/librpcz.so
#10 0xb6a0739f in boost::detail::function::void_function_obj_invoker2<boost::_bi::bind_t<void, boost::_mfi::mf3<void, rpcz::rpc_channel_impl, rpcz::rpc_response_context, rpcz::connection_manager::status, rpcz::message_iterator&>, boost::_bi::list4<boost::_bi::value<rpcz::rpc_channel_impl*>, boost::_bi::value<rpcz::rpc_response_context>, boost::arg<1>, boost::arg<2> > >, void, rpcz::connection_manager::status, rpcz::message_iterator&>::invoke(boost::detail::function::function_buffer&, rpcz::connection_manager::status, rpcz::message_iterator&) () from /usr/lib/librpcz.so
#11 0xb69fb718 in rpcz::worker_thread(rpcz::connection_manager*, zmq::context_t*, std::string) () from /usr/lib/librpcz.so
#12 0xb69fcd4b in boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(rpcz::connection_manager*, zmq::context_t*, std::string), boost::_bi::list3<boost::_bi::value<rpcz::connection_manager*>, boost::_bi::value<zmq::context_t*>, boost::_bi::value<std::string> > > >::run() () from /usr/lib/librpcz.so
#13 0xb4084681 in ?? () from /usr/lib/i386-linux-gnu/libboost_thread.so.1.54.0
#14 0xb740ef70 in start_thread (arg=0xa8fffb40) at pthread_create.c:312
#15 0xb6712bee in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129

It reminds me this issue:

https://code.google.com/p/rpcz/issues/detail?id=10

but it has been solved a long time ago.

Hi,

The following code hangs an application that uses RPCZ:

#include <rpcz/rpcz.hpp>
#include <memory>
#include <thread>

class SomeService;

int main() {
  std::unique_ptr<rpcz::application> application;
  std::unique_ptr<rpcz::server> server;

  application = std::make_unique<rpcz::application>();
  handle_thread = std::thread(&rpcz::application::run, application.get());

  server = std::make_unique<rpcz::server>(*application);
  SomeService service;
  server->registre_service(&service);
  server->bind("ipc://tmp/myservice");

  server->reset();
  application->terminate();
  handle_thread.join();
  application.reset();
}

The application hangs on rpcz::application destructor, trying to destroy own zmq context. I investigated the issue a bit and found that the destroying of the context hangs because it still has open sockets. I suppose that the socket under question is the connection_manager::socket_, which has type of boost::thread_specific_ptrzmq::socket_t

I changed the type of the socket to boost::scoped_ptr and the issue gone. However I'm not sure that the fix is valid.