owent / libatbus

用于搭建高性能、全异步、树形结构的BUS消息系统的跨平台框架库

Hi I'm OWenT

• Maintainer of @atframework and @xresloader .
• Member of @open-telemetry, Approver of @open-telemetry/cpp-approvers.
• Main personal projects
  • libcopp
  • wxwork_robotd
  • ddns-cli
  • coredns-nftables
  • coredns-filter
  • hiredis-happ

libatbus's Issues

请完善文档及示例

• 1
  目前只找到了1个例子, 在没有详细看代码的情况下无法快速判断上手步骤
  比如说一个树形结构:
  总根节点A, A下面挂着BC两个副节点, B下面挂着DE, C下面挂着FG
  其中ABC只负责数据转发和服务发现
  DEFG是用于提供业务的进程
  那么在使用库方面应当如何编写框架?

• 2
  常用结构只有node吗? 它既可以作为中心节点(listen), 也可以作为边缘节点(connect)?

• 3
  不listen可以吗? 数据统一走父节点转发

• 4
  busid 看起来只分了两层, 组织多级架构是库自己处理还是需要使用者处理?
  用最初的问题做例子是
  A listen, B connect A, B同时listen, D connect B, 其他类似
  若D要向F发消息那么数据流向是D->B->A->C->F
  这样理解对吗

• 5
  若要适配libcopp, protobuf.service (rpc), 希望有官方指导建议

• 6
  id_prefix是用于区分同一服务类型的不同实例么, 如game1, game2, game3

• 希望能有一份说明文档, 非常感谢!

centos7 编译报错

[root@localhost build]# cmake .. -G "Unix Makefiles"
-- Available Build Type: Unknown
-- Check Flag: -rdynamic -- yes
-- GCC Version 4.8.5 Found, -Wno-unused-local-typedefs added.
-- GCC Version 4.8.5 , using -std=c11/c++14.
-- Build Type: RelWithDebInfo
-- @/home/game/libatbus/build/deps/fmt-6.2.1/build_jobs_linux-x86_64-gnu Run: /usr/bin/cmake ../../../../3rd_party/fmtlib/pkg/fmt-6.2.1 -G 'Unix Makefiles' -DCMAKE_INSTALL_PREFIX=/home/game/libatbus/3rd_party/fmtlib/prebuilt/linux-x86_64-gnu -DCMAKE_POSITION_INDEPENDENT_CODE=YES;-DBUILD_SHARED_LIBS=OFF;-DFMT_DOC=OFF;-DFMT_INSTALL=ON;-DFMT_TEST=OFF;-DFMT_FUZZ=OFF;-DFMT_CUDA_TEST=OFF
-- CMake version: 3.9.4
-- Version: 6.2.1
-- Build type: Release
-- CXX_STANDARD: 11
-- Required features: cxx_variadic_templates
-- Configuring done
-- Generating done
-- Build files have been written to: /home/game/libatbus/build/deps/fmt-6.2.1/build_jobs_linux-x86_64-gnu
-- @/home/game/libatbus/build/deps/fmt-6.2.1/build_jobs_linux-x86_64-gnu Run: /usr/bin/cmake --build . --target install -j
Unknown argument -j
Usage: cmake --build

[options] [-- [native-options]]
Options:

= Project binary directory to be built. --target = Build instead of default targets. May only be specified once. --config = For multi-configuration tools, choose . --clean-first = Build target 'clean' first, then build. (To clean only, use --target 'clean'.) --use-stderr = Ignored. Behavior is default in CMake >= 3.0. -- = Pass remaining options to the native tool. CMake Warning at 3rd_party/atframe_utils/repo/project/cmake/modules/FindConfigurePackage.cmake:454 (find_package): By not providing "Findfmt.cmake" in CMAKE_MODULE_PATH this project has asked CMake to find a package configuration file provided by "fmt", but CMake did not find one.

Could not find a package configuration file provided by "fmt" with any of
the following names:

fmtConfig.cmake
fmt-config.cmake

Add the installation prefix of "fmt" to CMAKE_PREFIX_PATH or set "fmt_DIR"
to a directory containing one of the above files. If "fmt" provides a
separate development package or SDK, be sure it has been installed.
Call Stack (most recent call first):
3rd_party/fmtlib/fmtlib.cmake:69 (FindConfigurePackage)
3rd_party/3rd_party.macro.cmake:4 (include)
CMakeLists.txt:48 (include)

-- @/home/game/libatbus/build/deps/libuv-v1.38.1/build_jobs_linux-x86_64-gnu Run: /usr/bin/cmake ../../../../3rd_party/libuv/pkg/libuv-v1.38.1 -G 'Unix Makefiles' -DCMAKE_INSTALL_PREFIX=/home/game/libatbus/3rd_party/libuv/prebuilt/linux-x86_64-gnu -DCMAKE_POSITION_INDEPENDENT_CODE=YES;-DBUILD_SHARED_LIBS=OFF;-DBUILD_TESTING=OFF
-- summary of build options:
Install prefix: /home/game/libatbus/3rd_party/libuv/prebuilt/linux-x86_64-gnu
Target system: Linux
Compiler:
C compiler: /usr/bin/cc
CFLAGS:

-- Configuring done
-- Generating done
-- Build files have been written to: /home/game/libatbus/build/deps/libuv-v1.38.1/build_jobs_linux-x86_64-gnu
-- @/home/game/libatbus/build/deps/libuv-v1.38.1/build_jobs_linux-x86_64-gnu Run: /usr/bin/cmake --build . --target install -j
Unknown argument -j
Usage: cmake --build

[options] [-- [native-options]]
Options:

= Project binary directory to be built. --target = Build instead of default targets. May only be specified once. --config = For multi-configuration tools, choose . --clean-first = Build target 'clean' first, then build. (To clean only, use --target 'clean'.) --use-stderr = Ignored. Behavior is default in CMake >= 3.0. -- = Pass remaining options to the native tool. -- Could NOT find Libuv (missing: Libuv_INCLUDE_DIRS Libuv_LIBRARIES) -- Dependency: Libuv is required, we can not find prebuilt for libuv and can not find git to clone the sources CMake Error at 3rd_party/libuv/libuv.cmake:102 (message): Libuv not found Call Stack (most recent call first): 3rd_party/3rd_party.macro.cmake:5 (include) CMakeLists.txt:48 (include)

-- Configuring incomplete, errors occurred!
See also "/home/game/libatbus/build/CMakeFiles/CMakeOutput.log".
See also "/home/game/libatbus/build/CMakeFiles/CMakeError.log".

为啥msg_handler::on_recv_data_transfer_rsp没有处理transfer success的情况

int msg_handler::on_recv_data_transfer_rsp(node &n, connection *conn, protocol::msg &m, int /*status*/, int /*errcode*/) {
    if (NULL == m.body.forward || NULL == conn) {
        ATBUS_FUNC_NODE_ERROR(n, NULL == conn ? NULL : conn->get_binding(), conn, EN_ATBUS_ERR_BAD_DATA, 0);
        return EN_ATBUS_ERR_BAD_DATA;
    }

    _if (m.body.forward->to == n.get_id()) {
        ATBUS_FUNC_NODE_ERROR(n, conn->get_binding(), conn, m.head.ret, 0);
        n.on_send_data_failed(conn->get_binding(), conn, &m);
        return EN_ATBUS_ERR_SUCCESS;
    }_

parent node居然是child node的child node, 我是哪里用的不对吗？

CASE_TEST(atbus_endpoint, is_child1)
{
atbus::node::conf_t p_conf;
atbus::node::default_conf(&p_conf);
p_conf.children_mask = 16;

atbus::node::conf_t c_conf;
atbus::node::default_conf(&c_conf);
c_conf.children_mask = 8;
{
    atbus::node::ptr_t p_node = atbus::node::create();
    p_node->init(0x12345678, &p_conf);

    atbus::node::ptr_t c_node = atbus::node::create();
    c_node->init(0x12345679, &c_conf);
    
    CASE_EXPECT_TRUE(p_node->is_child_node(c_node->get_id()));
    
    CASE_EXPECT_FALSE(c_node->is_child_node(p_node->get_id()));
}

}

ubuntu 20.04下 编译不通过

cmake之后
直接make
提示内容如下，帮忙看一下
libatbus/atframework/atframe_utils/include/cli/cmd_option_bind_param_list.h:64:64: error: conversion from ‘long unsigned int’ to ‘int’ may change value [-Werror=conversion]
64 | struct build_args_index : build_args_index<N - 1, _Index..., sizeof...(_Index)> {};

全局路由表下发没有实现吗？

在代码里看到
// TODO 防止短时间内批量上报注册协议，所以合并上报数据包
// TODO 给所有需要全局路由表的子节点下发数据

问一下作者实现没有

使用 Windows + MinGW(msys2) 方式 编译报错 ？

$ make
Scanning dependencies of target atframe_utils
[ 3%] Building CXX object deps/atframe_utils/CMakeFiles/atframe_utils.dir/src/algorithm/base64.cpp.obj
[ 6%] Building CXX object deps/atframe_utils/CMakeFiles/atframe_utils.dir/src/algorithm/crc.cpp.obj
[ 9%] Building CXX object deps/atframe_utils/CMakeFiles/atframe_utils.dir/src/algorithm/crypto_cipher.cpp.obj
[ 12%] Building CXX object deps/atframe_utils/CMakeFiles/atframe_utils.dir/src/algorithm/crypto_dh.cpp.obj
[ 16%] Building CXX object deps/atframe_utils/CMakeFiles/atframe_utils.dir/src/algorithm/murmur_hash.cpp.obj
[ 19%] Building CXX object deps/atframe_utils/CMakeFiles/atframe_utils.dir/src/algorithm/xxtea.cpp.obj
[ 22%] Building CXX object deps/atframe_utils/CMakeFiles/atframe_utils.dir/src/cli/cmd_option_list.cpp.obj
[ 25%] Building CXX object deps/atframe_utils/CMakeFiles/atframe_utils.dir/src/cli/cmd_option_value.cpp.obj
[ 29%] Building CXX object deps/atframe_utils/CMakeFiles/atframe_utils.dir/src/cli/shell_font.cpp.obj
[ 32%] Building CXX object deps/atframe_utils/CMakeFiles/atframe_utils.dir/src/common/file_system.cpp.obj
E:/OpenSource/libatbus/3rd_party/atframe_utils/repo/src/common/file_system.cpp: In static member function 'static bool util::file_system::mkdir(const char*, bool, int)':
E:/OpenSource/libatbus/3rd_party/atframe_utils/repo/src/common/file_system.cpp:166:71: error: unused parameter 'mode' [-Werror=unused-parameter]
bool file_system::mkdir(const char *dir_path, bool recursion, int mode) {
~~~~^~~~
cc1plus.exe: all warnings being treated as errors
make[2]: *** [deps/atframe_utils/CMakeFiles/atframe_utils.dir/build.make:180：deps/atframe_utils/CMakeFiles/atframe_utils.dir/src/common/file_system.cpp.obj] 错误 1
make[1]: *** [CMakeFiles/Makefile2:91：deps/atframe_utils/CMakeFiles/atframe_utils.dir/all] 错误 2
make: *** [Makefile:141：all] 错误 2

Windows + MSVC 可以使用vcpkg 配置吗？

因为vs2017 调试功能很方便也很强大，很方便新手理解程序。
我在https://github.com/microsoft/vcpkg 发现已经有人移植好了libuv 和 msgpack，我也安装了，那怎么导入如这个工程？需要修改cmake 文件吗？

PS E:\OpenSource\vcpkg> .\vcpkg integrate install
Applied user-wide integration for this vcpkg root.

All MSBuild C++ projects can now #include any installed libraries.
Linking will be handled automatically.
Installing new libraries will make them instantly available.

CMake projects should use: "-DCMAKE_TOOLCHAIN_FILE=E:/OpenSource/vcpkg/scripts/buildsystems/vcpkg.cmake"
PS E:\OpenSource\vcpkg> .\vcpkg.exe install libuv
The following packages will be built and installed:
libuv[core]:x86-windows
Starting package 1/1: libuv:x86-windows
Building package libuv[core]:x86-windows...
-- Downloading https://github.com/libuv/libuv/archive/v1.28.0.tar.gz...
-- Extracting source E:/OpenSource/vcpkg/downloads/libuv-libuv-v1.28.0.tar.gz
-- Using source at E:/OpenSource/vcpkg/buildtrees/libuv/src/v1.28.0-c436f63419
-- Configuring x86-windows
-- Building x86-windows-dbg
-- Building x86-windows-rel
-- Performing post-build validation
-- Performing post-build validation done
Building package libuv[core]:x86-windows... done
Installing package libuv[core]:x86-windows...
Installing package libuv[core]:x86-windows... done
Elapsed time for package libuv:x86-windows: 47.68 s

Total elapsed time: 47.68 s

The package libuv:x86-windows provides CMake targets:

find_package(unofficial-libuv CONFIG REQUIRED)
target_link_libraries(main PRIVATE unofficial::libuv::libuv)

PS E:\OpenSource\vcpkg> .\vcpkg.exe install msgpack
The following packages will be built and installed:
msgpack[core]:x86-windows
Starting package 1/1: msgpack:x86-windows
Building package msgpack[core]:x86-windows...
-- Downloading https://github.com/msgpack/msgpack-c/archive/cpp-3.1.1.tar.gz...
-- Extracting source E:/OpenSource/vcpkg/downloads/msgpack-msgpack-c-cpp-3.1.1.tar.gz
-- Using source at E:/OpenSource/vcpkg/buildtrees/msgpack/src/cpp-3.1.1-eab3c56b41
-- Applying patch E:/OpenSource/vcpkg/ports/msgpack/add-static-lib-option.patch
-- Applying patch failed. This is expected if this patch was previously applied.
-- Configuring x86-windows
-- Building x86-windows-dbg
-- Building x86-windows-rel
-- Performing post-build validation
-- Performing post-build validation done
Building package msgpack[core]:x86-windows... done
Installing package msgpack[core]:x86-windows...
Installing package msgpack[core]:x86-windows... done
Elapsed time for package msgpack:x86-windows: 46.6 s

Total elapsed time: 46.6 s

The package msgpack:x86-windows provides CMake targets:

find_package(msgpack CONFIG REQUIRED)
target_link_libraries(main PRIVATE msgpackc msgpackc-cxx)

PS E:\OpenSource\vcpkg>

关于单读多写的实现机制

同机器之间的进程通信可以通过共享内存管道，想请教下这个单读多写队列的实现原理，网上看过的很多资料都是线程之间通过原子操作来实现，但是在进程之间，是怎么实现多个进程的同步写操作？

兄弟节点发现机制

尝试添加兄弟节点发现机制, 总是出现预期之外的效果, 希望作者能够支持
目前实践如下

• 添加开关

  struct conf_flag_t {
    enum type {
      EN_CONF_BRO_FOUND = 0, // 兄弟节点发现
      EN_CONF_MAX = 1,
    };
  };

• 修改proto

message node_bro_sync_data {
    repeated uint64 ids = 1;
}

message msg {
    msg_head head = 1;
    oneof    msg_body {
        // ...
        node_bro_sync_data  node_bro_lost      = 23;
        node_bro_sync_data  node_bro_found     = 24;
    }
}

• 添加dispatcher及handler
  这部分省略, 参照其他代码写的
• 触发

void node::on_new_child(bus_id_t new_one) {
	if (!conf_.flags.test(conf_flag_t::EN_CONF_BRO_FOUND)) {
		return;
	}

	std::set<bus_id_t> children;
	const auto& coll = node_routes_;

	for (const auto& i : coll) {
		auto id = i.second->get_id();
		if (is_child_node(id) && id != get_id() && id != new_one) {
			children.insert(id);
		}
	}

	::ATBUS_MACRO_PROTOBUF_NAMESPACE_ID::ArenaOptions arena_options;
	arena_options.initial_block_size = ATBUS_MACRO_RESERVED_SIZE;
	::ATBUS_MACRO_PROTOBUF_NAMESPACE_ID::Arena arena(arena_options);
	::atbus::msg_t *m = ::ATBUS_MACRO_PROTOBUF_NAMESPACE_ID::Arena::CreateMessage<atbus::protocol::msg>(&arena);

	if (!m) {
		return;
	}
	auto *head = m->mutable_head();
	auto *body = m->mutable_node_bro_found();

	if (!head || !body) {
		return;
	}

	head->set_version(get_protocol_version());
	head->set_type(0);
	head->set_src_bus_id(get_id());

	// tell the new one others
	for (auto id : children) {
		body->add_ids(id);
	}

	head->set_sequence(alloc_msg_seq());
	send_data_msg(new_one, *m);

	// tell others the new one
	body->clear_ids();
	body->add_ids(new_one);
	for (auto id : children) {
		head->set_sequence(alloc_msg_seq());
		send_data_msg(id, *m);
	}

	return;
}

• 调用触发
  msg_handler::on_recv_node_reg_req

  int ret;
  // 仅fd连接发回注册回包，否则忽略（内存和共享内存通道为单工通道）
  if (nullptr != conn && conn->check_flag(connection::flag_t::REG_FD)) {
    ret = send_reg(::atbus::protocol::msg::kNodeRegisterRsp, n, *conn, rsp_code, m.head().sequence());
    if (rsp_code < 0) {
      ATBUS_FUNC_NODE_ERROR(n, ep, conn, ret, errcode);
      conn->reset();
    }
  } else {
    ret = 0;
  }

  if (ret == 0 && conn->is_connected()) {
	  const auto id = ep->get_id();

	  if (n.is_child_node(id) && id != n.get_id()) {
		  n.on_new_child(id);
	  }
  }

  return ret;

• 问题
  目前发现此事件总是触发两次
  不明白的地方是为什么reg会走两遍
  代码理解的还不够深, 希望得到指点
  谢谢