owent / libatbus Goto Github PK

View Code? Open in Web Editor NEW

215.0 22.0 62.0 1.89 MB

用于搭建高性能、全异步、树形结构的BUS消息系统的跨平台框架库

License: MIT License

CMake 3.39% C++ 92.60% C 0.85% Shell 2.54% PowerShell 0.62%

cxx cpp shm bus linux windows socket tcp ip ipv4

libatbus's Introduction

libatbus

用于搭建高性能、全异步(a)、树形结构(t)的BUS消息系统的跨平台框架库

CI Job Matrix

Target System	Toolchain	Note
Linux	GCC
Linux	Clang	With libc++
Linux	GCC 4.8
MinGW64	GCC	Dynamic linking
Windows	Visual Studio 2022	Static linking
Windows	Visual Studio 2022	Dynamic linking
Windows	Visual Studio 2019	Static linking
macOS	AppleClang	With libc++

依赖

支持c++0x或c++11的编译器(为了代码尽量简洁,特别是少做无意义的平台兼容，依赖部分 C11和C++11的功能，所以不支持过低版本的编译器)
- GCC: 4.8 及以上
- Clang: 3.8 及以上
- VC: 12 及以上
cmake 3.16.0 以上

设计初衷和要点

[扩展性] 根据很多业务的需要，预留足够长的ID段（64位），用以给不同的ID段做不同类型的业务区分。

现有很多框架都是32位（比如腾讯的tbus和云风的skynet），在服务类型比较多的时候必须小心设计服务ID,以免冲突。

当然也有考虑到以后可能会扩展为带Hash的字符串，所以在编译选项上做了预留。但是目前还是uint64_t

[高性能] 同物理机之间可以直接使用共享内存通信，大幅提高消息分发的性能。跨物理机之间会使用tcp通信。并且这个通信方式的选择是完全自动并透明的（尽可能选择更快的方式发送消息），业务层完全不需要关心。
[跨平台] 拥有不同习惯的Developer可以使用自己熟悉的工具，提高开发速度
[动态路由] 父子节点间会至少保持一条连接，自动断线重连。同父的兄弟节点之间完全按需建立连接。并且这个过程都是自动完成的，不需要提前初始化。

同样，新增节点和移除节点也不需要做什么特别的初始化操作。不会影响到已经在线上的服务。

[低消耗] 采用无锁队列，提高CPU性能。（共享）内存通道支持多端写入，一端读取，减少内存浪费。

如果N个节点两两互联，每个节点可以只拥有一个（共享）内存通道。即总共只有N个通道，内存消耗为N*每个通道内存占用

一些其他的系统（比如tbus和我们目前的服务器框架）如果N个节点两两互联，每两个节点之间都要创建（共享）内存通道。即总共只有NN个通道，内存消耗为NN*每个通道内存占用。非常浪费

[简化设计] 根据一些实际的项目情况，把父子节点间的关系限定为Bus ID的后N位有包含关系，类似路由器的路由表的策略。

比如 0x12345678 可以控制的子节点有16位（0x12345678/16），那么0x12340000-0x1234FFFF都可以注册为它的子节点。

如同IP协议中 192.168.1.1/24 路由表能管理 192.168.1.0-192.168.1.255 一样。当然这里的24指前24位，而前面提到的16指后16位。

这么简化以后很多节点关系维护和通信都能简单很多并且能提高性能。

环境准备和构建流程

使用cmake标准构建方式，默认的编译目标为Debug版本，详见使用（编译）流程

注意：默认的编译选项是Debug模式，压测和正式环境请使用 cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo 源码目录 [其他选项] 编译（相当于gcc -O2 -g -ggdb -DNDEBUG -Wall -Werror或MSVC /O2）

注意： Windows下私有共享内存不允许跨进程共享，公有共享内存必须有管理员权限。如果Windows下出现初始化共享内存错误请使用管理员权限运行。

使用示例

简要的使用示例见使用示例

更加详细的请参考单元测试和tools目录内的代码

Benchmark

压力测试和对比见docs/Benchmark.md

支持工具

Linux下 GCC编译安装脚本(支持离线编译安装):

LICENSE

libatbus 采用MIT License
Flatbuffers 采用Apache License, Version 2.0
libuv 采用Node's license协议（类似MIT License）

libatbus's People

Contributors

Stargazers

Watchers

Forkers

days72115 laocheng simple555a weinvest dailypipsgxj phiau alchem-lab gezidan whalecold yeung908 kileven cnsuhao woozoo86 yeqiaojun lostedleo kingjeffli lxq2537664558 doublefox1981 littleboys wacr2008 a100q100 reed-lau std-vector superzscy 19317362 huaweidong darcyg researchmore huanglelh gitzhujia 13ice yanmeng-my merlinxyoung xyh922 tangzhenquan xmyeen johnnyvr gamark jackhakeem dotkt taoyuanyuan xingchaoet julian-younger platanus2000 zhaoyaogit yuufish kounowa sidyhe mincore feizai131 jeff-shen rdcentermrzhi cowboyyang vbirds pkafma-aon fishney dingdong1123 terry8210 stanhome jackylucifer ericyonng 724789975

libatbus's Issues

请完善文档及示例

1
目前只找到了1个例子, 在没有详细看代码的情况下无法快速判断上手步骤
比如说一个树形结构:
总根节点A, A下面挂着BC两个副节点, B下面挂着DE, C下面挂着FG
其中ABC只负责数据转发和服务发现
DEFG是用于提供业务的进程
那么在使用库方面应当如何编写框架?
2
常用结构只有node吗? 它既可以作为中心节点(listen), 也可以作为边缘节点(connect)?
3
不listen可以吗? 数据统一走父节点转发
4
busid 看起来只分了两层, 组织多级架构是库自己处理还是需要使用者处理?
用最初的问题做例子是
A listen, B connect A, B同时listen, D connect B, 其他类似
若D要向F发消息那么数据流向是D->B->A->C->F
这样理解对吗
5
若要适配libcopp, protobuf.service (rpc), 希望有官方指导建议
6
id_prefix是用于区分同一服务类型的不同实例么, 如game1, game2, game3
希望能有一份说明文档, 非常感谢!

cmake之后
直接make
提示内容如下，帮忙看一下
libatbus/atframework/atframe_utils/include/cli/cmd_option_bind_param_list.h:64:64: error: conversion from ‘long unsigned int’ to ‘int’ may change value [-Werror=conversion]
64 | struct build_args_index : build_args_index<N - 1, _Index..., sizeof...(_Index)> {};

兄弟节点发现机制

尝试添加兄弟节点发现机制, 总是出现预期之外的效果, 希望作者能够支持
目前实践如下

添加开关

  struct conf_flag_t {
    enum type {
      EN_CONF_BRO_FOUND = 0, // 兄弟节点发现
      EN_CONF_MAX = 1,
    };
  };

修改proto

message node_bro_sync_data {
    repeated uint64 ids = 1;
}

message msg {
    msg_head head = 1;
    oneof    msg_body {
        // ...
        node_bro_sync_data  node_bro_lost      = 23;
        node_bro_sync_data  node_bro_found     = 24;
    }
}

添加dispatcher及handler
这部分省略, 参照其他代码写的
触发

void node::on_new_child(bus_id_t new_one) {
	if (!conf_.flags.test(conf_flag_t::EN_CONF_BRO_FOUND)) {
		return;
	}

	std::set<bus_id_t> children;
	const auto& coll = node_routes_;

	for (const auto& i : coll) {
		auto id = i.second->get_id();
		if (is_child_node(id) && id != get_id() && id != new_one) {
			children.insert(id);
		}
	}

	::ATBUS_MACRO_PROTOBUF_NAMESPACE_ID::ArenaOptions arena_options;
	arena_options.initial_block_size = ATBUS_MACRO_RESERVED_SIZE;
	::ATBUS_MACRO_PROTOBUF_NAMESPACE_ID::Arena arena(arena_options);
	::atbus::msg_t *m = ::ATBUS_MACRO_PROTOBUF_NAMESPACE_ID::Arena::CreateMessage<atbus::protocol::msg>(&arena);

	if (!m) {
		return;
	}
	auto *head = m->mutable_head();
	auto *body = m->mutable_node_bro_found();

	if (!head || !body) {
		return;
	}

	head->set_version(get_protocol_version());
	head->set_type(0);
	head->set_src_bus_id(get_id());

	// tell the new one others
	for (auto id : children) {
		body->add_ids(id);
	}

	head->set_sequence(alloc_msg_seq());
	send_data_msg(new_one, *m);

	// tell others the new one
	body->clear_ids();
	body->add_ids(new_one);
	for (auto id : children) {
		head->set_sequence(alloc_msg_seq());
		send_data_msg(id, *m);
	}

	return;
}

调用触发
msg_handler::on_recv_node_reg_req

  int ret;
  // 仅fd连接发回注册回包，否则忽略（内存和共享内存通道为单工通道）
  if (nullptr != conn && conn->check_flag(connection::flag_t::REG_FD)) {
    ret = send_reg(::atbus::protocol::msg::kNodeRegisterRsp, n, *conn, rsp_code, m.head().sequence());
    if (rsp_code < 0) {
      ATBUS_FUNC_NODE_ERROR(n, ep, conn, ret, errcode);
      conn->reset();
    }
  } else {
    ret = 0;
  }

  if (ret == 0 && conn->is_connected()) {
	  const auto id = ep->get_id();

	  if (n.is_child_node(id) && id != n.get_id()) {
		  n.on_new_child(id);
	  }
  }

  return ret;

问题
目前发现此事件总是触发两次
不明白的地方是为什么reg会走两遍
代码理解的还不够深, 希望得到指点
谢谢

使用 Windows + MinGW(msys2) 方式编译报错？

$ make
Scanning dependencies of target atframe_utils
[ 3%] Building CXX object deps/atframe_utils/CMakeFiles/atframe_utils.dir/src/algorithm/base64.cpp.obj
[ 6%] Building CXX object deps/atframe_utils/CMakeFiles/atframe_utils.dir/src/algorithm/crc.cpp.obj
[ 9%] Building CXX object deps/atframe_utils/CMakeFiles/atframe_utils.dir/src/algorithm/crypto_cipher.cpp.obj
[ 12%] Building CXX object deps/atframe_utils/CMakeFiles/atframe_utils.dir/src/algorithm/crypto_dh.cpp.obj
[ 16%] Building CXX object deps/atframe_utils/CMakeFiles/atframe_utils.dir/src/algorithm/murmur_hash.cpp.obj
[ 19%] Building CXX object deps/atframe_utils/CMakeFiles/atframe_utils.dir/src/algorithm/xxtea.cpp.obj
[ 22%] Building CXX object deps/atframe_utils/CMakeFiles/atframe_utils.dir/src/cli/cmd_option_list.cpp.obj
[ 25%] Building CXX object deps/atframe_utils/CMakeFiles/atframe_utils.dir/src/cli/cmd_option_value.cpp.obj
[ 29%] Building CXX object deps/atframe_utils/CMakeFiles/atframe_utils.dir/src/cli/shell_font.cpp.obj
[ 32%] Building CXX object deps/atframe_utils/CMakeFiles/atframe_utils.dir/src/common/file_system.cpp.obj
E:/OpenSource/libatbus/3rd_party/atframe_utils/repo/src/common/file_system.cpp: In static member function 'static bool util::file_system::mkdir(const char*, bool, int)':
E:/OpenSource/libatbus/3rd_party/atframe_utils/repo/src/common/file_system.cpp:166:71: error: unused parameter 'mode' [-Werror=unused-parameter]
bool file_system::mkdir(const char *dir_path, bool recursion, int mode) {
~~~~^~~~
cc1plus.exe: all warnings being treated as errors
make[2]: *** [deps/atframe_utils/CMakeFiles/atframe_utils.dir/build.make:180：deps/atframe_utils/CMakeFiles/atframe_utils.dir/src/common/file_system.cpp.obj] 错误 1
make[1]: *** [CMakeFiles/Makefile2:91：deps/atframe_utils/CMakeFiles/atframe_utils.dir/all] 错误 2
make: *** [Makefile:141：all] 错误 2

centos7 编译报错

[root@localhost build]# cmake .. -G "Unix Makefiles"
-- Available Build Type: Unknown
-- Check Flag: -rdynamic -- yes
-- GCC Version 4.8.5 Found, -Wno-unused-local-typedefs added.
-- GCC Version 4.8.5 , using -std=c11/c++14.
-- Build Type: RelWithDebInfo
-- @/home/game/libatbus/build/deps/fmt-6.2.1/build_jobs_linux-x86_64-gnu Run: /usr/bin/cmake ../../../../3rd_party/fmtlib/pkg/fmt-6.2.1 -G 'Unix Makefiles' -DCMAKE_INSTALL_PREFIX=/home/game/libatbus/3rd_party/fmtlib/prebuilt/linux-x86_64-gnu -DCMAKE_POSITION_INDEPENDENT_CODE=YES;-DBUILD_SHARED_LIBS=OFF;-DFMT_DOC=OFF;-DFMT_INSTALL=ON;-DFMT_TEST=OFF;-DFMT_FUZZ=OFF;-DFMT_CUDA_TEST=OFF
-- CMake version: 3.9.4
-- Version: 6.2.1
-- Build type: Release
-- CXX_STANDARD: 11
-- Required features: cxx_variadic_templates
-- Configuring done
-- Generating done
-- Build files have been written to: /home/game/libatbus/build/deps/fmt-6.2.1/build_jobs_linux-x86_64-gnu
-- @/home/game/libatbus/build/deps/fmt-6.2.1/build_jobs_linux-x86_64-gnu Run: /usr/bin/cmake --build . --target install -j
Unknown argument -j
Usage: cmake --build

[options] [-- [native-options]]
Options:

= Project binary directory to be built. --target = Build instead of default targets. May only be specified once. --config = For multi-configuration tools, choose . --clean-first = Build target 'clean' first, then build. (To clean only, use --target 'clean'.) --use-stderr = Ignored. Behavior is default in CMake >= 3.0. -- = Pass remaining options to the native tool. CMake Warning at 3rd_party/atframe_utils/repo/project/cmake/modules/FindConfigurePackage.cmake:454 (find_package): By not providing "Findfmt.cmake" in CMAKE_MODULE_PATH this project has asked CMake to find a package configuration file provided by "fmt", but CMake did not find one.

Could not find a package configuration file provided by "fmt" with any of
the following names:

fmtConfig.cmake
fmt-config.cmake

Add the installation prefix of "fmt" to CMAKE_PREFIX_PATH or set "fmt_DIR"
to a directory containing one of the above files. If "fmt" provides a
separate development package or SDK, be sure it has been installed.
Call Stack (most recent call first):
3rd_party/fmtlib/fmtlib.cmake:69 (FindConfigurePackage)
3rd_party/3rd_party.macro.cmake:4 (include)
CMakeLists.txt:48 (include)

-- @/home/game/libatbus/build/deps/libuv-v1.38.1/build_jobs_linux-x86_64-gnu Run: /usr/bin/cmake ../../../../3rd_party/libuv/pkg/libuv-v1.38.1 -G 'Unix Makefiles' -DCMAKE_INSTALL_PREFIX=/home/game/libatbus/3rd_party/libuv/prebuilt/linux-x86_64-gnu -DCMAKE_POSITION_INDEPENDENT_CODE=YES;-DBUILD_SHARED_LIBS=OFF;-DBUILD_TESTING=OFF
-- summary of build options:
Install prefix: /home/game/libatbus/3rd_party/libuv/prebuilt/linux-x86_64-gnu
Target system: Linux
Compiler:
C compiler: /usr/bin/cc
CFLAGS:

-- Configuring done
-- Generating done
-- Build files have been written to: /home/game/libatbus/build/deps/libuv-v1.38.1/build_jobs_linux-x86_64-gnu
-- @/home/game/libatbus/build/deps/libuv-v1.38.1/build_jobs_linux-x86_64-gnu Run: /usr/bin/cmake --build . --target install -j
Unknown argument -j
Usage: cmake --build

[options] [-- [native-options]]
Options:

= Project binary directory to be built. --target = Build instead of default targets. May only be specified once. --config = For multi-configuration tools, choose . --clean-first = Build target 'clean' first, then build. (To clean only, use --target 'clean'.) --use-stderr = Ignored. Behavior is default in CMake >= 3.0. -- = Pass remaining options to the native tool. -- Could NOT find Libuv (missing: Libuv_INCLUDE_DIRS Libuv_LIBRARIES) -- Dependency: Libuv is required, we can not find prebuilt for libuv and can not find git to clone the sources CMake Error at 3rd_party/libuv/libuv.cmake:102 (message): Libuv not found Call Stack (most recent call first): 3rd_party/3rd_party.macro.cmake:5 (include) CMakeLists.txt:48 (include)

-- Configuring incomplete, errors occurred!
See also "/home/game/libatbus/build/CMakeFiles/CMakeOutput.log".
See also "/home/game/libatbus/build/CMakeFiles/CMakeError.log".

为啥msg_handler::on_recv_data_transfer_rsp没有处理transfer success的情况

int msg_handler::on_recv_data_transfer_rsp(node &n, connection *conn, protocol::msg &m, int /*status*/, int /*errcode*/) {
    if (NULL == m.body.forward || NULL == conn) {
        ATBUS_FUNC_NODE_ERROR(n, NULL == conn ? NULL : conn->get_binding(), conn, EN_ATBUS_ERR_BAD_DATA, 0);
        return EN_ATBUS_ERR_BAD_DATA;
    }

    _if (m.body.forward->to == n.get_id()) {
        ATBUS_FUNC_NODE_ERROR(n, conn->get_binding(), conn, m.head.ret, 0);
        n.on_send_data_failed(conn->get_binding(), conn, &m);
        return EN_ATBUS_ERR_SUCCESS;
    }_

全局路由表下发没有实现吗？

在代码里看到
// TODO 防止短时间内批量上报注册协议，所以合并上报数据包
// TODO 给所有需要全局路由表的子节点下发数据

问一下作者实现没有

parent node居然是child node的child node, 我是哪里用的不对吗？

CASE_TEST(atbus_endpoint, is_child1)
{
atbus::node::conf_t p_conf;
atbus::node::default_conf(&p_conf);
p_conf.children_mask = 16;

atbus::node::conf_t c_conf;
atbus::node::default_conf(&c_conf);
c_conf.children_mask = 8;
{
    atbus::node::ptr_t p_node = atbus::node::create();
    p_node->init(0x12345678, &p_conf);

    atbus::node::ptr_t c_node = atbus::node::create();
    c_node->init(0x12345679, &c_conf);
    
    CASE_EXPECT_TRUE(p_node->is_child_node(c_node->get_id()));
    
    CASE_EXPECT_FALSE(c_node->is_child_node(p_node->get_id()));
}

}

Windows + MSVC 可以使用vcpkg 配置吗？

因为vs2017 调试功能很方便也很强大，很方便新手理解程序。
我在https://github.com/microsoft/vcpkg 发现已经有人移植好了libuv 和 msgpack，我也安装了，那怎么导入如这个工程？需要修改cmake 文件吗？

PS E:\OpenSource\vcpkg> .\vcpkg integrate install
Applied user-wide integration for this vcpkg root.

All MSBuild C++ projects can now #include any installed libraries.
Linking will be handled automatically.
Installing new libraries will make them instantly available.

CMake projects should use: "-DCMAKE_TOOLCHAIN_FILE=E:/OpenSource/vcpkg/scripts/buildsystems/vcpkg.cmake"
PS E:\OpenSource\vcpkg> .\vcpkg.exe install libuv
The following packages will be built and installed:
libuv[core]:x86-windows
Starting package 1/1: libuv:x86-windows
Building package libuv[core]:x86-windows...
-- Downloading https://github.com/libuv/libuv/archive/v1.28.0.tar.gz...
-- Extracting source E:/OpenSource/vcpkg/downloads/libuv-libuv-v1.28.0.tar.gz
-- Using source at E:/OpenSource/vcpkg/buildtrees/libuv/src/v1.28.0-c436f63419
-- Configuring x86-windows
-- Building x86-windows-dbg
-- Building x86-windows-rel
-- Performing post-build validation
-- Performing post-build validation done
Building package libuv[core]:x86-windows... done
Installing package libuv[core]:x86-windows...
Installing package libuv[core]:x86-windows... done
Elapsed time for package libuv:x86-windows: 47.68 s

Total elapsed time: 47.68 s

The package libuv:x86-windows provides CMake targets:

find_package(unofficial-libuv CONFIG REQUIRED)
target_link_libraries(main PRIVATE unofficial::libuv::libuv)

PS E:\OpenSource\vcpkg> .\vcpkg.exe install msgpack
The following packages will be built and installed:
msgpack[core]:x86-windows
Starting package 1/1: msgpack:x86-windows
Building package msgpack[core]:x86-windows...
-- Downloading https://github.com/msgpack/msgpack-c/archive/cpp-3.1.1.tar.gz...
-- Extracting source E:/OpenSource/vcpkg/downloads/msgpack-msgpack-c-cpp-3.1.1.tar.gz
-- Using source at E:/OpenSource/vcpkg/buildtrees/msgpack/src/cpp-3.1.1-eab3c56b41
-- Applying patch E:/OpenSource/vcpkg/ports/msgpack/add-static-lib-option.patch
-- Applying patch failed. This is expected if this patch was previously applied.
-- Configuring x86-windows
-- Building x86-windows-dbg
-- Building x86-windows-rel
-- Performing post-build validation
-- Performing post-build validation done
Building package msgpack[core]:x86-windows... done
Installing package msgpack[core]:x86-windows...
Installing package msgpack[core]:x86-windows... done
Elapsed time for package msgpack:x86-windows: 46.6 s

Total elapsed time: 46.6 s

The package msgpack:x86-windows provides CMake targets:

find_package(msgpack CONFIG REQUIRED)
target_link_libraries(main PRIVATE msgpackc msgpackc-cxx)

PS E:\OpenSource\vcpkg>

关于单读多写的实现机制

同机器之间的进程通信可以通过共享内存管道，想请教下这个单读多写队列的实现原理，网上看过的很多资料都是线程之间通过原子操作来实现，但是在进程之间，是怎么实现多个进程的同步写操作？