Rpc: Http parse

呃这里find("\r\n")会不会更好一点, 如果字段中有\n会解析失败吧

Key: Value1\nValue2

flare/flare/rpc/protocol/http/buffer_io.cc

Line 207 in 8b67602

auto pos = view.find('\n');

logging

flare中的FLARE_LOG_INFO除了fmt::format优化还有其他的么? 有这么一个问题, 假设将日志等级设成WARN, 那么下面这条Hello World的日志一定不会打, 但是这条LOGMessage却还是会生成, 也就是说EXPECT会失败.

  FLARE_LOG_INFO("{}", []() -> const char* {
    EXPECT_FALSE(true);
    return "Hello World!";
  }());

在source code中改变日志等级, 只能用glog提供的FLAGS_minloglevel么? flare为什么没有提供, 我在logging_test中使用FLAGS_minloglevel还得链接上glog, 这个好像是blade的问题, 然后blade直接编译出来的binary也不能直接运行(我想通过-minloglevel改变日志等级), 这两点和bazel不太一样, 以前我还没注意.
./blade-bin/flare/base/internal/logging_test: error while loading shared libraries: libfmt.so.7: cannot open shared object file: No such file or directory

Blade(error): File "//../../BUILD" does not exist

编译环境 gcc 版本 11.2.0 (GCC)

这里缺少头文件, 加入头文件 optional 就行

这个我暂时将/usr/include/sys/cdefs.h中的__glibc_has_attribute (const)删了,希望有更好的解决办法

在thirdparty/curl/BUILD的configure_options里加入'--without-librtmp'和'--without-libpsl'

关于Function类的疑问

相关文件flare/base/function.h，flare没有使用std::function，而是采用自己实现的Function，请问有什么目的或者优势？

问一个非常诡异的用例

https://github.com/Tencent/flare/tree/master/flare/base/compression/compression_test.cc
运行./compression_test gzip

  std::string Compress(
      std::function<bool(Compressor*, CompressionOutputStream* out)> f) {
    if (!with_test_output_stream_) {
      NoncontiguousBufferBuilder builder;
      NoncontiguousBufferCompressionOutputStream out(&builder);
      auto&& c = MakeCompressor(method_);  //  调试这里一直返回nullptr
      EXPECT_TRUE(c);

https://github.com/Tencent/flare/blob/master/flare/base/compression_test.cc
这个用例正常运行。

gzip都已经在cpp文件FLARE_COMPRESSION_REGISTER_COMPRESSOR("gzip", GzipCompressor);注册过了，这是啥情况？

std::apply(_Fn&& __f, _Tuple&& __t) ? std::apply(f(t))

flare/flare/fiber/fiber.h

Line 114 in 2e2f59d

std::apply(std::forward<F>(f)(std::move(t)));

update blade

chen3feng/blade-build#996
现在Python版本高于3.10是没办法编译的

[yadcc ARM] spinlock 在 arm 构建中找不到符号 __aarch64_swp1_acq

flare/flare/base/thread/BUILD

Line 162 in 64d72fc

srcs = 'spinlock.cc',

在 arm 上构建 yadcc 的时候报错

gcc-10.3.0/include/c++/10.3.0/bits/atomic_base.h:443: undefined reference to `__aarch64_swp1_acq'

经查此符号只在 libatomic.a 中有，似乎 gcc 与 atomic 关系有点复杂，临时规避后继续通行

diff --git a/flare/base/thread/BUILD b/flare/base/thread/BUILD
index 60e4805..9a2bed7 100644
--- a/flare/base/thread/BUILD
+++ b/flare/base/thread/BUILD
@@ -163,6 +163,9 @@ cc_library(
   deps = [
     '//flare/base:likely',
   ],
+  linkflags = [
+    '-Wl,-Bstatic -latomic -Wl,-Bdynamic',
+  ],
   visibility = 'PUBLIC',
 )

无法编译项目, 提示没有 GIT LFS? 但是我我确定已经安装

uddf@tuf_test_server:~/flare
$ git lfs install
Updated git hooks.
Git LFS initialized.

uddf@tuf_test_server:~/flare
$ ./blade build
Blade(error): This repository need GIT LFS, see README.md
Blade(error): Blade will exit...
uddf@tuf_test_server:~/flare
$ ./blade build flare
Blade(error): This repository need GIT LFS, see README.md
Blade(error): Blade will exit...
uddf@tuf_test_server:~/flare
$ ./blade build flare/BUILD
Blade(error): This repository need GIT LFS, see README.md
Blade(error): Blade will exit...
uddf@tuf_test_server:~/flare

采用intrusive_ptr的考虑是什么？

base下面实现了一系列intrusive东西，例如intrusive doubly linked list、intrusive smart pointer(class RefPtr)，侵入式设计是比较少见的，是出于怎样的考虑呢？是为了减少一次内存分配取得更好性能考虑吗？另外， intrusive smart pointer 似乎没有循环引用问题，有考虑过这个方面吗？

build glog/press failed

I'm trying build in a docker environment and I have installed libunwind8 and libunwind-dev.

glog(master ✗) pwd                                                                                                                                                                            
/root/git/flare/thirdparty/glog

glog(master ✗) bb :press                                                                                                                                                                      root@dev 19:33:50
Blade: Entering directory `/root/git/flare'
Blade(info): Loading config file "/root/opt/blade-build/blade.conf"
Blade(info): Loading config file "/root/git/flare/BLADE_ROOT"
Blade(info): Loading BUILD files...
Blade(info): Loading done.
Blade(info): Analyzing dependency graph...
Blade(info): Analyzing done.
Blade(info): Generating backend build code...
Blade(info): Generating done.
Blade(info): Building...
Blade(info): Adjust build jobs number(-j N) to be 8
[1/1] LINK BINARY build64_release/thirdparty/glog/press
FAILED: build64_release/thirdparty/glog/press
g++ -o build64_release/thirdparty/glog/press  -Lthirdparty/jdk/lib  @build64_release/thirdparty/glog/press.rsp -lrt -lz -lpthread
/usr/bin/ld: build64_release/thirdparty/glog/lib/libglog.a(libglog_la-utilities.o): in function `google::GetStackTrace(void**, int, int)':
/root/git/flare/build64_release/thirdparty/glog/glog-0.4.0/src/stacktrace_libunwind-inl.h:65: undefined reference to `_Ux86_64_getcontext'
/usr/bin/ld: /root/git/flare/build64_release/thirdparty/glog/glog-0.4.0/src/stacktrace_libunwind-inl.h:66: undefined reference to `_ULx86_64_init_local'
/usr/bin/ld: /root/git/flare/build64_release/thirdparty/glog/glog-0.4.0/src/stacktrace_libunwind-inl.h:78: undefined reference to `_ULx86_64_step'
/usr/bin/ld: /root/git/flare/build64_release/thirdparty/glog/glog-0.4.0/src/stacktrace_libunwind-inl.h:70: undefined reference to `_ULx86_64_get_reg'
/usr/bin/ld: /root/git/flare/build64_release/thirdparty/glog/glog-0.4.0/src/stacktrace_libunwind-inl.h:78: undefined reference to `_ULx86_64_step'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
Blade(error): Build failure.
Blade(info): Cost time 0.267s
Blade(error): Failure

Question about looping for read_events_ and write_events_

Could anyone help to explain why to use a loop for read_events_ here.

if (read_events_.fetch_add(1, std::memory_order_acquire) == 0) {
      RefPtr self_ref(ref_ptr, this);

      do {
        auto rc = OnReadable();
        if (FLARE_LIKELY(rc == EventAction::Ready)) {
          continue;
        } else if (FLARE_UNLIKELY(rc == EventAction::Leaving)) {
          FLARE_CHECK(read_mostly_.seldomly_used->cleanup_reason.load(
                          std::memory_order_relaxed) != CleanupReason::None,
                      "Did you forget to call `Kill()`?");
          GetEventLoop()->AddTask([this] {
            read_events_.store(0, std::memory_order_relaxed);
            QueueCleanupCallbackCheck();
          });
          break;
        } else if (rc == EventAction::Suppress) {
          SuppressReadAndClearReadEventCount();
          break;
        }
      } while (read_events_.fetch_sub(1, std::memory_order_release) !=
               1);
      QueueCleanupCallbackCheck();
    });

We can only get read_events_ == 1 after the if condition, so is there any special case that looping to decrease read_events_ is necessary?

-Dconst= 在高版本gcc下会报错

类似于brpc，这里fiber实现中也定义了-D__const__=

https://github.com/Tencent/flare/blob/master/flare/fiber/BUILD#L209

目前brpc遇到了高版本gcc下编译错误的问题：apache/brpc#1693

fiber这里可能也会遇到

Timer

flare/flare/fiber/timer.cc

Lines 53 to 83 in 17b70d1

    
           std::uint64_t SetTimer(std::chrono::steady_clock::time_point at, 
        
                                  std::chrono::nanoseconds interval, 
        
                                  Function<void(std::uint64_t)>&& cb) { 
        
             // This is ugly. But since we have to start a fiber each time user's `cb` is 
        
             // called, we must share it. 
        
             // 
        
             // We also take measures not to call user's callback before the previous call 
        
             // has returned. Otherwise we'll likely crash user's (presumably poor) code. 
        
             struct UserCallback { 
        
               void Run(std::uint64_t tid) { 
        
                 if (!running.exchange(true, std::memory_order_acq_rel)) { 
        
                   cb(tid); 
        
                 } 
        
                 running.store(false, std::memory_order_relaxed); 
        
                 // Otherwise this call is lost. This can happen if user's code runs too 
        
                 // slowly. For the moment we left the behavior as unspecified. 
        
               } 
        
               Function<void(std::uint64_t)> cb; 
        
               std::atomic<bool> running{}; 
        
             }; 
        
             auto ucb = std::make_shared<UserCallback>(); 
        
             ucb->cb = std::move(cb); 
        
             auto sg = detail::NearestSchedulingGroup(); 
        
             auto timer_id = sg->CreateTimer(at, interval, [ucb](auto tid) mutable { 
        
               internal::StartFiberDetached([ucb, tid] { ucb->cb(tid); }); 
        
             }); 
        
             sg->EnableTimer(timer_id); 
        
             return timer_id; 
        
           }

没有看懂UserCallback的作用, Run函数完全没有调用啊, running也就显得乏力了

Why use std::common_type in TryParseTraits

Could anyone help to explain why std::common_type is need here instead of just using T()?

template <class T, class>
struct TryParseTraits {
  // The default implementation delegates calls to `TryParse` found by ADL.
  template <class... Args>
  static std::optional<T> TryParse(std::string_view s, const Args&... args) {
    return TryParse(std::common_type<T>(), s, args...);
  }
};

本地构建/开始使用打不开

你好，在使用时碰到两个问题：

1）本地构建

本地构建使用./blade build ...时，会提示以下错误：
Blade(error): This repository need GIT LFS, see README.md
Blade(error): Blade will exit...

打开对应提示文件，发现blade.zip 是小于判断的1024字节，请问这个是什么原因呢？

2）开始使用指向的文档链接打不开：https://github.com/Tencent/flare/blob/master/flare/doc/getting-started.md

谢谢

关于调度参数的疑问

请问一下，这里针对不同策略的计算是有什么依据吗（根据什么原理确定的参数）？

SchedulingParameters GetSchedulingParameters(SchedulingProfile profile,
                                             std::size_t numa_domains,
                                             std::size_t available_processors,
                                             std::size_t desired_concurrency) {
  if (profile == SchedulingProfile::ComputeHeavy) {
    return GetSchedulingParametersForComputeHeavy(desired_concurrency);
  } else if (profile == SchedulingProfile::Compute) {
    return GetSchedulingParametersForCompute(numa_domains, available_processors,
                                             desired_concurrency);
  } else if (profile == SchedulingProfile::Neutral) {
    // @sa: `SchedulingProfile` for the constants below.
    return GetSchedulingParametersOfGroupSize(numa_domains, desired_concurrency,
                                              16, 32);
  } else if (profile == SchedulingProfile::Io) {
    return GetSchedulingParametersOfGroupSize(numa_domains, desired_concurrency,
                                              12, 24);
  } else if (profile == SchedulingProfile::IoHeavy) {
    return GetSchedulingParametersOfGroupSize(numa_domains, desired_concurrency,
                                              8, 16);
  }
  FLARE_UNREACHABLE("Unexpected scheduling profile [{}].",
                    underlying_value(profile));
}

FLARE_CHECK 等字符过多，可以简化

首先要说代码质量相当不错，功能也相当充分，完全可以作为基础库了。

提个小建议

FLARE_LOG_INFO
FLARE_LOG_WARNING
FLARE_CHECK
FLARE_LIKELY
FLARE_DCHECK
FLARE_LOG_WARNING_EVERY_SECOND

等宏名字过于冗长，可以大气一些直接改为 FLOG_INFO, FCHECK 等

namespace flare 可能会用得比较多，如果有更简洁的方案，当采用之

这个owning_strs_2_为什么不用std::vector<std::string>, 从代码中看, 只用到了push_back和clear
还有就是internal::CaseInsensitiveHashMap<std::string_view, std::size_t> , 为什么不直接使用internal::CaseInsensitiveHashMap<std::string_view, std::string_view>, 感觉没有必要映射一个size_t, 再多加一个using NonowningFields = std::vector<std::pair<std::string_view, std::string_view>>;

flare/flare/net/http/http_headers.h

Lines 161 to 167 in bb5cd22

    
           std::deque<std::string> owning_strs_2_; 
        
           // For better lookup (which is likely to be done frequently if we're 
        
           // parsing, instead of building, headers) performance, we build this map for 
        
           // that purpose. Values are indices into `fields_`. 
        
           internal::CaseInsensitiveHashMap<std::string_view, std::size_t> header_idx_; 
        
           // Referencing either `buffer_` or `owning_strs_`. 
        
           NonowningFields fields_;

Update blade build tools to more common build tools Bazel

Noticed this framework is using Tencent Blade as build tools, but with open sources, Bazel is more common.
Please update blade to Bazel

About unboxed_type_t

flare/flare/base/future/basics.h

Lines 125 to 129 in d90635d

    
           template <class... Ts> 
        
           using unboxed_type_t = typename std::conditional_t< 
        
               sizeof...(Ts) == 0, std::common_type<void>, 
        
               std::conditional_t<sizeof...(Ts) == 1, std::common_type<Ts...>, 
        
                                  std::common_type<std::tuple<Ts...>>>>::type;

这个会不会更好点，更易于理解？ common_type有点难懂

template <class T0 = void, class... Ts>
struct unboxed_type {
  using type = std::conditional_t<sizeof...(Ts) == 0, T0, std::tuple<T0, Ts...>>;
};

template <class... Ts>
using unboxed_type_t = typename unboxed_type<Ts...>::type;

请问有合适的源码阅读顺序吗

代码质量很高，配合doc阅读也有点吃力，目前卡在flare/base这个目录，关联的东西很多，不知道从哪下手了。希望后续doc能有图~

if have any plan to support bazel

or how can i build with bazel

bazel build ... 失败

checking run-time libs availability... failed
configure: error: one or more libs available at link-time are not available run-time. Libs used at link-time: -lnghttp2   -lssl -lcrypto -lz 
_____ END BUILD LOGS _____
rules_foreign_cc: Build wrapper script location: bazel-out/k8-fastbuild/bin/external/com_github_curl_curl/curl_foreign_cc/wrapper_build_script.sh
rules_foreign_cc: Build script location: bazel-out/k8-fastbuild/bin/external/com_github_curl_curl/curl_foreign_cc/build_script.sh
rules_foreign_cc: Build log location: bazel-out/k8-fastbuild/bin/external/com_github_curl_curl/curl_foreign_cc/Configure.log

How to build flare using blade?

Hi there,
I have installed blade and ninja but don't know how to build flare or examples in flare project. I have no experience of blade and ninja. I think I installed blade and ninja correctly. Below is some log:

`luis@ubuntu:/mnt/hgfs/E/flare/flare/example$ blade -h
usage: blade [-h] [--version] {build,run,test,clean,query,dump} ...

blade [options...] [targets...]

positional arguments:
{build,run,test,clean,query,dump}
Available subcommands
build Build specified targets
run Build and runs a single target
test Build the specified targets and runs tests
clean Remove all blade-created output
query Execute a dependency graph query
dump Dump specified internal information

optional arguments:
-h, --help show this help message and exit
--version show program's version number and exit
luis@ubuntu:/mnt/hgfs/E/flare/flare/example$ ninja --version
1.10.0
luis@ubuntu:/mnt/hgfs/E/flare/flare/example$
`

About scheduler_lock

flare/flare/fiber/detail/fiber_entity.h

Lines 90 to 106 in 37be012

    
           // This lock is held when the fiber is in state-transition (e.g., from running 
        
           // to suspended). This is required since it's inherent racy when we add 
        
           // ourselves into some wait-chain (and eventually woken up by someone else) 
        
           // and go to sleep. The one who wake us up can be running in a different 
        
           // pthread, and therefore might wake us up even before we actually went sleep. 
        
           // So we always grab this lock before transiting the fiber's state, to ensure 
        
           // that nobody else can change the fiber's state concurrently. 
        
           // 
        
           // For waking up a fiber, this lock is grabbed by whoever the waker; 
        
           // For a fiber to go to sleep, this lock is grabbed by the fiber itself and 
        
           // released by *`SchedulingGroup`* (by the time we're sleeping, we cannot 
        
           // release the lock ourselves.). 
        
           // 
        
           // This lock also protects us from being woken up by several pthread 
        
           // concurrently (in case we waited on several waitables and have not removed 
        
           // us from all of them before more than one of then has fired.). 
        
           Spinlock scheduler_lock;

flare/flare/fiber/detail/fiber_entity.h

Lines 354 to 356 in 37be012

    
           // Argument `context` (i.e., `this`) is only used the first time the context 
        
           // is jumped to (in `FiberProc`). 
        
           jump_context(&caller->state_save_area, state_save_area, this);

或许可以利用一下这个Argument context， fiber切换过去再将caller fiber的State改变，状态改变的时候就不需要这把锁了？

inline void FiberEntity::Resume() noexcept {
  SetCurrentFiberEntity(this);
  state = FiberState::Running;
  // Argument `context`  set caller
  auto caller_ = jump_context(&caller->state_save_area, state_save_area, caller);
  // caller_ set nullptr when fiber return
  if (caller_) {
    static_cast<FiberEntity*>(caller_)->state = FiberState::Waiting;
  }
  ...
}

static void FiberProc(void* context) {
  auto caller = reinterpret_cast<FiberEntity*>(context);
  caller->state = FiberState::Waiting;
  //....
  current_fiber->state = FiberState::Dead;
  GetMasterFiberEntity()->M_return([](){...});
}

void FiberEntity::M_return(Function<void()>&& cb) noexcept {
  // set `resume_proc` ....
  SetCurrentFiberEntity(this);
  state = FiberState::Running;
  // set Argument `context`  nullptr
  jump_context(&caller->state_save_area, state_save_area, nullptr);
}

add constexpr

flare/flare/base/function.h

Line 305 in 2e2f59d

if (sizeof(std::decay_t<T>) <= kMaximumOptimizableSize) {

这里加个constexpr吧,虽然编译器会优化掉

Fail to build / run on Ubuntu 20.04

An undefined reference to pthread_once (even if we've already link against pthread);
link_all_symbols does not work correctly with shared linking.

Does the server example in the flare example support 1 thread start?

./build64_release/flare/example/rpc/server --port=5510 --logtostderr --flare_concurrency_hint=1 --flare_enable_watchdog=true

I1020 15:56:15.628629 27869 flare/init.cc:114] Flare started.
I1020 15:56:15.629772 27869 flare/fiber/runtime.cc:425] Using fiber scheduling profile [io-heavy].
I1020 15:56:15.629823 27869 flare/fiber/runtime.cc:220] Starting 1 worker threads per group, for a total of 1 groups. The system is treated as UMA.
I1020 15:56:15.635567 27869 flare/init.cc:122] Flare runtime initialized.

Unable to start successfully

How could I use flare in my own project?

Flare is an awesome rpc framework and flare/base has many usefull function capsulations, and I want to use them in my own project.
There is no CMakeFiles or configure script, I do no know how to build it as an indepedent lib.
So I tried to use BLADE too. My project has the following directory tree.

|__code
    |__BLADE_ROOT: my own BLADE_ROOT
    |__thirdparty
        |__flare: a git submodule pointing to Tencent/flare

I event wrote a tool (https://gist.github.com/ericuni/627cacb9978c9db58ae7360c4d31de6b) to process all BUILD files and BLADE_ROOT in flare.
But still failed.

blade Support generating compdb

I see generating compdb in blade-build

Fail to build

./blade build ... gave me the following errors

ERROR:root:code` for hash sha512 was not found.
Traceback (most recent call last):
  File "/home/server_dev/.blade_boost/lib/python2.7/hashlib.py", line 139, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/home/server_dev/.blade_boost/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor
    raise ValueError('unsupported hash type %s' % name)
ValueError: unsupported hash type sha512
...
... (similar error for other hashes)
...
Traceback (most recent call last):
  File "/home/server_dev/.blade_boost/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/home/server_dev/.blade_boost/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/server_dev/repo/flare/thirdparty/blade/blade.zip/__main__.py", line 14, in <module>
  File "/home/server_dev/repo/flare/thirdparty/blade/blade.zip/blade/__init__.py", line 6, in <module>
  File "/home/server_dev/repo/flare/thirdparty/blade/blade.zip/blade/config.py", line 422, in <module>
  File "/home/server_dev/repo/flare/thirdparty/blade/blade.zip/blade/config.py", line 44, in __init__
AttributeError: 'module' object has no attribute 'md5'

I though it was my openssl lib being missing, reinstalled but still no luck.

Question About Future and Async: Will it blocking until the the result has been product

Since that brpc do not has the future type, and std::async will:

If the std::future obtained from std::async is not moved from or bound to a reference, the destructor of the std::future will block at the end of the full expression until the asynchronous operation completes, essentially making code such as the following synchronous

And folly::Future will detach when destructor.

I have through the documents:

I didn't find anywhere that says if it will block or detach when destruct. And I go through the code, and find it will share a std::shared_ptr<Core>, and it will do like detached. Did I catched it?

clang-17 asan/tsan fiber coredump

environment

docker ubuntu22.04 clang 17 bazel

.bazelrc add new configs

build:tsan --config=san-common --copt=-fsanitize=thread --linkopt=-fsanitize=thread
build:tsan_llvm --config=llvm --config=tsan
build:asan_llvm --config=llvm --config=asan

future_test tsan coredump

build command : bazel build --config tsan_llvm //flare/fiber:*
log:

# ./bazel-bin/flare/fiber/future_test
Running main() from gmock_main.cc
[==========] Running 3 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 3 tests from Future
[ RUN      ] Future.BlockingGet
WARNING: Logging before InitGoogleLogging() is written to STDERR
I1102 03:38:13.659870 1061357 runtime.cc:425] Using fiber scheduling profile [neutral].
I1102 03:38:13.660069 1061357 runtime.cc:224] Starting 18 worker threads per group, for a total of 2 groups. The system is treated as UMA.
ThreadSanitizer:DEADLYSIGNAL
==1061357==ERROR: ThreadSanitizer: SEGV on unknown address 0x7f82f653bff8 (pc 0x558d1af98b66 bp 0x7f830f331df0 sp 0x7f830f331be8 T1061423)
==1061357==The signal is caused by a WRITE memory access.
ThreadSanitizer:DEADLYSIGNAL
ThreadSanitizer: nested bug in the same thread, aborting.

future_test asan coredump

build command : bazel build --config asan_llvm //flare/fiber:*
log:

./bazel-bin/flare/fiber/future_test
Running main() from gmock_main.cc
[==========] Running 3 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 3 tests from Future
[ RUN      ] Future.BlockingGet
WARNING: Logging before InitGoogleLogging() is written to STDERR
I1102 03:45:45.175843 1084756 runtime.cc:425] Using fiber scheduling profile [neutral].
I1102 03:45:45.176086 1084756 runtime.cc:224] Starting 18 worker threads per group, for a total of 2 groups. The system is treated as UMA.
AddressSanitizer:DEADLYSIGNAL
.......
=================================================================
==1084756==ERROR: AddressSanitizer: SEGV on unknown address 0x7f52e0199a00 (pc 0x55e2ca67c61d bp 0x7f52e05be6d0 sp 0x7f52e05be600 T42)
==1084756==The signal is caused by a WRITE memory access.
AddressSanitizer:DEADLYSIGNAL
AddressSanitizer:DEADLYSIGNAL
    #0 0x55e2ca67c61d in flare::internal::asan::StartSwitchFiber(void**, void const*, unsigned long) /proc/self/cwd/./flare/base/internal/annotation.h:161:1
    #1 0x55e2ca67af9c in flare::fiber::detail::FiberEntity::Resume() /proc/self/cwd/./flare/fiber/detail/fiber_entity.h:343:3
    #2 0x55e2ca684aef in flare::fiber::detail::FiberEntity::ResumeOn(flare::Function<void ()>&&) /proc/self/cwd/flare/fiber/detail/fiber_entity.cc:179:3
    #3 0x55e2ca6872eb in flare::fiber::detail::FiberProc(void*) /proc/self/cwd/flare/fiber/detail/fiber_entity.cc:133:29

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /proc/self/cwd/./flare/base/internal/annotation.h:161:1 in flare::internal::asan::StartSwitchFiber(void**, void const*, unsigned long)
Thread T42 created by T0 here:
    #0 0x55e2ca4ade7d in pthread_create (/root/.cache/bazel/_bazel_root/c5f028faa5e19a5e19c31dee93eeaf11/execroot/__main__/bazel-out/k8-dbg/bin/flare/fiber/future_test+0x111e7d) (BuildId: 40d8dc05e8ca6ce1fa4f444d1ddbaa8b42ab93e8)
    #1 0x7f5313b16328 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State>>, void (*)()) (/lib/x86_64-linux-gnu/libstdc++.so.6+0xdc328) (BuildId: e37fe1a879783838de78cbc8c80621fa685d58a2)
    #2 0x55e2ca6783a0 in flare::fiber::detail::FiberWorker::Start(bool) /proc/self/cwd/flare/fiber/detail/fiber_worker.cc:40:13
    #3 0x55e2ca575750 in flare::fiber::(anonymous namespace)::FullyFledgedSchedulingGroup::Start(bool) /proc/self/cwd/flare/fiber/runtime.cc:118:10
    #4 0x55e2ca57277a in flare::fiber::StartRuntime() /proc/self/cwd/flare/fiber/runtime.cc:490:11
    #5 0x55e2ca505034 in void flare::fiber::testing::RunAsFiber<flare::Future_BlockingGet_Test::TestBody()::$_0>(flare::Future_BlockingGet_Test::TestBody()::$_0&&) /proc/self/cwd/./flare/fiber/detail/testing.h:41:3
    #6 0x55e2ca504e9e in flare::Future_BlockingGet_Test::TestBody() /proc/self/cwd/flare/fiber/future_test.cc:33:3
    #7 0x55e2cab8f1ff in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2599:10
    #8 0x55e2cab552b6 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2635:14
    #9 0x55e2cab11bcd in testing::Test::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2674:5
    #10 0x55e2cab138f4 in testing::TestInfo::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2853:11
    #11 0x55e2cab14fc0 in testing::TestSuite::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:3012:30
    #12 0x55e2cab3937c in testing::internal::UnitTestImpl::RunAllTests() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:5870:44
    #13 0x55e2cab95daf in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2599:10
    #14 0x55e2cab5b86d in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2635:14
    #15 0x55e2cab38655 in testing::UnitTest::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:5444:10
    #16 0x55e2caaf1ca0 in RUN_ALL_TESTS() /proc/self/cwd/external/com_google_googletest/googletest/include/gtest/gtest.h:2293:73
    #17 0x55e2caaf1bfe in main /proc/self/cwd/external/com_google_googletest/googlemock/src/gmock_main.cc:70:10
    #18 0x7f531371ed8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

==1084756==ABORTING

http json 转 protobuf message 出错

flare/flare/rpc/protocol/protobuf/gdt_json_proto_conversion.cc

Line 326 in 6d3bf3c

} else if (auto opt = TryParse<valuetype>(value.asString())) { \

这里处理repeated字段，是不是应该写成 item.asString() , 而不是value.asString()，因为value是一个数组。

} else if (auto opt = TryParse(item.asString())) {

coredump when http overloaded

flare/flare/rpc/protocol/protobuf/message.cc

Line 51 in 193b1f9

return std::make_unique<ProtoMessage>(std::move(meta), nullptr);

这上面是不是要 return std::make_unique(std::move(meta), std::monostate());

在 http overloaded 后调用 NormalConnectionHandler::WriteOverloaded

flare/flare/rpc/internal/normal_connection_handler.cc

Lines 347 to 351 in 193b1f9

    
           auto msg = factory->Create(MessageFactory::Type::Overloaded, 
        
                                      corresponding_req.GetCorrelationId(), stream); 
        
           if (msg) {  // Note that `MessageFactory::Create` may return `nullptr`. 
        
             WriteMessage(*msg, protocol, controller, kFastCallReservedContextId); 
        
           }

然后一直执行到在下面 msg.msg_or_buffer.index() == 1 后拿到的 pb 是个空的，

flare/flare/rpc/protocol/protobuf/proto_over_http_protocol.cc

Lines 902 to 906 in 193b1f9

    
           } else if (msg.msg_or_buffer.index() == 1) { 
        
             auto&& pb = *std::get<1>(msg.msg_or_buffer); 
        
             std::string rc; 
        
             if (content_type_ == ContentType::kApplicationJson) { 
        
               FLARE_CHECK(ProtoMessageToJson(pb, &rc, nullptr));

导致 ProtoMessageToJson coredump

flare/flare/rpc/protocol/protobuf/gdt_json_proto_conversion.cc

Line 117 in 193b1f9

if (!message.IsInitialized()) {

使用`getaddrinfo`会不会更好一点

flare/flare/rpc/name_resolver/util/domain_name_resolver.cc

Lines 45 to 58 in bb5cd22

    
             struct hostent* he = nullptr; 
        
             int error = 0; 
        
           #ifdef _GNU_SOURCE 
        
             char buf[4096]; 
        
             struct hostent he_buf; 
        
             gethostbyname_r(hostname.c_str(), &he_buf, buf, sizeof(buf), &he, &error); 
        
           #else 
        
             he = gethostbyname(hostname.c_str()); 
        
             if (!he) error = h_errno; 
        
           #endif 
        
             if (!he) { 
        
               SetErrorCode(error_code, error); 
        
               return false; 
        
             }

针对K8s容器环境是否有需要注意的优化配置呢

是否有基础Docker镜像？

Cpu Profiler 是不是依赖的tcmalloc呢？

想请问一下，Cpu Profiler 这个功能是不是依赖了tcmalloc呀，看接口有点像..

DetachTimer的设计权衡是怎样的？

DetachTimer()是应用于怎样的场景呢？注释中说 can be helpful in fire-and-forget cases. 我的理解是单次定时任务（不重复定时），但已经存在了对应的CreateTimer()接口了，对于DetachTimer的设计思考以及具体作用不太理解。

对GetFreeCount的取值比较困惑

std::size_t GetFreeCount(std::size_t upto) {
  return std::min(upto, std::max(upto / 2, kMinimumFreePerWash));
}

当 upto > 2 * kMinimumFreePerWash，取 upto / 2 显然大于 kMinimumFreePerWash。
当 2 * kMinimumFreePerWash >= upto >= kMinimumFreePerWash ，取 kMinimumFreePerWash。
当 kMinimumFreePerWash > upto ，取 upto 显然小于 kMinimumFreePerWash。

请问这样取值的意义在哪？L32C1-L32C1

编译时间太长

bash -c "./blade test ..."

在WSL2 ubuntu20.04下编译要半个多小时，有时候我可能只改了一个.h文件都要这么久，请问有什么解决方案吗

Unable to link, ubuntu 20.04

blade build ... works, no errors, however when running blade test, many .so cannot be correctly linked.
Latest head of flare, latest head of blade-build

Details

blade test flare/base:endian_test
Blade(info): Loading BUILD files...
Blade(info): Loading done.
Blade(info): Analyzing dependency graph...
Blade(info): Analyzing done.
Blade(info): Generating backend build code...
Blade(info): Generating done.
Blade(info): Building...
Blade(info): Adjust build jobs number(-j N) to be 20
ninja: warning: bad deps log signature or version; starting over
[46/46] LINK BINARY build64_release/flare/base/endian_test
Blade(info): Build success.
Blade(info): Adjust test jobs number(-j N) to be 10
Blade(notice): 1 tests to run
Blade(info): Spawn 1 worker thread(s) to run concurrent tests
repo/flare/build64_release/flare/base/endian_test: error while loading shared libraries: libgtest_main.so: cannot open shared object file: No such file or directory
Blade(info): [1/0/1] Test //flare/base:endian_test finished : FAILED:127

ldd result:

~/repo/flare$ ldd blade-bin/flare/base/endian_test
linux-vdso.so.1 (0x00007ffc89d05000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007feb7aab2000)
libgtest_main.so => not found
libgtest.so => not found
libtcmalloc.so.4 => not found
libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007feb7a8d1000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007feb7a8b6000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007feb7a6c2000)
/lib64/ld-linux-x86-64.so.2 (0x00007feb7aafe000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007feb7a573000)

I wonder how does link work in blade?

Compile Failed(Third party ctemplate). Gcc version 11.2.0

Any plan to add c++20 coroutine support？

First of all, it's a great job. My question is that do u have any further plan to support native c++20 coroutine?

Add constexpr to `insert` function

flare/flare/base/experimental/byte_set.h

Line 33 in 64d72fc

insert(bytes);

./flare/base/experimental/byte_set.h: In constructor 'constexpr flare::experimental::ByteSet::ByteSet(std::string_view)':
./flare/base/experimental/byte_set.h:33:11: error: call to non-'constexpr' function 'void flare::experimental::ByteSet::insert(std::string_view)'
   33 |     insert(bytes);

为什么侵入式的链表采用的是组合而不是继承?

struct DoublyLinkedListEntry {
  DoublyLinkedListEntry* prev = this;
  DoublyLinkedListEntry* next = this;
};

template <class T>
class DoubleLyLinkedList {
public:
  static_assert(std::is_base_of_v<DoublyLinkedListEntry, T>);

private:
  static constexpr T* object_cast(DoublyLinkedListEntry* entry) noexcept {
    return reinterpret_cast<T*>(entry);
  }
  static constexpr DoublyLinkedListEntry* node_cast(T* ptr) noexcept {
    return reinterpret_cast<DoublyLinkedListEntry*>(ptr);
  }

  size_t size_{};
  DoublyLinkedListEntry head_;
};

  // 组合
  struct C {
    DoublyLinkedListEntry chain;
    int x;
  };
  DoublyLinkedList<C, &C::chain> list;
 // 继承
  struct C : public DoublyLinkedListEntry {
    int x;
  };
  DoublyLinkedList<C> list;

clang-format

噢噢是这样的我看到代码有些地方的格式不太对
@0x804d8000 你们用的clang-format版本是多少
关于google style 头文件的问题, clang-format文件能不能修改做到格式化

	std::uint64_t SetTimer(std::chrono::steady_clock::time_point at,
	std::chrono::nanoseconds interval,
	Function<void(std::uint64_t)>&& cb) {
	// This is ugly. But since we have to start a fiber each time user's `cb` is
	// called, we must share it.
	//
	// We also take measures not to call user's callback before the previous call
	// has returned. Otherwise we'll likely crash user's (presumably poor) code.
	struct UserCallback {
	void Run(std::uint64_t tid) {
	if (!running.exchange(true, std::memory_order_acq_rel)) {
	cb(tid);
	}
	running.store(false, std::memory_order_relaxed);
	// Otherwise this call is lost. This can happen if user's code runs too
	// slowly. For the moment we left the behavior as unspecified.
	}
	Function<void(std::uint64_t)> cb;
	std::atomic<bool> running{};
	};

	auto ucb = std::make_shared<UserCallback>();
	ucb->cb = std::move(cb);

	auto sg = detail::NearestSchedulingGroup();
	auto timer_id = sg->CreateTimer(at, interval, [ucb](auto tid) mutable {
	internal::StartFiberDetached([ucb, tid] { ucb->cb(tid); });
	});
	sg->EnableTimer(timer_id);
	return timer_id;
	}

	std::deque<std::string> owning_strs_2_;
	// For better lookup (which is likely to be done frequently if we're
	// parsing, instead of building, headers) performance, we build this map for
	// that purpose. Values are indices into `fields_`.
	internal::CaseInsensitiveHashMap<std::string_view, std::size_t> header_idx_;
	// Referencing either `buffer_` or `owning_strs_`.
	NonowningFields fields_;

	template <class... Ts>
	using unboxed_type_t = typename std::conditional_t<
	sizeof...(Ts) == 0, std::common_type<void>,
	std::conditional_t<sizeof...(Ts) == 1, std::common_type<Ts...>,
	std::common_type<std::tuple<Ts...>>>>::type;

	// This lock is held when the fiber is in state-transition (e.g., from running
	// to suspended). This is required since it's inherent racy when we add
	// ourselves into some wait-chain (and eventually woken up by someone else)
	// and go to sleep. The one who wake us up can be running in a different
	// pthread, and therefore might wake us up even before we actually went sleep.
	// So we always grab this lock before transiting the fiber's state, to ensure
	// that nobody else can change the fiber's state concurrently.
	//
	// For waking up a fiber, this lock is grabbed by whoever the waker;
	// For a fiber to go to sleep, this lock is grabbed by the fiber itself and
	// released by `SchedulingGroup` (by the time we're sleeping, we cannot
	// release the lock ourselves.).
	//
	// This lock also protects us from being woken up by several pthread
	// concurrently (in case we waited on several waitables and have not removed
	// us from all of them before more than one of then has fired.).
	Spinlock scheduler_lock;

	// Argument `context` (i.e., `this`) is only used the first time the context
	// is jumped to (in `FiberProc`).
	jump_context(&caller->state_save_area, state_save_area, this);

	auto msg = factory->Create(MessageFactory::Type::Overloaded,
	corresponding_req.GetCorrelationId(), stream);
	if (msg) { // Note that `MessageFactory::Create` may return `nullptr`.
	WriteMessage(*msg, protocol, controller, kFastCallReservedContextId);
	}

	} else if (msg.msg_or_buffer.index() == 1) {
	auto&& pb = *std::get<1>(msg.msg_or_buffer);
	std::string rc;
	if (content_type_ == ContentType::kApplicationJson) {
	FLARE_CHECK(ProtoMessageToJson(pb, &rc, nullptr));

	struct hostent* he = nullptr;
	int error = 0;
	#ifdef _GNU_SOURCE
	char buf[4096];
	struct hostent he_buf;
	gethostbyname_r(hostname.c_str(), &he_buf, buf, sizeof(buf), &he, &error);
	#else
	he = gethostbyname(hostname.c_str());
	if (!he) error = h_errno;
	#endif
	if (!he) {
	SetErrorCode(error_code, error);
	return false;
	}

tencent / flare Goto Github PK

flare's People

Stargazers

Watchers

Forkers

flare's Issues