venediktov / vanilla-rtb Goto Github PK
View Code? Open in Web Editor NEWReal Time Bidding (RTB) - Demand Side Platform framework
Home Page: http://forkbid.com
License: GNU General Public License v3.0
Real Time Bidding (RTB) - Demand Side Platform framework
Home Page: http://forkbid.com
License: GNU General Public License v3.0
Boost serialization is optional in our stack and only used for multi-bidder model.
It has to be drawn in dotted lines or diagram needs to have comment in the bottom.
Hi
I want to build your project and follow your instructions.
I use ubuntu 20.04
,boost Version: 1.71.0
and cmake version 3.23.2
.
as I run cmake -DCMAKE_BUILD_TYPE=Debug .. -G "Unix Makefiles"
:
-- The CXX compiler identification is GNU 9.4.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Boost: /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake (found suitable version "1.71.0", minimum required is "1.67.0")
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
CMAKE_INSTALL_PREFIX /home/parisa/Projecs/Parisa_G/vanilla-rtb/Release/examples
CMAKE_CURRENT_SOURCE_DIR /home/parisa/Projecs/Parisa_G/vanilla-rtb
CMAKE_CURRENT_BIN_DIR /home/parisa/Projecs/Parisa_G/vanilla-rtb/Release
CMake Deprecation Warning at jsonv/CMakeLists.txt:13 (cmake_minimum_required):
Compatibility with CMake < 2.8.12 will be removed from a future version of
CMake.
Update the VERSION argument <min> value or use a ...<max> suffix to tell
CMake that the project does not need compatibility with older versions.
-- The C compiler identification is GNU 9.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- JSONV version: 1.2.0
-- Found Boost: /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake (found version "1.71.0")
-- RUNTIME_INSTALL_DIR == /home/parisa/Projecs/Parisa_G/vanilla-rtb/Release/examples/bin
-- LIBRARY_INSTALL_DIR == /home/parisa/Projecs/Parisa_G/vanilla-rtb/Release/examples/lib64
-- INCLUDE_INSTALL_DIR == /home/parisa/Projecs/Parisa_G/vanilla-rtb/Release/examples/include
-- CONFIG_INSTALL_DIR == /home/parisa/Projecs/Parisa_G/vanilla-rtb/Release/examples/lib64/cmake/jsonv
CMake Error at /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component):
boost_find_component Macro invoked with incorrect arguments for macro
named: boost_find_component
Call Stack (most recent call first):
/usr/share/cmake-3.23/Modules/FindBoost.cmake:594 (find_package)
CRUD/service/CMakeLists.txt:10 (find_package)
CMake Error at /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component):
boost_find_component Macro invoked with incorrect arguments for macro
named: boost_find_component
Call Stack (most recent call first):
/usr/share/cmake-3.23/Modules/FindBoost.cmake:594 (find_package)
CRUD/service/CMakeLists.txt:10 (find_package)
-- Found Boost: /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake (found version "1.71.0") missing components: system regex
-- Found Boost: /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake (found version "1.71.0") found components: log system serialization program_options
CMake Error at /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component):
boost_find_component Macro invoked with incorrect arguments for macro
named: boost_find_component
Call Stack (most recent call first):
/usr/share/cmake-3.23/Modules/FindBoost.cmake:594 (find_package)
rtb/CMakeLists.txt:21 (find_package)
CMake Error at /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component):
boost_find_component Macro invoked with incorrect arguments for macro
named: boost_find_component
Call Stack (most recent call first):
/usr/share/cmake-3.23/Modules/FindBoost.cmake:594 (find_package)
rtb/CMakeLists.txt:21 (find_package)
CMake Error at /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component):
boost_find_component Macro invoked with incorrect arguments for macro
named: boost_find_component
Call Stack (most recent call first):
/usr/share/cmake-3.23/Modules/FindBoost.cmake:594 (find_package)
rtb/CMakeLists.txt:21 (find_package)
-- Found Boost: /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake (found version "1.71.0") missing components: log serialization system
CMake Error at /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component):
boost_find_component Macro invoked with incorrect arguments for macro
named: boost_find_component
Call Stack (most recent call first):
/usr/share/cmake-3.23/Modules/FindBoost.cmake:594 (find_package)
examples/CMakeLists.txt:6 (find_package)
CMake Error at /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component):
boost_find_component Macro invoked with incorrect arguments for macro
named: boost_find_component
Call Stack (most recent call first):
/usr/share/cmake-3.23/Modules/FindBoost.cmake:594 (find_package)
examples/CMakeLists.txt:6 (find_package)
CMake Error at /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component):
boost_find_component Macro invoked with incorrect arguments for macro
named: boost_find_component
Call Stack (most recent call first):
/usr/share/cmake-3.23/Modules/FindBoost.cmake:594 (find_package)
examples/CMakeLists.txt:6 (find_package)
CMake Error at /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component):
boost_find_component Macro invoked with incorrect arguments for macro
named: boost_find_component
Call Stack (most recent call first):
/usr/share/cmake-3.23/Modules/FindBoost.cmake:594 (find_package)
examples/CMakeLists.txt:6 (find_package)
CMake Error at /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component):
boost_find_component Macro invoked with incorrect arguments for macro
named: boost_find_component
Call Stack (most recent call first):
/usr/share/cmake-3.23/Modules/FindBoost.cmake:594 (find_package)
examples/CMakeLists.txt:6 (find_package)
-- Found Boost: /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake (found version "1.71.0") missing components: log program_options system serialization date_time
-- Found Boost: /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake (found version "1.71.0") found components: unit_test_framework
-- benchmarks disabled (google.benchmark not found)
-- Configuring incomplete, errors occurred!
See also "/home/parisa/Projecs/Parisa_G/vanilla-rtb/Release/CMakeFiles/CMakeOutput.log".
See also "/home/parisa/Projecs/Parisa_G/vanilla-rtb/Release/CMakeFiles/CMakeError.log".
I tried installing on my Macbook, OSX 10.14.5 Mojave, boost = 1.70.0, tried GCC=8 and 9, but once I'm nearly done with build, there are errors as listed below. Also, there appears to be no bin directory?
In file included from /Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:19:
/Users/bill.zhu/vanilla-rtb/rtb/datacache/generic_bidder_cache_loader.hpp:58:37: error:
'retrieve' following the 'template' keyword does not refer to a template
return entity.template retrieve(t, std::get<Idx>(std::forwa...
^
/Users/bill.zhu/vanilla-rtb/rtb/datacache/generic_bidder_cache_loader.hpp:50:27: note:
in instantiation of function template specialization
'vanilla::GenericBidderCacheLoader<DomainEntity<vanilla::config::config<ico_bidder_config_data>,
mpclmi::ipc::Shared, boost::interprocess::allocator<char,
boost::interprocess::segment_manager<char,
boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
iset_index> > >,
ICOCampaignEntity<vanilla::config::config<ico_bidder_config_data>,
mpclmi::ipc::Shared, boost::interprocess::allocator<char,
boost::interprocess::segment_manager<char,
boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
iset_index> > >,
AdDataEntity<vanilla::config::config<ico_bidder_config_data>,
mpclmi::ipc::Shared, boost::interprocess::allocator<char,
boost::interprocess::segment_manager<char,
boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
iset_index> > >,
vanilla::CampaignCache<vanilla::config::config<ico_bidder_config_data>,
mpclmi::ipc::Shared, boost::interprocess::allocator<char,
boost::interprocess::segment_manager<char,
boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
iset_index> > > >::retrieve<Domain,
std::__1::tuple<std::__1::basic_string<char> >, 0>' requested here
return this->retrieve(t, std::make_tuple(std::forward<Keys>...
^
/Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:95:25: note:
in instantiation of function template specialization
'vanilla::GenericBidderCacheLoader<DomainEntity<vanilla::config::config<ico_bidder_config_data>,
mpclmi::ipc::Shared, boost::interprocess::allocator<char,
boost::interprocess::segment_manager<char,
boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
iset_index> > >,
ICOCampaignEntity<vanilla::config::config<ico_bidder_config_data>,
mpclmi::ipc::Shared, boost::interprocess::allocator<char,
boost::interprocess::segment_manager<char,
boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
iset_index> > >,
AdDataEntity<vanilla::config::config<ico_bidder_config_data>,
mpclmi::ipc::Shared, boost::interprocess::allocator<char,
boost::interprocess::segment_manager<char,
boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
iset_index> > >,
vanilla::CampaignCache<vanilla::config::config<ico_bidder_config_data>,
mpclmi::ipc::Shared, boost::interprocess::allocator<char,
boost::interprocess::segment_manager<char,
boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
iset_index> > > >::retrieve<Domain, const std::__1::basic_string<char> &>'
requested here
if(!cacheLoader.retrieve(domain,dom)) {
^
/Users/bill.zhu/vanilla-rtb/rtb/core/bidder.hpp:60:17: note: in instantiation of
function template specialization
'vanilla::Bidder<DSL::GenericDSL<std::__1::basic_string<char>,
DSL::rapid_mapper>, vanilla::ad_selector<vanilla::BudgetManager, Ad>
>::buildImpResponse<openrtb::Impression<std::__1::basic_string<char> >,
std::__1::basic_string<char>, (lambda at
/Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:93:30),
(lambda at
/Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:101:36),
(lambda at
/Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:110:36)>'
requested here
buildImpResponse(request, imp, std::get<Idx>(std::forwar...
^
/Users/bill.zhu/vanilla-rtb/rtb/core/bidder.hpp:66:27: note: in instantiation of
function template specialization
'vanilla::Bidder<DSL::GenericDSL<std::__1::basic_string<char>,
DSL::rapid_mapper>, vanilla::ad_selector<vanilla::BudgetManager, Ad>
>::bid<openrtb::BidRequest<std::__1::basic_string<char> >,
std::__1::tuple<std::__1::basic_string<char>, (lambda at
/Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:93:30),
(lambda at
/Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:101:36),
(lambda at
/Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:110:36)>,
0, 1, 2, 3>' requested here
return this->bid(vanilla_request, std::make_tuple(std::forw...
^
/Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:136:27: note:
in instantiation of function template specialization
'vanilla::Bidder<DSL::GenericDSL<std::__1::basic_string<char>,
DSL::rapid_mapper>, vanilla::ad_selector<vanilla::BudgetManager, Ad>
>::bid<openrtb::BidRequest<std::__1::basic_string<char> >, const
std::__1::basic_string<char> &, (lambda at
/Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:93:30)
&, (lambda at
/Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:101:36)
&, (lambda at
/Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:110:36)
&>' requested here
return bidder.bid(request,
^
/Users/bill.zhu/vanilla-rtb/examples/matchers/domain.hpp:152:14: note: declared
as a non-template here
bool retrieve(Domain &domain, const std::string &url) {
^
In file included from /Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:19:
/Users/bill.zhu/vanilla-rtb/rtb/datacache/generic_bidder_cache_loader.hpp:58:37: error:
'retrieve' following the 'template' keyword does not refer to a template
return entity.template retrieve(t, std::get<Idx>(std::forwa...
^
/Users/bill.zhu/vanilla-rtb/rtb/datacache/generic_bidder_cache_loader.hpp:50:27: note:
in instantiation of function template specialization
'vanilla::GenericBidderCacheLoader<ICOCampaignEntity<vanilla::config::config<ico_bidder_config_data>,
mpclmi::ipc::Shared, boost::interprocess::allocator<char,
boost::interprocess::segment_manager<char,
boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
iset_index> > >,
AdDataEntity<vanilla::config::config<ico_bidder_config_data>,
mpclmi::ipc::Shared, boost::interprocess::allocator<char,
boost::interprocess::segment_manager<char,
boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
iset_index> > >,
vanilla::CampaignCache<vanilla::config::config<ico_bidder_config_data>,
mpclmi::ipc::Shared, boost::interprocess::allocator<char,
boost::interprocess::segment_manager<char,
boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
iset_index> > > >::retrieve<std::__1::vector<ICOCampaign,
std::__1::allocator<ICOCampaign> >, std::__1::tuple<unsigned int>, 0>'
requested here
return this->retrieve(t, std::make_tuple(std::forward<Keys>...
^
/Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:103:26: note:
in instantiation of function template specialization
'vanilla::GenericBidderCacheLoader<DomainEntity<vanilla::config::config<ico_bidder_config_data>,
mpclmi::ipc::Shared, boost::interprocess::allocator<char,
boost::interprocess::segment_manager<char,
boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
iset_index> > >,
ICOCampaignEntity<vanilla::config::config<ico_bidder_config_data>,
mpclmi::ipc::Shared, boost::interprocess::allocator<char,
boost::interprocess::segment_manager<char,
boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
iset_index> > >,
AdDataEntity<vanilla::config::config<ico_bidder_config_data>,
mpclmi::ipc::Shared, boost::interprocess::allocator<char,
boost::interprocess::segment_manager<char,
boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
iset_index> > >,
vanilla::CampaignCache<vanilla::config::config<ico_bidder_config_data>,
mpclmi::ipc::Shared, boost::interprocess::allocator<char,
boost::interprocess::segment_manager<char,
boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
iset_index> > > >::retrieve<std::__1::vector<ICOCampaign,
std::__1::allocator<ICOCampaign> >, unsigned int &>' requested here
if (!cacheLoader.retrieve(ico_campains,*dom_id)) {
^
/Users/bill.zhu/vanilla-rtb/rtb/core/bidder.hpp:60:17: note: in instantiation of
function template specialization
'vanilla::Bidder<DSL::GenericDSL<std::__1::basic_string<char>,
DSL::rapid_mapper>, vanilla::ad_selector<vanilla::BudgetManager, Ad>
>::buildImpResponse<openrtb::Impression<std::__1::basic_string<char> >,
std::__1::basic_string<char>, (lambda at
/Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:93:30),
(lambda at
/Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:101:36),
(lambda at
/Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:110:36)>'
requested here
buildImpResponse(request, imp, std::get<Idx>(std::forwar...
^
/Users/bill.zhu/vanilla-rtb/rtb/core/bidder.hpp:66:27: note: in instantiation of
function template specialization
'vanilla::Bidder<DSL::GenericDSL<std::__1::basic_string<char>,
DSL::rapid_mapper>, vanilla::ad_selector<vanilla::BudgetManager, Ad>
>::bid<openrtb::BidRequest<std::__1::basic_string<char> >,
std::__1::tuple<std::__1::basic_string<char>, (lambda at
/Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:93:30),
(lambda at
/Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:101:36),
(lambda at
/Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:110:36)>,
0, 1, 2, 3>' requested here
return this->bid(vanilla_request, std::make_tuple(std::forw...
^
/Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:136:27: note:
in instantiation of function template specialization
'vanilla::Bidder<DSL::GenericDSL<std::__1::basic_string<char>,
DSL::rapid_mapper>, vanilla::ad_selector<vanilla::BudgetManager, Ad>
>::bid<openrtb::BidRequest<std::__1::basic_string<char> >, const
std::__1::basic_string<char> &, (lambda at
/Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:93:30)
&, (lambda at
/Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:101:36)
&, (lambda at
/Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:110:36)
&>' requested here
return bidder.bid(request,
^
/Users/bill.zhu/vanilla-rtb/examples/matchers/ico_campaign.hpp:167:14: note:
declared as a non-template here
bool retrieve(ICOCampaignCollection &ico_campaigns, uint32_t dom...
^
2 errors generated.
make[2]: *** [examples/bidder_experimental/CMakeFiles/ico_campaign_bidder_test.dir/ico_campaign_bidder_test.cpp.o] Error 1
make[1]: *** [examples/bidder_experimental/CMakeFiles/ico_campaign_bidder_test.dir/all] Error 2
I think the best way is to augment incoming BidRequest is by segregating a call to user_matching in exchange_handler.
Basically , having a separate handler:
currently connection is non-persistent runs 11K QPS on a 4 core CPU 2.2Ghz
Preliminary tests with updated CRUD showing major speed improvement tested by ab.sh --test
it ran at 49K QPS .
From the wiki page, it seems that there's a part called Banker, but I can't find it.
As a matter of fact, when we tried to use this project, we need to do some custom work maybe refactor rtb/core/openrtb.hpp
which means many part of it will be modified. So can you give me some advice about this?
By the way, is there a branch that supports Redis so that we can handle data with other component written by Python .etc.
Thanks.
With 98000 records in current ad_geo file, our current implementation of datacache for geo lookup is slow, we are looking to speed up fetching of all ads for particular impression.geo down to
10 microseconds .
We want all lookups to contribute to no more then 12-15% of all overheads.
Currently most data lookups are within 10 microseconds, however ag_geo is much higher.
http://stackoverflow.com/questions/27819004/what-is-the-status-of-abbreviated-functions-in-c
We should use -Wpedantic-errors or -Wpedantic flag like jsonv , this way it guarantees we only use standard features of C++14.
looks like "abreviated functions" are not standard yet, I think we have to support standard .
template function( T && t)
vs.
function( auto && t)
if(WIN32)
else(WIN32)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wpedantic-errors -Wall -Wextra")
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_DEBUG} -ggdb")
endif(WIN32)
We need to show in examples how bidder can interface campaign budget
Our current implementation of exchange_handler
boost.asio/jsmn-parser/any_mapper is behind netty/openrtb-core ( Java ) by a lot
On the 4 core machine netty/openrtb-core can get up to 25K QPSl while our exchange_handler is doing about 20K QPS.
we need some Banker generic interface classes in our core library which then can
be used for some sample code in
examples/banker where we should provide some implementation possibly utilizing our rtb/datacache generic structures.
features :
auction handler's timeout in exchange_handler is implemented as std::future::wait_for and so it spawns
new thread for every incoming message.
This made our R&D tests fail with TCP/IP redis client and therefore we have decided to try continue
calling auction on the same thread with io_service timer.
/home/reoxey/CLionProjects/vanilla-rtb/rtb/datacache/any_str_ops.hpp:42:41: error: no match for call to ‘(const std::less<void>) (ufw::any_str_op<boost::interprocess::allocator<char, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family>, boost::interprocess::iset_index> >, std::less<void> >::str_view_t, ufw::any_str_op<boost::interprocess::allocator<char, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family>, boost::interprocess::iset_index> >, std::less<void> >::str_view_t)’
gmake[2]: *** [examples/CMakeFiles/country_cache_test.dir/country_cache_test.cpp.o] Error 1
gmake[2]: Leaving directory `/home/reoxey/CLionProjects/vanilla-rtb/Release'
gmake[1]: *** [examples/CMakeFiles/country_cache_test.dir/all] Error 2
gmake[1]: Leaving directory `/home/reoxey/CLionProjects/vanilla-rtb/Release'
gmake: *** [all] Error 2
Hi, these days I get stuck with another problem, can I return custom http status code?
I read some parts of this project and CRUD
project, and I find this,
request_parser_.reset();
reply_.headers.resize(0);
reply_.status = reply::status_type::ok;
reply_.content="";
request_ = request();
do_read();
https://github.com/venediktov/CRUD/blob/master/service/persistent_connection.hpp#L123
https://github.com/venediktov/CRUD/blob/master/service/mime_types.cpp#L42
Here's the situation, I want to return 204 http status code, but can't find a way to do this trick. Is this possible? Similarly, some platforms need the Content-Type
to be set application/json
, so maybe it's a improvement to make the http response header more customizable.
Many thanks.
After running make -j4 install i'm getting following error after 63% -
ubuntu@ip-172-26-15-204:~/vanilla-rtb/Release$
make -j4 install
Scanning dependencies of target jsonv
Scanning dependencies of target crud_service
Scanning dependencies of target parsers
Scanning dependencies of target generator
[ 1%] Building C object parsers/CMakeFiles/parsers.dir/jsmn.c.o
[ 3%] Building CXX object CRUD/service/CMakeFiles/crud_service.dir/mime_types.c pp.o
[ 5%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/algorithm_compar e.cpp.o
[ 5%] Built target generator
Scanning dependencies of target generator_experimental
[ 5%] Built target generator_experimental
Scanning dependencies of target vanilla_tests
[ 6%] Building CXX object tests/CMakeFiles/vanilla_tests.dir/vanilla_all.cpp.o
[ 8%] Linking CXX shared library libparsers.so
[ 8%] Built target parsers
[ 10%] Building CXX object tests/CMakeFiles/vanilla_tests.dir/main.cpp.o
[ 11%] Building CXX object CRUD/service/CMakeFiles/crud_service.dir/reply.cpp.o
[ 13%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/algorithm_map.cp p.o
[ 15%] Building CXX object CRUD/service/CMakeFiles/crud_service.dir/request_hand ler.cpp.o
[ 16%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/algorithm_traver se.cpp.o
[ 18%] Linking CXX executable vanilla_tests
[ 18%] Built target vanilla_tests
[ 20%] Building CXX object CRUD/service/CMakeFiles/crud_service.dir/request_pars er.cpp.o
[ 21%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/array.cpp.o
[ 23%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/char_convert.cpp .o
[ 25%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/coerce.cpp.o
/home/ubuntu/vanilla-rtb/jsonv/src/jsonv/coerce.cpp: In function ‘jsonv::value j sonv::coerce_merge(jsonv::value, jsonv::value)’:
/home/ubuntu/vanilla-rtb/jsonv/src/jsonv/coerce.cpp:197:9: warning: this stateme nt may fall through [-Wimplicit-fallthrough=]
if (can_coerce(b, kind::integer))
^~
/home/ubuntu/vanilla-rtb/jsonv/src/jsonv/coerce.cpp:200:5: note: here
case kind::decimal:
^~~~
[ 26%] Linking CXX shared library libcrud_service.so
[ 26%] Built target crud_service
Scanning dependencies of target my_service
[ 28%] Building CXX object CRUD/CMakeFiles/my_service.dir/main.cpp.o
[ 30%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/demangle.cpp.o
[ 31%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/detail.cpp.o
[ 33%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/detail/token_pat terns.cpp.o
[ 35%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/encode.cpp.o
[ 36%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/functional.cpp.o
[ 38%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/object.cpp.o
[ 40%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/parse.cpp.o
[ 41%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/path.cpp.o
[ 43%] Linking CXX executable my_service
[ 45%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/serialization.cp p.o
[ 45%] Built target my_service
[ 46%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/serialization_bu ilder.cpp.o
[ 48%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/tokenizer.cpp.o
[ 50%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/util.cpp.o
[ 51%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/value.cpp.o
/home/ubuntu/vanilla-rtb/jsonv/src/jsonv/util.cpp: In function ‘jsonv::value jso nv::merge_explicit(const jsonv::merge_rules&, jsonv::path, jsonv::value, jsonv:: value)’:
/home/ubuntu/vanilla-rtb/jsonv/src/jsonv/util.cpp:118:13: warning: this statemen t may fall through [-Wimplicit-fallthrough=]
if (b.kind() == kind::integer)
^~
/home/ubuntu/vanilla-rtb/jsonv/src/jsonv/util.cpp:121:9: note: here
case kind::decimal:
^~~~
[ 53%] Linking CXX shared library libjsonv.so
[ 53%] Built target jsonv
Scanning dependencies of target vanilla_rtb
[ 55%] Building CXX object rtb/CMakeFiles/vanilla_rtb.dir/core/logger.cpp.o
[ 56%] Linking CXX shared library libvanilla_rtb.so
[ 56%] Built target vanilla_rtb
Scanning dependencies of target mock_bidder_test
Scanning dependencies of target country_cache_test
Scanning dependencies of target exchange_handler_test
Scanning dependencies of target messaging_client_test
[ 58%] Building CXX object examples/CMakeFiles/country_cache_test.dir/country_ca che_test.cpp.o
[ 60%] Building CXX object examples/CMakeFiles/mock_bidder_test.dir/mock_bidder_ test.cpp.o
[ 61%] Building CXX object examples/CMakeFiles/messaging_client_test.dir/messagi ng_client_test.cpp.o
[ 63%] Building CXX object examples/CMakeFiles/exchange_handler_test.dir/exchang e_handler_test.cpp.o
[ 65%] Linking CXX executable country_cache_test
../rtb/libvanilla_rtb.so.1.1.0: undefined reference to boost::log::v2_mt_posix: :aux::code_convert_impl(wchar_t const*, unsigned long, std::__cxx11::basic_strin g<char, std::char_traits<char>, std::allocator<char> >&, unsigned long, std::loc ale const&)' ../rtb/libvanilla_rtb.so.1.1.0: undefined reference to
void boost::log::v2_mt_p osix::aux::put_integer(boost::log::v2_mt_posix::aux::basic_ostringstreambu f<char, std::char_traits, std::allocator >&, unsigned int, unsigned int, char)'
../rtb/libvanilla_rtb.so.1.1.0: undefined reference to boost::log::v2_mt_posix: :sinks::text_file_backend::construct(boost::filesystem::path const&, std::_Ios_O penmode, unsigned long, boost::log::v2_mt_posix::aux::light_function<bool ()> co nst&, bool, bool)' collect2: error: ld returned 1 exit status examples/CMakeFiles/country_cache_test.dir/build.make:120: recipe for target 'ex amples/country_cache_test' failed make[2]: *** [examples/country_cache_test] Error 1 CMakeFiles/Makefile2:518: recipe for target 'examples/CMakeFiles/country_cache_t est.dir/all' failed make[1]: *** [examples/CMakeFiles/country_cache_test.dir/all] Error 2 make[1]: *** Waiting for unfinished jobs.... [ 66%] Linking CXX executable mock_bidder_test CMakeFiles/mock_bidder_test.dir/mock_bidder_test.cpp.o: In function
boost::arch ive::detail::common_iarchiveboost::archive::binary_iarchive::vload(boost::arch ive::class_name_type&)':
mock_bidder_test.cpp:(.text._ZN5boost7archive6detail15common_iarchiveINS0_15bina ry_iarchiveEE5vloadERNS0_15class_name_typeE[_ZN5boost7archive6detail15common_iar chiveINS0_15binary_iarchiveEE5vloadERNS0_15class_name_typeE]+0x1): undefined ref erence to boost::archive::basic_binary_iarchive<boost::archive::binary_iarchive >::load_override(boost::archive::class_name_type&)' CMakeFiles/mock_bidder_test.dir/mock_bidder_test.cpp.o: In function
void boost: :serialization::throw_exceptionboost::archive::archive_exception(boost::archiv e::archive_exception const&)':
mock_bidder_test.cpp:(.text.ZN5boost13serialization15throw_exceptionINS_7archiv e17archive_exceptionEEEvRKT[ZN5boost13serialization15throw_exceptionINS_7archi ve17archive_exceptionEEEvRKT]+0x1d): undefined reference to boost::archive::ar chive_exception::archive_exception(boost::archive::archive_exception const&)' ../rtb/libvanilla_rtb.so.1.1.0: undefined reference to
boost::log::v2_mt_posix: :aux::code_convert_impl(wchar_t const*, unsigned long, std::__cxx11::basic_strin g<char, std::char_traits, std::allocator >&, unsigned long, std::loc ale const&)'
../rtb/libvanilla_rtb.so.1.1.0: undefined reference to void boost::log::v2_mt_p osix::aux::put_integer<char>(boost::log::v2_mt_posix::aux::basic_ostringstreambu f<char, std::char_traits<char>, std::allocator<char> >&, unsigned int, unsigned int, char)' ../rtb/libvanilla_rtb.so.1.1.0: undefined reference to
boost::log::v2_mt_posix: :sinks::text_file_backend::construct(boost::filesystem::path const&, std::_Ios_O penmode, unsigned long, boost::log::v2_mt_posix::aux::light_function<bool ()> co nst&, bool, bool)'
collect2: error: ld returned 1 exit status
examples/CMakeFiles/mock_bidder_test.dir/build.make:120: recipe for target 'exam ples/mock_bidder_test' failed
make[2]: *** [examples/mock_bidder_test] Error 1
CMakeFiles/Makefile2:577: recipe for target 'examples/CMakeFiles/mock_bidder_tes t.dir/all' failed
make[1]: *** [examples/CMakeFiles/mock_bidder_test.dir/all] Error 2
[ 68%] Linking CXX executable messaging_client_test
CMakeFiles/messaging_client_test.dir/messaging_client_test.cpp.o: In function b oost::archive::detail::common_iarchive<boost::archive::binary_iarchive>::vload(b oost::archive::class_name_type&)': messaging_client_test.cpp:(.text._ZN5boost7archive6detail15common_iarchiveINS0_1 5binary_iarchiveEE5vloadERNS0_15class_name_typeE[_ZN5boost7archive6detail15commo n_iarchiveINS0_15binary_iarchiveEE5vloadERNS0_15class_name_typeE]+0x1): undefine d reference to
boost::archive::basic_binary_iarchive<boost::archive::binary_iar chive>::load_override(boost::archive::class_name_type&)'
CMakeFiles/messaging_client_test.dir/messaging_client_test.cpp.o: In function v oid boost::serialization::throw_exception<boost::archive::archive_exception>(boo st::archive::archive_exception const&)': messaging_client_test.cpp:(.text._ZN5boost13serialization15throw_exceptionINS_7a rchive17archive_exceptionEEEvRKT_[_ZN5boost13serialization15throw_exceptionINS_7 archive17archive_exceptionEEEvRKT_]+0x1d): undefined reference to
boost::archiv e::archive_exception::archive_exception(boost::archive::archive_exception const& )'
../rtb/libvanilla_rtb.so.1.1.0: undefined reference to boost::log::v2_mt_posix: :aux::code_convert_impl(wchar_t const*, unsigned long, std::__cxx11::basic_strin g<char, std::char_traits<char>, std::allocator<char> >&, unsigned long, std::loc ale const&)' ../rtb/libvanilla_rtb.so.1.1.0: undefined reference to
void boost::log::v2_mt_p osix::aux::put_integer(boost::log::v2_mt_posix::aux::basic_ostringstreambu f<char, std::char_traits, std::allocator >&, unsigned int, unsigned int, char)'
../rtb/libvanilla_rtb.so.1.1.0: undefined reference to boost::log::v2_mt_posix: :sinks::text_file_backend::construct(boost::filesystem::path const&, std::_Ios_O penmode, unsigned long, boost::log::v2_mt_posix::aux::light_function<bool ()> co nst&, bool, bool)' collect2: error: ld returned 1 exit status examples/CMakeFiles/messaging_client_test.dir/build.make:120: recipe for target 'examples/messaging_client_test' failed make[2]: *** [examples/messaging_client_test] Error 1 CMakeFiles/Makefile2:606: recipe for target 'examples/CMakeFiles/messaging_clien t_test.dir/all' failed make[1]: *** [examples/CMakeFiles/messaging_client_test.dir/all] Error 2 [ 70%] Linking CXX executable exchange_handler_test /usr/bin/ld: warning: libboost_system.so.1.67.0, needed by ../CRUD/service/libcr ud_service.so, may conflict with libboost_system.so.1.58.0 CMakeFiles/exchange_handler_test.dir/exchange_handler_test.cpp.o: In function
b oost::archive::detail::common_iarchiveboost::archive::binary_iarchive::vload(b oost::archive::class_name_type&)':
exchange_handler_test.cpp:(.text._ZN5boost7archive6detail15common_iarchiveINS0_1 5binary_iarchiveEE5vloadERNS0_15class_name_typeE[_ZN5boost7archive6detail15commo n_iarchiveINS0_15binary_iarchiveEE5vloadERNS0_15class_name_typeE]+0x1): undefine d reference to boost::archive::basic_binary_iarchive<boost::archive::binary_iar chive>::load_override(boost::archive::class_name_type&)' CMakeFiles/exchange_handler_test.dir/exchange_handler_test.cpp.o: In function
b oost::re_detail_106700::perl_matcher<char const*, std::allocator<boost::sub_matc h<char const*> >, boost::regex_traits<char, boost::cpp_regex_traits > >::u nwind_extra_block(bool)':
exchange_handler_test.cpp:(.text._ZN5boost16re_detail_10670012perl_matcherIPKcSa INS_9sub_matchIS3_EEENS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEE18unwind_ext ra_blockEb[ZN5boost16re_detail_10670012perl_matcherIPKcSaINS_9sub_matchIS3_EEEN S_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEE18unwind_extra_blockEb]+0x22): und efined reference to boost::re_detail_106700::put_mem_block(void*)' CMakeFiles/exchange_handler_test.dir/exchange_handler_test.cpp.o: In function
v oid boost::re_detail_106700::raise_error<boost::regex_traits_wrapper<boost::rege x_traits<char, boost::cpp_regex_traits > > >(boost::regex_traits_wrapper<b oost::regex_traits<char, boost::cpp_regex_traits > > const&, boost::regex constants::error_type)':
exchange_handler_test.cpp:(.text._ZN5boost16re_detail_10670011raise_errorINS_20r egex_traits_wrapperINS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEEEEEvRKT_NS_15 regex_constants10error_typeE[_ZN5boost16re_detail_10670011raise_errorINS_20regex _traits_wrapperINS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEEEEEvRKT_NS_15rege x_constants10error_typeE]+0x98): undefined reference to boost::re_detail_106700 ::get_default_error_string(boost::regex_constants::error_type)' exchange_handler_test.cpp:(.text._ZN5boost16re_detail_10670011raise_errorINS_20r egex_traits_wrapperINS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEEEEEvRKT_NS_15 regex_constants10error_typeE[_ZN5boost16re_detail_10670011raise_errorINS_20regex _traits_wrapperINS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEEEEEvRKT_NS_15rege x_constants10error_typeE]+0xc9): undefined reference to
boost::re_detail_106700 ::raise_runtime_error(std::runtime_error const&)'
CMakeFiles/exchange_handler_test.dir/exchange_handler_test.cpp.o: In function b oost::re_detail_106700::perl_matcher<char const*, std::allocator<boost::sub_matc h<char const*> >, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::e xtend_stack()': exchange_handler_test.cpp:(.text._ZN5boost16re_detail_10670012perl_matcherIPKcSa INS_9sub_matchIS3_EEENS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEE12extend_sta ckEv[_ZN5boost16re_detail_10670012perl_matcherIPKcSaINS_9sub_matchIS3_EEENS_12re gex_traitsIcNS_16cpp_regex_traitsIcEEEEE12extend_stackEv]+0x2e): undefined refer ence to
boost::re_detail_106700::get_mem_block()'
CMakeFiles/exchange_handler_test.dir/exchange_handler_test.cpp.o: In function c har const* boost::re_detail_106700::re_is_set_member<char const*, char, boost::r egex_traits<char, boost::cpp_regex_traits<char> >, unsigned int>(char const*, ch ar const*, boost::re_detail_106700::re_set_long<unsigned int> const*, boost::re_ detail_106700::regex_data<char, boost::regex_traits<char, boost::cpp_regex_trait s<char> > > const&, bool)': exchange_handler_test.cpp:(.text._ZN5boost16re_detail_10670016re_is_set_memberIP KccNS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEjEET_S8_S8_PKNS0_11re_set_longI T2_EERKNS0_10regex_dataIT0_T1_EEb[_ZN5boost16re_detail_10670016re_is_set_memberI PKccNS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEjEET_S8_S8_PKNS0_11re_set_long IT2_EERKNS0_10regex_dataIT0_T1_EEb]+0x360): undefined reference to
boost::re_de tail_106700::cpp_regex_traits_implementation::transform_primary[abi:cxx11] (char const*, char const*) const'
exchange_handler_test.cpp:(.text._ZN5boost16re_detail_10670016re_is_set_memberIP KccNS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEjEET_S8_S8_PKNS0_11re_set_longI T2_EERKNS0_10regex_dataIT0_T1_EEb[_ZN5boost16re_detail_10670016re_is_set_memberI PKccNS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEjEET_S8_S8_PKNS0_11re_set_long IT2_EERKNS0_10regex_dataIT0_T1_EEb]+0x474): undefined reference to boost::re_de tail_106700::cpp_regex_traits_implementation<char>::transform[abi:cxx11](char co nst*, char const*) const' CMakeFiles/exchange_handler_test.dir/exchange_handler_test.cpp.o: In function
v oid boost::serialization::throw_exceptionboost::archive::archive_exception(boo st::archive::archive_exception const&)':
exchange_handler_test.cpp:(.text.ZN5boost13serialization15throw_exceptionINS_7a rchive17archive_exceptionEEEvRKT[ZN5boost13serialization15throw_exceptionINS_7 archive17archive_exceptionEEEvRKT]+0x1d): undefined reference to boost::archiv e::archive_exception::archive_exception(boost::archive::archive_exception const& )' CMakeFiles/exchange_handler_test.dir/exchange_handler_test.cpp.o: In function
b oost::re_detail_106700::perl_matcher<char const*, std::allocator<boost::sub_matc h<char const*> >, boost::regex_traits<char, boost::cpp_regex_traits > >::m atch_imp()':
exchange_handler_test.cpp:(.text._ZN5boost16re_detail_10670012perl_matcherIPKcSa INS_9sub_matchIS3_EEENS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEE9match_impEv [ZN5boost16re_detail_10670012perl_matcherIPKcSaINS_9sub_matchIS3_EEENS_12regex traitsIcNS_16cpp_regex_traitsIcEEEEE9match_impEv]+0x1c): undefined reference to boost::re_detail_106700::get_mem_block()' exchange_handler_test.cpp:(.text._ZN5boost16re_detail_10670012perl_matcherIPKcSa INS_9sub_matchIS3_EEENS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEE9match_impEv [_ZN5boost16re_detail_10670012perl_matcherIPKcSaINS_9sub_matchIS3_EEENS_12regex_ traitsIcNS_16cpp_regex_traitsIcEEEEE9match_impEv]+0x216): undefined reference to
boost::re_detail_106700::verify_options(unsigned int, boost::regex_constants:: _match_flags)'
exchange_handler_test.cpp:(.text._ZN5boost16re_detail_10670012perl_matcherIPKcSa INS_9sub_matchIS3_EEENS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEE9match_impEv [ZN5boost16re_detail_10670012perl_matcherIPKcSaINS_9sub_matchIS3_EEENS_12regex traitsIcNS_16cpp_regex_traitsIcEEEEE9match_impEv]+0x2c0): undefined reference to boost::re_detail_106700::put_mem_block(void*)' exchange_handler_test.cpp:(.text._ZN5boost16re_detail_10670012perl_matcherIPKcSa INS_9sub_matchIS3_EEENS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEE9match_impEv [_ZN5boost16re_detail_10670012perl_matcherIPKcSaINS_9sub_matchIS3_EEENS_12regex_ traitsIcNS_16cpp_regex_traitsIcEEEEE9match_impEv]+0x3f7): undefined reference to
boost::re_detail_106700::put_mem_block(void*)'
CMakeFiles/exchange_handler_test.dir/exchange_handler_test.cpp.o: In function h ttp::server::persistent_connection<http::crud::crud_dispatcher<http::server::req uest, http::server::reply, boost::match_results<char const*, std::allocator<boos t::sub_match<char const*> > >, boost::basic_regex<char, boost::regex_traits<char , boost::cpp_regex_traits<char> > > > >::do_read()::{lambda(boost::system::error _code, unsigned long)#1}::operator()(boost::system::error_code, unsigned long) c onst': exchange_handler_test.cpp:(.text._ZZN4http6server21persistent_connectionINS_4cru d15crud_dispatcherINS0_7requestENS0_5replyEN5boost13match_resultsIPKcSaINS6_9sub _matchIS9_EEEEENS6_11basic_regexIcNS6_12regex_traitsIcNS6_16cpp_regex_traitsIcEE EEEEEEE7do_readEvENKUlNS6_6system10error_codeEmE_clESN_m[_ZZN4http6server21persi stent_connectionINS_4crud15crud_dispatcherINS0_7requestENS0_5replyEN5boost13matc h_resultsIPKcSaINS6_9sub_matchIS9_EEEEENS6_11basic_regexIcNS6_12regex_traitsIcNS 6_16cpp_regex_traitsIcEEEEEEEEE7do_readEvENKUlNS6_6system10error_codeEmE_clESN_m ]+0x56a): undefined reference to
boost::re_detail_106700::perl_matcher<char con st*, std::allocator<boost::sub_match<char const*> >, boost::regex_traits<char, b oost::cpp_regex_traits > >::construct_init(boost::basic_regex<char, boost: :regex_traits<char, boost::cpp_regex_traits > > const&, boost::regex_const ants::_match_flags)'
../rtb/libvanilla_rtb.so.1.1.0: undefined reference to boost::log::v2_mt_posix: :aux::code_convert_impl(wchar_t const*, unsigned long, std::__cxx11::basic_strin g<char, std::char_traits<char>, std::allocator<char> >&, unsigned long, std::loc ale const&)' ../rtb/libvanilla_rtb.so.1.1.0: undefined reference to
void boost::log::v2_mt_p osix::aux::put_integer(boost::log::v2_mt_posix::aux::basic_ostringstreambu f<char, std::char_traits, std::allocator >&, unsigned int, unsigned int, char)'
../rtb/libvanilla_rtb.so.1.1.0: undefined reference to boost::log::v2_mt_posix: :sinks::text_file_backend::construct(boost::filesystem::path const&, std::_Ios_O penmode, unsigned long, boost::log::v2_mt_posix::aux::light_function<bool ()> co nst&, bool, bool)' collect2: error: ld returned 1 exit status examples/CMakeFiles/exchange_handler_test.dir/build.make:118: recipe for target 'examples/exchange_handler_test' failed make[2]: *** [examples/exchange_handler_test] Error 1 CMakeFiles/Makefile2:548: recipe for target 'examples/CMakeFiles/exchange_handle r_test.dir/all' failed make[1]: *** [examples/CMakeFiles/exchange_handler_test.dir/all] Error 2 Makefile:160: recipe for target 'all' failed make: *** [all] Error 2
It looks like coding a new bidder not based on Geo model we provided in examples requires quite few lines of code.
the pattern we observed is the following - output from one matcher becomes input to the next in the chain.
Following needs to be implemented to simplify bidder assembly process:
template<typename Entity, typename ...Entities>
struct GenericBidderCacheLoader<Entity, Entities ...> : GenericBidderCacheLoader<Entities ...>
.auction_async([&](const BidRequest &request) {
thread_local vanilla::Bidder<DSLT, Selector> bidder(std::move(Selector()));
return bidder.bid(request,
std::make_tuple(
request.site.get().ref,
retrieve_referer_f,
retrieve_ico_campaign_f,
retrieve_campaign_ads_f
)
);
});
If following implemented then this code
std::make_tuple(
request.site.get().ref,
retrieve_referer_f,
retrieve_ico_campaign_f,
retrieve_campaign_ads_f
)
Can be easily generated based on data model provided by DSP.
Create more wiki pages showing in greater details how to assemble our stack .Correct diagram with boost serialization with dotted lines as it's an option to our stack and not required feature .
https://github.com/venediktov/vanilla-rtb/blob/master/rtb/exchange/exchange_handler.hpp#L129
We have a new feature e.g.
when HTTP 204 is needed we have provided with exchange_handler.if_response() handler which
should be used and custom response should be sent .
The custom response is sent OK however exchange_handler redundantly converts standard response to
wire format causing lag in performance.
Proposing change remove line L129 and add it below
if (auction_response && timer.expires_from_now().total_milliseconds() > 0) {
if(if_response_handler) {
auto custom_reply = if_response_handler(auction_response);
if ( custom_reply ) {
custom_reply(r);
} else {
wire_response = parser.create_response(auction_response);
r << to_string(*wire_response) << http::server::reply::flush("json");
}
} else {
wire_response = parser.create_response(auction_response);
r << to_string(*wire_response) << http::server::reply::flush("json");
}
} else {
r << http::server::reply::flush("json");
}
And looking at it again maybe time to re-factor it as this code is too verbose .
we need a sample loader at least for caches we already have in place
examples/loader - proposed directory structure where loader supporting code will reside
The JavaScript is missing logic explained here:
https://stackoverflow.com/questions/44422561/how-to-select-drop-down-menu-in-angularjs-based-on-non-zero-value-of-json-column
We also need to figure out why javascript not very responsive , is it related to hosting it on CDN ?
our build process automatically produces test data for ad matching , sometimes it produces ads that
don't match impression in BID_REQUEST_BANNER.json
Is there a built-in payout calculator? Something that provides the total revenue per ad?
We have an old Angular-2 CDN based campaign budget UI management piece of code with HTML
We'd like to have :
For more questions either use our gitter chat channel or IM me directly via email or leave your email by replying to this issue on GitHub.
reference #40
I tried building the project in cmake 3.2.2 but gives errors like
it will better to provide the least version of cmake supported, for those who are new to this repo.
Regards.
Write user matching interfaces based on key /value storage - no implementation however possible easy integration with Redis.
Hello, while i'm compiling this project, i have encountered following errors:
vanilla-rtb/rtb/messaging/serialization.hpp:222:28: error: invalid initialization of non-const reference of type ‘long unsigned int&’ from an rvalue of type ‘jsonv::detail::string_view::size_type {aka long unsigned int}’
ar & value.size();
^
In file included from /usr/include/boost/archive/detail/common_oarchive.hpp:22:0,
from /usr/include/boost/archive/basic_binary_oarchive.hpp:34,
from /usr/include/boost/archive/binary_oarchive_impl.hpp:23,
from /usr/include/boost/archive/binary_oarchive.hpp:21,
from /home/pantianxiang/vanilla-rtb/rtb/messaging/communicator.hpp:31,
from /home/pantianxiang/vanilla-rtb/examples/exchange_handler_test.cpp:13:
/usr/include/boost/archive/detail/interface_oarchive.hpp:69:15: note: initializing argument 1 of ‘Archive& boost::archive::detail::interface_oarchive<Archive>::operator&(T&) [with T = long unsigned int; Archive = boost::archive::binary_oarchive]’
Archive & operator&(T & t){
^
In file included from /usr/include/c++/5/algorithm:62:0,
from /usr/include/boost/smart_ptr/shared_ptr.hpp:37,
from /usr/include/boost/log/core/core.hpp:19,
from /usr/include/boost/log/sources/basic_logger.hpp:37,
from /usr/include/boost/log/sources/severity_logger.hpp:23,
from /usr/include/boost/log/trivial.hpp:22,
from /home/pantianxiang/vanilla-rtb/examples/exchange_handler_test.cpp:2:
/usr/include/c++/5/bits/stl_algo.h:3761:5: error: ‘_Funct std::for_each(_IIter, _IIter, _Funct) [with _IIter = const char*; _Funct = boost::serialization::save(Archive&, const string_view&, unsigned int) [with Archive = boost::archive::binary_oarchive; jsonv::string_view = jsonv::detail::string_view]::<lambda(char)>]’, declared using local type ‘boost::serialization::save(Archive&, const string_view&, unsigned int) [with Archive = boost::archive::binary_oarchive; jsonv::string_view = jsonv::detail::string_view]::<lambda(char)>’, is used but never defined [-fpermissive]
for_each(_InputIterator __first, _InputIterator __last, _Function __f)
^
examples/CMakeFiles/exchange_handler_test.dir/build.make:62: recipe for target 'examples/CMakeFiles/exchange_handler_test.dir/exchange_handler_test.cpp.o' failed
Seem like the initialization od value.size() is invalid, yet i'm not aware how to initialize it correctly. Looking forward for your help.
We should try creating bidder per targeting or per other criteria to see if we
can segregate bidders and still process single request by multiple .
connecting all of those bidder-slaves to master endpoint ( exchange_handler ) via communicator
is the idea.
High level idea:
exchange_handler -> broadcast -> collect ( on master host )
bidders -> process -> reply ( on multiple or single slave machine )
within bidder-> process targeting rules should be applied and BidResponse should be sent back to
master ( exchange_handler ) , which will collect responses from all bidders.
Simple selection algo should be applied choosing highest bid.
Performance should be recorded with perf_timer and documented .
We don't need a separate loader for ico targeting , just add a REST handler and ico caches to
examples/loader/cache_loader_test.cpp and separate map
dispatcher.crud_match(boost::regex("/ico_cache_loader/(\\w*)"))
.put([&](http::server::reply & r, const http::crud::crud_match<boost::cmatch> & match) {
LOG(info) << "received cache update event url=" << match[0];
try {
ico_caches[match[1]]();
} catch (std::exception const& e) {
LOG(error) << e.what();
}
});
Also add map for ico targeting model
std::map<std::string, std::function<void()>> ico_caches = {
{"domain" , [&domain_cache](){geo_ad_cache.load();}},
{"ico_campaign", [&ico_campaign_cache](){co_campaign_cache.load();} },
{"ad" , [&ad_cache](){ad_cache.load();} },
{"" , [&ico_bidder_caches] (){ico_bidder_caches.load();} }
};
Looks like last stage when we pick up campaigns from the cache the complexity is O(N LogN) , for every matched campaign id N we use LogN complexity lookup in the last cached table ~ O(N LogN) .
Matching 20-30 campaign per request is within our benchmark goals however going above this number degrades performance.
on running run.sh
cd Release/examples/bin
./http_bidder_test --config etc/config.cfg
./cache_loader_test --config etc/config.cfg
./exchange_handler_test --config etc/config.cfg
./mock_bidder_test --config etc/config.cfg
./multi_bidder --config etc/config.cfg
./multi_exchange_handler --config etc/config.cfg
./notification_service_test --config etc/config.cfg
./slavebanker_service_test --config etc/config.cfg
./campaign_manager_test --config etc/config.cfg
i get this output in terminal..
reoxey@reoxey-desktop:~/CLionProjects/vanilla-rtb$ sh ./run.sh
[2017-06-17 12:42:39.832061] [0x00007f3f5e222900] [debug] bidder.ads_ipc_name vanilla-ads-ipc
bidder.ads_source data/ads
bidder.campaign_data_ipc_name vanilla-campaign-data-ipc
bidder.campaign_data_source data/campaign_data
bidder.concurrency 0
bidder.geo_ad_ipc_name vanilla-geo-ad-ipc
bidder.geo_ad_source data/ad_geo
bidder.geo_campaign_ipc_name vanilla-geo-campaign-ipc
bidder.geo_campaign_source data/geo_campaign
bidder.geo_ipc_name vanilla-geo-ipc
bidder.geo_source data/geo
bidder.host 0.0.0.0
bidder.key_value_host 0.0.0.0
bidder.key_value_port 0
bidder.log /tmp/bidder_log
bidder.port 9081
bidder.root .
bidder.timeout 50
config etc/config.cfg
12:42:39.924693: <info>
ad load : elapsed_sec=0|elapsed_ms=32|elapsed_mu=32341|elapsed_ns=32341990|
geo_data load : elapsed_sec=0|elapsed_ms=9|elapsed_mu=9912|elapsed_ns=9912520|
geo_campaign load : elapsed_sec=0|elapsed_ms=48|elapsed_mu=48943|elapsed_ns=48943310|
selector load : elapsed_sec=0|elapsed_ms=91|elapsed_mu=91217|elapsed_ns=91217176|
terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::system::system_error> >'
what(): resolve: Host not found (authoritative)
Aborted (core dumped)
[2017-06-17 12:42:41.621299] [0x00007f298fae9780] [debug] bidder.campaign_data_ipc_name vanilla-campaign-data-ipc
bidder.campaign_data_source data/campaign_data
bidder.geo_campaign_ipc_name vanilla-geo-campaign-ipc
bidder.geo_campaign_source data/geo_campaign
cache-loader.host 0.0.0.0
cache-loader.log /tmp/vanilla_cache_loader_log
cache-loader.port 10081
cache-loader.root .
config etc/config.cfg
datacache.ads_ipc_name vanilla-ads-ipc
datacache.ads_source bidder/data/ads
datacache.geo_ad_ipc_name vanilla-geo-ad-ipc
datacache.geo_ad_source bidder/data/ad_geo
datacache.geo_ipc_name vanilla-geo-ipc
datacache.geo_source bidder/data/geo
terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::system::system_error> >'
what(): resolve: Host not found (authoritative)
Aborted (core dumped)
[2017-06-17 12:42:41.733372] [0x00007f7cb43da800] [debug] config etc/config.cfg
exchange.host 0.0.0.0
exchange.log /tmp/openrtb_handler_log
exchange.port 8081
exchange.root .
exchange.v1.timeout 100
exchange.v2.timeout 80
mock-bidder.num_of_bidders 5
mock-bidder.port 5000
terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::system::system_error> >'
what(): resolve: Host not found (authoritative)
Aborted (core dumped)
12:42:41.845543: <info> Starting mock bidder pid=9545
12:42:41.846075: <info> Starting mock bidder pid=9546
12:42:41.846144: <info> Starting mock bidder pid=9547
12:42:41.846563: <info> Starting mock bidder pid=9548
12:42:41.846910: <info> Starting mock bidder pid=9549
when i try to open localhost:11081 or localhost:11081/campaign, this error shows up.
This site can’t be reached
localhost refused to connect.
what am i doing wrong?
It expanded too wide on the screen we need to make it smaller , ID should be no more then 4 digits
to hold up to 9999 campaigns by default and should be able to expand automatically if the number
of campaigns grows beyond 9999.
It's probably needs to be revised as we moved to structure below that guarantee only CPM, CPC or CPA as a single metric and python need to generate either a different structure or the code
loading from flat text files needs to be fixed .
Please see below the 2 campaigns generated with no values :
{
"budget": 2020807235,
"id": 1,
"metric": {
"id": 0,
"value": 0
},
"spent": 1309432428
},
{
"budget": 6362255854,
"id": 2,
"metric": {
"id": 0,
"value": 0
},
"spent": 1709407513
},
Need some way to test on different compilers and different OS to build the stack , looking into
integrating with Travis CI. Seems not as trivial as I thought , but hope to finish its script in few days
Even though we see people attempt to build our stack , sometimes it's required to update compilers and dependency libraries. To eliminate this leg work and make our stack build-ready for open source community our team decided to provide an isolated container based solution - container images .
The images are locked down Linux containers with file system containing complete vanilla stack and all
its build/run dependencies .
On our plate:
When we do lookup in our caches we convert process memory ( e.g. std::string ) into ipc memory with
with entity_cache.create_ipc_key.
This can be avoided by utilizing transient std::less
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.