venediktov / vanilla-rtb Goto Github PK

Create more wiki pages showing in greater details how to assemble our stack .Correct diagram with boost serialization with dotted lines as it's an option to our stack and not required feature .

Looks like last stage when we pick up campaigns from the cache the complexity is O(N LogN) , for every matched campaign id N we use LogN complexity lookup in the last cached table ~ O(N LogN) .
Matching 20-30 campaign per request is within our benchmark goals however going above this number degrades performance.

• Use relational cache entries to solve the issue , we should avoid last stage campaign lookup , rather have pointers saved in the preceding matcher cache and use those pointers to retrieve campaigns directly instead of performing O(LogN) lookup.

It's probably needs to be revised as we moved to structure below that guarantee only CPM, CPC or CPA as a single metric and python need to generate either a different structure or the code
loading from flat text files needs to be fixed .

Please see below the 2 campaigns generated with no values :

{
    "budget": 2020807235,
    "id": 1,
    "metric": {
        "id": 0,
        "value": 0
    },
    "spent": 1309432428
},
{
    "budget": 6362255854,
    "id": 2,
    "metric": {
        "id": 0,
        "value": 0
    },
    "spent": 1709407513
},

It looks like coding a new bidder not based on Geo model we provided in examples requires quite few lines of code.
the pattern we observed is the following - output from one matcher becomes input to the next in the chain.

Following needs to be implemented to simplify bidder assembly process:

• Create chained_slector to delegate output from previous matcher to the next in the chain
• Create generic version of bidder cache loader "GenericBidderCacheLoader" capable of loading all matchers data
• Same GenericBidderCacheLoader should be utilized to retrieve matched data from caches

template<typename Entity, typename ...Entities>
    struct GenericBidderCacheLoader<Entity, Entities ...> : GenericBidderCacheLoader<Entities ...>

• Simplify the use of bidder as following

.auction_async([&](const BidRequest &request) {
            thread_local vanilla::Bidder<DSLT, Selector> bidder(std::move(Selector()));
            return bidder.bid(request,
                              std::make_tuple(
                                  request.site.get().ref,
                                  retrieve_referer_f,
                                  retrieve_ico_campaign_f,
                                  retrieve_campaign_ads_f
                              )
            );
        });

If following implemented then this code

std::make_tuple(
      request.site.get().ref,
      retrieve_referer_f,
      retrieve_ico_campaign_f,
      retrieve_campaign_ads_f
)

Can be easily generated based on data model provided by DSP.

I think the best way is to augment incoming BidRequest is by segregating a call to user_matching in exchange_handler.
Basically , having a separate handler:

• user_matching
• auction ( e.g bidding interface )

The JavaScript is missing logic explained here:
https://stackoverflow.com/questions/44422561/how-to-select-drop-down-menu-in-angularjs-based-on-non-zero-value-of-json-column

We also need to figure out why javascript not very responsive , is it related to hosting it on CDN ?

• vanilla-rtb/rtb/supply ( all classes related to SSP go there )
• vanilla-rtb/examples/supply
  • ssp_test.cpp

Hi
I want to build your project and follow your instructions.
I use ubuntu 20.04,boost Version: 1.71.0 and cmake version 3.23.2.

as I run cmake -DCMAKE_BUILD_TYPE=Debug .. -G "Unix Makefiles" :

-- The CXX compiler identification is GNU 9.4.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Boost: /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake (found suitable version "1.71.0", minimum required is "1.67.0")  
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
CMAKE_INSTALL_PREFIX /home/parisa/Projecs/Parisa_G/vanilla-rtb/Release/examples
CMAKE_CURRENT_SOURCE_DIR /home/parisa/Projecs/Parisa_G/vanilla-rtb
CMAKE_CURRENT_BIN_DIR /home/parisa/Projecs/Parisa_G/vanilla-rtb/Release
CMake Deprecation Warning at jsonv/CMakeLists.txt:13 (cmake_minimum_required):
  Compatibility with CMake < 2.8.12 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


-- The C compiler identification is GNU 9.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- JSONV version: 1.2.0
-- Found Boost: /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake (found version "1.71.0")  
-- RUNTIME_INSTALL_DIR == /home/parisa/Projecs/Parisa_G/vanilla-rtb/Release/examples/bin
-- LIBRARY_INSTALL_DIR == /home/parisa/Projecs/Parisa_G/vanilla-rtb/Release/examples/lib64
-- INCLUDE_INSTALL_DIR == /home/parisa/Projecs/Parisa_G/vanilla-rtb/Release/examples/include
-- CONFIG_INSTALL_DIR == /home/parisa/Projecs/Parisa_G/vanilla-rtb/Release/examples/lib64/cmake/jsonv
CMake Error at /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component):
  boost_find_component Macro invoked with incorrect arguments for macro
  named: boost_find_component
Call Stack (most recent call first):
  /usr/share/cmake-3.23/Modules/FindBoost.cmake:594 (find_package)
  CRUD/service/CMakeLists.txt:10 (find_package)


CMake Error at /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component):
  boost_find_component Macro invoked with incorrect arguments for macro
  named: boost_find_component
Call Stack (most recent call first):
  /usr/share/cmake-3.23/Modules/FindBoost.cmake:594 (find_package)
  CRUD/service/CMakeLists.txt:10 (find_package)


-- Found Boost: /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake (found version "1.71.0")  missing components: system regex
-- Found Boost: /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake (found version "1.71.0") found components: log system serialization program_options 
CMake Error at /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component):
  boost_find_component Macro invoked with incorrect arguments for macro
  named: boost_find_component
Call Stack (most recent call first):
  /usr/share/cmake-3.23/Modules/FindBoost.cmake:594 (find_package)
  rtb/CMakeLists.txt:21 (find_package)


CMake Error at /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component):
  boost_find_component Macro invoked with incorrect arguments for macro
  named: boost_find_component
Call Stack (most recent call first):
  /usr/share/cmake-3.23/Modules/FindBoost.cmake:594 (find_package)
  rtb/CMakeLists.txt:21 (find_package)


CMake Error at /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component):
  boost_find_component Macro invoked with incorrect arguments for macro
  named: boost_find_component
Call Stack (most recent call first):
  /usr/share/cmake-3.23/Modules/FindBoost.cmake:594 (find_package)
  rtb/CMakeLists.txt:21 (find_package)


-- Found Boost: /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake (found version "1.71.0")  missing components: log serialization system
CMake Error at /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component):
  boost_find_component Macro invoked with incorrect arguments for macro
  named: boost_find_component
Call Stack (most recent call first):
  /usr/share/cmake-3.23/Modules/FindBoost.cmake:594 (find_package)
  examples/CMakeLists.txt:6 (find_package)


CMake Error at /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component):
  boost_find_component Macro invoked with incorrect arguments for macro
  named: boost_find_component
Call Stack (most recent call first):
  /usr/share/cmake-3.23/Modules/FindBoost.cmake:594 (find_package)
  examples/CMakeLists.txt:6 (find_package)


CMake Error at /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component):
  boost_find_component Macro invoked with incorrect arguments for macro
  named: boost_find_component
Call Stack (most recent call first):
  /usr/share/cmake-3.23/Modules/FindBoost.cmake:594 (find_package)
  examples/CMakeLists.txt:6 (find_package)


CMake Error at /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component):
  boost_find_component Macro invoked with incorrect arguments for macro
  named: boost_find_component
Call Stack (most recent call first):
  /usr/share/cmake-3.23/Modules/FindBoost.cmake:594 (find_package)
  examples/CMakeLists.txt:6 (find_package)


CMake Error at /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component):
  boost_find_component Macro invoked with incorrect arguments for macro
  named: boost_find_component
Call Stack (most recent call first):
  /usr/share/cmake-3.23/Modules/FindBoost.cmake:594 (find_package)
  examples/CMakeLists.txt:6 (find_package)


-- Found Boost: /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake (found version "1.71.0")  missing components: log program_options system serialization date_time
-- Found Boost: /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake (found version "1.71.0") found components: unit_test_framework 
-- benchmarks disabled (google.benchmark not found)
-- Configuring incomplete, errors occurred!
See also "/home/parisa/Projecs/Parisa_G/vanilla-rtb/Release/CMakeFiles/CMakeOutput.log".
See also "/home/parisa/Projecs/Parisa_G/vanilla-rtb/Release/CMakeFiles/CMakeError.log".

currently connection is non-persistent runs 11K QPS on a 4 core CPU 2.2Ghz
Preliminary tests with updated CRUD showing major speed improvement tested by ab.sh --test
it ran at 49K QPS .

• add persistent_connection.hpp to CRUD and use it in exchange_server<Handler,Connection>

It expanded too wide on the screen we need to make it smaller , ID should be no more then 4 digits
to hold up to 9999 campaigns by default and should be able to expand automatically if the number
of campaigns grows beyond 9999.

We don't need a separate loader for ico targeting , just add a REST handler and ico caches to
examples/loader/cache_loader_test.cpp and separate map

dispatcher.crud_match(boost::regex("/ico_cache_loader/(\\w*)"))
              .put([&](http::server::reply & r, const http::crud::crud_match<boost::cmatch> & match) {
              LOG(info) << "received cache update event url=" << match[0];
              try {
                  ico_caches[match[1]]();
              } catch (std::exception const& e) {
                  LOG(error) << e.what();
              }
    });

Also add map for ico targeting model

std::map<std::string, std::function<void()>> ico_caches = {
        {"domain" , [&domain_cache](){geo_ad_cache.load();}},
        {"ico_campaign", [&ico_campaign_cache](){co_campaign_cache.load();}   },
        {"ad"     , [&ad_cache](){ad_cache.load();}    },
        {""       , [&ico_bidder_caches] (){ico_bidder_caches.load();}    }
    };

• instructions how to build static boost libraries used by our cmake when building vanilla-RTB
• upgrade to "Visual Studio 15 2017" and rebuild stack .

I tried installing on my Macbook, OSX 10.14.5 Mojave, boost = 1.70.0, tried GCC=8 and 9, but once I'm nearly done with build, there are errors as listed below. Also, there appears to be no bin directory?

In file included from /Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:19:
/Users/bill.zhu/vanilla-rtb/rtb/datacache/generic_bidder_cache_loader.hpp:58:37: error: 
      'retrieve' following the 'template' keyword does not refer to a template
            return  entity.template retrieve(t, std::get<Idx>(std::forwa...
                                    ^
/Users/bill.zhu/vanilla-rtb/rtb/datacache/generic_bidder_cache_loader.hpp:50:27: note: 
      in instantiation of function template specialization
      'vanilla::GenericBidderCacheLoader<DomainEntity<vanilla::config::config<ico_bidder_config_data>,
      mpclmi::ipc::Shared, boost::interprocess::allocator<char,
      boost::interprocess::segment_manager<char,
      boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
      boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
      iset_index> > >,
      ICOCampaignEntity<vanilla::config::config<ico_bidder_config_data>,
      mpclmi::ipc::Shared, boost::interprocess::allocator<char,
      boost::interprocess::segment_manager<char,
      boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
      boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
      iset_index> > >,
      AdDataEntity<vanilla::config::config<ico_bidder_config_data>,
      mpclmi::ipc::Shared, boost::interprocess::allocator<char,
      boost::interprocess::segment_manager<char,
      boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
      boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
      iset_index> > >,
      vanilla::CampaignCache<vanilla::config::config<ico_bidder_config_data>,
      mpclmi::ipc::Shared, boost::interprocess::allocator<char,
      boost::interprocess::segment_manager<char,
      boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
      boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
      iset_index> > > >::retrieve<Domain,
      std::__1::tuple<std::__1::basic_string<char> >, 0>' requested here
            return  this->retrieve(t, std::make_tuple(std::forward<Keys>...
                          ^
/Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:95:25: note: 
      in instantiation of function template specialization
      'vanilla::GenericBidderCacheLoader<DomainEntity<vanilla::config::config<ico_bidder_config_data>,
      mpclmi::ipc::Shared, boost::interprocess::allocator<char,
      boost::interprocess::segment_manager<char,
      boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
      boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
      iset_index> > >,
      ICOCampaignEntity<vanilla::config::config<ico_bidder_config_data>,
      mpclmi::ipc::Shared, boost::interprocess::allocator<char,
      boost::interprocess::segment_manager<char,
      boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
      boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
      iset_index> > >,
      AdDataEntity<vanilla::config::config<ico_bidder_config_data>,
      mpclmi::ipc::Shared, boost::interprocess::allocator<char,
      boost::interprocess::segment_manager<char,
      boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
      boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
      iset_index> > >,
      vanilla::CampaignCache<vanilla::config::config<ico_bidder_config_data>,
      mpclmi::ipc::Shared, boost::interprocess::allocator<char,
      boost::interprocess::segment_manager<char,
      boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
      boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
      iset_index> > > >::retrieve<Domain, const std::__1::basic_string<char> &>'
      requested here
        if(!cacheLoader.retrieve(domain,dom)) {
                        ^
/Users/bill.zhu/vanilla-rtb/rtb/core/bidder.hpp:60:17: note: in instantiation of
      function template specialization
      'vanilla::Bidder<DSL::GenericDSL<std::__1::basic_string<char>,
      DSL::rapid_mapper>, vanilla::ad_selector<vanilla::BudgetManager, Ad>
      >::buildImpResponse<openrtb::Impression<std::__1::basic_string<char> >,
      std::__1::basic_string<char>, (lambda at
      /Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:93:30),
      (lambda at
      /Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:101:36),
      (lambda at
      /Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:110:36)>'
      requested here
                buildImpResponse(request, imp, std::get<Idx>(std::forwar...
                ^
/Users/bill.zhu/vanilla-rtb/rtb/core/bidder.hpp:66:27: note: in instantiation of
      function template specialization
      'vanilla::Bidder<DSL::GenericDSL<std::__1::basic_string<char>,
      DSL::rapid_mapper>, vanilla::ad_selector<vanilla::BudgetManager, Ad>
      >::bid<openrtb::BidRequest<std::__1::basic_string<char> >,
      std::__1::tuple<std::__1::basic_string<char>, (lambda at
      /Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:93:30),
      (lambda at
      /Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:101:36),
      (lambda at
      /Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:110:36)>,
      0, 1, 2, 3>' requested here
            return  this->bid(vanilla_request, std::make_tuple(std::forw...
                          ^
/Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:136:27: note: 
      in instantiation of function template specialization
      'vanilla::Bidder<DSL::GenericDSL<std::__1::basic_string<char>,
      DSL::rapid_mapper>, vanilla::ad_selector<vanilla::BudgetManager, Ad>
      >::bid<openrtb::BidRequest<std::__1::basic_string<char> >, const
      std::__1::basic_string<char> &, (lambda at
      /Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:93:30)
      &, (lambda at
      /Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:101:36)
      &, (lambda at
      /Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:110:36)
      &>' requested here
            return bidder.bid(request,
                          ^
/Users/bill.zhu/vanilla-rtb/examples/matchers/domain.hpp:152:14: note: declared
      as a non-template here
        bool retrieve(Domain &domain, const std::string &url) {
             ^
In file included from /Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:19:
/Users/bill.zhu/vanilla-rtb/rtb/datacache/generic_bidder_cache_loader.hpp:58:37: error: 
      'retrieve' following the 'template' keyword does not refer to a template
            return  entity.template retrieve(t, std::get<Idx>(std::forwa...
                                    ^
/Users/bill.zhu/vanilla-rtb/rtb/datacache/generic_bidder_cache_loader.hpp:50:27: note: 
      in instantiation of function template specialization
      'vanilla::GenericBidderCacheLoader<ICOCampaignEntity<vanilla::config::config<ico_bidder_config_data>,
      mpclmi::ipc::Shared, boost::interprocess::allocator<char,
      boost::interprocess::segment_manager<char,
      boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
      boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
      iset_index> > >,
      AdDataEntity<vanilla::config::config<ico_bidder_config_data>,
      mpclmi::ipc::Shared, boost::interprocess::allocator<char,
      boost::interprocess::segment_manager<char,
      boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
      boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
      iset_index> > >,
      vanilla::CampaignCache<vanilla::config::config<ico_bidder_config_data>,
      mpclmi::ipc::Shared, boost::interprocess::allocator<char,
      boost::interprocess::segment_manager<char,
      boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
      boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
      iset_index> > > >::retrieve<std::__1::vector<ICOCampaign,
      std::__1::allocator<ICOCampaign> >, std::__1::tuple<unsigned int>, 0>'
      requested here
            return  this->retrieve(t, std::make_tuple(std::forward<Keys>...
                          ^
/Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:103:26: note: 
      in instantiation of function template specialization
      'vanilla::GenericBidderCacheLoader<DomainEntity<vanilla::config::config<ico_bidder_config_data>,
      mpclmi::ipc::Shared, boost::interprocess::allocator<char,
      boost::interprocess::segment_manager<char,
      boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
      boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
      iset_index> > >,
      ICOCampaignEntity<vanilla::config::config<ico_bidder_config_data>,
      mpclmi::ipc::Shared, boost::interprocess::allocator<char,
      boost::interprocess::segment_manager<char,
      boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
      boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
      iset_index> > >,
      AdDataEntity<vanilla::config::config<ico_bidder_config_data>,
      mpclmi::ipc::Shared, boost::interprocess::allocator<char,
      boost::interprocess::segment_manager<char,
      boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
      boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
      iset_index> > >,
      vanilla::CampaignCache<vanilla::config::config<ico_bidder_config_data>,
      mpclmi::ipc::Shared, boost::interprocess::allocator<char,
      boost::interprocess::segment_manager<char,
      boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,
      boost::interprocess::offset_ptr<void, long, unsigned long, 0>, 0>,
      iset_index> > > >::retrieve<std::__1::vector<ICOCampaign,
      std::__1::allocator<ICOCampaign> >, unsigned int &>' requested here
        if (!cacheLoader.retrieve(ico_campains,*dom_id)) {
                         ^
/Users/bill.zhu/vanilla-rtb/rtb/core/bidder.hpp:60:17: note: in instantiation of
      function template specialization
      'vanilla::Bidder<DSL::GenericDSL<std::__1::basic_string<char>,
      DSL::rapid_mapper>, vanilla::ad_selector<vanilla::BudgetManager, Ad>
      >::buildImpResponse<openrtb::Impression<std::__1::basic_string<char> >,
      std::__1::basic_string<char>, (lambda at
      /Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:93:30),
      (lambda at
      /Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:101:36),
      (lambda at
      /Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:110:36)>'
      requested here
                buildImpResponse(request, imp, std::get<Idx>(std::forwar...
                ^
/Users/bill.zhu/vanilla-rtb/rtb/core/bidder.hpp:66:27: note: in instantiation of
      function template specialization
      'vanilla::Bidder<DSL::GenericDSL<std::__1::basic_string<char>,
      DSL::rapid_mapper>, vanilla::ad_selector<vanilla::BudgetManager, Ad>
      >::bid<openrtb::BidRequest<std::__1::basic_string<char> >,
      std::__1::tuple<std::__1::basic_string<char>, (lambda at
      /Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:93:30),
      (lambda at
      /Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:101:36),
      (lambda at
      /Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:110:36)>,
      0, 1, 2, 3>' requested here
            return  this->bid(vanilla_request, std::make_tuple(std::forw...
                          ^
/Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:136:27: note: 
      in instantiation of function template specialization
      'vanilla::Bidder<DSL::GenericDSL<std::__1::basic_string<char>,
      DSL::rapid_mapper>, vanilla::ad_selector<vanilla::BudgetManager, Ad>
      >::bid<openrtb::BidRequest<std::__1::basic_string<char> >, const
      std::__1::basic_string<char> &, (lambda at
      /Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:93:30)
      &, (lambda at
      /Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:101:36)
      &, (lambda at
      /Users/bill.zhu/vanilla-rtb/examples/bidder_experimental/ico_campaign_bidder_test.cpp:110:36)
      &>' requested here
            return bidder.bid(request,
                          ^
/Users/bill.zhu/vanilla-rtb/examples/matchers/ico_campaign.hpp:167:14: note: 
      declared as a non-template here
        bool retrieve(ICOCampaignCollection &ico_campaigns, uint32_t dom...
             ^
2 errors generated.
make[2]: *** [examples/bidder_experimental/CMakeFiles/ico_campaign_bidder_test.dir/ico_campaign_bidder_test.cpp.o] Error 1
make[1]: *** [examples/bidder_experimental/CMakeFiles/ico_campaign_bidder_test.dir/all] Error 2

1. From the wiki page, it seems that there's a part called Banker, but I can't find it.

2. As a matter of fact, when we tried to use this project, we need to do some custom work maybe refactor rtb/core/openrtb.hpp which means many part of it will be modified. So can you give me some advice about this?

3. By the way, is there a branch that supports Redis so that we can handle data with other component written by Python .etc.

Thanks.

Hi, these days I get stuck with another problem, can I return custom http status code?

I read some parts of this project and CRUD project, and I find this,

        request_parser_.reset();
        reply_.headers.resize(0);
        reply_.status = reply::status_type::ok;
        reply_.content="";
        request_ = request(); 
        do_read();

https://github.com/venediktov/CRUD/blob/master/service/persistent_connection.hpp#L123

https://github.com/venediktov/CRUD/blob/master/service/mime_types.cpp#L42

Here's the situation, I want to return 204 http status code, but can't find a way to do this trick. Is this possible? Similarly, some platforms need the Content-Type to be set application/json, so maybe it's a improvement to make the http response header more customizable.

Many thanks.

I have configured the project on linux , cmake 3.17.1 , boost 1.67.0 , gcc 7.5 .

After running make -j4 install i'm getting following error after 63% -

ubuntu@ip-172-26-15-204:~/vanilla-rtb/Release$ make -j4 install
Scanning dependencies of target jsonv
Scanning dependencies of target crud_service
Scanning dependencies of target parsers
Scanning dependencies of target generator
[ 1%] Building C object parsers/CMakeFiles/parsers.dir/jsmn.c.o
[ 3%] Building CXX object CRUD/service/CMakeFiles/crud_service.dir/mime_types.c pp.o
[ 5%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/algorithm_compar e.cpp.o
[ 5%] Built target generator
Scanning dependencies of target generator_experimental
[ 5%] Built target generator_experimental
Scanning dependencies of target vanilla_tests
[ 6%] Building CXX object tests/CMakeFiles/vanilla_tests.dir/vanilla_all.cpp.o
[ 8%] Linking CXX shared library libparsers.so
[ 8%] Built target parsers
[ 10%] Building CXX object tests/CMakeFiles/vanilla_tests.dir/main.cpp.o
[ 11%] Building CXX object CRUD/service/CMakeFiles/crud_service.dir/reply.cpp.o
[ 13%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/algorithm_map.cp p.o
[ 15%] Building CXX object CRUD/service/CMakeFiles/crud_service.dir/request_hand ler.cpp.o
[ 16%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/algorithm_traver se.cpp.o
[ 18%] Linking CXX executable vanilla_tests
[ 18%] Built target vanilla_tests
[ 20%] Building CXX object CRUD/service/CMakeFiles/crud_service.dir/request_pars er.cpp.o
[ 21%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/array.cpp.o
[ 23%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/char_convert.cpp .o
[ 25%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/coerce.cpp.o
/home/ubuntu/vanilla-rtb/jsonv/src/jsonv/coerce.cpp: In function ‘jsonv::value j sonv::coerce_merge(jsonv::value, jsonv::value)’:
/home/ubuntu/vanilla-rtb/jsonv/src/jsonv/coerce.cpp:197:9: warning: this stateme nt may fall through [-Wimplicit-fallthrough=]
if (can_coerce(b, kind::integer))
^~
/home/ubuntu/vanilla-rtb/jsonv/src/jsonv/coerce.cpp:200:5: note: here
case kind::decimal:
^~~~
[ 26%] Linking CXX shared library libcrud_service.so
[ 26%] Built target crud_service
Scanning dependencies of target my_service
[ 28%] Building CXX object CRUD/CMakeFiles/my_service.dir/main.cpp.o
[ 30%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/demangle.cpp.o
[ 31%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/detail.cpp.o
[ 33%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/detail/token_pat terns.cpp.o
[ 35%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/encode.cpp.o
[ 36%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/functional.cpp.o
[ 38%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/object.cpp.o
[ 40%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/parse.cpp.o
[ 41%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/path.cpp.o
[ 43%] Linking CXX executable my_service
[ 45%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/serialization.cp p.o
[ 45%] Built target my_service
[ 46%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/serialization_bu ilder.cpp.o
[ 48%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/tokenizer.cpp.o
[ 50%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/util.cpp.o
[ 51%] Building CXX object jsonv/CMakeFiles/jsonv.dir/src/jsonv/value.cpp.o
/home/ubuntu/vanilla-rtb/jsonv/src/jsonv/util.cpp: In function ‘jsonv::value jso nv::merge_explicit(const jsonv::merge_rules&, jsonv::path, jsonv::value, jsonv:: value)’:
/home/ubuntu/vanilla-rtb/jsonv/src/jsonv/util.cpp:118:13: warning: this statemen t may fall through [-Wimplicit-fallthrough=]
if (b.kind() == kind::integer)
^~
/home/ubuntu/vanilla-rtb/jsonv/src/jsonv/util.cpp:121:9: note: here
case kind::decimal:
^~~~
[ 53%] Linking CXX shared library libjsonv.so
[ 53%] Built target jsonv
Scanning dependencies of target vanilla_rtb
[ 55%] Building CXX object rtb/CMakeFiles/vanilla_rtb.dir/core/logger.cpp.o
[ 56%] Linking CXX shared library libvanilla_rtb.so
[ 56%] Built target vanilla_rtb
Scanning dependencies of target mock_bidder_test
Scanning dependencies of target country_cache_test
Scanning dependencies of target exchange_handler_test
Scanning dependencies of target messaging_client_test
[ 58%] Building CXX object examples/CMakeFiles/country_cache_test.dir/country_ca che_test.cpp.o
[ 60%] Building CXX object examples/CMakeFiles/mock_bidder_test.dir/mock_bidder_ test.cpp.o
[ 61%] Building CXX object examples/CMakeFiles/messaging_client_test.dir/messagi ng_client_test.cpp.o
[ 63%] Building CXX object examples/CMakeFiles/exchange_handler_test.dir/exchang e_handler_test.cpp.o
[ 65%] Linking CXX executable country_cache_test
../rtb/libvanilla_rtb.so.1.1.0: undefined reference to boost::log::v2_mt_posix: :aux::code_convert_impl(wchar_t const*, unsigned long, std::__cxx11::basic_strin g<char, std::char_traits<char>, std::allocator<char> >&, unsigned long, std::loc ale const&)' ../rtb/libvanilla_rtb.so.1.1.0: undefined reference to void boost::log::v2_mt_p osix::aux::put_integer(boost::log::v2_mt_posix::aux::basic_ostringstreambu f<char, std::char_traits, std::allocator >&, unsigned int, unsigned int, char)'
../rtb/libvanilla_rtb.so.1.1.0: undefined reference to boost::log::v2_mt_posix: :sinks::text_file_backend::construct(boost::filesystem::path const&, std::_Ios_O penmode, unsigned long, boost::log::v2_mt_posix::aux::light_function<bool ()> co nst&, bool, bool)' collect2: error: ld returned 1 exit status examples/CMakeFiles/country_cache_test.dir/build.make:120: recipe for target 'ex amples/country_cache_test' failed make[2]: *** [examples/country_cache_test] Error 1 CMakeFiles/Makefile2:518: recipe for target 'examples/CMakeFiles/country_cache_t est.dir/all' failed make[1]: *** [examples/CMakeFiles/country_cache_test.dir/all] Error 2 make[1]: *** Waiting for unfinished jobs.... [ 66%] Linking CXX executable mock_bidder_test CMakeFiles/mock_bidder_test.dir/mock_bidder_test.cpp.o: In function boost::arch ive::detail::common_iarchiveboost::archive::binary_iarchive::vload(boost::arch ive::class_name_type&)':
mock_bidder_test.cpp:(.text._ZN5boost7archive6detail15common_iarchiveINS0_15bina ry_iarchiveEE5vloadERNS0_15class_name_typeE[_ZN5boost7archive6detail15common_iar chiveINS0_15binary_iarchiveEE5vloadERNS0_15class_name_typeE]+0x1): undefined ref erence to boost::archive::basic_binary_iarchive<boost::archive::binary_iarchive >::load_override(boost::archive::class_name_type&)' CMakeFiles/mock_bidder_test.dir/mock_bidder_test.cpp.o: In function void boost: :serialization::throw_exceptionboost::archive::archive_exception(boost::archiv e::archive_exception const&)':
mock_bidder_test.cpp:(.text.ZN5boost13serialization15throw_exceptionINS_7archiv e17archive_exceptionEEEvRKT[ZN5boost13serialization15throw_exceptionINS_7archi ve17archive_exceptionEEEvRKT]+0x1d): undefined reference to boost::archive::ar chive_exception::archive_exception(boost::archive::archive_exception const&)' ../rtb/libvanilla_rtb.so.1.1.0: undefined reference to boost::log::v2_mt_posix: :aux::code_convert_impl(wchar_t const*, unsigned long, std::__cxx11::basic_strin g<char, std::char_traits, std::allocator >&, unsigned long, std::loc ale const&)'
../rtb/libvanilla_rtb.so.1.1.0: undefined reference to void boost::log::v2_mt_p osix::aux::put_integer<char>(boost::log::v2_mt_posix::aux::basic_ostringstreambu f<char, std::char_traits<char>, std::allocator<char> >&, unsigned int, unsigned int, char)' ../rtb/libvanilla_rtb.so.1.1.0: undefined reference to boost::log::v2_mt_posix: :sinks::text_file_backend::construct(boost::filesystem::path const&, std::_Ios_O penmode, unsigned long, boost::log::v2_mt_posix::aux::light_function<bool ()> co nst&, bool, bool)'
collect2: error: ld returned 1 exit status
examples/CMakeFiles/mock_bidder_test.dir/build.make:120: recipe for target 'exam ples/mock_bidder_test' failed
make[2]: *** [examples/mock_bidder_test] Error 1
CMakeFiles/Makefile2:577: recipe for target 'examples/CMakeFiles/mock_bidder_tes t.dir/all' failed
make[1]: *** [examples/CMakeFiles/mock_bidder_test.dir/all] Error 2
[ 68%] Linking CXX executable messaging_client_test
CMakeFiles/messaging_client_test.dir/messaging_client_test.cpp.o: In function b oost::archive::detail::common_iarchive<boost::archive::binary_iarchive>::vload(b oost::archive::class_name_type&)': messaging_client_test.cpp:(.text._ZN5boost7archive6detail15common_iarchiveINS0_1 5binary_iarchiveEE5vloadERNS0_15class_name_typeE[_ZN5boost7archive6detail15commo n_iarchiveINS0_15binary_iarchiveEE5vloadERNS0_15class_name_typeE]+0x1): undefine d reference to boost::archive::basic_binary_iarchive<boost::archive::binary_iar chive>::load_override(boost::archive::class_name_type&)'
CMakeFiles/messaging_client_test.dir/messaging_client_test.cpp.o: In function v oid boost::serialization::throw_exception<boost::archive::archive_exception>(boo st::archive::archive_exception const&)': messaging_client_test.cpp:(.text._ZN5boost13serialization15throw_exceptionINS_7a rchive17archive_exceptionEEEvRKT_[_ZN5boost13serialization15throw_exceptionINS_7 archive17archive_exceptionEEEvRKT_]+0x1d): undefined reference to boost::archiv e::archive_exception::archive_exception(boost::archive::archive_exception const& )'
../rtb/libvanilla_rtb.so.1.1.0: undefined reference to boost::log::v2_mt_posix: :aux::code_convert_impl(wchar_t const*, unsigned long, std::__cxx11::basic_strin g<char, std::char_traits<char>, std::allocator<char> >&, unsigned long, std::loc ale const&)' ../rtb/libvanilla_rtb.so.1.1.0: undefined reference to void boost::log::v2_mt_p osix::aux::put_integer(boost::log::v2_mt_posix::aux::basic_ostringstreambu f<char, std::char_traits, std::allocator >&, unsigned int, unsigned int, char)'
../rtb/libvanilla_rtb.so.1.1.0: undefined reference to boost::log::v2_mt_posix: :sinks::text_file_backend::construct(boost::filesystem::path const&, std::_Ios_O penmode, unsigned long, boost::log::v2_mt_posix::aux::light_function<bool ()> co nst&, bool, bool)' collect2: error: ld returned 1 exit status examples/CMakeFiles/messaging_client_test.dir/build.make:120: recipe for target 'examples/messaging_client_test' failed make[2]: *** [examples/messaging_client_test] Error 1 CMakeFiles/Makefile2:606: recipe for target 'examples/CMakeFiles/messaging_clien t_test.dir/all' failed make[1]: *** [examples/CMakeFiles/messaging_client_test.dir/all] Error 2 [ 70%] Linking CXX executable exchange_handler_test /usr/bin/ld: warning: libboost_system.so.1.67.0, needed by ../CRUD/service/libcr ud_service.so, may conflict with libboost_system.so.1.58.0 CMakeFiles/exchange_handler_test.dir/exchange_handler_test.cpp.o: In function b oost::archive::detail::common_iarchiveboost::archive::binary_iarchive::vload(b oost::archive::class_name_type&)':
exchange_handler_test.cpp:(.text._ZN5boost7archive6detail15common_iarchiveINS0_1 5binary_iarchiveEE5vloadERNS0_15class_name_typeE[_ZN5boost7archive6detail15commo n_iarchiveINS0_15binary_iarchiveEE5vloadERNS0_15class_name_typeE]+0x1): undefine d reference to boost::archive::basic_binary_iarchive<boost::archive::binary_iar chive>::load_override(boost::archive::class_name_type&)' CMakeFiles/exchange_handler_test.dir/exchange_handler_test.cpp.o: In function b oost::re_detail_106700::perl_matcher<char const*, std::allocator<boost::sub_matc h<char const*> >, boost::regex_traits<char, boost::cpp_regex_traits > >::u nwind_extra_block(bool)':
exchange_handler_test.cpp:(.text._ZN5boost16re_detail_10670012perl_matcherIPKcSa INS_9sub_matchIS3_EEENS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEE18unwind_ext ra_blockEb[ZN5boost16re_detail_10670012perl_matcherIPKcSaINS_9sub_matchIS3_EEEN S_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEE18unwind_extra_blockEb]+0x22): und efined reference to boost::re_detail_106700::put_mem_block(void*)' CMakeFiles/exchange_handler_test.dir/exchange_handler_test.cpp.o: In function v oid boost::re_detail_106700::raise_error<boost::regex_traits_wrapper<boost::rege x_traits<char, boost::cpp_regex_traits > > >(boost::regex_traits_wrapper<b oost::regex_traits<char, boost::cpp_regex_traits > > const&, boost::regex constants::error_type)':
exchange_handler_test.cpp:(.text._ZN5boost16re_detail_10670011raise_errorINS_20r egex_traits_wrapperINS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEEEEEvRKT_NS_15 regex_constants10error_typeE[_ZN5boost16re_detail_10670011raise_errorINS_20regex _traits_wrapperINS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEEEEEvRKT_NS_15rege x_constants10error_typeE]+0x98): undefined reference to boost::re_detail_106700 ::get_default_error_string(boost::regex_constants::error_type)' exchange_handler_test.cpp:(.text._ZN5boost16re_detail_10670011raise_errorINS_20r egex_traits_wrapperINS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEEEEEvRKT_NS_15 regex_constants10error_typeE[_ZN5boost16re_detail_10670011raise_errorINS_20regex _traits_wrapperINS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEEEEEvRKT_NS_15rege x_constants10error_typeE]+0xc9): undefined reference to boost::re_detail_106700 ::raise_runtime_error(std::runtime_error const&)'
CMakeFiles/exchange_handler_test.dir/exchange_handler_test.cpp.o: In function b oost::re_detail_106700::perl_matcher<char const*, std::allocator<boost::sub_matc h<char const*> >, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::e xtend_stack()': exchange_handler_test.cpp:(.text._ZN5boost16re_detail_10670012perl_matcherIPKcSa INS_9sub_matchIS3_EEENS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEE12extend_sta ckEv[_ZN5boost16re_detail_10670012perl_matcherIPKcSaINS_9sub_matchIS3_EEENS_12re gex_traitsIcNS_16cpp_regex_traitsIcEEEEE12extend_stackEv]+0x2e): undefined refer ence to boost::re_detail_106700::get_mem_block()'
CMakeFiles/exchange_handler_test.dir/exchange_handler_test.cpp.o: In function c har const* boost::re_detail_106700::re_is_set_member<char const*, char, boost::r egex_traits<char, boost::cpp_regex_traits<char> >, unsigned int>(char const*, ch ar const*, boost::re_detail_106700::re_set_long<unsigned int> const*, boost::re_ detail_106700::regex_data<char, boost::regex_traits<char, boost::cpp_regex_trait s<char> > > const&, bool)': exchange_handler_test.cpp:(.text._ZN5boost16re_detail_10670016re_is_set_memberIP KccNS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEjEET_S8_S8_PKNS0_11re_set_longI T2_EERKNS0_10regex_dataIT0_T1_EEb[_ZN5boost16re_detail_10670016re_is_set_memberI PKccNS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEjEET_S8_S8_PKNS0_11re_set_long IT2_EERKNS0_10regex_dataIT0_T1_EEb]+0x360): undefined reference to boost::re_de tail_106700::cpp_regex_traits_implementation::transform_primary[abi:cxx11] (char const*, char const*) const'
exchange_handler_test.cpp:(.text._ZN5boost16re_detail_10670016re_is_set_memberIP KccNS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEjEET_S8_S8_PKNS0_11re_set_longI T2_EERKNS0_10regex_dataIT0_T1_EEb[_ZN5boost16re_detail_10670016re_is_set_memberI PKccNS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEjEET_S8_S8_PKNS0_11re_set_long IT2_EERKNS0_10regex_dataIT0_T1_EEb]+0x474): undefined reference to boost::re_de tail_106700::cpp_regex_traits_implementation<char>::transform[abi:cxx11](char co nst*, char const*) const' CMakeFiles/exchange_handler_test.dir/exchange_handler_test.cpp.o: In function v oid boost::serialization::throw_exceptionboost::archive::archive_exception(boo st::archive::archive_exception const&)':
exchange_handler_test.cpp:(.text.ZN5boost13serialization15throw_exceptionINS_7a rchive17archive_exceptionEEEvRKT[ZN5boost13serialization15throw_exceptionINS_7 archive17archive_exceptionEEEvRKT]+0x1d): undefined reference to boost::archiv e::archive_exception::archive_exception(boost::archive::archive_exception const& )' CMakeFiles/exchange_handler_test.dir/exchange_handler_test.cpp.o: In function b oost::re_detail_106700::perl_matcher<char const*, std::allocator<boost::sub_matc h<char const*> >, boost::regex_traits<char, boost::cpp_regex_traits > >::m atch_imp()':
exchange_handler_test.cpp:(.text._ZN5boost16re_detail_10670012perl_matcherIPKcSa INS_9sub_matchIS3_EEENS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEE9match_impEv [ZN5boost16re_detail_10670012perl_matcherIPKcSaINS_9sub_matchIS3_EEENS_12regex traitsIcNS_16cpp_regex_traitsIcEEEEE9match_impEv]+0x1c): undefined reference to boost::re_detail_106700::get_mem_block()' exchange_handler_test.cpp:(.text._ZN5boost16re_detail_10670012perl_matcherIPKcSa INS_9sub_matchIS3_EEENS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEE9match_impEv [_ZN5boost16re_detail_10670012perl_matcherIPKcSaINS_9sub_matchIS3_EEENS_12regex_ traitsIcNS_16cpp_regex_traitsIcEEEEE9match_impEv]+0x216): undefined reference to boost::re_detail_106700::verify_options(unsigned int, boost::regex_constants:: _match_flags)'
exchange_handler_test.cpp:(.text._ZN5boost16re_detail_10670012perl_matcherIPKcSa INS_9sub_matchIS3_EEENS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEE9match_impEv [ZN5boost16re_detail_10670012perl_matcherIPKcSaINS_9sub_matchIS3_EEENS_12regex traitsIcNS_16cpp_regex_traitsIcEEEEE9match_impEv]+0x2c0): undefined reference to boost::re_detail_106700::put_mem_block(void*)' exchange_handler_test.cpp:(.text._ZN5boost16re_detail_10670012perl_matcherIPKcSa INS_9sub_matchIS3_EEENS_12regex_traitsIcNS_16cpp_regex_traitsIcEEEEE9match_impEv [_ZN5boost16re_detail_10670012perl_matcherIPKcSaINS_9sub_matchIS3_EEENS_12regex_ traitsIcNS_16cpp_regex_traitsIcEEEEE9match_impEv]+0x3f7): undefined reference to boost::re_detail_106700::put_mem_block(void*)'
CMakeFiles/exchange_handler_test.dir/exchange_handler_test.cpp.o: In function h ttp::server::persistent_connection<http::crud::crud_dispatcher<http::server::req uest, http::server::reply, boost::match_results<char const*, std::allocator<boos t::sub_match<char const*> > >, boost::basic_regex<char, boost::regex_traits<char , boost::cpp_regex_traits<char> > > > >::do_read()::{lambda(boost::system::error _code, unsigned long)#1}::operator()(boost::system::error_code, unsigned long) c onst': exchange_handler_test.cpp:(.text._ZZN4http6server21persistent_connectionINS_4cru d15crud_dispatcherINS0_7requestENS0_5replyEN5boost13match_resultsIPKcSaINS6_9sub _matchIS9_EEEEENS6_11basic_regexIcNS6_12regex_traitsIcNS6_16cpp_regex_traitsIcEE EEEEEEE7do_readEvENKUlNS6_6system10error_codeEmE_clESN_m[_ZZN4http6server21persi stent_connectionINS_4crud15crud_dispatcherINS0_7requestENS0_5replyEN5boost13matc h_resultsIPKcSaINS6_9sub_matchIS9_EEEEENS6_11basic_regexIcNS6_12regex_traitsIcNS 6_16cpp_regex_traitsIcEEEEEEEEE7do_readEvENKUlNS6_6system10error_codeEmE_clESN_m ]+0x56a): undefined reference to boost::re_detail_106700::perl_matcher<char con st*, std::allocator<boost::sub_match<char const*> >, boost::regex_traits<char, b oost::cpp_regex_traits > >::construct_init(boost::basic_regex<char, boost: :regex_traits<char, boost::cpp_regex_traits > > const&, boost::regex_const ants::_match_flags)'
../rtb/libvanilla_rtb.so.1.1.0: undefined reference to boost::log::v2_mt_posix: :aux::code_convert_impl(wchar_t const*, unsigned long, std::__cxx11::basic_strin g<char, std::char_traits<char>, std::allocator<char> >&, unsigned long, std::loc ale const&)' ../rtb/libvanilla_rtb.so.1.1.0: undefined reference to void boost::log::v2_mt_p osix::aux::put_integer(boost::log::v2_mt_posix::aux::basic_ostringstreambu f<char, std::char_traits, std::allocator >&, unsigned int, unsigned int, char)'
../rtb/libvanilla_rtb.so.1.1.0: undefined reference to boost::log::v2_mt_posix: :sinks::text_file_backend::construct(boost::filesystem::path const&, std::_Ios_O penmode, unsigned long, boost::log::v2_mt_posix::aux::light_function<bool ()> co nst&, bool, bool)' collect2: error: ld returned 1 exit status examples/CMakeFiles/exchange_handler_test.dir/build.make:118: recipe for target 'examples/exchange_handler_test' failed make[2]: *** [examples/exchange_handler_test] Error 1 CMakeFiles/Makefile2:548: recipe for target 'examples/CMakeFiles/exchange_handle r_test.dir/all' failed make[1]: *** [examples/CMakeFiles/exchange_handler_test.dir/all] Error 2 Makefile:160: recipe for target 'all' failed make: *** [all] Error 2

https://github.com/venediktov/vanilla-rtb/blob/master/rtb/exchange/exchange_handler.hpp#L129
We have a new feature e.g.
when HTTP 204 is needed we have provided with exchange_handler.if_response() handler which
should be used and custom response should be sent .
The custom response is sent OK however exchange_handler redundantly converts standard response to
wire format causing lag in performance.

Proposing change remove line L129 and add it below

if (auction_response && timer.expires_from_now().total_milliseconds() > 0) {
                    if(if_response_handler) {
                       auto custom_reply = if_response_handler(auction_response);
                       if ( custom_reply ) {
                          custom_reply(r);
                       } else {
                          wire_response = parser.create_response(auction_response);
                          r << to_string(*wire_response) << http::server::reply::flush("json");
                       }
                    } else {
                       wire_response = parser.create_response(auction_response);
                       r << to_string(*wire_response) << http::server::reply::flush("json");
                    }
                } else {
                    r << http::server::reply::flush("json");
                }

And looking at it again maybe time to re-factor it as this code is too verbose .

With 98000 records in current ad_geo file, our current implementation of datacache for geo lookup is slow, we are looking to speed up fetching of all ads for particular impression.geo down to
10 microseconds .
We want all lookups to contribute to no more then 12-15% of all overheads.
Currently most data lookups are within 10 microseconds, however ag_geo is much higher.

Is there a built-in payout calculator? Something that provides the total revenue per ad?

http://stackoverflow.com/questions/27819004/what-is-the-status-of-abbreviated-functions-in-c

We should use -Wpedantic-errors or -Wpedantic flag like jsonv , this way it guarantees we only use standard features of C++14.

looks like "abreviated functions" are not standard yet, I think we have to support standard .

template function( T && t)
vs.
function( auto && t)

if(WIN32)
else(WIN32)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wpedantic-errors -Wall -Wextra")
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_DEBUG} -ggdb")
endif(WIN32)

Even though we see people attempt to build our stack , sometimes it's required to update compilers and dependency libraries. To eliminate this leg work and make our stack build-ready for open source community our team decided to provide an isolated container based solution - container images .
The images are locked down Linux containers with file system containing complete vanilla stack and all
its build/run dependencies .

On our plate:

• Create docker directory with base, dev, prod docker files
• Integrate github.com with hub.docker.com for automated builds
• Make sure hub.docker.com pre-builds vanilla image in Release mode
• Create readme on how to build and release docker containers manually
• Introduce GitHub tags and make a first release of existing vanilla stack on GitHub
• Make sure the hub.docker.com only builds based on our GitHub tags : basically skip intermediate updates to git repository and only release docker images based on our GitHub tags.

reference #40
I tried building the project in cmake 3.2.2 but gives errors like

• cannot run .c
• threads not found
  but when i upgraded to cmake 3.8.2, the build was successfull
  so right now i'm using gcc - 7.1.0 & boost - 1.61

it will better to provide the least version of cmake supported, for those who are new to this repo.

Regards.

Hello, while i'm compiling this project, i have encountered following errors:

vanilla-rtb/rtb/messaging/serialization.hpp:222:28: error: invalid initialization of non-const reference of type ‘long unsigned int&’ from an rvalue of type ‘jsonv::detail::string_view::size_type {aka long unsigned int}’
             ar & value.size();
                            ^
In file included from /usr/include/boost/archive/detail/common_oarchive.hpp:22:0,
                 from /usr/include/boost/archive/basic_binary_oarchive.hpp:34,
                 from /usr/include/boost/archive/binary_oarchive_impl.hpp:23,
                 from /usr/include/boost/archive/binary_oarchive.hpp:21,
                 from /home/pantianxiang/vanilla-rtb/rtb/messaging/communicator.hpp:31,
                 from /home/pantianxiang/vanilla-rtb/examples/exchange_handler_test.cpp:13:
/usr/include/boost/archive/detail/interface_oarchive.hpp:69:15: note:   initializing argument 1 of ‘Archive& boost::archive::detail::interface_oarchive<Archive>::operator&(T&) [with T = long unsigned int; Archive = boost::archive::binary_oarchive]’
     Archive & operator&(T & t){
               ^
In file included from /usr/include/c++/5/algorithm:62:0,
                 from /usr/include/boost/smart_ptr/shared_ptr.hpp:37,
                 from /usr/include/boost/log/core/core.hpp:19,
                 from /usr/include/boost/log/sources/basic_logger.hpp:37,
                 from /usr/include/boost/log/sources/severity_logger.hpp:23,
                 from /usr/include/boost/log/trivial.hpp:22,
                 from /home/pantianxiang/vanilla-rtb/examples/exchange_handler_test.cpp:2:
/usr/include/c++/5/bits/stl_algo.h:3761:5: error: ‘_Funct std::for_each(_IIter, _IIter, _Funct) [with _IIter = const char*; _Funct = boost::serialization::save(Archive&, const string_view&, unsigned int) [with Archive = boost::archive::binary_oarchive; jsonv::string_view = jsonv::detail::string_view]::<lambda(char)>]’, declared using local type ‘boost::serialization::save(Archive&, const string_view&, unsigned int) [with Archive = boost::archive::binary_oarchive; jsonv::string_view = jsonv::detail::string_view]::<lambda(char)>’, is used but never defined [-fpermissive]
     for_each(_InputIterator __first, _InputIterator __last, _Function __f)
     ^
examples/CMakeFiles/exchange_handler_test.dir/build.make:62: recipe for target 'examples/CMakeFiles/exchange_handler_test.dir/exchange_handler_test.cpp.o' failed

Seem like the initialization od value.size() is invalid, yet i'm not aware how to initialize it correctly. Looking forward for your help.

We have an old Angular-2 CDN based campaign budget UI management piece of code with HTML
We'd like to have :

• Modern MVC framework UI widget in AngularJS / ReactJS or other presumably in TypeScript
• It has to be integrated with existing backend service with existing REST API
• UI developer will work with our backend engineer
• UI is npm installable with project.json
• The bounty will be paid via Beer Pay / Stripe

For more questions either use our gitter chat channel or IM me directly via email or leave your email by replying to this issue on GitHub.

• create docker-compose-vanilla.yaml
  this command "docker-compose -f docker-compose-vanilla.yaml"
  should start NGINX configured for vanilla-rtb docker image + vanilla-rtb image with 3-4 http_handler_test running on 3-4 different ports .
  docker compose file should map volume for both nginx and vanilla-rtb , so that we could alter configuration if needed.
• add performance tests results to our wiki

on running run.sh
cd Release/examples/bin
./http_bidder_test --config etc/config.cfg
./cache_loader_test --config etc/config.cfg
./exchange_handler_test --config etc/config.cfg
./mock_bidder_test --config etc/config.cfg
./multi_bidder --config etc/config.cfg
./multi_exchange_handler --config etc/config.cfg
./notification_service_test --config etc/config.cfg
./slavebanker_service_test --config etc/config.cfg
./campaign_manager_test --config etc/config.cfg

i get this output in terminal..
reoxey@reoxey-desktop:~/CLionProjects/vanilla-rtb$ sh ./run.sh

[2017-06-17 12:42:39.832061] [0x00007f3f5e222900] [debug]   bidder.ads_ipc_name vanilla-ads-ipc
bidder.ads_source data/ads
bidder.campaign_data_ipc_name vanilla-campaign-data-ipc
bidder.campaign_data_source data/campaign_data
bidder.concurrency 0
bidder.geo_ad_ipc_name vanilla-geo-ad-ipc
bidder.geo_ad_source data/ad_geo
bidder.geo_campaign_ipc_name vanilla-geo-campaign-ipc
bidder.geo_campaign_source data/geo_campaign
bidder.geo_ipc_name vanilla-geo-ipc
bidder.geo_source data/geo
bidder.host 0.0.0.0
bidder.key_value_host 0.0.0.0
bidder.key_value_port 0
bidder.log /tmp/bidder_log
bidder.port 9081
bidder.root .
bidder.timeout 50
config etc/config.cfg

12:42:39.924693: <info> 
ad load : elapsed_sec=0|elapsed_ms=32|elapsed_mu=32341|elapsed_ns=32341990|
geo_data load : elapsed_sec=0|elapsed_ms=9|elapsed_mu=9912|elapsed_ns=9912520|
geo_campaign load : elapsed_sec=0|elapsed_ms=48|elapsed_mu=48943|elapsed_ns=48943310|
selector load : elapsed_sec=0|elapsed_ms=91|elapsed_mu=91217|elapsed_ns=91217176|
terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::system::system_error> >'
  what():  resolve: Host not found (authoritative)
Aborted (core dumped)
[2017-06-17 12:42:41.621299] [0x00007f298fae9780] [debug]   bidder.campaign_data_ipc_name vanilla-campaign-data-ipc
bidder.campaign_data_source data/campaign_data
bidder.geo_campaign_ipc_name vanilla-geo-campaign-ipc
bidder.geo_campaign_source data/geo_campaign
cache-loader.host 0.0.0.0
cache-loader.log /tmp/vanilla_cache_loader_log
cache-loader.port 10081
cache-loader.root .
config etc/config.cfg
datacache.ads_ipc_name vanilla-ads-ipc
datacache.ads_source bidder/data/ads
datacache.geo_ad_ipc_name vanilla-geo-ad-ipc
datacache.geo_ad_source bidder/data/ad_geo
datacache.geo_ipc_name vanilla-geo-ipc
datacache.geo_source bidder/data/geo

terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::system::system_error> >'
  what():  resolve: Host not found (authoritative)
Aborted (core dumped)
[2017-06-17 12:42:41.733372] [0x00007f7cb43da800] [debug]   config etc/config.cfg
exchange.host 0.0.0.0
exchange.log /tmp/openrtb_handler_log
exchange.port 8081
exchange.root .
exchange.v1.timeout 100
exchange.v2.timeout 80
mock-bidder.num_of_bidders 5
mock-bidder.port 5000

terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::system::system_error> >'
  what():  resolve: Host not found (authoritative)
Aborted (core dumped)
12:42:41.845543: <info> Starting mock bidder pid=9545
12:42:41.846075: <info> Starting mock bidder pid=9546
12:42:41.846144: <info> Starting mock bidder pid=9547
12:42:41.846563: <info> Starting mock bidder pid=9548
12:42:41.846910: <info> Starting mock bidder pid=9549

when i try to open localhost:11081 or localhost:11081/campaign, this error shows up.

This site can’t be reached
localhost refused to connect.

what am i doing wrong?

Our current implementation of exchange_handler
boost.asio/jsmn-parser/any_mapper is behind netty/openrtb-core ( Java ) by a lot
On the 4 core machine netty/openrtb-core can get up to 25K QPSl while our exchange_handler is doing about 20K QPS.

1. Introduce rapidjson library to the stack
2. remove extra step encoding to jsonv::value or boost::any and encode directly into openrtb::BidRequest
3. implement json writers with rapidjson API

Write user matching interfaces based on key /value storage - no implementation however possible easy integration with Redis.

we need a sample loader at least for caches we already have in place
examples/loader - proposed directory structure where loader supporting code will reside

When we do lookup in our caches we convert process memory ( e.g. std::string ) into ipc memory with
with entity_cache.create_ipc_key.
This can be avoided by utilizing transient std::less

we need some Banker generic interface classes in our core library which then can
be used for some sample code in
examples/banker where we should provide some implementation possibly utilizing our rtb/datacache generic structures.

features :

• add / withdraw funds
• audit transactions
• put holds on bidded amounts
• revert bid if no win or timer

• Add google proto
• Add DSL mapper for it.
• Add bidder example

We should try creating bidder per targeting or per other criteria to see if we
can segregate bidders and still process single request by multiple .

connecting all of those bidder-slaves to master endpoint ( exchange_handler ) via communicator
is the idea.

High level idea:

exchange_handler -> broadcast -> collect ( on master host )
bidders -> process -> reply ( on multiple or single slave machine )

within bidder-> process targeting rules should be applied and BidResponse should be sent back to
master ( exchange_handler ) , which will collect responses from all bidders.
Simple selection algo should be applied choosing highest bid.

Performance should be recorded with perf_timer and documented .

Boost serialization is optional in our stack and only used for multi-bidder model.
It has to be drawn in dotted lines or diagram needs to have comment in the bottom.

auction handler's timeout in exchange_handler is implemented as std::future::wait_for and so it spawns
new thread for every incoming message.
This made our R&D tests fail with TCP/IP redis client and therefore we have decided to try continue
calling auction on the same thread with io_service timer.

We need to show in examples how bidder can interface campaign budget

• running on single machine configured to decrease funds from master cache by notification_service and bidders reading budget directly from master cache - single machine model .
• running multi-bidder examples and decreasing funds in the master cache and reading updated budget from the slave cache updated by slavebanker_service_test.cpp

• see if travis can handle C++17 either gcc 7.1 or clang 5.0
• move our C++17 implementation of audit_buffer.hpp , audit_codec.hpp into official build
• change code as needed to compile on gcc >= 7.1 and on clang >= 5.0
• update README.md

Need some way to test on different compilers and different OS to build the stack , looking into
integrating with Travis CI. Seems not as trivial as I thought , but hope to finish its script in few days

our build process automatically produces test data for ad matching , sometimes it produces ads that
don't match impression in BID_REQUEST_BANNER.json