stiffstream / restinio Goto Github PK

I've been doing benchmarks for multiple web-servers/REST frameworks inlcuding rwasa, asmttpd, C++ REST SDK, Spring MVC, Spring Webflux and Restinio.
The performance for sending files with Restinio was among the best. Especially that it didn't have the deadlocking problem serving files with multiple requests that the C++ REST SDK had.
However, I've noticed a strange behavior with Restinio: I make 4 comparative benchmarks with Gatling: 2 with 300 simultaneous users and 2 with 3000 users, each one is done with 70 bytes and 700kb files respectively. The smaller file is an HTTP file and the larger one is a binary file.
For the larger file the response time has been like this:

and for the smaller file like this:

As you can see, the smaller file has a larger average time for some reason! I've done that multiple times and always had the same results. Any explanation for that?
All the tests were done on the sendfile sample on Linux

in the example section i think the good solution is:

 http_router->non_matched_request_handler(
                    [](auto req) {
                        return req->create_response(restinio::status_not_found()).set_body("Not Found").done();
                    });

create_response take only one argument on my machine (0.5.1 full version)

https://github.com/Stiffstream/restinio/blob/master/dev/restinio/request_handler.hpp#L78

Is there a specific reason that all find methods in http_header_fields_t are private?
If I want to check for the existence of a field and get its value if it exists, I have to either use has_field and get_field doing the lookup twice, or I could use the public begin and end methods with std::find_if duplicating what find already does.

Hi,
I would like to try your elegnant C++ REST library on Linux.
I read the documentation + test cases, but i couldnt find async TLS client implementation.
May u please refer me to relevant documentation or code snippet?

Thanks,
David.

Hi devs!

Is it possible to define extra HTTP methods even if in their definition it states that its their mapping to nodejs ones? What is this relationship? Basically, we need to use ENCRYPT, SIGN, etc. Currently, I wrote a RESTful oriented ones like /:resource/verb but it would break the current RESTful API which we don't want to. I want to find the right way to do it. I could always make a PR if necessary!

Thank you for your time!
Seva

It seems that there is an ambiguity as to when an asterisk must be percent-encoded and when it is allowed to be used as-is (see this Stack Overflow question).

We have the problem here that some of our REST requests come from JavaScript's encodeURIComponent which apparently does not escape an asterisk. The problem is that parse_query throws an exception when given a query like this: http://example.com/find?name=A*.

So would you be willing to let parse_query (maybe as an option) accept unescaped asterisks?

Hello.
Is there any sample or documentation about how to handle files uploaded from the client?

Hi devs,

As I'm finishing the integration of RESTinio as a proxy server, I'm wondering how would I properly redirect the logging using the restinio::traits_t or perhaps, another way that you would suggest?

Basically, we have our own logger, and I would like to redirect it to the same one to avoid clashing and format a little bit the output to be standard to ours.

Cheers,
Seva

Hi,

I couldn't compile restinio with this flag: /std:c++17. It worked flawlessly if i downgrade the C++ version to c++14.

Environment:

Visual Studio 2017 - x64
Compiler Version: 19.15.26730

Error message:

Severity	Code	Description	Project	File	Line	Suppression State
Error	C2440	'<function-style-cast>': cannot convert from 'initializer list' to 'std::cregex_iterator'	sample	include\restinio\path2regex\path2regex.hpp	685	
Error	C2789	'escaped': an object of const-qualified type must be initialized	sample	include\restinio\path2regex\path2regex.hpp	708	
Error	C3536	'escaped': cannot be used before it is initialized	sample	include\restinio\path2regex\path2regex.hpp	709	
Error	C2660	'restinio::path2regex::impl::check_no_unescaped_brackets': function does not take 1 arguments	sample	include\restinio\path2regex\path2regex.hpp	705	
Error	C2660	'restinio::path2regex::impl::check_no_unescaped_brackets': function does not take 1 arguments	sample	include\restinio\path2regex\path2regex.hpp	727	
Error	C3536	'token_it': cannot be used before it is initialized	sample	include\restinio\path2regex\path2regex.hpp	688	
Error	C2679	binary '!=': no operator found which takes a right-hand operand of type 'std::cregex_iterator' (or there is no acceptable conversion)	sample	include\restinio\path2regex\path2regex.hpp	694	
Error	C2679	binary '==': no operator found which takes a right-hand operand of type 'std::cregex_iterator' (or there is no acceptable conversion)	sample	include\restinio\path2regex\path2regex.hpp	688	
Error	C2679	binary '==': no operator found which takes a right-hand operand of type 'std::cregex_iterator' (or there is no acceptable conversion)	sample	include\restinio\path2regex\path2regex.hpp	723	
Error	C2100	illegal indirection	sample	include\restinio\path2regex\path2regex.hpp	696	
Error	C2109	subscript requires array or pointer type	sample	include\restinio\path2regex\path2regex.hpp	708	
Error	C2109	subscript requires array or pointer type	sample	include\restinio\path2regex\path2regex.hpp	712

Previous version:

include(FetchContent)
FetchContent_Declare(
        restinio
        URL https://bitbucket.org/sobjectizerteam/restinio/downloads/restinio-0.5.1-full.zip
)
FetchContent_MakeAvailable(restinio)
add_subdirectory(${restinio_SOURCE_DIR}/dev/fmt ${restinio_BINARY_DIR}/fmt)
add_subdirectory(${restinio_SOURCE_DIR}/dev/nodejs/http_parser ${restinio_BINARY_DIR}/http_parser)
add_subdirectory(${restinio_SOURCE_DIR}/dev ${restinio_BINARY_DIR})

add_library(restiniofull INTERFACE)
target_link_libraries(restiniofull INTERFACE restinio::restinio)
target_include_directories(restiniofull INTERFACE ${restinio_SOURCE_DIR}/dev/asio/include)
target_compile_definitions(restiniofull INTERFACE $<$<CXX_COMPILER_ID:MSVC>:_SILENCE_ALL_CXX17_DEPRECATION_WARNINGS>
        $<$<CXX_COMPILER_ID:MSVC>:_WIN32_WINNT=0x0A00> -DASIO_STANDALONE -DASIO_HAS_STD_CHRONO -DASIO_DISABLE_STD_STRING_VIEW)
add_library(myproject::restinio ALIAS restiniofull)
target_link_libraries(my_exec PUBLIC myproject::restinio)

new version:

include(FetchContent)
FetchContent_Declare(
        restinio
        URL https://bitbucket.org/sobjectizerteam/restinio/downloads/restinio-0.5.1.1-full.zip
)
FetchContent_MakeAvailable(restinio)
add_subdirectory(${restinio_SOURCE_DIR}/dev/fmt ${restinio_BINARY_DIR}/fmt)
add_subdirectory(${restinio_SOURCE_DIR}/dev/nodejs/http_parser ${restinio_BINARY_DIR}/http_parser)
add_subdirectory(${restinio_SOURCE_DIR}/dev ${restinio_BINARY_DIR})

add_library(restiniofull INTERFACE)
target_link_libraries(restiniofull INTERFACE restinio::restinio)
target_include_directories(restiniofull INTERFACE ${restinio_SOURCE_DIR}/dev/asio/include)
target_compile_definitions(restiniofull INTERFACE $<$<CXX_COMPILER_ID:MSVC>:_SILENCE_ALL_CXX17_DEPRECATION_WARNINGS>
        $<$<CXX_COMPILER_ID:MSVC>:_WIN32_WINNT=0x0A00> -DASIO_STANDALONE -DASIO_HAS_STD_CHRONO -DASIO_DISABLE_STD_STRING_VIEW)
add_library(myproject::restinio ALIAS restiniofull)

target_link_libraries(my_exec PUBLIC myproject::restinio)

Second version seem's to call CMake before building each time (only the version downloaded change) very annoying.

like:

cmake -DCMAKE_BUILD_TYPE=Debug -DCMAKE_CXX_COMPILER=/usr/local/opt/llvm/bin/clang++ ../

make # this line call cmake again with new version, why ?

Hi,

Is there a filter feature to intercept all request?
I am going to implement generic JWT authentication and metrics handler but couldn't figure out the optimal solution.

Thanks

Hi Stiffstream,

I'm trying to implement a user_controlled_output_t as a response inside my get handler.
However, the connection is getting closed on a flush() as seen in this simple:

request_status Server::get(restinio::request_handle_t request,
                         restinio::router::route_params_t params)
{
    printf("connection_id: %lu\n", request->connection_id());

    using output_t = restinio::user_controlled_output_t;
    auto response = this->init_http_resp(request->create_response<output_t>(

    ));
    //response.set_content_length(request->body().size());
    response.flush();
    std::this_thread::sleep_for(std::chrono::seconds(2));

    std::string part1 = "i like";
    response.set_body(part1);
    response.set_content_length(part1.size());
    response.flush();
    std::this_thread::sleep_for(std::chrono::seconds(2));

    std::string part2 = " waffles!";
    response.set_body(part2);
    response.set_content_length(part2.size());
    response.flush();
    std::this_thread::sleep_for(std::chrono::seconds(2));

    return response.done();
}

I'm implementing your amazing RESTinio server into OpenDHT project that needs to be able to send get responses by parts i.e. head first then, once a callback is fired for a value found on a node, another one and so forth until done callback.

I also went into debugging your code base with gdb and the flags are properly set:

gdb$ b /usr/local/include/restinio/impl/connection.hpp:794
Breakpoint 1 at 0xb10b1: file /usr/local/include/restinio/impl/connection.hpp, line 794.
...
gdb$ p response_output_flags
$1 = {m_response_parts = restinio::response_parts_attr_t::not_final_parts, m_response_connection = restinio::response_connection_attr_t::connection_keepalive}

But it seems that the connection is getting closed on a flush() which is strange to me because the definition of a flush would be considered a different one from done() in terms of not closing the connection with response_parts_attr_t::not_finals_parts whereas a done() is sending response_parts_attr_t::final_parts flag.

The goal is to be able to perform an async yield like this part that I am removing.

Thank you for your time!

Sincerely,
Seva

Suggested feature.

Ie to avoid having to do this for each parameter:

auto thing_param = restinio::utils::unescape_percent_encoding(params["thing"]);

The ::CreateFileA used to open the file for sendfile does not have the FILE_SHARE_READ option set. This means that the file is opened exclusively and the next attempt to open the file will fail until the sendfile operation has finished and the handle is closed.

There are no helpers for dealing with values of HTTP-fields in RESTinio at the current moment. For example, if someone wants to handle file upload request, then he/she has to deal with the value of 'Content-Type' HTTP-field by him/herself. It means that the value of 'Content-Type' should be manually parsed, the presence of 'multipart/form-data" should be checked and the fragment with 'boundary' should be found and handled.

I think it's not a good situation because RESTinio is intended to be easy-to-use but still a rather performant library. So I have an idea about adding some simple helpers for parsing values of HTTP-fields. And I want to discuss some design decisions here.

There is just a sketch of how it can look like:

#include <restinio/all.hpp>
#include <restinio/helpers/http_field_parser.hpp> // NOTE: additional header!

void handle_request(const restinio::request_handle_t & req) {
   // We expect that Content-Type header is present.
   const auto content_type = req->header().value_of(restinio::http_field::content_type);

   // Try to ensure that it has 'multipart/form-data' value and 'boundary' fragment.
   std::string boundary; // A receiver for the value of 'boundary'
   if(!restinio::http_field_parser::try_parse_field_value(content_type,
         ';', // The separator that should separate fragments of HTTP-field values.
         restinio::http_field_parser::expect("multipart/form-data"),
         restinio::http_field_parser::name_value("boundary", boundary)))
      throw std::runtime_error("Unexpected value of 'Content-Type'!");

   ... // Now boundary variable contains the value of 'boundary' fragment.
}

This call to try_parse_field_value allows to parse the content of field in the form multipart/form-data; boundary=Some_Boundary_Value or multipart/form-data; boundary="Some Boundary Value".

There can be also a function try_parse_whole_field that allows to parse the whole HTTP-field, without stripped field header (as in the previous example):

std::string whole_field = "Content-Disposition: form-data; name=\"file-name\"; filename=\"demo.txt\"";
std::string name;
std::string filename;
if(restinio::http_field_parser::try_parse_whole_field(whole_field,
      "content-disposition", // Expected field name.
      ';', // The separator that should separate fragments of HTTP-field values.
      restinio::http_field_parser::expect("form-data"),
      restinio::http_field_parser::name_value("name", name),
      restinio::http_field_parser::name_value("filename", filename)))
{
   assert("file-name" == name);
   assert("demo.txt" == filename);
   ...
}

So the first question is: is this set of helper functions good enough as a starting point?

The second question is related to the way of acquiring a value from name_value(). At the current moment the value is stored to a variable of type std::string (reference to that variable is passed to name_value() function).

But that design has at least two drawbacks:

The first one is rewriting of the current value of the variable even in the case when try_parse_* function returns false. For example:

std::string whole_field = "Content-Disposition: form-data; name=\"file-name\"";
std::string name;
std::string filename;
if(restinio::http_field_parser::try_parse_whole_field(whole_field,
      "content-disposition", // Expected field name.
      ';', // The separator that should separate fragments of HTTP-field values.
      restinio::http_field_parser::expect("form-data"),
      restinio::http_field_parser::name_value("name", name),
      restinio::http_field_parser::name_value("filename", filename)))
{
   assert("file-name" == name);
   assert("demo.txt" == filename);
   ...
}

In that case try_parse_whole_field will return false, but name variable will hold file-name value after the return from try_parse_whole_field. It's a consequence of parser working scheme.

The second drawback is inability to work with other types of destinations. For example, someone may want to use std::vector<char> instead of std::string. Or something else. So maybe there should be a version of name_value that accepts inserter:

std::vector<char> boundary; // A receiver for the value of 'boundary'
if(!restinio::http_field_parser::try_parse_field_value(
      req->header().value_of(restinio::http_field::content_type),
      ';',
      restinio::http_field_parser::expect("multipart/form-data"),
      restinio::http_field_parser::name_value("boundary", std::back_inserter(boundary))))
...

But there is another approach for returning parsed values. It seems that try_parse_* can return a tuple where the first item will be bool and all following items will be std::string. In that case we can write code like that:

std::string whole_field = "Content-Disposition: form-data; name=\"file-name\"; filename=\"demo.txt\"";
const auto result = restinio::http_field_parser::try_parse_whole_field(whole_field,
      "content-disposition", // Expected field name.
      ';', // The separator that should separate fragments of HTTP-field values.
      restinio::http_field_parser::expect("form-data"),
      restinio::http_field_parser::name_value("name"),
      restinio::http_field_parser::name_value("filename"));
if(std::get<0>(result))
{
   assert("file-name" == std::get<1>(result));
   assert("demo.txt" == std::get<2>(result));
   ...
}

or, in the case of C++17 it can look like that:

std::string whole_field = "Content-Disposition: form-data; name=\"file-name\"; filename=\"demo.txt\"";
const auto & [parsed, name, filename] =
   restinio::http_field_parser::try_parse_whole_field(whole_field,
      "content-disposition", // Expected field name.
      ';', // The separator that should separate fragments of HTTP-field values.
      restinio::http_field_parser::expect("form-data"),
      restinio::http_field_parser::name_value("name"),
      restinio::http_field_parser::name_value("filename"));
if(parsed)
{
   assert("file-name" == name);
   assert("demo.txt" == filename);
   ...
}

Personally I would prefer a version that returns a tuple. But it will be great to hear other opinions.

So any constructive suggestions are welcome Please tell me what you think.

Hello,

We want to test restinio under Windows 10. Apparently, we can use the vcpkg tool. We have installed all the necessary libraries but we can't build our example. Do you have an example CMakeLists.txt file that works?
Here is the error message :

CMake Error at C:/Users/jm/vcpkg/scripts/buildsystems/vcpkg.cmake:141 (_add_executable):
  Target "helloworld" links to target "unofficial::http_parser::http_parser"
  but the target was not found.  Perhaps a find_package() call is missing for
  an IMPORTED target, or an ALIAS target is missing?
Call Stack (most recent call first):
  CMakeLists.txt:5 (add_executable)

And the contents of the CMakeLists.txt file :

cmake_minimum_required(VERSION 3.8.0)
project (helloworld)
add_executable(helloworld helloworld.cpp)
find_package(restinio REQUIRED)
find_package(fmt REQUIRED)
find_path(RESTINIO_INCLUDE_DIR restinio/all.hpp)
include_directories(${RESTINIO_INCLUDE_DIR})
target_link_libraries(helloworld PRIVATE restinio::restinio)

#include <restinio/all.hpp>

int main()
{
    restinio::run(
        restinio::on_this_thread()
        .port(8080)
        .address("localhost")
        .request_handler([](auto req) {
            return req->create_response().set_body("Hello, World!").done();
        }));
    return 0;
}

Feature request.

Building a server that has to support multiple restful APIs it's nice to keep things modular. It would be nice to be able to run a server built with one request handler that dispatches to the other routers to handle individual APIs. This would allow for better separation, as apposed to having a "god class" with knowledge of everything.

Attempt at an example:

namespace rr = restinio::router;
using router_t = rr::express_router_t<>;

auto getFooApiRouter(){
   auto router = std::make_unique< router_t >();
   router->http_get(  "/:version" , &fooVersion );
   router->http_get(  "/:version/one" , &fooOne );
   router->http_put(  "/:version/one" , &fooOne );
   //... etc
   return router;
}

auto getBarApiRouter(){
   auto router = std::make_unique< router_t >();
   router->http_get(  "/:version" , &barVersion );
   router->http_get(  "/:version/thing" , &barThing );
   router->http_put(  "/:version/buzz" , &barBuzz );
   //... etc
   return router;
}

auto router = std::make_unique< router_t >();

router->http_get( "/", &rootPage );
router->add_routes(  "/foo" , getFooApiRouter() );
router->add_routes(  "/bar" , getBarApiRouter() );

restinio::run(
	restinio::on_this_thread< traits_t >()
	.address( "localhost" )
	.request_handler( router ) );

I wanted to see if it was possible at all, tried this prince-chrismc/restinio@6a9edbd7828ff75d61c7b52c2d71ca62d835222f and prince-chrismc/restinio@e9b74ec8aafef9cfad5fd865d15373bcf7900a0b, which seems like a sufficient solution to preferred concept. The individual API routers, would need the full path but I could live with that.

Hello,

We are testing Restinio 0.4.1 and we have a problem with the express_router_tutorial example.
When we use the URL http://localhost:8080/many/2018.06.01, the route used is non_matched_request_handler.
For the other requests, everything works correctly.

Jean-Marc

Title says it

Hi,

Browsers won't send the URL fragment but there is fragment() method in restinio::http_request_header_t struct.

As far as i know, Java servlet or similar libraries does not have a such method.

What is the use case of this method?

Thanks

Hi,

From the Restinio doc.:

Request handler

Body can be accesed by request_t::body() function which returns a reference to std::string.

Why do you preferred std::string instead of a suitable stream type? I couldn't figure out what happen if http client sends request body which does not fit in the memory?

Got this error that I never saw before while integrating RESTinio into our Autotools GNU build system of the ring-daemon and it seems related to this https://github.com/citra-emu/citra/issues/4766 however, this did not fix it but removing the ; at the specified lines fixed this errors, any insights on this matter?

make[1]: Entering directory '/home/n0t/jami/ring-daemon/contrib/build/opendht'
Making install in src
make[2]: Entering directory '/home/n0t/jami/ring-daemon/contrib/build/opendht/src'
/bin/sh ../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I..  -I/home/n0t/jami/ring-daemon/contrib/x86_64-pc-linux-gnu/include -fPIC -DOPENDHT_JSONCPP=1 -DOPENDHT_PROXY_HTTP_PARSER_FORK -DOPENDHT_PROXY_SERVER -DOPENDHT_PROXY_CLIENT -DOPENDHT_PUSH_NOTIFICATIONS -I../include/opendht    -I/home/n0t/jami/ring-daemon/contrib/x86_64-pc-linux-gnu/include -fPIC -DOPENDHT_JSONCPP=1 -DOPENDHT_PROXY_HTTP_PARSER_FORK -DOPENDHT_PROXY_SERVER -DOPENDHT_PROXY_CLIENT -DOPENDHT_PUSH_NOTIFICATIONS  -I/home/n0t/jami/ring-daemon/contrib/x86_64-pc-linux-gnu/include -DNDEBUG=1 -O3 -std=c++14 -fPIC -O3 -pedantic-errors -DMSGPACK_DISABLE_LEGACY_NIL -DMSGPACK_DISABLE_LEGACY_CONVERT -c -o libopendht_la-dhtrunner.lo `test -f 'dhtrunner.cpp' || echo './'`dhtrunner.cpp
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -I/home/n0t/jami/ring-daemon/contrib/x86_64-pc-linux-gnu/include -fPIC -DOPENDHT_JSONCPP=1 -DOPENDHT_PROXY_HTTP_PARSER_FORK -DOPENDHT_PROXY_SERVER -DOPENDHT_PROXY_CLIENT -DOPENDHT_PUSH_NOTIFICATIONS -I../include/opendht -I/home/n0t/jami/ring-daemon/contrib/x86_64-pc-linux-gnu/include -fPIC -DOPENDHT_JSONCPP=1 -DOPENDHT_PROXY_HTTP_PARSER_FORK -DOPENDHT_PROXY_SERVER -DOPENDHT_PROXY_CLIENT -DOPENDHT_PUSH_NOTIFICATIONS -I/home/n0t/jami/ring-daemon/contrib/x86_64-pc-linux-gnu/include -DNDEBUG=1 -O3 -std=c++14 -fPIC -O3 -pedantic-errors -DMSGPACK_DISABLE_LEGACY_NIL -DMSGPACK_DISABLE_LEGACY_CONVERT -c dhtrunner.cpp  -fPIC -DPIC -o libopendht_la-dhtrunner.o
In file included from /home/n0t/jami/ring-daemon/contrib/x86_64-pc-linux-gnu/include/restinio/http_headers.hpp:16,
                 from /home/n0t/jami/ring-daemon/contrib/x86_64-pc-linux-gnu/include/restinio/request_handler.hpp:15,
                 from /home/n0t/jami/ring-daemon/contrib/x86_64-pc-linux-gnu/include/restinio/settings.hpp:18,
                 from /home/n0t/jami/ring-daemon/contrib/x86_64-pc-linux-gnu/include/restinio/all.hpp:12,
                 from ../include/opendht/http.h:28,
                 from ../include/opendht/dht_proxy_client.h:30,
                 from dhtrunner.cpp:27:
/home/n0t/jami/ring-daemon/contrib/x86_64-pc-linux-gnu/include/fmt/format.h:3475:55: error: ISO C++ did not adopt string literal operator templates taking an argument pack of characters [-Wpedantic]
 3475 | FMT_CONSTEXPR internal::udl_formatter<Char, CHARS...> operator""_format() {
      |                                                       ^~~~~~~~
In file included from /home/n0t/jami/ring-daemon/contrib/x86_64-pc-linux-gnu/include/restinio/request_handler.hpp:15,
                 from /home/n0t/jami/ring-daemon/contrib/x86_64-pc-linux-gnu/include/restinio/settings.hpp:18,
                 from /home/n0t/jami/ring-daemon/contrib/x86_64-pc-linux-gnu/include/restinio/all.hpp:12,
                 from ../include/opendht/http.h:28,
                 from ../include/opendht/dht_proxy_client.h:30,
                 from dhtrunner.cpp:27:
/home/n0t/jami/ring-daemon/contrib/x86_64-pc-linux-gnu/include/restinio/http_headers.hpp:38:2: error: extra ‘;’ [-Wpedantic]
   38 | };
      |  ^
In file included from /home/n0t/jami/ring-daemon/contrib/x86_64-pc-linux-gnu/include/restinio/uri_helpers.hpp:18,
                 from /home/n0t/jami/ring-daemon/contrib/x86_64-pc-linux-gnu/include/restinio/all.hpp:21,
                 from ../include/opendht/http.h:28,
                 from ../include/opendht/dht_proxy_client.h:30,
                 from dhtrunner.cpp:27:
/home/n0t/jami/ring-daemon/contrib/x86_64-pc-linux-gnu/include/restinio/utils/percent_encoding.hpp:83:2: error: extra ‘;’ [-Wpedantic]
   83 | };
      |  ^
/home/n0t/jami/ring-daemon/contrib/x86_64-pc-linux-gnu/include/restinio/utils/percent_encoding.hpp:109:2: error: extra ‘;’ [-Wpedantic]
  109 | };
      |  ^
make[2]: *** [Makefile:904: libopendht_la-dhtrunner.lo] Error 1
make[2]: Leaving directory '/home/n0t/jami/ring-daemon/contrib/build/opendht/src'
make[1]: *** [Makefile:515: install-recursive] Error 1
make[1]: Leaving directory '/home/n0t/jami/ring-daemon/contrib/build/opendht'
make: *** [../../contrib/src/opendht/rules.mak:42: .opendht] Error 2

Hello reading code through a tool let me know about an alert in code, i share it with you:

missing const ref parameters or std::move:

(restinio/impl/connection_settings.hpp)

Although a query string usually contains key value pairs, it is perfectly legal to have a query like this:

http://example.com/api/foo?1234

This is used for example to implement tracking beacons.
The current parse_query() implementation throws an exception given this query.
I am not sure how to fix this. parse_query returns a map of key value pairs. So it cannot deal with this in its current form.
In my particular use case I would simply like to ignore such queries. So maybe there could be an option for parse_query similar to express_router_t::http_get's option to enable strict parse rules.

Another potential problem with parse_query is that it only accepts & as a separator. But W3C also recommends that servers support semicolon separators.

Small suggestion - It would be good if instead of:

auto status =restinio::status_bad_request();
status.reason_phrase("Only Basic authorization currently supported");

we could do this:

auto status =restinio::status_bad_request("Only Basic authorization currently supported");

However I also read that HTTP/2 does away with text reason phrases: httpwg/http2-spec#202 ... so if you are planning http/2 support soon then it might not be worth it.

Your code is absolutely awesome, its really what I need.And I found that it seems no http client module or not documented. How to communicate with other servers? Do I need another http client library?

The open_file() function in sendfile_defs_win.h uses CreateFile to open the requested file path. This does not work if an application is compiled in unicode mode (i.e. UNICODE is defined). In this case CreateFile will be defined as CreateFileW and its first parameter will effectively be a wchar_t*.

There are two options to fix this.
i) Just use CreateFileA (the ASCII version of the function) in either unicode and non-unicode mode.
ii) Declare that the const char * parameter is a utf8-encoded string and convert it into utf16 in unicode mode.

Option i) is simple to implement but restricts sendfile to ASCII paths.
Option ii) is what one would actually want but needs a conversion from utf8 to utf16 (this code needs come from somewhere).

Hello,

I installed restinio with vcpkg. I test your WebSocket example with the Chrome module "Simple WebSocket Client".

I can open the connection but when I close it, I get these errors :

$ LANG=C ./websocket 
[2019-02-26 10:17:47.081] TRACE: starting server on 127.0.0.1:8080
[2019-02-26 10:17:47.081]  INFO: init accept #0
[2019-02-26 10:17:47.081]  INFO: server started on 127.0.0.1:8080
[2019-02-26 10:17:56.707] TRACE: accept connection from 127.0.0.1:41250 on socket #0
[2019-02-26 10:17:56.707] TRACE: [connection:1] start connection with 127.0.0.1:41250
[2019-02-26 10:17:56.707] TRACE: [connection:1] start waiting for request
[2019-02-26 10:17:56.707] TRACE: [connection:1] continue reading request
[2019-02-26 10:17:56.707] TRACE: [connection:1] received 702 bytes
[2019-02-26 10:17:56.708] TRACE: [connection:1] upgrade request received: GET /chat/; Upgrade: 'websocket';
[2019-02-26 10:17:56.708]  INFO: [connection:1] handle upgrade request (#0): GET /chat/
[2019-02-26 10:17:56.708] TRACE: [connection:1] move socket to [ws_connection:1]
[2019-02-26 10:17:56.708] TRACE: [ws_connection:1] start connection with 127.0.0.1:41250
[2019-02-26 10:17:56.708] TRACE: [ws_connection:1] start next write group, size: 1
[2019-02-26 10:17:56.708] TRACE: [ws_connection:1] sending data with buf count: 1, total size: 129
[2019-02-26 10:17:56.708] TRACE: [ws_connection:1] start reading header
[2019-02-26 10:17:56.708] TRACE: [ws_connection:1] continue reading message
[2019-02-26 10:17:56.708] TRACE: [connection:1] destructor called
[2019-02-26 10:17:56.708] TRACE: [ws_connection:1] outgoing data was sent: 129 bytes
[2019-02-26 10:17:56.708] TRACE: [ws_connection:1] finishing current write group
[2019-02-26 10:18:07.562] TRACE: [ws_connection:1] received 6 bytes
[2019-02-26 10:18:07.562] TRACE: [ws_connection:1] start handling connection_close_frame (0x8)
websocket: /donnees/jm/vcpkg/installed/x64-linux/include/restinio/third_party/string-view-lite/string_view.hpp:492: constexpr const CharT& nonstd::sv_lite::basic_string_view<CharT, Traits>::operator[](nonstd::sv_lite::basic_string_view<CharT, Traits>::size_type) const [with CharT = char; Traits = std::char_traits<char>; nonstd::sv_lite::basic_string_view<CharT, Traits>::const_reference = const char&; nonstd::sv_lite::basic_string_view<CharT, Traits>::size_type = long unsigned int]: Assertion `pos < size()' failed.
Abandon (core dumped)

Jean-Marc

Hi,

I am trying to integrate RESTinio to Java platform. It works quite well for sync handler. I would like to add continuation and asynchronous feature. I know that there is a timer based async handler but could you please help me to how to add asleep & awake logic.

Thanks

Hello,

I work with your example https://github.com/Stiffstream/restinio/blob/master/dev/sample/websocket/main.cpp

I want to send (for example every 30 seconds) ping-pong frames from my websocket server to clients to keep connections open or remove invalid connections from the registry.

What is the right method with Restinio?

Thanks,

Jean-Marc

I would like to use RESTinio's logger inside of a request handler lambda function. Is there any way to do so?
I have had a looked through the examples but could not find anything where things are actually written to the logger.

Let's say I want to perform a standard rfc2818_verification using the tls_socket adapter implementation, how would one be able to set_verify_callback for this purpose using thessl::stream containing our socket if that interface is not exposed (private)?

I've been following RESTinio a bit as I plan to use it for a game and also because I've been making a Macports portfile. Initially, the cmake file was so broken I had to implement a custom install routine in the portfile (which worked but is a PITA to maintain). As such, I was looking into the changes introduced in recent versions and I noticed that in fact most of the issues appear to have been fixed.

However, SObjectizer is still always being expected/included by CMake. Shouldn't this be wrapped in a conditional for tests, samples or somesuch?

Building minimalistic hello world sample results in errors like:

message_builders.hpp(182): error C2719: 'body': formal parameter with requested alignment of 8 won't be aligned

This is because buffer_storage_t is decorated with alignas( std::max_align_t ). It seems that there is no easy way to bypass this in VS 2015, which does not support aligned function arguments.

Commenting out alignas( std::max_align_t ) in Lines 86 and 211 in buffers.hpp, the sample works for VS 2015. I am however unsure about the impact of removing the alignment requirement.

Is conditional compilation to handle VS2015 acceptable in this case? What is your opinion? Thanks.

I'm on Windows 10 using CLion and Visual Studio 2019 compiler with CMake. Latest version of all of these tools. Here are my CMake flags:

-DCMAKE_TOOLCHAIN_FILE=D:/cpp/vcpkg/scripts/buildsystems/vcpkg.cmake
-DRESTINIO_INSTALL=ON
-DRESTINIO_TEST=ON
-DRESTINIO_SAMPLE=ON
-DRESTINIO_INSTALL_SAMPLES=ON
-DRESTINIO_BENCH=ON
-DRESTINIO_INSTALL_BENCHES=ON
-DRESTINIO_FIND_DEPS=OFF

All the dependencies were downloaded using WSL since Mxx_ru kept on giving weird errors both on Windows CMD and MSYS.

Now I could build all the of the modules, run the samples and even create projects based on Restinio. But there are three specific targets:

add_subdirectory(express_router_pcre_bench)
add_subdirectory(express_router_pcre2_bench)
add_subdirectory(express_router_boost_regex_bench)

That I cannot build. They give the following errors:

D:\cpp\restinio\dev\test\router\express_router_pcre_bench../express_router_bench/main.cpp(83): error C2039: 'http_method_id_t': is not a member of 'restinio'
D:\cpp\vcpkg\installed\x64-windows\include\restinio/router/pcre_regex_engine.hpp(17): note: see declaration of 'restinio'
D:\cpp\restinio\dev\test\router\express_router_pcre_bench../express_router_bench/main.cpp(83): error C3646: 'm_method': unknown override specifier
D:\cpp\restinio\dev\test\router\express_router_pcre_bench../express_router_bench/main.cpp(83): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
D:\cpp\restinio\dev\test\router\express_router_pcre_bench../express_router_bench/main.cpp(111): error C2039: 'http_method_id_t': is not a member of 'restinio'
D:\cpp\vcpkg\installed\x64-windows\include\restinio/router/pcre_regex_engine.hpp(17): note: see declaration of 'restinio'
D:\cpp\restinio\dev\test\router\express_router_pcre_bench../express_router_bench/main.cpp(111): error C2065: 'http_method_id_t': undeclared identifier
D:\cpp\restinio\dev\test\router\express_router_pcre_bench../express_router_bench/main.cpp(111): error C2146: syntax error: missing ';' before identifier 'mm'
D:\cpp\restinio\dev\test\router\express_router_pcre_bench../express_router_bench/main.cpp(111): error C2065: 'mm': undeclared identifier
D:\cpp\restinio\dev\test\router\express_router_pcre_bench../express_router_bench/main.cpp(112): error C2039: 'http_method_id_t': is not a member of 'restinio'
D:\cpp\vcpkg\installed\x64-windows\include\restinio/router/pcre_regex_engine.hpp(17): note: see declaration of 'restinio'
D:\cpp\restinio\dev\test\router\express_router_pcre_bench../express_router_bench/main.cpp(112): error C2065: 'http_method_id_t': undeclared identifier
D:\cpp\restinio\dev\test\router\express_router_pcre_bench../express_router_bench/main.cpp(112): error C2146: syntax error: missing ';' before identifier 'method'
D:\cpp\restinio\dev\test\router\express_router_pcre_bench../express_router_bench/main.cpp(112): error C2065: 'method': undeclared identifier
D:\cpp\restinio\dev\test\router\express_router_pcre_bench../express_router_bench/main.cpp(114): error C2065: 'mm': undeclared identifier
D:\cpp\restinio\dev\test\router\express_router_pcre_bench../express_router_bench/main.cpp(114): error C2039: 'default_http_methods_t': is not a member of 'restinio'
D:\cpp\vcpkg\installed\x64-windows\include\restinio/router/pcre_regex_engine.hpp(17): note: see declaration of 'restinio'
D:\cpp\restinio\dev\test\router\express_router_pcre_bench../express_router_bench/main.cpp(114): error C3083: 'default_http_methods_t': the symbol to the left of a '::' must be a type
D:\cpp\restinio\dev\test\router\express_router_pcre_bench../express_router_bench/main.cpp(114): error C2039: 'from_nodejs': is not a member of 'restinio'
D:\cpp\vcpkg\installed\x64-windows\include\restinio/router/pcre_regex_engine.hpp(17): note: see declaration of 'restinio'
D:\cpp\restinio\dev\test\router\express_router_pcre_bench../express_router_bench/main.cpp(114): error C3861: 'from_nodejs': identifier not found
D:\cpp\restinio\dev\test\router\express_router_pcre_bench../express_router_bench/main.cpp(117): error C2065: 'mm': undeclared identifier
D:\cpp\restinio\dev\test\router\express_router_pcre_bench../express_router_bench/main.cpp(119): error C2065: 'method': undeclared identifier
D:\cpp\restinio\dev\test\router\express_router_pcre_bench../express_router_bench/main.cpp(119): error C2065: 'mm': undeclared identifier
D:\cpp\restinio\dev\test\router\express_router_pcre_bench../express_router_bench/main.cpp(124): error C2065: 'method': undeclared identifier
D:\cpp\restinio\dev\test\router\express_router_pcre_bench../express_router_bench/main.cpp(129): error C2039: 'm_method': is not a member of 'route_line_t'
D:\cpp\restinio\dev\test\router\express_router_pcre_bench../express_router_bench/main.cpp(81): note: see declaration of 'route_line_t'
D:\cpp\restinio\dev\test\router\express_router_pcre_bench../express_router_bench/main.cpp(129): error C2065: 'method': undeclared identifier
D:\cpp\restinio\dev\test\router\express_router_pcre_bench../express_router_bench/main.cpp(152): error C2039: 'm_method': is not a member of 'route_line_t'
D:\cpp\restinio\dev\test\router\express_router_pcre_bench../express_router_bench/main.cpp(81): note: see declaration of 'route_line_t'
D:\cpp\restinio\dev\test\router\express_router_pcre_bench../express_router_bench/main.cpp(156): error C2039: 'm_method': is not a member of 'route_line_t'
D:\cpp\restinio\dev\test\router\express_router_pcre_bench../express_router_bench/main.cpp(81): note: see declaration of 'route_line_t'

Any help would be appreciated.

Hello,
I am asked if it is possible to use amqp-cpp with restinio. As I don't know, I ask the question to the developers of restinio:)

Do you think this is possible easily enough?

Jean-Marc

Hi devs,

We have one last operation on the server called LISTEN which keeps a connection open at the maximum defined delay in the settings handle_request_timeout. Everything is good and I can send chunks and close it using the response with response.connection_close(). I can find its ID with request->connection_id() which returns a connection_id_t.

However, we need to be able to check the connection state (open, closed) using the connection_id_t or any other object in a different thread to close the connections gracefully in case we shutdown the server as well as use this observer thread to close the active listeners on the OpenDHT thread.

Thank you for the heads up! 🚀
Seva

Hi,

I am using restinio on a server receiving a lot of connections. I saw that sometimes, the server is hanging and doesn't accepts any new connections (but current connections continue to work without issues).

Just before hanging, this is what I see in the logs:

an exception in 'accept_current_connection': remote_endpoint: Transport endpoint is not connected

So, my assumption is that

I'm currently investigating, but if you have any clue, it will be appreciated :)

Hello i'm looking the sample,

I try to retrieve params from a getter url: /api/restcall?option1=value&option2=value

How to retrieve option1 and option2 in the code ?

When using sendfile in a response the file being sent will be closed two times (with the second attempt obviously failing).

The first closing (i.e. a call to ::CloseHandle) is made when the file sending has finished and the writable_item_t wrapping the file is destroyed. This is triggered from connection_t::finish_handling_current_write_ctx.

After this the a send_file_operation_runner_t is destroyed. This class has a member m_file_handle of type asio::windows::random_access_handle. The problem is now that this member also got the file handle when it was constructed. And it wants to close the file in its destruction as well.

I don't really understand the code so I cannot tell which part is supposed to close the file handle. But asio::windows::random_access_handle looks like it is designed to take ownership of the handle. So to fix the problem file_descriptor_holder_t inside send_file_t needs to be released explicitly somewhere to prevent its destructor from closing the handle prematurely.

In golang i simply do:

router.ServeFiles("/logs/*filepath", config.GConfig.LogsPath)

ServeFiles serves files from the given file system root. The path must end with "/*filepath", files are then served from the local path /defined/root/dir/*filepath. For example if root is "/etc" and *filepath is "passwd", the local file "/etc/passwd" would be served. Internally a http.FileServer is used, therefore http.NotFound is used instead of the Router's NotFound handler.

(i want to serve the logs file of my server so we can read it online)

i dont find the equivalent with restinio in the documentation :'(

Right here you can see the change:
7ab2fd3#diff-014dfa4235c0e34420ccb97dba6181d7R97

I had to comment this macro out for the compilation to proceed. It looks like the vector header that ships with Visual Studio 2019 defines erase() as a noexcept.

Hello i followed the example from the markdown for creating a request, but now i got this warning from LGTM security services:

Can we pass the router as const reference, or it's always as copy (like your example)

Hi,

I am unable to compile restinio.

Here is my configuration:

• Windows 10
• Visual Studio 2017 - 15.9.13

c:\tools\vcpkg\installed\x64-windows-static\include\restinio\sendfile.hpp(321): error C2589: '(': illegal token on right side of '::' 
c:\tools\vcpkg\installed\x64-windows-static\include\restinio\sendfile.hpp(321): error C2059: syntax error: ')' 
c:\tools\vcpkg\installed\x64-windows-static\include\restinio\sendfile.hpp(346): error C2589: '(': illegal token on right side of '::' 
c:\tools\vcpkg\installed\x64-windows-static\include\restinio\sendfile.hpp(346): error C2059: syntax error: ')' 

There was a discussion after one of the articles on Habr.com (note: it's in Russian) about the necessity of such a thing as "tls_inspector" in addition to "ip_blocker" and "state_listener" (both added in RESTinio v.0.5). I've tried to implement such "tls_inspector". The first version can be found in "0.5-dev--tls-inspector" branch on BitBucket.

An example of very simple usage of tls_inspector can be found here. This example shows how to extract commonName from client's certificate and use this value as a user name to control access of the user to resources. A user "Alice" has access to all URLs, but user "Bob" has access only to /all and haven't access to /limited.

This implementation works, but I don't like it. It seems that several significant flaws should be fixed before "tls_inspector" will become a part of RESTinio.

The first moment I don't like is the return of inspection_result_t from tls_inspector's inspect method.

It seems logical that tls_inspector can check some client credentials or TLS parameters and allow or deny the connection. But, maybe Asio's set_verify_callback is more appropriate for that purpose?

The second moment can be seen in the implementation of a simple example of tls_inspector usage (this example was already mentioned above). If a tls_inspector has to store connection-related information somewhere (like in user_connections_t container in the example) then there should be a way to say tls_inspector that a connection is gone (closed or upgraded to WebSocket connection).

It seems that tls_inspector and state_listener will be a single thing most of the time. If so, why do we need a separate entity for tls_inspector? What if state_listener is enough and all that we need is the way to access TLS-socket from state_listener's state_changed method?

There can be another point of view on that issue. Maybe there should be a way to store some user-provided data inside a connection with a possibility to extract that via requiest_handle_t. It will allow doing things like:

class my_tls_inspector {
   ...
   restinio::tls_inspector::inspection_result_t
   inspect(restinio::tls_inspector::incoming_info_t & info) noexcept {
      auto name = extract_user_name(info);
      info.add_connection_user_data<user_name_t>(some_id, user_name_t{name});
   }
};
...
auto limited_resource_request_handler(
   restinio::request_handle_t & req ) {
   ...
   auto & name = req->query_connection_user_data<user_name_t>(some_id);
   ...
}

That approach has a benefit: a user data associated with a connection will automatically be destroyed when the connection is gone (closed or upgraded to WebSocket). So we can create a tls_inspector as a separate entity not related to state_listener.

That approach can also be used for different purposes. For example, if a client uses "keep-alive" for a connection and issues several requests via that connection, then connection_user_data allows storing some connection- or user-specific information without a need to implement own container for that purpose.

It'll be handy if someone can tell his or her thoughts on that topic. I hope someone finds time and desire to answer these questions or share their own opinion (any suggestions are welcome):

Does tls_inspector look like a valuable addition to RESTinio?

If does, should tls_inspector allow or deny new connections?

Is it a good idea to have something like connection-related user data associated with a connection?

When trying to create a tls-enabled server the MSVC compiler complains about this:

...restinio/impl/tls_socket.hpp(93): error C3779: 'restinio::impl::tls_socket_t::lowest_layer': a function that returns 'auto' cannot be used before it is defined
...restinio/impl/tls_socket.hpp(60): note: see declaration of 'restinio::impl::tls_socket_t::lowest_layer'
...restinio/impl/tls_socket.hpp(93): note: This diagnostic occurred in the compiler generated function 'void restinio::impl::tls_socket_t::cancel(Args &&...)'
...

Simply not using lowest_layer() inside tls_socket_t's methods and instead replacing it with m_socket->lowest_layer() fixes the problem.

Hey,

Great work with Restinio, greatly simplifies my work :)

On question:
I'd like to log all requests that come in, a little like the apache access log.
I can set a logging trait, and I already have that. But that uses the trace function and is very verbose.

I'd just like to output a simple log, without having to call it in every handler.
Is that possible?

Thanks
Dom

Is the IPV6 supported or is it planned to be supported in a near future?

Thank you!

If one is using {fmt} to format a response to be sent by restinio, it would be great if needed_storage_max_size were big enough to accommodate a fmt::basic_memory_buffer<char, 1> (at least 40 bytes on x64), and then we could send the {fmt} buffer directly without memory copying.