jeremycw / httpserver.h Goto Github PK

View Code? Open in Web Editor NEW

1.8K 1.8K 141.0 298 KB

Single header library for writing non-blocking HTTP servers in C

License: MIT License

Makefile 0.25% C 94.23% Shell 1.15% CMake 1.10% M4 0.40% Ragel 2.86%

httpserver.h's People

Contributors

Stargazers

Watchers

Forkers

chronologist89 migzone bhatti danbruder yut148 jhines2k7 petterla nghiatranuit harlowja igordzreyev seekaddo zhou3968322 cidermole drawcode allenxuxu jimjag smazumder05 ubuntu-repo jburon affix mbrukman tisma skyformat99 dendisuhubdy manniru vladimirfomene hoangpq ramasai1 daxaar rrozestw poooooooooke ahmah2009 mnaza hhy5277 kyungkoo daodaoliang razvanst 0xaf jgowdy chorkei dontepanlin icakir javisantana junxie6 mkll hide5stm amasad blissseven chenlucan s-you studiovc markusvaikjarv nexround pjreid819 ioppermann asherho progyuan freezesoul 5l1v3r1 zeta1999 gx1727 plsmaop zhouzu hl0521 shunchuan sternenseemann xiebaiyuan ctyb wshhja paulstuart jungkwangho leoagle xixiha0 zabrane yashade weinbery userhdisk developercyrus smdhj-cell kwon-young-hu 0xcccccccccccc mazhuwo xunmenging hanliinter zhongqingyang shengzhe8688 rafidavidov satchelwu schlang alvendarthy blueszeng wisebuilder amdfansheng hoathienvu8x adas-eye suyjuris lwjjhy wanjiadenghuo111 linghhan ayouboss

httpserver.h's Issues

SO_REUSEPORT is not defined on Ubuntu

When compiling on WSL2 Ubuntu I get the following error:

nick:~/dev/_lib/httpserver.h master$ make
cc -O3 -std=c99 -Wall -Wextra -Werror test/main.c -o http-server
In file included from test/main.c:2:
test/../httpserver.h: In function ‘http_listen’:
test/../httpserver.h:1223:40: error: ‘SO_REUSEPORT’ undeclared (first use in this function); did you mean ‘SO_REUSEADDR’?
 1223 |   setsockopt(serv->socket, SOL_SOCKET, SO_REUSEPORT, &flag, sizeof(flag));
      |                                        ^~~~~~~~~~~~
      |                                        SO_REUSEADDR
test/../httpserver.h:1223:40: note: each undeclared identifier is reported only once for each function it appears in
make: *** [Makefile:24: http-server] Error 1

Should this be replaced with SO_REUSEADDR? I'm not sure if they are the same

Test crashes on OpenBSD

OS: OpenBSD 7.2 snapshot amd64
Compiler: GCC 11.2.0

I compiled and started test/main.c first, then ran test/run.
All tests before Chunked Request close: (expect empty) are OK, but the test server crashes when running this test.

Output from test/run:

Small Response Body:
Hello, World!

Empty Response:


Echo Body:
Echo test

Get Header:
localhost:8080

Request Body larger than max in mem size:
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 48.0M  100 24.0M  100 24.0M  31.5M  31.5M --:--:-- --:--:-- --:--:-- 63.0M


Chunked Response:
Hello, World!Hello, World!Hello, World!

Chunked Response keep-alive:
Hello, World!Hello, World!Hello, World!Hello, World!Hello, World!Hello, World!

Chunked Response close:
Hello, World!Hello, World!Hello, World!Hello, World!Hello, World!Hello, World!

Chunked Request keep-alive: (expect empty)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  512k  100  256k  100  256k  12.5M  12.5M --:--:-- --:--:-- --:--:-- 26.3M
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  512k  100  256k  100  256k  21.9M  21.9M --:--:-- --:--:-- --:--:-- 45.4M


Chunked Request close: (expect empty)
(The test server crashed here)

I try to figure it out with gdb. Here is what i got:

☁  test [master] ⚡  egdb ./test             
GNU gdb (GDB) 9.2
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-unknown-openbsd7.1".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./test...
(gdb) r
Starting program: /home/niko/src/httpserver.h/test/test 

(Reaching test "chunked Request close: (expect empty)")
Program received signal SIGABRT, Aborted.
thrkill () at /tmp/-:3
3	/tmp/-: No such file or directory.
(gdb) bt
#0  thrkill () at /tmp/-:3
#1  0x000000a6d65096ae in _libc_abort () at /usr/src/lib/libc/stdlib/abort.c:51
#2  0x000000a6d6511aaf in memcpy (dst0=<optimized out>, src0=<optimized out>, length=<optimized out>) at /usr/src/lib/libc/string/memcpy.c:74
#3  0x000000a401d319ce in hs_stream_shift (stream=0xa67da65318) at ./../httpserver.h:824
#4  0x000000a401d32553 in http_parse (parser=0xa67da65348, stream=0xa67da65318) at ./../httpserver.h:983
#5  0x000000a401d331c9 in hs_read_and_process_request (request=0xa67da65300) at ./../httpserver.h:1121
#6  0x000000a401d3341a in http_session (request=0xa67da65300) at ./../httpserver.h:1164
#7  0x000000a401d33608 in hs_session_io_cb (ev=0x7f7ffffd1cd0) at ./../httpserver.h:1578
#8  0x000000a401d34f03 in http_server_listen_addr (serv=0xa67da96d00, ipaddr=0x0) at ./../httpserver.h:1604
#9  0x000000a401d34f43 in http_server_listen (serv=0xa67da96d00) at ./../httpserver.h:1611
#10 0x000000a401d356bb in main () at main.c:119

(gdb) frame #2
Invalid character '#' in expression.

(gdb) frame 2
#2  0x00000b4cfcc73aaf in memcpy (dst0=<optimized out>, src0=<optimized out>, length=<optimized out>) at /usr/src/lib/libc/string/memcpy.c:74
74	/usr/src/lib/libc/string/memcpy.c: No such file or directory.

(gdb) frame 3
#3  0x00000b4af9ca99ce in hs_stream_shift (stream=0xb4d4ef4eb18) at ./../httpserver.h:824
824	    memcpy(dst, src, bytes);

(gdb) list
819	  if (stream->token.index == stream->anchor) return;
820	  if (stream->token.len > 0) {
821	    char* dst = stream->buf + stream->anchor;
822	    char const* src = stream->buf + stream->token.index;
823	    int bytes = stream->length - stream->token.index;
824	    memcpy(dst, src, bytes);
825	  }
826	  stream->token.index = stream->anchor;
827	  stream->index = stream->token.len + stream->anchor;
828	  stream->length = stream->anchor + stream->token.len;

(gdb) print *dst
$1 = 102 'f'

(gdb) print *src
$2 = -32 '\340'

(gdb) print bytes
$3 = 32710

(gdb) print stream->length
$4 = 32883

(gdb) print stream->token.len
$5 = 32710

(gdb) frame 4
#4  0x00000b4af9caa553 in http_parse (parser=0xb4d4ef4eb48, stream=0xb4d4ef4eb18) at ./../httpserver.h:983
983	  if (parser->state == CB) hs_stream_shift(stream);

(gdb) list
978	    parser->state = to;
979	    http_token_t emitted = hs_transition_action(parser, stream, c, from, to);
980	    hs_stream_consume(stream);
981	    if (emitted.type != HS_TOK_NONE) return emitted;
982	  }
983	  if (parser->state == CB) hs_stream_shift(stream);
984	  token = hs_meta_emit(parser);
985	  http_token_t current = hs_stream_current_token(stream);
986	  if (
987	    current.type != HS_TOK_CHUNK_BODY &&

(gdb) print *stream
$6 = {
  buf = 0xb4d992cb000 "POST /chunked-req HTTP/1.1\r\nHost: localhost:8080\r\nUser-Agent: curl/7.84.0\r\nAccept: */*\r\nTransfer-Encoding: chunked\r\nContent-Type: application/x-www-form-urlencoded\r\n\r\nfff4\r\n\340\257\240\312\375\317\343\364\253\264<\363Fj}p\252F\207\377\275\n\033$\236\021M"..., total_bytes = 32883, capacity = 65536, length = 32883, index = 32883, anchor = 167, token = {index = 173, len = 32710, type = 6}, flags = 0 '\000'}

(gdb) frame 5
#5  0x00000b4af9cab1c9 in hs_read_and_process_request (request=0xb4d4ef4eb00) at ./../httpserver.h:1121
1121	    token = http_parse(&request->parser, &request->stream);

(gdb) frame 6
#6  0x00000b4af9cab41a in http_session (request=0xb4d4ef4eb00) at ./../httpserver.h:1164
1164	      hs_read_and_process_request(request);

(gdb) print *request
$7 = {handler = 0xb4af9cab5a0 <hs_session_io_cb>, chunk_cb = 0xb4af9cad140 <chunk_req_cb>, data = 0xb4d4ef56bc0, stream = {
    buf = 0xb4d992cb000 "POST /chunked-req HTTP/1.1\r\nHost: localhost:8080\r\nUser-Agent: curl/7.84.0\r\nAccept: */*\r\nTransfer-Encoding: chunked\r\nContent-Type: application/x-www-form-urlencoded\r\n\r\nfff4\r\n\340\257\240\312\375\317\343\364\253\264<\363Fj}p\252F\207\377\275\n\033$\236\021M"..., total_bytes = 32883, capacity = 65536, length = 32883, index = 32883, anchor = 167, token = {index = 173, len = 32710, type = 6}, flags = 0 '\000'}, parser = {
    content_length = 65524, body_consumed = 32710, match_index = 0, header_count = 5, state = 17 '\021', meta = 11 '\v'}, state = 1, socket = 7, timeout = 20, server = 0xb4d4ef5d500, tokens = {buf = 0xb4d4ef34c00, capacity = 32, 
    size = 14}, flags = 9 '\t'}

(gdb)

Hey -
I've noticed that when I send an empty header value to the parser, the parser will consume the CRLF as whitespace and take the next header's key as a value,
for example, let's say this is our header section :

content-type: application/xml\r\n
host: example.com\r\n
empty: \r\n
empty2:....

When parsing the "empty" header - it will consume the CRLF as whitespace and continue to the next line until it sees "empty2" which will be taken.

This leads to more bugs, another one that I encountered as a result of that is that if there is an odd number of empty headers in the end of the header section - empty body will not get parsed correctly (also, didn't check - but if there is a body it might be counted as header value, not sure about that though)

According to the RFC, it is legitimate to have an empty field as a header value - https://datatracker.ietf.org/doc/html/rfc7230#section-3.2
Also - there are applications that actually require you to have those empty values - like WOPI

Missing Headers Causes Request Timeout

First of all thank you so much for this project, it's amazing!

The hello world server seems to be timing out depending on the headers sent:

Steps to repro

1. Compile and run the server:

#define HTTPSERVER_IMPL
#include "httpserver.h"

#define RESPONSE "Hello, World!"

void handle_request(struct http_request_s* request) {
  struct http_response_s* response = http_response_init();
  http_response_status(response, 200);
  http_response_header(response, "Content-Type", "text/plain");
  http_response_body(response, RESPONSE, sizeof(RESPONSE) - 1);
  http_respond(request, response);
}

int main() {
  struct http_server_s* server = http_server_init(8080, handle_request);
  http_server_listen(server);
}

2. Make a request

With the server running, attempt to send a request to POST http://localhost:8080 with Postman (using the default headers). Notice that going to Headers, clicking "7 hidden" and then un-checking Content-Length: 0 will allow the request to be processed successfully.

Alternatively, you can run the following code in the browser which also will cause the request to timeout:

fetch('http://localhost:8080/', { method: 'POST' }).then(resp => resp.text()).then(console.log);

While this seems to work OK:

fetch('http://localhost:8080/', {
    method: 'POST',
    body: JSON.stringify(null),
  }).then(resp => resp.text()).then(console.log);

Note that removing body or making body: "" also will cause a timeout.

Let me know if there's anything I should be doing to fix this. Thanks!

Example not compiling

$ env LANG=C gcc q.c
In file included from q.c:2:
httpserver.h: In function 'http_listen':
httpserver.h:1223:40: error: 'SO_REUSEPORT' undeclared (first use in this function); did you mean 'SO_REUSEADDR'?
 1223 |   setsockopt(serv->socket, SOL_SOCKET, SO_REUSEPORT, &flag, sizeof(flag));
      |                                        ^~~~~~~~~~~~
      |                                        SO_REUSEADDR
httpserver.h:1223:40: note: each undeclared identifier is reported only once for each function it appears in

glibc: 2.33
gcc: 10.2.0
This #43 PR resolves issue

Option to break the epoll main loop

Hello,

In order to exit from the server cleaning the way, may be doing this change in the line 1700:

while (1) {

change to

int nev = 0;
while (1) {
    if (nev != -1) break;
    [delete int from the next line]

To quit the loop only it is needed:

int res = epoll_ctl (server->loop, EPOLL_CTL_DEL, 1, NULL);
close (server->loop);

It can be more stylished but works.

how to use in c++11 project?

I use gcc version 4.8.5 with compile flag -W -Wall -Wfatal-errors -g -std=c99 -std=c++11, build error:

./deps/httpserver.h/jeremycw/httpserver.h: In function ‘void hs_init_session(http_request_t*)’:
./deps/httpserver.h/jeremycw/httpserver.h:1047:35: error: expected primary-expression before ‘)’ token
   session->parser = (http_parser_t){ };
                                   ^
compilation terminated due to -Wfatal-errors.

Please verify the memory leaks bug

void _hs_accept_and_begin_request_cycle(http_server_t *server,
hs_io_cb_t on_client_connection_cb,
hs_io_cb_t on_timer_event_cb) {
http_request_t *request = NULL;
while ((request = hs_server_accept_connection(server, on_client_connection_cb,
on_timer_event_cb))) {
if (server->memused > HTTP_MAX_TOTAL_EST_MEM_USAGE) {
hs_request_respond_error(request, 503, "Service Unavailable",
hs_request_begin_write);
} else {
hs_request_begin_read(request);
}
// ================ added by xiao ================ //
if (request) {
hs_request_terminate_connection(request); // We should free request here.
}
// ================ added by xiao end ================ //
}
}

server crashes with certain image/media payloads

hi! thanks for this server. it is really impressive and helpful.

i'm trying to set it up to send some media data to a device via a POST. i'm able to make an empty file of X bytes and send it successfully:

$ mkfile -n 76369 76369.txt
$ curl -X POST -v --data-binary @76369.txt  "http://192.168.7.58:9000/"
Note: Unnecessary use of -X or --request, POST is already inferred.
*   Trying 192.168.7.58...
* TCP_NODELAY set
* Connected to 192.168.7.58 (192.168.7.58) port 9000 (#0)
> POST / HTTP/1.1
> Host: 192.168.7.58:9000
> User-Agent: curl/7.64.1
> Accept: */*
> Content-Length: 76369
> Content-Type: application/x-www-form-urlencoded
> Expect: 100-continue
> 
* Done waiting for 100-continue
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Date: Mon, 21 Sep 2020 16:53:30 GMT
< Connection: keep-alive
< Content-Type: text/plain
< Content-Length: 13
< 
* Connection #0 to host 192.168.7.58 left intact
Hello, World!* Closing connection 0

however. when i try to do this same request with a .png, .jpeg, .mkv, etc, i will often see failures. below is a file of the same size:

$ curl -X POST -v --data-binary @image.jpg  "http://192.168.7.58:9000/"
Note: Unnecessary use of -X or --request, POST is already inferred.
*   Trying 192.168.7.58...
* TCP_NODELAY set
* Connected to 192.168.7.58 (192.168.7.58) port 9000 (#0)
> POST / HTTP/1.1
> Host: 192.168.7.58:9000
> User-Agent: curl/7.64.1
> Accept: */*
> Content-Length: 76369
> Content-Type: application/x-www-form-urlencoded
> Expect: 100-continue
> 
* Done waiting for 100-continue
* We are completely uploaded and fine
* Recv failure: Connection reset by peer
* Closing connection 0
curl: (56) Recv failure: Connection reset by peer

any thoughts as to why this is happening and how i can fix? maybe a character encoding issue?

thanks!

Apparent bug with http_request_method

When I add a call to http_request_method to the simple example in the readme it seems to return the entire request, as opposed to just the method.

Example code:

#include <stdio.h>
#define HTTPSERVER_IMPL
#include "httpserver.h"

#define RESPONSE "Hello, World!"

void handle_request(struct http_request_s* request) {
    struct http_response_s* response = http_response_init();
    printf("test %s\n", http_request_method(request).buf);
    http_response_status(response, 200);
    http_response_header(response, "Content-Type", "text/plain");
    http_response_body(response, RESPONSE, sizeof(RESPONSE) - 1);
    http_respond(request, response);
}

int main() {
    struct http_server_s* server = http_server_init(8080, handle_request);
    http_server_listen(server);
}

Compile and execute:

$ gcc -o srv main.c
$ ./srv

After hitting in my browser it prints the following to the terminal:

test GET / HTTP/1.1
Host: localhost:8080
Connection: keep-alive
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.122 Safari/537.36 Edg/80.0.361.62
Sec-Fetch-Dest: document
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Sec-Fetch-Site: none
Sec-Fetch-Mode: navigate
Sec-Fetch-User: ?1
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9

Support chunked transfer encoding

https://tools.ietf.org/html/rfc2616#section-3.6.1

Benchmark hardware question

I have a brief question about the kind of hardware you ran the benchmark in the readme on.
I tried the same on my local 2016 era Macbook pro and my avg req/sec are dramatically different.

Am I compiling correctly? Or is this just my laptop being terrible?

Results (mean of 8200 req/sec):

$ ab -k -c 200 -n 100000 http://127.0.0.1:8080/
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:
Server Hostname:        127.0.0.1
Server Port:            8080

Document Path:          /
Document Length:        13 bytes

Concurrency Level:      200
Time taken for tests:   12.165 seconds
Complete requests:      100000
Failed requests:        0
Keep-Alive requests:    100000
Total transferred:      13400000 bytes
HTML transferred:       1300000 bytes
Requests per second:    8220.54 [#/sec] (mean)
Time per request:       24.329 [ms] (mean)
Time per request:       0.122 [ms] (mean, across all concurrent requests)
Transfer rate:          1075.73 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   1.8      0      42
Processing:     1   24   2.0     24      56
Waiting:        1   24   2.0     23      56
Total:          1   24   3.0     24      83

Percentage of the requests served within a certain time (ms)
  50%     24
  66%     24
  75%     24
  80%     25
  90%     27
  95%     29
  98%     30
  99%     31
 100%     83 (longest request)

Is it possible to make thread per route?

Hello, is it possible to make thread per route? if its possible, can you give me example code

Thanks

Memory leak when application is terminated while a keep-alive connection is still open

When dealing with HTTP keep-alive connections in practice, terminating a httpserver.h based application via a SIGTERM handler similar to the one in test/main.c will leak memory as free(http_server) isn't enough to get rid of whatever is allocated per connection.

To reproduce:

valgrind ./http-server &
pid=$!
firefox http://localhost:8080/ # or any other typical browser
kill $pid

I haven't looked into it yet, it'd probably be possible to provide a cleanup function to free everything related to keep-alive connections as well?

How can deal with image file upload?

Hello, everyone

For example:
curl localhost:4000/test -F "filename=@/tmp/xxx.jpg"

In the program code, how can I get this image file?

Improve C++ Compatibility

I currently use some C99 features in the implementation portion. This means that for use in C++ projects you have to compile the file that includes the implementation with --std=c99

I could probably remove the C++ incompatibilities to makes this easier to use in C++ projects.

Potential Null Pointer Dereference

Used throughout, realloc may return a null pointer if there is a failure during allocation. The return value from realloc is never validated prior to use may lead to a null pointer dereference or information disclosure where an attacker may be able to leak information from memory.

tes/run curl not testing Chunked Request close

In line 38, curl needs -H'Connection: close' to test correct. If not, it's the same test as keep-alive.

echo "\n\nChunked Request close: (expect empty)"
curl -H'Expect:' -H'Connection: close' -H'Transfer-Encoding: chunked' -XPOST --data-binary @test.dat -o r3.dat http://localhost:8080/chunked-req -o r4.dat http://localhos

Add ability to poll server in a loop instead of a blocking listen call

This will allow for integration into applications that have a constant update loop like game engines.

About : warning: "_POSIX_C_SOURCE" redefined

When I compile my project with httpserver.h, occur warning :

./src/httpserver.h:295:0: warning: "_POSIX_C_SOURCE" redefined
 #define _POSIX_C_SOURCE 199309L
 
In file included from /usr/include/x86_64-linux-gnu/sys/types.h:25:0,
                 from /usr/include/x86_64-linux-gnu/curl/system.h:399,
                 from /usr/include/x86_64-linux-gnu/curl/curl.h:38,
                 from ./src/hr_darknet.c:1:
/usr/include/features.h:265:0: note: this is the location of the previous definition
# define _POSIX_C_SOURCE 200809L

Its must be '199309L' in the httpserver.h?

For fix the warning, can I modify httpserver.h like this:

#ifndef _POSIX_C_SOURCE
#define _POSIX_C_SOURCE
#endif

Thanks.

socket api bind is a little bit confused with std::bind

I changed
int rc = bind(s, (struct sockaddr *)addr, sizeof(struct sockaddr_in));
to
int rc = ::bind(s, (struct sockaddr *)addr, sizeof(struct sockaddr_in));
problem solved.

Add automated testing

I've written a basic server for testing but mostly test against it by hand. I'd like to add automated tests to improve this processes.

B

IPv6 Support

I haven't looked into this much but it would be nice to have

Wrong behaviour of the http_request_header function

I am coding the following:

        http_string_t path = http_request_header(request, "x-file-path");
        http_string_t output_path = http_request_header(request, "x-output-path");

but getting wrong values from these variables, for example:

        sds path_str = sdstrim(sdsnew(path.buf), "\n\r");
        log_debug("Path: %s", path_str);

curl -vvvv  -H "X-File-Path: /home/yehor/i_1.raw" -H "X-Output-Path: /home/yehor/test1" http://localhost:8080/task

2020-09-15 13:22:54.7908 DEBUG [rest.denoiser] Path: /home/yehor/i_1.raw
X-Output-Path: /home/yehor/test1

Optionally support HTTPS with libtls

I took a look at libtls and it seems like it shouldn't be too difficult to add HTTPS support with it. This will be disabled by default and available through a preprocessor define

Behaviour will be undefined when content length overflows an int

Currently I just use an int to store the content length this will overflow for request bodies that are multiple gigabytes in size causing undefined behaviour. I need to protect against this in the request and response. I'll need to look into how to best handle this. I think I can probably just reject requests with a content length over some max value.

error LNK1120: n unresolved externals

this is kind of my code:

#include "include/httpserver.h"
...

#define HTTPSERVER_IMPL
#define RESPONSE "Hello, World!"

...

void handle_request(struct http_request_s *request)
{
	struct http_response_s *response = http_response_init();
	http_response_status(response, 200);
	http_response_header(response, "Content-Type", "text/plain");
	http_response_body(response, RESPONSE, sizeof(RESPONSE) - 1);
	http_respond(request, response);
}

...

void Webserver_Runner(bool *run, int *i)
{
	struct http_server_s *server = http_server_init(8080, handle_request);
	http_server_listen(server);
}

main(){
        ....
        int counter = 0;
	ThreadHandler<int> *th3 = new ThreadHandler<int>();
	th3->setRunner(Webserver_Runner, &counter);
	th3->start();
        ...
}

But when I try to build it using nmake I get the following result:

main.obj : error LNK2019: unresolved external symbol _http_server_init referenced in function "void __cdecl Webserver_Runner(bool *,int *)" (?Webserver_Runner@@YAXPA_NPAH@Z)
main.obj : error LNK2019: unresolved external symbol _http_server_listen referenced in function "void __cdecl Webserver_Runner(bool *,int *)" (?Webserver_Runner@@YAXPA_NPAH@Z)
main.obj : error LNK2019: unresolved external symbol _http_response_init referenced in function "void __cdecl handle_request(struct http_request_s *)" (?handle_request@@YAXPAUhttp_request_s@@@Z)
main.obj : error LNK2019: unresolved external symbol _http_response_status referenced in function "void __cdecl handle_request(struct http_request_s *)" (?handle_request@@YAXPAUhttp_request_s@@@Z)
main.obj : error LNK2019: unresolved external symbol _http_response_header referenced in function "void __cdecl handle_request(struct http_request_s *)" (?handle_request@@YAXPAUhttp_request_s@@@Z)
main.obj : error LNK2019: unresolved external symbol _http_response_body referenced in function "void __cdecl handle_request(struct http_request_s *)" (?handle_request@@YAXPAUhttp_request_s@@@Z)
main.obj : error LNK2019: unresolved external symbol _http_respond referenced in function "void __cdecl handle_request(struct http_request_s *)" (?handle_request@@YAXPAUhttp_request_s@@@Z)
main.exe : fatal error LNK1120: 7 unresolved externals
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2019\VC\Tools\MSVC\14.28.29333\bin\HostX86\x86\cl.EXE"' : return code '0x2'
Stop.

Do I need to link the h somehow?

Broken connection after interrupted write

Thank you for writing this library! I find it quite useful.

I encountered an issue where the server sometimes does not seem to receive requests. (This is on an application I am developing, and the requests originate from Firefox.) Digging a bit deeper, I think the following code is able to reliably reproduce the issue:

#include <unistd.h>
#undef _POSIX_C_SOURCE

#define HTTPSERVER_IMPL
#include "httpserver.h"

char buf[1024 * 1024 * 8];

void handle_request(struct http_request_s* request) {
    puts("Received request, sending answer");
    struct http_response_s* response = http_response_init();
    http_response_status(response, 200);
    http_response_header(response, "Content-Type", "text/plain");
    http_response_body(response, buf, sizeof(buf));
    http_respond(request, response);
}

void read_n(int sock, ssize_t n) {
    // Read in chunks. This is off-by-one intentionally, to also read the headers.
    int wait = 1;
    while (n >= 0) {
        n -= read(sock, buf, 4096);
        if (wait) usleep(1000);
        wait = 0;
    }
}

int main(int argc, char** argv) {
    memset(buf, '.', sizeof(buf));
    for (int i = 127; i < sizeof(buf); i+= 128) {
        buf[i] = '\n';
    }
    
    if (argc == 1) {
        struct http_server_s* server = http_server_init(8080, handle_request);
        puts("Starting server");
        http_server_listen(server);
    } else {
        int sock = socket(AF_INET, SOCK_STREAM, 0);

        // Connect to 127.0.0.1:8080
        struct sockaddr_in addr;
        memset(&addr, 0, sizeof(addr));
        addr.sin_family = AF_INET;
        addr.sin_port = 0x901f;
        addr.sin_addr.s_addr = 0x0100007f;
        connect(sock, (struct sockaddr*)&addr, sizeof(addr));

        char req[] = "GET / HTTP/1.1\r\nHost: localhost\r\n\r\n";
        
        puts("Sending first request");
        write(sock, req, sizeof(req));
        puts("Sent.");
        read_n(sock, sizeof(buf));
        puts("Received answer");
        
        puts("Sending second request");
        write(sock, req, sizeof(req));
        puts("Sent.");
        read_n(sock, sizeof(buf));
        puts("Received answer");
    }
}

To reproduce, compile this code and run two instances: the first without arguments, and the second with one argument (it does not matter which). The first instance starts the server, while the second tries to send two simple requests. On my machine, only the first of these requests is handled. I believe that the following happens:

The first request is sent.
The server receives the first request, and returns a large(-ish) response (8 MiB).
The client reads this response, but waits after reading a small chunk of it.
This interrupts the write on the server and triggers the logic in hs_write_response, calling hs_add_write_event.
The client continues to read the rest of the response.
The server completes its handling of the request and re-initialises the session for future requests (third case in hs_write_response).
The client sends another request.
Nothing happens, as the call to hs_add_write_event cleared the EPOLLIN flag

Based on the above, I think that restoring the EPOLLIN flag at step 6 should fix this issue, and indeed, that is what happens on my machine. I will be making a pull request with the code shortly.

add support for Windows

Perhaps I'm one of the only developers who likes to do web things on Windows, but it would be awesome if this project had support for Win32.

I'm not entirely sure what the equivalent of kqueue / epoll on Windows would be, but according to the docs I think it would be Overlapped I/O. There's also this thread, which also suggests WaitForMultipleObjects among other things. I'm not sure if using these APIs would cleanly fit into the platform specific section you've got at the bottom or would require a slightly larger refactor.

I might take a stab at this at some point even just for my own edification, but wanted to get your thoughts!

"fatal error: sys/event.h: No such file or directory"

When i trying to compile this code with gcc main.c

#define HTTPSERVER_IMPL
#include "httpserver.h"

#define RESPONSE "Hello, World!"

void handle_request(struct http_request_s* request) {
  struct http_response_s* response = http_response_init();
  http_response_status(response, 200);
  http_response_header(response, "Content-Type", "text/plain");
  http_response_body(response, RESPONSE, sizeof(RESPONSE) - 1);
  http_respond(request, response);
}

int main() {
  struct http_server_s* server = http_server_init(8080, handle_request);
  http_server_listen(server);
}

I got this error

In file included from main.c:2:
common.h:38:3: error: unknown type name ‘epoll_cb_t’
common.h:91:3: error: unknown type name ‘epoll_cb_t’
common.h:92:3: error: unknown type name ‘epoll_cb_t’
common.h:112:3: error: unknown type name ‘epoll_cb_t’
common.h:113:3: error: unknown type name ‘epoll_cb_t’
server.c:14:10: fatal error: sys/event.h: No such file or direct
compilation terminated.

Info about system

$ gcc --version
gcc (GCC) 13.2.0
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ uname -a
Linux s2me 6.5.8-200.fc38.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Oct 20 15:53:48 UTC 2023 x86_64 GNU/Linux

$ cat /etc/os-release 
NAME="Fedora Linux"
VERSION="38.20231101.0 (Silverblue)"
ID=fedora
VERSION_ID=38
VERSION_CODENAME=""
PLATFORM_ID="platform:f38"
PRETTY_NAME="Fedora Linux 38.20231101.0 (Silverblue)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:38"
DEFAULT_HOSTNAME="fedora"
HOME_URL="https://silverblue.fedoraproject.org"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora-silverblue/"
SUPPORT_URL="https://ask.fedoraproject.org/"
BUG_REPORT_URL="https://github.com/fedora-silverblue/issue-tracker/issues"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=38
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=38
SUPPORT_END=2024-05-14
VARIANT="Silverblue"
VARIANT_ID=silverblue
OSTREE_VERSION='38.20231101.0'

gcc installed through rpm-ostree

http header parse invalid when request multiple url request in one session

to produce
start the ./functional-test-server
then

curl -X GET --header "content-type: application/json"  -v   http://127.0.0.1:8080/echo --next --header "content-type: application/json"  -v   http://127.0.0.1:8080/headers

the expected behavior is the receive on client side

Host: 127.0.0.1:8080
User-Agent: curl/8.1.2
Accept: */*
content-type: application/json

the acutul behavior is no response on server side, for the server the url is not valid parsed, added the screenshot for information