riolet / wafer Goto Github PK

On OS X, building nope.c fails for me with:

> make
gcc -W -Wall -O2 -c -o nope.o nope.c
nope.c:13:10: fatal error: 'sys/sendfile.h' file not found
#include <sys/sendfile.h>
         ^
1 error generated.
make: *** [nope.o] Error 1

gcc --version output:

Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn)
Target: x86_64-apple-darwin13.3.0
Thread model: posix

Line:
https://github.com/riolet/nope.c/blob/master/nope.c#L224
getLine(client, buf, sizeof(buf));

All requests with a first line longer than 1024 bytes will fail.
As the server stops ignores the rest of the line.

As @MagicalTux explained in issue #19, our select() is not very effective since "While this new method allows a single worker to process multiple requests at the same time, it is quite unlikely to happen - especially since the process() method will not return the processing of that query is done."

Hi, I read the code of the whole project, it's now only supporting the select method. And there are someother multiplexing I/O methods, such as epoll, kqueue. Will nope.c support these methods? If so, I think the project needs a simple event driven library which packages multiple multiplexing I/O methods. And I'm writing a simple event driven library libse, I hope I could make some contributions to nope.c. Looking forward to your reply!
thank you very much!

I carried out a load test for nope example.

./wrk -t1 -c1 -d10s http://localhost:4242/factor

The config is: NOPE_EPOLL, NOPE_THREAD 2
The bug looks like this:

GET /factorGET /factorGET /factorGET /factorGET /factorGET /factorGET /factorGET /factorGET .... /factorGET /factorGEclose: Bad file descriptor

notify_parent() is not used in your curent version and timespec is not used too. Could you clean this up? The way you called getnameinfo without checking its failure mode is a bit confusing to me.

WAFer is: ultra-light ... wafer-thin

Can you be more specific in the README?
We would like to see approximate LOC, size in memory, size on disk. (compared with node.js etc.)

Writing web applications in C is a great idea. Still, people who want to write their own app using C can't legally do it without opening their sources. That's because you distribute it under GPL terms as a set of source files which should be extended by users to get their functionality:

2. You may modify your copy or copies of the Program or any portion
of it, thus forming a work based on the Program, and copy and
distribute such modifications or work under the terms of Section 1
above, provided that you also meet all of these conditions:

    a) You must cause the modified files to carry prominent notices
    stating that you changed the files and the date of any change.

    b) You must cause any work that you distribute or publish, that in
    whole or in part contains or is derived from the Program or any
    part thereof, to be licensed as a whole at no charge to all third
    parties under the terms of this License.

    c) If the modified program normally reads commands interactively
    when run, you must cause it, when started running for such
    interactive use in the most ordinary way, to print or display an
    announcement including an appropriate copyright notice and a
    notice that there is no warranty (or else, saying that you provide
    a warranty) and that users may redistribute the program under
    these conditions, and telling the user how to view a copy of this
    License.  (Exception: if the Program itself is interactive but
    does not normally print such an announcement, your work based 

It would be more appropriate to license it under dual MIT/GPL license so people could use your framework without opening the application source code.

In nope.c, buffer is used everywhere, I think we may need a automatically expandable buffer, which could automatically expand the memory space when add or copy data into the buffer. I wrote an simple one, see this .

Perhaps not an issue, may be my short sightedness. Is there any way to set the mimetype of the response. I would like to use nope.c to produce small rest servers serving simple dynamic json. i would like to be able to set response mimetype as "application/json", based on the response.

Incriminated code:
http_request req;
strcpy(req.filename,url);

"url" is 1024 bytes, req.filename is 512 bytes. Buffer overflow ensues by passing an URL between 512 bytes and 1024 bytes (plus "req" is helpfully allocated on the stack).

Suggestion: use strncpy().

After having a quick look at the code I think you should put a big warning on the README that this code is not at all ready for anything that listens on a public port. Besides the stack overflow that was already reported by MagicalTux there's also this code which will cause a bufferoverflow if msg & longmsg are too long:

void client_error(int fd, int status, char *msg, char *longmsg){
    char buf[MAXLINE];
    sprintf(buf, "HTTP/1.1 %d %s\r\n", status, msg);
    sprintf(buf + strlen(buf),
            "Content-length: %lu\r\n\r\n", strlen(longmsg));
    sprintf(buf + strlen(buf), "%s", longmsg);
    writen(fd, buf, strlen(buf));
}

Pretty sure this is not the only problem remaining.

It's in POSIX ≥ 2008.

Now in nope.c, send function is called without checking whether the kernel send buffer is enough using select. For example, there are 100 active clients, we need to send 1k data to each client, but the total kernel buffer is 10k (just as an example), we know that when send function is finished, it only ensures the data of process is copied into kernel send buffer, maybe when dealing with the 50th client, the kernel buffer is full, then the process will blocked until the kernel buffer has enough space for data.

strcpy(req.filename,url);
url can be 1024 chars
req.filename can be 512

malloc(n * size) is an antipattern as n * size can overflow (especially from an attacker). You'll want to use calloc or create your own malloc_array function. Something like the following could work:

static inline void * malloc_array(size_t nmemb, size_t size)
{
    /* Make the check so that a constant size deletes the zero check
     * and the division.
     */
    if (size != 0u && SIZE_MAX / size < nmemb) {
        errno = ENOMEM;
        return NULL;
    }

    return malloc(size * nmemb);
}

Let's have a brief discussion about nope.c architecture. What I have in mind for nope.c is a layered architecture that embodies the Unix philosophy with three main strata:

1. The core (nope.c): A minimalist HTTP server, mostly focussing on the event loop. We are using select() currently, but we could provide additional event notification systems (poll(),epoll() etc) as drop in replacements.
2. The shell (nopeutils.c - optional): Provides a programming interface to the core, abstracting over most of the cores operations.
3. The modules (optional): Provides various enhancements to make it easier to write useful web applications. nope.c does not need many specific nope.c modules as there are thousands of c libraries that nope.c can use natively. That said, we need modules for ssl, multi-part parsing and web sockets.

What are your thoughts?

nprintf does not check bounds of its buffer.

Example code:

char *testbuf = calloc(1, sizeof(char) * 1000001);
memset(testbuf, 'A', 1000000);
printf("Testbuf len: %ld\n", strlen(testbuf));
nprintf(client, testbuf);

As informed by /u/jmhodges

an alternative solution might be

void server(Request request)
{
    int is_404 = 0;
    is_404 |= nope_route(request, "/factor", &factor, true);
    is_404 |= something else...
    if(!is_404)
    {
        handler_404(request);
    }
}

void handler_404(Request request)
{
    not_found(request.client);
}

file nope.c:
line 90 and line 92 repeated?
line 108 and line 110 repeated?

nope.c, line 67, the <p> tag is never closed.

I was a bit confused while going through your code particularly when I noticed that socket used "AF-Unix" . Why using it? I also noticed that now you have epoll, thread, and select. I fcntl and thread, but what do you again with epoll

Is there any interest in supporting a simple logging option for requests? I would be willing to help implement it.

Malloc can fail.

http://pubs.opengroup.org/onlinepubs/009695399/functions/malloc.html

Would you like to add more error handling for return values from functions like the following?

• pthread_mutex_init ⇒ initialize_threads
• strdup ⇒ getQueryPath

The way nope.c works, each worker thread will wait for an incoming connection, then process it.

While this new method allows a single worker to process multiple requests at the same time, it is quite unlikely to happen - especially since the process() method will not return the processing of that query is done.

I would guess the purpose is to have multiple open sockets and wait for incoming data, but if one sender is very slow and sends fragmented request elements (ie. only "GET " at first), it will block that child while the child may have accepted other connections.

Anyway I believe it would make sense to either keep nope.c's original simple design (listen, process), or if we really want to do this right we could:

• listen is only done in parent process
• each child process has a unix pipe to the parent process and keeps waiting for data to come
• incoming connections are accepted and data is received async in a temporary buffer (until empty newline or buffer full which would cause an error to be sent)
• once full buffer received, it is passed to an available child process through the pipe, alongside the client's fd (UNIX domain sockets can pass file descriptor between processes)
• parent closes the file descriptor, free memory
• child processes the request, responds directly to the client

From there, the parent process could automatically increase the number of child processes should - for example - all children be busy, and do a lot of other interesting things.

Pros:

• If someone opens multiple connections but doesn't send anything, it doesn't cause a denial of service
• Parent process has more control on the distribution of work
• If a child dies for some reason (bug in the server for example) parent will notice and will be able to respawn some children

Cons:

• More complex to implement
• Could potentially cause parent process to block if a child is frozen
• Things like OpenSSL will have troubles dealing with the fd moving to a different process (could be solved by having the parent act as a proxy rather than sending over the fd - also would improve security as child code will not have access to certificates/etc)

would make a nice addendum to what we use in machinekit.io - unfortunately we're unable to incorporate GPL-only code

any chance to relicense with a 'postwar' license ;-?