luukdegram / apple_pie Goto Github PK

The following flow should be implemented

• If/Else
• Loop

Should we stick close to Zig's syntax?
i.e.

{if (x)
<div></div>
}

{for(iterator)|x|
<div></div>
}

self descriptive :)

Hi,

I think there's a bug with any when requesting /.

Taking the router example and replacing this line with router.any instead I would expect the code to still work, but instead it panics:

thread 111030 panic: attempt to use null value
/home/vincent/dev/stuff/apple_pie/src/trie.zig:104:51: 0x252f4f in .apple_pie.trie.Trie(u8).get (router)
                return .{ .static = self.root.data.? };
                                                  ^
/home/vincent/dev/stuff/apple_pie/src/router.zig:106:60: 0x2518df in .apple_pie.router.Router(*Context,[]const .apple_pie.router.Route{(struct .apple_pie.router.Route constant),(struct .apple_pie.router.Route constant),(struct .apple_pie.router.Route constant),(struct .apple_pie.router.Route constant),(struct .apple_pie.router.Route constant),(struct .apple_pie.router.Route constant)}).serve (router)
            switch (trees[@enumToInt(request.method())].get(request.path())) {
                                                           ^
/home/vincent/dev/stuff/apple_pie/src/server.zig:197:24: 0x254ee8 in .apple_pie.server.ClientFn(*Context,.apple_pie.router.Router(*Context,[]const .apple_pie.router.Route{(struct .apple_pie.router.Route constant),(struct .apple_pie.router.Route constant),(struct .apple_pie.router.Route constant),(struct .apple_pie.router.Route constant),(struct .apple_pie.router.Route constant),(struct .apple_pie.router.Route constant)}).serve).handle (router)
                handler(context, &response, parsed_request) catch |err| {
                       ^
/home/vincent/dev/stuff/apple_pie/src/server.zig:151:24: 0x254962 in .apple_pie.server.ClientFn(*Context,.apple_pie.router.Router(*Context,[]const .apple_pie.router.Route{(struct .apple_pie.router.Route constant),(struct .apple_pie.router.Route constant),(struct .apple_pie.router.Route constant),(struct .apple_pie.router.Route constant),(struct .apple_pie.router.Route constant),(struct .apple_pie.router.Route constant)}).serve).run (router)
            self.handle(gpa, clients, context) catch |err| {
                       ^
/home/vincent/dev/stuff/apple_pie/src/server.zig:113:26: 0x253f77 in .apple_pie.server.Server.run (router)
                .frame = async client.run(gpa, &clients, context),
                         ^
/home/vincent/dev/stuff/apple_pie/src/server.zig:56:28: 0x24bc3d in .apple_pie.server.listenAndServe (router)
    try (Server.init()).run(gpa, address, context, handler);
                           ^
/home/vincent/dev/stuff/apple_pie/examples/router.zig:22:28: 0x243a33 in main (router)
    try http.listenAndServe(
                           ^
/home/vincent/dev/devtools/zig/lib/std/start.zig:527:37: 0x23c09a in std.start.callMain (router)
            const result = root.main() catch |err| {
                                    ^
/home/vincent/dev/devtools/zig/lib/std/start.zig:469:12: 0x22014e in std.start.callMainWithArgs (router)
    return @call(.{ .modifier = .always_inline }, callMain, .{});
           ^
/home/vincent/dev/devtools/zig/lib/std/start.zig:383:17: 0x21f1d6 in std.start.posixCallMainAndExit (router)
    std.os.exit(@call(.{ .modifier = .always_inline }, callMainWithArgs, .{ argc, argv, envp }));
                ^
/home/vincent/dev/devtools/zig/lib/std/start.zig:296:5: 0x21efe2 in std.start._start (router)
    @call(.{ .modifier = .never_inline }, posixCallMainAndExit, .{});
    ^
fish: Job 1, './zig-out/bin/router' terminated by signal SIGABRT (Abort)

as far as I understand the problem is that:

• the router looks up the trie for the request method
• that trie always has a root node matching / with no data
• the trie assumes the data is non-null if the path is /

the hashmap currently provided by Request.headers(Allocator) does not currently support this

According to https://datatracker.ietf.org/doc/html/rfc2068#section-19.7.1, an HTTP/1.0 client is allowed to have a persistent connection using the Connection: keep-alive header. This means that we should verify/check this header also for HTTP/1.0 connections, and not only for 1.1.

trying to implement a function that can detect if the request was made with HTTPS, so i need this in conjunction with the headers

build.zig file needs updating. It no longer works on 0.11 version.

I would do this, but I don't know how build system works, sorry 😢.

Instead of this:

const std = @import("std");
const http = @import("apple_pie");

pub const io_mode = .evented;

pub fn main() !void {
    var gpa = std.heap.GeneralPurposeAllocator(.{}){};
    defer _ = gpa.deinit();
    const allocator = gpa.allocator();

    const builder = router.Builder(void);

    try http.listenAndServe(
        allocator,
        try std.net.Address.parseIp("127.0.0.1", 8080),
        {},
        comptime router.Router(void, &.{
            builder.get("/hello/:name", []const u8, hello),
        }),
    );
}

/// Shows "Hello {name}" where {name} is /hello/:name
fn hello(_: void, resp: *http.Response, req: http.Request, captures: ?*const anyopaque) !void {
    _ = req;
    const name = @ptrCast(
        *const []const u8,
        @alignCast(
            @alignOf(*const []const u8),
            captures,
        ),
    );
    try resp.writer().print("Hello {s}\n", .{name.*});
}

The parameters would be extracted from the function.

const std = @import("std");
const http = @import("apple_pie");

pub const io_mode = .evented;

pub fn main() !void {
    var gpa = std.heap.GeneralPurposeAllocator(.{}){};
    defer _ = gpa.deinit();
    const allocator = gpa.allocator();

    const builder = router.Builder(void);

    try http.listenAndServe(
        allocator,
        try std.net.Address.parseIp("127.0.0.1", 8080),
        {},
        comptime router.Router(void, &.{
            builder.get("/hello/:name", hello),
        }),
    );
}

/// Shows "Hello {name}" where {name} is /hello/:name
fn hello(_: void, resp: *http.Response, req: http.Request, name: []const u8) !void {
    _ = req;
    try resp.writer().print("Hello {s}\n", .{name});
}

Not sure if it should match the url parameters by name or position.

Would throw an error if the names/number of url matches is not the same as the number of parameters.

I'm interested in creating a prototype for this.

If we run the router example:

$ zig build example -Dexample=router

We can use curl to craft a request for any file that the user running the server can access:

$ curl "http://127.0.0.1:8080" --request-target "/files/../README.md"
# Apple Pie

Apple pie is HTTP Server implementation in [Zig](https://ziglang.org). The initial goal... (content clipped)

I'm not sure the best way to handle this situation, but two ideas off the top of my head:

1. Disallow paths with .. in them. Browsers (or at least Chromium and Firefox) handle this case client side, so it wouldn't interfere with browsers
2. Create a modified std.fs.path.resolve that keeps track of the root directory and prevents reaching beyond it

Some features the router should support

• Grouping: router.group("/path", my_group)
• HTTP Method support: (i.e. router.GET("/", ...))
• Routing based on query parameters (example.com/path?name=apple_pie)
• Routing based on path capturing (blog.com/posts/{id})
• Routing should be performant and preferably non-allocating

I have noticed that for loop syntax is not up to date with the latest zig.

Previous syntax for index of for loop elements:
for (items) |item, index|

New syntax for index of for loop elements:
for (items, 0..) |item, index|

I have created pull request #87.

I have already made pull request (which got merged) to cappy, which is dependent on apple pie.

When a client requests with a header Expect: 100-continue, we must send a status 100 continue. We can send this immediately after verifying the request headers and MUST send this before sending the 'regular' response.

As we're currently relying on buffered writers for performance benefits, the 100 status code must be flushed immediately to the client, so they can send their body (if they haven't already).

This can then be used by the Router in #3 for fast routing.

Is it possible to support https protocol in apple pie?

If I add the following testcase to the end of src/trie.zig, the second result will return Result.none instead of Result.static as you would expect.

If change the insertion order, the first route with a /api prefix will work, but the rest will return Result.none.

test "Similar looking paths" {
    comptime var trie = Trie(u32){};
    comptime trie.insert("/api", 1);
    comptime trie.insert("/api/users", 2);
    comptime trie.insert("/api/events", 3);
    comptime trie.insert("/api/events/:id", 4);

    const res = trie.get("/api");
    const res2 = trie.get("/api/users");
    const res3 = trie.get("/api/events");
    const res4 = trie.get("/api/events/1337");
    const res5 = trie.get("/foo");

    std.log.warn("res2: {}", .{res2});
    std.testing.expectEqual(@as(u32, 1), res.static);
    std.testing.expectEqual(@as(u32, 2), res2.static);
    std.testing.expectEqual(@as(u32, 3), res3.with_params.data);
    std.testing.expectEqual(@as(u32, 4), res4.with_params.data);
    std.testing.expect(res4 == .none);

    std.testing.expectEqualStrings("1337", res4.with_params.params[0].value);
}

Console output:

zig test src/trie.zig
Test [2/2] test "Similar looking paths"... [default] (warn): res2: Result{ .none = void }
access of inactive union field

Hi,

When running in io_mode = .blocking, the connection is always closed here, however the response always writes Connection: keep-alive here.

Some clients are fine with this but for example the scraper from VictoriaMetrics isn't.

I fixed this by adding a close field in Response and writing the Connection: close header accordingly.
I'm not sure if that's the best way to do it but if it's acceptable I can make a PR.

Any advice ?

last meaningful updates was about one year ago, is this project under development now?

Reported by @mattnite

[meg@nixos:~/other/dev/aquila]$ zig version
0.10.0-dev.536+557a09752

[meg@nixos:~/other/dev/aquila]$ zig build
/home/meg/other/dev/aquila/.zigmod/deps/git/github.com/Luukdegram/apple_pie/src/router.zig:19:14: error: expected type expression, found 'anytype'
    handler: anytype,
             ^

ziglang/zig@3eb8d01

@Luukdegram, the build fails on latest Zig.
zig build example router fails because the build.zig format has been changed.
When including apple_pie from another project, the following error is thrown:

.zigmod/deps/git/github.com/Luukdegram/apple_pie/src/router.zig:199:72: error: function with comptime-only return type 'router.Route(*main.Context)' requires all parameters to be comptime
        pub fn get(path: []const u8, comptime handlerFn: anytype) Route(Context) {
                                                                  ~~~~~^~~~~~~~~
.zigmod/deps/git/github.com/Luukdegram/apple_pie/src/router.zig:23:18: note: struct requires comptime because of this field
        handler: Handler,
                 ^~~~~~~
.zigmod/deps/git/github.com/Luukdegram/apple_pie/src/router.zig:23:18: note: use '*const fn(*main.Context, *response.Response, Request, []const trie.Entry) anyerror!void' for a function pointer type
        handler: Handler,
                 ^~~~~~~
.zigmod/deps/git/github.com/Luukdegram/apple_pie/src/router.zig:199:20: note: param 'path' is required to be comptime
        pub fn get(path: []const u8, comptime handlerFn: anytype) Route(Context) {
                   ^~~~

Currently the request parser is quite annoying to maintain and not very readable.
We should probably have some event/state driven request parser. This should also make it easier to support chunk-based requests.

Currently the suspension point is at the request reader, which means we do not accept any new connections until the reader is being suspended. By using runDetached we can assure the request handler function is coroutined and we do not have to wait for a suspension point before we can accept a new connection.

Unfortunately didn't get a stack trace but this showed up in my logs recently, thought I'd make a tracking issue

Users only have access to a *Response and Request instance.
This means that if they require access to some allocator, database connection, or w/e, the variable must be global. This is very annoying to work with, and also prone to footguns.

The idea is to allow providing a context variable to the server, which is then accessible within the handler.
Using comptime we can ensure the types match.

This would look as follow:

try server.run(
...,
some_variable,
myHandle);

fn myHandle(ctx: @TypeOf(some_variable), resp: *Response, req: Request) !void {
    // access to `some_variable` is possible here.
    // Allows both constants and mutable pointers, depending on what was provided in the `run` function.
}

Currently, apple_pie only reads from the TCP stream once to read in the headers (here: https://github.com/Luukdegram/apple_pie/blob/master/src/request.zig#L199). However, the headers may not all be sent by that point, which makes the function return with error.IncorrectHeader.

Implement accordingly to https://tools.ietf.org/html/rfc2616#section-13

Steps to reproduce

Apply this patch to the router example:

diff --git a/examples/router.zig b/examples/router.zig
index 5b7d909..eabe465 100644
--- a/examples/router.zig
+++ b/examples/router.zig
@@ -28,7 +28,7 @@ pub fn main() !void {
             builder.get("/", null, index),
             builder.get("/headers", null, headers),
             builder.get("/files/*", null, serveFs),
-            builder.get("/hello/:name", []const u8, hello),
+            builder.get("/hello/:name", u32, hello),
             builder.get("/route", null, route),
             builder.get("/posts/:post/messages/:message", struct {
                 post: []const u8,
@@ -74,13 +74,13 @@ fn hello(ctx: *Context, resp: *http.Response, req: http.Request, captures: ?*con
     _ = req;
     _ = ctx;
     const name = @ptrCast(
-        *const []const u8,
+        *const u32,
         @alignCast(
-            @alignOf(*const []const u8),
+            @alignOf(*const u32),
             captures,
         ),
     );
-    try resp.writer().print("Hello {s}\n", .{name.*});
+    try resp.writer().print("Hello {}\n", .{name.*});
 }
 
 /// Serves a file

Expected behavior

joachim@dustbowl ~> curl http://localhost:8080/hello/123
Hello 123

Actual behavior

joachim@dustbowl ~> curl http://localhost:8080/hello/123
curl: (52) Empty reply from server
joachim@dustbowl ~ [52]> 

The server crashes with this backtrace:

thread 32959 panic: incorrect alignment
/home/joachim/src/apple_pie/examples/router.zig:80:13: 0x26b11b in hello (router)
            captures,
            ^
/home/joachim/src/apple_pie/src/router.zig:100:33: 0x2723be in .apple_pie.router.Router(*Context,[]const .apple_pie.router.Route(*Context){(struct .apple_pie.router.Route(*Context) constant),(struct .apple_pie.router.Route(*Context) constant),(struct .apple_pie.router.Route(*Context) constant),(struct .apple_pie.router.Route(*Context) constant),(struct .apple_pie.router.Route(*Context) constant),(struct .apple_pie.router.Route(*Context) constant)}).handle (router)
            return route.handler(ctx, res, req, @ptrCast(?*const anyopaque, &param));
                                ^
/home/joachim/src/apple_pie/src/router.zig:129:47: 0x270f15 in .apple_pie.router.Router(*Context,[]const .apple_pie.router.Route(*Context){(struct .apple_pie.router.Route(*Context) constant),(struct .apple_pie.router.Route(*Context) constant),(struct .apple_pie.router.Route(*Context) constant),(struct .apple_pie.router.Route(*Context) constant),(struct .apple_pie.router.Route(*Context) constant),(struct .apple_pie.router.Route(*Context) constant)}).serve (router)
                            return Self.handle(route, object.params[0..object.param_count], context, response, request);
                                              ^
/home/joachim/src/apple_pie/src/server.zig:113:26: 0x273092 in .apple_pie.server.Server.run (router)
                .frame = async client.run(gpa, &clients, context),
                         ^
/home/joachim/src/zig/lib/std/event/loop.zig:1415:25: 0x2b935d in std.event.loop.Loop.workerRun (router)
                        resume handle;
                        ^
/home/joachim/src/zig/lib/std/event/loop.zig:702:23: 0x25287d in std.event.loop.Loop.run (router)
        self.workerRun();
                      ^
/home/joachim/src/zig/lib/std/start.zig:488:21: 0x2243f9 in std.start.callMainWithArgs (router)
            loop.run();
                    ^
/home/joachim/src/zig/lib/std/start.zig:409:17: 0x222506 in std.start.posixCallMainAndExit (router)
    std.os.exit(@call(.{ .modifier = .always_inline }, callMainWithArgs, .{ argc, argv, envp }));
                ^
/home/joachim/src/zig/lib/std/start.zig:322:5: 0x222312 in std.start._start (router)
    @call(.{ .modifier = .never_inline }, posixCallMainAndExit, .{});
    ^
The following command terminated unexpectedly:
cd /home/joachim/src/apple_pie && /home/joachim/src/apple_pie/zig-out/bin/router 

Implement form body as per rfc7231 & rfc2046.

Currently, we only expose Response from response.zig.
This means users cannot use the Status type definition anywhere.

There's 2 solutions to this:

• Expose response.zig in its entirety.
• Expose Status in addition to Response

Clients can send the header If-Modified-Since to save bandwidth. Although clients can ignore this, servers must comply with this header.

This means we need to be able to parse the following three formats (due to earlier HTTP versions):
If-Modified-Since: Fri, 31 Dec 1999 23:59:59 GMT
If-Modified-Since: Friday, 31-Dec-99 23:59:59 GMT
If-Modified-Since: Fri Dec 31 23:59:59 1999

This proposal includes multiple changes to the way responses handle headers.

The first idea is to make the key of a header an union(enum).
This allows us to provide pre-defined keys, and also allow the user to provide a custom key.
Meaning we reduce the amount of allocations required as the keys are a simple enum in most cases.
The header key would look like:

const HeaderKey = union(enum) {
    content_type,
    content_length,
    custom: []const u8,
    
    fn toString(self: HeaderKey) {
        switch(self) {
            .content_type => return "Content-Type",
            .custom => |custom| return custom,
            ...
        }
    }
};

All keys apart from custom are of type void and do not require any allocations therefore while also preventing typos.

The second change is to provide a method to write a header directly, rather than appending it to some set/map.
This means it requires no allocation, but also means the user does not have to clean up any memory afterward.
However, this does come with some consideration. As headers come after the status line, we must either ensure the status code is written before writing any headers to the client. How to solve this must still be considered.
The method could look like:

fn writeHeader(self: *Response, key: HeaderKey, value: []const u8)!void

A worthwhile benefit to this approach is that we do not have to iterate over the map to detect if certain headers are set. As we can simply check this during the writeHeader method.

Hey there, i am a zig newbie here.
It looks to me that the router is splitting the called url path by slash and matching on those substrings.
The trie is the correct data structure for this but it will be much more efficient to match on characters or bytes.

https://github.com/darpi-rs/darpi/blob/master/gonzales/src/lib.rs

If you think its worthwhile to improve this, you can checkout my router implementation in my rust web framework.
It also has case insensitive capability without runtime costs.

HTTP 1.1 describes the requirement of having to include a Date header, containing the current date in GMT. We can use the following format for this:
Date: Fri, 31 Dec 1999 23:59:59 GMT

Currently, there's an open PR on Zig's std: #9040 that we could leverage for this.
But we will have to build upon it to get the current day of the week. (mon, thu, etc)

As title, will it be?

Just running the basic example, Im seeing 100% CPU on 1 core.

?? Not sure if I have configured something wrong, or if Im on a particularly bad release of zig-master with a broken std.net.

Anyway - ApplePie is idling in var connection = stream.accept() catch |err| switch (err) { in src/server.zig ... so that suggest that zig std.net is eating all the CPU.

Curious, will have a deeper play tomorrow.