stephank / hyper-staticfile

Static file-serving for Hyper 1.0

hyper-staticfile

Static file-serving for Hyper 1.0.

See examples/doc_server.rs for a complete example that you can compile.

Documentation

hyper-staticfile's Issues

TLS support

Are there any plans for TLS support in this crate?

please discard

Hyper 0.12

Currently working on this, so leaving a note here. :)

Header Last-Modified should be in GMT instead of local time

Last-Modified

But I am getting header like this:

last-modified: Tue, 21 Apr 2020 21:32:30 +0200

hyper-staticfile/src/util/file_response_builder.rs

Lines 101 to 115 in ea9805f

	let modified: DateTime<LocalTz> = modified.into();
	match self.if_modified_since {
	// Truncate before comparison, because the `Last-Modified` we serve
	// is also truncated through `DateTime::to_rfc2822`.
	Some(v) if modified.trunc_subsecs(0) <= v.trunc_subsecs(0) => {
	return ResponseBuilder::new()
	.status(StatusCode::NOT_MODIFIED)
	.body(Body::empty())
	}
	_ => {}
	}
	res = res
	.header(header::LAST_MODIFIED, modified.to_rfc2822().as_str())

Set Travis CI to build master weekly.

Tokio is a bit of a moving target. It's good to get a heads up on incompatiblity. Without #3, master currently does not build as it does not allow use of deprecated elements. Yet the badge is green as of this moment. The denial of deprecated elements declaration in this crate would be stronger by enabling the cron job functionality in Travis CI.

Please look into enabling this.

invalid state when serving default file

Resolving / to /index.html fails with a panic (invalid state) because of the following line:

hyper-staticfile/src/resolve.rs

Line 101 in bc92710

let mut full_path = self.full_path.take().expect("invalid state");

The take() call removes the value from the self.full_path Option, which is then empty for the next state of the future:

hyper-staticfile/src/resolve.rs

Line 119 in bc92710

self.full_path.as_ref().expect("invalid state"),

Modifying the self.full_path with as_mut() fixes the issue.

command line interface for hyper-staticfile

@stephank Is a command line interface for hyper-staticfile already developed somewhere?
A small command line tool that serves the current directory via HTTP, like a replacement for python -m http.server?
If not I'll create such a project, I just want to make sure I didn't miss it and end up with two.

I developed something like that already directly on top of hyper back in the synchronous days. I now wanted to rewrite it for async hyper, had no idea how to feed files into tokio through hyper, but luckily I found this crate. 😉

RFE: mime_guess 2.x

It would be great if you could update dependency to 2.x so we don't have to ship 1.x in Fedora.

Thanks!

Question from beginning rustacean about the MainService in the example code.

hyper-staticfile/examples/doc_server.rs

Line 60 in 5542150

let s = MainService::new(&handle);

I try to figure out how to use hyper properly, so thanks for the example code. I'm wondering about the service being instantiated for each new connexion. Would this be a recommended workflow or is it just to simplify the example.

Should production code reuse service objects?

Sorry if this spams your issues ;(

why were prior versions yanked?

We had code that is building fine with hyper 0.14 and hyper-staticfile 0.6. It seems the prior versions were yanked - why?

Example code deprecated due to tokio-core ?

When I compiled and ran the code, I got this message:

warning: use of deprecated item 'hyper::server::server_proto::<impl hyper::server::Http>::bind_connection': All usage of the tokio-proto crate is going away.

I'm using this for the Cargo.toml:

[dependencies]
hyper-staticfile = "*"
futures = "0.1"
hyper = "0.11"
tokio-core = "0.1"
url = "1.1"

Should I be using a different version of these crates?

Static file server does not correctly resolve directories containing index.html on Windows

This bug applies to Windows operating system only and can be replicated using the doc server example app.

Steps to reproduce:

• Start server by running cargo doc && cargo run --example doc_server
• Navigate to http://localhost:3000/hyper_staticfile/` in the browser (tested with Firefox). Server will respond with 403 Forbidden
• Instead navigate to http://localhost:3000/hyper_staticfile/index.html`. Page loads correctly

Presumably this is due to open_with_metadata returning an Err on windows when it is resolving the index.html

[Tracking] Potential performance improvements

In my quick tests, there is some room for improvement in the performance of this crate. We discussed some ideas in #23, mainly:

• increasing the read buffer size or making it configurable
• using BytesMut to reduce allocations
• introducing a small amount buffering so that one chunk can be read from disk while another is being sent over the network

I'm filing this as somewhat of a tracking issue, because I don't think it's worth trying to work on it right now. The buffer size change is partially blocked because tokio limits the read size to 16 KB (tokio-rs/tokio#1976). And in my tests I couldn't get consistent performance, likely because of extra context switching in tokio (tokio-rs/tokio#1844).

resolve should take a Path instead of a Request (IMO)

First off, thanks for making the library! I was able to integrate this into a proxy that we deploy to customer sites and get huge speed improvements by pre-caching files then serving them with staticfile! So.. great work!

One minor nit that definitely tripped me up. It is documented, but I missed it, IMO we should use types to enforce this.

Presently, resolve takes a Request, but if there is anything in the Request except a path Part, it fails at runtime, and in a sort of a confusing way.

In addition, constructing an http::Request that only contains a path is actually a bit of a PITA. I haven't found an ergonomic way to create them.

My proposal is to make resolve take 2 pathPath or a AsRef<Path> parameter, then there can be no ambiguity about what kind of data resolve expects.

This would also streamline the resolve function as well, since it wouldn't need to perform the checks on the 'request` and unpack the uri().path() into a Path, it would already be done.

The ergonomics actually get better for the user. Instead of having to do this (Note that I was not able to make Request::get('/path/to/file') work.. it failed as an invalid URI)

let hpath: http::uri::PathAndQuery = filename.parse().unwrap();
let mut parts = http::uri::Parts::default();
parts.path_and_query = Some(hpath);
let uri = http::uri::Uri::from_parts(parts).unwrap();
let newreq = http::Request::get(&uri).body(()).unwrap();
let resolve_future = hyper_staticfile::resolve(&root, &newreq);

The new appproach would simply be:

let mypath = Path::new(filename);
let resolve_future = hyper_staticfile::resolve(&root, mypath); 

We could take it a step further and make the resolve function into a Resolver struct which can be initialized with &root so that we only have to run

   let mypath = Path::new(filename);
   let resolve_future = my_resolver::resolve(mypath);

Thoughts?

Serving either JSON string or static file (hyper 1.0+)

Previously with hyper 0.14, I was able to design a server with hyper and hyper-staticfile (0.9.x) to serve different types of requests, depending on the URI that requests plain JSON string or the others for static files.

Now with the new version of hyper-staticfile w/ hyper 1.0+, it became a bit more difficult. Body enum seems to have Empty or Stream variants so this by default can't handle { "foo": "1" } type of response. Any idea where I should work to make this ideally work?

Make FileBytesStreamRange public

I would like to use FileBytesStreamRange but it is not exported.

...
let file_handle = tokio::fs::File::open(&p).await?;
let b = hyper_staticfile::util::FileBytesStreamRange::new(file_handle, range);
let body = Body::wrap_stream(b);

let mime_type = mime_guess::from_path(&af.1)
    .first_or_octet_stream()
    .to_string();
Ok(Response::builder()
    .status(200)
    .header("Content-Type", mime_type)
    //.header("Cache-Control", "public, max-age=31536000")
    .body(body)
    .unwrap())
...

Thank you and all the best!

Support Range Header streaming

For media files a lot of browsers are using the Range header to retrieve only parts of bigger files.

It would be nice to have support for this in hyper-staticfile. If it is out of scope of this project I would like to be pointed to a crate handling this kind of requests to media files/streaming.

Thank you.

Resolve url paths without trailing '/' to corresponding directory with index.html

Actual behavior

When you request a path /dashboard and a corresponding {root}/dashboard/index.html file exists, the resolver returns a ResolveResult::IsDirectory variant which is essentially an error.

It works when you add a trailing slash to the url path, that is /dashboard/, but the user may not be aware of this and it becomes annoying after a while.

For context, my frontend is developed with svelte-kit and in production static ssr build, it maps each route to separate directories containing index.html files. If one of those directories is {root}/profile, it would be nice if users that visit mysite.com/profile would get the expected result.

The behavior is a result of an assumption made here.

hyper-staticfile/src/util/requested_path.rs

Lines 35 to 46 in 85eee8d

	pub fn resolve(root_path: impl Into<PathBuf>, request_path: &str) -> Self {
	let is_dir_request = request_path.as_bytes().last() == Some(&b'/');
	let request_path = PathBuf::from(decode_percents(request_path));
	let mut full_path = root_path.into();
	full_path.extend(&normalize_path(&request_path));
	RequestedPath {
	full_path,
	is_dir_request,
	}
	}

Expected behavior

The expected behavior is to return ResolveResult::Found when there is a corresponding directory with index.html and construct a response for it.

Reproduction

Add a new directory with an index.html file to your server root and try navigating to it without the trailing slash.

Leverage of sendfile syscall ?

sendfile(2) allows data to be transferred from a file descriptor to a TCP socket descriptor without paying the price of a copy round-trip through user space.

This is a great optimization for static file servers.

If the kernel(Version 4.13+) can handle KTLS(Kernel TLS offload), sendfile(2) can be used also for encrypted connections.

However, I am afraid the Rust ecosystem (hyper / rustls / etc) are ready.
Just raise the idea here -- leverage of sendfile when it is avaiable.

How to get sanitized path from ResolveResult::Found ?

Thank you for building a very nice library!

I am using 0.9.5 version of the library.

I would like to write some logic to control cache response headers differently based on the resolved file path and file name.

Here is roughly the code I have:

        let resolve_result = hyper_staticfile::resolve(&root, hyper_request)
            .await
            .unwrap();

        let cache_headers = if let ResolveResult::Found(file, metadata, mime) = &resolve_result {
            // more logic here based on sanitized path
            Some(300)
        } else {
            None
        };

        let response = hyper_staticfile::ResponseBuilder::new()
            .request(hyper_request)
            .cache_headers(cache_headers)
            .build(resolve_result)
            .unwrap();

I'm trying to find a way to get the sanitized path from ResolveResult::Found(File, Metadata, Mime)

It seems to be difficult/impossible to get full path from a std::fs::File object: https://stackoverflow.com/questions/62840979/how-do-i-get-the-filename-of-an-open-stdfsfile-in-rust

Is there some simple solution I am missing ? If ResolveResult::Found had another field containing the sanitized path this would be easy.

I could use the unsanitized path from hyper_request for my logic, or call RequestedPath::resolve on my own but this seems not ideal.

Thanks!

Accessing arbitrary files

On windows it is possible to access arbitrary files from other drives than the server is started from.

For example, if the server is started from D:\..., you can get files from C: by requests like this: http://127.0.0.1/anypath/c:/windows/win.ini

This is the same bug as found here: tower-rs/tower-http#204