fermyon / spin-fileserver

A static file server implemented as a Spin component

Static file server for Spin applications

A simple static file server as a Spin HTTP component, written in Rust.

This component is now fully componentized and can be used with any runtime that supports wasi:[email protected], such as Spin 2.0, wasmtime and NGINX Unit.

• Building from source
• Testing
• Using the component
  • Running the file server
  • Composing with the file server
• Configuration options

Building from source

Prerequisites:

• Rust at 1.72+ with the wasm32-wasi target configured
• cargo-component
• Spin v2.0 to run the component/examples

Compiling the component:

$ cargo component build --release

See the examples directory for examples of using and composing spin-fileserver with applications.

Testing

Prerequisites:

• Rust at 1.72+ with the wasm32-wasi target configured

Running test cases:

$ make test

Using the component as part of a Spin application

The easiest way to use this the Spin fileserver component in your application is to add it via its Spin template.

To create a new Spin app based on this component, run:

$ spin new -t static-fileserver

To add this component to your existing Spin app, run:

$ spin add -t static-fileserver

If you're looking to upgrade the version of this component in your application from one of the releases, select the release and corresponding checksum and update the component's source in the application's spin.toml, e.g.:

source = { url = "https://github.com/fermyon/spin-fileserver/releases/download/v0.1.0/spin_static_fs.wasm", digest = "sha256:96c76d9af86420b39eb6cd7be5550e3cb5d4cc4de572ce0fd1f6a29471536cb4" }

Next, we'll look at running the file server directly as well as using component composition to integrate this component in with your application logic.

Running the file server

Let's have a look at the component definition (from spin.toml):

[[trigger.http]]
route = "/..."
component = "fs"

# For more on configuring a component, see: https://developer.fermyon.com/spin/writing-apps
[component.fs]
source = "target/wasm32-wasi/release/spin_static_fs.wasm"
files = [{ source = "", destination = "/" }]
[component.fs.build]
command = "make"

This component will recursively mount all files from the current directory and will serve them at the configured route. If an index.html file is in the source root, it will be served if no file is specified.

Running the static server:

$ spin up --listen 127.0.0.1:3000 --file spin.toml

At this point, the component is going to serve all files in the current directory, and this can be tested using curl:

$ curl localhost:3000/LICENSE
                              Apache License
                        Version 2.0, January 2004
                    http://www.apache.org/licenses/

TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
...

See also the rust-standalone example showing use of the file server alongside a simple Rust-based application.

Component composition with the file server

The file server can also be composed with application logic to form one binary that can be run as a Spin application. See the following examples using the language and toolchains of your choice:

• Rust
• Javascript
• Python

Configuration options

The Spin fileserver supports various configuration options.

Setting the cache header

Currently, this file server has a single cache header that it can set through the CACHE_CONTROL environment variable. If no value is set, the default max-age=60 is used instead for all media types.

Setting the fallback path

You can configure a FALLBACK_PATH environment variable that points to a file that will be returned instead of the default 404 Not Found response. If no environment value is set, the default behavior is to return a 404 Not Found response. This behavior is useful for Single Page Applications that use view routers on the front-end like React and Vue.

# For more on configuring a component, see: https://developer.fermyon.com/spin/writing-apps#adding-environment-variables-to-components
[component.fs]
source = "target/wasm32-wasi/release/spin_static_fs.wasm"
files = [{ source = "test", destination = "/" }]
environment = { FALLBACK_PATH = "index.html" }

Using a custom 404 document

You can configure a CUSTOM_404_PATH environment variable and point to a file that will be served instead of returning a plain 404 Not Found response. Consider the following sample where the spin-fileserver component is configured to serve all files from the test folder. The desired page must exist in the test folder to send a custom 404 HTML page (here, 404.html) instead of a plain 404 Not Found response.

# For more on configuring a component, see: https://developer.fermyon.com/spin/writing-apps#adding-environment-variables-to-components
[component.fs]
source = "target/wasm32-wasi/release/spin_static_fs.wasm"
files = [{ source = "test", destination = "/" }]
environment = { CUSTOM_404_PATH = "404.html" }

Fallback favicon

If you haven't specified a favicon in your HTML document, spin-fileserver will serve the Spin logo as the fallback favicon. The spin-fileserver also serves the fallback favicon if the file (called favicon.ico or favicon.png) specified in your <link rel="shortcut icon" ...> element does not exist.

Remember that there are situations where spin-fileserver cannot serve the fallback favicon if no <link rel="shortcut icon" ...> element is specified. Browsers try to find the favicon in the root directory of the origin (somedomain.com/favicon.ico). If the application doesn't listen for requests targeting that route, it can't intercept requests to non-existing favicons.

spin-fileserver's Issues

Add CI

A GitHub action that runs the build and integration tests would be a great start.

After this a release action that publishes the module on a new git tag would be even better.

Support routing for SPA web apps

I'm working on a Vue.js single-page web app that will be hosted by the static fileserver and I'd like to suggest an improvement. My Vue.js app uses vue-router to provide routing inside my application and currently I can only use the "hash mode" routing where the URL's hash is used for routing. This is not ideal for SEO and deep linking inside my application. For instance, the basic vue template has routes for:

/      -> index.html -> HomePage.vue
/about -> ...        -> AboutPage.vue

But there is not a file at the /about path so it returns a 404 like it should. For my use case (and this is likely to cover all SPA web apps), it would be awesome if I could configure the fileserver to return the content for index.html with headers to use the browser cache. This would allow the vue-router component to route when the page is re-loaded and I can manually handle any 404 in the UI with a catch all route. See the vue-router docs for more info.

Option 1: URL Rewrites

Looking at other platforms like Netlify and Vercel, it seems like they both support URL re-writes at the file server level. Something like this would allow me to specify the priority in which paths are checked and also create a "fallback" or "catch all" route. Some samples for comparison:

# nginx
location / {
  try_files $uri $uri/ /index.html;
}

# netlify
/* /index.html 200

# vercel
{
  "rewrites": [{ "source": "/:path*", "destination": "/index.html" }]
}

I'm not sure what the best way to express this is for the spin.toml but it seems do-able. This seems like the most complete solution but it also comes with an additional level of complexity.

Option 2: Fallback file

This is the simplest option I can think of that would catch a "file not found" error and instead of returning a 404 it would return a file at some configurable path. Obviously this would have to be opt-in as you don't want to break functionality for existing users but supplying a fallback_route config item and checking for its existence should suffice.

I have tried to solve this with the redirect module but I don't want the window's location to change so I think it needs to be done in the fileserver component unless I'm missing something else, open to any other options or ideas you may have. Thanks!

Release v0.2.0

Cut the 0.2.0 release of this component and update the corresponding Spin template

Subdirectory fallback path

Consider the following website structure.
Currently, this cannot be served with this fileserver, since the fallback path is always the top-level index.html.

Ideally, the fileserver should check for an index.html in the last part of the path and return it if present, unless there is no direct match.

❯ tree _site
_site
├── 404.html
├── about
│   └── index.html
├── blog
│   ├── fifthpost
│   │   └── index.html
│   ├── firstpost
│   │   └── index.html
│   ├── fourthpost
│   │   └── index.html
│   ├── index.html
│   ├── secondpost
│   │   └── index.html
│   └── thirdpost
│       └── index.html
├── css
│   ├── index.css
│   ├── message-box.css
│   ├── prism-diff.css
│   └── prism-okaidia.css
├── feed
│   ├── feed.json
│   └── feed.xml
├── img
│   ├── IdthKOzqFA-350.avif
│   ├── IdthKOzqFA-350.png
│   └── IdthKOzqFA-350.webp
├── index.html
├── sitemap.xml
└── tags
    ├── another-tag
    │   └── index.html
    ├── index.html
    ├── number-2
    │   └── index.html
    ├── posts-with-two-tags
    │   └── index.html
    └── second-tag
        └── index.html

Support default files on path

If the fileserver is being used to serve static files for a front-end, it would be great if it could serve e.g. index.html on the root URL, without the user having to specifically request index.html.

One concept could be to make it a component config with a true | false | root_only enumerator, or to make a configuration option to specify this per path - e.g., "/" = "index.html", "/path1" = "path1_index.html" etc.

Fallback path for sub directories does not work with release 0.0.2

Hey 👋🏻 ,

I was playing with spin-fileserver today. And it happens that index.html for sub-directories is not resolved as mentioned in #30 and actually fixed as part of #31

I updated my local spin CLI and all the templates using spin templates upgrade so I ended up with having spin CLI in spin 1.4.1 (e0bd911 2023-07-12)

When creating a new app using spin new and selecting the static-fileserver (Serves static files from an asset directory) template, I end up with the following Spin.toml:

spin_manifest_version = "1"
authors = ["Thorsten Hans <[email protected]>"]
description = ""
name = "app"
trigger = { type = "http", base = "/" }
version = "0.1.0"

[[component]]
source = { url = "https://github.com/fermyon/spin-fileserver/releases/download/v0.0.2/spin_static_fs.wasm", digest = "sha256:65456bf4e84cf81b62075e761b2b0afaffaef2d0aeda521b245150f76b96421b" }
id = "app"
files = [ { source = "assets", destination = "/" } ]
[component.trigger]
route = "/static/..."

From here, I updated my Spin.toml (I changed route, files::sources, and added the FALLBACK_PATH env var), making my Spin.toml looks like this:

spin_manifest_version = "1"
authors = ["Thorsten Hans <[email protected]>"]
description = ""
name = "app"
trigger = { type = "http", base = "/" }
version = "0.1.0"

[[component]]
source = { url = "https://github.com/fermyon/spin-fileserver/releases/download/v0.0.2/spin_static_fs.wasm", digest = "sha256:65456bf4e84cf81b62075e761b2b0afaffaef2d0aeda521b245150f76b96421b" }
id = "app"
files = [ { source = "public", destination = "/" } ]
environment = { FALLBACK_PATH = "index.html" }
[component.trigger]
route = "/..."

Actual behavior

After starting the application using spin up I can browse http://localhost:3000 which correctly returns http://localhost:3000/index.html.

However, when I try to access http://localhost:3000/foo (which is a directory containing an index.html), I receiver the index.html from the root folder and the following is printed to stdout:

Cannot find file /foo/. Attempting fallback.

Expected behavior

The static fileserver should return index.html from the subfolder.

Where things get strange

I cloned fermyon/spin-fileserver (this repository) and compiled it using make. Then I replaced the component::source and point it to my local version of spin-fileserver

[[component]]
source = "../../spin-fileserver/target/wasm32-wasi/release/spin_static_fs.wasm"

When running this, the index.html from the sub-directory foo is served correctly. Besides adding some println! statements to see how paths were resolved I have not made any changes! (See my git diff below)

It looks like something is broken/wrong with the release 0.0.2 of the fermyon/spin-fileserver.

Recognize changes in files.source at runtime

I noticed that spin-fileserver currently does not recognize file changes at runtime.

Scenario to Reproduce:

1. Create following spin.toml

cat <<EOF > spin.toml
spin_manifest_version = "1"
authors = ["Christoph Voigt <[email protected]>"]
description = "This Spin application serves static files"
name = "static-server"
trigger = { type = "http", base = "/" }
version = "0.1.0"

[[component]]
source = { url = "https://github.com/fermyon/spin-fileserver/releases/download/v0.0.3/spin_static_fs.wasm", digest = "sha256:38bf971900228222f7f6b2ccee5051f399adca58d71692cdfdea98997965fd0d" }
id = "static-server"
files = [{ source = "content", destination = "/" }]
[component.trigger]
route = "/..."
EOF

2. Create directory & file to be served:

$ mkdir -p content
$ cat <<EOF > content/index.html
<html>
<head>
    <title>Spin Website</title>
    <body>
        <h1>Welcome from Spin!</h1>
    </body>
</html>
EOF

3. Serve files & request:

$ spin up
$ curl http://127.0.0.1:3000
<html>
<head>
<title>Hello from Spin</title>
</head>
<body>
<h1>Hello from Spin!</h1>
</body>
</html>

4. Update index.html

$ cat <<EOF > content/index.html
<html>
<head>
    <title>Spin Website</title>
    <body>
        <h1>Welcome from Spin! - UPDATED</h1>
    </body>
</html>
EOF

5. Request with the result, that the update is not served.

$ curl http://127.0.0.1:3000
<html>
<head>
<title>Hello from Spin</title>
</head>
<body>
<h1>Hello from Spin!</h1>
</body>
</html>

Expected behavior:

$ curl http://127.0.0.1:3000
<html>
<head>
    <title>Spin Website</title>
    <body>
        <h1>Welcome from Spin! - UPDATED</h1>
    </body>
</html>

Of course, I could restart Spin - but I don't think this is a desired workaround.

In a specific use case, I wanted to host Spin in Kubernetes and provide content via a sidecar container. This pattern is currently not possible.

Default favicon

The issue of web browsers requesting /favicon.ico is starting to become confusing.

I would like to suggest that we supply a default favicon icon, compiled into the file server — for example, the Spin logo.

Users could always override that by providing their own file at the root of the files.

Thoughts?

Update dependency on spin sdk

Update dependency to new rev of spin-sdk in Cargo.toml when fermyon/spin#208 is merged:

Example spin.toml includes .git files as assets, throws permission error

While starting work on #1, setting up the project using the default configuration results in an hard error as a result of including and attempting to access files under .git/. This throws a few permission errors before stopping, eg:

2022-06-06T17:39:18.252463Z ERROR spin_loader::local::assets: Error copying asset file  '/home/<snip>/spin-fileserver/.git/objects/pack/pack-0871eab75bd2eabcbf84e0e0fe6c77cf498c73a0.idx'

Caused by:
    Permission denied (os error 13)
2022-06-06T17:39:18.252503Z ERROR spin_loader::local::assets: Error copying asset file  '/home/<spin>/fermyon/spin-fileserver/.git/objects/pack/pack-0871eab75bd2eabcbf84e0e0fe6c77cf498c73a0.pack'

Caused by:
    Permission denied (os error 13)
Error: Failed to prepare configuration

Caused by:
    Error copying assets: 2 file(s) not copied

Looking at the spin.toml file, I assume the component's files attribute is to blame:

[[component]]
files = ["**/*"]

Re-write unit tests as integration tests

In a review for the fallback path feature, I received feedback to switch from std::env::var calls to spin_sdk::config::get calls for the configurable fallback path. Once I made this change, I was no longer able to run unit tests locally because the spin sdk only has the wasm32-wasi target. In order to use Spin Configuration, we should re-write the unit tests as integration tests because they currently only interact with the HTTP endpoint anyways and that will allow us to use spin's configuration. There will be another task for switching from environment variables to spin config but this issue is the predecessor to that work.

Source: #27

Add support for custom redirects

Users who want to host their apps using spin-fileserver may wish to configure custom redirects. This is especially useful when users may want to modify URL structures or relocate content that is already indexed by different search engines or where external links are pointed to.

Based on feedback provided by @jpflueger on Discord, I tried to find a spec that defines how redirects should be configured. However, I was not able to find anything that fits the needs.

As an alternative I would suggest outsourcing the configuration to a dedicated .toml file (e.G. redirects.toml). Users could configure redirects as shown in the following snippet:

[[redirects]]
source = "old.html"
destination = "new.html"
status = 301

[[redirects]]
source = "old2.html"
destination = "new2.html"
status = 302

# have another source pointing to the desitination new2.html
[[redirects]]
source = "old3.html"
destination = "new2.html"
status = 302

In spin-fileserver, the toml crate could be used to deserialise the configuration into a vector of a corresponding Redirect struct.

I looked at the current implementation of spin-fileserver and tried to come up with a proper way of altering the current implementation to support handling redirects while minimising modifications to the existing code-base.

I would suggest modifying the resolve function of FileServer to return Option<PathBuf, StatusCode> instead of Option<PathBuf>, this would allow handling redirects during resolution.

FileServer::send could pickup the status code received as part of the tuple returned by resolve and send proper status code along with the actual content.

The corresponding part of server would be changed to something like this:

let doc = match FileServer::resolve(path) {
    Some(res) => (FileServer::read(&res.0, &enc).ok(), res.1),
    None => (None, StatusCode::NOT_FOUND),
};

let etag = FileServer::get_etag(doc.0.clone());
FileServer::send(doc.0, path, enc, &etag, if_none_match, doc.1)

Although using a tuple would be a simple solution, using a custom struct here would be way more readable.

Support cache revalidation headers (ETag/Last-Updated)

Support either (or both) of:

• etag/if-none-match: Server sends etag header that must change if the file changes (e.g. hash of the file contents). If a client sends if-none-match with a matching value, server may return empty 204 Not Modified. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/If-None-Match
• last-modified/if-modified-since: Server sends last-modified with file modification timestamp. If a client sends if-modified-since >= modification timestamp server may return empty 204 Not Modified. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/If-Modified-Since

content-type header application/javascript; charset=utf-8?

Request appending "; charset=utf-8" to the content-type header if the mime type value is application/javascript.

Asserting that .js files would be in UTF-8 across the board...

Are there other mine types that should get the same/similar treatment?

Thanks!

Use `PATH_INFO` instead of request URI

The URI field in a request in Spin now contains the absolute path of the URI, which means it can no longer be used reliably to select files (if the application has a base path, or if the component defines complex routes).

The PATH_INFO header should be used for this instead.

Plans and time frame for release `0.0.4`?

Hey @jpflueger,

do you have plans on cutting a new release (0.0.4)? Since 0.0.3, several smaller features/improvements found its way into main.

Are there other features that you want to be implemented in order to release a new version?

Switch configuration from environment variables to spin configuration

In a review for the fallback path feature, I received feedback to use spin configuration instead of environment variables. I ran into issues with the unit tests as spin config only has a target for wasm32-wasi. To follow up on that pull request, I've created this issue to change from environment variables to spin configuration.

Source: #27
Predecessor: #28

Add tests

This repository should have a few simple integration tests to ensure the right files are served on the right routes with the right content types.