aspenweb / aspen.py

A filesystem dispatch library for Python web frameworks

Aspen is a filesystem router for Python web frameworks. See aspen.io for framework wrappers.

Installation, Testing, and License

aspen is available on GitHub and on PyPI:

$ pip install aspen

We test against 64-bit Python 3.6, 3.7, 3.8 and 3.9 on Linux:

And we test against 32- and 64-bit Python 3.6, 3.7, 3.8 and 3.9 on Windows:

aspen is MIT-licensed.

aspen.py's Issues

Compression of static files

Aspen supports storing static files in RAM. Compressing this data with brotli or gzip could save both RAM and CPU cycles.

Push a first release of the new aspen

Porting pando to aspen (AspenWeb/pando.py#548) is complicated by the fact that the new aspen hasn't been released yet.

Repoint aspen.io to RtD

We still link to https://hackerone.com/gratipay from the footer of http://aspen.io/. Rather than update in docs/, I thought it might just be easier to switch http://aspen.io/ to point to http://aspen.readthedocs.io/en/latest/ instead of Heroku. Any objections, @Changaco?

Unexpected `output` in template

I am upgrading a (closed-source) project to Aspen 1.0rc2. In one simplate I conditionally assign a variable output in the request-time Python section, and then conditionally show a message based on the truthiness of the variable. Under 1.0rc2, there is an output object in the default simplate context, so the value is always truthy. I know we've touched on this area before, but I forget where things stand: am I supposed to know this somehow and just be careful with a name like that?

I think the only place we ever really hit this was with response, because we'd do an HTTP call from inside a simplate and bind the result to response, which would overwrite the in-progress Aspen response. Should we rename this to aspen_output to avoid the conflict? Should we prefix other names, like path and qs?

if foo:
    output = 'bar'
[---]
{% if output %}{{ output }}{% endif %}

Not optimized for production use

When changes_reload is False: resources.get() should not touch the filesystem at all (no stat() call), and the dispatcher shouldn't touch the filesystem either (it should build a dispatch tree in RAM when the RequestProcessor is created).

The syntax of path variables is ambiguous

When the dispatcher encounters a file named %varname.xxx.spt, it can't know for sure whether xxx is a typecast or a file extension. Should we change the separator used for typecasting from a dot . to a colon :?

Static.render doesn't check the request method

I think it should raise a 405 if the method is not GET or HEAD.

Pure-python alternative to simplates

Adding support for pure-python (.py) dynamic resources, in addition to simplates, could be a solution to several issues. Here's what it could look like:

import things

do_one_time_stuff
a_global_var = None

def GET(website, response):  # dependency injection here
    if bad:
        return response.error(400, 'xxx')
    do_GET_stuff
    return locals()

def POST(request, response):
    do_POST_stuff
    response.redirect('…')

PAGES = []

PAGES.append("""text/html
<p>Lorem ipsum</p>
""")

PAGES.append("""application/json
{'lorem': 'ipsum'}
""")

# or

PAGES = """
[---] text/html
<p>Lorem ipsum</p>
[---] application/json
{'lorem': 'ipsum'}
"""

Notes:

• it's a little more verbose than a simplate, but not much
• variables aren't passed implicitly to the templates by default (AspenWeb/pando.py#215)
• pyflakes friendly (no need for AspenWeb/pando.py#30 and AspenWeb/pando.py#546)
• also more friendly to other tools, e.g. test coverage measurement
• single source of truth for supported methods (that's even better than AspenWeb/pando.py#204)
• possible to decorate functions
• for proper error reporting we can scan the file to determine the line number each template starts at
• syntax highlighting of templates is still possible, but instead of making a new syntax highlighter you need to modify the python one

@whit537 What do you think? Why did you originally go with a custom file format? Is there some issue with using pure python that I missed/forgot?

What about a rc3 release to PyPI to finalize support for Python 3?

Last release was 9 months ago... https://github.com/AspenWeb/aspen.py/releases

`mmap` for static files

Aspen currently supports two possibilities for static files: open and read them every time they're requested; or store their contents in Python bytestrings. Mapping them using mmap could be an interesting third possibility, and probably a good default for files that aren't too large.

Efficient file transfer isn't implemented

WSGI supports Optional Platform-Specific File Handling, i.e. sendfile, through the environ key wsgi.file_wrapper. It would be nice if Aspen used that.

The number of system calls made by dispatchers could be reduced

Python 3.5 added a new function, os.scandir, that Aspen's dispatchers could use instead of the less efficient os.listdir.

No easy way to pass the output of a single simplate page through a filter

In Liberapay we have a jinja2_html_jswrapped renderer, it could be nice to be able to replace it with something like via jinja2 | jswrap in the simplate. jswrap would be a simple function, either defined/imported in the simplate's first page, or registered globally like renderers.

Configuration is still taken from the environment

We went with that because it was the simple backward-compatible solution, but eventually Aspen should probably stop using environment variables.

Simplates are assumed to be ASCII even under python 3

In Python 3.0 the default encoding was changed from ASCII to UTF-8 (PEP 3120), so a project that no longer supports Python 2.7 should be able to remove all comments like # coding: utf8, but currently the ones in simplates can't be pruned because Aspen always assumes ASCII by default.

Drop state chain?

The aspen.request_processor.algorithm module only contains 7 functions, and they're all very simple. I'm not sure using a state chain in Aspen still makes sense, framework wrappers may be better off gluing the pieces together themselves.

README isn't rendered correctly on PyPI

Link.

(Reticketed from #2 (comment).)

Broken test

https://travis-ci.org/AspenWeb/aspen.py/builds/210710915

The dispatcher should not raise redirects

It should only give the canonical URL in the DispatchResult, then a state chain function can turn that into redirects if desired. This would remove the need for an ugly function in Liberapay: https://github.com/liberapay/liberapay.com/blob/master/liberapay/utils/state_chain.py#L85-L125

Remove pjz as Maintainer of Aspen on pypi

@whit537 needs to go to the pypi aspen page and under 'roles' remove 'pjz' as a Maintainer.

I changed the 'owner' of aspen-{pystache,jinja2} to whit537 there.

Keeping static files open

Aspen currently supports keeping the contents of static files in RAM (the store_static_files_in_ram setting). I think it should also support keeping open file descriptors instead. It would be a middle ground alternative that improves performance and reliability compared to opening the files over and over, without wasting RAM like store_static_files_in_ram does.

The performance problem of opening the files over and over is that we have to call os.path.realpath() every time to check that the file isn't outside the allowed directories. That function is slow because it has to make a system call for every parent directory (and because it's written in pure Python, but it would still be slowish if it was written in C).

The names of the codec settings are confusing

The charset_dynamic and charset_static configuration knobs are improperly named, see #9 (diff) for details.

Static resources are always cached in RAM

We need a configuration knob to turn this on/off. The default should probably be off instead of on.

Related issue: AspenWeb/pando.py#127 (if you don't cache the file in RAM you have to deal with the fact that it can be modified by another process).

3.9 dropped support for Aspen "magic directories"

I was wondering if this relates to you guys. Looks like the link in the venv creation code is dead. From some googling and some common sense I deduced this might be the project that has been referred to.

Please see
https://bugs.python.org/issue37663
python/cpython#14941

If I'm correct, I suggest you propose a fix to revert the issue.