fukamachi / clack

Web server abstraction layer for Common Lisp

Clack - Web Application Environment for Common Lisp

Clack is a web application environment for Common Lisp inspired by Python's WSGI and Ruby's Rack.

Usage

(defvar *handler*
    (clack:clackup
      (lambda (env)
        (declare (ignore env))
        '(200 (:content-type "text/plain") ("Hello, Clack!")))))

Open your web browser and go to http://localhost:5000/. You should get "Hello, Clack!".

To stop the server, use (clack:stop *handler*).

Command-line interface

Clack provides a script to start a web server. It's useful when you deploy to production environment.

NOTE: Install Roswell before as it depends on it.

When you execute ros install clack, it copies clackup script to $HOME/.roswell/bin. Make sure the path is in your shell $PATH.

$ ros install clack
$ which clackup
/Users/nitro_idiot/.roswell/bin/clackup

$ cat <<EOF >> app.lisp
(lambda (env)
  (declare (ignore env))
  '(200 (:content-type "text/plain") ("Hello, Clack!")))
EOF
$ clackup app.lisp
Hunchentoot server is started.
Listening on localhost:5000.

Installation

(ql:quickload :clack)

Documentation

• Quickdocs Page

Resources

• How to build a web app with Clack/Lack (1)
• jasom/clack-tutorial

Server

• Hunchentoot
• Wookie
• Toot
• Woo

How to contribute

See CONTRIBUTING.md.

See Also

• Lack: Clack application builder

Author

• Eitaro Fukamachi ([email protected])

Copyright

Copyright (c) 2011 Eitaro Fukamachi & contributors

License

Licensed under the MIT License.

clack's Issues

(intern "UPPERCASE") doesn't work on modern environment

It should be (intern (string '#:uppercase)).

URLs with %23 aren't matched by clack.util.route:match

I'm using URLs with # encoded as %23, and they are never matched by clack.util.route.

Typically:

(clack.util.route:match (clack.util.route:make-url-rule "/:id" :method 'GET) 'GET "/123#abc")

returns nil

Rename req to env

The name "req" is confusable because it has two meanings: an environment plist and an instance of clack.request . I think the former should be called "env" as in plack.

http://search.cpan.org/~miyagawa/Plack-0.9978/lib/Plack/Request.pm#env

Clack.Middleware.AccessLog

Move Clack.Request::parameters->plist to Clack.Util

Clack.Request::parameters->plist is a function for parse URL and convert into a plist.
However it is not exported, it seems to be useful for many frameworks.
It should be exported from Clack.Util.

Then, consider to implement plist->parameters, opposite from that.

clack.app.route.defroute is not defined

clack document has route sample as follows
(defroute app (req)
(GET "/" #'index)
(GET "/login" #'login)
(POST "/login" #'authorize)
(GET "/member/:id" #'member))

but defroute is not defined, otherwise defroutes is defined on clack.app.route package.

Cannot start on port 80

Clack.Handler class

stores the handler name, where listening to and the handler instance.

Clack.App.Directory

how about specifying file variables header

I think it is helpful for a slime user that each file having header like

;; -*- slime-buffer-package: "clack.component" -*-

I don't know the format is appropriate or not.
but it is helpful for the people who try to read clack.

Clack.Middleware.Conditional

http://search.cpan.org/~miyagawa/Plack-0.9974/lib/Plack/Middleware/Conditional.pm

Bind *standard-output* of the environment when a server is started

Clack.Middleware.Stdout

merge Clack.Request `uploads` into `body-parameter`

Currently, POST requests with enctype="multipart/form-data" are parsed into uploads in Clack.Request.

<form action="/" method="POST" enctype="multipart/form-data">
  <input type="text" name="name" value="fukamachi" />
  <input type="file" name="profile-icon" />
  <input type="submit" value="Send" />
</form>

(body-parameter (make-request env))
;=> NIL
(uploads (make-request env))
;=> '(("name" . "fukamachi") ("profile-icon" "/tmp/hunchentoot-1" "profile-icon.png"))

This looks strange.

Reduce dependencies

Clack depends on Drakma for Clack.Test. Because of that, Clack doesn't work on Windows.

Not only this component, there are many components should be separated from the Core, such like CL-Base64 (This is used in Clack.Middleware.Auth.Basic). In this issue, separating needless components into other systems to reduce dependencies.

Separate to Clack-Contrib or each systems

"src/contrib" should be separated from Clack core, because of the dependency. Clack core don't have to depend on Rucksack, for example.

clack.el

clack.util.route: loop warning

(defmethod compile-rule ((this <url-rule>))
  (loop with list = (split "(?::([\\w-]+)|(\\*))" (url this)
                           :with-registers-p t :omit-unmatched-p t)
        while list
        for prefix = (pop list)
        for name = (pop list)
        ...))

WARNING: LOOP: FOR clauses should occur before the loop's main body
GNU CLISP 2.48 (2009-07-28) (built 3528290451)

This would be better, in my opinion:

  (loop with list = (split "(?::([\\w-]+)|(\\*))" (url this)
                           :with-registers-p t :omit-unmatched-p t)
        for (prefix name) on list by #'cddr
        ...)

Improvement proposals

Hello,

First of all, I'd like to say, that it is very nice to see the development of this project, especially since I appreciate the simplicity of Ring, as well as similar approaches in other languages (Rack, WSGI). I also contemplated created a similar library.

But there's one major concert I wanted to raise regarding the current implementation. As the saying goes, "LISP programmers know the value of anything, and the cost of nothing". Yet I don't think, this should stand for Common Lisp programmers :) What I mean is that such software as Clack should be very robust, stable and fast, so that people could reliably build on top of it. As it stands currently, I think there are some choices in the main request handling pipeline, which are not optimal in regards of performance.

1. Using of plists for request and response data. It is a common knowledge, that working with lists, that contain more than 5-6 elements, in random-access mode is much less efficient, than using other data-structures. The obvious candidates are hash-tables, structs and objects. As the requests and responses may have fields, that cannot be anticipated, using structs or objects would require an additional level of indirection for holding additional fields, which is, obviously, overkill. So what remains is only hash-tables. I don't know, why you didn't choose them for the task - is it related to lacking syntactic support for their handling in the standard library? If so, this can be solved. In my library RUTILS I provide a readtable, that allows to use a literal syntax for creating them, like this: #{:status-code 200 :text "ok"} (source code), as well as several convenience functions for working with them, like get# (an alias of gethash), set# (an alias of (setf (gethash ...)), take#, and a utility for printing them in the same manner. These small syntactic additions make working with hash-tables just a breeze - as convenient as in other languages with native support for them, like Clojure or Python.

2. Using keywords, interned at runtime, for handling additional fields of requests. Surely, keywords are the most convenient and appropriate representation for request fields, known beforehand, but handling other fields as keywords, created with intern will make all the applications incur a significant and unnecessary performance hit. Here I'd like to refer you to the Google Common Lisp Styleguide. I think, that leaving this arguments as strings will not pose any inconveniences for the users. The only problem I see is that this will necessitate that the request hash-table has equal test instead of eql, but I think, this will not degrade its performance significantly (although this assumption needs additional testing). Yet, if it will be the case, the other approach may be to use a plain struct for the request object with the additional-headers field, designated to unanticipated headers, that can be an equal hash-table or even a plist/alist (since there shouldn't be many such headers in the majority of cases).

3. Not using multiple-values for returning results from the app. This issue is more of a convenience proposal, but still, I believe, it should be raised.

   AFAIU, currently Clack applications should return a list, consisting of a status-code, headers and body, which will be transformed by Clack stack into response plist and eventually into appropriate response objects. I think, this is both not convenient and not very efficient.

   I think it will be much clearer, as well as more efficient, if the application returned multiple-values. The first value may have a double meaning: if it's a string — it's a response body, otherwise if it's a number — it's a status code. The second value is a hash-table of response headers. The optional third value may hold a status-code.

   So the normal control flow will return just a string. The simple page not found will return 404, while a more involved example may be:

     (values "[\"/variant1\",\"/variant2\"]"
             #{:content-type "application/json"}
             300)
   

test-app hangs even with Multi-threads

https://github.com/quicklisp/quicklisp-projects/issues#issue/120

Clack.Handler.Araneida

Clack.Session.Store.File

Clack allows to store sessions only in memory.

Yason API change removes JSON package nickname.

Yason no longer has a JSON nickname, and this breaks clack's use of json:parse in request.lisp.

:mount URL of Clack.Builder

(:mount "/foo" )

Clack.Middleware.Dbi

Clack Middleware for management a connection of CL-DBI.

Can not parse Cookie header

If a content of Cookie header includes semicolons, parsing the cookie will be failed. This bug has been fixed in arielnetworks' branch. See https://github.com/arielnetworks/clack.

Clack.Handler.AppEngine

array parameters not properly parsed

credentials[login] => :|credentials%5Blogin%5D|

I'm going to send the patch in two or three days.
How to represent this? as list or as hashtable? I prefer hashtable.

Clack.Handler.Mongrel2

Implement a handler to work behind Mongrel2 (http://mongrel2.org)

(might actually do some work on it later)

Clack.Handler.Aserve

with trivial-types

https://github.com/m2ym/trivial-types

Many Clack.Middleware.Logger won't work

If many Clack.Middleware.Logger were specified in builder, the most inner logger read the log stream until it ends.

Clack.Middleware.Lint

Delayed Reponse and Streaming Body

porting Clack.App.Route's routing to Clack.Util

Clack.App.Route's routing rule is very general and reusable thing. In fact, Caveman is heavily depending on it. It must be useful for other Clack developers.

Shouldn't it have to be supported in Clack core, as Clack.Util.Route, for instance?

Lazy loading of Handler

split Clack.Util.Stream::replay-input-stream to an individual library

The idea of Clack.Util.Stream::<replay-input-stream> is really useful for general purpose. It will be an individual library, like "circular-streams".

Clack.Handler.Toot

working on progress

Expiring of sessions in clack.middleware.session

When trying to expire a session by setting the :expire keyword in the hash map :clack.session.options the <clack-session-state-cookie> middleware raises a type miss-match. This is because <clack-middleware-session> calls clack.session.state:expire with the options hash-table converted to a plist while <clack-session-state-cookie> expects the original hash-table:

;;; from core/middleware/session.lisp
(defmethod expire ((this <clack-middleware-session>) id res env)
  (state:expire
   (state this)
   id res
   (hash-table-plist (getf env :clack.session.options))))

;;; from core/middleware/state/cookie.lisp
(defmethod expire ((this <clack-session-state-cookie>)
                              id res &optional options)
  (setf (gethash :expires options) 0) ;; here is the problem
  (finalize this id res options))

Clack.Handler.Fcgi

Clack.Middleware.Csrf

For CSRF easily protection.

with CL-Syntax

CL-Syntax is an useful library for read macros and allows SLIME integration.
And, now cl-annot doesn't work properly on SLIME without CL-Syntax.

https://github.com/m2ym/cl-syntax

I will rewrite Clack using CL-Syntax after it will be added to Quicklisp (maybe next month, May).

defroutes: compile time and run time messed up

defroutes (clack.app.route)
*** - ENDP: A proper list must not end with #:PARAMS12669
GNU CLISP 2.48 (2009-07-28) (built 3528290451)

I've no idea what the behavior of defroutes is expected. One works is here:

(defmacro defroutes (name (env) &body clauses &aux (otherwise (car (last clauses))))
  (if (member (car otherwise) '(otherwise t))
      (setf clauses (butlast clauses))
    (setf otherwise nil))
  (with-gensyms (nil env request-method path-info patten method cb matched p)
        `(defun ,name (,env)
           (let ((,request-method (getf ,env :request-method))
             (,path-info (getf ,env :path-info)))
             (or ,@(loop :for (patten method cb) :in clauses
                 :collect `(multiple-value-bind (,matched ,p)
                           (match (make-url-rule ,patten :method ,method) ,request-method ,path-info)
                         (if ,matched
                         (funcall ,cb (concatenate 'list ,p ,env)))))
             ,(if otherwise
                  `(funcall ,(cadr otherwise) ,env)
                ''(404 nil nil)))))))

The signature of with-gensyms in clisp is different from others (maybe), and here I use funcall directly instead of call (could be improved if necessary).

Make clackup be able to handle pathname as an app

Write Tutorial

http://clacklisp.org/tutorial/

Does not build in SBCL due to lock on CL:METHOD

Defining and exporting a function named CL:METHOD violates 11.1.2.1.2 of the spec, and SBCL signals an error as a result. This was triggered by routes.lisp and the url-rule class.

Fail to parse a JSON Body with a Space.

Disclaimer: I'm not sure if the error is in the yason library's skip-whitespace method or in the clack package. But because the test case requieres ningle & clack I figured I report it here. By the way, I think the parsing the body with yason and storing it as an extra env attribute should probably be implemented as a middleware.

So bare bones test case is defining an endpoint that just spits the env out

(setf (ningle:route *app* "/word/" :method :PUT)
      #'(lambda (params)
          (format nil "~a" (slot-value ningle:*request* 'clack.request::env))))

and then hit the endpoint with curl:

 => curl -v -XPUT -H"Content-Type: application/json" -d "{\"cat\": \"dog\"}" http://localhost:5000/word/

and I get the following error (note that if I remove - H option the endpoint works fine):

#\  fell through ECASE expression.
Wanted one of (#\t #\f #\n #\[ #\{ #\- #\0 #\1 #\2 #\3 #\4 #\5
               #\6 #\7 #\8 #\9 #\").
   [Condition of type SB-KERNEL:CASE-FAILURE]
Restarts:
 0: [ABORT] Abort thread (#<THREAD "hunchentoot-worker-127.0.0.1:55430" RUNNING {100892DA93}>)

Upon inspecting the #\ symbol the slime inspector shows me the following:

#<STANDARD-CHAR {2049}>
--------------------
Char code: 32
Lower cased: @0=#\ 
Upper cased: @0=#\ 

It appears to be whitespace, but that is weird as the parse% method is supposed to skip-whitespace

(ecase (peek-char-skipping-whitespace input)
    (#\"
     (parse-string input))
    ((#\- #\0 #\1 #\2 #\3 #\4 #\5 #\6 #\7 #\8 #\9)
     (parse-number input))

    (#\{
     (parse-object input))
    (#\[
     (parse-array input))
    ((#\t #\f #\n)
     (parse-constant input)))))

;; ======
(defun peek-char-skipping-whitespace (input &optional (eof-error-p t))
  (skip-whitespace input)
  (peek-char nil input eof-error-p))

(defun skip-whitespace (input)
  (loop
     while (and (listen input)
                (whitespace-p (peek-char nil input)))
     do (read-char input)))

(defun whitespace-p (char)
  (member char '(#\Space #\Newline #\Tab #\Linefeed #\Return)))

Also, I checked that #\ and #\Space are equivalent

(member #\ '(#\Space))

* (eq #\Space #\ )
T

P.D. Thank you not only for your libraries but also because of your strong focus on documentation (even if it not done yet)

"body-parameter" will be NIL when create Clack.Request twice

(lambda (env)
  (body-parameter (make-request env))
  ;=> (:|name| "fukamachi")
  (make-request env)
  ;=> NIL
  )

Emit Clack.Util.Ducktype

It isn't used at all.

Does not build on SBCL due to package lock

I get this error:

unhandled SB-EXT:SYMBOL-PACKAGE-LOCKED-ERROR in thread
#<SB-THREAD:THREAD "initial thread" RUNNING {1002FC9501}>: Lock on
package COMMON-LISP violated when proclaiming CONDITION as a
function while in package CLACK.MIDDLEWARE.CONDITIONAL.

See also:
  The SBCL Manual, Node "Package Locks"
  The ANSI Standard, Section 11.1.2.1.2

File uploads on browsers causes I/O timeout

File uploads on browsers causes I/O timeout.

(clack:clackup (lambda (env) (list 200 '(:content-type "text/html") '("<html><body><form action=\"/\" method=\"post\" enctype=\"multipart/form-data\"><input name=\"myfile\" type=\"file\" /><input type=\"submit\" value=\"Submit\" /></form></body></html>"))))