inhabitedtype / ocaml-webmachine

A REST toolkit for OCaml

ocaml-webmachine

ocaml-webmachine is a layer on top of cohttp that implements a state-machine-based HTTP request processor. It's particularly well-suited for writing RESTful APIs. As the name suggests, this is an OCaml port of the webmachine project.

Installation

Install the library and its depenencies via OPAM:

opam install webmachine

Getting Started

webmachine implements this decision diagram to determine how an HTTP request should be handled. This includes validation, authentication, content negotiation, and caching. A resource specifies the decision that should be made at each node in the diagram by defining the appropriate method in a resource subclass. The correspondence is suggested by the name of the method for now. This will be better-documented in the future.

Examples

To build the examples in the examples/ subdirectory:

dune build examples/hello_lwt.exe
dune build examples/crud_lwt.exe
dune build examples/hello_async.exe

Development

To install development dependencies, pin the package from the root of the repository:

opam pin add -n webmachine .
opam install --deps-only webmachine

After this, you may install a development version of the library using the install command as usual.

For building and running the tests during development, you will need to install the oUnit package and run tests:

opam install oUnit
dune runtest

License

BSD3, see LICENSE file for its text.

ocaml-webmachine's Issues

webmachine 0.3.1 can't be installed by opam with ocaml 4.02.3 and 4.03.0

Looks like there is an incompatibility with cohttp.

Installation logs:

4.02.3:

$ opam install webmachine
The following actions will be performed:
  - install conf-m4         1                           [required by ocamlfind]
  - install result          1.2                         [required by dispatch]
  - install ocamlbuild      0                           [required by dispatch]
  - install ocamlfind       1.6.2                       [required by webmachine]
  - install cmdliner        0.9.8                       [required by cohttp]
  - install ppx_tools       5.0+4.02.0                  [required by ppx_fields_conv]
  - install magic-mime      1.0.0                       [required by cohttp]
  - install js-build-tools  113.33.04                   [required by fieldslib, ppx_fields_conv]
  - install dispatch        0.3.0                       [required by webmachine]
  - install cppo            1.3.2                       [required by ocplib-endian]
  - install base-bytes      base                        [required by cohttp]
  - install sexplib         113.33.03                   [required by cohttp]
  - install ppx_core        113.33.03                   [required by ppx_fields_conv]
  - install fieldslib       113.33.03                   [required by cohttp]
  - install ppx_deriving    3.3                         [required by ppx_type_conv]
  - install stringext       1.4.2                       [required by cohttp]
  - install re              1.6.1                       [required by webmachine]
  - install ocplib-endian   0.8                         [required by cstruct]
  - install base64          2.0.0                       [required by cohttp]
  - install ppx_optcomp     113.33.03                   [required by conduit]
  - install cstruct         2.1.0                       [required by conduit]
  - install ppx_driver      113.33.03                   [required by conduit]
  - install ppx_type_conv   113.33.03                   [required by ppx_fields_conv]
  - install ppx_sexp_conv   113.33.03                   [required by cohttp]
  - install ppx_fields_conv 113.33.03                   [required by cohttp]
  - install uri             1.9.2                       [required by cohttp]
  - install ipaddr          2.7.0                       [required by conduit]
  - install conduit         0.12.0                      [required by cohttp]
  - install cohttp          0.21.0                      [required by webmachine]
  - install webmachine      0.3.1     
===== 30 to install =====
Do you want to continue ? [Y/n] y

=-=- Gathering sources =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
[base64] Archive in cache
[cmdliner] Archive in cache
[cohttp] Archive in cache
[conduit] Archive in cache
[cppo] Archive in cache
[default] https://opam.ocaml.org/archives/fieldslib.113.33.03+opam.tar.gz downloaded
[ipaddr] Archive in cache
[default] https://opam.ocaml.org/archives/dispatch.0.3.0+opam.tar.gz downloaded
[magic-mime] Archive in cache
[default] https://opam.ocaml.org/archives/js-build-tools.113.33.04+opam.tar.gz downloaded
[ocplib-endian] Archive in cache
[default] https://opam.ocaml.org/archives/ppx_core.113.33.03+opam.tar.gz downloaded
[ppx_deriving] Archive in cache
[default] https://opam.ocaml.org/archives/cstruct.2.1.0+opam.tar.gz downloaded
[default] https://opam.ocaml.org/archives/ocamlfind.1.6.2+opam.tar.gz downloaded
[default] https://opam.ocaml.org/archives/ppx_driver.113.33.03+opam.tar.gz downloaded
[default] https://opam.ocaml.org/archives/ppx_fields_conv.113.33.03+opam.tar.gz downloaded
[default] https://opam.ocaml.org/archives/ppx_optcomp.113.33.03+opam.tar.gz downloaded
[default] https://opam.ocaml.org/archives/ppx_tools.5.0+4.02.0+opam.tar.gz downloaded
[default] https://opam.ocaml.org/archives/ppx_sexp_conv.113.33.03+opam.tar.gz downloaded
[result] Archive in cache
[default] https://opam.ocaml.org/archives/ppx_type_conv.113.33.03+opam.tar.gz downloaded
[stringext] Archive in cache
[uri] Archive in cache
[webmachine] Archive in cache
[default] https://opam.ocaml.org/archives/re.1.6.1+opam.tar.gz downloaded
[default] https://opam.ocaml.org/archives/sexplib.113.33.03+opam.tar.gz downloaded

=-=- Processing actions -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
-> installed conf-m4.1
-> installed ocamlbuild.0
-> installed cmdliner.0.9.8
-> installed ocamlfind.1.6.2
-> installed base-bytes.base
-> installed base64.2.0.0
-> installed cppo.1.3.2
-> installed js-build-tools.113.33.04
-> installed fieldslib.113.33.03
-> installed magic-mime.1.0.0
-> installed ocplib-endian.0.8
-> installed ppx_tools.5.0+4.02.0
-> installed ppx_core.113.33.03
-> installed ppx_deriving.3.3
-> installed ppx_optcomp.113.33.03
-> installed ppx_driver.113.33.03
-> installed ppx_type_conv.113.33.03
-> installed ppx_fields_conv.113.33.03
-> installed re.1.6.1
-> installed result.1.2
-> installed dispatch.0.3.0
-> installed sexplib.113.33.03
-> installed cstruct.2.1.0
-> installed ppx_sexp_conv.113.33.03
-> installed ipaddr.2.7.0
-> installed stringext.1.4.2
-> installed uri.1.9.2
-> installed conduit.0.12.0
-> installed cohttp.0.21.0
[ERROR] The compilation of webmachine failed at "ocaml setup.ml -build".
Processing 30/30: [webmachine: ocamlfind remove]
#=== ERROR while installing webmachine.0.3.1 ==================================#
# opam-version 1.2.2
# os           linux
# command      ocaml setup.ml -build
# path         /home/louis/.opam/ocamlirc/build/webmachine.0.3.1
# compiler     4.02.3
# exit-code    1
# env-file     /home/louis/.opam/ocamlirc/build/webmachine.0.3.1/webmachine-5618-09199b.env
# stdout-file  /home/louis/.opam/ocamlirc/build/webmachine.0.3.1/webmachine-5618-09199b.out
# stderr-file  /home/louis/.opam/ocamlirc/build/webmachine.0.3.1/webmachine-5618-09199b.err
### stdout ###
# [...]
# /home/louis/.opam/ocamlirc/bin/ocamlfind ocamldep -package re.str -package re -package dispatch -package cohttp -modules lib/webmachine.mli > lib/webmachine.mli.depends
# /home/louis/.opam/ocamlirc/bin/ocamlfind ocamlc -c -g -annot -bin-annot -package re.str -package re -package dispatch -package cohttp -I lib -o lib/webmachine.cmi lib/webmachine.mli
# /home/louis/.opam/ocamlirc/bin/ocamlfind ocamldep -package re.str -package re -package dispatch -package cohttp -modules lib/webmachine.ml > lib/webmachine.ml.depends
# /home/louis/.opam/ocamlirc/bin/ocamlfind ocamldep -package re.str -package re -package dispatch -package cohttp -modules lib/wm_util.ml > lib/wm_util.ml.depends
# /home/louis/.opam/ocamlirc/bin/ocamlfind ocamlc -c -g -annot -bin-annot -package re.str -package re -package dispatch -package cohttp -I lib -o lib/wm_util.cmo lib/wm_util.ml
# /home/louis/.opam/ocamlirc/bin/ocamlfind ocamlc -c -g -annot -bin-annot -package re.str -package re -package dispatch -package cohttp -I lib -o lib/webmachine.cmo lib/webmachine.ml
# + /home/louis/.opam/ocamlirc/bin/ocamlfind ocamlc -c -g -annot -bin-annot -package re.str -package re -package dispatch -package cohttp -I lib -o lib/webmachine.cmo lib/webmachine.ml
# File "lib/webmachine.ml", line 65, characters 24-35:
# Error: Unbound record field Request.uri
# Command exited with code 2.
### stderr ###
# E: Failure("Command ''/home/louis/.opam/ocamlirc/bin/ocamlbuild' lib/webmachine.cma lib/webmachine.cmxa lib/webmachine.a lib/webmachine.cmxs -use-ocamlfind -tag debug' terminated with error code 10")



=-=- Error report -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
The following actions failed
  - install webmachine 0.3.1
The following changes have been performed
  - install base-bytes      base      
  - install base64          2.0.0     
  - install cmdliner        0.9.8     
  - install cohttp          0.21.0    
  - install conduit         0.12.0    
  - install conf-m4         1         
  - install cppo            1.3.2     
  - install cstruct         2.1.0     
  - install dispatch        0.3.0     
  - install fieldslib       113.33.03 
  - install ipaddr          2.7.0     
  - install js-build-tools  113.33.04 
  - install magic-mime      1.0.0     
  - install ocamlbuild      0         
  - install ocamlfind       1.6.2     
  - install ocplib-endian   0.8       
  - install ppx_core        113.33.03 
  - install ppx_deriving    3.3       
  - install ppx_driver      113.33.03 
  - install ppx_fields_conv 113.33.03 
  - install ppx_optcomp     113.33.03 
  - install ppx_sexp_conv   113.33.03 
  - install ppx_tools       5.0+4.02.0
  - install ppx_type_conv   113.33.03 
  - install re              1.6.1     
  - install result          1.2       
  - install sexplib         113.33.03 
  - install stringext       1.4.2     
  - install uri             1.9.2     

The former state can be restored with:
    opam switch import "~/.opam/ocamlirc/backup/state-20160521122246.export"

4.03.0

$ opam install webmachine
The following actions will be performed:
  - install conf-m4         1                           [required by ocamlfind]
  - install result          1.2                         [required by dispatch]
  - install ocamlbuild      0.9.2                       [required by dispatch]
  - install ocamlfind       1.6.2                       [required by webmachine]
  - install cmdliner        0.9.8                       [required by cohttp]
  - install sexplib         113.33.00+4.03              [required by cohttp]
  - install ppx_tools       5.0+4.03.0                  [required by ppx_fields_conv]
  - install magic-mime      1.0.0                       [required by cohttp]
  - install fieldslib       113.24.00                   [required by cohttp]
  - install dispatch        0.3.0                       [required by webmachine]
  - install cppo            1.3.2                       [required by ocplib-endian]
  - install base-bytes      base                        [required by cohttp]
  - install ppx_core        113.33.01+4.03              [required by ppx_fields_conv]
  - install ppx_deriving    3.3                         [required by ppx_type_conv]
  - install stringext       1.4.2                       [required by cohttp]
  - install re              1.6.1                       [required by webmachine]
  - install ocplib-endian   0.8                         [required by cstruct]
  - install base64          2.0.0                       [required by cohttp]
  - install ppx_optcomp     113.33.00+4.03              [required by conduit]
  - install cstruct         2.1.0                       [required by conduit]
  - install ppx_driver      113.33.01+4.03              [required by conduit]
  - install ppx_type_conv   113.33.01+4.03              [required by ppx_fields_conv]
  - install ppx_sexp_conv   113.33.01+4.03              [required by cohttp]
  - install ppx_fields_conv 113.33.00+4.03              [required by cohttp]
  - install uri             1.9.2                       [required by cohttp]
  - install ipaddr          2.7.0                       [required by conduit]
  - install conduit         0.12.0                      [required by cohttp]
  - install cohttp          0.21.0                      [required by webmachine]
  - install webmachine      0.3.1         
===== 29 to install =====
Do you want to continue ? [Y/n] y

=-=- Gathering sources =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
[base64] Archive in cache
[cmdliner] Archive in cache
[cohttp] Archive in cache
[conduit] Archive in cache
[cppo] Archive in cache
[cstruct] Archive in cache
[dispatch] Archive in cache
[ipaddr] Archive in cache
[magic-mime] Archive in cache
[ocamlbuild] Archive in cache
[ocamlfind] Archive in cache
[ocplib-endian] Archive in cache
[ppx_core] Archive in cache
[ppx_deriving] Archive in cache
[ppx_driver] Archive in cache
[ppx_fields_conv] Archive in cache
[ppx_optcomp] Archive in cache
[ppx_sexp_conv] Archive in cache
[ppx_type_conv] Archive in cache
[re] Archive in cache
[result] Archive in cache
[sexplib] Archive in cache
[stringext] Archive in cache
[uri] Archive in cache
[webmachine] Archive in cache
Processing 29/29: [fieldslib: from default] [ppx_tools: from default]
[default] https://opam.ocaml.org/archives/fieldslib.113.24.00+opam.tar.gz downloaded
[default] https://opam.ocaml.org/archives/ppx_tools.5.0+4.03.0+opam.tar.gz downloaded

=-=- Processing actions -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
-> installed conf-m4.1
-> installed ocamlbuild.0.9.2
-> installed cmdliner.0.9.8
-> installed ocamlfind.1.6.2
-> installed base-bytes.base
-> installed base64.2.0.0
-> installed cppo.1.3.2
-> installed fieldslib.113.24.00
-> installed magic-mime.1.0.0
-> installed ocplib-endian.0.8
-> installed ppx_tools.5.0+4.03.0
-> installed ppx_core.113.33.01+4.03
-> installed ppx_deriving.3.3
-> installed ppx_optcomp.113.33.00+4.03
-> installed ppx_driver.113.33.01+4.03
-> installed ppx_type_conv.113.33.01+4.03
-> installed ppx_fields_conv.113.33.00+4.03
-> installed re.1.6.1
-> installed result.1.2
-> installed dispatch.0.3.0
-> installed sexplib.113.33.00+4.03
-> installed cstruct.2.1.0
-> installed ppx_sexp_conv.113.33.01+4.03
-> installed ipaddr.2.7.0
-> installed stringext.1.4.2
-> installed uri.1.9.2
-> installed conduit.0.12.0
-> installed cohttp.0.21.0
[ERROR] The compilation of webmachine failed at "ocaml setup.ml -build".
Processing 29/29: [webmachine: ocamlfind remove]
#=== ERROR while installing webmachine.0.3.1 ==================================#
# opam-version 1.2.2
# os           linux
# command      ocaml setup.ml -build
# path         /home/louis/.opam/403playground/build/webmachine.0.3.1
# compiler     4.03.0
# exit-code    1
# env-file     /home/louis/.opam/403playground/build/webmachine.0.3.1/webmachine-24443-527847.env
# stdout-file  /home/louis/.opam/403playground/build/webmachine.0.3.1/webmachine-24443-527847.out
# stderr-file  /home/louis/.opam/403playground/build/webmachine.0.3.1/webmachine-24443-527847.err
### stdout ###
# [...]
# File "lib/webmachine.ml", line 65, characters 24-35:
# Error: Unbound record field Request.uri
# Command exited with code 2.
# + ocamlfind ocamlopt unix.cmxa -I /home/louis/.opam/403playground/lib/ocamlbuild /home/louis/.opam/403playground/lib/ocamlbuild/ocamlbuildlib.cmxa -linkpkg myocamlbuild.ml /home/louis/.opam/403playground/lib/ocamlbuild/ocamlbuild.cmx -o myocamlbuild
# File "myocamlbuild.ml", line 518, characters 43-62:
# Warning 3: deprecated: Ocamlbuild_plugin.String.uncapitalize
# Use String.uncapitalize_ascii instead.
# File "myocamlbuild.ml", line 531, characters 51-70:
# Warning 3: deprecated: Ocamlbuild_plugin.String.uncapitalize
# Use String.uncapitalize_ascii instead.
### stderr ###
# [...]
# File "./setup.ml", line 1831, characters 16-35:
# Warning 3: deprecated: String.uncapitalize
# Use String.uncapitalize_ascii instead.
# File "setup.ml", line 5847, characters 11-28:
# Warning 3: deprecated: String.capitalize
# Use String.capitalize_ascii instead.
# File "setup.ml", line 5848, characters 11-30:
# Warning 3: deprecated: String.uncapitalize
# Use String.uncapitalize_ascii instead.
# E: Failure("Command ''/home/louis/.opam/403playground/bin/ocamlbuild' lib/webmachine.cma lib/webmachine.cmxa lib/webmachine.a lib/webmachine.cmxs -use-ocamlfind -tag debug' terminated with error code 10")



=-=- Error report -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
The following actions failed
  - install webmachine 0.3.1
The following changes have been performed
  - install base-bytes      base          
  - install base64          2.0.0         
  - install cmdliner        0.9.8         
  - install cohttp          0.21.0        
  - install conduit         0.12.0        
  - install conf-m4         1             
  - install cppo            1.3.2         
  - install cstruct         2.1.0         
  - install dispatch        0.3.0         
  - install fieldslib       113.24.00     
  - install ipaddr          2.7.0         
  - install magic-mime      1.0.0         
  - install ocamlbuild      0.9.2         
  - install ocamlfind       1.6.2         
  - install ocplib-endian   0.8           
  - install ppx_core        113.33.01+4.03
  - install ppx_deriving    3.3           
  - install ppx_driver      113.33.01+4.03
  - install ppx_fields_conv 113.33.00+4.03
  - install ppx_optcomp     113.33.00+4.03
  - install ppx_sexp_conv   113.33.01+4.03
  - install ppx_tools       5.0+4.03.0    
  - install ppx_type_conv   113.33.01+4.03
  - install re              1.6.1         
  - install result          1.2           
  - install sexplib         113.33.00+4.03
  - install stringext       1.4.2         
  - install uri             1.9.2         

The former state can be restored with:
    opam switch import "~/.opam/403playground/backup/state-20160521123030.export"

Finishing porting Erlang tests

The decision diagram tests in lib_test/test_logic.ml contain commented out tests from the Erlang project. Some will not be applicable to the OCaml version, in which case they should be deleted from the file. However the majority of them should be ported over to OCaml and become part of the test suite.

Implement parser for HTTP date format

See the specification in RFC7231§7.1.1.1. Angstrom may help in this effort, or maybe something hand-rolled would work just as well.

Illustrate the use of forms

We would need a very simple example showing how to prepare a simple form and handling the corresponding post request. Demonstrating how to post files would also be a plus.

As a toy example, we could provide a service counting the length of an input string or of a file.

Wm_util.Date?

while reviewing the diff between 0.5 and 0.6, I couldn't resist to lookup the specification of HTTP 1.1 and especially the Date header (see (obsoleted) RFC 2616 Sec 3.3 and RFC 7231 Section 7.1.1.1) as references):

• An HTTP-date value represents time as an instance of Coordinated Universal Time (UTC). -- why does the parser in Wm_util include (i.e. are there clients out there who actually set a different timezone which webmachine needs to support - and what is the reason to hardcode the 8 timezones below)?

          | "" | "Z" | "GMT" | "UTC" | "UT" -> 0
          | "PST" -> -480
          | "MST" | "PDT" -> -420
          | "CST" | "MDT" -> -360
          | "EST" | "CDT" -> -300
          | "EDT" -> -240
          | s -> Scanf.sscanf s "%c%02d%_[:]%02d" (fun sign hour min ->
              min + hour * (if sign = '-' then -60 else 60))

• HTTP-date is case sensitive. this doesn't seem to be taken care of in webmachine at all
• I fail to understand why year is bound to if year < 50 then 2000 + year else if year < 1000 then year + 1000 after %4d was used in sscanf (which parses a 4 digit number)

The function is named parse_rfc1123_date (which may actually parse a full RFC 1123 timestamp), but in HTTP the HTTP-Date is slightly different (a subset thereof, plus allowing other formats) according to the RFCs mentioned above.

Using sscanf is dangerous: there can be characters at the end of the input which are not matched by the expression, thus a date such as Mon, 10 Mar 1994 10:20:30 a0040abcdef is accepted by webmachine.

Should we revise the implementation to (a) include some test cases and (b) be more strict about its input? It is not clear to me whether there are still clients out there (which we would like to talk to) that don't use IMF-fixdate, but RFC 850 (Sunday, 06-Nov-94 08:49:37 GMT) or asctime () (Sun Nov 6 08:49:37 1994), which are both required by 7231. If this is worthwhile, I suspect using angstrom for the parser would be more convenient than the OCaml stdlib utilities. or is a dependency onto angstrom in webmachine intentionally avoided?

/cc @ansiwen @seliopou

No handler called on PUT request?

I'm not sure if it's the right way to handle PUT request but with the following code neither the to_json method nor the process_post is called with a PUT request:

class id = object(self)
  inherit [Cohttp_lwt_body.t] Wm.resource

  method allowed_methods rd =
    Wm.continue [`GET; `POST; `PUT; `DELETE] rd

  method content_types_provided rd =
    Wm.continue [
      ("application/json", self#to_json);
    ] rd

  method content_types_accepted rd =
    Wm.continue [
      ("application/json", (Wm.continue true));
    ] rd

  method process_post rd =
    print_endline "  process_post method  " ; flush_all () ;
    Wm.continue true

  method private to_json rd =
    print_endline "  to_json method  " ; flush_all () ;
    let json = ... in
    Wm.continue (`String json) rd

end

The states path is: v3b13, v3b12, v3b11, v3b10, v3b9, v3b8, v3b7, v3b6, v3b5, v3b4, v3b3, v3c3, v3c4, v3d4, v3d5, v3e5, v3f6, v3f7, v3g7, v3g8, v3h10, v3i12, v3l13, v3m16, v3n16, v3o16, v3o14, v3p11, v3o20

Implement missing POST functionality

Just a heads up I'm going part way through implementing this.

Basically I'm going to follow how erlang web machine does it.

content_type_provided called twice, or where to set headers

Hello,

I want to set some headers (cache-control) but there is no method to do so in the resource class. So I set it in the content_type_provided method:

class wm_file content_type blob =
  let md5 = Digest.to_hex (Digest.string blob) in
  fun () ->
  object(self)
    inherit [Cohttp_lwt_body.t] Wm.resource
    method allowed_methods rd = Wm.continue [`GET ] rd
    method content_types_provided rd =
      let resp_headers =
          Cohttp.Header.add_multi rd.Wm.Rd.resp_headers
          "cache-control" ["public"; "must-revalidate"]
      in
      let rd = { rd with Wm.Rd.resp_headers } in
      Wm.continue [ content_type, Wm.continue (`String blob) ] rd
    method content_types_accepted rd = Wm.continue [] rd
    method generate_etag rd = Wm.continue (Some md5) rd
  end

But it seems that this method is called twice (I can see it by adding print instructions). Is this a bug ?
And is there another way to specify headers ? Currently, I end up with duplicated values in the header I specify.

v3o20 condition seems reversed

An empty body should return 204, but thats not what the code does:

      | `Empty -> self#v3o18
      | _      -> self#respond ~status:`No_content ()

PUT fails when content_types_provided is empty

Hello,

I am trying to write a write only controller that supports only PUT and POST operations. To do so, I define the following:

method content_types_provided rd =
  Wm.continue [ ] rd

method content_types_accepted rd =
  Wm.continue [ "application/json", self#of_json ] rd

Interestingly, this fails to process either PUT of POST operations with the following error:

406 - PUT / - v3b13, v3b12, v3b11, v3b10, v3b9, v3b8, v3b7, v3b6, v3b5, v3b4, v3b3, v3c3, v3c4

However, if I also define "application/json" in content_types_provided, the same request works:

204 - PUT / - v3b13, v3b12, v3b11, v3b10, v3b9, v3b8, v3b7, v3b6, v3b5, v3b4, v3b3, v3c3, v3c4, v3d4, v3e5, v3f6, v3g7, v3g8, v3h10, v3i12, v3l13, v3m16, v3n16, v3o16, v3o14, v3p11, v3o20

It seems that the FSM has been implemented that way, but is that intuitively correct? I mean is it intended for a service to accept a type only if also generates it?

Thanks !

Expose connection information in Rd.t

Cohttp connection handlers take connection information as their second argument. This argument does not appear in handlers that webmachine generates because the type of this connection information is not uniformly specified in cohttp. There is a IO.conn type that cohttp exposes, but this is simply unit for the async implementation. This is different from the type of the connection argument in async connection handlers.

Ideally, the IO module type in this library would include a conn type that would specify the connection type, and that would be compatible with the lwt and async IO modules exported by cohttp.

Illustrate how to read cookies

It would be great to have an example showing how to read cookies from requests. I think I can contribute this.

Finish implementing cache handlers

Time parsing and comparison have not yet been implemented, mostly due to hesitance in adding calendarlib as a dependency. It may be worthwhile considering ptime for this use case as well.

The decision nodes in question are v3h11, v3h12, and v3l13.

Invalid Accept crashes the server

I'm not sure if this is a bug on webmachine itself or somewhere else.
Using hello_async.ml as a starting point:

$ curl -H'Accept: asdf/asdf' -v localhost:8080
> GET / HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.58.0
> Accept: asdf/asdf
>
< HTTP/1.1 406 Not Acceptable
< connection: keep-alive
< content-length: 0
<
* Connection #0 to host localhost left intact

Which is expected.

Compare with this:

$ curl -H'Accept: asdf' -v localhost:8080
* Rebuilt URL to: localhost:8080/
*   Trying ::1...
* TCP_NODELAY set
* connect to ::1 port 8080 failed: Verbinding is geweigerd
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET / HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.58.0
> Accept: asdf
>
* Empty reply from server
* Connection #0 to host localhost left intact
curl: (52) Empty reply from server

I also get this in the logs:

(((pid 9100) (thread_id 0)) "2018-02-08 13:06:33.527658191Z"
 "unhandled exception in Async scheduler"
 ("unhandled exception"
  ((monitor.ml.Error Parsing.Parse_error
    ("Raised at file \"parsing.ml\", line 138, characters 8-25" 
     "Called from file \"parsing.ml\", line 164, characters 4-28" 
     "Re-raised at file \"parsing.ml\", line 183, characters 8-17" 
     "Called from file \"lib/wm_util.ml\", line 112, characters 14-33"
     "Called from file \"lib/webmachine.ml\", line 542, characters 14-62"
     "Called from file \"src/deferred0.ml\", line 61, characters 64-69"
     "Called from file \"src/job_queue.ml\", line 159, characters 6-47" 
     "Caught by monitor Tcp.collect_errors"))
   ((pid 9100) (thread_id 2)))))

The server crashes and exits. Really scary, bad input can kill the server! I'm willing to provide a pull request, but I don't even know where parsing.ml lives :)

Illustrate how to set cookies

We need a small example illustrating how to set cookies on an answer, and possibly how to do simple session management.

Tests / build broken by cohttp 2.5.3

Test suite output:

  .......F..............................
  ==============================================================================
  Failure: test logic:7:head method not allowed
  
  405 from head method now allowed
  expected: <none> but got: GET,POST,PUT
  ------------------------------------------------------------------------------
  Ran: 38 tests in: 0.03 seconds.
  FAILED: Cases: 38 Tried: 38 Errors: 0 Failures: 1 Skip:  0 Todo: 0 Timeouts: 0.

Builds correctly with cohttp 2.5.1 and otherwise unchanged environment. I suspect the breaking change was introduced in cohttp 2.5.2. Happens on webmachine 0.6.1 and 0.6.2.

Wrong test for content_types_accepted

The same function Wm_util.MediaType.match_header is used so that the following checks are performed:

• content types provided by server are subtypes of content types required by client
• content types accepted by server are subtypes of content types provided by client.

In the second case this is wrong, as the control should be the other way: content types provided by client must be subtypes of content types accepted by server.

I will provide a patch.

examples are failing to build

Example fail to build.

Reproducing

1. git clone the project.
2. Run dev commands:

opam pin add -n webmachine .
opam install --deps-only webmachine

3. Update jbuilder to dune per command line message with dune upgrade.

4. Run examples:

dune build _build/default/examples/hello_lwt.exe
dune build _build/default/examples/crud_lwt.exe
dune build _build/default/examples/hello_async.exe

output is for first two build commands:

➜  GitHub git clone https://github.com/inhabitedtype/ocaml-webmachine.git
Cloning into 'ocaml-webmachine'...
remote: Enumerating objects: 857, done.
remote: Total 857 (delta 0), reused 0 (delta 0), pack-reused 857
Receiving objects: 100% (857/857), 281.13 KiB | 2.47 MiB/s, done.
Resolving deltas: 100% (550/550), done.
➜  GitHub ocaml-webmachine
➜  ocaml-webmachine [master]
➜  ocaml-webmachine [master]
➜  ocaml-webmachine [master]opam pin add -n webmachine .
[webmachine.0.6.1] synchronised from git+file:///Users/prisc_000/working/GitHub/ocaml-webmachine#master
webmachine is now pinned to git+file:///Users/prisc_000/working/GitHub/ocaml-webmachine#master (version 0.6.1)
➜  ocaml-webmachine [master]opam install --deps-only webmachine

<><> Synchronising pinned packages ><><><><><><><><><><><><><><><><><><><><>  🐫
[webmachine.0.6.1] no changes from git+file:///Users/prisc_000/working/GitHub/ocaml-webmachine#master

Nothing to do.
➜  ocaml-webmachine [master]opam install oUnit
[NOTE] Package ounit is already installed (current version is 2.2.1).
➜  ocaml-webmachine [master]dune runtest
File "examples/jbuild", line 1, characters 0-0:
Error: jbuild files are no longer supported, please convert this file to a
dune file instead.
Note: You can use "dune upgrade" to convert your project to dune.
➜  ocaml-webmachine [master]dune upgrade
Info: Creating file dune-project with this contents:
| (lang dune 1.0)
| (name webmachine)
Upgrading webmachine.opam...
Upgrading lib_test/jbuild to lib_test/dune...
Upgrading lib/jbuild to lib/dune...
Upgrading examples/jbuild to examples/dune...
➜  ocaml-webmachine [master*]dune runtest
  test_logic alias lib_test/runtest
......................................
Ran: 38 tests in: 0.07 seconds.
OK
test_dispatch alias lib_test/runtest
.....
Ran: 5 tests in: 0.01 seconds.
OK
➜  ocaml-webmachine [master*]dune build _build/default/examples/hello_lwt.exe
zsh: correct 'build' to '_build' [nyae]? n
File "examples/hello_lwt.ml", line 25, characters 2-55:
Error (warning 7): the method allowed_methods is overridden.
File "examples/hello_lwt.ml", line 2, characters 0-15:
Error (warning 33): unused open Cohttp_lwt.
File "examples/hello_lwt.ml", line 101, characters 16-18:
Error (warning 27): unused variable ch.
File "examples/hello_lwt.ml", line 101, characters 19-23:
Error (warning 27): unused variable conn.
File "examples/hello_lwt.ml", line 139, characters 22-26:
Error (warning 27): unused variable conn.
➜  ocaml-webmachine [master*]code .
➜  ocaml-webmachine [master*]dune build _build/default/examples/hello_lwt.exe
zsh: correct 'build' to '_build' [nyae]? n
File "examples/crud_lwt.ml", line 110, characters 9-24:
Error: Syntax error
➜  ocaml-webmachine [master*]dune build _build/default/examples/crud_lwt.exe
zsh: correct 'build' to '_build' [nyae]? n
File "examples/crud_lwt.ml", line 110, characters 2-69:
Error (warning 7): the method allowed_methods is overridden.
File "examples/crud_lwt.ml", line 121, characters 2-223:
Error (warning 7): the method process_post is overridden.
File "examples/crud_lwt.ml", line 150, characters 2-77:
Error (warning 7): the method allowed_methods is overridden.
File "examples/crud_lwt.ml", line 153, characters 2-150:
Error (warning 7): the method resource_exists is overridden.
File "examples/crud_lwt.ml", line 169, characters 2-283:
Error (warning 7): the method delete_resource is overridden.
File "examples/crud_lwt.ml", line 193, characters 16-18:
Error (warning 27): unused variable ch.
File "examples/crud_lwt.ml", line 193, characters 20-24:
Error (warning 27): unused variable conn.
File "examples/crud_lwt.ml", line 225, characters 23-27:
Error (warning 27): unused variable conn.

Thanks for any guidance updating this project.

How to return HTTP 201 (created) after a POST?

Hello,

It seems that the only path which can return a 201 is from P11 state. And in https://raw.githubusercontent.com/wiki/webmachine/webmachine/images/http-headers-status-v3.png we can see that there is a new resource check from this state. But how to implement this with webmachine? Currently, the v3p11 method only checks if a location header exists.

Thanks.

What about httpaf support?

Hello,

does webmachine work (yet!?) with httpaf -- was curious whether there is a branch for using these two projects together?

thanks for all your hard work (to @seliopou esp. and all the other contributors)!

Illustrate how to prepare a simple class serving static content

It would be interesting to show how to prepare a simple class serving static content. I will take a look at this.

Question about content_types_accepted

I have need for the following code:

  method content_types_accepted rd =
    Wm.continue [
      ("application/json;domain-type=Provision", self#provision_facility);
      ("application/json;domain-type=Activate",  self#activate_facility);
      ("application/json;domain-type=Deactivate", self#deactivate_facility);
    ] rd

The purpose is to implement a RESTful interface for CQRS. I need to dispatch on the different domain types independently of the verb used.

Any suggestions? I am new to webmachine, so I am sure I am missing something...

Thanks,
Hadil

Implement Challenge variant of auth type

As described in #60.

Cannot build from source: Cohttp_async_io is unbounded (OS X 10.11.6)

When trying to build from sources with tests enabled:

./configure --enable-examples
make clean && make

yields the following error:

Zoom: Error: Unbound module Cohttp_async_io

context:

File "examples/hello_async.ml", line 1, characters 5-13:
Warning 3: deprecated: module Core.Std
[since 2017-02] Use Core. The Std sub-module is no longer needed
File "examples/hello_async.ml", line 2, characters 5-14:
Warning 3: deprecated: module Async.Std
[since 2017-02] use Async. The Std sub-module is no longer needed
File "examples/hello_async.ml", line 5, characters 5-20:
Error: Unbound module Cohttp_async_io
Hint: Did you mean Cohttp_async?
Command exited with code 2.

even after I run opam install --deps-only webmachine:

$ opam install --deps-only webmachine

=-=- Synchronising pinned packages =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=  🐫
[webmachine] /Users/erwan/ocaml-webmachine/ already up-to-date

My uname -a :

Darwin bastiat.local 15.6.0 Darwin Kernel Version 15.6.0: Thu Jun 23 18:25:34 PDT 2016; root:xnu-3248.60.10~1/RELEASE_X86_64 x86_64

My ocaml --version:

 The OCaml toplevel, version 4.04.2

I obtain the same error when install from opam (via opam install webmachine).

Provide a body when reporting errors

When webmachine rejects a request, it usually returns a status code and no body. Firefox and Safari at least just display a blank page in this case, which is confusing for users. It would be nice to return some kind of message in this case. Even just "Error: nnn" would be better than nothing.

Corrupted PUT functionality in crud_lwt example

A PUT on localhost:8080/item/2 changes all items put item 2. Change in crud_lwt.ml on lines 74 and 75

          then begin found := true; (id', e') end
          else (id', e))

to

          then begin found := true; (id', e) end
          else (id', e'))

to solve the issue.

Dependency on calendar breaks usage with mirage

We are using webmachine in a mirage unikernel, and version 0.4.0 contains a dependency on calendar which breaks xen builds:

Error: No implementations provided for the following modules:
         Str referenced from /home/svanders/.opam/4.04.1/lib/calendar/calendarLib.cmxa(CalendarLib)
         Unix referenced from /home/svanders/.opam/4.04.1/lib/calendar/calendarLib.cmxa(CalendarLib)

Installed package versions are:

$ opam list
# Installed packages for 4.04.1:
asn1-combinators               0.1.3  Define ASN.1 grammars in OCaml
astring                        0.8.3  Alternative String module for OCaml
base                          v0.9.2  Full standard library replacement for OCam
base-bigarray                   base  Bigarray library distributed with the OCam
base-bytes                      base  Bytes library distributed with the OCaml c
base-threads                    base  Threads library distributed with the OCaml
base-unix                       base  Unix library distributed with the OCaml co
base64                         2.1.2  For OCaml
biniou                        1.0.13  Binary data format designed for speed, saf
bos                            0.1.6  Basic OS interaction for OCaml
calendar                      2.03.2  Library for handling dates and times in yo
camlp4                        4.04+1  Camlp4 is a system for writing extensible 
cmdliner                       1.0.0  Declarative definition of command line int
cohttp                        0.22.0  HTTP(S) library for Lwt, Async and Mirage
conduit                       0.15.3  Network connection library for TCP and SSL
conf-gmp                           1  Virtual package relying on a GMP lib syste
conf-m4                            1  Virtual package relying on m4
conf-perl                          1  Virtual package relying on perl
conf-pkg-config                  1.0  Virtual package relying on pkg-config inst
conf-which                         1  Virtual package relying on which
cppo                           1.5.0  Equivalent of the C preprocessor for OCaml
cpuid                          0.1.1  Detect CPU features
crunch                         2.0.0  Convert a filesystem into a static OCaml m
cstruct                        2.4.1  access C structures via a camlp4 extension
depext                         1.0.4  Query and install external dependencies of
dispatch                       0.3.0  Path-based dispatching for client- and ser
dns                           0.20.0  DNS client and server implementation
duration                       0.1.0  Conversions to various time units
easy-format                    1.2.0  High-level and functional interface to the
fieldslib                     v0.9.0  Syntax extension to define first class val
fmt                            0.8.3  OCaml Format pretty-printer combinators
fpath                          0.7.2  File system paths for OCaml
functoria                      2.0.2  A DSL to organize functor applications.
functoria-runtime              2.0.0  
gmp-xen                        6.0.0  The GNU Multiple Precision Arithmetic Libr
hashcons                       1.0.1  OCaml hash-consing library
io-page                        1.6.1  Allocate memory pages suitable for aligned
ipaddr                         2.7.2  IP (and MAC) address manipulation
jbuilder                   1.0+beta8  Fast, portable and opinionated build syste
logs                           0.6.2  Logging infrastructure for OCaml
lwt                            2.7.1  Monadic promises and concurrent I/O
magic-mime                     1.0.0  Convert file extensions to MIME types
minios-xen                       0.9  A minimal OS for running under the Xen hyp
mirage                         3.0.2  The MirageOS library operating system
mirage-block                   1.0.0  Utilities and module definitions for deali
mirage-block-lwt               1.0.0  Utilities and module definitions for deali
mirage-bootvar-xen             0.4.0  Library for reading MirageOS unikernel boo
mirage-channel                 3.0.0  MirageOS channels
mirage-channel-lwt             3.0.0  MirageOS channels
mirage-clock                   1.2.0  Libraries and module types for two kinds o
mirage-clock-freestanding      1.2.0  
mirage-clock-lwt               1.2.0  Libraries and module types for two kinds o
mirage-conduit                 2.3.1  Virtual package for the MirageOS Conduit t
mirage-console                 2.2.0  A Mirage-compatible Console library for Xe
mirage-console-lwt             2.2.0  A Mirage-compatible Console library for Xe
mirage-console-xen             2.2.0  A Mirage-compatible Console library for Xe
mirage-console-xen-proto       2.2.0  Protocol for communicating with Xen consol
mirage-device                  1.0.0  Foundational module types for devices.
mirage-dns                     2.7.0  Virtual package for the MirageOS DNS trans
mirage-entropy                 0.4.0  Entropy source for MirageOS unikernels.
mirage-flow                    1.2.0  Various implementations of the MirageOS FL
mirage-flow-lwt                1.2.0  Various implementations of the MirageOS FL
mirage-fs                      1.0.0  MirageOS filesystem utilities
mirage-fs-lwt                  1.0.0  Module type definitions for filesystems, w
mirage-http                    3.1.0  MirageOS-compatible implementation of the 
mirage-kv                      1.0.0  MirageOS utilities for interfacing with ke
mirage-kv-lwt                  1.0.0  MirageOS utilities for interfacing with ke
mirage-logs                    0.3.0  A reporter for the Logs library that write
mirage-net                     1.0.0  MirageOS TCP/IP networking library
mirage-net-lwt                 1.0.0  MirageOS TCP/IP networking library
mirage-net-xen                 1.7.0  Ethernet network device driver for MirageO
mirage-no-solo5                    1  Virtual package conflicting with mirage-so
mirage-os-shim                 3.0.0  Portable shim for MirageOS APIs
mirage-profile                 0.7.0  Collect profiling information
mirage-protocols               1.1.0  Module type definitions for network protoc
mirage-protocols-lwt           1.1.0  
mirage-random                  1.0.0  Random signatures for MirageOS, and an imp
mirage-runtime                 3.0.0  A bundle of useful runtime functions for a
mirage-stack                   1.0.0  Module type definitions for network stacks
mirage-stack-lwt               1.0.0  Module type definitions for network stacks
mirage-time                    1.0.0  Module type definitions for dealing with t
mirage-time-lwt                1.0.0  Module type definitions for dealing with t
mirage-types                   3.0.0  Module type definitions for Mirage-compati
mirage-types-lwt               3.0.0  Lwt module type definitions for Mirage-com
mirage-xen                     3.0.0  MirageOS library for Xen compilation
mirage-xen-minios              0.8.0  Xen MiniOS guest operating system library
mirage-xen-ocaml               3.0.0  MirageOS headers for the OCaml runtime
mirage-xen-posix               2.6.0  MirageOS library for posix headers
nocrypto                       0.5.4  Simpler crypto
oasis                          0.4.8  Tooling for building OCaml libraries and a
ocaml-compiler-libs           v0.9.0  OCaml compiler libraries repackaged
ocaml-migrate-parsetree          1.0  Convert OCaml parsetrees between different
ocaml-src                     4.04.0  Compiler sources
ocamlbuild                    0.11.0  OCamlbuild is a build system with builtin 
ocamlfind                      1.7.1  A library manager for OCaml
ocamlgraph                     1.8.7  A generic graph library for OCaml
ocamlify                       0.0.1  Include files in OCaml code
ocamlmod                       0.0.8  Generate OCaml modules from source files
ocb-stubblr                    0.1.1  OCamlbuild plugin for C stubs
ocplib-endian                    1.0  Optimised functions to read and write int1
ounit                          2.0.0  Unit testing framework loosely based on HU
parse-argv                     0.0.3  process strings into sets of command-line 
ppx_ast                       v0.9.1  OCaml AST used by Jane Street ppx rewriter
ppx_core                      v0.9.0  Standard library for ppx rewriters
ppx_deriving                     4.1  Type-driven code generation for OCaml >=4.
ppx_driver                    v0.9.1  Feature-full driver for OCaml AST transfor
ppx_fields_conv               v0.9.0  Generation of accessor and iteration funct
ppx_metaquot                  v0.9.0  Write OCaml AST fragment using OCaml synta
ppx_optcomp                   v0.9.0  Optional compilation for OCaml
ppx_sexp_conv                 v0.9.0  Generation of S-expression conversion func
ppx_tools                        5.0  Tools for authors of ppx rewriters and oth
ppx_tools_versioned            5.0.1  A variant of ppx_tools based on ocaml-migr
ppx_traverse_builtins         v0.9.0  Builtins for Ppx_traverse
ppx_type_conv                 v0.9.0  Support Library for type-driven code gener
ptime                          0.8.3  POSIX time for OCaml
randomconv                     0.1.0  Convert from random bytes to random native
re                             1.7.1  RE is a regular expression library for OCa
result                           1.2  Compatibility Result module
rresult                        0.5.0  Result value combinators for OCaml
sexplib                       v0.9.1  Library for serializing OCaml values to an
shared-memory-ring             1.3.0  Shared memory rings for RPC and bytestream
stdio                         v0.9.0  Standard IO library for OCaml
stringext                      1.4.3  Extra string functions for OCaml
tcpip                          3.1.0  Userlevel TCP/IP stack
tls                            0.8.0  Transport Layer Security purely in OCaml
topkg                          0.9.0  The transitory OCaml software packager
uchar                          0.0.1  Compatibility library for OCaml's Uchar mo
uri                            1.9.2  RFC3986 URI/URL parsing library
webmachine                     0.4.0  A REST toolkit for OCaml
x509                           0.5.3  X.509 certificate (RFC5280) library
xen-evtchn                     1.0.7  Xen event channel bindings.
xen-gnt                        2.2.3  Xen grant table bindings
xenstore                       1.3.0  Xenstore protocol clients and server
yojson                         1.3.3  Yojson is an optimized parsing and printin
zarith                           1.4  Implements arithmetic and logical operatio
zarith-xen                       1.4  Implements arithmetic and logical operatio

Redirect does not set the HTTP code

When calling the redirect function, the location header is set, but the return code stays 200, which is (at least for me, possibly others) confusing.

The following fairly minimal example illustrates the issue:

open Cohttp_lwt_unix
open Lwt.Infix

module Wm = struct
  module Rd = Webmachine.Rd
  include Webmachine.Make(Cohttp_lwt_unix_io)
end

class redirect () = object(self)
  inherit [Cohttp_lwt_body.t] Wm.resource

  method private to_json rd =
    Wm.Rd.redirect "https://github.com/inhabitedtype/ocaml-webmachine/issues/62" rd
    |> Wm.continue `Empty

  method content_types_provided rd =
    Wm.continue [
      "application/json", self#to_json
    ] rd

  method content_types_accepted rd =
    Wm.continue [] rd
end

let main () =
  let port = 8080 in
  let routes = [
    ("/", fun () -> new redirect ()) ;
  ] in
  let callback (ch,conn) request body =
    let open Cohttp in
    Wm.dispatch' routes ~body ~request
    >|= (function
      | None        -> (`Not_found, Header.init (), `String "Not found", [])
      | Some result -> result)
    >>= fun (status, headers, body, path) ->
      Server.respond ~headers ~body ~status ()
  in
  let config = Server.make ~callback () in
  Server.create ~mode:(`TCP(`Port port)) config
  >>= fun () -> Lwt.return_unit

let () = Lwt_main.run @@ main ()

Request from the running server via curl -v localhost:8080/ and see that the location header is sent but the return code stays 200.