Coder Social home page Coder Social logo

vase's Introduction

CircleCI

Clojars Project

Vase: Data-driven microservices

This system provides a data-driven and extensible way to describe and run HTTP APIs. Building blocks make it very easy to use Datomic as a durable database. Other databases can be added via an extension mechanism.

Major Changes Since v0.9

We learned from using Vase on real-world projects and from hearing feedback in our user community. People like the promise of Vase but found it hard to use.

This release aims to make Vase much easier for common cases:

  1. A new input format called Fern lets us produce much better error messages when something goes wrong.
  2. Step-by-step examples show how to grow an API from scratch rather than relying on the Leiningen template.
  3. A new public API allows applications to skip the files altogether and directly feed Vase with data structures to describe the application. You can produce those however you like, from code, files, values in a database... whatever.
  4. Datomic has been "demoted" by one namespace level so we can add support for other external integrations. Instead of vase/query, for example, you'll now use vase.datomic/query.

Releases and Dependency Information

If you would like to use the latest developments in Vase, you will need to clone this repository and install it locally:

git clone https://github.com/cognitect-labs/vase
cd vase
lein install

Stable versions are currently deployed to the Clojars repository.

Leiningen dependency information:

 [com.cognitect/pedestal.vase "0.9.3"]

Maven dependency information:

<dependency>
  <groupId>com.cognitect</groupId>
  <artifactId>pedestal.vase</artifactId>
  <version>0.9.3</version>
</dependency>

Before you get started

Vase is built on top of Pedestal and Datomic. While you don't need to be a Pedestal or Datomic expert to use Vase, a little introductory material goes a long way. Newcomers to either will find these resources especially helpful.

Pedestal

Pedestal is a collection of libraries for building services and applications. The core components of Pedestal are Interceptors, the Context Map and the Interceptor Chain Provider.

Interceptors implement functionality and the Context Map controls how Pedestal behaves (i.e., when interceptor chain execution terminates, going async, etc...). The Interceptor Chain Provider connects the interceptor chain to a given platform, creates an initial context and executes the interceptor chain against it. Each interceptor has access to the context map during execution which means that interceptors can alter how Pedestal behaves. What about routes?

Routes are data structures that relate request paths to interceptors. After expansion, They are consumed by a Router which is implemented as an interceptor. When a matching route is found for a given request, the interceptor(s) it relates to are enqueued on the interceptor chain.

Pedestal ships with support for the Jetty, Tomcat and Immutant platforms. Although these are all servlet-based platforms, interceptor providers can be implemented for other platforms.

It should come as no surprise that Pedestal interceptors are a crucial part of Vase so it is helpful to understand what they are and how they work.

As you dive into more advanced Vase usage scenarios, you'll benefit from a deeper understanding of Pedestal. Here's where you should look for more information:

Datomic

Datomic is a database of facts and Vase uses it as its backend store. You will immediately be confronted by three Datomic concepts as you work with Vase: schema, query and transaction. Of the three, Datomic queries offer the most variety and, possibly, confusion. Datomic uses a declarative query language called Datomic Datalog for queries. Learn Datalog Today will help you get up to speed with it.

The Datomic docs site has in-depth resources covering schema, query, transactions and more. These are good resources to dive into as you move to more advanced Vase usage scenarios.

Getting Started

Your path to get running depends on what you need to do:

  1. Just build an API for CRUD operations: Run Vase standalone.
  2. Integrate with hand-written routes: Add Vase to an existing Pedestal project.
  3. Use Vase in a new project, for more than just CRUD: Use the template
  4. Extend Vase with new actions: Create Actions.

Prerequisites

By default, Vase uses an in-memory Datomic database, using the publicly available Datomic-free version located in Clojars.

Run Vase Standalone

If you just want to run an API that does CRUD (create, read, update, delete) operations on a database, then you can run Vase directly. Download the latest uberjar JAR from Clojars and use it with java -jar.

java -jar vase-standalone.jar my-service.fern

This path does assume you're using Fern for your input syntax. Vase will look for the top-level key vase/service in your Fern environment.

Use the template

If you want to do more than CRUD, you will need a project. This repository includes a Leiningen template for getting started. Look at the template's README for local/developer setup, otherwise

lein new vase my-service

Look at `my-service/src/

Adding Vase to an Existing Pedestal project

Vase turns API descriptions into Pedestal routes. The API descriptions specify actions for routes, these are turned into interceptors in the Pedestal routes. This is done by vase/routes.

vase/routes takes an API base URL and an API specification. The routes it returns can be used directly by Pedestal like this:

(require '[io.pedestal.http.route.definition.table :as table])
(require '[com.cognitect.vase]))

(defn make-master-routes
  [spec]
  (table/table-routes
   {}
   (vase/routes "/api" spec)))

The routes that vase/routes returns can also be combined with hand-written Pedestal routes by concatenating the input to table/table-routes:

(require '[io.pedestal.http.route.definition.table :as table])
(require '[com.cognitect.vase]))

(defn make-master-routes
  [spec]
  (table/table-routes
   {}
   (concat
     [["/hello" :get [hello]]]
     (vase/routes "/api" spec))))

Documentation

Contributing

Contributing guidelines for Vase are the same as Pedestal.

For code contribution, you'll need a signed Cognitect Contributor Agreement. For small changes and doc updates, no CA is required.

If you're proposing a significant change, please open an Issue to discuss the design of the change before submitting a Pull Request.

Support

Don't hesitate to reach out if you run into issues or have questions! The Vase community can be found in the the pedestal-users mailing list or the #pedestal slack channel.

Copyright

Copyright © 2015-2018 Cognitect, Inc. All rights reserved.

vase's People

Contributors

alexanderkiel avatar bherrmann7 avatar calebmacdonaldblack avatar dball avatar ddeaguiar avatar deg avatar jimmyhmiller avatar levand avatar livtanong avatar luskwater avatar malchmih avatar mtnygard avatar ohpauleez avatar solussd avatar yokolet avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

vase's Issues

Status code does not correctly handle errors

In the last major refactor, the response generation tooling was modified in util and within the actions themselves.

The status code's error considerations should be based on the :body not on the context's :errors. A context's :errors will be consumed and handle by Pedestal or other interceptors.

Support edn coercion in vase/transact

The current Vase syntax does not make it easy to create new entities with non-string types.
:edn-coerce is supported for #vase/query, but not for #vase/transact.

So, this common operation can only be done currently with an interceptor.
This pattern appears a couple of times in the current samples, e.g. https://github.com/cognitect-labs/vase/blob/master/samples/petstore-full/src/petstore_full/interceptors.clj#L10. To me, this pattern appears verbose and fragile, especially for such a common operation.

I see several possible ways to extend the syntax:

  1. Support :edn-coerce in #vase/transact maps. But, this is awkward, since the current syntax refers only to the fully-namespaced keywords in the :properties vector, with no :params bindings.
  2. Extend the syntax of :properties. Currently this is a vector of keywords. Each keyword could optionally be a map including, e.g., :edn-coerce true.
  3. Extend the syntax of #vase/transact. Add a :params bindings clause. This can then support end coercion in a syntax exactly parallel to #vase/query. It could also offer an option to define entity-creating routes where the exposed API parameter name does not have to exactly match the internal database attribute keyword.
  4. Extend :params in both #vase/transact and #vase/query. This addresses an additional issue too. :edn-coerce feels a bit unnatural to me. It would be cleaner to have this typing request be part of each parameter in the :params vector.

Query descriptors with Datalog

In the original Vase code, it was possible to ensure the existence or absence of properties within your descriptor with Datomic's datalog. This was used for all sorts analysis, including enforcing business rules/restrictions on a Descriptor.

We need to bring that code back.

samples don't work out of the box

Description

samples only work with the newest dependencies of pedestal, clojure, and vase.

Expected Behavior

lein run produces no error

Actual Behavior

Caused by: clojure.lang.ExceptionInfo: Call to clojure.core/refer-clojure did not conform to spec:
In: [2 1] val: :as fails at: [:args :exclude :op :quoted-spec :spec] predicate: #{:exclude}

Steps to reproduce

Replace

[[org.clojure/clojure "1.9.0-alpha14"]
                 [io.pedestal/pedestal.service "0.5.2"]
                 [com.cognitect/pedestal.vase "0.9.1-SNAPSHOT"]

                 ;; Remove this line and uncomment one of the next lines to
                 ;; use Immutant or Tomcat instead of Jetty:
                 [io.pedestal/pedestal.jetty "0.5.2"]
                 ;; [io.pedestal/pedestal.immutant "0.5.2-SNAPSHOT"]
                 ;; [io.pedestal/pedestal.tomcat "0.5.2-SNAPSHOT"]

                 [ch.qos.logback/logback-classic "1.1.8" :exclusions [org.slf4j/slf4j-api]]
                 [org.slf4j/jul-to-slf4j "1.7.22"]
                 [org.slf4j/jcl-over-slf4j "1.7.22"]
                 [org.slf4j/log4j-over-slf4j "1.7.22"]]

with

[[org.clojure/clojure "1.10.0-alpha4"]
                 [io.pedestal/pedestal.service "0.5.4"]
                 [com.cognitect/pedestal.vase "0.9.4-SNAPSHOT"]

                 ;; Remove this line and uncomment one of the next lines to
                 ;; use Immutant or Tomcat instead of Jetty:
                 [io.pedestal/pedestal.jetty "0.5.4"]
                 ;; [io.pedestal/pedestal.immutant "0.5.2-SNAPSHOT"]
                 ;; [io.pedestal/pedestal.tomcat "0.5.2-SNAPSHOT"]

                 [ch.qos.logback/logback-classic "1.1.8" :exclusions [org.slf4j/slf4j-api]]
                 [org.slf4j/jul-to-slf4j "1.7.22"]
                 [org.slf4j/jcl-over-slf4j "1.7.22"]
                 [org.slf4j/log4j-over-slf4j "1.7.22"]]

or something else that works

Environment

Apple

Operating System (including version).

17.6.0 Darwin Kernel Version 17.6.0: Tue May 8 15:22:16 PDT 2018; root:xnu-4570.61.1~1/RELEASE_X86_64 x86_64

Your current Leiningen/Boot/Maven version (lein --version)

Leiningen 2.8.1 on Java 1.8.0_171 GraalVM 1.0.0-rc2

Pedestal and Vase version

com.cognitect/pedestal.vase "0.9.1-SNAPSHOT"
io.pedestal/pedestal.service "0.5.2"

HTTP status codes

Not a bug as such, but seems that Vase should follow HTTP conventions for status codes - especially for following cases:

  • Nothing found should be a 404 (currently 200) on routes that include a resource identifier in the path
  • Bad requests should be a 400 (currently 500 and exposing stack trace to client)

Vase microservices to use clj-client to connect to peers.

Description

According to Datomic documentation in a microservice environment the peers are recommended to be running outside of the microservice process that way the actual microservice memory footprint can stay small, a peer can be shared amongst microservice instances and not be restarted as often as the microservice instance itself. It looks to me that the vase services do not have the option of using the datomic API client (http://docs.datomic.com/project-setup.html).
If that's the case, are there any plans on supporting using the client right out-of-the-box in the future vase versions? If not, that it might be prohibitive for us to use Vase, since we plan on using Ditomic peers.

Vase to provide standard microservice APIs

Description

It's typical for a mature microservice framework to provide a number of endpoints out-of-the box. For instance, Dropwizard and SpringBoot provide /health, /metrics, etc. They both it easy to add custom healthchecks that are displayed in /health and /metrics displays all the metrics that can be generated from the endpoint declaration such as counts per HTTP codes for calls made per endpoint, timing statistics for the times taken by calling each endpoint. For health, for vase a default health endpoint can check connectivity to Datomic. For metrics, all the default Datomics metrics can be displayed. Dropwizard also makes it very easy to add any custom metrics explicitly in code. Our in-house web services also return their swagger-formatted documentation from /docs which can be either static and provided in a file or generated at runtime by looking at the service routes and documentations. In Java it's done by looking at the custom custom annotations.

Expected Behavior

Vase services provided health, metrics, other common endpoints out of box and makes it easy to add content to those endpoints in a standard way.

Actual Behavior

The standards endpoints are not provided.

Transact literal should accept EDN and Transit as well as JSON

Description

Expected Behavior

Vase literals should be consistent in their allowed inputs. Any of them that take data in (e.g., query and transact) should accept all the data-like representations we support:

  • JSON
  • EDN
  • Transit

Actual Behavior

The transact literal only accepts JSON payloads.

Environment

Pedestal and Vase version

Vase 0.9.1

Vasebi sample throws a NullPointerException when loading the root url.

Description

The application errors when following the Getting Started section of the README.

Expected Behavior

It should not error.

Actual Behavior

I get this error:

clojure.lang.ExceptionInfo: Interceptor Exception:
	at clojure.core$ex_info.invokeStatic(core.clj:4725)
	at clojure.core$ex_info.invoke(core.clj:4725)
...
Caused by: java.lang.NullPointerException: null
	at clojure.lang.Numbers.divide(Numbers.java:3796)
	at vasebi.server$eval17758$fn__17759$fn__17760.invoke(server.clj:45)
	at clojure.core$mapv$fn__9368.invoke(core.clj:6788)
	at clojure.lang.PersistentVector.reduce(PersistentVector.java:341)
	at clojure.core$reduce.invokeStatic(core.clj:6703)
	at clojure.core$mapv.invokeStatic(core.clj:6779)
	at clojure.core$mapv.invoke(core.clj:6779)
	at vasebi.server$eval17758$fn__17759.invoke(server.clj:45)
	at io.pedestal.interceptor.chain$try_f.invokeStatic(chain.clj:52)

Steps to reproduce

Follow the steps in the Getting Started section of the sample's README.

Environment

Operating System (including version).

macOS 10.12.3

Your current Leiningen/Boot/Maven version (lein --version)

Leiningen 2.7.1 on Java 1.8.0_112 Java HotSpot(TM) 64-Bit Server VM

Pedestal and Vase version

Refer to the sample's project.clj

[io.pedestal/pedestal.service "0.5.2"]
[com.cognitect/pedestal.vase "0.9.1-SNAPSHOT"]

your_first_api.md edits

Description

Given a route:

"/do-stuff" {:post #vase/transact {:name :example/do-stuff-to-things
                                   :properties [:a :b]}

Trying to run invoke a transaction

"curl -H "Content-Type: application/json" -X POST -d '{"payload": [{"a": "Hello", "b": "World"}]}' http://localhost:8080/api/example/do-stuff"

results in an exception

Expected Behavior

"When the transaction executes, it will create a new entity in Datomic with the entity map {:a "Hello" :b "World"}"

Actual Behavior

clojure.lang.ExceptionInfo: Interceptor Exception: java.lang.IllegalArgumentException: :db.error/not-an-entity Unable to resolve entity: :a

(full trace below)

Steps to reproduce

use the route and http request as shown above.

Operating System (including version).

os x 10.9.5

Your current Leiningen/Boot/Maven version (lein --version)

Leiningen 2.7.1 on Java 1.8.0_40 Java HotSpot(TM) 64-Bit Server VM

Pedestal and Vase version

project.clj defaults from "lein new vase

Full Error Trace

ERROR i.p.http.impl.servlet-interceptor - {:msg "error-ring-response triggered", :context {:response nil, :io.pedestal.interceptor.chain/stack (#Interceptor{:name :io.pedestal.http.impl.servlet-interceptor/stylobate} #Interceptor{:name :io.pedestal.http.impl.servlet-interceptor/terminator-injector}), :request {:request-id "YlgEGRMLPxE", :json-params {:payload [{:a "xxx", :b "yyy"}]}, :protocol "HTTP/1.1", :db datomic.db.Db@ace8a948, :async-supported? true, :remote-addr "127.0.0.1", :servlet-response #object[org.eclipse.jetty.server.Response 0x79d7c641 "HTTP/1.1 200 \nDate: Tue, 07 Feb 2017 05:39:03 GMT\r\n\r\n"], :servlet #object[io.pedestal.http.servlet.FnServlet 0x18bef68d "io.pedestal.http.servlet.FnServlet@18bef68d"], :headers {"user-agent" "curl/7.30.0", "host" "localhost:8080", "accept" "/", "content-length" "36", "content-type" "application/json", "vaserequest-id" "YlgEGRMLPxE"}, :server-port 8080, :servlet-request #object[org.eclipse.jetty.server.Request 0x1a098c30 "Request(POST //localhost:8080/api/accounts/v1/dostuff)@1a098c30"], :content-length 36, :content-type "application/json", :path-info "/api/accounts/v1/dostuff", :character-encoding "UTF-8", :received-time #object[org.joda.time.DateTime 0x2dc42ba1 "2017-02-07T05:39:03.697Z"], :url-for #object[io.pedestal.http.route$url_for_routes$fn__9809 0x2dd7f1f6 "io.pedestal.http.route$url_for_routes$fn__9809@2dd7f1f6"], :uri "/api/accounts/v1/dostuff", :server-name "localhost", :query-string nil, :path-params {}, :body #object[org.eclipse.jetty.server.HttpInputOverHTTP 0x4ffffc23 "HttpInputOverHTTP@4ffffc23[c=36,q=0,[0]=null,s=STREAM]"], :scheme :http, :request-method :post, :conn #object[datomic.peer.LocalConnection 0x70a9b689 "datomic.peer.LocalConnection@70a9b689"]}, :bindings {#'io.pedestal.http.route/url-for #object[io.pedestal.http.route$url_for_routes$fn__9809 0x2dd7f1f6 "io.pedestal.http.route$url_for_routes$fn__9809@2dd7f1f6"]}, :enter-async [#object[io.pedestal.http.impl.servlet_interceptor$start_servlet_async 0x32b1e8c4 "io.pedestal.http.impl.servlet_interceptor$start_servlet_async@32b1e8c4"]], :io.pedestal.interceptor.chain/terminators (#object[io.pedestal.http.impl.servlet_interceptor$terminator_inject$fn__12757 0x34ab12f7 "io.pedestal.http.impl.servlet_interceptor$terminator_inject$fn__12757@34ab12f7"]), :servlet-response #object[org.eclipse.jetty.server.Response 0x79d7c641 "HTTP/1.1 200 \nDate: Tue, 07 Feb 2017 05:39:03 GMT\r\n\r\n"], :route {:path "/api/accounts/v1/dostuff", :method :post, :path-re #"/\Qapi\E/\Qaccounts\E/\Qv1\E/\Qdostuff\E", :path-parts ["api" "accounts" "v1" "dostuff"], :interceptors [#Interceptor{:name :attach-received-time} #Interceptor{:name :attach-request-id} #Interceptor{:name :io.pedestal.http/json-body} #Interceptor{:name :com.cognitect.vase.datomic/insert-datomic} #Interceptor{:name :io.pedestal.http.body-params/body-params} #Interceptor{:name :forward-headers} #Interceptor{:name :accounts.v1/dostuff}], :route-name :accounts.v1/dostuff, :path-params {}, :io.pedestal.http.route.prefix-tree/satisfies-constraints? #object[clojure.core$constantly$fn__6693 0x7316ad67 "clojure.core$constantly$fn__6693@7316ad67"]}, :servlet #object[io.pedestal.http.servlet.FnServlet 0x18bef68d "io.pedestal.http.servlet.FnServlet@18bef68d"], :servlet-request #object[org.eclipse.jetty.server.Request 0x1a098c30 "Request(POST //localhost:8080/api/accounts/v1/dostuff)@1a098c30"], :url-for #object[io.pedestal.http.route$url_for_routes$fn__9809 0x2dd7f1f6 "io.pedestal.http.route$url_for_routes$fn__9809@2dd7f1f6"], :io.pedestal.interceptor.chain/execution-id 1, :servlet-config #object[org.eclipse.jetty.servlet.ServletHolder$Config 0x6335c430 "org.eclipse.jetty.servlet.ServletHolder$Config@6335c430"], :async? #object[io.pedestal.http.impl.servlet_interceptor$servlet_async_QMARK_ 0x763d616c "io.pedestal.http.impl.servlet_interceptor$servlet_async_QMARK_@763d616c"]}, :line 252}
clojure.lang.ExceptionInfo: Interceptor Exception: java.lang.IllegalArgumentException: :db.error/not-an-entity Unable to resolve entity: :a
at clojure.core$ex_info.invokeStatic(core.clj:4725)
at clojure.core$ex_info.invoke(core.clj:4725)
at io.pedestal.interceptor.chain$throwable__GT_ex_info.invokeStatic(chain.clj:33)
at io.pedestal.interceptor.chain$throwable__GT_ex_info.invoke(chain.clj:32)
at io.pedestal.interceptor.chain$try_f.invokeStatic(chain.clj:55)
at io.pedestal.interceptor.chain$try_f.invoke(chain.clj:42)
at io.pedestal.interceptor.chain$process_all_with_binding.invokeStatic(chain.clj:169)
at io.pedestal.interceptor.chain$process_all_with_binding.invoke(chain.clj:144)
at io.pedestal.interceptor.chain$process_all$fn__8786.invoke(chain.clj:186)
at clojure.lang.AFn.applyToHelper(AFn.java:152)
at clojure.lang.AFn.applyTo(AFn.java:144)
at clojure.core$apply.invokeStatic(core.clj:657)
at clojure.core$with_bindings_STAR_.invokeStatic(core.clj:1963)
at clojure.core$with_bindings_STAR_.doInvoke(core.clj:1963)
at clojure.lang.RestFn.invoke(RestFn.java:425)
at io.pedestal.interceptor.chain$process_all.invokeStatic(chain.clj:184)
at io.pedestal.interceptor.chain$process_all.invoke(chain.clj:180)
at io.pedestal.interceptor.chain$enter_all.invokeStatic(chain.clj:233)
at io.pedestal.interceptor.chain$enter_all.invoke(chain.clj:227)
at io.pedestal.interceptor.chain$execute.invokeStatic(chain.clj:377)
at io.pedestal.interceptor.chain$execute.invoke(chain.clj:350)
at io.pedestal.interceptor.chain$execute.invokeStatic(chain.clj:387)
at io.pedestal.interceptor.chain$execute.invoke(chain.clj:350)
at io.pedestal.http.impl.servlet_interceptor$interceptor_service_fn$fn__12782.invoke(servlet_interceptor.clj:350)
at io.pedestal.http.servlet.FnServlet.service(servlet.clj:28)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:838)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:543)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1228)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:481)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1130)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:564)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:318)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:112)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:672)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:590)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: :db.error/not-an-entity Unable to resolve entity: :a
at datomic.promise$throw_executionexception_if_throwable.invokeStatic(promise.clj:10)
at datomic.promise$throw_executionexception_if_throwable.invoke(promise.clj:6)
at datomic.promise$settable_future$reify__1048.deref(promise.clj:54)
at clojure.core$deref.invokeStatic(core.clj:2310)
at clojure.core$deref.invoke(core.clj:2296)
at com.cognitect.vase.actions$apply_tx.invokeStatic(actions.clj:110)
at com.cognitect.vase.actions$apply_tx.invoke(actions.clj:101)
at vasetest.server$eval14784$fn__14786.invoke(server.clj:40)
at io.pedestal.interceptor.chain$try_f.invokeStatic(chain.clj:52)
... 39 common frames omitted
Caused by: datomic.impl.Exceptions$IllegalArgumentExceptionInfo: :db.error/not-an-entity Unable to resolve entity: :a
at datomic.error$arg.invokeStatic(error.clj:57)
at datomic.error$arg.invoke(error.clj:52)
at datomic.error$arg.invokeStatic(error.clj:55)
at datomic.error$arg.invoke(error.clj:52)
at datomic.db$require_id.invokeStatic(db.clj:588)
at datomic.db$require_id.invokePrim(db.clj)
at datomic.db$require_attrid.invokeStatic(db.clj:683)
at datomic.db$require_attrid.invoke(db.clj:680)
at datomic.db$expand_map$fn__2568.invoke(db.clj:2409)
at clojure.core.protocols$fn__9143.invokeStatic(protocols.clj:167)
at clojure.core.protocols$fn__9143.invoke(protocols.clj:124)
at clojure.core.protocols$fn__9098$G__9093__9107.invoke(protocols.clj:19)
at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:31)
at clojure.core.protocols$fn__9126.invokeStatic(protocols.clj:75)
at clojure.core.protocols$fn__9126.invoke(protocols.clj:75)
at clojure.core.protocols$fn__9072$G__9067__9085.invoke(protocols.clj:13)
at clojure.core$reduce.invokeStatic(core.clj:6704)
at clojure.core$reduce.invoke(core.clj:6686)
at datomic.db$expand_map.invokeStatic(db.clj:2404)
at datomic.db$expand_map.invoke(db.clj:2398)
at datomic.db.ProcessInpoint.inject(db.clj:2452)
at datomic.db$with_tx$inject_all__2660$fn__2661.invoke(db.clj:2698)
at clojure.lang.PersistentVector.reduce(PersistentVector.java:341)
at clojure.core$reduce.invokeStatic(core.clj:6703)
at clojure.core$reduce.invoke(core.clj:6686)
at datomic.db$with_tx$inject_all__2660.invoke(db.clj:2698)
at datomic.db$with_tx.invokeStatic(db.clj:2702)
at datomic.db$with_tx.invoke(db.clj:2691)
at datomic.peer.LocalConnection$fn__8221.invoke(peer.clj:564)
at datomic.peer.LocalConnection.transactAsync(peer.clj:564)
at datomic.peer.LocalConnection.transact(peer.clj:556)
at datomic.api$transact.invokeStatic(api.clj:94)
at datomic.api$transact.invoke(api.clj:92)
at com.cognitect.vase.actions$apply_tx.invokeStatic(actions.clj:108)
... 42 common frames omitted
INFO i.p.http.impl.servlet-interceptor - {:msg "sending error", :message "Internal server error: exception", :line 214}

#vase/query should allow empty params

Description

A descriptor defined that omits :params should process the request (not 400)

Expected Behavior

The defined :query should execute and the respond be returned as normal.

Actual Behavior

HTTP 400 response, with error message:
Missing required query parameters; One or more parameters was nil. Got: Required: []

Steps to reproduce

Define an api route such as:

:vase/apis
 {:myapp/v1/gene
     "/count"
     {:get
      #vase/query {:name :roscoff.gene/count
                   :params nil
                   :query [:find (count ?e) .
                           :in $
                           :where [?e :gene/id _]]}}

Environment

Clojure 1.9.0-alpha13
lein

Operating System (including version).

DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.2 LTS"

Your current Leiningen/Boot/Maven version (lein --version)

Leiningen 2.7.1 on Java 1.8.0_131 OpenJDK 64-Bit Server VM

Pedestal and Vase version

Snipped from output of lein deps :tree | grep -E "(pedestal|vase)":

 [com.cognitect/pedestal.vase "0.9.0"]
 [io.pedestal/pedestal.jetty "0.5.1"]
 [io.pedestal/pedestal.service-tools "0.5.1" :scope "test"]
 [io.pedestal/pedestal.service "0.5.1"]
   [io.pedestal/pedestal.interceptor "0.5.1"]
   [io.pedestal/pedestal.log "0.5.1"]
   [io.pedestal/pedestal.route "0.5.1"]

Query parameters, path and parameters in request entity are mixed together

Pedestal merges all parameters related to a request, i.e. query parameters, path parameters and request entity data. One problem is that the priority of the different parameters sources is arbitrary (however documented). This problem is a fundamental one, which I like to point out because vase declares to be "way to describe and run RESTful APIs". The query and path parameters are part of the URL addressing a resource and not part of the state that a client wants to be transferred as the new state of the resource. In my view, mixing all parameters together does not play well with a clean REST architecture.

In the context of pedestal this means that all the data for a transact action should come from the request entity, i.e. the json or edn encoded body.

Clarify docs in template

Description

The docstring in part of the vase template is ambiguous/hard to understand.

   ;; You can bring your own non-default interceptors. Make
   ;; sure you include routing and set it up right for
   ;; dev-mode. If you do, many other keys for configuring
   ;; default interceptors will be ignored.
   ;; ::http/interceptors []

;; You can bring your own non-default interceptors. Make
;; sure you include routing and set it up right for
;; dev-mode. If you do, many other keys for configuring
;; default interceptors will be ignored.
;; ::http/interceptors []

There's three sentences here:

  1. You can bring your own non-default interceptors.
  2. Make sure you include routing and set it up right for dev-mode.
  3. If you do, many other keys for configuring default interceptors will be ignored.

It's not clear if they are separate or dependent.

  1. I think I'm ok with, although I think it would be clearer to say "You can bring your own non-default interceptors to Vase" if that is the intent? Is 2) in relation to bringing non-default interceptors, or does it stand alone? I don't understand how 3) relates to 1) and 2) (if at all?).

Are you able to clarify this documentation section?

Trying to figure out how to run the tests in the lein template

Description

I'm attempting to try out some changes to the lein template but I'm running into issues getting tests to run without editing the tests themselves.

In order to get tests to run I need to change line 21 in template/test/com/cognitect/vase/new_server_integration_test.clj
from:

(def project-dir
  (->
   (ClassLoader/getSystemResource *file*)
   io/file .getParent io/file .getParent))

to:

(def project-dir (ClassLoader/getSystemResource *file*))

I then run lein test from within the template directory.

I've had a look around and I can't seem to find where these tests are being run. So I wonder if these tests are being run at all, even on CircleCI as I cannot see anything testing the lein template.

If the tests are in fact not being run on CircleCI and this is not intended, I could submit a PR to fix this. I think the readme could be improved in the template also on how to test.

Scalar/non-sequential query results yield errors

Description

When I use the #vase/query literal to perform a query without any trailing interceptors, queries with a non sequential find-spec (like a single scalar value or a pulled map) generate an error. Queries with nil results also generate a spec error, but don't throw.

Expected Behavior

When I write a query like so:

"/users/:id" {:get #vase/query {:name :query-result-error.v1/user-id-page
                                     :params [id]
                                     :edn-coerce [id]
                                     :query [:find ?e .
                                             :in $ ?id
                                             :where
                                             [?e :user/userId ?id]]}}

I should get a result that is just the user's eid.

Actual Behavior

If the user exists, It throws:

HTTP/1.1 500 Server Error
Connection: close
Date: Wed, 08 Mar 2017 13:48:41 GMT
Content-Type: text/plain

Error processing request!
Exception:

clojure.lang.ExceptionInfo: Interceptor Exception: Don't know how to create ISeq from: java.lang.Long
 at clojure.core$ex_info.invokeStatic (core.clj:4725)
    clojure.core$ex_info.invoke (core.clj:4725)
    io.pedestal.interceptor.chain$throwable__GT_ex_info.invokeStatic (chain.clj:33)
    io.pedestal.interceptor.chain$throwable__GT_ex_info.invoke (chain.clj:32)
    io.pedestal.interceptor.chain$try_f.invokeStatic (chain.clj:55)
    io.pedestal.interceptor.chain$try_f.invoke (chain.clj:42)
    io.pedestal.interceptor.chain$process_all_with_binding.invokeStatic (chain.clj:169)
    io.pedestal.interceptor.chain$process_all_with_binding.invoke (chain.clj:144)
    io.pedestal.interceptor.chain$process_all$fn__8778.invoke (chain.clj:186)
    clojure.lang.AFn.applyToHelper (AFn.java:152)
    clojure.lang.AFn.applyTo (AFn.java:144)
    clojure.core$apply.invokeStatic (core.clj:657)
    clojure.core$with_bindings_STAR_.invokeStatic (core.clj:1963)
    clojure.core$with_bindings_STAR_.doInvoke (core.clj:1963)
    clojure.lang.RestFn.invoke (RestFn.java:425)
    io.pedestal.interceptor.chain$process_all.invokeStatic (chain.clj:184)
    io.pedestal.interceptor.chain$process_all.invoke (chain.clj:180)
    io.pedestal.interceptor.chain$enter_all.invokeStatic (chain.clj:233)
    io.pedestal.interceptor.chain$enter_all.invoke (chain.clj:227)
    io.pedestal.interceptor.chain$execute.invokeStatic (chain.clj:377)
    io.pedestal.interceptor.chain$execute.invoke (chain.clj:350)
    io.pedestal.interceptor.chain$execute.invokeStatic (chain.clj:387)
    io.pedestal.interceptor.chain$execute.invoke (chain.clj:350)
    io.pedestal.http.impl.servlet_interceptor$interceptor_service_fn$fn__12774.invoke (servlet_interceptor.clj:350)
    io.pedestal.http.servlet.FnServlet.service (servlet.clj:28)
    org.eclipse.jetty.servlet.ServletHolder.handle (ServletHolder.java:838)
    org.eclipse.jetty.servlet.ServletHandler.doHandle (ServletHandler.java:543)
    org.eclipse.jetty.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:188)
    org.eclipse.jetty.server.handler.ContextHandler.doHandle (ContextHandler.java:1228)
    org.eclipse.jetty.server.handler.ScopedHandler.nextScope (ScopedHandler.java:168)
    org.eclipse.jetty.servlet.ServletHandler.doScope (ServletHandler.java:481)
    org.eclipse.jetty.server.handler.ScopedHandler.nextScope (ScopedHandler.java:166)
    org.eclipse.jetty.server.handler.ContextHandler.doScope (ContextHandler.java:1130)
    org.eclipse.jetty.server.handler.ScopedHandler.handle (ScopedHandler.java:141)
    org.eclipse.jetty.server.handler.HandlerWrapper.handle (HandlerWrapper.java:132)
    org.eclipse.jetty.server.Server.handle (Server.java:564)
    org.eclipse.jetty.server.HttpChannel.handle (HttpChannel.java:318)
    org.eclipse.jetty.server.HttpConnection.onFillable (HttpConnection.java:251)
    org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded (AbstractConnection.java:279)
    org.eclipse.jetty.io.FillInterest.fillable (FillInterest.java:112)
    org.eclipse.jetty.io.ChannelEndPoint$2.run (ChannelEndPoint.java:124)
    org.eclipse.jetty.util.thread.QueuedThreadPool.runJob (QueuedThreadPool.java:672)
    org.eclipse.jetty.util.thread.QueuedThreadPool$2.run (QueuedThreadPool.java:590)
    java.lang.Thread.run (Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Don't know how to create ISeq from: java.lang.Long
 at clojure.lang.RT.seqFrom (RT.java:547)
    clojure.lang.RT.seq (RT.java:527)
    clojure.core$seq__6422.invokeStatic (core.clj:137)
    clojure.core.protocols$seq_reduce.invokeStatic (protocols.clj:24)
    clojure.core.protocols$fn__9120.invokeStatic (protocols.clj:75)
    clojure.core.protocols/fn (protocols.clj:75)
    clojure.core.protocols$fn__9072$G__9067__9085.invoke (protocols.clj:13)
    clojure.core$reduce.invokeStatic (core.clj:6704)
    clojure.core$into.invokeStatic (core.clj:6771)
    clojure.core$into.invoke (core.clj:6763)
    clojure.core$eval27928$fn__27930.invoke (NO_SOURCE_FILE:0)
    io.pedestal.interceptor.chain$try_f.invokeStatic (chain.clj:52)
    io.pedestal.interceptor.chain$try_f.invoke (chain.clj:42)
    io.pedestal.interceptor.chain$process_all_with_binding.invokeStatic (chain.clj:169)
    io.pedestal.interceptor.chain$process_all_with_binding.invoke (chain.clj:144)
    io.pedestal.interceptor.chain$process_all$fn__8778.invoke (chain.clj:186)
    clojure.lang.AFn.applyToHelper (AFn.java:152)
    clojure.lang.AFn.applyTo (AFn.java:144)
    clojure.core$apply.invokeStatic (core.clj:657)
    clojure.core$with_bindings_STAR_.invokeStatic (core.clj:1963)
    clojure.core$with_bindings_STAR_.doInvoke (core.clj:1963)
    clojure.lang.RestFn.invoke (RestFn.java:425)
    io.pedestal.interceptor.chain$process_all.invokeStatic (chain.clj:184)
    io.pedestal.interceptor.chain$process_all.invoke (chain.clj:180)
    io.pedestal.interceptor.chain$enter_all.invokeStatic (chain.clj:233)
    io.pedestal.interceptor.chain$enter_all.invoke (chain.clj:227)
    io.pedestal.interceptor.chain$execute.invokeStatic (chain.clj:377)
    io.pedestal.interceptor.chain$execute.invoke (chain.clj:350)
    io.pedestal.interceptor.chain$execute.invokeStatic (chain.clj:387)
    io.pedestal.interceptor.chain$execute.invoke (chain.clj:350)
    io.pedestal.http.impl.servlet_interceptor$interceptor_service_fn$fn__12774.invoke (servlet_interceptor.clj:350)
    io.pedestal.http.servlet.FnServlet.service (servlet.clj:28)
    org.eclipse.jetty.servlet.ServletHolder.handle (ServletHolder.java:838)
    org.eclipse.jetty.servlet.ServletHandler.doHandle (ServletHandler.java:543)
    org.eclipse.jetty.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:188)
    org.eclipse.jetty.server.handler.ContextHandler.doHandle (ContextHandler.java:1228)
    org.eclipse.jetty.server.handler.ScopedHandler.nextScope (ScopedHandler.java:168)
    org.eclipse.jetty.servlet.ServletHandler.doScope (ServletHandler.java:481)
    org.eclipse.jetty.server.handler.ScopedHandler.nextScope (ScopedHandler.java:166)
    org.eclipse.jetty.server.handler.ContextHandler.doScope (ContextHandler.java:1130)
    org.eclipse.jetty.server.handler.ScopedHandler.handle (ScopedHandler.java:141)
    org.eclipse.jetty.server.handler.HandlerWrapper.handle (HandlerWrapper.java:132)
    org.eclipse.jetty.server.Server.handle (Server.java:564)
    org.eclipse.jetty.server.HttpChannel.handle (HttpChannel.java:318)
    org.eclipse.jetty.server.HttpConnection.onFillable (HttpConnection.java:251)
    org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded (AbstractConnection.java:279)
    org.eclipse.jetty.io.FillInterest.fillable (FillInterest.java:112)
    org.eclipse.jetty.io.ChannelEndPoint$2.run (ChannelEndPoint.java:124)
    org.eclipse.jetty.util.thread.QueuedThreadPool.runJob (QueuedThreadPool.java:672)
    org.eclipse.jetty.util.thread.QueuedThreadPool$2.run (QueuedThreadPool.java:590)
    java.lang.Thread.run (Thread.java:745)

Context:

{:response nil,
 :cors-headers
 {"Access-Control-Allow-Origin" "",
  "Access-Control-Allow-Credentials" "true"},
 :io.pedestal.interceptor.chain/stack
 ({:name :io.pedestal.http.impl.servlet-interceptor/ring-response,
   :enter nil,
   :leave
   #function[io.pedestal.http.impl.servlet-interceptor/leave-ring-response],
   :error
   #function[io.pedestal.http.impl.servlet-interceptor/error-ring-response]}
  {:name :io.pedestal.http.impl.servlet-interceptor/stylobate,
   :enter
   #function[io.pedestal.http.impl.servlet-interceptor/enter-stylobate],
   :leave
   #function[io.pedestal.http.impl.servlet-interceptor/leave-stylobate],
   :error
   #function[io.pedestal.http.impl.servlet-interceptor/error-stylobate]}
  {:name
   :io.pedestal.http.impl.servlet-interceptor/terminator-injector,
   :enter #function[io.pedestal.interceptor.helpers/before/fn--8985],
   :leave nil,
   :error nil}),
 :request
 {:request-id "T1UaC1shBGI",
  :protocol "HTTP/1.1",
  :db datomic.db.Db@cc2ea76c,
  :async-supported? true,
  :remote-addr "127.0.0.1",
  :servlet-response
  #object[org.eclipse.jetty.server.Response 0x29523707 "HTTP/1.1 200 \nConnection: close\r\nDate: Wed, 08 Mar 2017 13:48:41 GMT\r\n\r\n"],
  :servlet
  #object[io.pedestal.http.servlet.FnServlet 0x122db885 "io.pedestal.http.servlet.FnServlet@122db885"],
  :headers
  {"connection" "close",
   "user-agent" "Paw/3.0.16 (Macintosh; OS X/10.12.3) GCDHTTPRequest",
   "host" "localhost:8080",
   "origin" "",
   "vaserequest-id" "T1UaC1shBGI"},
  :server-port 8080,
  :servlet-request
  #object[org.eclipse.jetty.server.Request 0x1d76ebe2 "Request(GET //localhost:8080/api/query-result-error/v1/users/1)@1d76ebe2"],
  :path-info "/api/query-result-error/v1/users/1",
  :received-time
  #object[org.joda.time.DateTime 0x2cb041d0 "2017-03-08T13:48:41.238Z"],
  :url-for #function[io.pedestal.http.route/url-for-routes/fn--9801],
  :uri "/api/query-result-error/v1/users/1",
  :server-name "localhost",
  :query-string nil,
  :path-params {:id "1"},
  :body
  #object[org.eclipse.jetty.server.HttpInputOverHTTP 0x22107cb2 "HttpInputOverHTTP@22107cb2[c=0,q=0,[0]=null,s=STREAM]"],
  :scheme :http,
  :request-method :get,
  :conn
  #object[datomic.peer.LocalConnection 0x2dce6a20 "datomic.peer.LocalConnection@2dce6a20"]},
 :bindings
 {#'io.pedestal.http.route/*url-for*
  #function[io.pedestal.http.route/url-for-routes/fn--9801]},
 :enter-async
 [#function[io.pedestal.http.impl.servlet-interceptor/start-servlet-async]],
 :io.pedestal.interceptor.chain/terminators
 (#function[io.pedestal.http.impl.servlet-interceptor/terminator-inject/fn--12749]),
 :servlet-response
 #object[org.eclipse.jetty.server.Response 0x29523707 "HTTP/1.1 200 \nConnection: close\r\nDate: Wed, 08 Mar 2017 13:48:41 GMT\r\n\r\n"],
 :route
 {:path "/api/query-result-error/v1/users/:id",
  :method :get,
  :path-constraints {:id "([^/]+)"},
  :io.pedestal.http.route.prefix-tree/satisfies-constraints?
  #function[io.pedestal.http.route.prefix-tree/add-satisfies-constraints?/fn--9676],
  :path-re #"/\Qapi\E/\Qquery-result-error\E/\Qv1\E/\Qusers\E/([^/]+)",
  :path-parts ["api" "query-result-error" "v1" "users" :id],
  :interceptors
  [{:name :attach-received-time,
    :enter #function[com.cognitect.vase.interceptor/fn--14423],
    :leave nil,
    :error nil,
    :doc "Attaches a timestamp to every request."}
   {:name :attach-request-id,
    :enter #function[com.cognitect.vase.interceptor/fn--14426],
    :leave nil,
    :error nil,
    :doc
    "Attaches a request ID to every request;\n            If there's a 'request_id' header, it will use that value, otherwise it will generate a short hash"}
   {:name :io.pedestal.http/json-body,
    :enter nil,
    :leave
    #function[io.pedestal.interceptor.helpers/on-response/fn--9062],
    :error nil}
   {:name :com.cognitect.vase.datomic/insert-datomic,
    :enter
    #function[com.cognitect.vase.datomic/insert-datomic/fn--13969],
    :leave nil,
    :error nil}
   {:name :io.pedestal.http.body-params/body-params,
    :enter
    #function[io.pedestal.interceptor.helpers/on-request/fn--9041],
    :leave nil,
    :error nil}
   {:name :forward-headers,
    :enter nil,
    :leave
    #function[com.cognitect.vase.interceptor/forward-headers/fn--14430],
    :error nil,
    :doc
    "Given an interceptor name and list of headers to forward,\n            return an interceptor that attaches those headers to reponses IFF\n            they are in the request"}
   {:name :query-result-error.v1/user-id-page,
    :enter #function[clojure.core/eval27928/fn--27930],
    :leave nil,
    :error nil,
    :action-literal :vase/query}],
  :route-name :query-result-error.v1/user-id-page,
  :path-params {:id "1"}},
 :servlet
 #object[io.pedestal.http.servlet.FnServlet 0x122db885 "io.pedestal.http.servlet.FnServlet@122db885"],
 :servlet-request
 #object[org.eclipse.jetty.server.Request 0x1d76ebe2 "Request(GET //localhost:8080/api/query-result-error/v1/users/1)@1d76ebe2"],
 :url-for #function[io.pedestal.http.route/url-for-routes/fn--9801],
 :io.pedestal.interceptor.chain/execution-id 12,
 :servlet-config
 #object[org.eclipse.jetty.servlet.ServletHolder$Config 0x45d22a43 "org.eclipse.jetty.servlet.ServletHolder$Config@45d22a43"],
 :async?
 #function[io.pedestal.http.impl.servlet-interceptor/servlet-async?]}

If the user doesn't exist, I get:

HTTP/1.1 400 Bad Request
Connection: close
Date: Wed, 08 Mar 2017 13:55:05 GMT
X-Frame-Options: DENY
X-XSS-Protection: 1; mode=block
X-Download-Options: noopen
Strict-Transport-Security: max-age=31536000; includeSubdomains
X-Permitted-Cross-Domain-Policies: none
vaserequest-id: FF8LMCphCQM
Access-Control-Allow-Credentials: true
Access-Control-Expose-Headers: Strict-Transport-Security, X-Frame-Options, X-Content-Type-Options, X-Xss-Protection, X-Download-Options, X-Permitted-Cross-Domain-Policies, Content-Security-Policy, Vaserequest-Id
X-Content-Type-Options: nosniff
Content-Security-Policy: object-src 'none'; script-src 'unsafe-inline' 'unsafe-eval' 'strict-dynamic' https: http:;
Content-Type: text/plain

Missing required query parameters; One or more parameters was `nil`.  Got: (:id)  Required: [:id]

Steps to reproduce

  1. Use the vase template to make a fresh project with the example svc file.
  2. Add a user via PUT
  3. Modify the /user/:id GET route query to return a scalar result (add .)
  4. Try to GET /user/:id
    (5). Query /user/:id for a user that doesn't exist (to see the spec error)

I made an example repo with the template + modifications here

Environment

Operating System (including version).

Mac OS X 10.12.3

Your current Leiningen/Boot/Maven version (lein --version)

Leiningen 2.7.1 on Java 1.8.0_111 Java HotSpot(TM) 64-Bit Server VM

Pedestal and Vase version

0.5.2/0.9.0

License

Have you considered whether the EPL is really the correct choice of license?

The EPL requires that source code be made available if a derivative work of the licensed work is distributed. Given the use case for Vase most uses are likely to create a derivative work.

Re EPL/derivative work, distribution etc please see the below:

Some open source software communities specify what they mean by a "derivative work". Does the Eclipse Foundation have a position on this?
As described in article 1(b)(ii) of the Eclipse Public License, "...Contributions do not include additions to the Program which: (i) are separate modules of software distributed in conjunction with the Program under their own license agreement, and (ii) are not derivative works of the Program." The definition of derivative work varies under the copyright laws of different jurisdictions. The Eclipse Public License is governed under U.S. law. Under the U.S. Copyright Act, a "derivative work" is defined as "...a work based upon one or more preexisting works, such as a translation, musical arrangement, dramatization, fictionalization, motion picture version, sound recording, art reproduction, abridgment, condensation, or any other form in which a work may be recast, transformed, or adapted. A work consisting of editorial revisions, annotations, elaborations, or other modifications which, as a whole, represent an original work of authorship, is a "derivative work"." The Eclipse Foundation interprets the term "derivative work" in a way that is consistent with the definition in the U.S. Copyright Act, as applicable to computer software. You will need to seek the advice of your own legal counsel in deciding whether your program constitutes a derivative work.

enhance Transact literal to support facts on transactions

Description

A key Datomic capability is the ability to reify transactions with arbitrary facts/metadata. E.g. the :user/id associated with the (transact). Continuing this example, perhaps a vase interceptor could extract the user identifier from a JWT, making it available for the subsequent transact.

Expected Behavior

Transact literal would allow user to specific attributes/values that should be associated with the transaction.

Query action doesn't handle multiple parameters properly

Description

An attempt to bind multiple parameters into a query like so:

"/:foo/:bar"  {:get #vase/query {:name :myapi.v1/a-query
                                      :params [foo bar]
                                      :query [:find ?e
                                              :in $ ?foo ?bar
                                              :where
                                              [?e :foo ?foo]
                                              [?e :bar ?bar]]}}

Will fail, with an error related to parsing params:

clojure.lang.ExceptionInfo: Interceptor Exception: Wrong number of args (7) passed to: actions/coerce-arg-val

Expected Behavior

It should allow queries with multiple inputs bound.

Actual Behavior

Error processing request!
Exception:

clojure.lang.ExceptionInfo: Interceptor Exception: Wrong number of args (7) passed to: actions/coerce-arg-val
 at clojure.core$ex_info.invokeStatic (core.clj:4725)
    clojure.core$ex_info.invoke (core.clj:4725)
    io.pedestal.interceptor.chain$throwable__GT_ex_info.invokeStatic (chain.clj:33)
    io.pedestal.interceptor.chain$throwable__GT_ex_info.invoke (chain.clj:32)
    io.pedestal.interceptor.chain$try_f.invokeStatic (chain.clj:55)
    io.pedestal.interceptor.chain$try_f.invoke (chain.clj:42)
    io.pedestal.interceptor.chain$process_all_with_binding.invokeStatic (chain.clj:169)
    io.pedestal.interceptor.chain$process_all_with_binding.invoke (chain.clj:144)
    io.pedestal.interceptor.chain$process_all$fn__8778.invoke (chain.clj:186)
    clojure.lang.AFn.applyToHelper (AFn.java:152)
    clojure.lang.AFn.applyTo (AFn.java:144)
    clojure.core$apply.invokeStatic (core.clj:657)
    clojure.core$with_bindings_STAR_.invokeStatic (core.clj:1963)
    clojure.core$with_bindings_STAR_.doInvoke (core.clj:1963)
    clojure.lang.RestFn.invoke (RestFn.java:425)
    io.pedestal.interceptor.chain$process_all.invokeStatic (chain.clj:184)
    io.pedestal.interceptor.chain$process_all.invoke (chain.clj:180)
    io.pedestal.interceptor.chain$enter_all.invokeStatic (chain.clj:233)
    io.pedestal.interceptor.chain$enter_all.invoke (chain.clj:227)
    io.pedestal.interceptor.chain$execute.invokeStatic (chain.clj:377)
    io.pedestal.interceptor.chain$execute.invoke (chain.clj:350)
    io.pedestal.interceptor.chain$execute.invokeStatic (chain.clj:387)
    io.pedestal.interceptor.chain$execute.invoke (chain.clj:350)
    io.pedestal.http.impl.servlet_interceptor$interceptor_service_fn$fn__12774.invoke (servlet_interceptor.clj:350)
    io.pedestal.http.servlet.FnServlet.service (servlet.clj:28)
    org.eclipse.jetty.servlet.ServletHolder.handle (ServletHolder.java:838)
    org.eclipse.jetty.servlet.ServletHandler.doHandle (ServletHandler.java:543)
    org.eclipse.jetty.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:188)
    org.eclipse.jetty.server.handler.ContextHandler.doHandle (ContextHandler.java:1228)
    org.eclipse.jetty.server.handler.ScopedHandler.nextScope (ScopedHandler.java:168)
    org.eclipse.jetty.servlet.ServletHandler.doScope (ServletHandler.java:481)
    org.eclipse.jetty.server.handler.ScopedHandler.nextScope (ScopedHandler.java:166)
    org.eclipse.jetty.server.handler.ContextHandler.doScope (ContextHandler.java:1130)
    org.eclipse.jetty.server.handler.ScopedHandler.handle (ScopedHandler.java:141)
    org.eclipse.jetty.server.handler.HandlerWrapper.handle (HandlerWrapper.java:132)
    org.eclipse.jetty.server.Server.handle (Server.java:564)
    org.eclipse.jetty.server.HttpChannel.handle (HttpChannel.java:318)
    org.eclipse.jetty.server.HttpConnection.onFillable (HttpConnection.java:251)
    org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded (AbstractConnection.java:279)
    org.eclipse.jetty.io.FillInterest.fillable (FillInterest.java:112)
    org.eclipse.jetty.io.ChannelEndPoint$2.run (ChannelEndPoint.java:124)
    org.eclipse.jetty.util.thread.QueuedThreadPool.runJob (QueuedThreadPool.java:672)
    org.eclipse.jetty.util.thread.QueuedThreadPool$2.run (QueuedThreadPool.java:590)
    java.lang.Thread.run (Thread.java:745)
Caused by: clojure.lang.ArityException: Wrong number of args (7) passed to: actions/coerce-arg-val
 at clojure.lang.AFn.throwArity (AFn.java:429)
    clojure.lang.AFn.invoke (AFn.java:57)
    clojure.core$eval30709$fn__30711.invoke (NO_SOURCE_FILE:0)
    io.pedestal.interceptor.chain$try_f.invokeStatic (chain.clj:52)
    io.pedestal.interceptor.chain$try_f.invoke (chain.clj:42)
    io.pedestal.interceptor.chain$process_all_with_binding.invokeStatic (chain.clj:169)
    io.pedestal.interceptor.chain$process_all_with_binding.invoke (chain.clj:144)
    io.pedestal.interceptor.chain$process_all$fn__8778.invoke (chain.clj:186)
    clojure.lang.AFn.applyToHelper (AFn.java:152)
    clojure.lang.AFn.applyTo (AFn.java:144)
    clojure.core$apply.invokeStatic (core.clj:657)
    clojure.core$with_bindings_STAR_.invokeStatic (core.clj:1963)
    clojure.core$with_bindings_STAR_.doInvoke (core.clj:1963)
    clojure.lang.RestFn.invoke (RestFn.java:425)
    io.pedestal.interceptor.chain$process_all.invokeStatic (chain.clj:184)
    io.pedestal.interceptor.chain$process_all.invoke (chain.clj:180)
    io.pedestal.interceptor.chain$enter_all.invokeStatic (chain.clj:233)
    io.pedestal.interceptor.chain$enter_all.invoke (chain.clj:227)
    io.pedestal.interceptor.chain$execute.invokeStatic (chain.clj:377)
    io.pedestal.interceptor.chain$execute.invoke (chain.clj:350)
    io.pedestal.interceptor.chain$execute.invokeStatic (chain.clj:387)
    io.pedestal.interceptor.chain$execute.invoke (chain.clj:350)
    io.pedestal.http.impl.servlet_interceptor$interceptor_service_fn$fn__12774.invoke (servlet_interceptor.clj:350)
    io.pedestal.http.servlet.FnServlet.service (servlet.clj:28)
    org.eclipse.jetty.servlet.ServletHolder.handle (ServletHolder.java:838)
    org.eclipse.jetty.servlet.ServletHandler.doHandle (ServletHandler.java:543)
    org.eclipse.jetty.server.handler.ScopedHandler.nextHandle (ScopedHandler.java:188)
    org.eclipse.jetty.server.handler.ContextHandler.doHandle (ContextHandler.java:1228)
    org.eclipse.jetty.server.handler.ScopedHandler.nextScope (ScopedHandler.java:168)
    org.eclipse.jetty.servlet.ServletHandler.doScope (ServletHandler.java:481)
    org.eclipse.jetty.server.handler.ScopedHandler.nextScope (ScopedHandler.java:166)
    org.eclipse.jetty.server.handler.ContextHandler.doScope (ContextHandler.java:1130)
    org.eclipse.jetty.server.handler.ScopedHandler.handle (ScopedHandler.java:141)
    org.eclipse.jetty.server.handler.HandlerWrapper.handle (HandlerWrapper.java:132)
    org.eclipse.jetty.server.Server.handle (Server.java:564)
    org.eclipse.jetty.server.HttpChannel.handle (HttpChannel.java:318)
    org.eclipse.jetty.server.HttpConnection.onFillable (HttpConnection.java:251)
    org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded (AbstractConnection.java:279)
    org.eclipse.jetty.io.FillInterest.fillable (FillInterest.java:112)
    org.eclipse.jetty.io.ChannelEndPoint$2.run (ChannelEndPoint.java:124)
    org.eclipse.jetty.util.thread.QueuedThreadPool.runJob (QueuedThreadPool.java:672)
    org.eclipse.jetty.util.thread.QueuedThreadPool$2.run (QueuedThreadPool.java:590)
    java.lang.Thread.run (Thread.java:745)

Steps to reproduce

Define a get method with a vase query literal (as above), and specify more than one param from the path, query string, etc.

Environment

Operating System (including version).

Mac OS 10.12.3

Your current Leiningen/Boot/Maven version (lein --version)

Leiningen 2.7.1 on Java 1.8.0_111 Java HotSpot(TM) 64-Bit Server VM

Pedestal and Vase version

Pedestal: 0.5.2
Vase: 0.9.0 - current master

How to get started? (missing doc?)

tl;dr - The startup docs have some gaps that are hurting me as a newbie.

I'm taking a first look at Vase, and have never used Datomic. I'm trying to find the easiest path to a minimal working example which, I guess, would include a running service that I can deploy locally or to some cloud service, and that can persist data in the face of a system reboot.

I did 'lein new vase your-first-api', and am indeed up and seeing Hello World.

I don't know if I already have a database configured, nor how to test. I see the following lines in the docs:

In docs/your_first_api.md:

... be sure you have a Datomic transactor running. See ... docs ... [T]he template project uses Datomic Pro. (You can change the project.clj dependency to use Datomic Free. ...)

But:

  • I did not do any Datomic setup, yet something seems to be working. Do I need to do more, or is Vase complete as is.
  • I don't have a copy of Datamic Pro. Yet, I did not change anything to use Datamic Free.
  • There is no mention of Datomic in my project.clj.

And, then, once I get Datomic setup, how I understand that it starts as an in-memory database. How do I persist to disk? This is just a hobbyist project now, so I'm fine with something simple. I see http://docs.datomic.com/backup.html, which may be all I need, but I don't yet see what/how I need to integrate into my project.

And, maybe out of scope, but it would be good to have a few lines about how to deploy to a production server.

API discovery should be customizable

Currently Vase exposes simple API discovery (cognitect.vase.routes/describe-api) as an endpoint at the root of each API tree.

This should be configurable, presumably via an option to the API definition in the service .edn file since:

  • On the one extreme, I might not want to reveal my entire API.
  • On the other extreme, I might want to offer a much richer discovery mechanism. At the very least, I can see often wanting to return the information as a navigable data structure, rather than a flat string.

isComponent not supported in the current version

Description

In the current version 0.9.0, the keyword :component is not supported. I have seen it exists in the source code, so guess it will come with the next version. So could I ask when the next version will be released? We would like to use :isComponent in our project, it does affect our developing.

petstore-full not working with datomic free

Description

Following the petstore-full instructions, "lein run" fails with dependency errors.

$ lein run
Could not find artifact com.datomic:datomic-pro:jar:0.9.5544 in central (https://repo1.maven.org/maven2/)
Could not find artifact com.datomic:datomic-pro:jar:0.9.5544 in clojars (https://clojars.org/repo/)
Could not find artifact com.cognitect:pedestal.vase:jar:0.9.0-SNAPSHOT in clojars (https://clojars.org/repo/)

I made the following edits to project.clj :

com.cognitect/pedestal.vase "0.9.0-SNAPSHOT" ==> Changed to "0.9.0"
com.datomic/datomic-pro "0.9.5544" ==> Deleted

Then I removed the datomic-free exclusion from pedestal.

Try again:

petstore-full $ lein run
WARNING: seqable? already refers to: #'clojure.core/seqable? in namespace: clojure.core.incubator, being replaced by: #'clojure.core.incubator/seqable?
WARNING: boolean? already refers to: #'clojure.core/boolean? in namespace: clojure.tools.analyzer.utils, being replaced by: #'clojure.tools.analyzer.utils/boolean?
WARNING: boolean? already refers to: #'clojure.core/boolean? in namespace: clojure.tools.analyzer, being replaced by: #'clojure.tools.analyzer.utils/boolean?
WARNING: bounded-count already refers to: #'clojure.core/bounded-count in namespace: clojure.core.async, being replaced by: #'clojure.core.async/bounded-count
Exception in thread "main" java.lang.IllegalArgumentException: :db.error/unsupported-protocol Unsupported protocol :dev, compiling:(petstore_full/server.clj:7:1)
at clojure.lang.Compiler.load(Compiler.java:7441)
at clojure.lang.RT.loadResourceScript(RT.java:374)
at clojure.lang.RT.loadResourceScript(RT.java:365)
at clojure.lang.RT.load(RT.java:455)
at clojure.lang.RT.load(RT.java:421)
... more...

Expected Behavior

"lein run" should build and run with datomic-free with no changes

Actual Behavior

dependency errors and the build fail when they are corrected.

Steps to reproduce

cd petstore-full
lein run

Operating System (including version).

OS X 10.9.5

Your current Leiningen/Boot/Maven version (lein --version)

Leiningen 2.7.1 on Java 1.8.0_40 Java HotSpot(TM) 64-Bit Server VM

Pedestal and Vase version

from project.clj

Post is not creating new entities

Description

I followed the Tutorial "Building your first API" to "Getting data in with transact", but I can't get :post to be transacting the new entity. It boils down to the following problem.

My minimal EDN:

{:activated-apis
 [:accounts/v1]

 :datomic-uri
 "datomic:free://localhost:4334/example" ;; or "datomic:mem://example", same response

 :descriptor
 {:vase/norms
  {:accounts/user
   {:vase.norm/txes [#vase/schema-tx
       [[:user/userId :one :string :unique "The unique identifier for a user"]
        [:user/email :one :string :unique "The email address for a given user"]]]}}

  :vase/specs
  {}

  :vase/apis
  {:accounts/v1
   {:vase.api/routes
    {"/user" {:post #vase/transact {:name       :accounts.v1/user-create
                                    :properties [:db/id :user/userId :user/email]}}}

    :vase.api/schemas
    [:accounts/user]

    :vase.api/forward-headers
    ["vaserequest-id"]}}}}

Request-Variations: (same response)

curl --form user/userId=42 --form user/[email protected] http://localhost:8080/api/accounts/v1/user
curl --form :user/userId=42 --form :user/[email protected] http://localhost:8080/api/accounts/v1/user

curl -H "Content-Type: application/json" -X POST -d '{"user/userId":"42","user/email":"[email protected]"}' http://localhost:8080/api/accounts/v1/user
curl -H "Content-Type: application/json" -X POST -d '{":user/userId":"42",":user/email":"[email protected]"}' http://localhost:8080/api/accounts/v1/user

(also tried same requests in postman, same results)

Response:

{"whitelist":[],"transaction":[[null,null,null],[null,null,null],[null,null,null],[null,null,null]]}

Log: (for one request)

INFO  io.pedestal.http - {:msg "POST /api/accounts/v1/user", :line 78}
INFO  io.pedestal.http.cors - {:msg "cors request processing", :origin "", :allowed true, :line 84}
INFO  datomic.kv-cluster - {:event :kv-cluster/get-pod, :pod-key "pod-catalog", :phase :begin, :pid 96750, :tid 70}
INFO  datomic.kv-cluster - {:event :kv-cluster/get-pod, :pod-key "pod-catalog", :msec 1.08, :phase :end, :pid 96750, :tid 70}
INFO  datomic.peer - {:event :peer/transact, :uuid #uuid "5895a3de-ff72-49a5-a318-f1d45ad8ad66", :phase :start, :pid 96750, :tid 85}
INFO  datomic.peer - {:event :peer/transact, :uuid #uuid "5895a3de-ff72-49a5-a318-f1d45ad8ad66", :phase :end, :pid 96750, :tid 52}
INFO  datomic.peer - {:event :peer/notify-data, :msec 1, :id #uuid "5895a3de-ff72-49a5-a318-f1d45ad8ad66", :pid 96750, :tid 52}
INFO  io.pedestal.http.cors - {:msg "cors response processing", :cors-headers {"Access-Control-Allow-Origin" "", "Access-Control-Allow-Credentials" "true", "Access-Control-Expose-Headers" "Strict-Transport-Security, X-Frame-Options, X-Content-Type-Options, X-Xss-Protection, X-Download-Options, X-Permitted-Cross-Domain-Policies, Content-Security-Policy, Vaserequest-Id, Content-Type"}, :line 111}

Datomic: (checked with datomic console, schema is applied, only the transaction is created)
13194139534317 :db/txInstant #inst "2017-02-04T09:50:22.443-00:00" true

Environment

OS: Mac El Captain - 10.11.6
JVM: jdk1.8.0_74
Leiningen: 2.7.1
Clojure: 1.9.0-alpha14
Pedestal: 0.5.2
Vase: 0.9.0

Datomic Transactor: datomic-free-0.9.5544

(generated on 3. Feb. with lein new vase anytitle)

petstore-full not starting

Description

lein run in petstore-full example throws exception on start

petstore-full $ lein run
WARNING: seqable? already refers to: #'clojure.core/seqable? in namespace: clojure.core.incubator, being replaced by: #'clojure.core.incubator/seqable?
WARNING: boolean? already refers to: #'clojure.core/boolean? in namespace: clojure.tools.analyzer.utils, being replaced by: #'clojure.tools.analyzer.utils/boolean?
WARNING: boolean? already refers to: #'clojure.core/boolean? in namespace: clojure.tools.analyzer, being replaced by: #'clojure.tools.analyzer.utils/boolean?
WARNING: bounded-count already refers to: #'clojure.core/bounded-count in namespace: clojure.core.async, being replaced by: #'clojure.core.async/bounded-count
INFO datomic.kv-cluster - {:event :kv-cluster/retry, :StorageGetBackoffMsec 0, :attempts 0, :max-retries 20, :cause "java.net.ConnectException", :pid 34127, :tid 13}
INFO datomic.kv-cluster - {:event :kv-cluster/retry, :StorageGetBackoffMsec 50, :attempts 1, :max-retries 20, :cause "java.net.ConnectException", :pid 34127, :tid 13}
INFO datomic.kv-cluster - {:event :kv-cluster/retry, :StorageGetBackoffMsec 100, :attempts 2, :max-retries 20, :cause "java.net.ConnectException", :pid 34127, :tid 13}
INFO datomic.kv-cluster - {:event :kv-cluster/retry, :StorageGetBackoffMsec 150, :attempts 3, :max-retries 20, :cause "java.net.ConnectException", :pid 34127, :tid 13}
INFO datomic.kv-cluster - {:event :kv-cluster/retry, :StorageGetBackoffMsec 200, :attempts 4, :max-retries 20, :cause "java.net.ConnectException", :pid 34127, :tid 13}
INFO datomic.kv-cluster - {:event :kv-cluster/retry, :StorageGetBackoffMsec 250, :attempts 5, :max-retries 20, :cause "java.net.ConnectException", :pid 34127, :tid 13}
INFO datomic.kv-cluster - {:event :kv-cluster/retry, :StorageGetBackoffMsec 300, :attempts 6, :max-retries 20, :cause "java.net.ConnectException", :pid 34127, :tid 13}
Exception in thread "main" java.util.concurrent.ExecutionException: org.h2.jdbc.JdbcSQLException: Connection is broken: "java.net.ConnectException: Connection refused: localhost:4335" [90067-171], compiling:(petstore_full/server.clj:7:1)
at clojure.lang.Compiler.load(Compiler.java:7441)
at clojure.lang.RT.loadResourceScript(RT.java:374)
at clojure.lang.RT.loadResourceScript(RT.java:365)
at clojure.lang.RT.load(RT.java:455)
at clojure.lang.RT.load(RT.java:421)
at clojure.core$load$fn__7840.invoke(core.clj:6008)
at clojure.core$load.invokeStatic(core.clj:6007)
at clojure.core$load.doInvoke(core.clj:5991)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invokeStatic(core.clj:5812)
at clojure.core$load_one.invoke(core.clj:5807)
at clojure.core$load_lib$fn__7785.invoke(core.clj:5852)
at clojure.core$load_lib.invokeStatic(core.clj:5851)
at clojure.core$load_lib.doInvoke(core.clj:5832)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invokeStatic(core.clj:659)
at clojure.core$load_libs.invokeStatic(core.clj:5889)
at clojure.core$load_libs.doInvoke(core.clj:5873)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invokeStatic(core.clj:659)
at clojure.core$require.invokeStatic(core.clj:5911)
at clojure.core$require.doInvoke(core.clj:5911)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at user$eval15$fn__19.invoke(form-init1832586691876862531.clj:1)
at user$eval15.invokeStatic(form-init1832586691876862531.clj:1)
at user$eval15.invoke(form-init1832586691876862531.clj:1)
at clojure.lang.Compiler.eval(Compiler.java:6977)
at clojure.lang.Compiler.eval(Compiler.java:6967)
at clojure.lang.Compiler.load(Compiler.java:7429)
at clojure.lang.Compiler.loadFile(Compiler.java:7367)
at clojure.main$load_script.invokeStatic(main.clj:277)
at clojure.main$init_opt.invokeStatic(main.clj:279)
at clojure.main$init_opt.invoke(main.clj:279)
at clojure.main$initialize.invokeStatic(main.clj:310)
at clojure.main$null_opt.invokeStatic(main.clj:344)
at clojure.main$null_opt.invoke(main.clj:341)
at clojure.main$main.invokeStatic(main.clj:423)
at clojure.main$main.doInvoke(main.clj:386)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:700)
at clojure.main.main(main.java:37)
Caused by: java.util.concurrent.ExecutionException: org.h2.jdbc.JdbcSQLException: Connection is broken: "java.net.ConnectException: Connection refused: localhost:4335" [90067-171]
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at clojure.core$deref_future.invokeStatic(core.clj:2290)
at clojure.core$future_call$reify__9371.deref(core.clj:6847)
at clojure.core$deref.invokeStatic(core.clj:2310)
at clojure.core$deref.invoke(core.clj:2296)
at datomic.coordination$lookup_transactor_endpoint$fn__5161.invoke(coordination.clj:132)
at datomic.coordination$lookup_transactor_endpoint.invokeStatic(coordination.clj:130)
at datomic.coordination$lookup_transactor_endpoint.invoke(coordination.clj:127)
at datomic.coordination$lookup_compatible_transactor_endpoint.invokeStatic(coordination.clj:148)
at datomic.coordination$lookup_compatible_transactor_endpoint.invoke(coordination.clj:146)
at datomic.peer$send_admin_request$fn__8304.invoke(peer.clj:759)
at datomic.peer$send_admin_request.invokeStatic(peer.clj:753)
at datomic.peer$send_admin_request.invoke(peer.clj:751)
at datomic.peer$create_database.invokeStatic(peer.clj:773)
at datomic.peer$create_database.invoke(peer.clj:763)
at datomic.peer$create_database.invokeStatic(peer.clj:765)
at datomic.peer$create_database.invoke(peer.clj:763)
at clojure.lang.Var.invoke(Var.java:379)
at datomic.Peer.createDatabase(Peer.java:117)
at datomic.api$create_database.invokeStatic(api.clj:19)
at datomic.api$create_database.invoke(api.clj:17)
at com.cognitect.vase.datomic$connect.invokeStatic(datomic.clj:13)
at com.cognitect.vase.datomic$connect.invoke(datomic.clj:9)
at com.cognitect.vase$ensure_schema$fn__14531.invoke(vase.clj:45)
at clojure.core.protocols$iter_reduce.invokeStatic(protocols.clj:49)
at clojure.core.protocols$fn__9124.invokeStatic(protocols.clj:75)
at clojure.core.protocols$fn__9124.invoke(protocols.clj:75)
at clojure.core.protocols$fn__9066$G__9061__9079.invoke(protocols.clj:13)
at clojure.core$reduce.invokeStatic(core.clj:6704)
at clojure.core$reduce.invoke(core.clj:6686)
at com.cognitect.vase$ensure_schema.invokeStatic(vase.clj:44)
at com.cognitect.vase$ensure_schema.invoke(vase.clj:26)
at petstore_full.service$service.invokeStatic(service.clj:53)
at petstore_full.service$service.invoke(service.clj:49)
at petstore_full.server$eval14649.invokeStatic(server.clj:7)
at petstore_full.server$eval14649.invoke(server.clj:7)
at clojure.lang.Compiler.eval(Compiler.java:6977)
at clojure.lang.Compiler.load(Compiler.java:7429)
... 40 more
Caused by: org.h2.jdbc.JdbcSQLException: Connection is broken: "java.net.ConnectException: Connection refused: localhost:4335" [90067-171]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:329)
at org.h2.message.DbException.get(DbException.java:158)
at org.h2.engine.SessionRemote.connectServer(SessionRemote.java:399)
at org.h2.engine.SessionRemote.connectEmbeddedOrServer(SessionRemote.java:287)
at org.h2.jdbc.JdbcConnection.(JdbcConnection.java:109)
at org.h2.jdbc.JdbcConnection.(JdbcConnection.java:93)
at org.h2.Driver.connect(Driver.java:72)
at org.apache.tomcat.jdbc.pool.PooledConnection.connectUsingDriver(PooledConnection.java:266)
at org.apache.tomcat.jdbc.pool.PooledConnection.connect(PooledConnection.java:175)
at org.apache.tomcat.jdbc.pool.ConnectionPool.createConnection(ConnectionPool.java:684)
at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:616)
at org.apache.tomcat.jdbc.pool.ConnectionPool.init(ConnectionPool.java:479)
at org.apache.tomcat.jdbc.pool.ConnectionPool.(ConnectionPool.java:135)
at org.apache.tomcat.jdbc.pool.DataSourceProxy.pCreatePool(DataSourceProxy.java:114)
at org.apache.tomcat.jdbc.pool.DataSourceProxy.createPool(DataSourceProxy.java:101)
at org.apache.tomcat.jdbc.pool.DataSourceProxy.getConnection(DataSourceProxy.java:125)
at datomic.sql$connect.invokeStatic(sql.clj:16)
at datomic.sql$connect.invoke(sql.clj:13)
at datomic.sql$select.invokeStatic(sql.clj:77)
at datomic.sql$select.invoke(sql.clj:75)
at datomic.kv_sql.KVSql.get(kv_sql.clj:60)
at datomic.kv_cluster.KVCluster$fn__3396$fn__3399$fn__3400.invoke(kv_cluster.clj:226)
at datomic.kv_cluster$retry_fn$fn__3312.invoke(kv_cluster.clj:82)
at datomic.kv_cluster$retry_fn.invokeStatic(kv_cluster.clj:82)
at datomic.kv_cluster$retry_fn.invoke(kv_cluster.clj:57)
at clojure.lang.AFn.applyToHelper(AFn.java:178)
at clojure.lang.AFn.applyTo(AFn.java:144)
at clojure.core$apply.invokeStatic(core.clj:663)
at clojure.core$partial$fn__6855.doInvoke(core.clj:2616)
at clojure.lang.RestFn.invoke(RestFn.java:421)
at datomic.kv_cluster.KVCluster$fn__3396$fn__3399.invoke(kv_cluster.clj:224)
at datomic.kv_cluster.KVCluster$fn__3396.invoke(kv_cluster.clj:222)
at clojure.core$binding_conveyor_fn$fn__6766.invoke(core.clj:2020)
at clojure.lang.AFn.call(AFn.java:18)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.h2.util.NetUtils.createSocket(NetUtils.java:119)
at org.h2.util.NetUtils.createSocket(NetUtils.java:100)
at org.h2.engine.SessionRemote.initTransfer(SessionRemote.java:93)
at org.h2.engine.SessionRemote.connectServer(SessionRemote.java:395)
... 35 more

Operating System (including version).

os x 10.9.5

Your current Leiningen/Boot/Maven version (lein --version)

Leiningen 2.7.1 on Java 1.8.0_40 Java HotSpot(TM) 64-Bit Server VM

Pedestal and Vase version

from petstore-full project.clj

Behavior of :find [(pull ... is under-defined and has changed recently

The behavior of a Vase pull query does not seem to be fully defined.
Worse, it has recently changed incompatibly somewhere between Vase commits 8f61d36 and bf96190.

In the old version :find [(pull ?e [*]) ...] returned the tuples wrapped in an extra layer of vector.
This extra wrapping no longer occurs. FWIW, the new behavior seems more sensible, but the change breaks client code.

Also, surprising at least to me, a simpler :find (pull ?e [*]) returned a more complex structure. Prior to the recent changes, if I recall correctly, there were two extra layers. I've not tested what is returned in the latest version.

Suggestions:

  1. Document the interactions between pull and Vase, and the expected values. Some additional tests would be good too. The code is likely fragile, given that the current change appears to have been introduced accidentally.
  2. Warn that the behavior has changed, in the 0.9.1 release notes

Cannot build Docker image

Description

Small one:
The pet=store project.clj defines version "0.9.0-SNAPSHOT" while the Dockerfile contains an ADD for target/pet-store-0.0.1-SNAPSHOT-standalone.jar that cannot be found when building the image.

Consider change to descriptor files

Description

Right now, the top level of descriptor files has the keys :activated-apis, :datomic-uri, and :descriptor. Two of these are much more like environment-specific configuration, specifically :activated-apis and :datomic-uri.

Also, I've found it challenging to get the Datomic URI from the descriptor file to use in any other non-Vase interceptors in my application. (The Vase template doesn't offer any place to get access to the connection that's created for the standard DB interceptor.)

Proposal

This proposal has three parts:

  1. Restrict the descriptor file to just the contents that are currently under the top-level :descriptor key.
  2. Adjust the Vase template to look for a separate config.edn file for the system configuration.
  3. Adjust the top-level API functions vase/routes and vase/descriptor-facts to just take what is now the contents of the :descriptor key. vase/routes should take one new parameter with a map for configuration.

Considerations

As a library, Vase should be silent on the issue of how applications will load their config. It should make no assumptions about whether config comes from a file, environment variables, ZooKeeper, or anything else. When config is needed, Vase functions should just accept a maplike value.

This is a breaking change, so it would be better to do this before we cut release 1.0.

Exploring implementing new action literals that require access to shared state

Taking the proposition Paul suggested in the Cognicast to heart, I've been exploring what it would take to add action literals to do things not among vase's design goals, namely express sql queries and commands, with an eye towards higher-level routes that do CRUD operations on sql tables.

The implementation is fairly straightforward, but the stumbling block is that vase doesn't provide any affordance for interceptors offering access to novel resources to get mixed into the default interceptor chain. I realize that this is essentially one of your design non-goals, but I posit that most interesting and useful custom action literals will need this. Would you be interested in exploring what it might look like for vase to offer some e.g. declarative approach, or do you feel like it's just best in such cases to build a custom interceptor chain?

Reduce boilerplate when adding Vase to a service

Currently, Vase offers all the individual functions needed to bootstrap Vase into an existing Pedestal application. Much liked Pedestal's create-server, we need to make the common-case fast. Another top-level function should be introduced that performs the standard bootstrapping steps.

Float coercion

Javascript does not have distinct integer and float types. Therefore, 2.0 and 2 are equal.

When 2.0 is sent to the server (at least in JSON; probably also in Transit, though I've not tested), the server sees it as 2.

If my Vase schema has an attribute of type :float, a run-time error will be generated if the client passes a number that happens to be an integer (even if it was created by code that thinks it is dealing with floats).

This risk of surprise could be avoided if Vase would accept and coerce integers into float fields.

For now, I can work around this with a trivial interceptor, but it would be better if automated.
The following ten lines are a much too verbose way to ensure that one float field works safely!

(def float-conversion
  (i/interceptor
   {:name ::float-conversion
    :enter (fn [context]
             (let [payloads (get-in context [:request :json-params :payload])
                   payloads (map (fn [m] (if (:purchase/price m)
                                           (update m :purchase/price float)
                                           m))
                                 payloads)]
               (assoc-in context [:request :json-params :payload] payloads)))}))

Cannot transact false values from payload.

False values are removed from the payload before transacting. False values should be allowed and transacted into the database.

I believe the problem is here:

#(into {} (filter second (select-keys % ~(vec properties))))

This line should be changed from
#(into {} (filter second (select-keys % ~(vec properties))))
to
#(into {} (remove (comp nil? second) (select-keys % ~(vec properties))))

Or, just
#(select-keys % ~(vec properties))

The result of select-keys doesn't include keys not found in the map, and if nil is actually a value that the user supplied, then this function shouldn't silently remove it.

Consider adding an FAQ doc with small demonstrations.

Description

I think Bite size demonstrations would be a good supplement to full on sample applications.
The writethedocs site has a good writeup on this.

I'm bringing this up because I recently shared a snippet on the #pedestal Slack channel that would have been a good candidate.

I'm more than happy to kick this off if folks are interested.

There should be better handling for multiple descriptors

The top-level Vase API supports multiple descriptors to be into the main functions. This is nice for end-users because it's common to break large descriptors down into smaller, cohesive descriptors.

Currently those functions map over the descriptors individually, but Vase injects routes and other artifacts when processing descriptors (routes like the describe route used to reflect the APIs back out to the user for discovery/docs). Injecting the reflection capabilities should be a separate function called from the top-level API after process routes. Additionally, in the new format, it's cleaner to merge Descriptor subsections (norms, specs, apis) -- the top-level functions should merge descriptor subsections when processing them, instead of mapping over descriptors individually.

Schema-tx literal surprising behavior when :db/doc omitted

Description

Expected Behavior

Consider the following sample:

[:pet/legs :ref :many]

This is not accepted and throws an exception because the docstring is missing. This is reasonable if the docstring is required. However:

[:pet/legs :ref :many :indexed]

This also fails but silently. The :indexed toggle is misinterpreted as a docstring. It has to be changed to:

[:pet/legs :ref :many :indexed "Why would you search by leg number? And which one is #1?"]

This arises because the toggles are optional and may be multiple.

We should go one of two ways on this. Either:

  1. Make the docstring required in all cases and throw when it is not a string, or
  2. Make the docstring optional in all cases and accept as many toggles as we find.

I lean toward #2, since the whole point of the schema-tx literal is convenience and concision.

Pedestal and Vase version

Present in 0.9.1-snapshot

API description route getting applied over existing route at "/"

Description

When one defines an endpoint that handles :get at "/" within an API, it can get replaced by the API descriptor route. This is confusing, as in the Pet Store Full example, there are API endpoints defined at "/". If this is not allowed, it should be documented somewhere, or throw an error. I am happy to submit a pull request when that decision is made.

Expected Behavior

Whatever is defined at "/" works as described in the descriptor file.

Actual Behavior

Either the defined handler or route description handlers get called for a GET request to "/". It seems to pick randomly at server start, and then does not change.

Steps to reproduce

Define a vase api with an endpoint at "/" with a :get. Bring the server up and issue a GET request to the API root, and observe. Bring the server down and back up a few times, and see that the behavior is inconsistent.

Environment

lein run

Operating System (including version).

OSX 10.13.3

Your current Leiningen/Boot/Maven version (lein --version)

Leiningen 2.8.1 on Java 1.8.0_131 Java HotSpot(TM) 64-Bit Server VM

Pedestal and Vase version

[org.clojure/clojure "1.9.0"]
[io.pedestal/pedestal.service "0.5.3"]
[com.cognitect/pedestal.vase "0.9.3"]

Add ability to call arbitrary database function with payload

Hi, and thanks for a fine library.

Currently it does not seem easy to call an arbitrary database function on the given JSON payload.

I would like the following possibility:

"/user"             {:post #vase/transact {:name  :accounts.v1/user-create
                                                  :db-fn :my-great-fn
                                                  :properties [:db/id
                                                               :user/userId
                                                               :user/email]}}}

and that this should expand to:

[[:my-great-fn {:user/userId 42 :user/email "[email protected]"}]]

It is then up to :my-great-fn to generate / expand to the proper data as needed.

I think this is preferable over using custom interceptors with request db or request conn as this will preserve ACID properties.

Would this be possible and desirable to add to Vase?

Regards

Generating vase template project with a kebab-case project name results in inconstant naming of the service.edn file

Description

Generating a vase template project with a kebab-case project name results in inconstant naming of the service.edn file. For example, running lein new vase foo-bar creates the file resources/foo-bar_service.edn with hyphens and underscores.

I'm aware that namespaces must be separated by hyphen and have a corresponding filename separated by an underscore in order to work at all. And that the service.edn is just a file that is read and is not required to follow that patten at all and will work anyway.

I do believe it should be consistent though and sanitising the namespace as is done with clj filenames will make this file look a little less strange when generated.

Expected Behavior

Running lein new vase foo-bar creates the file resources/foo_bar_service.edn

Actual Behavior

Running lein new vase foo-bar creates the file resources/foo-bar_service.edn

I can submit a PR to fix this if this change is wanted.

your_first_api.md suggested changes

Description

Documentation for creating your first API in https://github.com/cognitect-labs/vase/blob/master/docs/your_first_api.md needs some edits

  1. The vase.api/schemas value is inconsistent. The first 2 instances show:

:vase.api/schemas [:account/item]

but after that it changes to:

:vase.api/schemas [:accounts/user]

  1. The schema description is confusing. It starts with a cookbook process saying "Start by clearing out the current value of :vase/norms and replace it with this:
     :vase/norms
      {:your-first-api/accounts {}
      }

But after that ":your-first-api/accounts" is replaced with 2 schemas: :accounts/item and :accounts/user, and there is no schema ending in /accounts. That makes for confusion in trying to reconcile the initial edit instructions with the followup snippets. It would be really helpful to have an "end result" edn example available.

  1. This is probably obvious and planned, but it wold be help to have curl examples at each stage, esp. POST examples - since it's not obvious that you need to use "payload" (as mentioned in a recent issue report).

Thanks, this is great stuff! I'm enjoying learning vase. Hope this is helpful. Wasn't sure if you'd prefer pull requests or issue reports for these doc changes.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.