luminus-framework / conman Goto Github PK

The lein compile for clojure fails during deploy which was working until yesterday.
Below is stack trace
23:33:19 - java.lang.ClassCastException: clojure.lang.Keyword cannot be cast to java.base/java.lang.String 23:33:19 - at cemerick.pomegranate.aether$authentication.invokeStatic (aether.clj:173) 23:33:19 - cemerick.pomegranate.aether$authentication.invoke (aether.clj:173) 23:33:19 - cemerick.pomegranate.aether$set_authentication.invokeStatic (aether.clj:184) 23:33:19 - cemerick.pomegranate.aether$set_authentication.invoke (aether.clj:181) 23:33:19 - cemerick.pomegranate.aether$make_repository.invokeStatic (aether.clj:207) 23:33:19 - cemerick.pomegranate.aether$make_repository.invoke (aether.clj:200) 23:33:19 - cemerick.pomegranate.aether$resolve_dependencies_STAR_$fn__219.invoke (aether.clj:798) 23:33:19 - clojure.core$map$fn__4785.invoke (core.clj:2644) 23:33:19 - clojure.lang.LazySeq.sval (LazySeq.java:40) 23:33:19 - clojure.lang.LazySeq.seq (LazySeq.java:49) 23:33:19 - clojure.lang.ChunkedCons.chunkedNext (ChunkedCons.java:59) 23:33:19 - clojure.lang.ChunkedCons.next (ChunkedCons.java:43) 23:33:19 - clojure.lang.PersistentVector.create (PersistentVector.java:73) 23:33:19 - clojure.lang.LazilyPersistentVector.create (LazilyPersistentVector.java:44) 23:33:19 - clojure.core$vec.invokeStatic (core.clj:377) 23:33:19 - clojure.core$vec.invoke (core.clj:367) 23:33:19 - cemerick.pomegranate.aether$resolve_dependencies_STAR_.invokeStatic (aether.clj:798) 23:33:19 - cemerick.pomegranate.aether$resolve_dependencies_STAR_.doInvoke (aether.clj:707) 23:33:19 - clojure.lang.RestFn.applyTo (RestFn.java:137) 23:33:19 - clojure.core$apply.invokeStatic (core.clj:646) 23:33:19 - clojure.core$apply.invoke (core.clj:641) 23:33:19 - cemerick.pomegranate.aether$resolve_dependencies.invokeStatic (aether.clj:815) 23:33:19 - cemerick.pomegranate.aether$resolve_dependencies.doInvoke (aether.clj:809)

Is there some built-in functionality that already does that?

I added the following code to my ns:

(defmacro rebind-connection []
  (cons 'do
    (->> (io/file (io/resource "sql"))
     (file-seq)
     (filter #(.isFile %))
     (map #(str "sql/" (.getName %)))
     (map #((fn [] `(conman/bind-connection *db* ~%)))))))

It works, but feels hackish, because bind-connection requires a string as first argument and passing a symbol does not work.

First up, a disclaimer: maybe this is already supported and I don't quite see how. Please let me know if that's the case.

In Hughsql, we can currently create hugsql-result-fns and hugsql-command-fns. These allow us to transform the results from queries, say, converting keys from underscores to dashes.

I don't see that we can apply a similar transformation to the input values of a queries in a global manner.

This means you if you have a column called first_name, you can get a result out where the key is called :first-name. You still have to name your parameter :first_name if you want the query file to be proper SQL (at least in the case of Postgres).

While Hugsql does recognize :first-name on the sql script as a valid parameter, and sets it accordingly, any tool that operates over the sql file will throw up errors. For instance, IDEA will convert it to :first - name if you reformat the code.

It'll be great to be able to add an option to conman to convert the parameter names from keywords to snake-case during binding (or any other transformation).

I've cloned the repo and will take a stab at it during this week, but will filing it in case anyone can get to it before me.

Hello,

following the simple steps in the README for the setup and usage of conman, I get the following error:

IllegalArgumentException Invalid configuration options: (:adapter) sun.reflect.NativeConstructorAccessorImpl.newInstance0 (NativeConstructorAccessorImpl.java:-2)

when doing this:

(require 'myapp.db :verbose)
(in-ns 'myapp.db)
(mount/start)

The cause of this is the :adapter declaration, if you use the following (as per the README instructions):

(def pool-spec
  {:adapter    :postgresql
   :init-size  1
   :min-idle   1
   :max-idle   4
   :max-active 32
   :jdbc-url "jdbc:postgresql://localhost/myapp"})

If I comment out :adapter line, the problem disapears and I can sucessfully

(mount/start)
{:started ["#'myapp.db/*db*"]}

In order to replicate this, I created a demo app with all the steps in the README here, for which I create a database using

create database myapp;

Is this a bug, or am I doing something wrong here?

Thank you for your time in advance

Not 💯 sure if this is an issue with conman or with my setup but im getting this error
whenever i try to deploy a new luminus app to heroku:

java.lang.Exception: conman could not find the query file:sql/queries.sql, compiling:(core.clj:22:1)
remote: Exception in thread "main" java.lang.Exception: conman could not find the query file:sql/queries.sql, compiling:(core.clj:22:1)

Any clues as to what could be happening ?
Using [conman "0.6.2"] and Luminus version "2.9.11.14"
Thanks in advance!

I have a default luminus project with the +auth and +swagger options, with two query files, queries1.sql and queries2.sql.

So I edit my conman/bind-connection in project.db.core like so:
(conman/bind-connection *db* "sql/queries1.sql" "sql/queries2.sql")

But when I start the server with lein run I'm getting

Exception in thread "main" java.lang.ExceptionInInitializerError
at clojure.main.(main.java:20)
Caused by: java.lang.IllegalArgumentException: db-spec mount.core.DerefableState@2e62903f is missing a required parameter, compiling:(geo.clj:9:34)

The line the error references is where I call a function I defined in sql/queries2.sql.

It's a simple function, so I know there are no mistakes in that file:

-- :name get-some-data :? :*
SELECT * FROM sometable

The weird thing is that if I define this function in sql/queries1.sql I don't get this error. Only functions defined AND called from the second file throw this error.

It's also weird that from the repl I can call the functions defined in the second sql file with no errors:

(start)
(require '[myproject.db.core :as db])
(db/some-function)
; function returns as expected

It feels that the current library is too tightly couple with yesql and that will limit it and its adoption. I propose we at least start by having a very clear statement that this library is intended to be used with any other DSL and that we welcome pull requests of other integrations.

I think as a second step we should try to decouple yesql from the library while still keeping an easy integration. One of the issues we'll encounter here is dependencies. People using one DSL won't want to include all others, so I think we should have a base library, conman and then conman-yesql, conman-honeysql and so on that depend on conman and provide the higher level abstractions/integrations.

What do you think?

I just don't want someone to look at this, think "Oh, this is a good idea, I'll make another one for my own DSL" when they could just extend this one.

In this line:

(with-meta (:name (meta yesql-query#)) (meta yesql-queries#))

https://github.com/luminus-framework/conman/blob/master/src/conman/core.clj#L22

I think meta should refer to yesql-query, not yesql-queries. Am I wrong?

When trying to run the tests, I found this error:

Exception in thread "main" org.h2.jdbc.JdbcBatchUpdateException: Table "FRUITS" not found; SQL statement:
DROP TABLE fruits [42102-187], compiling:(/private/var/folders/k9/pgnpg2h92bv157mw70mhznk80000gn/T/form-init6355575859006062318.clj:1:124)
    at clojure.lang.Compiler.load(Compiler.java:7142)
    at clojure.lang.Compiler.loadFile(Compiler.java:7086)
    at clojure.main$load_script.invoke(main.clj:274)
    at clojure.main$init_opt.invoke(main.clj:279)
    at clojure.main$initialize.invoke(main.clj:307)
    at clojure.main$null_opt.invoke(main.clj:342)
    at clojure.main$main.doInvoke(main.clj:420)
    at clojure.lang.RestFn.invoke(RestFn.java:421)
    at clojure.lang.Var.invoke(Var.java:383)
    at clojure.lang.AFn.applyToHelper(AFn.java:156)
    at clojure.lang.Var.applyTo(Var.java:700)
    at clojure.main.main(main.java:37)
Caused by: org.h2.jdbc.JdbcBatchUpdateException: Table "FRUITS" not found; SQL statement:
DROP TABLE fruits [42102-187]
    at org.h2.jdbc.JdbcStatement.executeBatch(JdbcStatement.java:676)
    at clojure.java.jdbc$execute_batch.invoke(jdbc.clj:404)
    at clojure.java.jdbc$db_do_commands$fn__659.invoke(jdbc.clj:684)
    at clojure.java.jdbc$db_transaction_STAR_.doInvoke(jdbc.clj:587)
    at clojure.lang.RestFn.invoke(RestFn.java:425)
    at clojure.java.jdbc$db_do_commands.doInvoke(jdbc.clj:683)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invoke(core.clj:628)
    at clojure.java.jdbc$db_do_commands.doInvoke(jdbc.clj:690)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invoke(core.clj:630)
    at clojure.java.jdbc$db_do_commands.doInvoke(jdbc.clj:677)
    at clojure.lang.RestFn.invoke(RestFn.java:442)
    at conman.core_test$create_test_table.invoke(core_test.clj:19)
    at conman.core_test$eval2131$fn__2132.invoke(core_test.clj:31)
    at clojure.test$compose_fixtures$fn__7181$fn__7182.invoke(test.clj:681)
    at clojure.test$default_fixture.invoke(test.clj:674)
    at clojure.test$compose_fixtures$fn__7181.invoke(test.clj:681)
    at clojure.test$test_vars.invoke(test.clj:718)
    at clojure.test$test_all_vars.invoke(test.clj:728)
    at clojure.test$test_ns.invoke(test.clj:747)
    at clojure.core$map$fn__4245.invoke(core.clj:2559)
    at clojure.lang.LazySeq.sval(LazySeq.java:40)
    at clojure.lang.LazySeq.seq(LazySeq.java:49)
    at clojure.lang.Cons.next(Cons.java:39)
    at clojure.lang.RT.boundedLength(RT.java:1654)
    at clojure.lang.RestFn.applyTo(RestFn.java:130)
    at clojure.core$apply.invoke(core.clj:626)
    at clojure.test$run_tests.doInvoke(test.clj:762)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invoke(core.clj:624)
    at user$eval85$fn__140$fn__171.invoke(form-init6355575859006062318.clj:1)
    at user$eval85$fn__140$fn__141.invoke(form-init6355575859006062318.clj:1)
    at user$eval85$fn__140.invoke(form-init6355575859006062318.clj:1)
    at user$eval85.invoke(form-init6355575859006062318.clj:1)
    at clojure.lang.Compiler.eval(Compiler.java:6703)
    at clojure.lang.Compiler.eval(Compiler.java:6693)
    at clojure.lang.Compiler.load(Compiler.java:7130)
    ... 11 more
Tests failed.

Simply, the first time the tests are running, there's no fruits table.

I think it would be nice of conman offered some tools for testing. In luminus-template we already have the fixture that connects to the database, runs migrations. I also ended up writing this fixture:

(use-fixtures
  :each
  (fn [test-function]
    (conman/with-transaction
      [t-conn db/conn]
      (jdbc/db-set-rollback-only! t-conn)
      (test-function))))

I ended up putting them both in a function:

(defn set-up-database-test []
  (use-fixtures
   :once
   (fn [f]
     (db/connect!)
     (migrations/migrate ["migrate"])
     (f)))

  (use-fixtures
    :each
    (fn [test-function]
      (conman/with-transaction
        [t-conn db/conn]
        (jdbc/db-set-rollback-only! t-conn)
        (test-function)))))

It would require some work to find the right set of interfaces here, to make sure we can operate with any or many connections. This also ties with the migration library, which I don't think it's a bad thing as we use yesql already.

Hi,

I'm not 100% sure whether this is an issue with conman or hugsql (or even if something wrong with my code). I have the following sql query:

-- :name get-status
-- :doc fetch the status of a specific message
-- :command :query
-- :return :many
/* :require [clojure.string :as s] */
SELECT
    *
FROM
    message_status
/*~
(let [params (filter (comp some? val) params)]
  (when (not (empty? params))
    (str "WHERE "
      (s/join ", "
              (for [[field _] params]
                (str (name field) " = " field))))))
~*/

The idea is that I want to be able to do

(get-status {:message_id 2 :user_id 1 :status "sent"})
;; or
(get-status {:message_id 2})
;; or
(get-status {:user_id 1})
;; etc.

The query seems ok to me:

myapp.db.core=> (get-status-sqlvec {:message_id 2 :user_id 1})
["SELECT\n    *\nFROM\n    message_status\nWHERE message_id = ?, user_id = ?"
 2
 1]

But when I execute it I get an un-helpful error message, and this is where I'm stuck:

myapp.db.core=> (get-status {:message_id 2})
Execution error (ClassCastException) at (REPL:1).
null

myapp.db.core=> *e
#error {
 :cause nil
 :via
 [{:type java.lang.Exception
   :message "Exception in :get-status"
   :at [conman.core$try_query$fn__12775$fn__12776 invoke "core.clj" 32]}
  {:type java.lang.ClassCastException
   :message nil}]
 :trace
 []}

Do you have any idea about what is wrong with this request?

Recently I read the book Web development with clojure, when I read the chapter 8 Picture Gallery, the task A:account registration, there is a function named register!.

When I finish the function and test it, it tell me the mount error *mount.core.DerefableState is missing a required parameter *, and I found it solution in the pragmatic book forum.

But I continue to execute it, it handle me a java nullpointerexception, I use the cider, so I debug it.

And it break in the db/create-user!, it tell me
Cannot issue data manipulation statements with executeQuery()

I know a litte about java, I found this question in stackoverflow
http://stackoverflow.com/questions/1905607/cannot-issue-data-manipulation-statements-with-executequery

it say the executequery() only in select query, so up to now, it seems that a error about sql query, and I can't found it solution,does the hugsql package this method?And how can I fix this problem?

here is my code:

(defn register! [{:keys [session]} user]
  (if (registration-errors user)
    (response/precondition-failed {:result :error})
    (try
      (db/create-user!
       (-> user
           (dissoc :pass-confirm)
           (update :pass hashers/derive)))
      (-> {:result :ok}
          (response/ok)
          (assoc :session (assoc session :identity (:id user))))
      (catch Exception e
        (handle-registration-error e)))))

(register! {} {:id "foo" :pass "12345678" :pass-confirm "12345678"})

here is the error message:

=> #error {
:cause "Can not issue data manipulation statements with executeQuery()."
:via
[{:type java.sql.SQLException
:message "Can not issue data manipulation statements with executeQuery()."
:at [com.mysql.jdbc.SQLError createSQLException "SQLError.java" 1055]}]

Do you think this library should call to-jdbc-url? That could be another nice automatic think that people could take out of this, so people using conman won't even think about the fact that there are two different URL formats out there.

I would even be happy integrating to-jdbc-url into this library and killing that other one, but this is not necessary.

How to connect to oracle db?
What will be the url how to change the library?

It would be really nice if a function could be provided to conman/bind-connection that would wrap database-touching functions.

Something like

(conman/bind-connection *db*
  {:wrapper-func log-it}
  "sql/queries.sql")

where a theoretical create-user! call would be defined as

(defn create-user! [map]
  (REAL-create-user (log-it map)))

or something to that effect.

Hello,

Is there any way to use dynamic table name like SELECT SAMPLETIME, FROM :r_table ?

regards,

Fernando

super minor but should this line be updated to:
(with-transaction [conn {:isolation level :read-only? true}]

I think it would be nice to have a function that creates a connection only if there are no connections at the moment, something like connect-if-not-connected or ensure-connected

I'm using sqlite and yesql in my project (but not luminus). I need the snake-case and camel-case functionality and thought I'd try and give conman a go.

I was connecting successfully using pure yesql with the following

(def db-spec
  {:classname "org.sqlite.JDBC"
   :subprotocol "sqlite"
   :subname "data/metime.sqlite"})

Now when I'm trying to use conman I guessed at the following spec:

(def db-spec
  {:dataSourceClassName "org.sqlite.SQLiteDataSource"
   :jdbc-url "jdbc:sqlite/data/metime.sqlite"
 })

When I try (cc/connect! conn db-spec)

I get the following error:

SQLException No suitable driver  java.sql.DriverManager.getDriver (DriverManager.java:278)

Any ideas what my db-spec should be set to? and indeed will conman work in such a situation. i.e. Just using Yesql and not luminus?

I have a complex query which works fine in h2 web console and when I use clojure.java.jdbc api and returns meaningful results . When I try to use conman it returns nil values.

--:name get-metals-by-id :? :1
with recursive sostav (pref, num, name, qnt, gold, silver, pl, pal, id) as (
SELECT u2.prefix, u2.num, u2.name, c.qnt,
u2.gold, u2.silver, u2.pl, u2.pal, u2.id
FROM (unit u inner join contain c on u.id = c.cont_id)
inner join unit u2 on u2.id  = c.unit_id
where u.id = :id
union all
select u.prefix, u.num, u.name, c.qnt, u.gold, u.silver, u.pl, u.pal, u.id
from ( sostav s inner join contain c on s.id = c.cont_id)
inner join unit u on c.unit_id = u.id)
select sum(cast(gold as double)*cast(qnt as int)) as gold,
sum(cast(silver as double)*cast(qnt as int)) as silver,
sum(cast(pl as double)*cast(qnt as int)) as pl,
sum(cast(pal as double)*cast(qnt as int)) as pal
from sostav;

As i see there is no access to HugSQL sqlvec functions. Is there a way to solve my problem?

To my knowledge it is currently not possible to make use of the options parameter of the encapsulated hugsql.core/def-db-fns function (docs) when using conman.core/bind-connection to set connection specific configurations.

I propose the use of an options map for bind-connection, similar showcased and helpful in #19, which is then handed to def-db-fns.

In my case I need to enable :quoting (:mysql) for HugSQL functions per default. Currently this is solved by always providing the database and options map for each call of a generated HugSQL function (example in first snippet), which is far from ideal (second snippet), since connection aware functions are one key feature of conman.

(db/get-user *db* {:id "foo"} {:quoting :mysql})

(db/get-user {:id "foo"})

Since 0.8.2, when passing an option map to bind-connection or bind-connection-deref, that map is never evaluated. Instead the contents are passed to the bound Hugsql functions as literals.

This is a problem when doing something like:

(bind-connection-deref conn {:adapter (hugsql-adapter-clojure-java-jdbc)} "queries.sql")

Hugsql then tries to use that literal list '(hugsql-adapter-clojure-java-jdbc) as adapter instead of the value of the expression, resulting in obscure exceptions such as java.lang.IllegalArgumentException: No implementation of method: :on-exception of protocol: #'hugsql.adapter/HugsqlAdapter found for class: clojure.lang.PersistentList.

This bug was introduced in the refactor of 8702e2b. Previously the options were split out and then unquoted. Now the filenames list is emitted untouched and then quoted so the options remain unevaluated, while load-queries expects evaluated args.

From what I can tell the easiest fix is to splicing-unquote filenames (ie. (conman.core/load-queries ~@filenames)) and change load-queries to take & args. The logic in load-queries to split out the options from the filenames should then just work.

I am getting the following value from a SELECT where val is of type JSON.

#object[org.postgresql.util.PGobject
               0x7496f3d3
               "[json-string-here]"]

After downgrading back to 0.8.4, I am getting JSON parsed as Clojure map correctly. The parsing is done by this code, based on luminus template:

(extend-protocol jdbc/IResultSetReadColumn
  Timestamp
  (result-set-read-column [v _2 _3]
    (.toLocalDateTime v))
  Date
  (result-set-read-column [v _2 _3]
    (.toLocalDate v))
  Time
  (result-set-read-column [v _2 _3]
    (.toLocalTime v))
  Array
  (result-set-read-column [v _ _] (vec (.getArray v)))
  PGobject
  (result-set-read-column [pgobj _metadata _index]
    (let [type (.getType pgobj)
          value (.getValue pgobj)]
      (case type
        "json" (parse-string value true)
        "jsonb" (parse-string value true)
        "citext" (str value)
        value))))

I'm not quite sure what in fce1c01 causes the issue, but we've been having a fight here today in a luminus project.

AFAICT datasource-config in make-config is being called multiple times. In latter calls the pool-spec value is null, blowing up hikari-cp.core/validate-options.

To reproduce with luminus (2.9.10.50), simply generate a new project with +postgres and then add [app.db.core :as db] to the home handler and boot the app up. Alternatively start a lein repl and call (conman/connect! {...}) manually.

It should fail with the following exception: java.lang.IllegalArgumentException: Invalid configuration options: (:driver-class-name), because the pool-spec is null.

We confirmed this on two different dev machines.

Let me know if I can provide more information.

I am using the user namespace in my luminous application to restart parts of my application from the REPL. But this does not work with changing database queries in the queries.sql file. Here is what I have tried that does not work:

(defn restart-db []
  (mount/stop #'mokens.db.core/*db*)
  (mount/start #'mokens.db.core/*db*)
  (conman/bind-connection #'mokens.db.core/*db* "sql/queries.sql"))

How can I make this work so my queries are reloaded when I call user/restart-db from the REPL?

Hi,
it seems Hikari 3.x is not compatible with Java 8 anymore, so the bump gives this for conman 0.9.6:

Caused by: java.lang.UnsupportedClassVersionError: com/zaxxer/hikari/HikariConfig has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0

conman 0.9.5 still works with 1.8.0_352.

I'm not advocating for keeping backwards compatibility, but I think it would warrant a note in the Changelog.

I was using conman version 0.6.2 for few months in my project. Today upgraded to latest 0.7.3 and started getting this;
java.lang.ClassCastException: clojure.lang.PersistentArrayMap cannot be cast to java.util.concurrent.Future
at clojure.core$deref_future.invokeStatic(core.clj:2206) ~[clojure-1.8.0.jar:na]
at clojure.core$deref.invokeStatic(core.clj:2228) ~[clojure-1.8.0.jar:na]
at clojure.core$deref.invoke(core.clj:2214) ~[clojure-1.8.0.jar:na]

Switching to 0.7.2 solved it though.

Getting this when modifying the postgres schema... the current app just errors out when querying on the edited table.

I will sometimes get this error on load (AOT compiled). It only happens around 20% of the time-ish, and it seems to be AOT related since restarting the process consistently has the error.

No such var: adp/hugsql-adapter-clojure-java-jdbc

Related to layerware/hugsql#46

I'm struggling connecting luminus to oracle db using the following URL
jdbc:oracle:thin:username/[email protected]:1521:orcl

I'm getting -- Error creating DB connection for jdbc:oracle:thin....

This change 632d579
prevents us from catching and handling more specific exceptions such as java.sql.SQLException and its subclasses such as java.sql.SQLIntegrityConstraintViolationException that we were relying on.

First, I have to say that I'm a beginner in Clojure/Luminus/Conman and so : )

I have had a problem counting tuples after an insertion. The function counting tuples seems to be out of the transaction. I tested it by creating the project like this:
lein new luminus test_transaction +postgres +cider +swagger +service +war
After setting up the database in profiles.clj, I added the next code to resources/sql/queries.sql:

-- :name get-users :? :*
-- :doc retrieve all users.
SELECT * FROM users

Next I modified the function test-users by adding the last line.

(deftest test-users
  (jdbc/with-db-transaction [t-conn *db*]
    (jdbc/db-set-rollback-only! t-conn)
    (is (= 1 (db/create-user!
               t-conn
               {:id         "1"
                :first_name "Sam"
                :last_name  "Smith"
                :email      "[email protected]"
                :pass       "pass"})))
    (is (= {:id         "1"
            :first_name "Sam"
            :last_name  "Smith"
            :email      "[email protected]"
            :pass       "pass"
            :admin      nil
            :last_login nil
            :is_active  nil}
           (db/get-user t-conn {:id "1"})))
    (is (= 1 (count (db/get-users t-conn))))))

Finally this is the result of lein test:

$ lein test                                                     
Retrieving org/webjars/bower/tether/1.3.7/tether-1.3.7.pom from central                               
Retrieving org/postgresql/postgresql/9.4.1211/postgresql-9.4.1211.pom from central                    
Retrieving org/postgresql/pgjdbc-core-parent/1.1.1/pgjdbc-core-parent-1.1.1.pom from central          
Retrieving org/postgresql/pgjdbc-versions/1.1.1/pgjdbc-versions-1.1.1.pom from central                
Retrieving org/webjars/bower/tether/1.3.7/tether-1.3.7.jar from central                               
Retrieving org/postgresql/postgresql/9.4.1211/postgresql-9.4.1211.jar from central                    
2016-10-05 10:23:37,429 [main] DEBUG org.jboss.logging - Logging Provider: org.jboss.logging.Slf4jLoggerProvider                                                                                            

lein test test-transaction.test.db.core                                                               
read config from resource: "config.edn"                                                               
read config from file: ".lein-env"
2016-10-05 10:23:46,882 [main] INFO  migratus.core - Starting migrations 
2016-10-05 10:23:46,939 [main] INFO  migratus.database - creating migration table 'schema_migrations' 
2016-10-05 10:23:46,989 [main] INFO  migratus.core - Running up for [20161005101947] 
2016-10-05 10:23:46,990 [main] INFO  migratus.core - Up 20161005101947-add-users-table 
2016-10-05 10:23:47,000 [main] DEBUG migratus.database - found 1 up migrations 
2016-10-05 10:23:47,010 [main] DEBUG migratus.database - marking 20161005101947 complete 
2016-10-05 10:23:47,028 [main] INFO  migratus.core - Ending migrations 

lein test :only test-transaction.test.db.core/test-users

FAIL in (test-users) (core.clj:37)
expected: 1
  actual: 0
    diff: - 1
          + 0

lein test test-transaction.test.handler

lein test :only test-transaction.test.handler/test-app

FAIL in (test-app) (handler.clj:9)
main route
expected: 200
  actual: 404
    diff: - 200
          + 404

Ran 2 tests containing 5 assertions.
2 failures, 0 errors.
Tests failed.

I updated conman in my dependencies to 0.4.7 and changed my code, according to your documentation. But when I now execute a database query, the following error is thrown:

db-spec mount.core.DerefableState@3cb8256c is missing a required parameter clojure.java.jdbc/get-connection (jdbc.clj:292)

My connection code is the following:

(ns wamp-authenticator.db.core
  (:require
    [clojure.java.jdbc :as jdbc]
    [mount.core :refer [defstate]]
    [conman.core :as conman]
    [environ.core :refer [env]])
  (:import [java.sql
            BatchUpdateException
            PreparedStatement]))

(def pool-spec
  {:adapter    :mysql
   :init-size  1
   :min-idle   1
   :max-idle   4
   :max-active 32})

(defn connect! []
    (conman/connect!
        (assoc
            pool-spec
            :jdbc-url (env :database-url))))

(defn disconnect! [conn]
    (conman/disconnect! conn))

(defstate ^:dynamic *db*
          :start (connect!)
          :stop (disconnect! *db*))

(conman/bind-connection *db* "queries.sql")

I also looked into the code of the luminus template and in the documentation, but didn't find a clue. Maybe you could help me with this.

Is this how connect!/disconnect! is supposed to be used?

(let [conn (connect! {:jdbc-url "..."})]
  ; do stuff with conn
  ; ...
  ; then release conn:
  (disconnect! conn))

If so, and given that conn does not have a :datasource key, I think disconnect! never does anything: https://github.com/luminus-framework/conman/blob/master/src/conman/core.clj#L162

EDIT:
Noticed this because I was getting PSQLExceptions with "FATAL: sorry, too many clients already" when I reloaded the system.

The IBM® Data Server Driver for JDBC and SQLJ supports two types of connectivity: type 2 connectivity and type 4 connectivity.
Type 2 connectivity requires a DB2 client (or server) installation and has a restriction in that the "bitness" of the java application must match with the DB2 client ( or server) instance.
The Data Server Driver for Type 4 connection does not require the use of the native DB2 libraries.
So Oracle:
Oracle Pool API

 void   setDriverType(java.lang.String dt)
          Set the JDBC driver type

It would be nice if:

(defn- make-config
  [{:keys [jdbc-url datasource datasource-classname username
           password auto-commit? conn-timeout idle-timeout
           max-lifetime min-idle max-pool-size pool-name]}]
  (let [cfg (HikariConfig.)
        uri (when jdbc-url (to-jdbc-uri jdbc-url))]
    (when uri                  (.setJdbcUrl cfg uri))
    (when datasource           (.setDataSource cfg datasource))
    (when datasource-classname (.setDataSourceClassName cfg datasource-classname))
    (when username             (.setUsername cfg username))
    (when password             (.setPassword cfg password))
    (when (some? auto-commit?) (.setAutoCommit cfg auto-commit?))
    (when conn-timeout         (.setConnectionTimeout cfg conn-timeout))
    (when idle-timeout         (.setIdleTimeout cfg conn-timeout))
    (when max-lifetime         (.setMaxLifetime cfg max-lifetime))
    (when max-pool-size        (.setMaximumPoolSize cfg max-pool-size))
    (when min-idle             (.setMinimumIdle cfg min-idle))
    (when pool-name            (.setPoolName cfg pool-name))
    cfg))

have a .setDriverType

int my project, we can use
(conman/bind-connection db "sql/queries.sql")
to bind a sql file to a connection,
but, when i compile the project to a jar package, and run it, then i connect to the server by nrepl and when i modify the sql, so how can i rebind the sql file to the connection?

I was about to submit a pull request for this, but given I'm not that familiar with conman, decided to open an issue instead in case I'm missing something.

There are a couple of things that look like errors on the documentation for with-transaction:

• The example refers to with-db-transaction . Was this the old name?
• The same snippet seems to receive two name and connection before it goes to the k-v ones. The example on the README only uses to the name. Is that correct?
• It refers the reader to jdbc/db-transaction* for more details. Shouldn't that be jdbc/with-db-transaction?

Would be nice to get a plan to support next.jdbc!

if we face thousands of millions of data, we need to split the databases and tables, there is sharding-jdbc for java, then it is need to port to clojure ?

Hello,

Is there any way to do something like this?
(conman/bind-connection *db* (str "sql/" database ".sql")) ?
I tried but it don´t compile by lein uberjar.

Regards

The options that Yesql offers for specifying processors in its query functions are sometimes useful, and it would be nice if conman offered some support for that feature in those cases.

For instance, this could be done relatively clearly and at little cost with a 3-ary form of the query functions in bind-connection, something like

([args# conn# options#] (yesql-query# args# (merge {:connection conn#} options#)))

where options# is a map of processors. This could even replace the 2-ary form directly
with

([args# conn# & [options#]] (yesql-query# args# (merge {:connection conn#} options#)))

if that were desirable.

What do you think?
Thanks.

On my luminus app I have one test file, when I try to run it with the command:

$ lein with-profile test test

I'm getting

Caused by: java.lang.IllegalArgumentException: db-spec mount.core.DerefableState@5cb2f9e7 is missing a required parameter

Digging a little I see that the problem is the app is unable to connect with the database.

The weird thing is that if I run

$ lein with-profile test run

I mean, run the whole app in the "test" profile, all works perfectly and I can insert/update/delete in the database. I'm guessing that the function:

(start-app args)

in the core.clj file, does something that "repl" and "test" don't do, but since I'm kind of newbie in the clojure world, I don't know what part could be. Maybe do I need to add a function to run the tests in luminus?

This are my project and core files:

https://github.com/aarkerio/ZentaurLMS/blob/development/project.clj
https://github.com/aarkerio/ZentaurLMS/blob/development/src/clj/zentaur/core.clj

when I use oracle db with this library , it is not work.

I'm trying to use the batch form of an insert statement;
INSERT INTO Table (column1, column2) VALUES ("value1", 1),("value2",2)
This because I want to be able to insert thousands of records per second.

According to the HugSQL documentation this is possible by using :t*:params in the query definition in the sql file. However for this to work the function 'map-of-sqlvec-fns' should be called when reading the sql query definitions. After this it is possible to call a database function with the suffix -sqlvec if you want to use the batch version.

I think it is easy to add this to the conman bind-connection function by changing this line:
(hugsql/map-of-db-fns file options)
to
(merge (hugsql/map-of-db-fns file options) (hugsql/map-of-sqlvec-fns file options)

It is then up to the user which function to call, the one with or without the suffix -sqlvec.

I would very much appreciate if you can add this functionality.

Hi DS,

It seems that the (def pool-spec ... sections within the Conman readme / Luminus website documentation both contain stale references to config keys which were valid for Conman when it used clj-dbcp, but no longer valid now that it uses @tomekw's HikariCP wrapper.

For example, the :init-size key doesn't mean anything in HikariCP, and hence won't do anything.

This may trip users up - it may make sense to link to @tomekw's readme so that users understand what exact parameters HikariCP takes?

When a function generated by bind-connection throws an exception, the stack frame looks something like myapp.queries/eval29331/fn--29332. The name for this function is known at function definition time, but it's not included in the function.

I believe it would be more debuggable to construct a fn that has an internal name in order to improve stack traces. However, I think the way things are structured right now this would require using eval--something like this:

       (doseq [[id# {fn# :fn {doc# :doc} :meta}] fns#]
         (intern *ns* (with-meta (symbol (name id#)) {:doc doc#})
                 (eval (list 'fn (symbol name #id)
                             '([] (fn# ~conn {}))
                             '([params#] (fn# ~conn params#))
                             '([conn# params#] (fn# conn# params#))
                             '([conn# params# opts# & command-opts#]
                              (apply fn# conn# params# opts# command-opts#))))))

Thoughts?