incanter / incanter Goto Github PK

When running lein deps with [incanter "1.3.0-SNAPSHOT"] as a dependency, I get this error:

Retrieving org/mongodb/mongo-java-driver/2.6.5/mongo-java-driver-2.6.5.pom (1k)
    from http://repo1.maven.org/maven2/
Could not transfer artifact org.mongodb:mongo-java-driver:pom:2.6.5 from/to central (http://repo1.maven.org/maven2): Checksum validation failed, no checksums available from the repository
Could not find artifact org.mongodb:mongo-java-driver:pom:2.6.5 in clojars (https://clojars.org/repo/) 

I fixed it for now by excluding incanter/incanter-mongodb like this : [incanter "1.3.0-SNAPSHOT" :exclusions [incanter/incanter-mongodb]]

But I hope incanter-mongodb's mongodb driver dependency can be updated!

Just not to forget...

We need to fix scripts so they can accept lein2 command name as optional parameter, or obtain it through environment variable

The following program (adapted from Shiffman's processing code) doesn't work correctly because the drawable height of the created window is 178 pixels instead of 200 as coded. It does work correctly when the height parameters are changed to 222. It looks suspiciously like the missing 22 pixels are hiding under the window title bar since it is about the same size.

Using the prebuilt zip.

Java version "1.6.0_20" on OS X 10.6.3. Running using script/swank and Emacs.

(ns multiprob
(:use [incanter.core]
    [incanter.processing]))
(let [
  p1 (ref 0.05)
  p2 (ref (+ @p1 0.8))
  r (ref nil)
  sktch (sketch
         (setup []
                (doto this
                  (size 200 200)
                  smooth
                  (framerate 30)
                  (background 63 63 63)
                  (color-mode RGB 255 255 255)
                                        ))
         (draw []
               (dosync
                (ref-set r (rand)))

               (doto this
               (fill 0.0 1.0)
               (rect 0 0 (width this) (height this))
               (fill (if (< @r @p1)
                        255
                        (if (< @r @p2)
                          150
                          0)))

                 (stroke 200)
                 (rect (rem (* 10 (frame-count this)) (width this))
                       (rem (* 10 (quot (* 10 (frame-count this)) (width this))) (height this))
                       10 10)
                 (println (width this) (height this))
                 )))
  ]
  (view sktch :size [200 200]))

As a side note, I had an earlier version using the original clj-processing lib working without this problem.

I just need to use decomp-qr when it hits me that decomp-qr is not defined. Then I went through the source tree and found that decomp-qr is commented out. I'm wondering when it would be back again.

using with-rotation from incanter.processing it doesnt work because the required sketch arguments for push-matrix rotate and pop-matrix aren't present in the macro or passed in from without.

In incanter.core, in (defmethod sel [incanter.Matrix true], line 295 has the call (filter filter mat). filter here has been bound to the optional function provided by the :filter keyword; the call to the core filter function needs to have the clojure.core namespace specified (as was correctly done in
(defmethod sel [::dataset true]

given:

(def A (matrix (range 1 7) 2))

(sel A :all 1)

returns the entire matrix, rather than just the second column. This is because the dispatch function for sel (a multimethod) simply checks whether the second argument is a keyword. This method:

(defmethod sel [incanter.Matrix false]

has the code which handles :all passed as the rows parameter; however that code will never be reached, because such calls will be sent to the method for [incanter.Matrix true].

(This can be worked around by using true instead of :all to select all rows).

There is a typo in the denominator of the cosine similarity. It reads ||a|| * ||a|| but should read ||a||*||b||. I'm guessing this also affects tanimoto-coefficient.

Documentation to add-lines macro states that it works on line plots which could someone misinterpret as a line-chart, whereas the multimethod add-lines* is defined for xy-plots instead.
Just a minor issue though.

Version 1.3.0 works. This example breaks with the following exception returned using 1.4.0:

(use '(incanter core stats charts datasets))

(with-data (get-dataset :co2)
(view (bar-chart :Type :uptake
:title "CO2 Uptake"
:group-by :Treatment
:x-label "Grass Types" :y-label "Uptake"
:legend true)))

java.lang.IllegalArgumentException: No matching method found: addValue for class org.jfree.data.category.DefaultCategoryDataset

Just pulled incanter down today, and its complaining about a function call to applyCurrentTheme in the charts namespace. Error is being called at incanter/charts.clj:2447

Hi,
I have two vectors x and y, and create a matrix using the Kronecker product of them. What I would like back is a matrix with dimension length(x) and length(y). I think I just get back a one dimensional matrix. Is it possible to resize this matrix or the have the kronecker function return a specific dimension size?

Also is there an Incanter function to return the dimensions of a given matrix?

incanter.internal has a macro called combine-with which is used to implement some of the arithmetic functions (like plus). It is basically a big cond block. One of the conditions (starting on line 115) is:

  (and (number? ~A) (coll? ~B) (coll? (first ~B)))
    (let [~mA (make-matrix ~A (.rows ~B) (.columns ~B))
          ~mB (make-matrix ~B)]
      (.assign ~mA ~mB ~df))

Clearly the calls to .rows and .columns are mistakes; presumably, what was intended was something like:

    (let [~mA (make-matrix ~A (count ~B) (count (first ~B)))

(conj-rows
(dataset [:a :b] [[1 10]])
(dataset [:b :a] [[10 1]]))

; [:a :b]
; [1 10]
; [10 1]

stats.clj, in sweep, stat-fn is used instead of stat in the :or map

Incanter echoes input not evaluating it.

Screenshot: https://img.skitch.com/20110823-pepqfx3dxcq9qgymd5hsfgm8d1.png

Varsion: Clojure 1.2.0-master-SNAPSHOT http://incanter.org/downloads/Incanter.dmg

Hi Liebke,

When I use the "read-dataset" function to import data from CSV, I get the following error. The value "akshay" is not at the end of the vector. Instead it is put in front of what should be a bunch of nil values.

CSV:
1303938000,fraglimit,,,,akshay

Dataset (imported by "read-dataset"):
[1303938000 "fraglimit" "akshay" nil nil nil]

Are nil values allowed in the CSV?

Noticed this line:
[swank-clojure "1.3.0-SNAPSHOT"]

in the dependencies for incanter, is this an actual runtime dependency, or can it be moved to dev-dependencies?

When using set-stroke to try to increase the thickness of lines of a line-chart that's plotting data grouped into groups, only the first group seems to be affected. For example:

(doto (line-chart :parallel :min
                  :group-by :config
                  :legend true
                  :data (...))
  (set-stroke :width 3)
  (view
    :width 800
    :height 600))

Is it currently possible to create polar charts with incanter?

I couldn't find any functions in incanter.charts or google so I'm about to create my own JFreeChart wrapper function. Thought I might as well give a shout incase I'm missing something? :)

Incanter's infix feature suggest a surprising ordering of simple arithmetic expressions. Usually, + and - are given an equal precedence value, as well as * and / are given an equal one and higher than + and -.

The current setup makes simple expressions result in unexpected values, such as:

($= 10 - 1 + 10) ;=> -1, expected 19
($= 1 / 2 * 3) ;=> 1/6, expected 3/2

The fix is to correct lines 2625–2628 in modules/incanter-core/src/incanter/core.clj to read:

(defop '- 60 'incanter.core/minus)
(defop '+ 60 'incanter.core/plus)
(defop '/ 80 'incanter.core/div)
(defop '* 80 'incanter.core/mult)

(prod ()) is zero but mathematically it should be 1 so the recursive definition of itself has the correct edge case

The Chebyshev distance metric is missing an abs.

(chebyshev-distance '(0 3 4 5) '(7 6 3 -1))
=> 6 (expected 7)

I conformed these work in the 'sketch' as (.key this) , (.pmouseX this) and (.pmouseY this) at incanter/incanter-processing "1.2.3".

(def ds (to-dataset (partition 3 (range 9)))) (= (to-map ds) (to-map ds)) ;; true (= ds (to-dataset (to-map ds))) ;; false, dataset becomes somehow transposed

PS: I do not mean that = does not work but that the data is really different, i.e., col-0 is (0 3 6) first and after mapping forth and back it is (8 7 6)

I noticed that (xy-plot [0 1] [2 4] :x-label "hello") no longer sets the x axis label nor does :y-label set the y axis label. (set-x-label chart label) still works.

Hi! First off, thanks so much for Incanter!

The chart macros don't take care to avoid multiple expansions of the arguments, which makes them unsafe if called with expressions that have side effects, and inefficient if called with expressions that require significant computation. For example, the 'values' parameter to bar-chart is expanded three times. (I believe this is a common gotcha in writing macros.)

I spotted this because the on-hover labels on the charts showed the raw expressions, not the values, so that's another undesirable side effect.

IMHO, the best solution may be to convert these macros to regular functions -- it's not at all clear to me why they must be macros.

Here's an example: http://gist.github.com/259024

Posted on clojuredocs by laughingboy: http://clojuredocs.org/v/3324#comments

http://github.com/liebke/incanter/blob/master/modules/incanter-core/src/incanter/stats.clj#L1875

When I was using Incanter to draw scatter plots I ended up running into a case in which the data provided only had one row, because the data was provided by a user. A simple test case that shows the problem:

(scatter-plot :col-0 :col-1 :data (to-dataset [[1 2 3]]))
java.lang.RuntimeException: java.lang.IllegalArgumentException: Don't know how to create ISeq from: java.lang.Integer (NO_SOURCE_FILE:0)

I tried to figure out what was going wrong and it seems that basically when ($ :col some-dataset) is called it usually returns a sequence, but in the case in which there is only one row it instead returns a single item. However, the scatter-plot call doesn't anticipate this. It does something like:

_x (if (coll? x) (to-list x)  (if (coll? x) (to-list x) ($ x data))

I got things working in this case by replacing that section with

   _x (if (coll? x) (to-list x) 
     (if-let [x-list (sel data :cols x)]
       (if (seq? x-list) x-list (list x-list))))

The equivalent expression was also needed in y.

I haven't checked to see if this same problem is in other charts.

SNAPSHOTs don't count as releases (because you can't depend on them in your own released artefacts).

I installed incanter 1.4.0 on lein but was unable to create matrix in REPL, but it works fine for 1.3.0. Maybe a package dependency issue?

Bug details can be found in the following link:
http://stackoverflow.com/questions/14087222/unable-to-create-matrix-in-incanter-1-4-0

I like to be able to run the Incanter repl from wherever I like. To achieve this, I modified the repl shell script a little. Here's the diff if you think it'd be useful.

Thanks for making Incanter.

-Ed

diff --git a/script/repl b/script/repl
index dace2e0..d554c18 100755
--- a/script/repl
+++ b/script/repl
@@ -1,8 +1,11 @@
#!/bin/sh
-CLASSPATH=src:test:config:data:.

-for f in lib/.jar; do
+BASEDIR=dirname $0
+
+CLASSPATH=$BASDIR/../src:$BASDIR/../test:$BASDIR/../config:$BASDIR/../data:$BASDIR/..
+
+for f in $BASEDIR/../lib/.jar; do
CLASSPATH="$CLASSPATH":$f
done

-java -Xmx1G -cp "$CLASSPATH" jline.ConsoleRunner clojure.main -i script/run.clj -r
+java -Xmx1G -cp "$CLASSPATH" jline.ConsoleRunner clojure.main -i $BASEDIR/run.clj -r

I just tried to upgrade to incanter 1.2.4 and noticed that it keeps the modules in version 1.2.3.
A look into the project.clj of the incanter 1.2.4 jar reveals that the dependencies have not change in there.

So far, every graph I've attempted to display over a forwarded X connection presents a blank, grey window. The same graphs display correctly when logged in to the console directly.

I don't know if this is an issue with Incanter, or one of its dependencies, but if possible, it'd be nice to have it fixed.

Thanks

Rob Browning

In the SVD rather than returning S as a diagonal matrix, per the docs and behaviour of Colt, the diagonal of that matrix is returned as a vector. See

https://github.com/liebke/incanter/blob/master/modules/incanter-core/src/incanter/core.clj#L899

The patch is just to remove the call to (diag ...) and I'm happy to submit it if we're agreed that this is the sensible behaviour.

(:t-probs (linear-model [-1 1 3 0] [2 3 -1 0]))
=> (1.6454972140743414 1.5345224792532302)

Looks like it's using ":lower-tail" instead of ":lower-tail?"

The graph from the xy-plot example is rendered without the title and axis labels:

(def x2 (range 0 20 0.1))
(def gamma-plot (xy-plot x2 (pdf-gamma x2 :shape 1 :rate 2)
                           :legend true
                           :title "Gamma PDF"
                           :y-label "Density"))
(view gamma-plot)

In incanter.stats/t-test, there seems to be an issue with the one-sided-p calculation.

The lower-tail? argument should be passed as ":lower-tail? lower-tail?" instead of ":lower-tail lower-tail?"

I have a small patch.

Hello,

Would there be interest in the Incanter project in working towards a common interface for vector / matrix maths in Clojure?

The potential idea is to create an API / abstraction for matrix maths that is fairly generic and will support multiple possible matrix maths implementations (e.g. pure Java vs. JBLAS vs. Colt vs. some other custom format used by an external program).

This would free projects like Incanter from having to fix on a specific matrix implementation, and allow generic matrix functionality to be developed on a common base that is useful across the Clojure landscape.

I believe Clojure protocols are fast enough that performance would not be a concern.

See this thread in the Clatrix project: tel/clatrix#7

Obviously, Incanter would be a big and important consumer of such an API / abstraction so wanted to test for interest in the Incanter project before deciding if this is a project worth committing effort to.

(box-plot :col-0 :data (to-dataset [[1 2 3] [4 5 6]]) :group-by :col-2)

throws exception

java.lang.IllegalArgumentException: No matching method found: add for class org.jfree.data.statistics.DefaultBoxAndWhiskerCategoryDataset
 at clojure.lang.Reflector.invokeMatchingMethod (Reflector.java:80)
    clojure.lang.Reflector.invokeInstanceMethod (Reflector.java:28)
    incanter.charts$box_plot_STAR_.doInvoke (charts.clj:2305)
    clojure.lang.RestFn.applyTo (RestFn.java:139)
    clojure.core$apply.invoke (core.clj:601)
    incanter.charts_tests/fn (charts_tests.clj:117)
    clojure.test$test_var$fn__6926.invoke (test.clj:701)
    clojure.test$test_var.invoke (test.clj:701)
    clojure.test$test_all_vars$fn__6930$fn__6937.invoke (test.clj:717)
    clojure.test$default_fixture.invoke (test.clj:671)
    clojure.test$test_all_vars$fn__6930.invoke (test.clj:717)
    clojure.test$default_fixture.invoke (test.clj:671)
    clojure.test$test_all_vars.invoke (test.clj:713)
    clojure.test$test_ns.invoke (test.clj:736)
    clojure.core$map$fn__4087.invoke (core.clj:2434)
    clojure.lang.LazySeq.sval (LazySeq.java:42)
    clojure.lang.LazySeq.seq (LazySeq.java:60)
    clojure.lang.Cons.next (Cons.java:39)
    clojure.lang.RT.boundedLength (RT.java:1633)
    clojure.lang.RestFn.applyTo (RestFn.java:130)
    clojure.core$apply.invoke (core.clj:603)
    clojure.test$run_tests.doInvoke (test.clj:751)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.core$apply.invoke (core.clj:601)
    user$eval5066$fn__5069.invoke (NO_SOURCE_FILE:1)
    user$eval5066.invoke (NO_SOURCE_FILE:1)
    clojure.lang.Compiler.eval (Compiler.java:6511)
    clojure.lang.Compiler.eval (Compiler.java:6501)
    clojure.lang.Compiler.eval (Compiler.java:6501)
    clojure.lang.Compiler.eval (Compiler.java:6477)
    clojure.core$eval.invoke (core.clj:2797)
    clojure.main$eval_opt.invoke (main.clj:297)
    clojure.main$initialize.invoke (main.clj:316)
    clojure.main$null_opt.invoke (main.clj:349)
    clojure.main$main.doInvoke (main.clj:427)
    clojure.lang.RestFn.invoke (RestFn.java:421)
    clojure.lang.Var.invoke (Var.java:419)
    clojure.lang.AFn.applyToHelper (AFn.java:163)
    clojure.lang.Var.applyTo (Var.java:532)
    clojure.main.main (main.java:37)

It appears that this happens when there is only one element in a box plot group AND there is more than one element in the dataset in total.

The following

(box-plot :col-0 :data (to-dataset [[1 2 3] [4 5 3]]) :group-by :col-2)

doesn't throw an exception.

Launching script/repl leads to:
Exception in thread "main" java.lang.NoClassDefFoundError: jline/ConsoleRunner
Caused by: java.lang.ClassNotFoundException: jline.ConsoleRunner
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)

Copying jline-0.9.94.jar from lib/dev to lib fixes this issue.
Is this a problem with the dependencies?

The documentation at http://liebke.github.com/incanter/ seems to be missing, there are only 6 public methods total, drilling down the documentation seems to be missing completely for most namespaces (example: http://liebke.github.com/incanter/datasets-api.html )

Noticed that $joining two datasets returns different columns depending on the order of datasets defined. Let's say I have datasets a and b. First 3 rows of a (take 3 (:rows a))

{:qual 0.68, :pos "714038", :chrom "1", :chrom-pos "1_714038"}
{:qual 9.43, :pos "742584", :chrom "1", :chrom-pos "1_742584"}
{:qual 13.06, :pos "798494", :chrom "1", :chrom-pos "1_798494"}

First 3 rows of b:

{:is-present "yes", :id-golden "1_742584"}
{:is-present "yes", :id-golden "1_831535"}
{:is-present "yes", :id-golden "1_835153"}

The chrom-pos column in a matches the id-golden column in b. When I do

(col-names ($join [:chrom-pos :id-golden] a b))

I get the following columns: [:id-golden :is-present :qual :pos :chrom]. However, when I turn this around, I miss the is-present column

(col-names ($join [:id-golden :chrom-pos] b a)) ; => [:chrom-pos :chrom :pos :qual]

There are records that are in a but not in b, but also the other way around. I suppose that the second version of $join should also include the :found column?

jan.

(use '(incanter core datasets))
(def plant-growth (to-matrix (get-dataset :plant-growth)))
(group-on plant-growth 1)

clojure.lang.ArityException: Wrong number of args (2) passed to: core$group-on$filter-fn--1130$fn
at clojure.lang.AFn.throwArity (AFn.java:437)
clojure.lang.AFn.invoke (AFn.java:43)
incanter.core$eval721$fn__723.doInvoke (core.clj:295)
clojure.lang.RestFn.invoke (RestFn.java:439)
clojure.lang.MultiFn.invoke (MultiFn.java:172)
incanter.core$group_on$fn__1149.invoke (core.clj:1121)
clojure.core$map$fn__3811.invoke (core.clj:2432)
clojure.lang.LazySeq.sval (LazySeq.java:42)
clojure.lang.LazySeq.seq (LazySeq.java:60)
clojure.lang.RT.seq (RT.java:466)
clojure.core$seq.invoke (core.clj:133)
clojure.core$print_sequential.invoke (core_print.clj:46)
clojure.core/fn (core_print.clj:140)
clojure.lang.MultiFn.invoke (MultiFn.java:167)
clojure.core$pr_on.invoke (core.clj:3264)
clojure.core$pr.invoke (core.clj:3276)
clojure.lang.AFn.applyToHelper (AFn.java:161)
clojure.lang.RestFn.applyTo (RestFn.java:132)
clojure.core$apply.invoke (core.clj:600)
clojure.core$prn.doInvoke (core.clj:3309)
clojure.lang.RestFn.invoke (RestFn.java:408)

using with-rotation from incanter.processing it doesnt work because the required sketch arguments for push-matrix rotate and pop-matrix aren't present in the macro or passed in from without.
I'm using the latest build (downloaded yesterday) on Solaris 10 java 1.6r17 (ithink)

Sorry for the duplicate but I accidentally closed it and could find the reopen button

incanter.main=>    (use '(incanter core stats datasets charts))
nil
incanter.main=>     (def iris (to-matrix (get-dataset :iris) :dummies true))
#'incanter.main/iris
incanter.main=>     (def y (sel iris :cols 0))
#'incanter.main/y
incanter.main=>     (def x (sel iris :cols (range 1 6)))
#'incanter.main/x
incanter.main=>     (def iris-lm (linear-model y x)) ; with intercept term
ClassCastException clojure.lang.LazySeq cannot be cast to java.lang.Number  clojure.lang.Numbers.lt (Numbers.java:219)

This is copied from (doc linear-model)

Right now, scripts continue to execute even when there is an error during execution of one of the tasks, for example, lein test. We need to fix scripts to return earlier if error happens.

Making a scatter-plot of 300k doubles is extremely slow compared to R or Excel with the same data. Perhaps JFreeChart isn't being used optimally?

A single CPU core is utilized 100% during this time. Also after the plot is generated, re-referencing it is quick enough.

This came up on the IRC channel today:

OSX 10.6.3
lein 1.2.0-RC1

Steps to repro:

1. Create a new lien project using http://pastebin.com/QYLnBF50
2. lein clean && lein deps
3. lein repl
4. user=> (use '(incanter core stats))
   java.lang.RuntimeException: java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Symbol (core.clj:2)
5. modify project.clj to use Clojure beta1: http://pastebin.com/BXVeHWWV
6. lein clean && lein deps && lein repl
7. user=> (use '(incanter core stats))
8. it works!

Please contact me if more info is needed.

According to Wikipedia, factorial(0) is special-cased to be 0. incanter.core/factorial has an incorrect pre-condition, it checks for (pos? k) rather than (not (neg? k)), throwing an exception on (factorial 0).

I've tested that (cern.jet.math.tdouble.DoubleArithmetic/factorial 0) handles it correctly, so only the precondition needs to be modified.