riemann / riemann Goto Github PK

Folks seem to love talks! Let's put a video up on the main site. Maybe the devopsdays 5 minute talk?

Cores should have a Service interface for things that require lifecycle management. This will allow us to free resource pools when we shut down cores for reloading.

Stop confusing ipv6 jvm users by default.

When running this combination, java complains about an unsupported argument:

$ /usr/bin/riemann config.clj
Unrecognized VM option '+UseCompressedOops'
Could not create the Java virtual machine.

$ java -version
java version "1.6.0_31"
Java(TM) SE Runtime Environment (build 1.6.0_31-b04)
Java HotSpot(TM) Client VM (build 20.6-b01, mixed mode, sharing)

$ dpkg -l|grep riemann
ii  riemann                           0.1.3                                           A network event stream processor. Intended f

$ uname -a
Linux <redacted> 2.6.32-347-ec2 #53-Ubuntu SMP Tue Aug 21 13:45:32 UTC 2012 i686 GNU/Linux

(let [x #"foo"](where %28service x%29 ...)))

will fail because the macro compiles the equality check based on the symbol x, not the value it refers to.

The rollup stream primitive uses periodically-until-expired (by use of part-time-fast), which stops when an expired event occurs. This is sensible for some use cases but creates problems in others, for instance when trying to roll up all expired events per host.

Maybe part-time-fast could be refactored to provide a parameter as to whether periodic/every or periodically-until-expired should be used?

Split is the stream analogy to cond. It takes a list of test/stream pairs. Test expressions are evaluated using the same substitution/matching rules as (where). If a test returns logical true, the corresponding stream is evaluated. A catchall stream can be provided with or without a test.

(split
  (> 75 metric) (with :state "ok" index)
  (> 95 metric) (with :state "warning" index)
  :else (with :state "oh no" index email))

split* is to split as where* is to where: instead of macro-substituted tests, it takes a list of functions to apply to the current event.

(split*
  #(> 75 (:metric %)) (with :state "ok" index)
  #(> 95 (:metric %)) (with :state "warning" index)
  (with state "oh no" index email))

splitp takes a binary predicate, an expression evaluated with where-substitution rules, and a set of clauses comprising a value and a stream. Clauses are evaluated with (pred value expr), and the first to return true determines which stream is called.

(splitp > metric
  75 (with :state "ok" index)
  95 (with :state "warning" index)
  (with state "oh no" index email))

Get rid of the obsolete metric_f: it's actually a bug now. Simplify everything, and add pointers to clojure and streams documentation, and the cookbook.

(riemann.bin/reload)
(count (Thread/getAllStackTraces))

... rises with each reload. My guess is we're leaking a threadpool in riemann.server.

As discussed on IRC, please see attached gist:

https://gist.github.com/4112f72078071ce19b9b

Hi, I was recently trying to create a 99.9th percentile graph in Graphite, but couldn't figure out for the life of me where the data was going. I could see the events being emitted in the stream, and everything looked fine. Finally figured out the offending bit is this:

https://github.com/aphyr/riemann/blob/master/src/riemann/graphite.clj#L21

Seems that it's assuming two digits of precision for percentiles at all times. I'm mulling over some options for dealing with this, without breaking existing deployments.

Normalize all the protobuf parsing functions; right now the code path is split between riemann-client and clojure-protobuf. Should improve performance. We can also introduce defrecords for events and messages.

490 FAIL in (rate-fast) (streams.clj:613)
491 expected: (approx-equal total (reduce + (map :metric (clojure.core/deref output))))
492 actual: (not (approx-equal 10000 9849))

https://github.com/clojure/tools.nrepl

Whoops.

603 FAIL in (rate-fast) (streams.clj:523)
604 expected: (approx-equal total (reduce + (map :metric o)))
605 actual: (not (approx-equal 10000 9880

A timing bug that slipped through the cracks, perhaps.

(where) should take an (else child1 child2 ...) clause, similar to the way (catch) works in Clojure.

481 FAIL in (expires) (core.clj:115)
482 expected: (= (clojure.core/deref res) {:service 1, :host nil, :time 0.011, :state "expired"})
483 actual: (not (= nil {:host nil, :state "expired", :service 1, :time 0.011}))

The TCP server seems to create new threadpools on a slow but regular basis. Not sure why.

https://gist.github.com/0a96223a1b1cc6680375

Takes a list of functions f1, f2, f3, and returns f such that (f event) calls (f1 event) (f2 event) (f3 event).

(sdo index graph) -> (fn [event] (call-rescue event [index graph]))

Hi,

there's a debian packaged offered as a download here:

http://aphyr.com/riemann/riemann_0.1.2.deb

But the naming is not conform debian naming policy, so tools like apt-ftparchive and so on do not understand whether to index it as 'all' or 'i386' etc. If you change the name to this for example, it works just fine and our internal apt repo offers it up for installation:

riemann_0.1.2-1_all.deb 

When starting riemann with oracle java 1.6.27 I get the following message:

Unrecognized VM option '+UseCompressedOops'
Could not create the Java virtual machine.

Removing the '+UseCompressedOops' solves the issue.

$ java -version
java version "1.6.0_27"
Java(TM) SE Runtime Environment (build 1.6.0_27-b07)
Java HotSpot(TM) Client VM (build 20.2-b06, mixed mode, sharing)

$ uname -rvp
3.6.3-1.fc17.i686 #1 SMP Mon Oct 22 16:10:29 UTC 2012 i686

On http://aphyr.github.com/riemann/quickstart.html, copy and pasting the 'Start the server' command returns not found. Changing the name of the bin file to riemann works.

Trying to start riemann 0.1.3 on a vm with fedora 17 gives the following error:

Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

$ java -version
java version "1.7.0_09-icedtea"
OpenJDK Runtime Environment (fedora-2.3.3.fc17.1-i386)
OpenJDK Client VM (build 23.2-b09, mixed mode)

$ uname -rvp
3.6.3-1.fc17.i686 #1 SMP Mon Oct 22 16:10:29 UTC 2012 i686

This is more of a Clojure best-practice question/discussion than an issue...

There are a couple uses of multimethods in streams.clj that I'm not sure are best expressed with multimethods. In my mind, multimethods are useful when the potential values for the dispatch func are unbounded, and unknown at the time of writing the function. The cool thing about mutlimethods is that they are open, but in this case, you're necessarily covering all the possibilities at once. Hope this makes sense...

I think it's something along the lines of:

lein protobuf compile
lein test

We need a section that explains how to accomplish simple tasks. Syntax highlighting is all ready to go; just need a page with an index.