Coder Social home page Coder Social logo

sensu_influxdb_handler's Introduction

Requirements

influxdb ruby gem

Usage

The configuration options are pretty straight forward. Note that the files are called influx not influxdb.

Metrics are inserted into the database with a table per check name. So if you're using the rabbitmq_overview_metrics plugin, you'd have a table in the defined database called rabbitmq_overview_metrics with the following columns:

  • host
  • metric
  • value

Additionally a duration column would be present based on the time it took the check to run (this is gleaned from the sensu event data).

The value for metric is determined based on the value of strip_metric described below. You can query it like so:

an image

Extension not a handler

Note that the first push of this was a handler that could be called via pipe. This is now an actual extension that's more performant since it's actually in the sensu-server runtime. Additionally it's now using batch submission to InfluxDB by writing all the points for a given series at once.

Just drop the file in /etc/sensu/extensions and add it to your metrics configuration (/etc/sensu/conf.d/handlers/metrics.json:

{
  "handlers": {
    "metrics": {
      "type": "set",
      "handlers": [ "debug", "influx"]
    }
  }
}

Handler config (/etc/sensu/conf.d/influx.json)

{
  "influx": {
    "host": "localhost",
    "port": "8086",
    "user": "stats",
    "password": "stats",
    "database": "stats",
    "strip_metric": "somevalue"
  }
}

Host, port, user, password and database are pretty straight forward. strip_metric however might not be. This is used to "clean up" the data sent to influxdb. Normally everything sent to handlers is akin to the graphite/stats style:

something.host.metrictype.foo.bar

or

host.stats.something.foo.bar

Really the pattern is irrelevant. People have different tastes. Adding much of that data to the column name in InfluxDB is rather silly so strip_metric provides you with a chance to add a value that strips off everything up to (and including that value). This allows you to continue sending to graphite or statsd or whatever and still use this handler.

Using the examples above, if you set the strip_metric to host, then the column in InfluxDB would be called metrictype.foo.bar or stats.something.foo.bar. If you set the value to foo then the column would simply be called foo

Note that strip_metric isn't required.

Quickstart (for Chef sensu cookbook users)

sensu_gem "influxdb"

cookbook_file "/etc/sensu/extensions/influx.rb" do
  source "extensions/influx.rb"
  mode 0755
end

sensu_snippet "influx" do
  content(
    :host => node['sensu']['rabbitmq']['host'],
    :port => '8086',
    :user => 'stats',
    :password => 'stats',
    :database => 'stats',
    :strip_metric => node.name
  )
end

sensu_influxdb_handler's People

Contributors

lusis avatar ves avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

sensu_influxdb_handler's Issues

exception

/etc/sensu/extensions/influx.rb:18:in post_init': undefined method[]' for nil:NilClass (NoMethodError)
from /opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-extension-1.1.2/lib/sensu/extension.rb:21:in block in initialize' from /opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/eventmachine-1.0.3/lib/eventmachine.rb:959:incall'
from /opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/eventmachine-1.0.3/lib/eventmachine.rb:959:in block in run_deferred_callbacks' from /opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/eventmachine-1.0.3/lib/eventmachine.rb:956:intimes'
from /opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/eventmachine-1.0.3/lib/eventmachine.rb:956:in run_deferred_callbacks' from /opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/eventmachine-1.0.3/lib/eventmachine.rb:187:inrun_machine'
from /opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/eventmachine-1.0.3/lib/eventmachine.rb:187:in run' from /opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.20.3/lib/sensu/server/process.rb:23:inrun'
from /opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.20.3/bin/sensu-server:10:in <top (required)>' from /opt/sensu/bin/sensu-server:23:inload'
from /opt/sensu/bin/sensu-server:23:in `

'

query optimization

Hi,
I'm using Influxdb v0.8.8 with this handler.

this handler created a Serie with the same plugin name, exemple:
metric_cpu_pcnt
with the columns below:

time sequence_number value host metric
1421090211000 193692360001 1 i.5e2e894b.lb.tnt.portal.core.uat.sa.east.1.test.net lb.tnt-portal.core.uat.sa-east-1.cpu.total.user

I would like be a query to find a CPU consume per host like this:

select value from "metric_cpu_pcnt" where host =~ /^i.5e2e894b/ AND metric =~ /cpu.total.user$/ and time > now() - 15m;

This query is very slow because when the query use regex in the columns the regex needs to be match for EVERY measurement, and measurements for ALL "metrics" needs to be loaded.

Why this extension work this away?

The correct are not this: (serie name = hostname + metric) ?
series name: i.5e2e894b.lb.tnt-portal.core.uat.sa-east-1.cpu.total.user , columns: [time, value]

Because:
when the query use regex on the series names, not the column values, the series names are in memory in influxdb and the lookup and regex mach is quick as it only needs to be done once.

tks
Joao

Error while using the extension

Hi I get the following error after configuration.

{"timestamp":"2014-08-18T13:14:04.221609+0000","level":"error","message":"InfluxDB: Error posting event - ["/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/influxdb-0.1.8/lib/influxdb/client.rb:151:in write_point'\", \"/etc/sensu/extensions/influx.rb:49:inrun'", "/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-extension-1.0.0/lib/sensu/extension.rb:99:in safe_run'\", \"/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.13.1/lib/sensu/server.rb:307:inblock (2 levels) in handle_event'", "/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.13.1/lib/sensu/server.rb:206:in call'\", \"/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.13.1/lib/sensu/server.rb:206:inblock in mutate_event_data'", "/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-extensions-1.0.0/lib/sensu/extensions/mutators/json.rb:16:in run'\", \"/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-extension-1.0.0/lib/sensu/extension.rb:99:insafe_run'", "/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.13.1/lib/sensu/server.rb:227:in mutate_event_data'\", \"/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.13.1/lib/sensu/server.rb:253:inblock in handle_event'", "/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.13.1/lib/sensu/server.rb:238:in each'\", \"/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.13.1/lib/sensu/server.rb:238:inhandle_event'", "/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.13.1/lib/sensu/server.rb:437:in block (4 levels) in process_result'\", \"/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/em-redis-unified-0.5.0/lib/em-redis/redis_protocol.rb:457:incall'", "/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/em-redis-unified-0.5.0/lib/em-redis/redis_protocol.rb:457:in dispatch_response'\", \"/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/em-redis-unified-0.5.0/lib/em-redis/redis_protocol.rb:411:inprocess_cmd'", "/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/em-redis-unified-0.5.0/lib/em-redis/redis_protocol.rb:385:in receive_data'\", \"/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-em-2.4.0/lib/eventmachine.rb:187:inrun_machine'", "/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-em-2.4.0/lib/eventmachine.rb:187:in run'\", \"/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.13.1/lib/sensu/server.rb:13:inrun'", "/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.13.1/bin/sensu-server:10:in <top (required)>'\", \"/opt/sensu/bin/sensu-server:23:inload'", "/opt/sensu/bin/sensu-server:23:in `

'"]"}
{"timestamp":"2014-08-18T13:14:04.221678+0000","level":"info","message":"handler extension output","extension":{"type":"extension","name":"influx"},"output":"InfluxDB: Handler finished"}

What am I doing wrong?

I have this extension installed on my rhel 6 server sensu 0.12 with rvm 2.1, and 1.9.3 and the influxdb, timeout and the rubygems gems installed and when I restart the sensu-server I get the following in the logs, what am I doing wrong?

{"timestamp":"2014-05-22T00:34:00.654791-0400","level":"error","message":"failed to require extension","extension_file":"/etc/sensu/extensions/handlers/influx.rb","error":{"path":"influxdb","~mesg":"cannot load such file -- influxdb","~bt":["/opt/sensu/embedded/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:53:in require'","/opt/sensu/embedded/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:53:inrequire'","/etc/sensu/extensions/handlers/influx.rb:3:in <top (required)>'","/opt/sensu/embedded/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:53:inrequire'","/opt/sensu/embedded/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:53:in require'","/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.12.6/lib/sensu/extensions.rb:31:inblock in require_directory'","/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.12.6/lib/sensu/extensions.rb:29:in each'","/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.12.6/lib/sensu/extensions.rb:29:inrequire_directory'","/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.12.6/lib/sensu/base.rb:59:in extensions'","/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.12.6/lib/sensu/server.rb:24:ininitialize'","/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.12.6/lib/sensu/server.rb:13:in new'","/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.12.6/lib/sensu/server.rb:13:inrun'","/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.12.6/bin/sensu-server:10:in <top (required)>'","/opt/sensu/bin/sensu-server:23:inload'","/opt/sensu/bin/sensu-server:23:in `

'"]}}
{"timestamp":"2014-05-22T00:34:00.655158-0400","level":"warn","message":"ignoring extension","extension_file":"/etc/sensu/extensions/handlers/influx.rb"}

"cannot load such file -- influxdb"

I have done a "gem install influxdb", and i setup my /etc/sensu/handlers/metrics.json to look like the example. Is there something else I'm missing in this error or in the requirements?

From sensu-server log:

{"timestamp":"2014-06-24T08:30:31.857970-0700","level":"error","message":"failed to require extension","extension_file":"/etc/sensu/extensions/influx.rb","error":{"path":"influxdb","~mesg":"cannot load such file -- influxdb","~bt":["/opt/sensu/embedded/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:53:in require'","/opt/sensu/embedded/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:53:inrequire'","/etc/sensu/extensions/influx.rb:4:in <top (required)>'","/opt/sensu/embedded/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:53:inrequire'","/opt/sensu/embedded/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:53:in require'","/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.12.6/lib/sensu/extensions.rb:31:inblock in require_directory'","/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.12.6/lib/sensu/extensions.rb:29:in each'","/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.12.6/lib/sensu/extensions.rb:29:inrequire_directory'","/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.12.6/lib/sensu/base.rb:59:in extensions'","/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.12.6/lib/sensu/server.rb:24:ininitialize'","/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.12.6/lib/sensu/server.rb:13:in new'","/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.12.6/lib/sensu/server.rb:13:inrun'","/opt/sensu/embedded/lib/ruby/gems/2.0.0/gems/sensu-0.12.6/bin/sensu-server:10:in <top (required)>'","/opt/sensu/bin/sensu-server:23:inload'","/opt/sensu/bin/sensu-server:23:in `

'"]}}
{"timestamp":"2014-06-24T08:30:31.863903-0700","level":"warn","message":"ignoring extension","extension_file":"/etc/sensu/extensions/influx.rb"}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.