Coder Social home page Coder Social logo

yabeda-rails's Introduction

Yabeda::Rails

Built-in metrics for out-of-the box Rails applications monitoring.

If your monitoring system already collects Rails metrics (e.g. NewRelic) then most probably you don't need this gem.

Sample Grafana dashboard ID: 11668

Installation

Add this line to your application's Gemfile:

gem 'yabeda-rails'
# Then add monitoring system adapter, e.g.:
# gem 'yabeda-prometheus'

And then execute:

$ bundle

Registering metrics on server process start

Currently, yabeda-rails automatically registers rails metrics when a server is started via rails server, puma -C config/puma.rb or unicorn -c. However, other application servers or launching via rackup aren't supported at the moment.

A possible workaround is to detect server process and manually activate yabeda-rails in an initializer:

# config/initializers/yabeda.rb

if your_app_server_process? # Your logic here
  Yabeda::Rails.install!
end

You always can add support for your app server to lib/yabeda/rails/railtie.rb. Pull Requests are always welcome!

Metrics

  • Total web requests received: rails_requests_total
  • Web request duration: rails_request_duration (in seconds)
  • Views rendering duration: rails_view_runtime (in seconds)
  • DB request duration: rails_db_runtime (in seconds)

Hooks

  • on_controller_action: Allows to collect

    Yabeda::Rails.on_controller_action do |event, labels|
      next unless event.payload[:ext_service_runtime]
      time_in_seconds = event.payload[:ext_service_runtime] / 1000.0
      rails_ext_service_runtime.measure(labels, time_in_seconds)
    end

Custom tags

You can add additional tags to the existing metrics by adding custom payload to your controller.

# This block is optional but some adapters (like Prometheus) requires that all tags should be declared in advance
Yabeda.configure do
  default_tag :importance, nil
end

class ApplicationController < ActionController::Base
  def append_info_to_payload(payload)
    super
    payload[:importance] = extract_importance(params)
  end
end

append_info_to_payload is a method from ActionController::Instrumentation

Development

After checking out the repo, run bin/setup to install dependencies. Then, run rake spec to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the .gem file to rubygems.org.

Releasing

  1. Bump version number in lib/yabeda/rails/version.rb

    In case of pre-releases keep in mind rubygems/rubygems#3086 and check version with command like Gem::Version.new(Yabeda::Rails::VERSION).to_s

  2. Fill CHANGELOG.md with missing changes, add header with version and date.

  3. Make a commit:

    git add lib/yabeda/rails/version.rb CHANGELOG.md
    version=$(ruby -r ./lib/yabeda/rails/version.rb -e "puts Gem::Version.new(Yabeda::Rails::VERSION)")
    git commit --message="${version}: " --edit
  4. Create annotated tag:

    git tag v${version} --annotate --message="${version}: " --edit --sign
  5. Fill version name into subject line and (optionally) some description (list of changes will be taken from changelog and appended automatically)

  6. Push it:

    git push --follow-tags
  7. You're done!

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/yabeda-rb/yabeda-rails.

License

The gem is available as open source under the terms of the MIT License.

yabeda-rails's People

Contributors

daffydowden avatar dks17 avatar envek avatar lautis avatar lewispb avatar liaden avatar raivil avatar vast avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

yabeda-rails's Issues

Rails server not booting with gem enabled

Hi everyone, I'm experiencing this weird issue while trying to boot my Rails app:

Ruby: 2.7.0
Rails: 6.0.2.2


ArgumentError: wrong number of arguments (given 2, expected 1; required keyword: docstring)
  ../.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/prometheus-client-2.0.0/lib/prometheus/client/registry.rb:56:in `gauge'
  ../.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/yabeda-prometheus-0.1.5/lib/yabeda/prometheus/adapter.rb:23:in `register_gauge!'
  ../.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/yabeda-0.5.0/lib/yabeda/base_adapter.rb:9:in `register!'
  ../.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/yabeda-0.5.0/lib/yabeda/dsl/class_methods.rb:71:in `block in register_metric'
  ../.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/yabeda-0.5.0/lib/yabeda/dsl/class_methods.rb:71:in `each_value'
  ../.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/yabeda-0.5.0/lib/yabeda/dsl/class_methods.rb:71:in `register_metric'
  ../.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/yabeda-0.5.0/lib/yabeda/dsl/class_methods.rb:48:in `gauge'
  ../.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/yabeda-puma-plugin-0.3.0/lib/puma/plugin/yabeda.rb:18:in `block in start'
  ../.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/yabeda-0.5.0/lib/yabeda.rb:68:in `class_eval'
  ../.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/yabeda-0.5.0/lib/yabeda.rb:68:in `block in configure!'
  ../.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/yabeda-0.5.0/lib/yabeda.rb:66:in `each'
  ../.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/yabeda-0.5.0/lib/yabeda.rb:66:in `configure!'
  ../.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/yabeda-rails-0.5.0/lib/yabeda/rails/railtie.rb:20:in `block in <class:Railtie>'

This is my current setup:

Gemfile

# Prometheus instrumentation library for Ruby applications
gem "prometheus-client"

# Collect and export basic metrics for Rails applications
gem "yabeda-rails"

# Collects Puma web-server metrics from puma control panel
gem "yabeda-puma-plugin"

# Prometheus integration to export evil metrics
gem "yabeda-prometheus"

Gemfile.lock

prometheus-client (2.0.0)
yabeda (0.5.0)
      concurrent-ruby
      dry-initializer
    yabeda-prometheus (0.1.5)
      yabeda
    yabeda-puma-plugin (0.3.0)
      json
      puma
      yabeda (~> 0.2)
    yabeda-rails (0.5.0)
      rails
      yabeda (~> 0.4)

config.ru

require_relative "config/environment"

# Insert _before_ the main application:
use Yabeda::Prometheus::Exporter

run Rails.application

config/puma.rb

# Allow Yabeda to collect puma metrics
#
activate_control_app
plugin :yabeda

By removing yabeda-rails, with the current setup, it's working as expected and the app boots.

Not working for clustered puma

When running yabeda rails for clustered puma and does not work for streaming metrics, any ideas?
We are running it via puma config bundle exec puma -C config/puma.rb

# ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.7  1.3 1415016 217088 ?      Ssl  13:05   0:09 puma 3.12.0 (tcp://0.0.0.0:3000) [app]
root        22  0.1  1.3 1632116 214932 ?      Sl   13:06   0:01 puma: cluster worker 0: 1 [app]
root        27  0.0  1.1 1565552 189348 ?      Sl   13:06   0:00 puma: cluster worker 1: 1 [app]
root        32  0.4  1.6 2806432 264888 ?      Sl   13:06   0:05 puma: cluster worker 2: 1 [app]
root        37  0.0  1.1 1565552 189680 ?      Sl   13:06   0:00 puma: cluster worker 3: 1 [app]
root       307  0.0  0.0   4296   728 pts/0    Ss   13:15   0:00 sh
root       581  0.0  0.0  36644  2828 pts/0    R+   13:26   0:00 ps aux

Difference between nginx-ingress and yabeda metrics

Hi. I have a problem with metrics value consistency.
There is a backend deployed in k8s. Also there is a nginx-ingress controller with metrics enabled. Let's compare RPS through the backend service.

Here is yabeda graph
image

Here is a ingress-nginx graph
image

Looks like yabeda's metrics are doubled.

Limit the set of controller metrics

Hi!
Is it possible to do that?
I tried smth like that

Yabeda::Rails.on_controller_action do |event, labels|
  next unless Rails.configuration.measured_actions.include? "#{labels[:controller]}##{labels[:action]}"
  ...

and

config.measured_actions = [
      'api/entry#index',
      'api/offers#index'
    ]

but not working, I see all controllers in /metrics

Not working with Rails API mode

Hi! I have rails app with only API mode. I tried setup yabeda-prometheus, yabeda-rails, but It's not worked.
::1 - - [22/May/2020:20:36:01 +08] "GET /metrics HTTP/1.1" 404 17945
I think, Prometheus expects text exposition format, and send request to app as Content-Type: text/html. But my app hasn't route GET /metrics with html-response.
How can I resolve this problem for Rails api app?

Is there a way to serve the rails metrics in same endpoint as yabeda-sidekiq?

Hi, first thanks for your time and work!

From my understanding, the rails metrics are exposed to the same URL as the application host, e.g if the rails app is at localhost:3000, the metrics are available at localhost:3000/metrics.

It is possible to merge the rails metrics to the same endpoint of yabeda-sidekiq (when using the start_metrics_server! method)?
Or for example, serve the rails metrics in another port? (e.g localhost:5000/metrics)

Thanks!

Is there a way to disable specific metrics?

I'm considering turning off certain metrics that I think negligible because our metrics have too high cardinality. Is there any way to do that? If not, I'd love to make a pull request.

Custom on_controller_action: Event missing headers?

Hi,

I've added this custom metric via the hooks:

Yabeda::Rails.on_controller_action do |event, labels|
  next unless event.payload[:headers] && event.payload[:headers]["X-Request-Start"]

  now = DateTime.now
  request_start = DateTime.strptime(event.payload[:headers]["X-Request-Start"].gsub("t=", ""), "%Q")

  queue_time_in_seconds = (now.to_time.to_f - request_start.to_time.to_f)

  request_queue_time.measure(labels, queue_time_in_seconds)
end

However it seems the headers are missing from the event. According to the Rails docs they should be present. Are they filtered out by this gem?

Metrics are not loaded when running a rails application without rails server

The rails metrics are not automatically added when you, instead of using rails server, call the webserver directly (e.g. puma -C config/puma.rb).

Calling the webserver directly is a fairly common practice in production, the puma documentation even promotes it when you need to specify puma-specific configuration flags.

I wonder if there is a better way to check if we're running in a rails server then calling const_defined?(:Server).

Is removing the check, and thus always defining the metrics, an option? In that case there are a couple of unused metrics in the /metrics endpoint, but I don't know if that is acceptable.

Metrics not collected

Hi! I use yabeda-rb + yabeda-rails + yabeda-sidekiq

gem 'yabeda'
gem 'yabeda-rails'
gem 'yabeda-sidekiq'
gem 'yabeda-prometheus'
gem "prometheus-client" 

on /metrics i see all of rails, sidekiq and my custom metrics

# HELP sidekiq_jobs_waiting_count The number of jobs waiting to process in sidekiq.
sidekiq_jobs_waiting_count{queue="default"} 12.0
...
# TYPE rails_requests_total counter
# HELP rails_requests_total A counter of the total number of HTTP requests rails processed.
rails_requests_total{controller="...",action="...",status="200",format="json",method="post"} 2.0
...
# TYPE git_git_tag gauge
# HELP git_git_tag Current git tag of the application.
git_git_tag{tag="..."} 1.0

but when i call

Yabeda.collect!
Prometheus::Client::Push.new(job: JOB_NAME, gateway: PROMETHEUS_URL).add(Yabeda::Prometheus.registry)

it pushes only sidekiq + default metrics, for rails metrics are not collected

...
# TYPE rails_requests_total counter
# HELP rails_requests_total A counter of the total number of HTTP requests rails processed.
# TYPE rails_request_duration_seconds histogram
# HELP rails_request_duration_seconds A histogram of the response latency.
# TYPE rails_view_runtime_seconds histogram
# HELP rails_view_runtime_seconds A histogram of the view rendering time.
# TYPE rails_db_runtime_seconds histogram
# HELP rails_db_runtime_seconds A histogram of the activerecord execution time.
...

in console it looks like

pry(main)> Yabeda.collect!
Cache read: metrics ({:compress=>true, :compress_threshold=>1024, :expires_in=>15 minutes})
Cache generate: metrics ({:compress=>true, :compress_threshold=>1024, :expires_in=>15 minutes})
Cache write: metrics ({:compress=>true, :compress_threshold=>1024, :expires_in=>15 minutes})
=> [#<Proc:0x00007f9d42be01b8 /usr/local/bundle/gems/yabeda-sidekiq-0.12.0/lib/yabeda/sidekiq.rb:67>,
 #<Proc:0x00007f9d42bf68f0 .../config/initializers/yabeda_metrics.rb:11>,
 #<Proc:0x00007f9d42bf5450 .../initializers/yabeda_metrics.rb:23>]

is there way to correctly collect all metrics for pushgateway?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.