carwow / honeykiq

Sidekiq → Honeycomb 🐝

Honeykiq

Sidekiq → Honeycomb 🐝

Send Sidekiq-related events to Honeycomb.

Installation

Add this line to your application's Gemfile:

gem 'honeykiq'

Usage

The library provides three use cases:

• Honeykiq::ClientMiddleware
• Honeykiq::ServerMiddleware
• Honeykiq::PeriodicReporter

Honeykiq::ClientMiddleware

Add Honeykiq to your Sidekiq client middleware chain. It will propagate tracing fields when used in combination with Honeykiq::ServerMiddleware.

This middleware is only configured to work with the Honeycomb beeline gem, not with a custom Libhoney client.

# Configure Honeycomb beeline
Honeycomb.configure do |config|
  config.write_key = ENV.fetch('HONEYCOMB_WRITE_KEY')
  config.dataset = ENV.fetch('HONEYCOMB_DATASET')
end

# Add the middleware to Sidekiq chain
Sidekiq.configure_client do |config|
  config.client_middleware do |chain|
    chain.add Honeykiq::ClientMiddleware
  end
end

# Also add it to the server client chain
Sidekiq.configure_server do |config|
  # Configure the server client, used when a worker enqueues a job itself.
  config.client_middleware do |chain|
    chain.add Honeykiq::ClientMiddleware
  end

  # Configure ServerMiddleware with a tracing mode
  config.server_middleware do |chain|
    # tracing_mode: options are nil (default), :child, :child_trace, :link
    #
    # - nil (default) starts a span and trace when the job executes.
    #
    # - :child starts a span as a child of the enqueuing trace,
    #   requires ClientMiddleware to be configured when enqueuing the job.
    #
    # - :child_trace (experimental) starts a span and a trace
    #   with trace.parent_id set to the enqueuing trace id,
    #   requires ClientMiddleware to be configured when enqueuing the job.
    #
    # - :link (experimental) starts a span and trace
    #   with a link event that points to the enqueuing trace,
    #   requires ClientMiddleware to be configured when enqueuing the job.
    #   https://docs.honeycomb.io/getting-data-in/tracing/send-trace-data/#links
    chain.add Honeykiq::ServerMiddleware, tracing_mode: :child
  end
end

Honeykiq::ServerMiddleware

Add Honeykiq to your Sidekiq server middleware chain. It will send an event to Honeycomb once a job finishes or fails. Have a look at server_middleware.rb to see what kind of information it sends.

# Configure Honeycomb beeline
Honeycomb.configure do |config|
  config.write_key = ENV.fetch('HONEYCOMB_WRITE_KEY')
  config.dataset = ENV.fetch('HONEYCOMB_DATASET')
end

# Add the middleware to Sidekiq chain
Sidekiq.configure_server do |config|
  config.server_middleware do |chain|
    chain.add Honeykiq::ServerMiddleware
  end
end

# Or pass the libhoney client to the middleware
Sidekiq.configure_server do |config|
  config.server_middleware do |chain|
    chain.add Honeykiq::ServerMiddleware,
      libhoney: Libhoney::Client.new(
        write_key: ENV.fetch('HONEYCOMB_WRITE_KEY'),
        dataset: ENV.fetch('HONEYCOMB_DATASET')
      )
  end
end

You can add your own data or functions to the Honeycomb event by subclassing Honeykiq::ServerMiddleware, and overriding the extra_fields method with your own hash. The contents will be serialized into individual items in the event:

class MyServerMiddleware < Honeykiq::ServerMiddleware
  def extra_fields(job) # Sidekiq::Job instance
    {
      my_data: 'evaluated and added to the event after the job has finished/errored',
      my_function: -> { Time.now }
    }
  end
end

Sidekiq.configure_server do |config|
  config.server_middleware do |chain|
    chain.add MyServerMiddleware
  end
end

Note: If you have long running jobs, an event is only sent to Honeycomb when the job finishes. Therefore, it may appear as though no jobs are currently running. Additionally, if the process receives a SIGKILL then no event is sent about that job, and the job may keep retrying without appearing in Honeycomb. The PeriodicReporter provides visibility for these cases.

Honeykiq::PeriodicReporter

The periodic reporter should be scheduled to report every few seconds, depending on your use case. Every time the #report method is called it will send a total of 1 + P + Q events to Honeycomb where P and Q are the number of processes and queues respectively.

It sends three types of events: instance, process, and queue. Have a look at periodic_reporter.rb to see what kind of information we send for each type.

A setup using clockwork to report every 30 seconds would look like this:

require 'honeycomb-beeline'
require 'clockwork'
require 'honeykiq'

Honeycomb.configure do |config|
  config.write_key = ENV.fetch('HONEYCOMB_WRITE_KEY')
  config.dataset = ENV.fetch('HONEYCOMB_DATASET')
end

module Clockwork
  every(30, 'Honeykiq::PeriodicReporter') do
    Honeykiq::PeriodicReporter.new.report
  end
end

Contributing

Pull requests are very welcome!

Please report bugs in a new issue.

Everyone is expected to follow the code of conduct.

License

The gem is available as open source under the terms of the MIT License.

honeykiq's Issues

Prepare for 1.0.0

Documentation

• Project has a LICENSE file with an open source license
• Project has basic documentation (CHANGELOG, README, CONTRIBUTING, CODE_OF_CONDUCT, usage and examples)
• The name is easy to remember, gives some idea of what the project does, and does not conflict with an existing project or infringe on trademarks
• The issue queue is up-to-date, with issues clearly organized and labeled

Code

• Project uses consistent code conventions and clear function/method/variable names
• The code is clearly commented, documenting intentions and edge cases
• There are no sensitive materials in the revision history, issues, or pull requests (for example, passwords or other non-public information)
• Use of SemVer, CHANGELOG standards
• Automated tests

People

• Someone is committed to managing community interactions (responding to issues, reviewing and merging pull requests)
• At least two people have administrative access to the project

uninitialized constant Sidekiq::Job with Sidekiq 6.2.2

sidekiq/sidekiq#4973

https://github.com/mperham/sidekiq/blob/master/Changes.md#622

Result:
NameError
uninitialized constant Sidekiq::Job
honeykiq (1.3.0) lib/honeykiq/server_middleware.rb in call at line 11

https://github.com/carwow/honeykiq/blob/master/lib/honeykiq/server_middleware.rb#L11

Link spans do not respect sampling configuration

It'd be good to have some way to avoid sending a link span if you sample out the job span.

As it stands the link mode is unusable if you have any real volume, because it gets sent even if neither the span for the job nor the enqueuing request is being sent.

Database queries are not linked to their job

In server_middleware.rb a new event is created, but as it's not done via start_span anything else you instrument (eg database queries, AWS service use) is not linked to the event.

Is there a reason to use libhoney instead of honeycomb-beeline? PRs accepted?