Problems with delayed_job_sampler about newrelic-ruby-agent

[benweint]

Hi @mhaylock!

You're completely right that DelayedJobSampler#setup is dead code. This is one of the older and crustier parts of the agent's code, and it looks like that method is a left-over from a half-finished change to the sampler to record DJ queue length metrics by queue.

We'd like to fix this, but to be frank, it's unlikely to happen in the near future - momentum in the community seems to be shifting away from DelayedJob and towards Sidekiq and Resque, and the wide variety of backends that can be used with DJ multiplied by the numerous (often quite divergent) versions of DJ that are in use makes testing a feature like this quite time-consuming. (Not to mention the fact that anything we did implement in this regard would likely not be useful for your unsupported version of DJ.)

Your best bet for the moment is probably to write a custom sampler. The basic idea here is to subclass NewRelic::Agent::Sampler, and override the poll method (and possibly initialize if you have setup work to do). From your overridden poll method, you'll want to use our custom metric recording APIs to record your metrics. Here's a simple example:

class MySampler < NewRelic::Agent::Sampler
  def initialize(*args)
    # Do any initialization work here
    super
  end

  def poll
    sampled_value = ... # derive your measurement here 
    NewRelic::Agent.record_metric('Custom/my/metric', sampled_value)
  end
end

Your sampler will be automatically instantiated by the agent (by virtue of it inheriting from the NewRelic::Agent::Sampler class), and it's poll method will be called every minute, just before the agent transmits data to New Relic.

Let us know how this goes and if you run into any trouble with your custom sampler, we'll be happy to help. Sorry for the disappointing answer re: the built-in DelayedJob sampler, but I didn't want to give you any false hope about it working for your usage in the future.

[mhaylock]

Thank you for your response @benweint, I’ve gone ahead and tried to implement a custom sampler, but I seem to have a bit of a chicken and egg problem - the samplers are loaded when newrelic_rpm boots, but I can’t define a subclass of NewRelic::Agent::Sampler before newrelic_rpm has loaded.

Is there a supported way to include newrelic_rpm without booting it first, or do you have another suggestion for getting my subclass of NewRelic::Agent::Sampler defined at the right moment?

[mhaylock]

Actually nevermind @benweint, this was an issue with newrelic_rpm version 3.7.2.192, but not with the recently released 3.8.0.218 - due to this change to the way the agent is loaded when the dispatched is for delayed_job.

A solution that did work with 3.7.2.192 was to call NewRelic::Agent.instance.add_harvest_sampler(CustomDelayedJobSampler) after defining CustomDelayedJobSampler.

[mhaylock]

I’ll close now as I have a working solution for our situation.