cronitorio / cronitor-sidekiq Goto Github PK

View Code? Open in Web Editor NEW

5.0 5.0 8.0 111 KB

Wrap Sidekiq jobs with monitoring by Cronitor

Home Page: https://cronitor.io

License: MIT License

Ruby 99.48% Shell 0.52%

cronitor-sidekiq's People

Contributors

Stargazers

Watchers

Forkers

babylist huguesbr podia cloudforecast kevintom dudo mevo-limited

cronitor-sidekiq's Issues

Support for Sidekiq6

Just a note here for adding support to Sidekiq6. Currently this gem is pinned to Sidekiq5.

Add support for Sidekiq 7

When attempting to use this gem with Sidekiq 7, I receive the following error:

Could not find compatible versions

Because sidekiq-cronitor >= 3.5.0 depends on sidekiq ~> 6.0
  and Gemfile depends on sidekiq ~> 7.0.3,
  sidekiq-cronitor >= 3.5.0 is forbidden.
So, because Gemfile depends on sidekiq-cronitor ~> 3.5.0,
  version solving has failed.

`Sidekiq::Periodic:Loop` Options serialization misalignment

Summary

The cronitor-sidekiq gem relies on the options method of Sidekiq::Periodic::Loop objects to return a hash. However, due to inconsistencies in Sidekiq's own handling of @options serialization and deserialization, the method returns a JSON string instead. This causes a NoMethodError in cronitor-sidekiq. See this related issue sidekiq/sidekiq#6092 I filed with Sidekiq.

Description

The core issue arises from the misalignment between how Sidekiq::Periodic::Loop handles @options serialization and how cronitor-sidekiq expects to consume it.

In Sidekiq::Periodic::Loop, @options is serialized to JSON when stored in Redis:

args.append("options", Sidekiq.dump_json(lop.options))

However, it is not deserialized back to a hash when read:

@options = opts.delete("options")

In cronitor-sidekiq, the assumption is made that @options will be a hash:

periodic_job.present? && periodic_job.options.fetch('cronitor_key', nil)

Example error

2023-10-31 19:20:00.112938 I [6:default/processor] Sidekiq -- fail
2023-10-31 19:20:00.113108 W [6:default/processor] Sidekiq -- {"context":"Job raised exception","job":{"retry":true,"queue":"default","class":"TestLoanProConnection","args":[],"cronitor_enabled":true,"jid":"50ce858a9af4a387a7277a26","created_at":1698780000.085093,"enqueued_at":1698780000.085868},"_config":"#<Sidekiq::Config:0x0000ffffb0e65810>"}
2023-10-31 19:20:00.114064 W [6:default/processor] Sidekiq -- NoMethodError: undefined method `fetch' for "{\"cronitor_enabled\":true}":String

      periodic_job.present? && periodic_job.options.fetch('cronitor_key', nil)
                                                   ^^^^^^
2023-10-31 19:20:00.114091 W [6:default/processor] Sidekiq -- /rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-cronitor-3.7.1/lib/sidekiq/cronitor.rb:55:in `periodic_job_key'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-cronitor-3.7.1/lib/sidekiq/cronitor.rb:44:in `job_key'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-cronitor-3.7.1/lib/sidekiq/cronitor.rb:76:in `rescue in ping'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-cronitor-3.7.1/lib/sidekiq/cronitor.rb:67:in `ping'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-cronitor-3.7.1/lib/sidekiq/cronitor.rb:21:in `rescue in call'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-cronitor-3.7.1/lib/sidekiq/cronitor.rb:16:in `call'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-7.0.3/lib/sidekiq/middleware/chain.rb:177:in `block in invoke'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-pro-7.0.10/lib/sidekiq/batch/middleware.rb:58:in `call'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-7.0.3/lib/sidekiq/middleware/chain.rb:177:in `block in invoke'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-ent-7.0.8/lib/sidekiq-ent/limiter/middleware.rb:40:in `call'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-7.0.3/lib/sidekiq/middleware/chain.rb:177:in `block in invoke'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-7.0.3/lib/sidekiq/metrics/tracking.rb:24:in `track'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-7.0.3/lib/sidekiq/metrics/tracking.rb:120:in `call'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-7.0.3/lib/sidekiq/middleware/chain.rb:177:in `block in invoke'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-7.0.3/lib/sidekiq/middleware/chain.rb:180:in `invoke'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-7.0.3/lib/sidekiq/processor.rb:173:in `block in process'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-7.0.3/lib/sidekiq/processor.rb:140:in `block (6 levels) in dispatch'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-7.0.3/lib/sidekiq/job_retry.rb:113:in `local'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-7.0.3/lib/sidekiq/processor.rb:139:in `block (5 levels) in dispatch'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-7.0.3/lib/sidekiq/rails.rb:16:in `block in call'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.4.3/lib/active_support/execution_wrapper.rb:92:in `wrap'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.4.3/lib/active_support/reloader.rb:72:in `block in wrap'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.4.3/lib/active_support/execution_wrapper.rb:92:in `wrap'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.4.3/lib/active_support/reloader.rb:71:in `wrap'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-7.0.3/lib/sidekiq/rails.rb:15:in `call'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-7.0.3/lib/sidekiq/processor.rb:135:in `block (4 levels) in dispatch'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-7.0.3/lib/sidekiq/processor.rb:267:in `stats'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-7.0.3/lib/sidekiq/processor.rb:130:in `block (3 levels) in dispatch'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-7.0.3/lib/sidekiq/job_logger.rb:13:in `call'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-7.0.3/lib/sidekiq/processor.rb:129:in `block (2 levels) in dispatch'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-7.0.3/lib/sidekiq/job_retry.rb:80:in `global'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-7.0.3/lib/sidekiq/processor.rb:128:in `block in dispatch'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-7.0.3/lib/sidekiq/job_logger.rb:39:in `prepare'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-7.0.3/lib/sidekiq/processor.rb:127:in `dispatch'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-7.0.3/lib/sidekiq/processor.rb:172:in `process'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-7.0.3/lib/sidekiq/processor.rb:82:in `process_one'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-7.0.3/lib/sidekiq/processor.rb:72:in `run'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-7.0.3/lib/sidekiq/component.rb:8:in `watchdog'
/rails_terraform_docker/vendor/bundle/ruby/3.1.0/gems/sidekiq-7.0.3/lib/sidekiq/component.rb:17:in `block in safe_thread'

Initializer

require 'sidekiq'
require 'sidekiq/cronitor/periodic_jobs'

sidekiq_config = { url: ENV.fetch('DEDICATED_REDIS_URI', 'redis://localhost:6379/0') }

Sidekiq.configure_server do |config|
  config.server_middleware do |chain|
    chain.add Sidekiq::Cronitor::ServerMiddleware
  end
  LogstopGuard.guard!(config.logger)
  config.redis = sidekiq_config
  config.periodic do |mgr|
    mgr.register('*/5 * * * 1-5', 'TestLoanProConnection', cronitor_enabled: true)
  end
end

Suggested Fix

Option Parsing: To ensure compatibility with both the current and any fixed version of Sidekiq Enterprise, a version-agnostic approach could be to check the type of @options before accessing its keys.

if periodic_job.present?
  options = JSON.parse(periodic_job.options) if periodic_job.options.is_a?(String)
  options ||= periodic_job.options
  options.fetch('cronitor_key', nil)
end

Probably would want to rescue JSON::ParserError

Can't get jobs to show up after deleting

I started using Cronitor this morning, everything was going great!

One issue I'm having is, I wanted to start over after configuring environments, so I deleted some jobs on the Cronitor web dashboard. But now it seems that Cronitor is not picking up my new jobs as they run?

I'm not sure, is there something I have to do to have the jobs detected again?

I tried adding a cronitor_key to each worker but that didn't seem to work either.

Thanks!

Standardizing Cronitor Libraries

Hi @sj26,

August from Cronitor here. Thanks for making this really cool sidekiq wrapper! We'd like to start taking responsibility for providing Cronitor supported open-source client libraries. Our Github account would host the clients and take a more active roll in the development of all clients. We'd like to start with Ruby, Python, Java, PHP, Elixir, Javascript, and Bash - https://cronitor.io/docs/open-source-libraries - as well as open-source our own Go client.

Our goal is to help standardize documentation, nomenclature, feature development, and testing, as well as keeping the APIs current, and, in some cases, adding or consolidating Ping and Monitor APIs.

Ideally, we'd like to transfer ownership of existing libraries for continuity. We'll clearly note original client authors on the README. And any maintainers who would like to continue contributing to the projects will continue to have maintainer access. I use sidekiq regularly, and would love to make this our official sidekiq gem. Would you be interested in helping us do this for with this library?

Add initializer option to opt-in vs opt-out

Hey @aflanagan

Thank you for this one.

We would love to use it, but would rather have it behave like an opt-in rather than opt-out.

Any chance you could add an initializer (I guess at the cronitor gem level) to set default behavior (requiring opt-in vs opt-out)

cronitor-sidekiq/lib/sidekiq/cronitor.rb

Line 35 in e0534c7

disabled = worker.class.sidekiq_options.fetch("cronitor_disabled", nil)

require 'cronitor'
Cronitor.api_key = 'api_key_123'
Cronitor.environment = 'development' #default: 'production'
Cronitor.sidekiq_opt_in = true

class MyJob
  include Sidekiq::Job
  sidekiq_options cronitor_key: 'abc123', cronitor_enabled: true

  def perform
  end
end

Happy to do it if you think it's interesting

🙏

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.