flapjack / flapjack-diner

Consumer of the Flapjack API

Flapjack

Flapjack is a flexible monitoring notification routing system that handles:

• Alert routing (determining who should receive alerts based on interest, time of day, scheduled maintenance, etc)
• Alert summarisation (with per-user, per media summary thresholds)
• Your standard operational tasks (setting scheduled maintenance, acknowledgements, etc)

Flapjack will be immediately useful to you if:

• You want to identify failures faster by rolling up your alerts across multiple monitoring systems.
• You monitor infrastructures that have multiple teams responsible for keeping them up.
• Your monitoring infrastructure is multitenant, and each customer has a bespoke alerting strategy.
• You want to dip your toe in the water and try alternative check execution engines like Sensu, Icinga, or cron in parallel to Nagios.

Try it out with the Quickstart Guide

The Quickstart guide will help you get Flapjack up and running in a VM locally using Vagrant and VirtualBox.

The technical low-down

Flapjack provides a scalable method for dealing with events representing changes in system state (OK -> WARNING -> CRITICAL transitions) and alerting appropriate people as necessary.

At its core, Flapjack processes events received from external check execution engines, such as Nagios. Nagios provides a 'perfdata' event output channel, which writes to a named pipe. flapjack-nagios-receiver then reads from this named pipe, converts each line to JSON and adds them to the events queue.

Flapjack sits downstream of check execution engines (like Nagios, Sensu, Icinga, or cron), processing events to determine:

• if a problem has been detected
• who should know about the problem
• how they should be told

Additional check engines can be supported by adding additional receiver processes similar to the nagios receiver.

Installing

NB: v2 packages will be ready soon -- for the moment these instructions will not work

Ubuntu Precise 64 (12.04):

Tell apt to trust the Flapjack package signing key:

gpg --keyserver keys.gnupg.net --recv-keys 803709B6
gpg -a --export 803709B6 | sudo apt-key add -

Add the Flapjack Debian repository to your Apt sources:

echo "deb http://packages.flapjack.io/deb/v2 precise main" | sudo tee /etc/apt/sources.list.d/flapjack.list

Install the latest Flapjack package:

sudo apt-get update
sudo apt-get install flapjack

Alternatively, download the deb and install using sudo dpkg -i <filename>

The Flapjack package is an Omnibus package and as such contains most dependencies under /opt/flapjack, including Redis.

Installing the package will start Redis (non standard port) and Flapjack. You should now be able to access the Flapjack Web UI at:

http://localhost:3080/

And consume the REST API at:

http://localhost:3081/

N.B. The Redis installed by Flapjack runs on a non-standard port (6380), so it doesn't conflict with other Redis instances you may already have installed.

Other OSes:

Currently we only make a package for Ubuntu Precise (amd64). If you feel comfortable getting a ruby environment going on your preferred OS, then you can also just install Flapjack from rubygems.org:

gem install flapjack

Using a tool like rbenv or rvm is recommended to keep your Ruby applications from intefering with one another.

Alternatively, you can add support for your OS of choice to omnibus-flapjack and build a native package. Pull requests welcome, and we'll help you make this happen!

You'll also need Redis >= 2.6.12.

Configuring

Have a look at the default config file and modify things as required. The package installer copies this to /etc/flapjack/flapjack_config.toml if it doesn't already exist.

# edit the config
sudo vi /etc/flapjack/flapjack_config.toml

# reload the config
sudo /etc/init.d/flapjack reload

Running

Ubuntu Precise 64:

After installing the Flapjack package, Redis and Flapjack should be automatically started.

First up, start Redis if it's not already started:

# status:
sudo /etc/init.d/redis-flapjack status

# start:
sudo /etc/init.d/redis-flapjack start

Operating Flapjack:

# status:
sudo /etc/init.d/flapjack status

# reload:
sudo /etc/init.d/flapjack reload

# restart:
sudo /etc/init.d/flapjack restart

# stop:
sudo /etc/init.d/flapjack stop

# start:
sudo /etc/init.d/flapjack start

Usage

Please see the documentation.

Developing Flapjack

Information on developing more Flapjack components or contributing to core Flapjack development can be found in the Developing section of the docs.

Note that the master branch corresponds to Flapjack 2; maintenance builds for Flapjack 1 are built from the maint/1.x branch.

Documentation Submodule

We have the documentation for this project on a github wiki and also referenced as a submodule at /doc in this project. Run the following commands to populate the local doc/ directory:

git submodule init
git submodule update

If you make changes to the documentation locally, here's how to publish them:

• Checkout master within the doc subdir, otherwise you'll be commiting to no branch, a.k.a. no man's land.
• git add, commit and push from inside the doc subdir
• Add, commit and push the doc dir from the root (this updates the pointer in the main git repo to the correct ref in the doc repo, we think...)

RTFM

All of the documentation.

flapjack-diner's Issues

flapjack-diner should return a created resource's unique ID

Right now, Diner returns 'true' if a request was sent successfully, and 'false' if it failed. Shouldn't it return any data that was sent to it from Flapjack instead?

For example, when I create a new Contact, I would expect to get its unique ID back in response so I can begin tying that contact to rules or entities. The JSON API does actually return this data, and Diner writes it to the API log - but there's no easy way to retrieve it in your code without performing a separate GET action, filtering it, and hoping that there are no duplicated names in your Flapjack setup.

Is there a reason that the simple 'true' and 'false' behavior was chosen over this? Should it be changed?

Read timeout needs to be configurable

HTTParty provides a timeout method which sets the connection and read timeout(s). We need to expose this in a method on diner eg

# set http timeout to 5 minutes
Flapjack::Diner.timeout = 300

include cli with with flapjack-diner

Having a cli included with flapjack-diner would be pretty nice. Initially just supporting enabling/disabling maintenance mode would be great to have the framework in place and allow others to add to it as the need arises.

We have been using jwoods flapjack-client which wraps flapjack-diner for this until now. The flapjack-client will need some work to be compatible with the new flapjack-diner and I thought the time may be better spent getting a cli closer to the source to ensure future compatibility.

In our application deploy process we make extensive use of setting/ending maintenance modes from the client. For each potentially affected service the clients check if that service is in a maint mode and then schedules a new one before doing the deploy and at the end of the deploy the maint mode is cleared if the status check are currently OK.

Additionally we have scripts that self check that status of specific checks and then do one thing or another depending on if it's OK or CRITICAL.

Add /metrics/* endpoints from flapjack maint/1.x to diner

Once this PR is merged - flapjack/flapjack#828 - we'll need to add the endpoints to diner.

Notification Rule Time Restrictions Can't be Updated

Input from our app into FlapjackDiner#update_notification_rules

id: "3eb72c3a-76d7-412a-ad0e-29e9a90d2c70"

attributes:

{"tags"=>nil,
 "regex_tags"=>nil,
 "entities"=>["some-entity-id"],
 "regex_entities"=>nil,
 "time_restrictions"=>
  [{"end_time"=>"2014-10-28 00:00:00",
    "rrules"=>[{"validations"=>{"day"=>[1, 2, 5, 6, 0]}, "rule_type"=>"Weekly", "interval"=>1, "week_start"=>0}],
    "exrules"=>[],
    "rtimes"=>[],
    "extimes"=>[],
    "summary"=>"Weekly on Sundays, Mondays, Tuesdays, Fridays, and Saturdays",
    "start_time"=>"2014-10-27 00:00:00"}],
 "unknown_media"=>nil,
 "warning_media"=>["email", "sms"],
 "critical_media"=>["email"],
 "unknown_blackhole"=>false,
 "warning_blackhole"=>false,
 "critical_blackhole"=>false,
 "links"=>{"contacts"=>[""]},
 "entity_tags"=>[]}

Flapjack Diner log

PATCH http://flapjack-server-example.org/notification_rules/3eb72c3a-76d7-412a-ad0e-29e9a90d2c70

[
    {
        : op=>"replace",
        : path=>"/notification_rules/0/tags",
        : value=>nil
    },
    {
        : op=>"replace",
        : path=>"/notification_rules/0/regex_tags",
        : value=>nil
    },
    {
        : op=>"replace",
        : path=>"/notification_rules/0/entities",
        : value=>[
            "some-entity-id"
        ]
    },
    {
        : op=>"replace",
        : path=>"/notification_rules/0/regex_entities",
        : value=>nil
    },
    {
        : op=>"replace",
        : path=>"/notification_rules/0/unknown_media",
        : value=>nil
    },
    {
        : op=>"replace",
        : path=>"/notification_rules/0/warning_media",
        : value=>[
            "email",
            "sms"
        ]
    },
    {
        : op=>"replace",
        : path=>"/notification_rules/0/critical_media",
        : value=>[
            "email"
        ]
    },
    {
        : op=>"replace",
        : path=>"/notification_rules/0/unknown_blackhole",
        : value=>false
    },
    {
        : op=>"replace",
        : path=>"/notification_rules/0/warning_blackhole",
        : value=>false
    },
    {
        : op=>"replace",
        : path=>"/notification_rules/0/critical_blackhole",
        : value=>false
    }
]

It looks like that time_restrictions is skipped in FlapjackDiner

Get a single notification rule

https://github.com/flpjck/flapjack-diner#notification_rule

This method is supposed to take 2 arguments according to the documentation. I've just tried it in the console while debugging and it takes just the id.

FlapjackDiner#status is requesting new bulk format from flapjack

"Flapjack::Diner.checks" apparently does nothing

README.md lists a checks method that will return all checks, or a specific set if IDs are passed to it as arguments. I've tried running it both ways, and all that gets returned is a blank line.

The Flapjack JSON API docs don't list an endpoint for GETting checks - is this another leftover from before the JSON API was added?

README.md omissions and errors

Hi, let me know if I should split these into separate issues.

Flapjack::Diner.create_contacts([{
  :id         => '123',
  :first_name => 'john',
  :last_name  => 'smith',
  :email      => '[email protected]',
  :timezone   => 'UTC',
  :tags       => ['foo', 'bar', 'baz'],
  :media => {
    :email => {
      :address => '[email protected]', 
      :interval => '60',
      :rollup_threshold => '120'
    }
  }
}])

1. If I create a new contact with Diner with the above code, it will actually create both the contact and the email address medium for it. This is very useful, but I had to discover it myself as README.md doesn't even say that ":media" can be set.
2. Several update methods reference ":add_tag" and ":remove_tag" in README.md. So far at least, they don't appear to exist for any of the resource types. Are these just going to be added later?
3. The header for "create_contact_pagerduty_credentials"'s section says "create_contact_media", and the header for "delete_notification_rules" says "delete_contacts."

be consistent about symbols as keys

currently we require keys to be symbols for eg create_contacts (at least for the :id key) but when retrieving data the keys returned are strings

lets symbolize all top level keys in returned hashes (and think about deeper level key symbolization)

spaces in check names in paths are encoded as +

Only the query string should use + (or %20), the paths must use %20, per http://en.wikipedia.org/wiki/URL_encoding (and see the linked RFCs for more details).

Unclear documentation for multiple check_ids on create_scheduled_maintenances_checks

Flapjack::Diner.create_test_notifications_checks(CHECK_ID(S), [TEST_NOTIFICATION, ...])

Is this supposed to be:

Flapjack::Diner.create_test_notifications_checks('foo:http', 'bar:http', [TEST_NOTIFICATION, ...])

or

Flapjack::Diner.create_test_notifications_checks( { 'foo:http', 'bar:http' } , [TEST_NOTIFICATION, ...])

or something else entirely?

The README should be updated to clarify this.

Please update version on Rubygems

The published gem on Rubygems is 1.3.0 from March 19th, which doesn't have important patches added since then, namely d691742.. Can you please bump patch and push a new release?

Proposal for syntax change

What do you think about a syntax like this?

def unscheduled_maintenances(entity, options = {})
  path_options = {:entity => entity, :check => options.delete(:check)}

  perform_get_request('unscheduled_maintenances', path_options, options) do
    validate :path  => :entity, :as => :required
    validate :query => [:start_time, :end_time], :as => [:time, :required]
  end
end

This is only a cosmetic change, and therefore maybe not that important. However, I think it is much more obvious to the developer, what the expected functionality is.

Flapjack::Diner.entities returns a hash rather than an array

According to the docs, Flapjack::Diner.entities should return an array, but instead it returns a hash. So rather than Flapjack::Diner.entities.each I have to do Flapjack::Diner.entities['entities'].each.

Perhaps this is a bug with flapjack rather than flapjack-diner, but I'll just file it here and let you decide :)

Support bulk operation calls

The way the old API operated was useful in scenarios when performing a "sync" operation from another system, such as a CMDB. For example, we have all our contacts, media and notification rules defined in our CMDB, and run a build sync job to create/update/delete those in flapjack.

With the new JSON API, and the corresponding flapjack-diner library, we're no longer able to perform this same type of "sync" operation. We'll need to write code to pull all the existing contacts, media, and rules and make decisions for each as whether we should create, update or delete the contact, media and rules.

This initially sounds like a reasonable amount of work, so I figured I'd get something filed. @jessereynolds mentioned he was working on some code to do this, and I'm looking at doing the same. Preferably, it would be great to have this part of the flapjack-diner package so it can be easily reused/collaborated on.

update_contacts(:id, :timezone) doesn't work

I tried to update a contact's timezone; however it doesn't work:

irb(main):002:0> Flapjack::Diner.contacts("1398")
=> [{:id=>"1398", :first_name=>"J", :last_name=>"S (BP)", :email=>"[email protected]", :timezone=>"UTC", :links=>{...}}]

irb(main):003:0> Flapjack::Diner.update_contacts("1398", :timezone => "Asia/Bangkok")
=> true

irb(main):004:0> Flapjack::Diner.contacts("1398")
=> [{:id=>"1398", :first_name=>"J", :last_name=>"S (BP)", :email=>"[email protected]", :timezone=>"UTC", :links=>{...}}]

Tried this on both our integration and production instances. Also exit and signin to the console again in case it does caching.

Missing links[:checks] on entities

If you get an entity, its links[:checks] array will contain all check ids for that entity

This currently doesn't happen:

pry(main)> entity = Flapjack::Diner.entities_matching(/^foo}/)
=> [{:id=>"123",
:name=>"foo.bar",
:links=>{:contacts=>["142", "182", "352", "852", "46", "40", "4"]}}]

Flapjack::Diner.downtime_report_entities throws a type error

Just going through some of the commands in the README:

pry(main)> Flapjack::Diner.downtime_report_entities
TypeError: no implicit conversion of String into Integer
from /Users/sarah/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/bundler/gems/flapjack-diner-0c144b4f97db/lib/flapjack-diner.rb:391:in `[]'

change 'entity_tags' to 'tags' on notification rules

Refer: flapjack/flapjack#270

Document minimum ruby version in README

Ruby 1.9, see #43 (comment).

Support flapjack's JSONAPI gateway

Once Flapjack's JSONAPI gateway is a little further along, enhance flapjack-diner to support that alongside its current API gateway support.

Run rubocop over the code, improve code style

https://github.com/bbatsov/rubocop

We'll probably choose to disable a few cops and configure some others differently to the defaults. If this is easy to do, a larger effort against Flapjack proper might be worthwhile.

Error going to second page

When I try and go to any page other than the first page on checks, I get the following error. Here is an example url that I would go to. http://flapjack:3080/checks?page=2

2016-09-28 14:10:16 - ArgumentError - '2' must be an Integer greater than 0.:
/usr/local/rvm/gems/ruby-2.2.2/gems/flapjack-diner-2.0.0b1/lib/flapjack-diner/argument_validator.rb:22:in 'validate'
/usr/local/rvm/gems/ruby-2.2.2/gems/flapjack-diner-2.0.0b1/lib/flapjack-diner/request.rb:22:in 'block (2 levels) in initialize'
/usr/local/rvm/gems/ruby-2.2.2/gems/flapjack-diner-2.0.0b1/lib/flapjack-diner/request.rb:22:in 'each_pair'
/usr/local/rvm/gems/ruby-2.2.2/gems/flapjack-diner-2.0.0b1/lib/flapjack-diner/request.rb:22:in 'block in initialize'
/usr/local/rvm/gems/ruby-2.2.2/gems/flapjack-diner-2.0.0b1/lib/flapjack-diner/query.rb:16:in 'instance_eval'
/usr/local/rvm/gems/ruby-2.2.2/gems/flapjack-diner-2.0.0b1/lib/flapjack-diner/query.rb:16:in 'validate_params'
/usr/local/rvm/gems/ruby-2.2.2/gems/flapjack-diner-2.0.0b1/lib/flapjack-diner/request.rb:21:in 'initialize'
/usr/local/rvm/gems/ruby-2.2.2/gems/flapjack-diner-2.0.0b1/lib/flapjack-diner/resources.rb:27:in 'new'
/usr/local/rvm/gems/ruby-2.2.2/gems/flapjack-diner-2.0.0b1/lib/flapjack-diner/resources.rb:27:in 'block (3 levels) in included'
/usr/local/rvm/gems/ruby-2.2.2/gems/flapjack-2.0.0/lib/flapjack/gateways/web.rb:251:in 'block in class:Web'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:1611:in 'call'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:1611:in 'block in compile!'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:975:in '[]'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:975:in 'block (3 levels) in route!'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:994:in 'route_eval'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:975:in 'block (2 levels) in route!'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:1015:in 'block in process_route'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:1013:in 'catch'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:1013:in 'process_route'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:973:in 'block in route!'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:972:in 'each'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:972:in 'route!'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:1085:in 'block in dispatch!'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:1067:in 'block in invoke'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:1067:in 'catch'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:1067:in 'invoke'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:1082:in 'dispatch!'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:907:in 'block in call!'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:1067:in 'block in invoke'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:1067:in 'catch'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:1067:in 'invoke'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:907:in 'call!'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:895:in 'call'
/usr/local/rvm/gems/ruby-2.2.2/gems/rack-1.6.4/lib/rack/session/abstract/id.rb:225:in 'context'
/usr/local/rvm/gems/ruby-2.2.2/gems/rack-1.6.4/lib/rack/session/abstract/id.rb:220:in 'call'
/usr/local/rvm/gems/ruby-2.2.2/gems/rack-1.6.4/lib/rack/commonlogger.rb:33:in 'call'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:219:in 'call'
/usr/local/rvm/gems/ruby-2.2.2/gems/rack-1.6.4/lib/rack/methodoverride.rb:22:in 'call'
/usr/local/rvm/gems/ruby-2.2.2/gems/flapjack-2.0.0/lib/flapjack/gateways/web/middleware/request_timestamp.rb:12:in 'call'
/usr/local/rvm/gems/ruby-2.2.2/gems/rack-protection-1.5.3/lib/rack/protection/xss_header.rb:18:in 'call'
/usr/local/rvm/gems/ruby-2.2.2/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in 'call'
/usr/local/rvm/gems/ruby-2.2.2/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in 'call'
/usr/local/rvm/gems/ruby-2.2.2/gems/rack-protection-1.5.3/lib/rack/protection/json_csrf.rb:18:in 'call'
/usr/local/rvm/gems/ruby-2.2.2/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in 'call'
/usr/local/rvm/gems/ruby-2.2.2/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in 'call'
/usr/local/rvm/gems/ruby-2.2.2/gems/rack-protection-1.5.3/lib/rack/protection/frame_options.rb:31:in 'call'
/usr/local/rvm/gems/ruby-2.2.2/gems/rack-1.6.4/lib/rack/session/abstract/id.rb:225:in 'context'
/usr/local/rvm/gems/ruby-2.2.2/gems/rack-1.6.4/lib/rack/session/abstract/id.rb:220:in 'call'
/usr/local/rvm/gems/ruby-2.2.2/gems/rack-1.6.4/lib/rack/nulllogger.rb:9:in 'call'
/usr/local/rvm/gems/ruby-2.2.2/gems/rack-1.6.4/lib/rack/head.rb:13:in 'call'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:182:in 'call'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:2013:in 'call'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:1487:in 'block in call'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:1787:in 'synchronize'
/usr/local/rvm/gems/ruby-2.2.2/gems/sinatra-1.4.7/lib/sinatra/base.rb:1487:in 'call'
/usr/local/rvm/gems/ruby-2.2.2/gems/puma-3.6.0/lib/puma/server.rb:578:in 'handle_request'
/usr/local/rvm/gems/ruby-2.2.2/gems/puma-3.6.0/lib/puma/server.rb:415:in 'process_client'
/usr/local/rvm/gems/ruby-2.2.2/gems/puma-3.6.0/lib/puma/server.rb:275:in 'block in run'
/usr/local/rvm/gems/ruby-2.2.2/gems/puma-3.6.0/lib/puma/thread_pool.rb:116:in 'call'
/usr/local/rvm/gems/ruby-2.2.2/gems/puma-3.6.0/lib/puma/thread_pool.rb:116:in 'block in spawn_thread'

Running version 2.0 of Flapjack.

Release master as 1.2 ?

Now Flapjack is at 1.2 and there's been 52 commits to diner since the last release (1.0.0), time for a new diner release? '1.2.0' ?

Creating PagerDuty credentials requires user/pass/subdomain

I'm trying to use flapjack-diner to create PagerDuty credentials, but it's failing to submit the request whenever I don't pass a username, password, or subdomain.

None of these is actually required for PagerDuty access - I tested by submitting only a service key to Flapjack using a curl call, and it was able to push alerts.