opendistro-for-elasticsearch / alerting-kibana-plugin

📟 Open Distro Kibana Alerting Plugin

Open Distro for Elasticsearch Alerting Kibana

The Open Distro for Elasticsearch Alerting Kibana plugin lets you manage your Open Distro for Elasticsearch alerting plugin to monitor your data and send notifications when certain criteria are met---all from Kibana.

Highlights

• Create and schedule monitors, which run period queries against data in Elasticsearch.
• Evaluate query results against triggers to see if they meet certain criteria.
• If trigger criteria are met, generate alerts and perform actions (e.g. post a message in a Slack channel).

Documentation

Please see our documentation.

Setup

1. Download Elasticsearch for the version that matches the Kibana version specified in package.json.

2. Download and install the appropriate Open Distro for Elasticsearch Alerting plugin.

3. Download the Kibana source code for the version specified in package.json you want to set up.

   See the Kibana contributing guide for more instructions on setting up your development environment.

4. Change your node version to the version specified in .node-version inside the Kibana root directory.

5. cd into the plugins directory of the Kibana source code directory.

6. Check out this package from version control into the plugins directory.

7. Run yarn kbn bootstrap inside kibana/plugins/alerting-kibana-plugin.

Ultimately, your directory structure should look like this:

.
├── kibana
│   └── plugins
│       └── alerting-kibana-plugin

Build

To build the plugin's distributable zip simply run yarn build.

Example output: ./build/opendistroAlertingKibana-1.12.0.0.zip

Run

• yarn start

  • Starts Kibana and includes this plugin. Kibana will be available on localhost:5601.
  • Please run in the Kibana root directory
  • You must have Elasticsearch running with the Alerting plugin

Test

There are unit/stubbed integration tests and Cypress e2e/integration tests.

• yarn test:jest

  • Runs the plugin unit tests.

• yarn run cypress open

  • Opens the Cypress test runner.

• yarn run cypress run

  • Runs all Cypress tests headlessly in the Electron browser.

To run the Cypress tests, you must have both Elasticsearch and Kibana running with the Alerting plugin.

If you are running Cypress tests with Kibana development server, pass these 2 options to yarn start:

1. --no-base-path: opt out the BasePathProxy.
2. --no-watch: make sure your server is not restarted.

Contributing to Open Distro for Elasticsearch Alerting Kibana

• Refer to CONTRIBUTING.md.
• Since this is a Kibana plugin, it can be useful to review the Kibana contributing guide alongside the documentation around Kibana plugins and plugin development.

License

This code is licensed under the Apache 2.0 License.

Copyright

Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.

alerting-kibana-plugin's Issues

Alerting UI for Creating Monitors down arrow goes negative

To reproduce, create a new Monitor, Select frequency: by interval, then type or use the arrows to add a negative number to the box. I see that this is post-validated, but the down-arrow should cut off at 0.

Create visual monitor, X Axes are illegible for smaller time values

See the image. This is a 5-minute monitor. The axes should adjust to every 10 seconds or something, otherwise the times run into one another

Root mapping definition has unsupported parameters when creating monitor

Hi,
Opendistro version: 1.1.0
Issue:
When trying to create monitor in kibana alert panel, while could not found any data in UI, and check elasticsearch logs, get below errors.
Expected could successfully get data in create monitor page.
Appreciate if any kindly help, thanks.
[2019-09-06T09:00:39,700][ERROR][c.a.o.a.MonitorRunner ] [elasticcluster] Error loading alerts for monitor: _na_ org.elasticsearch.index.mapper.MapperParsingException: Failed to parse mapping [_doc]: Root mapping definition has unsupported parameters: [_doc : {}] at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:394) ~[elasticsearch-7.1.1.jar:7.1.1] at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:323) ~[elasticsearch-7.1.1.jar:7.1.1] at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$IndexCreationTask.execute(MetaDataCreateIndexService.java:476) ~[elasticsearch-7.1.1.jar:7.1.1] at org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:47) ~[elasticsearch-7.1.1.jar:7.1.1] at org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:687) ~[elasticsearch-7.1.1.jar:7.1.1] at org.elasticsearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:310) ~[elasticsearch-7.1.1.jar:7.1.1] at org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:210) ~[elasticsearch-7.1.1.jar:7.1.1] at org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:142) ~[elasticsearch-7.1.1.jar:7.1.1] at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150) ~[elasticsearch-7.1.1.jar:7.1.1] at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188) ~[elasticsearch-7.1.1.jar:7.1.1] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:681) ~[elasticsearch-7.1.1.jar:7.1.1] at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:252) ~[elasticsearch-7.1.1.jar:7.1.1] at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:215) ~[elasticsearch-7.1.1.jar:7.1.1] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:835) ~[?:?] Caused by: org.elasticsearch.index.mapper.MapperParsingException: Root mapping definition has unsupported parameters: [_doc : {}] at org.elasticsearch.index.mapper.DocumentMapperParser.checkNoRemainingFields(DocumentMapperParser.java:152) ~[elasticsearch-7.1.1.jar:7.1.1] at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:140) ~[elasticsearch-7.1.1.jar:7.1.1] at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:92) ~[elasticsearch-7.1.1.jar:7.1.1] at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:392) ~[elasticsearch-7.1.1.jar:7.1.1] ... 15 more

Monitoring tab disappears if there are no monitors

If you open
Alerting tab -> "Monitors" section

For the first time (no monitors have been created yet) the Alerting tab disappears and I can see the following error in my browser console:

manifest.json:1 Manifest: Line: 1, column: 1, Unexpected token.
vendors.bundle.js:70 TypeError: Cannot read property 'length' of undefined
at EuiBasicTable.value (opendistro-alerting.bundle.js:6)
at EuiBasicTable.value (opendistro-alerting.bundle.js:6)
at EuiBasicTable.value (opendistro-alerting.bundle.js:6)

After creating a monitor, I can see the "Monitors" section. Looks like the 'length' has been updated after creating a monitor, and now even if I delete the monitor, the "Monitors" section is visible (as should be in the first place).

Overriding slack channel per trigger

Hi, I'd like to use same Slack destination for all my triggers. The only thing that I want to change in the trigger is the destination slack channel.

Is it possible to accomplish or I need to create Slack destination per slack channel?

Btw, Slack Incoming Webhook API support this...

Support for Kibana 7.2.x

Hi.

Could you provide a release for Kibana 7.2.x ?

Remove hard dependency of lodash from the plugin.

Kibana 6.6.2 used forked version of lodash which is 3.10.x and doesn't have couple new function that we uses.

Evaluate and add only require lodash. utils instead of adding entire lodash.

Custom webhook info

We are getting quite a few questions regarding custom webhook and how to properly create / format these destinations. Some more information on the Kibana UI would be helpful, or maybe in the documentation? Open to discussion

References
https://discuss.opendistrocommunity.dev/t/custom-webhook-params-documentation-for-create-destination-api/538/4

https://discuss.opendistrocommunity.dev/t/custom-webhook-to-tellegram-chanell/976/3

Kibana 6.6 Compatibility

Build and test against Kibana 6.6 to fix any compatibility issue

yarn kbn bootstrap failed

kibana v6.6.2

root@ubuntu-s-1vcpu-1gb-nyc1-01:~/kibana-extra/opendistro-elasticsearch-alerting-kibana# yarn kbn bootstrap
yarn run v1.15.2
$ node ../../kibana/scripts/kbn bootstrap
Running [bootstrap] command from [/root/kibana]:

Found [28] projects:

kibana
├── packages
│   ├── elastic-datemath (@elastic/datemath)
│   ├── eslint-config-kibana (@elastic/eslint-config-kibana)
│   ├── eslint-plugin-kibana-custom (@elastic/eslint-plugin-kibana-custom)
│   ├── kbn-babel-code-parser (@kbn/babel-code-parser)
│   ├── kbn-babel-preset (@kbn/babel-preset)
│   ├── kbn-config-schema (@kbn/config-schema)
│   ├── kbn-dev-utils (@kbn/dev-utils)
│   ├── kbn-es-query (@kbn/es-query)
│   ├── kbn-es (@kbn/es)
│   ├── kbn-eslint-import-resolver-kibana (@kbn/eslint-import-resolver-kibana)
│   ├── kbn-eslint-plugin-license-header (@kbn/eslint-plugin-license-header)
│   ├── kbn-i18n (@kbn/i18n)
│   ├── kbn-interpreter (@kbn/interpreter)
│   ├── kbn-plugin-generator (@kbn/plugin-generator)
│   ├── kbn-plugin-helpers (@kbn/plugin-helpers)
│   ├── kbn-pm (@kbn/pm)
│   ├── kbn-system-loader (@kbn/system-loader)
│   ├── kbn-test-subj-selector (@kbn/test-subj-selector)
│   ├── kbn-test (@kbn/test)
│   └── kbn-ui-framework (@kbn/ui-framework)
├── test/plugin_functional/plugins
│   ├── kbn_tp_custom_visualizations
│   ├── kbn_tp_sample_app_plugin
│   ├── kbn_tp_sample_panel_action
│   └── kbn_tp_visualize_embedding
├── x-pack (x-pack)
│   └── plugins/infra
└── ../kibana-extra/opendistro-elasticsearch-alerting-kibana (opendistro-alerting)

Running installs in topological order:


Installing dependencies in [kibana]:

$ node ./preinstall_check
[1/5] Validating package.json...
[2/5] Resolving packages...
success Already up-to-date.
Skipping workspace project: @elastic/datemath
Skipping workspace project: @elastic/eslint-config-kibana
Skipping workspace project: @elastic/eslint-plugin-kibana-custom
Skipping workspace project: @kbn/babel-code-parser
Skipping workspace project: @kbn/babel-preset
Skipping workspace project: @kbn/config-schema
Skipping workspace project: @kbn/dev-utils
Skipping workspace project: @kbn/es-query
Skipping workspace project: @kbn/es
Skipping workspace project: @kbn/eslint-import-resolver-kibana
Skipping workspace project: @kbn/eslint-plugin-license-header
Skipping workspace project: @kbn/i18n
Skipping workspace project: @kbn/interpreter
Skipping workspace project: @kbn/plugin-generator
Skipping workspace project: @kbn/plugin-helpers
Skipping workspace project: @kbn/pm
Skipping workspace project: @kbn/system-loader
Skipping workspace project: @kbn/test-subj-selector
Skipping workspace project: @kbn/test
Skipping workspace project: @kbn/ui-framework
Skipping workspace project: kbn_tp_custom_visualizations
Skipping workspace project: kbn_tp_sample_app_plugin
Skipping workspace project: kbn_tp_sample_panel_action
Skipping workspace project: kbn_tp_visualize_embedding
Skipping workspace project: x-pack
Skipping workspace project: infra


Installing dependencies in [opendistro-alerting]:

[1/4] Resolving packages...
success Already up-to-date.

Installs completed, linking package executables:

[x-pack] plugin-helpers -> ../packages/kbn-plugin-helpers/bin/plugin-helpers.js
[opendistro-alerting] plugin-helpers -> ../../kibana/packages/kbn-plugin-helpers/bin/plugin-helpers.js

Linking executables completed, running `kbn:bootstrap` scripts

@elastic/datemath: $ yarn build --quiet
@kbn/config-schema: $ yarn build
@kbn/config-schema: $ tsc
@elastic/datemath: $ babel src --out-dir target --copy-files --quiet
@kbn/es-query: $ yarn build
@kbn/dev-utils: $ yarn build --quiet
@kbn/babel-code-parser: $ yarn build --quiet
@kbn/babel-code-parser: $ babel src --out-dir target --quiet
@kbn/dev-utils: $ babel src --out-dir target --quiet
@kbn/es-query: $ babel src --out-dir target
@kbn/es-query: src/es_query/__tests__/_migrate_filter.js -> target/es_query/__tests__/_migrate_filter.js
@kbn/es-query: src/es_query/__tests__/build_es_query.js -> target/es_query/__tests__/build_es_query.js
@kbn/es-query: src/es_query/__tests__/decorate_query.js -> target/es_query/__tests__/decorate_query.js
@kbn/es-query: src/es_query/__tests__/from_filters.js -> target/es_query/__tests__/from_filters.js
@kbn/es-query: src/es_query/__tests__/from_kuery.js -> target/es_query/__tests__/from_kuery.js
@kbn/es-query: src/es_query/__tests__/from_lucene.js -> target/es_query/__tests__/from_lucene.js
@kbn/es-query: src/es_query/__tests__/lucene_string_to_dsl.js -> target/es_query/__tests__/lucene_string_to_dsl.js
@kbn/es-query: src/es_query/build_es_query.js -> target/es_query/build_es_query.js
@kbn/es-query: src/es_query/decorate_query.js -> target/es_query/decorate_query.js
@kbn/es-query: src/es_query/from_filters.js -> target/es_query/from_filters.js
@kbn/es-query: src/es_query/from_kuery.js -> target/es_query/from_kuery.js
@kbn/es-query: src/es_query/from_lucene.js -> target/es_query/from_lucene.js
@kbn/es-query: src/es_query/index.js -> target/es_query/index.js
@kbn/es-query: src/es_query/lucene_string_to_dsl.js -> target/es_query/lucene_string_to_dsl.js
@kbn/es-query: src/es_query/migrate_filter.js -> target/es_query/migrate_filter.js
@kbn/es-query: src/filters/__tests__/phrase.js -> target/filters/__tests__/phrase.js
@kbn/es-query: src/filters/__tests__/query.js -> target/filters/__tests__/query.js
@kbn/es-query: src/filters/__tests__/range.js -> target/filters/__tests__/range.js
@kbn/es-query: src/filters/exists.js -> target/filters/exists.js
@kbn/es-query: src/filters/index.js -> target/filters/index.js
@kbn/es-query: src/filters/phrase.js -> target/filters/phrase.js
@kbn/es-query: src/filters/phrases.js -> target/filters/phrases.js
@kbn/es-query: src/filters/query.js -> target/filters/query.js
@kbn/es-query: src/filters/range.js -> target/filters/range.js
@kbn/es-query: src/index.js -> target/index.js
@kbn/es-query: src/kuery/ast/__tests__/ast.js -> target/kuery/ast/__tests__/ast.js
@kbn/es-query: src/kuery/ast/ast.js -> target/kuery/ast/ast.js
@kbn/es-query: src/kuery/ast/index.js -> target/kuery/ast/index.js
@kbn/es-query: src/kuery/ast/kuery.js -> target/kuery/ast/kuery.js
@kbn/es-query: src/kuery/ast/legacy_kuery.js -> target/kuery/ast/legacy_kuery.js
@kbn/es-query: src/kuery/filter_migration/__tests__/exists.js -> target/kuery/filter_migration/__tests__/exists.js
@kbn/es-query: src/kuery/filter_migration/__tests__/filter_to_kuery.js -> target/kuery/filter_migration/__tests__/filter_to_kuery.js
@kbn/es-query: src/kuery/filter_migration/__tests__/geo_bounding_box.js -> target/kuery/filter_migration/__tests__/geo_bounding_box.js
@kbn/es-query: src/kuery/filter_migration/__tests__/geo_polygon.js -> target/kuery/filter_migration/__tests__/geo_polygon.js
@kbn/es-query: src/kuery/filter_migration/__tests__/phrase.js -> target/kuery/filter_migration/__tests__/phrase.js
@kbn/es-query: src/kuery/filter_migration/__tests__/range.js -> target/kuery/filter_migration/__tests__/range.js
@kbn/es-query: src/kuery/filter_migration/exists.js -> target/kuery/filter_migration/exists.js
@kbn/es-query: src/kuery/filter_migration/filter_to_kuery.js -> target/kuery/filter_migration/filter_to_kuery.js
@kbn/es-query: src/kuery/filter_migration/geo_bounding_box.js -> target/kuery/filter_migration/geo_bounding_box.js
@kbn/es-query: src/kuery/filter_migration/geo_polygon.js -> target/kuery/filter_migration/geo_polygon.js
@kbn/es-query: src/kuery/filter_migration/index.js -> target/kuery/filter_migration/index.js
@kbn/es-query: src/kuery/filter_migration/phrase.js -> target/kuery/filter_migration/phrase.js
@kbn/es-query: src/kuery/filter_migration/range.js -> target/kuery/filter_migration/range.js
@kbn/es-query: src/kuery/functions/__tests__/and.js -> target/kuery/functions/__tests__/and.js
@kbn/es-query: src/kuery/functions/__tests__/exists.js -> target/kuery/functions/__tests__/exists.js
@kbn/es-query: src/kuery/functions/__tests__/geo_bounding_box.js -> target/kuery/functions/__tests__/geo_bounding_box.js
@kbn/es-query: src/kuery/functions/__tests__/geo_polygon.js -> target/kuery/functions/__tests__/geo_polygon.js
@kbn/es-query: src/kuery/functions/__tests__/is.js -> target/kuery/functions/__tests__/is.js
@kbn/es-query: src/kuery/functions/__tests__/not.js -> target/kuery/functions/__tests__/not.js
@kbn/es-query: src/kuery/functions/__tests__/or.js -> target/kuery/functions/__tests__/or.js
@kbn/es-query: src/kuery/functions/__tests__/range.js -> target/kuery/functions/__tests__/range.js
@kbn/es-query: src/kuery/functions/__tests__/utils/get_fields.js -> target/kuery/functions/__tests__/utils/get_fields.js
@kbn/es-query: src/kuery/functions/and.js -> target/kuery/functions/and.js
@kbn/es-query: src/kuery/functions/exists.js -> target/kuery/functions/exists.js
@kbn/es-query: src/kuery/functions/geo_bounding_box.js -> target/kuery/functions/geo_bounding_box.js
@kbn/es-query: src/kuery/functions/geo_polygon.js -> target/kuery/functions/geo_polygon.js
@kbn/es-query: src/kuery/functions/index.js -> target/kuery/functions/index.js
@kbn/es-query: src/kuery/functions/is.js -> target/kuery/functions/is.js
@kbn/es-query: src/kuery/functions/not.js -> target/kuery/functions/not.js
@kbn/es-query: src/kuery/functions/or.js -> target/kuery/functions/or.js
@kbn/es-query: src/kuery/functions/range.js -> target/kuery/functions/range.js
@kbn/es-query: src/kuery/functions/utils/get_fields.js -> target/kuery/functions/utils/get_fields.js
@kbn/es-query: src/kuery/index.js -> target/kuery/index.js
@kbn/es-query: src/kuery/node_types/__tests__/function.js -> target/kuery/node_types/__tests__/function.js
@kbn/es-query: src/kuery/node_types/__tests__/literal.js -> target/kuery/node_types/__tests__/literal.js
@kbn/es-query: src/kuery/node_types/__tests__/named_arg.js -> target/kuery/node_types/__tests__/named_arg.js
@kbn/es-query: src/kuery/node_types/__tests__/wildcard.js -> target/kuery/node_types/__tests__/wildcard.js
@kbn/es-query: src/kuery/node_types/function.js -> target/kuery/node_types/function.js
@kbn/es-query: src/kuery/node_types/index.js -> target/kuery/node_types/index.js
@kbn/es-query: src/kuery/node_types/literal.js -> target/kuery/node_types/literal.js
@kbn/es-query: src/kuery/node_types/named_arg.js -> target/kuery/node_types/named_arg.js
@kbn/es-query: src/kuery/node_types/wildcard.js -> target/kuery/node_types/wildcard.js
@kbn/interpreter: $ node scripts/build --dev
@kbn/i18n: $ node scripts/build --source-maps
@kbn/test: $ yarn build --quiet
@kbn/interpreter:  info Deleting old output
@kbn/i18n:  info Deleting old output
@kbn/interpreter:  info Starting babel and webpack
@kbn/interpreter:  info [babel  ] > babel src --ignore src/plugin,*.test.js --out-dir target --copy-files --source-maps inline --quiet
@kbn/i18n:  info Starting babel and typescript
@kbn/i18n:  info [babel:web ] > babel src --config-file /root/kibana/packages/kbn-i18n/babel.config.js --out-dir /root/kibana/packages/kbn-i18n/target/web --extensions .ts,.js,.tsx --quiet --source-map inline
@kbn/interpreter:  info [webpack] > webpack --config tasks/build/webpack.config.js --env.sourceMaps true
@kbn/i18n:  info [babel:node] > babel src --config-file /root/kibana/packages/kbn-i18n/babel.config.js --out-dir /root/kibana/packages/kbn-i18n/target/node --extensions .ts,.js,.tsx --quiet --source-map inline
@kbn/i18n:  info [tsc       ] > tsc --emitDeclarationOnly --declarationMap true
@kbn/test: $ babel src --out-dir target --quiet
@kbn/i18n:  proc [babel:web ] Browserslist: caniuse-lite is outdated. Please run next command `yarn upgrade caniuse-lite browserslist`
@kbn/i18n:  proc [babel:node] Successfully compiled 18 files with Babel.
@kbn/i18n:  info [babel:node] exited with 0 after a few seconds
@kbn/interpreter:  info [babel  ] exited with 0 after a few seconds
@kbn/i18n:  proc [babel:web ] Successfully compiled 18 files with Babel.
@kbn/i18n:  info [babel:web ] exited with 0 after a few seconds
@kbn/interpreter:  info [webpack] exited with 0 after a few seconds
@kbn/interpreter:  succ Complete
@kbn/i18n:  info [tsc       ] exited with 0 after a few seconds
@kbn/i18n:  succ Complete
x-pack: $ gulp canvas:plugins:build
x-pack: [14:44:48] /root/kibana/x-pack/plugins/canvas/canvas_plugin
x-pack: [14:44:48] Using gulpfile ~/kibana/x-pack/gulpfile.js
x-pack: [14:44:48] Starting 'canvas:plugins:build'...
✖ x-pack: Killed
x-pack: info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
✖ x-pack: error Command failed with exit code 137.

[bootstrap] failed:

Error: Command failed: yarn run kbn:bootstrap
Killed
error Command failed with exit code 137.

$ gulp canvas:plugins:build
[14:44:48] /root/kibana/x-pack/plugins/canvas/canvas_plugin
[14:44:48] Using gulpfile ~/kibana/x-pack/gulpfile.js
[14:44:48] Starting 'canvas:plugins:build'...
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

    at makeError (/root/kibana/packages/kbn-pm/dist/index.js:14111:9)
    at Promise.all.then.arr (/root/kibana/packages/kbn-pm/dist/index.js:14215:16)
    at process._tickCallback (internal/process/next_tick.js:68:7)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Opendistro configuration support

Hi all,
I use elastiflow to monitor network traffic, and I'm stuck in the opendistro configure, I want to use opendistro to trigger alert, for example when the client IP total traffic exceed 1GB in last 15 min, opendistro will trigger an alert via email or webhook. but now I don't know opendistro how to get the index data, and how to configure condition when client traffic sum exceed 1GB, here is my index templet, I need the filed "flow"{"bytes:", "client_hostname":},hope someone can help me, thanks in advance.
{
"_index": "elastiflow-3.4.1-2019.06.13",
"_type": "doc",
"_id": "XjDTTmsBwJmiCTy6dE1w",
"_version": 1,
"_score": null,
"_source": {
"@Version": "3.4.1",
"event": {
"host": "10.10.101.1",
"type": "netflow"
},
"node": {
"ipaddr": "10.10.101.1",
"hostname": "10.10.101.1"
},
"flow": {
"tos": 20,
"src_hostname": "172.217.24.206",
"input_snmp": 24,
"client_autonomous_system": "private",
"ip_version": "IPv4",
"server_hostname": "172.217.24.206",
"bytes": 2917,
"dst_mask_len": 23,
"dst_port": 60813,
"server_autonomous_system": "Google LLC (15169)",
"client_hostname": "10.213.221.184",
"dst_addr": "10.213.221.184",
"src_country_code": "US",
"sampling_interval": 0,
"traffic_locality": "public",
"src_geo_location": {
"lon": -97.822,
"lat": 37.751
},
"output_ifname": "index: 3",
"autonomous_system": "Google LLC (15169)",
"service_port": "443",
"src_port_name": "https (TCP/443)",
"service_name": "https (TCP/443)",
"server_geo_location": "37.751,-97.822",
"country_code": "US",
"server_country": "United States",
"dst_port_name": "TCP/60813",
"dst_hostname": "10.213.221.184",
"ip_protocol": "TCP",
"input_ifname": "index: 24",
"dst_autonomous_system": "private",
"src_addr": "172.217.24.206",
"server_asn": "15169",
"application": "Google Docs/Drive",
"client_addr": "10.213.221.184",
"src_mask_len": 0,
"src_country": "United States",
"server_country_code": "US",
"src_port": 443,
"src_autonomous_system": "Google LLC (15169)",
"packets": 31,
"country": "United States",
"server_addr": "172.217.24.206",
"src_asn": 15169,
"direction": "unspecified",
"output_snmp": 3
},
"@timestamp": "2019-06-13T03:13:28.000Z",
"netflow": {
"first_switched": "2019-06-13T03:12:28.999Z",
"in_bytes": 2917,
"version": 9,
"flowset_id": 289,
"ipv4_dst_prefix": "10.213.220.0",
"flow_seq_num": 4902055,
"last_switched": "2019-06-13T03:13:20.999Z",
"in_pkts": 31
}
},
"fields": {
"netflow.first_switched": [
"2019-06-13T03:12:28.999Z"
],
"@timestamp": [
"2019-06-13T03:13:28.000Z"
],
"netflow.last_switched": [
"2019-06-13T03:13:20.999Z"
]
},
"sort": [
1560395608000
]
}

Compatibility for Elastic Stack Basic

Has anyone had any success getting this plugin to work with the Basic license? I'm not sure what all is involved but I know that generally the plugin cannot create incides with a "." in them. But obviously other plugin maintainers have done it.

Warn before deleting/acknowledging an alert

If click “Delete” button, will delete the trigger directly without any warning. Can we pop up some warning to avoid user's wrong operation.
The warning can contains the trigger to be deleted and the result may caused by the action, such as all alerts triggered by the trigger will become Deleted too.

Same to acknowledge action.

Trigger does not send the alert if trigger condition is set to less than 0 and value of monitoring metric is negative

Hello,

problem: When setting up monitor/trigger alerting on negative values no alert is being sent even though the monitoring value is way below 0 (negative).
expected result: send an alert if a value is below 0.

Details:

I've set up a monitor using visual graph that checks the min() of a value for the past hour.
See: https://i.imgur.com/tKfOg50.png

And a trigger to send an alert on slack if the monitoring value is less than -1. You can see on the chart that the value is way below the threshold (from -30 to -40 on the example chart) and alert should be sent. However, no alert is sent, ever for any time period.
https://i.imgur.com/evbcsq7.png

Alert history is green all the way. https://i.imgur.com/PibHc4n.png
Should be red and triggered.

Testing the alert with "send test message" under configure actions works fine and the test alert is indeed sent so it's not a delivery issue.

Add visual graph to monitor overview page

To the see monitor chart for visual based monitors, users need to hit edit. It would be nice if the chart was available in the monitor overview.

Support for Kibana 7.x

Enable support for kibana 7.0

Can't filter deleted alerts

We can filter alerts on Kibana by these status:
1.All alerts
2.Active
3.Acknowledged
4.Completed
5.Error

If a trigger is deleted, the alerts triggered by it will change to "Deleted" status, can we support filtering by "Deleted" status?

Invalid new values for parameters and headers on Custom Webhook Destination

On create/edit destination screen, when click on "Add parameter" or "Add header" the application add an empty object { } instead of object with empty values { key: "", value "" }.

Because of that the validation of those fields are not working as expected. If you add a new parameter or header and try save, the new fields are not being marked as invalid but will prevent the request to be saved.

Alerting in kibana.

Hello team,
While creating the alerts in kibana using extraction query we get no of hits results. for example:
while creating trigger on the first hit I do the below.
ctx.results[0].hits.hits[1]._source.rabbitmq.queue.messages.ready.count>1000

what if there are multiple hits? How do I denote this with iterative?
EX: ctx.results[0].hits.hits[i>0]._source.rabbitmq.queue.messages.ready.count>1000 so that it sends alert for each and every result hit. Is there any way to do that or do we need to create additional triggers like below for each hit?
ctx.results[0].hits.hits[2]._source.rabbitmq.queue.messages.ready.count>1000
ctx.results[0].hits.hits[3]._source.rabbitmq.queue.messages.ready.count>1000
ctx.results[0].hits.hits[4]._source.rabbitmq.queue.messages.ready.count>1000 and so on...

Add error messaging detail for Alerts in "Error" state

Currently when an Alert is in an error state, it can be unclear what the cause of the error is from the UI. By providing more details on what is causing the error, it will help users root cause and mitigate the issue.

Run the monitor schedule for every second.

Hello Team,
I am trying to run the monitor schedule to run every second. I tried the cron expression but its not working. Is there any way to do so?

Allow non-indexed fields to be referenced in creation of Alerts visually.

I have observed that, if a field is not indexed/searchable, then i cannot choose that field when i try to create alert visually which aggregates over a field. But if i make it indexed/searchable, then i can select that field.

For logs visualization and alerting, we disable source fields, hence we cannot create alerts based on the fields.
It is possible in Watchers, to create alerts using non-indexed fields.

Please make this possible with Alerting also.

Thanks.

Alerting using metricbeat parameters

I am getting this error whwnever I try to select system.memory.actual.used.pct from metricbeat index.

Indexed fields do not show up in OVER option in Create Monitor

I have an Elastic Search cluster in AWS Ireland, version 6.2.3. When attempting to build a HTTP status 500 error monitor in the Alerts section, there is no way to select a field to filter on when defining a monitor using visual graph.

The index pattern specified matches the index pattern in Settings.

All Documents returns the expected number of documents.

See screenshot.

Clarify the monitor creation text for visual editor

Currently the text follows this pattern Match the following condition WHEN sum() OF select a field OVER all documents FOR THE LAST 1 hour(s)

This reads like a trigger but it for a monitor. We can revamp the text for clarity.

URL validation for custom webhooks is incorrect

The validation for custom webhooks located here:
https://github.com/opendistro-for-elasticsearch/alerting-kibana-plugin/blob/master/public/pages/Destinations/components/createDestinations/CustomWebhook/validate.js#L18-L26

Does not allow specifying IPs and must be a valid DNS when in reality it could be different.

Insert information about Trigger in Action message text

When a new Action is created, I can specify a text body with Mustache Syntax getting information from "ctx" variable.

Say I have inserted in my DSL Monitor query an aggregation clause to count events in sub-buckets (according to the value of a specific field). Let's suppose I have simply checked those counters against a fixed threshold to trigger the alert.

How can I show in the text an information about which sub-bucket caused the alert?

PDF generation

Hi, a much-sought after feature in Kibana with X-Pack is the ability to export dashboards and visualizations as PDF.

Does OpenDistro Kibana support this feature? If not, is it currently on the roadmap?

Thanks for your time.

Enhancement - replace raw response(true/false) with more friendly message

Repro:

1. Open Kibana Alerting Plugin,
2. Create/open a query based monitor
3. Navigate to Trigger creation page
4. run default painless script, it return back raw result of true.

Enhancement:
Should this raw result be converted to more readable message, such as "Trigger ran successfully" - as suggested by @ylwu-amzn

Standardize plugin name

As noted in https://github.com/opendistro-for-elasticsearch/security-kibana-plugin/issues/19, the alerting plugin uses a dash, whereas everything else uses an underscore:

./bin/elasticsearch-plugin list
opendistro_alerting
opendistro_performance_analyzer
opendistro_security
opendistro_sql

./bin/kibana-plugin list
[email protected]
[email protected]

Expected behavior would be [email protected] (or a dash if we can get all plugins to switch).

Wrong size parameter for fetching Destinations in creation of Trigger UI.

Hi, while creating Trigger, the list of destinations is fetched using this query
https://kibana.example.com:5601/api/alerting/destinations?_searchsize=200_

The API does not respect the "searchsize parameter", hence, even though i have 30+ destinations, it will fetch only 20, which is default Elasticsearch return size.

The right parameter would be "size", which would fetch the number of given destinations.
https://kibana.eample.in:5601/api/alerting/destinations?_size=200_

I am running Opendistro 0.9.0.0 and ES version 6.7.1

how to include iterative value in ctx.results.hits[i>=0]

Hello team, In the message body I want to make the hits result as iterative.
If suppose the alert triggered has 10 hits. The trigger condition met at the 3rd hit result, But in my message body if i want to get that hostname in the particular result I will have to give it as hits.3 but what if the hit is on 4th? Again I need to configure for the remaining individually or is there any iterative thing we can mention in the body so that it checks and throws particular hit.

Ex:
Trigger condition:
[boolean flag = false;

for (int i = 0; i< ctx.results[0].hits.hits.length; i++) {
if ctx.results[0].hits.hits[i]._source.mongodb.replstatus.members.down.count > 0
flag = true;
}
return flag;]
The above one checks for the condition in every hit irrespective of 'n' no.of hits.

For suppose the error is on 4th hit result
In my message body I given as:

"Member have reached down state in host:{{ctx.results.0.hits.hits.4._source.mongodb.replstatus.members.down.hosts}} and the count is: {{ctx.results.0.hits.hits.4._source.mongodb.replstatus.members.down.count}}"
Then only it gives the exact result.

If suppose the hit result is on 3rd then trigger condition will work as it is iterative but the message body gives the 4th hit result which gives wrong host name.
So to avoid this, Is there any way we can make the message body also iterative??
ex:
"Member have reached down state in host:{{ctx.results.0.hits.hits.i._source.mongodb.replstatus.members.down.hosts}} and the count is: {{ctx.results.0.hits.hits.i._source.mongodb.replstatus.members.down.count}}???

Please provide any update on this.

Check if the aggregation value is null before comparing threshold in painless script

Currently we just do a comparison of the aggregation.when.value and the threshold, but if there were no matching documents, the aggregation value is null and painless will throw an error when doing null > x.

Update documentation links to link to specific sections

Currently some of our help links direct to the Alerting main page rather than specific sections that might be more helpful.

Document the fields available in ctx for trigger actions

In the documentation:
https://opendistro.github.io/for-elasticsearch-docs/docs/alerting/monitors/#add-actions

How do I know which fields I have available in ctx? I gather I have some or all of the monitor, trigger, and period. What other fields are there?

The UI includes an "info" button which is not helpful:

Error while updating a monitor

While updating a monitor and trying to create a different alert rule in the graphical editor I received the following error:

I am running an ElasticSearch Server cluster in US Oregon and software version R20190221.

Disable the Query parameter box in Custom webhook

While creating Custom webhook, Query Parameter editor is enabled by default in either case of URL selection (i.e fully qualified URL , custom (host /port/path) which creates confusion and customer provide Query parameter in editor instead of URL.

To aligned with Slack / Chime we should consider the endpoint to be fully qualified URL.

This will require to disable Query parameter editor in case user has chosen endpoint and no custom URL.

More details on : https://discuss.opendistrocommunity.dev/t/alerting-webhook-destination/295/

Show blank bar for period before trigger created

Currently, the period before the trigger created shown as “No alerts”. That confuses me a bit. I think it's better to show as blank bar for time before trigger created. It’s impossible to ask user remember when the trigger was created. We can also add the trigger creation time on Triggers table

grouping in kibana does not work

When creating an alert in kibana with "define using visual graph" the field "OVER ALL DOCUMENTS" should support aggregation but it currently does not. It always shows "over all documents" regardless of what settings are configured in other fields.

screenshot: https://i.imgur.com/4Iwf2zy.png

This is a very basic and highly used feature in other alerting tools (x-pack or elastalert) which allows use cases such as grouping on beat.name so one alert can cover multiple hosts. Right now separating alerts by host can only be implemented with "define using extraction query" (removes simplicity) or by implementing one alert for each host (creates a mess).

Using elasticsearch 7.1.1 with kibana 7.1.1 on linux
kibana-alerting 1.1.0.0 and kibana-alerting-elasticsearch 1.1.0.0 built from github then installed into elasticsearch as a plugin.

also reported on opendistro forums: https://discuss.opendistrocommunity.dev/t/grouping-aggregation-does-not-work-when-using-visual-graph/1104

Calendar range test cases break after migration to 6.6.2

Integration with 6.6.2 removed the hard dependency of EUI to rely on Kibana dependency. The current version of EUI consumed by Kibana 6.6.2 enabled default accessibility mode. This has broken existing test cases for .

Incorrect plugin name for uninstalling

While trying to uninstall the plugin from Kibana, it is not possible to uninstall using the display name of the plugin. Instead it is required to remove the version and uninstall for it to be removed.

Kibana version: 6.7.1 (non-OSS)
Alerting: 0.9.0

The plugin should be displayed without it's version while being listed. This would make it consistent with other plugins.

The above error is not observed with the Elasticsearch plugin as it does not display the version along with it's name.

Command logs

bin/kibana-plugin list
[email protected]

bin/kibana-plugin remove [email protected]
Unable to remove plugin because of error: "Plugin [[email protected]] is not installed"

sudo bin/kibana-plugin remove opendistro-alerting
Removing opendistro-alerting...
Plugin removal complete

Add "Where" clause to visual editor for monitors

We should add a where clause to the visual editor when creating a monitor to make it easier for users to be able to add a condition to their chart. For example being able to do WHEN count() OF response OVER all documents FOR THE LAST 5 minutes **where response_code >= 400**

Error when i trying install as plugin

bash-4.2$ bin/kibana-plugin install https://github.com/opendistro-for-elasticsearch/alerting-kibana-plugin/archive/v0.8.0.0.zip
Attempting to transfer from https://github.com/opendistro-for-elasticsearch/alerting-kibana-plugin/archive/v0.8.0.0.zip
Transferring unknown number of bytes
Transfer complete
Retrieving metadata from plugin archive
Plugin installation was unsuccessful due to error "No kibana plugins found in archive"

What i am doing wrong?
I want use original kibana from elasticsearch, and install some opendistro plugins. It's possible?

Monitoring UI does not allow wildcards in index names

Creating an alert, in the Define Monitor portion of the page, you specify the index. When I start typing, I get a drop down with possible choices that are specific indexes. I want to run the alert over an index pattern, like logs-*. I need to have that choice in the menu.

If, I type logs-* into the text box, and escape from the drop down, then the UI errors, asking me to set an index name.

Does this plugin support other languages other than "mustache" and "painless"

Does this plugin support other languages other than "mustache" and "painless" ?

i can't find other in the source code

Implement auto-complete for creating monitor triggers

When creating a monitor trigger and specifying an extraction query, auto-complete would be very helpful. I'm used to the auto-complete in the developer tools tab and miss it here. Possibly link to the Dev Tools tab instead, so I can build my query and then copy-paste.

Support ES 6.7.1

Build / test against Kibana 6.7.1 and fix any compatibility issue.

Release notes -> https://www.elastic.co/guide/en/kibana/current/release-notes-6.7.1.html

Clicking a filtered field in OfExpression when creating a visual graph monitor throws an error

Original issue:
https://discuss.opendistrocommunity.dev/t/define-monitor-using-visual-graph-went-wrong/312

To replicate:
Search in input box and click one of the results

Workaround for now:
Don't use the search filter or use search filter w/ arrow keys and enter

This seems to be happening inside the Eui Combobox component when calling findPopoverPosition. Need to look into why this is happening.

Alerting dashboard MUST auto-update

Once I've set an alert, I must manually reload the alerting dashboard page to see when it goes into the Active state. Either use Kibana's auto-refresh setting, or add a setting to this page that allows me to set a refresh.

Yes, I will get a notification when something goes active. But if I return to the page and don't refresh, I won't see the right thing.

yarn version

Hello,

Which version of yarn should be used to build this project? I built with the latest version 1.15.2 and got an error.

yarn run v1.15.2
$ plugin-helpers build
/bin/sh: 1: plugin-helpers: not found
error Command failed with exit code 127.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Note I have not used yarn before, yarn --version, yarn, and yarn install appear to work.