gyselroth / kube-icinga Goto Github PK

View Code? Open in Web Editor NEW

35.0 4.0 4.0 683 KB

Monitor kubernetes services / resources using icinga2 (including autodiscovery support)

License: MIT License

JavaScript 0.41% TypeScript 99.50% Dockerfile 0.09%

kubernetes monitoring icinga2 icinga autodiscovery

kube-icinga's People

Contributors

Stargazers

Watchers

Forkers

arjonbu pavels1 adfinis-forks aland-zhang

kube-icinga's Issues

Enable provisioning on a resource basis

Is your feature request related to a problem? Please describe

Missing the ability to enable/disable provisioning on a single resource directly.

Describe the solution you'd like

Apply a nee annotation:

kube-icinga/provisioning: "false" or "true".

Note that this shall overturn the global provisioning setting for given resource type.

none string environment variables not parsed correctly into config

config values like booleans are objects are not correctly taken from env variables.

LoadBalancers services using node port instead of target port

Describe the bug

Using MetalLB on Raspberry Pi and X86 servers, Loadbalanced services end up with an Icinga service definition that uses a NodePort instead of the targetPort. NodePort is assigned but not used.

To Reproduce

Create a service based on MetalLB such as this:

$ kubectl -n icinga describe service icinga-server
Name:                     icinga-server
Namespace:                icinga
Labels:                   <none>
Annotations:              kube-icinga/host: icinga-sec.thesniderpad.com
                          kubectl.kubernetes.io/last-applied-configuration:
                            {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{"kube-icinga/host":"icinga-sec.thesniderpad.com","metallb.universe.tf/allow...
                          metallb.universe.tf/allow-shared-ip: icinga
Selector:                 app=icinga-server
Type:                     LoadBalancer
IP:                       10.108.186.186
IP:                       10.9.9.206
LoadBalancer Ingress:     10.9.9.206
Port:                     api  5665/TCP
TargetPort:               5665/TCP
NodePort:                 api  31657/TCP
Endpoints:                10.244.6.31:5665
Session Affinity:         None
External Traffic Policy:  Cluster
Events:
  Type    Reason        Age                From                Message
  ----    ------        ----               ----                -------
  Normal  IPAllocated   54m                metallb-controller  Assigned IP "10.9.9.206"
  Normal  nodeAssigned  31m (x7 over 53m)  metallb-speaker     announcing from node "ia01"

In the above example, the container is listening on the TargetPort/Endpoint, not the NodePort as can be validated by running these commands from a container inside the cluster:

Failure example using IP and Nodeport

curl -k -s -u user:pass 'https://10.108.186.186:31657/v1'

< nothing was returned >

Success example using IP and Targetport

root@icinga-web:/# curl -k -s -u user:pass 'https://10.108.186.186:5665/v1'
<html><head><title>Icinga 2</title></head><h1>Hello from Icinga 2 (Version: r2.10.5-1)!</h1><p>You are authenticated as <b>root</b>. Your user has the following permissions:</p> <ul><li>*</li></ul><p>More information about API requests is available in the <a href="https://docs.icinga.com/icinga2/latest" target="_blank">documentation</a>.</p></html>root@icinga-web:/#

Success example using endpoint

curl -k -s -u user:pass 'https://10.244.6.31:5665/v1'
<html><head><title>Icinga 2</title></head><h1>Hello from Icinga 2 (Version: r2.10.5-1)!</h1><p>You are authenticated as <b>root</b>. Your user has the following permissions:</p> <ul><li>*</li></ul><p>More information about API requests is available in the <a href="https://docs.icinga.com/icinga2/latest" target="_blank">documentation</a>.</p></html>root@icinga-web:/#

Expected behavior

Service created in Icinga using IP/Target Port or Endpoint

Environment

kube-icinga version: latest tag on docker, changelog last stamp is 2.0.1
kubernetes version: 1.15.3
icinga2 version: r2.10.5-1

Additional context

none

Add support for authenticated services

Provide credentials for icinga via the kubernetes credential store

Handle authentication denied for resource type

Is behaviour

(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2288)
(node:1) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'kind' of undefined
    at Volume.<anonymous> (/opt/kube-icinga/build/kube/volume.js:132:39)
    at Generator.next (<anonymous>)
    at /opt/kube-icinga/build/kube/volume.js:7:71
    at new Promise (<anonymous>)
    at __awaiter (/opt/kube-icinga/build/kube/volume.js:3:12)
    at module.exports.JSONStream.stream.on (/opt/kube-icinga/build/kube/volume.js:130:47)
    at module.exports.JSONStream.emit (events.js:189:13)
    at addChunk (_stream_readable.js:284:12)
    at readableAddChunk (_stream_readable.js:265:11)
    at module.exports.JSONStream.Readable.push (_stream_readable.js:220:10)

Should behaviour

Check if kube api response is invalid (authentication denied)

Upgrade to typescript 3.x / jest 24.x

Describe the change

Upgrade to latest TypeScript/jest.

Current situation

Currently TypeScript 2.9.x is in use with an older jest version.

Should

Latest TypeScript/jest.

Additional context

Add any other context about the problem here.

Deploy pods to icinga2

Is your feature request related to a problem? Please describe

Missing pods.

Describe the solution you'd like

Include pods as a new resource.

There are reasons to monitor single pods besides services:

There are pods which do not have a service but should be monitored
Cronjobs (Lead to pods in kubernetes, failed pods can be discovered that way)

Note that pod provisioning shall be disabled by default.

Can we use it without "ip" ?

Describe the change

i want to watch my cluster from an Icinga master.

Current situation

My cluster doesn't have public IP and he is not in the same network of my icinga master (master got public ip).

Should

Can we run the "script" and command from my cluster directly, to send informations to my icinga master.

Additional context

I know its complicated to explain but i want know if i can send kube status form my cluster to my icinga master. In this case agent icinga send informations to master icinga but master icinga has nothing to do except display on icingaweb2 because i can't sent check_command to my icinga agent from master.

thanks for your help

UnhandledPromiseRejectionWarning: TypeError: result is not iterable

Hi, I try to use kube-icinga, but pod is crashing at start with

info: start cleanup, removing all kubernetes objects from icinga
(node:1) UnhandledPromiseRejectionWarning: TypeError: result is not iterable
at Icinga.<anonymous> (/opt/kube-icinga/build/icinga.js:201:43)
at Generator.next (<anonymous>)
at /opt/kube-icinga/build/icinga.js:7:71
at new Promise (<anonymous>)
at __awaiter (/opt/kube-icinga/build/icinga.js:3:12)
at icingaClient.getServiceFiltered (/opt/kube-icinga/build/icinga.js:199:119)
at IncomingMessage.<anonymous> (/opt/kube-icinga/node_modules/icinga2-api/index.js:147:24)
at emitNone (events.js:111:20)
at IncomingMessage.emit (events.js:208:7)
at endReadableNT (_stream_readable.js:1064:12)
(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:1) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
(node:1) UnhandledPromiseRejectionWarning: TypeError: result is not iterable
at Icinga.<anonymous> (/opt/kube-icinga/build/icinga.js:209:40)
at Generator.next (<anonymous>)
at /opt/kube-icinga/build/icinga.js:7:71
at new Promise (<anonymous>)
at __awaiter (/opt/kube-icinga/build/icinga.js:3:12)
at icingaClient.getHostFiltered (/opt/kube-icinga/build/icinga.js:207:113)
at IncomingMessage.<anonymous> (/opt/kube-icinga/node_modules/icinga2-api/index.js:110:24)
at emitNone (events.js:111:20)
at IncomingMessage.emit (events.js:208:7)
at endReadableNT (_stream_readable.js:1064:12)
(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)

I'm seeing api call coming to incinga2 api, user/pass is ok.
This inciga2 server is also use to monitor others servers not related to kubernetes

Environment

kube-icinga : gyselroth/kube-icinga:latest
kubernetes version: :1.11
icinga2 version: 2.10.2-1~bpo9+1 (debian stretch)

Add support for service/host definitions in kubernetes annotations

https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/

Discover persistent volumes and create icinga services for those

Use kube annotation kube-icinga/check_command to define a check_command directly on the resource

This can already be done by using portNameAsCommand but an annotation would be a nice addition.

Headless service monitoring

Is your feature request related to a problem? Please describe

Headless services (https://kubernetes.io/docs/concepts/services-networking/service/#headless-services) can't be monitored since they have no cluster ip.

Describe the solution you'd like

Monitor the endpoints of the service directly or monitor via DNS name instead of IP.

Describe alternatives you've considered

Additional context

Change servicegroup definition

Is your feature request related to a problem? Please describe

Currently we can not change the servicegroup definition. This is for instance required
to set a custom zone.

Describe the solution you'd like

New config: kubernetes.namespaces.serviceGroupDefinition

Describe alternatives you've considered

There is not really a workaround if you need to put certain objects into other zones but a servicegroup is not available on all endpoints. Therefore changing the zone to another one is required, for example to a global zone.

Additional context

[2019-03-19 13:19:41 +0000] critical/ApiListener: Error: Validation failed for object 'kubernetes-clusterip-services!kube-system-kubernetes-dashboard-tcp:443' of type 'Service'; Attribute 'groups': Object 'kube-system' of type 'ServiceGroup' does not exist.
Location: in /var/lib/icinga2/api/packages/_api/3f7882be-2789-462c-a90e-be5a11caaafd/conf.d/services/kubernetes-clusterip-services!kube-system-kubernetes-dashboard-tcp%3A443.conf: 7:2-7:27
[2019-03-19 13:19:41 +0000] critical/config: Error: Validation failed for object 'kubernetes-clusterip-services!syslog-elasticsearch-discovery-tcp:9300' of type 'Service'; Attribute 'groups': Object 'syslog' of type 'ServiceGroup' does not exist.
Location: in /var/lib/icinga2/api/packages/_api/3f7882be-2789-462c-a90e-be5a11caaafd/conf.d/services/kubernetes-clusterip-services!syslog-elasticsearch-discovery-tcp%3A9300.conf: 7:2-7:22
/var/lib/icinga2/api/packages/_api/3f7882be-2789-462c-a90e-be5a11caaafd/conf.d/services/kubernetes-clusterip-services!syslog-elasticsearch-discovery-tcp%3A9300.conf(5):  command_endpoint = "kubernetes"
/var/lib/icinga2/api/packages/_api/3f7882be-2789-462c-a90e-be5a11caaafd/conf.d/services/kubernetes-clusterip-services!syslog-elasticsearch-discovery-tcp%3A9300.conf(6):  display_name = "syslog-elasticsearch-discovery-tcp:9300"
/var/lib/icinga2/api/packages/_api/3f7882be-2789-462c-a90e-be5a11caaafd/conf.d/services/kubernetes-clusterip-services!syslog-elasticsearch-discovery-tcp%3A9300.conf(7):  groups = [ "syslog" ]
                                                                                                                                                                          ^^^^^^^^^^^^^^^^^^^^^
/var/lib/icinga2/api/packages/_api/3f7882be-2789-462c-a90e-be5a11caaafd/conf.d/services/kubernetes-clusterip-services!syslog-elasticsearch-discovery-tcp%3A9300.conf(8):  host_name = "kubernetes-clusterip-services"
/var/lib/icinga2/api/packages/_api/3f7882be-2789-462c-a90e-be5a11caaafd/conf.d/services/kubernetes-clusterip-services!syslog-elasticsearch-discovery-tcp%3A9300.conf(9):  vars["_kubernetes"] = true

Do not add NodePort services for each worker

Describe the change

Only create single icinga services for NodePort (Like all other objects.)

Current situation

Currently an icinga service is made for each worker/NodePort service.

Should

Add a nodeport service via the kube cluster.

Additional context

Add any other context about the problem here.

Better documentation on check_command annotations

Is your feature request related to a problem? Please describe

I have created a command that will check Kubernetes objects, it requires vars.name, vars.namespace, vars.type (service, deployment, etc). As far as I can tell, I can create an annotation that creates a check_command override, but I cannot see how to specify what the vars.* should be, can I set those in an annotation?

Describe the solution you'd like

A few examples on how to do this in the documentation would be good. As an example, there is extensive metadata defined in the Kubernetes variable for the service, is there a way to address that in the check_command? Or is there a way to specify it in a definition annotation?

Describe alternatives you've considered

I've considered writing a custom check command for each service to override, but this doesn't feel like it will scale very well

Additional context

Here are the check_command definitions for Icinga and Kubernetes

object CheckCommand "check_kube" {
        import "plugin-check-command"
        command = [ "/etc/icinga2/scripts/check_kube.sh" ]
        arguments = {
          "-t" = "$kube_type$"
          "-c" = "$kube_check$"
          "-n" = "$kube_namespace$"
          "-o" = "$kube_object$"
        }
}

object CheckCommand "check_kube_deployment" {
        import "plugin-check-command"
        command = [ "/etc/icinga2/scripts/check_kube.sh" ]
        arguments = {
          "-t" = "deployment"
          "-c" = "Available"
          "-n" = "$kube_namespace$"
          "-o" = "$kube_object$"
        }
}

ClusterIP service check for UDP fails

Describe the bug

If a ClusterIP service uses UDP, the generated check fails in a default icinga2 setup:

Error: Non-optional macro 'udp_expect' used in argument '-e' is missing.
(0) Executing check for object 'kubernetes-clusterip-services!kube-system-kube-dns-dns'

check_udp requires packet content to send/receive to verify that a port is open.

To Reproduce

Steps to reproduce the behavior:

Expected behavior

No error message :-)
not really sure how to fix in a generic way.

Environment

kube-icinga version: gyselroth/kube-icinga:latest
kubernetes version: v1.14.1
icinga2 version: r2.10.4-1

line 32: did not find expected alphabetic or numeric character

Describe the bug

PS C:\project\kube-icinga> kubectl apply --validate -f .\kube-icinga.yaml
secret/icinga-credentials unchanged
clusterrole.rbac.authorization.k8s.io/kube-icinga configured
clusterrolebinding.rbac.authorization.k8s.io/kubernetes-icinga created
serviceaccount/kube-icinga unchanged
error: error parsing .\kube-icinga.yaml: error converting YAML to JSON: yaml: line 32: did not find expected alphabetic or numeric character

To Reproduce

git clone repos
kubectl apply -f kube-icinga.yaml

Expected behavior

work

Environment

kubectl ver 1.20
kubernetes server 1.17

Additional context

Add any other context about the problem here.

gyselroth / kube-icinga Goto Github PK

kube-icinga's People

Contributors

Stargazers

Watchers

Forkers

kube-icinga's Issues

Is your feature request related to a problem? Please describe

Describe the solution you'd like

Describe the bug

To Reproduce

Expected behavior

Environment

Additional context

Is behaviour

Should behaviour

Describe the change

Current situation

Should

Additional context

Is your feature request related to a problem? Please describe

Describe the solution you'd like

Describe the change

Current situation

Should

Additional context

Environment

Is your feature request related to a problem? Please describe

Describe the solution you'd like

Describe alternatives you've considered

Additional context

Is your feature request related to a problem? Please describe

Describe the solution you'd like

Describe alternatives you've considered

Additional context

Describe the change

Current situation

Should

Additional context

Is your feature request related to a problem? Please describe

Describe the solution you'd like

Describe alternatives you've considered

Additional context

Describe the bug

To Reproduce

Expected behavior

Environment

Describe the bug

To Reproduce

Expected behavior

Environment

Additional context

Recommend Projects

Recommend Topics

Recommend Org