vmware-archive / kubewatch

Watch k8s events and trigger Handlers

WARNING: Kubewatch is no longer actively maintained by VMware.

VMware has made the difficult decision to stop driving this project and therefore we will no longer actively respond to issues or pull requests. The project will be externally maintained in the following fork: https://github.com/robusta-dev/kubewatch

Thank You.

kubewatch is a Kubernetes watcher that currently publishes notification to available collaboration hubs/notification channels. Run it in your k8s cluster, and you will get event notifications through webhooks.

Usage

$ kubewatch -h

Kubewatch: A watcher for Kubernetes

kubewatch is a Kubernetes watcher that publishes notifications
to Slack/hipchat/mattermost/flock channels. It watches the cluster
for resource changes and notifies them through webhooks.

supported webhooks:
 - slack
 - hipchat
 - mattermost
 - flock
 - webhook
 - smtp

Usage:
  kubewatch [flags]
  kubewatch [command]

Available Commands:
  config      modify kubewatch configuration
  resource    manage resources to be watched
  version     print version

Flags:
  -h, --help   help for kubewatch

Use "kubewatch [command] --help" for more information about a command.

Install

Cluster Installation

Using helm:

When you have helm installed in your cluster, use the following setup:

helm install --name kubewatch bitnami/kubewatch --set='rbac.create=true,slack.channel=#YOUR_CHANNEL,slack.token=xoxb-YOUR_TOKEN,resourcesToWatch.pod=true,resourcesToWatch.daemonset=true'

You may also provide a values file instead:

rbac:
  create: true
resourcesToWatch:
  deployment: false
  replicationcontroller: false
  replicaset: false
  daemonset: false
  services: true
  pod: true
  job: false
  node: false
  clusterrole: true
  serviceaccount: true
  persistentvolume: false
  namespace: false
  secret: false
  configmap: false
  ingress: false
slack:
  channel: '#YOUR_CHANNEL'
  token: 'xoxb-YOUR_TOKEN'

And use that:

$ helm upgrade --install kubewatch bitnami/kubewatch --values=values-file.yml

Using kubectl:

In order to run kubewatch in a Kubernetes cluster quickly, the easiest way is for you to create a ConfigMap to hold kubewatch configuration.

An example is provided at kubewatch-configmap.yaml, do not forget to update your own slack channel and token parameters. Alternatively, you could use secrets.

Create k8s configmap:

$ kubectl create -f kubewatch-configmap.yaml

Create the Pod directly, or create your own deployment:

$ kubectl create -f kubewatch.yaml

A kubewatch container will be created along with kubectl sidecar container in order to reach the API server.

Once the Pod is running, you will start seeing Kubernetes events in your configured Slack channel. Here is a screenshot:

To modify what notifications you get, update the kubewatch ConfigMap and turn on and off (true/false) resources:

resource:
  deployment: false
  replicationcontroller: false
  replicaset: false
  daemonset: false
  services: true
  pod: true
  job: false
  node: false
  clusterrole: false
  serviceaccount: false
  persistentvolume: false
  namespace: false
  secret: false
  configmap: false
  ingress: false

Working with RBAC

Kubernetes Engine clusters running versions 1.6 or higher introduced Role-Based Access Control (RBAC). We can create ServiceAccount for it to work with RBAC.

$ kubectl create -f kubewatch-service-account.yaml

If you do not have permission to create it, you need to become an admin first. For example, in GKE you would run:

$ kubectl create clusterrolebinding cluster-admin-binding --clusterrole=cluster-admin --user=REPLACE_EMAIL_HERE

Edit kubewatch.yaml, and create a new field under spec with serviceAccountName: kubewatch, you can achieve this by running:

$ sed -i '/spec:/a\ \ serviceAccountName: kubewatch' kubewatch.yaml

Then just create pod as usual with:

$ kubectl create -f kubewatch.yaml

Local Installation

Using go package installer:

# Download and install kubewatch
$ go get -u github.com/bitnami-labs/kubewatch

# Configure the notification channel
$ kubewatch config add slack --channel <slack_channel> --token <slack_token>

# Add resources to be watched
$ kubewatch resource add --po --svc
INFO[0000] resource svc configured
INFO[0000] resource po configured

# start kubewatch server
$ kubewatch
INFO[0000] Starting kubewatch controller                 pkg=kubewatch-service
INFO[0000] Starting kubewatch controller                 pkg=kubewatch-pod
INFO[0000] Processing add to service: default/kubernetes  pkg=kubewatch-service
INFO[0000] Processing add to service: kube-system/tiller-deploy  pkg=kubewatch-service
INFO[0000] Processing add to pod: kube-system/tiller-deploy-69ffbf64bc-h8zxm  pkg=kubewatch-pod
INFO[0000] Kubewatch controller synced and ready         pkg=kubewatch-service
INFO[0000] Kubewatch controller synced and ready         pkg=kubewatch-pod

Using Docker:

To Run Kubewatch Container interactively, place the config file in $HOME/.kubewatch.yaml location and use the following command.

docker run --rm -it --network host -v $HOME/.kubewatch.yaml:/root/.kubewatch.yaml -v $HOME/.kube/config:/opt/bitnami/kubewatch/.kube/config --name <container-name> bitnami/kubewatch

Example:

$ docker run --rm -it --network host -v $HOME/.kubewatch.yaml:/root/.kubewatch.yaml -v $HOME/.kube/config:/opt/bitnami/kubewatch/.kube/config --name kubewatch-app bitnami/kubewatch

==> Writing config file...
INFO[0000] Starting kubewatch controller                 pkg=kubewatch-service
INFO[0000] Starting kubewatch controller                 pkg=kubewatch-pod
INFO[0000] Starting kubewatch controller                 pkg=kubewatch-deployment
INFO[0000] Starting kubewatch controller                 pkg=kubewatch-namespace
INFO[0000] Processing add to namespace: kube-node-lease  pkg=kubewatch-namespace
INFO[0000] Processing add to namespace: kube-public      pkg=kubewatch-namespace
INFO[0000] Processing add to namespace: kube-system      pkg=kubewatch-namespace
INFO[0000] Processing add to namespace: default          pkg=kubewatch-namespace
....

To Demonise Kubewatch container use

$ docker run --rm -d --network host -v $HOME/.kubewatch.yaml:/root/.kubewatch.yaml -v $HOME/.kube/config:/opt/bitnami/kubewatch/.kube/config --name kubewatch-app bitnami/kubewatch

Configure

Kubewatch supports config command for configuration. Config file will be saved at $HOME/.kubewatch.yaml

$ kubewatch config -h

config command allows admin setup his own configuration for running kubewatch

Usage:
  kubewatch config [flags]
  kubewatch config [command]

Available Commands:
  add         add webhook config to .kubewatch.yaml
  test        test handler config present in .kubewatch.yaml
  view        view .kubewatch.yaml

Flags:
  -h, --help   help for config

Use "kubewatch config [command] --help" for more information about a command.

Example:

slack:

• Create a slack Bot

• Edit the Bot to customize its name, icon and retrieve the API token (it starts with xoxb-).

• Invite the Bot into your channel by typing: /invite @name_of_your_bot in the Slack message area.

• Add Api token to kubewatch config using the following steps

  $ kubewatch config add slack --channel <slack_channel> --token <slack_token>

  You have an altenative choice to set your SLACK token, channel via environment variables:

  $ export KW_SLACK_TOKEN='XXXXXXXXXXXXXXXX'
  $ export KW_SLACK_CHANNEL='#channel_name'

flock:

• Create a flock bot.

• Add flock webhook url to config using the following command.

  $ kubewatch config add flock --url <flock_webhook_url>

  You have an altenative choice to set your FLOCK URL

  $ export KW_FLOCK_URL='https://api.flock.com/hooks/sendMessage/XXXXXXXX'

Testing Config

To test the handler config by send test messages use the following command.

$ kubewatch config test -h

Tests handler configs present in .kubewatch.yaml by sending test messages

Usage:
  kubewatch config test [flags]

Flags:
  -h, --help   help for test

Example:

$ kubewatch config test

Testing Handler configs from .kubewatch.yaml
2019/06/03 12:29:23 Message successfully sent to channel ABCD at 1559545162.000100

Viewing config

To view the entire config file $HOME/.kubewatch.yaml use the following command.

$ kubewatch config view
Contents of .kubewatch.yaml

handler:
  slack:
    token: xoxb-xxxxx-yyyy-zzz
    channel: kube-watch
  hipchat:
    token: ""
    room: ""
    url: ""
  mattermost:
    channel: ""
    url: ""
    username: ""
  flock:
    url: ""
  webhook:
    url: ""
resource:
  deployment: false
  replicationcontroller: false
  replicaset: false
  daemonset: false
  services: false
  pod: true
  job: false
  node: false
  clusterrole: false
  serviceaccount: false
  persistentvolume: false
  namespace: false
  secret: false
  configmap: false
  ingress: false
namespace: ""

Resources

To manage the resources being watched, use the following command, changes will be saved to $HOME/.kubewatch.yaml.

$ kubewatch resource -h

manage resources to be watched

Usage:
  kubewatch resource [flags]
  kubewatch resource [command]

Available Commands:
  add         adds specific resources to be watched
  remove      remove specific resources being watched

Flags:
      --clusterrole   watch for cluster roles
      --cm            watch for plain configmaps
      --deploy        watch for deployments
      --ds            watch for daemonsets
  -h, --help          help for resource
      --ing           watch for ingresses
      --job           watch for jobs
      --node          watch for Nodes
      --ns            watch for namespaces
      --po            watch for pods
      --pv            watch for persistent volumes
      --rc            watch for replication controllers
      --rs            watch for replicasets
      --sa            watch for service accounts
      --secret        watch for plain secrets
      --svc           watch for services

Use "kubewatch resource [command] --help" for more information about a command.

Add/Remove resource:

$ kubewatch resource add -h

adds specific resources to be watched

Usage:
  kubewatch resource add [flags]

Flags:
  -h, --help   help for add

Global Flags:
      --clusterrole   watch for cluster roles
      --cm            watch for plain configmaps
      --deploy        watch for deployments
      --ds            watch for daemonsets
      --ing           watch for ingresses
      --job           watch for jobs
      --node          watch for Nodes
      --ns            watch for namespaces
      --po            watch for pods
      --pv            watch for persistent volumes
      --rc            watch for replication controllers
      --rs            watch for replicasets
      --sa            watch for service accounts
      --secret        watch for plain secrets
      --svc           watch for services

Example:

# rc, po and svc will be watched
$ kubewatch resource add --rc --po --svc

# rc, po and svc will be stopped from being watched
$ kubewatch resource remove --rc --po --svc

Build

Using go

Clone the repository anywhere:

$ git clone https://github.com/bitnami-labs/kubewatch.git
$ cd kubewatch
$ go build

or

You can also use the Makefile directly:

$ make build

Prerequisites

• You need to have Go (v1.5 or later) installed. Make sure to set $GOPATH

Using Docker

$ make docker-image
$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED              SIZE
kubewatch           latest              919896d3cd90        3 minutes ago       27.9MB

Prerequisites

• you need to have docker installed.

Contribution

Refer to the contribution guidelines to get started.

kubewatch's Issues

Store sensitive information in k8s secrets

k8s offers secrets for holding sensitive information, we should use that feature

read kubernetes api endpoint from kubectl

kubewatch will read k8s endpoint from the current context of kubectl

Enhance/fix: core functionality

Hi ,

We could improve core functionality of Kubewatch by enhancing/fixing the following functionalities

1. Spamming of notifications during start.

   kubewatch bombards the notification channel (say slack) with all the resources that were already present in the cluster. This could be controlled by using CreationTimestamp properties available as part of object's metadata.

   // compare CreationTimestamp of objects with Kubewatch serverStartTime
   // and triggers ObjectCreated() only on latest events 
   func (c *Controller) processItem(key string) error {
     obj, exists, err := c.informer.GetIndexer().GetByKey(key)
     ...
     
     // get object's metedata
     objectMeta := utils.GetObjectMetaData(obj)
     
     ...
     if objectMeta.CreationTimestamp.Sub(serverStartTime).Seconds() > 0 {
       c.eventHandler.ObjectCreated(obj)
     }
   }
   

   Note: The code change for the above functionality is ready and tested locally. If this fix looks good, we could go ahead raise a PR, review it once and make it live.

2. func ObjectCreated() used for Update and Sync Events.

   Currently c.eventHandler.ObjectCreated(obj) method is being used for the following events

   • Create (valid)
   • Update : invalid. we need to use ObjectUpdated() function instead
   • Sync : invalid, this could be ignored.

   This causes the multiple create notifications during a delete event, since internally a delete event is accompanied with an update and a sync event. we could discuss about this issues and come up a with a solution.

   Note: Correct me, if the inference about the delete event is wrong.

Thanks.

Change `Slack API Tester` to `Kubewatch`

correct tests

Add a Dockerfile

Add a simple dockerfile,maybe using the golang:onbuild image.

Render URL of external service

If a external service or ingress has been created render the external HTTP(S) URL in slack

Investigate the controller framework

https://github.com/kubernetes/kubernetes/blob/master/pkg/controller/framework/controller.go

State of the project

Hello.

I've experimented with the bot and used the default kubernetes deployment descriptor which fetched the "0.0.1" tag of the docker image. WIth this version I get pod changes notifications and service change notifications (although I get a log of spurious service changes for some reason). I've also tried the latest code from the repository which has a lot of commented out code and it seems only pod notifications are working.

What is the current state of the project? Is there some refactoring going on?

probably an issue with current vendoring

I tried to make a release but it had problem when building the image:

$ make binary-image
Sending build context to Docker daemon  170.5MB
Step 1/7 : FROM golang:alpine
 ---> 00371bbb49d5
Step 2/7 : MAINTAINER "Cuong Manh Le <[email protected]>"
 ---> Running in 079fb66eb765
 ---> 473d417e13cb
Removing intermediate container 079fb66eb765
Step 3/7 : RUN apk update &&     apk add git build-base &&     rm -rf /var/cache/apk/* &&     mkdir -p "$GOPATH/src/github.com/bitnami-labs/kubewatch"
 ---> Running in 06f757787adc
fetch http://dl-cdn.alpinelinux.org/alpine/v3.4/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.4/community/x86_64/APKINDEX.tar.gz
v3.4.6-270-g88aa8f7 [http://dl-cdn.alpinelinux.org/alpine/v3.4/main]
v3.4.6-160-g14ad2a3 [http://dl-cdn.alpinelinux.org/alpine/v3.4/community]
OK: 5981 distinct packages available
(1/25) Upgrading musl (1.1.14-r14 -> 1.1.14-r16)
(2/25) Installing binutils-libs (2.26-r1)
(3/25) Installing binutils (2.26-r1)
(4/25) Installing gmp (6.1.0-r0)
(5/25) Installing isl (0.14.1-r0)
(6/25) Installing libgomp (5.3.0-r0)
(7/25) Installing libatomic (5.3.0-r0)
(8/25) Installing libgcc (5.3.0-r0)
(9/25) Installing pkgconf (0.9.12-r0)
(10/25) Installing pkgconfig (0.25-r1)
(11/25) Installing mpfr3 (3.1.2-r0)
(12/25) Installing mpc1 (1.0.3-r0)
(13/25) Installing libstdc++ (5.3.0-r0)
(14/25) Installing gcc (5.3.0-r0)
(15/25) Installing make (4.1-r1)
(16/25) Installing musl-dev (1.1.14-r16)
(17/25) Installing libc-dev (0.7-r0)
(18/25) Installing fortify-headers (0.8-r0)
(19/25) Installing g++ (5.3.0-r0)
(20/25) Installing build-base (0.4-r1)
(21/25) Installing libcurl (7.58.0-r0)
(22/25) Installing expat (2.2.0-r1)
(23/25) Installing pcre (8.38-r1)
(24/25) Installing git (2.8.6-r0)
(25/25) Upgrading musl-utils (1.1.14-r14 -> 1.1.14-r16)
Executing busybox-1.24.2-r12.trigger
OK: 173 MiB in 35 packages
 ---> 9d2552ea088b
Removing intermediate container 06f757787adc
Step 4/7 : ADD . "$GOPATH/src/github.com/bitnami-labs/kubewatch"
 ---> 2a79ff9dae97
Step 5/7 : RUN cd "$GOPATH/src/github.com/bitnami-labs/kubewatch" &&     CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a --installsuffix cgo --ldflags="-s" -o /kubewatch
 ---> Running in e105944a79c9
# github.com/bitnami-labs/kubewatch/vendor/k8s.io/apimachinery/pkg/util/net
vendor/k8s.io/apimachinery/pkg/util/net/http.go:279: req.URL.Hostname undefined (type *url.URL has no field or method Hostname)
The command '/bin/sh -c cd "$GOPATH/src/github.com/bitnami-labs/kubewatch" &&     CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a --installsuffix cgo --ldflags="-s" -o /kubewatch' returned a non-zero code: 2
make: *** [builder-image] Error 2

It probably happens because of vendoring. Will investigate more tomorrow morning and try to cut a release.

Filters for events

The bot is now quite chatty and some messages are not well formatted.

We need to break up the filtering and formatting of events from the general handler.

Ideally we could easily add simple configuration that only send notifications for configured events. For example a yaml file with:

[reason]
Created

would only send notifications for events with the reason 'created'.

We could support wildcard:

[reason]
*

Refactor to easily add new handlers

If someone wants to publish in IRC or telegram...etc.

We need a clean way to separate handlers and pass the handler config via command line option

hipchat

Would be awesome to support hipchat

build kubewatch slack app

Need to optimize the Dockerfile.run

I had to use alpine to get the ca-certificates bundle, otherwise post to SLACK might fail due to missing CA.

We need to optimize that file so the image is smaller. I did it quickly and it reached ~500MB.

Add TravisCI

It's great to add a TravisCI for better integration.

check kubewatch running in Kubernetes cluster

Replace imports from skippbox with bitnamilabs

Right now, all the imports inside of kubewatch still refer to the skippbox repo. I'm assuming this isn't breaking anything because there are some redirects happening at this point. But to reduce any confusion (and any future breaks should the redirect dissapear), it's probably smart to update these to point directly to bitnamilabs.

ObjectUpdated not used?

Sorry for a dummy question, but looking at controller.go I only see ObjectDeleted and ObjectCreated invoked, but it seems that ObjectUpdate only exists on handlers, or am I missing something obvious?

Watching new event only

From @Runseb comment #25 (comment)

We should only watch events occur at the time kubewatch start.

bump client-go version

Kubewatch appears to currently be running utilizing version = "v3.0.0-beta.0" . We probably want to bump this to a later (latest?) release to take advantage of new functionality and resource types (CRD, etc.)

Build recommendation library/hooks

With a working kubewatch we can now create a library of rules that trigger specific messages.

For example:

Say a Pod/RC contains an image with a tag latest, then we detect that and automatically send a message "Hey you are using a latest tag, this is bad practice"....

We need to structure the code so that adding specific triggers like these is cleanly architected.

[discuss] Having kubewatch cli to control kubewatch daemon

I found in some cases user has a need to reconfigure filters to get what he want to see while kubewatch is still running. What do you think if we add kubewatch cli (like kubectl) in order to control kubewatch daemon ? Some more actions to be considered to declare in kubewatch cli:

• start/stop/restart kubewatch daemon
• change slack message format
• create/list/delete handles, specify handle

Configurable resource types to be watched

Add Kubernetes replication controller

based on the docker image built, create a Kubernetes RC

Check Openshift observe command

Openshift just release observe, merged with:
https://github.com/openshift/origin/pull/4196/files

We should look at it, it uses the prometheus golang library to watch resources...

Building failed

Building failed with error:
controller/controller.go:131:4: undefined: batch_v1

It seems batch_v1 from import is missed.

More Robust Testing

Currently it looks like travis does the bare minimum in validating that kubewatch builds. What I would like to see is something along these lines:

Build validation for the CLI
Build validation for the docker based image
Output validation from the handlers (slack,mattermost,etc.)
Output validation from the CLI

We can keep making this more robust from there, but I think that's a good start compared to where things stand now.

Set kubewatch to start listening only when it starts

currently kubewatch, gets all the events and pushes that to slack.

We need to have a time filter, which only notifies slack of events from the time kubewatch starts till it is stopped...

Add license headers in all go files

See bottom of LICENSE file.

Add a copyright to Skippbox, Ltd 2016 and put the header in every go file.

support third-party resources

Optimize build to reduce docker image size

Maybe do a static build to avoid having a Docker image that is too big.

Where to get a slack token?

I'd like to run this app in my cluster, but I'm not sure where to fetch a token from. Is there supposed to be a "Kubedash" bot that I should install?

As alternative, perhaps using slack webhooks provides a easier way to integrate?

Improve formatting of SLACK messages

Look at how to better format the messages sent to SLACK.
Maybe add color ...:)

For instance we could color code the the messages based on their type or status of the object.

CrashLoopBackOff when resourcesToWatch.pod is false or not set

When the resourcesToWatch is an empty dictionary, or the pod setting is omitted, kubewatch crashes on startup. While this seems like a minor issue, the default incubator helm chart follows this same pattern!

I've worked around this by adding resourcesToWatch to the improved README (#72), but it seems useful to have a proper solution out of the box.

Kubewatch for Jobs monitoring

Hello!

Does Kubewatch support jobs at the moment? It seems like jobs are not included in 0.0.3 release. But I see jobs in the current state of Master. I manually added "jobs" resource to configamp and clusterrole and of course it did not start working. So my questions are:

1. Is it possible to make Kubewatch working with jobs at the current state of the project?
   If not,
2. Are you going to add jobs support in your next release? And what is the approximate date of the next release?
   Thanks a lot!
   Pavel

architecture document for kubewatch

That would be nice to have a document to help explain kubewatch internals and motivate new members to get involved.

Improve slack messages

Hello there,
I've installed kubewatch on my cluster and enabled slack integration on it, here is sample messages that I receive:

kubewatch
A  in namespace  has been :

kubewatch
A pod in namespace gitlab has been created: runner-e34aa186-project-62-concurrent-0vchfv
kubewat 
A pod in namespace gitlab has been created: runner-e34aa186-project-62-concurrent-0vchfv
kubewatch
A pod in namespace gitlab has been created: runner-e34aa186-project-62-concurrent-0vchfv

There is an incomplete message about namespace and three duplicate messages which I guess they seem duplicate because they don't have enough information.
The other issue is in slack notification I see "message undefined" and "(message missing text)" and leave me no option other and checking channel to see notification content.

objects name in messages

I found the object name is missing in slack messages that makes less sense.

For example if I delete a pod, the pod name is missing:

kubewatch
----------------
In *Namespace* default *Kind* Pod from *Component* kubelet on *Host* ip-172-20-0-179.ap-southeast-1.compute.internal had *Reason* Killing

Instead of using slack token, we can use webhooks?

I guess the slack token used in kubewatch is API tokens which is binding with user account directly.

https://get.slack.help/hc/en-us/articles/215770388-Create-and-regenerate-API-tokens

But I prefer to support incoming webhooks, so I don't worry if someone can represent me to do something bad using my token, if I expose it accidently.

https://api.slack.com/incoming-webhooks

https://my.slack.com/services/new/incoming-webhook/

Services Keep Alerting

Hi,

For some reason every 30 minutes we keep getting alerts on services but no issues have occurred. Is there somewhere to check why? Nothing wrong is showing up in the logs other than successfully posting messages into slack.

We're running K8s 1.54 if it matters.

2017-12-08 16:50:10.306312 I | Message successfully sent to channel G8BBR991D at 1512751810.000104
2017-12-08 16:50:10.469756 I | Message successfully sent to channel G8BBR991D at 1512751810.000194
2017-12-08 16:50:10.585881 I | Message successfully sent to channel G8BBR991D at 1512751810.000261
2017-12-08 16:50:10.697563 I | Message successfully sent to channel G8BBR991D at 1512751810.000325
2017-12-08 16:50:10.804384 I | Message successfully sent to channel G8BBR991D at 1512751810.000406
2017-12-08 16:50:11.022848 I | Message successfully sent to channel G8BBR991D at 1512751810.000550
2017-12-08 16:50:11.126990 I | Message successfully sent to channel G8BBR991D at 1512751810.000625
2017-12-08 16:50:11.239222 I | Message successfully sent to channel G8BBR991D at 1512751810.000687
2017-12-08 16:50:11.367775 I | Message successfully sent to channel G8BBR991D at 1512751810.000761
2017-12-08 16:50:11.483517 I | Message successfully sent to channel G8BBR991D at 1512751811.000077
2017-12-08 16:50:11.604525 I | Message successfully sent to channel G8BBR991D at 1512751811.000159
2017-12-08 16:53:59.056162 I | Message successfully sent to channel G8BBR991D at 1512752038.000661
2017-12-08 16:58:05.028826 I | Message successfully sent to channel G8BBR991D at 1512752284.000578
2017-12-08 17:02:12.410720 I | Message successfully sent to channel G8BBR991D at 1512752531.000804
2017-12-08 17:04:48.564726 I | Message successfully sent to channel G8BBR991D at 1512752688.000195
2017-12-08 17:20:10.395830 I | Message successfully sent to channel G8BBR991D at 1512753610.000134
2017-12-08 17:20:10.505617 I | Message successfully sent to channel G8BBR991D at 1512753610.000191
2017-12-08 17:20:10.597144 I | Message successfully sent to channel G8BBR991D at 1512753610.000241
2017-12-08 17:20:10.686559 I | Message successfully sent to channel G8BBR991D at 1512753610.000293
2017-12-08 17:20:10.765702 I | Message successfully sent to channel G8BBR991D at 1512753610.000354
2017-12-08 17:20:10.905213 I | Message successfully sent to channel G8BBR991D at 1512753610.000408
2017-12-08 17:20:11.043680 I | Message successfully sent to channel G8BBR991D at 1512753610.000492
2017-12-08 17:20:11.139338 I | Message successfully sent to channel G8BBR991D at 1512753610.000535
2017-12-08 17:20:11.230527 I | Message successfully sent to channel G8BBR991D at 1512753611.000040
2017-12-08 17:20:11.351597 I | Message successfully sent to channel G8BBR991D at 1512753611.000110
2017-12-08 17:20:11.451932 I | Message successfully sent to channel G8BBR991D at 1512753611.000167
2017-12-08 17:50:10.326576 I | Message successfully sent to channel G8BBR991D at 1512755409.000657
2017-12-08 17:50:10.422760 I | Message successfully sent to channel G8BBR991D at 1512755409.000719
2017-12-08 17:50:10.738609 I | Message successfully sent to channel G8BBR991D at 1512755410.000161
2017-12-08 17:50:10.830598 I | Message successfully sent to channel G8BBR991D at 1512755410.000220
2017-12-08 17:50:10.959868 I | Message successfully sent to channel G8BBR991D at 1512755410.000305
2017-12-08 17:50:11.109078 I | Message successfully sent to channel G8BBR991D at 1512755410.000387
2017-12-08 17:50:11.212455 I | Message successfully sent to channel G8BBR991D at 1512755410.000460
2017-12-08 17:50:11.336679 I | Message successfully sent to channel G8BBR991D at 1512755411.000066
2017-12-08 17:50:11.433150 I | Message successfully sent to channel G8BBR991D at 1512755411.000107
2017-12-08 17:50:11.535122 I | Message successfully sent to channel G8BBR991D at 1512755411.000170

Add Makefile

Create a Makefile:

to build and create docker image.

can't access slack bahind corporate proxy

2016-12-19 05:55:45.587255 I | Post https://slack.com/api/chat.postMessage: dial tcp 54.192.xxx.xx:443: i/o timeout

There is a container named proxy, but I don't think it is for proxy access.

  - image: gcr.io/skippbox/kubectl:v1.3.0
    args:
      - proxy
      - "-p"
      - "8080"
    name: proxy
    imagePullPolicy: Always

Consider to adopt a cli library for kubewatch

I would prefer to use a cli lib for kubewatch in order to make its help looks more senses, compare to the current one:

Usage of ./kubewatch:
  -alsologtostderr
        log to standard error as well as files
  -config-file string
        Configuration file
  -handler string
        Handler for event, can be [slack, default], default handler is printing event (default "default")
  -log_backtrace_at value
        when logging hits line file:N, emit a stack trace (default :0)
  -log_dir string
        If non-empty, write log files in this directory
  -logtostderr
        log to standard error instead of files
  -reason string
        Filter event by events, comma separated string
  -slack-channel string
        Slack channel
  -slack-token string
        Slack token
  -stderrthreshold value
        logs at or above this threshold go to stderr
  -v value
        log level for V logs
  -vmodule value
        comma-separated list of pattern=N settings for file-filtered logging

Extend support watching deployment, daemonset, replicaset

make a Helm chart

with the helm client installed simply do:

$ helm create kubewatch

Then stick the RC/Deployment yaml file in the Chart /templates directory, edit the Chart.yaml file

Test that you can deploy kubewatch with helm.

that's it.

Building docker image causes config error

Sorry I am new to kubernetes but I am trying to build the image and run it from my own repo. So I ran these commands

go get -u github.com/skippbox/kubewatch
make binary-image
docker tag kubewatch hantaowang/kubewatch
docker push hantaowang/kubewatch #repo already set up on dockerhub

I then modify tuna/kubewatch:v0.0.1 to hantaowang/kubewatch in kubewatch.yaml. Then I run

kubectl create -f kubewatch-configmap.yaml
kubectl create -f kubewatch.yaml

However the kubewatch container will not start. Running kubectl logs kubewatch kubewatch gives:

time="2017-12-30T08:41:32Z" level=fatal msg="Can not get kubernetes config: stat /root/.kube/config: no such file or directory"

Does anyone have any idea as to why? I am not sure what this error means. Thank you.

Reference Kubeconfig File?

Hi,

Is there a way to point Kubewatch to our kubeconfig file that contains the auth token and K8s API url?

Check why services don't seem to send events

If we create or delete a service, it does not seem to trigger an event.

something to check, maybe there is no watch api for services ? seems strange.

Add usage and ENV check in main

The main does not have any usage

plus it does not check whether the ENV var for SLACK token and channel are presents at startup...