Welcome to The Falco Project's collaborative testing initiatives in partnership with the CNCF Environmental Sustainability Technical Advisory Group (TAG ENV) - Green Reviews Working Group.
This repository functions as the hosting platform for Falcos' daemonset configurations intended for testing with the CNCF Green Reviews Working Group. These configurations will be used within the following repository: https://github.com/cncf-tags/green-reviews-tooling/, leveraging the Flux framework (see the Falco Flux Config).
The primary directory structure is outlined below:
├── kustomize
│ ├── falco-driver
│ │ ├── ebpf
│ │ │ ├── configmap.yaml
│ │ │ ├── daemonset.yaml
│ │ │ ├── falco-event-generator.yaml
│ │ │ ├── redis.yaml
│ │ │ └── stress-ng.yaml
│ │ ├── kmod
│ │ │ ├── configmap.yaml
│ │ │ ├── daemonset.yaml
│ │ │ ├── falco-event-generator.yaml
│ │ │ ├── redis.yaml
│ │ │ └── stress-ng.yaml
│ │ └── modern_ebpf
│ │ ├── configmap.yaml
│ │ ├── daemonset.yaml
│ │ ├── falco-event-generator.yaml
│ │ ├── redis.yaml
│ │ └── stress-ng.yaml
│ ├── falco-generic
│ │ ├── falcoctl-configmap.yaml
│ │ └── serviceaccount.yaml
│ └── kustomization.yaml
├── LICENSE
├── OWNERS
└── README.md
The Falco daemonset definitions under ./kustomize/driver/{ebpf,kmod,modern_ebpf}/daemonset.yaml resemble existing templates available at https://github.com/falcosecurity/deploy-kubernetes/, but are customized to cater to specific purposes and requirements (e.g. namespace falco and a driver specific nodeSelector, e.g. cncf-project-sub: "falco-driver-modern-ebpf").
Furthermore, there's a customized setup within the Falco container entrypoint and falco.yaml settings, focusing on benchmarking Falco's performance. Notably, we direct Falco alerts and internal metrics solely to log-rotated files, unlike real-world scenarios where this data is usually sent off the knode to a data lake.
For our testing process, each Falco driver type undergoes testing on its own dedicated knode.
Each Falco driver-specific deployment under ./kustomize/driver/{ebpf,kmod,modern_ebpf}/ also contains deployments for microservices or teststress frameworks aimed at generating synthetic workloads on the CNC testbed servers.
| Knode | Falco Driver | Namespace | Node Selector |
|---|---|---|---|
| knode A | modern-ebpf | falco | cncf-project: "falco" |
| cncf-project-sub: "falco-driver-modern-ebpf" | |||
| knode B | ebpf | falco | cncf-project: "falco" |
| cncf-project-sub: "falco-driver-ebpf" | |||
| knode C | kmod | falco | cncf-project: "falco" |
| cncf-project-sub: "falco-driver-kmod" |
| Knode | Kernel Version Requirement | Additional Requirements | BPF Stats Enabled |
|---|---|---|---|
| knode A | >= 5.8 | eBPF supported | 1 |
| knode B | >= 4.14 | eBPF supported, Kernel headers installed | 1 |
| knode C | >= 2.6.32 | DKMS package, Kernel headers installed | N/A |
Notes:
- The Falco Deployment enables
kernel.bpf_stats_enabledby default. - For both
ebpfandkmod, additional host mounts are required, such as/usr/src/and/lib/modules. Please refer to the respective daemonset configuration for more details. - We anticipate
containerdto be the container runtime socket located at/run/k3s/containerd/containerd.sock. - The CNCF test Kubernetes cluster offers machines with 16 CPUs.
Expand Testing Instructions
To test these configurations on localhost using minikube, make sure you have minikube and kubectl installed and running. In order to test kmod and ebpf drivers, additional host mounts are required. Minikube needs a specific setting to accommodate this, as shown below:
minikube start --mount --mount-string="/usr/src:/usr/src" --mount --mount-string="/dev:/dev" --driver=docker --nodes 4
NOTE: You won't be able to properly test Falco's container engine using minikube. Please be aware of this limitation, and there can still be issues with host mounts.
NOTE: For localhost testing reduce the number of replicas for the synthetic workload deployments.
NOTE: Finally, we recommend testing on Ubuntu 22.04 to reflect the CNCF testbed setup. You can use the Vagrant VM config shared here.
Proceed by executing the following setup commands:
kubectl create namespace falco;
kubectl get nodes;
# Test cncf-project-sub=falco-driver-modern-ebpf (easiest)
kubectl label nodes minikube-m02 cncf-project=falco cncf-project-sub=falco-driver-modern-ebpf --overwrite;
# Test cncf-project-sub=falco-driver-ebpf
kubectl label nodes minikube-m03 cncf-project=falco cncf-project-sub=falco-driver-ebpf --overwrite;
# Test cncf-project-sub=falco-driver-kmod
# WARNING: Testing kernel modules on a local dev box is more risky,
# remember to unload the module `sudo rmmod falco`
# Testing kmod within a smaller VM with minikube likely crashes, only test w/ minikube on a larger native box
# kubectl label nodes minikube-m04 cncf-project=falco cncf-project-sub=falco-driver-kmod --overwrite;
kubectl get nodes --show-labels;Apply the configurations by executing the following command:
kubectl apply -k ./kustomize
# Tear-down
kubectl delete -k ./kustomizeVerify if the pods are up and running (Note that the output below is not regularly updated, and there might be more pods and containers running than displayed):
kubectl get pods -n falco
NAME READY STATUS RESTARTS AGE
falco-driver-ebpf-bjvgc 1/1 Running 0 5m26s
falco-driver-modern-ebpf-fpph9 1/1 Running 0 5m26s
falco-event-generator-driver-ebpf-785c6cc7dc-58wjr 1/1 Running 0 5m27s
falco-event-generator-driver-modern-ebpf-64674f78bf-fjvn7 1/1 Running 0 5m27s
redis-driver-ebpf-cbdd47b74-4drg4 3/3 Running 0 5m27s
redis-driver-ebpf-cbdd47b74-lb6wt 3/3 Running 0 5m27s
redis-driver-ebpf-cbdd47b74-lt6q7 3/3 Running 0 5m27s
redis-driver-ebpf-cbdd47b74-pcm8g 3/3 Running 0 5m27s
redis-driver-ebpf-cbdd47b74-rv2ww 3/3 Running 0 5m27s
redis-driver-modern-ebpf-7c4bdd9d58-2fqp9 3/3 Running 0 5m27s
redis-driver-modern-ebpf-7c4bdd9d58-2ms8j 3/3 Running 0 5m27s
redis-driver-modern-ebpf-7c4bdd9d58-k5vtw 3/3 Running 0 5m27s
redis-driver-modern-ebpf-7c4bdd9d58-kztgj 3/3 Running 0 5m27s
redis-driver-modern-ebpf-7c4bdd9d58-rf9m2 3/3 Running 0 5m27s
stress-ng-driver-ebpf-78766f6fbd-cxljg 2/2 Running 0 5m27s
stress-ng-driver-ebpf-78766f6fbd-rb9wn 2/2 Running 0 5m27s
stress-ng-driver-modern-ebpf-7885fdc996-mkb78 2/2 Running 0 5m27s
stress-ng-driver-modern-ebpf-7885fdc996-rzl4h 2/2 Running 0 5m26s
...
To drop interactively into the Falco container, execute the exec command as follows:
kubectl -n falco exec -it falco-driver-modern-ebpf-5vwl6 -c falco -- bashExecute dummy suspicious commands and examine Falco's alert outputs and native metrics logs:
cat /etc/shadow
# Falco alerts outputs
cat /tmp/falco/events.jsonl
# Falco native metrics logs; recommend adjusting `interval: 1m` for quicker testing
cat /tmp/stats/falco_stats.jsonlThe Falco container includes utilities installed for ad-hoc checks on the Falco process:
ps aux
htopExtra Tips
# Check if Falco's kmod was loaded
lsmod | grep falco
# Inspect possible issues with a pod
kubectl -n falco describe pod falco-driver-modern-ebpf-5vwl6The respective CONFIG_VERSION environment variable within the daemonset deployment contains the semver-compatible version of the testbed setup. We inject it (as a suffix) into the FALCO_HOSTNAME environment variable to maintain a version record extending beyond the Falco version in the native Falco metrics. Every merge into the main branch necessitates a (mostly minor) version increment.
Please refer to the contributing guide and the code of conduct for more information on how to contribute.
This project is licensed to you under the Apache 2.0 open source license.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent