Comments (56)
Hi @fjogeleit and @fonru I was able to test multi-cluster deployment. Thanks!
from policy-reporter.
@fjogeleit sry for long answer, i fetched repo and install...now there is no problem...i will continue testing)))
from policy-reporter.
@fjogeleit nice, fix it
I continue testing)))
from policy-reporter.
You can expose the api of cluster 2 to cluster 1 and configure it as external cluster:
Docs: https://kyverno.github.io/policy-reporter/guide/helm-chart-core#external-clusters
Auth will be supported in v2 of the UI, an alpha release is available here:
https://github.com/kyverno/policy-reporter/tree/3.x
from policy-reporter.
@fjogeleit yeah i know that i can expose the api of cluster 2 to cluster 1, but i cant understand how to install only UI on cluster 1
my values
rest:
enabled: true
ui:
enabled: true
plugins:
kyverno: true
clusterName: cluster1
clusters:
- name: cluster2
api: http://kyverno.local/
kyvernoApi: http://kyverno.local/
kyvernoPlugin:
enabled: true
from policy-reporter.
The helm chart is not intended to install only the UI without the core app right now.
also the Kyverno api needs to be the url of the Kyverno plugin app, but this is optional
from policy-reporter.
@fjogeleit i try to install the alpha version https://github.com/kyverno/policy-reporter/tree/3.x
but get error
Error: INSTALLATION FAILED: unable to build kubernetes objects from release manifest: error validating "": error validating data: unknown object type "nil" in Secret.data.config.yaml
my values set as:
plugin:
kyverno:
enabled: true
rest:
enabled: true
api:
logging: true
from policy-reporter.
I take a look
from policy-reporter.
@fjogeleit
i think problem somewhere .Files.Get "kyverno-plugin.tmpl"
from policy-reporter.
I already pushed a fix. Will release it now
from policy-reporter.
Should be available soo, can you please update and try it again?
from policy-reporter.
@fjogeleit now there is no problems with deployment, thx, but kyverno-plugin api not work (as i think)
values
plugin:
kyverno:
enabled: true
rest:
enabled: true
api:
logging: true
log_from_kyverno-plugin:
1.712140075207587e+09 info cmd/run.go:103 server starts {"port": 8080}
no errors, but when i open port-forward on kyverno-plugin pod and try to curl http://localhosts:8080/ready or http://localhosts:8080/healthz (404 error)
from policy-reporter.
The Service has only 3 APIs
http://localhost:8080/api/v1/policies
http://localhost:8080/api/v1/policies/{name}
http://localhost:8080/api/v1/policies/exception
The http://localhost:8080/api/v1/policies
is used for health checks.
from policy-reporter.
@fjogeleit
ok, thx
On another cluster i also install policy reporter from 3.0.0 branch with such values
ui:
enabled: true
clusters:
- name: cluster2
host: http://kyverno.local/policy-reporter/ # As i understand this path to policy-reporter API
plugins:
- name: kyverno
host: http://kyverno.local/ # As i understand this path to kyverno-plugin API
I open ui. and cant find how can i change cluster (as it was in previous) version
from policy-reporter.
This config overrides the default, so you have to add the default cluster above
ui:
enabled: true
clusters:
- name: Default
secretRef: policy-report-ui-default-cluster
- name: cluster2
host: http://kyverno.local/policy-reporter/ # As i understand this path to policy-reporter API
plugins:
- name: kyverno
host: http://kyverno.local/ # As i understand this path to kyverno-plugin API
from policy-reporter.
@fjogeleit ok, now i can switch, but when i switch to cluster2, i dont see any results by pod ( i check all kinds)
on Default(local) cluster all is well
from policy-reporter.
I'll take a look
from policy-reporter.
@fonru can you add the following config to your values:
ui:
server:
overwriteHost: true
It will be default true in the next release, I forgot to set the default value to true
from policy-reporter.
You can also update to the latest release which sets this value to true by default.
from policy-reporter.
thanks a lot for helping
from policy-reporter.
@fjogeleit one more problem
after i configured ingress to access web-ui by kyverno.local/ui
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
name: policy-reporter-ui
namespace: policy-reporter
spec:
ingressClassName: nginx
rules:
- host: kyverno.local
http:
paths:
- backend:
service:
name: policy-reporter-ui
port:
number: 8080
path: /ui/(.*)
pathType: Prefix
but the page not loaded, cause of problem with some js (404 error)
what i doing wrong?
from policy-reporter.
Let me take a look
from policy-reporter.
logs in ui pod
1.7126716913341775e+09 info auth/middleware.go:91 abort request {"path": "/", "err": "missing session key"}
values
ui:
enabled: true
ingress:
className: nginx
enabled: true
hosts:
- host: kyverno.mydomain.mydomain.tech
paths:
- path: /
pathType: ImplementationSpecific
openIDConnect:
callbackUrl: https://kyverno.mydomain.mydomain.tech
clientId: kyverno.mydomain.mydomain.tech
clientSecret: XXXXXXXXXXXXXXXX
when i auth with keycloak it redirect loop me to https://kyverno.mydomain.mydomain.tech/login but as i understand nothing is listen to this path? and in see 307 status
from policy-reporter.
You need to redirect to the /callback route of the ui
from policy-reporter.
The latest version should support a subpath configuration
from policy-reporter.
@fjogeleit thx
1.71272961207395e+09 info auth/middleware.go:91 abort request {"path": "/", "err": "missing session key"}
1.712729612074096e+09 error auth/middleware.go:41 profile not found
i configured /call, auth is ok. Is that normal, that logs like upper always generated in UI pod?
from policy-reporter.
Yeah but I can check again if they still needed. Most of them were necessary during the development
from policy-reporter.
@fjogeleit and one more question, are external-secrets like Vault normally work in secret_ref
key?
from policy-reporter.
Currently it supports secrets with a predefined set of keys. How would a vault secret look like? Is it a json or something similar?
from policy-reporter.
@fjogeleit i will look
and about openidconnect? can i restrict access to policy reporter ui ? Not found anything in docs and values...
from policy-reporter.
Right now it supports only authentication in general, no authorization via roles or similar. Thats on my todo list but I have to check how I can implement a generic way for the different providers.
from policy-reporter.
@fjogeleit thx a lot
have one more problem....i deploy PR on two cluster.
cluster1_values
ui:
enabled: true
server:
overwriteHost: true
openIDConnect:
enabled: true
discoveryUrl: 'xxxxxxxxx'
callbackUrl: https://cluster1.mydomain.tech/callback
clientId: "cluster1.mydomain.tech"
clientSecret: "xxxxxxxxx"
clusters:
- name: Default
secretRef: policy-report-ui-default-cluster
- name: cluster2
host: https://cluster2.mydomain.tech/policy-reporter/
plugins:
- name: kyverno
host: https://cluster2.mydomain.tech/plugin/
ingress:
enabled: true
className: "nginx"
hosts:
- host: cluster1.mydomain.tech
paths:
- path: /
pathType: ImplementationSpecific
plugin:
kyverno:
enabled: true
rest:
enabled: true
api:
logging: true
cluster2_values
rest:
enabled: true
ingress:
enabled: true
className: "nginx"
hosts:
- host: cluster2.mydomain.tech
paths:
- path: /
pathType: ImplementationSpecific
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
plugin:
kyverno:
enabled: true
ingress:
enabled: true
className: "nginx"
hosts:
- host: cluster2.mydomain.tech
paths:
- path: /plugin
pathType: ImplementationSpecific
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
ui:
enabled: true
ingress:
enabled: true
className: "nginx"
hosts:
- host: cluster2.mydomain.tech
paths:
- path: /
pathType: ImplementationSpecific
I open web-ui on https://cluster1.mydomain.tech/ (auth with OIDC work fine) and then when i switch in web-ui to cluster2, page open without any info.
And in logs of ui pod cluster1 have such errors log
│ 1.712840284955799e+09 error api/handler.go:176 failed to call core API {"error": "json: cannot unmarshal number into Go value of type []core.SourceCategoryTree"} ││ 1.712840285036027e+09 error api/handler.go:176 failed to call core API {"error": "json: cannot unmarshal number into Go value of type []core.SourceCategoryTree"} │
│ 1.7128402850471404e+09 error api/handler.go:234 failed to call core api {"error": "json: cannot unmarshal number into Go value of type []string"} ││ 1.7128402851375322e+09 error api/handler.go:234 failed to call core api {"error": "json: cannot unmarshal number into Go value of type []string"}
from policy-reporter.
thanks for reporting, I will take a look on it
from policy-reporter.
does the Cluster switch work without authentication enabled?
You configured:
ingress:
enabled: true
className: "nginx"
hosts:
- host: cluster2.mydomain.tech
paths:
- path: /
pathType: ImplementationSpecific
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
but you use https://cluster2.mydomain.tech/policy-reporter/
. I don't see the subpath in your ingress config
from policy-reporter.
@fjogeleit i fix (forgot .*) and redeploy this ingress annotations. it`s work fine with auth.
1.7129048219484746e+09 error api/handler.go:271 failed to load policies from plugin {"cluster": "cluster2", "plugin": "kyverno", "error": "EOF"}
1.7129048249718714e+09 error service/service.go:67 failed to load policy details from plugin {"cluster": "cluster2", "source": "kyverno", "policy": "podsecurity-subrule-restricted"
when i switch to cluster2 and open some policies in kyverno_plugin there is error (above) in UI pod of cluster1. Is that normal?? As i see all work fine, but can`t understand why some errors showing in ui pod(.
I continue testing)))
from policy-reporter.
Oh yeah, you need to add /api
to your cluster.plugin config. The UI is also working without the plugin but it will not show the policy details.
So it should be like this:
clusters:
- name: Default
secretRef: policy-report-ui-default-cluster
- name: cluster2
host: https://cluster2.mydomain.tech/policy-reporter/
plugins:
- name: kyverno
host: https://cluster2.mydomain.tech/plugin/api
from policy-reporter.
Hi @fonru have you tested multi-cluster deployment? Could you please provide the configuration you used to get it to run?
from policy-reporter.
What issue or question do you have?
a basic setup would be to add a new item to the cluster list:
clusters:
- name: Default
secretRef: policy-report-ui-default-cluster
- name: Cluster 2
host: https://policy-reporter-api.com # URL to the REST API of the Policy Reporter instance of your second cluster
the first item is the default Policy Reporter API in the same cluster. Its configured as secret which has a host
with the Core API URL and optional additional config like plugin URLs, HTTPBasic Auth credentials, etc.
from policy-reporter.
Hi @devang704 this is my two clusters values
#Policy reporter with GUI
ui:
enabled: true
server:
overwriteHost: true
openIDConnect:
enabled: true
discoveryUrl: 'https://kk.mydomain.local/realms/Common'
callbackUrl: https://kyverno-ui.mydomain.local/callback
clientId: "kyverno-ui.mydomain.local"
clientSecret: "xxxxxxxxxxxxxxxxxx"
clusters:
- name: Default
secretRef: policy-report-ui-default-cluster
- name: second-cluster
host: https://mydomain.local/policy-reporter/
plugins:
- name: kyverno
host: https://mydomain.local/plugin/api
ingress:
enabled: true
className: "nginx"
hosts:
- host: kyverno-ui.mydomain.local
paths:
- path: /
pathType: ImplementationSpecific
plugin:
kyverno:
enabled: true
rest:
enabled: true
api:
logging: true
#Policy reporter without GUI
rest:
enabled: true
ingress:
enabled: true
className: "nginx"
hosts:
- host: kyverno.mydomain.local
paths:
- path: /policy-reporter/(.*)
pathType: ImplementationSpecific
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
plugin:
kyverno:
enabled: true
ingress:
enabled: true
className: "nginx"
hosts:
- host: kyverno.mydomain.local
paths:
- path: /plugin/(.*)
pathType: ImplementationSpecific
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
it`s work fine, continue testing
from policy-reporter.
Hi @fjogeleit ,
I am using policy-reporter/policy-reporter helm chart with version 2.22.5, as part of this helm chart I can see two secrets are getting created 1) policy-reporter-kyverno-plugin-config 2) policy-reporter-config.
Which Secret needs to be configured in secretRef for Default cluster?
from policy-reporter.
This issue relates to the new UI v2 which is currently in an alpha state.
If you refer to the stable chart you can check: https://kyverno.github.io/policy-reporter/guide/helm-chart-core/#external-clusters
In this version the default cluster is not defined as secret and you only need to add additional external clusters without the default one.
from policy-reporter.
ok - I will test it. Thanks!
from policy-reporter.
Hi, I am getting below error in multi cluster configuration.
error app/main.go:110 failed to configure api proxies {"name": "Default", "error": "secrets "policy-report-ui-default-cluster" not found"}
error app/main.go:110 failed to configure api proxies {"name": "cluster2", "error": "missing core api configuration"}
`ui:
enabled: true
server:
overwriteHost: true
clusters:
- name: Default
secretRef: policy-reporter-config - name: cluster2
host: http://cluster-2.mydomain.com/policy-reporter/
plugins:- name: kyverno
host: http://cluster-2.mydomain.com/plugin/api`
- name: kyverno
from policy-reporter.
as I wrote in the last comment, you are using the current stable v1 UI which has a total different configuration structure. Please check the posted documentation link which shows the setup for Policy Reporter UI v1.
from policy-reporter.
@fjogeleit hi, pls help...
i use secrete to define telegram target (chatID and token)
target:
telegram:
chatID:
- secretRef: "policy-reporter-tg"
token:
- secretRef: "policy-reporter-tg"
minimumPriority: "warning"
skipExistingOnStartup: true
in "policy-reporter-tg" secret there are two key-value "chatID" and "token", but target not working(
what i doing wrong? if i use without secret all work fine
target:
telegram:
chatID: {my_chat_id}
token: {my_token}
minimumPriority: "warning"
skipExistingOnStartup: true
from policy-reporter.
and logs in PR pod:
1.7156132601547685e+09 error Telegram: PUSH FAILED {"statusCode": 404, "body": "{\"ok\":false,\"error_code\":404,\"description\":\"Not Found\"}"}
from policy-reporter.
target:
telegram:
chatID: {my_chat_id}
secretRef: "policy-reporter-tg"
minimumPriority: "warning"
skipExistingOnStartup: true
should be the correct structure. The secret needs the token key, ChatID is currently not supported as secret value.
from policy-reporter.
@fjogeleit i try such values:
target:
telegram:
chatID: "-XXXXXXXX" #my chat id starts with minus)
secretRef: "policy-reporter-tg"
minimumPriority: "warning"
skipExistingOnStartup: true
and it`s not work...
telegram policy reporter config in secrete looks like:
telegram:
config:
chatID: "-XXXXXXX"
token: ""
webhook:
certificate: ""
skipTLS: false
name:
path:
secretRef: "policy-reporter-tg"
mountedSecret: ""
minimumPriority: "warning"
skipExistingOnStartup: true
is that normal that token is empty??
and errors in PR pod now changed:
1.7156680474722931e+09 error Telegram: PUSH FAILED {"error": "Post \"https://api.telegram.org/bot6317557123:AAG2_JcepYUCqjlQ_lVM522a311Kfl0gAdw/sendMessage\": dial tcp 149.154.167.220:443: i/o timeout"}
if i delete secrete-ref and put token all works...((
from policy-reporter.
apiVersion: v1
kind: Secret
metadata:
name: policy-reporter-tg
data:
token: dG9rZW4=
this is how your secret needs to look like, only a token key with the related value
from policy-reporter.
first i think that problem is with external-secrets, but external-secret create secrete equvalient to secret, that u mension above.
i try to create secrete manually
apiVersion: v1
kind: Secret
metadata:
name: policy-reporter-tg
namespace: kyverno-pr
data:
token: {base64 decoded value of my token}
describe of my created secret
│ apiVersion: v1 │
│ data: │
│ token: {my token in base64} │
│ kind: Secret │
│ metadata: │
│ annotations: │
│ kubectl.kubernetes.io/last-applied-configuration: | │
│ {"apiVersion":"v1","data":{"token":"{my token in base64}"},"kind":"Secret","metadata":{"annotations":{},"name":"policy-reporter-tg","namespace":"kyverno-pr"}} │
│ creationTimestamp: "2024-05-14T10:12:54Z" │
│ name: policy-reporter-tg │
│ namespace: kyverno-pr│
│ resourceVersion: "434105395" │
│ uid: 19c9b36b-edc0-4ee4-9d11-1ace1cb7086e │
│ type: Opaque
my values:
target:
telegram:
chatID: "-4196593432"
secretRef: "policy-reporter-tg"
minimumPriority: "warning"
skipExistingOnStartup: true
but it`s not working (((((
from policy-reporter.
I will take a look
from policy-reporter.
1.7156680474722931e+09 error Telegram: PUSH FAILED {"error": "Post \"https://api.telegram.org/bot6317557123:AAG2_JcepYUCqjlQ_lVM522a311Kfl0gAdw/sendMessage\": dial tcp 149.154.167.220:443: i/o timeout"}
if i delete secrete-ref and put token all works...((
But if you check the error message above:
The structure is: https://api.telegram.org/bot{your-token}/sendMessage
And its not empty, so it looks like your token is correctly set.
from policy-reporter.
@fjogeleit i find problem...i reinstall helm with new values, but i need to restart pod with PR
from policy-reporter.
hm oh yeah .. sure ... the secret is only picked up on startup, its not updating when you change it during a running pod.
from policy-reporter.
Hi Frank,
The policy-reporter is working perfect with multiple cluster/ multi-tenant.
As of now we can select the cluster from dropdown and get the detailed information about kyverno policies, but if I want to see the summary for all the cluster.
Is there any feature in UI to view the policy-report for all the clusters in single panel?
from policy-reporter.
Hey, right now this is not possible. I can take a look on this but if so only for the new UI. The structure of the current one does not allow a feature like this.
from policy-reporter.
Related Issues (20)
- Policy Reporter Targets as CRDs HOT 4
- Include the timestamp HOT 7
- Policy Reporter not aligned with Kyverno HOT 5
- Helm chart won't start due to "Error: unknown flag: --template-dir" HOT 2
- Kyverno Policy Reporter writes extensively on node disk storage in certain ocassions. HOT 2
- strange S3 config error when using SecurityHub as target HOT 3
- Helm: Ingress template has wrong selection for pathType
- Allow for not rendering a Kubernetes Secret HOT 2
- policy-reporter presents incomplete results HOT 7
- Slack Channel Override does not work HOT 4
- Support for new reports.x-k8s.io reports group
- Support for multicluster HOT 1
- Loki not receiving/showing logs HOT 6
- Wrong key "label" instead of "labels" in redis mainfest file HOT 1
- The Secrets do not have the namespace in manifests HOT 1
- Question: Integration with Security Hub HOT 6
- CrashLoopBackOff policy-reporter-kyverno-plugin when deploying the manifest HOT 1
- Prometheus: Out-of-order samples HOT 4
- Feature request: Support IAM role for service accounts for AWS Security Hub integration HOT 3
- General question about persistance in Policy Reporter HOT 8
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from policy-reporter.