open-policy-agent / conftest Goto Github PK
View Code? Open in Web Editor NEWWrite tests against structured configuration data using the Open Policy Agent Rego query language
Home Page: https://conftest.dev
License: Other
Write tests against structured configuration data using the Open Policy Agent Rego query language
Home Page: https://conftest.dev
License: Other
As we both add more features, more examples and more integrations (Helm, GitHub Actions, etc.) the README won't be the best place to put all the documentation.
It would be good to create a separate docs site with multiple pages.
I've had good luck with MkDocs (https://www.mkdocs.org/) before, for instance with
https://github.com/instrumenta/kubeval. This just involves a simple config file and a directory of markdown files managed in the same repo.
I would like to test if the namespace of the resource is the same as folder name where my configuration files are present. In this case, I would like to get the parent folder name.
Has there been any talk on moving some of the private functions like buildCompile
to a separate package in the repo so they could be used as a library in other projects? Or would you have any interest in a PR that does this?
Hello,
I am wondering if the rules being names deny
and warn
is mandatory ?
I've been first writing my policies with opa
using the test framework available with it. It allows a TDD approach when writing the policies.
When testing conftest
(which have a better output for a CI integration), I have been kind of disappointed to see that all the rules should be named deny
or warn
, It forced me to rename my rules, and I am no more able to use opa
tests anymore :/
Did I miss something ?
Would it be possible for conftest
to look for rules through a regex ?
It would allow to write rules named deny_some_explicit_name
or warn_some_other_name
so that conftest
finds the rules and one can still test the policies through opa
test framework.
Version 0.6.0 worked great, but with 0.7.0 and 0.9.0, conftest seems to be rerunning my policies multiple times when I run conftest test
. Is there some change I need to make? My policies don't differ structurally from the kubernetes example in the repo.
With the work started in #55 we are soon to be able to support multiple output formats. Some discussion in #26 and #42.
Opening this issue to discuss TAP output support. I think the best we can do is:
1..3
not ok 1 - deployment.yaml - Containers must not run as root
not ok 2 - deployment.yaml - Deployments are not allowed
# Warnings
not ok 3 - deployment.yaml - Services not allowed
ok
responses at present#
is a diagnostic information, it's not machine readable be is at least human readable. The alternative would be to not include the warnings which I feel is worseThe TAP specification can be found at http://testanything.org/tap-specification.html
TAPs advantage is its simplicity, so a number of tools can consume it and display the results in whatever way they choose.
To expand the potential places where Conftest could be used, it would be good to support HCL as used in Terraform and GitHub Actions as an input. Conftest currently supports JSON or YAML only as input, but should work with any format we can unmarshall.
This should just be about either using something more generic, or adding some logic around the current use of the YAML module around this line.
As a Conftest user, I want to know that how the inputs transformed into structured data. This feature especially needed for when we are writing tests against Dockerfile
, hcl2
, or ini
like files, that needs to be parsed.
The JSON output for the parsing process would be nice, in order to understand exactly how the files are transformed and how we can write tests against them:
For instance, to understand which one is correct, input.x.y["z.t"] = foo
or input["x"].y["z.t"] = foo
, I have to take a look at the parsers.
I think this can be achieved with a flag like --reveal-structure
but other suggested ideas are welcomed. btw, I'm not sure if this feature is needed for the users, so let's discuss it!
Any thoughts?
Another proof of concept, I build a Helm plugin for Conftest https://github.com/instrumenta/helm-conftest
helm conftest <path-to-chart>
Currently this simply allows accessing conftest from Helm. Helm runs helm template under the hood to render the template, and conftest then tests that document against the defined policies.
Not supported in the above, but probably desirable, would be to be able to pass a number of the helm template
arguments, in particular --set
and --set-file
.
The initial proof of concept is a very simple Bash script, but it would probably be good to create a standalone binary which uses the relevant packages from Helm. Similar to #52, this would be another case for testing the public interface for Conftest as a library.
Assume we got a dummy.yaml
and contains:
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: xapi
spec:
replicas: 15
template:
metadata:
labels:
app: xapi
spec:
hostAliases:
- ip: "127.0.0.1"
hostnames:
- "foo.org"
- ip: "127.0.0.1"
hostnames:
- "bar.org"
containers:
- name: nginx
image: nginx:alpine
ports:
- containerPort: 80
resources:
requests:
cpu: "1000m"
and a deny.rego
file in policy
folder,
package main
deny[msg] {
output := sprintf("%s", [input.spec.template.spec])
not contains(output, "{\"hostnames\": [\"foo.org\"], \"ip\": \"127.0.0.1\"}")
msg = output
}
And when we execute test commands several times manually or
while true; do echo "conftest test dummy.yaml"; conftest test dummy.yaml; done
$ conftest test dummy.yaml
$ conftest test dummy.yaml
$ conftest test dummy.yaml
FAIL - dummy.yaml - {"containers": [{"image": "nginx:alpine", "name": "nginx", "ports": [{"containerPort": 80}], "resources": {"requests": {"cpu": "1000m"}}}], "hostAliases": [{"ip": "127.0.0.1", "hostnames": ["foo.org"]}, {"hostnames": ["bar.org"], "ip": "127.0.0.1"}]}
$ conftest test dummy.yaml
$ conftest test dummy.yaml
FAIL - dummy.yaml - {"containers": [{"image": "nginx:alpine", "name": "nginx", "ports": [{"containerPort": 80}], "resources": {"requests": {"cpu": "1000m"}}}], "hostAliases": [{"ip": "127.0.0.1", "hostnames": ["foo.org"]}, {"hostnames": ["bar.org"], "ip": "127.0.0.1"}]}
We sometimes fail, sometimes pass. This bug is probably caused by Rego
but can affect tests.
Do we have to open this issue to OPA and close this one?
Any thoughts?
Just tracking here for visibility, and probably for linking into the documentation. See the PR tektoncd/catalog#60 for the Task itself.
I've been super happy to see the interest in Conftest, both from folks wanting to use it and from several folks who have already contributed fixes and features. One of the reasons to start this under an org rather than my own name was to make it a touch easier to add other folks if the project proved interesting.
With that, I thought it would be easiest to open an issue to discuss.
Pinging @brendanjryan, @jpreesem, @Blokje5, @xchapter7x and @Proplex just in case that is something of interest?
Right now conftest compiles the Rego policies at the given directory. However, it does not recursively search that directory, which makes it hard to structure a set of policies.
For example I was trying the following directory structure for policies:
policy/
terraform/
aws/
aws.rego
kubernetes/
main.rego
surprisingly it did not error. However my policies just did not work as expected.
To reproduce try the following:
aws.rego:
package aws
minimum_tags = {"ApplicationRole", "Owner", "Project"}
tags_contain_proper_keys(tags) {
keys := {key | tags[key]}
leftover := minimum_tags - keys
leftover == set()
}
tags_contain_minimum_set[i] = resources {
changeset := input.resource_changes[i]
tags := changeset.change.after.tags
resources := [resource | resource := changeset.address; not tags_contain_proper_keys(changeset.change.after.tags)]
}
main.rego
package main
import data.aws
deny[msg] {
resources := aws.tags_contain_minimum_set[_]
resources != []
msg := sprintf("Invalid tags (missing minimum required tags) for the following resources: %v", [resources])
}
input.json
{
"format_version": "0.1",
"terraform_version": "0.12.4",
"planned_values": {
"root_module": {
"resources": [
{
"address": "aws_s3_bucket.profile_picture_storage",
"mode": "managed",
"type": "aws_s3_bucket",
"name": "profile_picture_storage",
"provider_name": "aws",
"schema_version": 0,
"values": {
"acl": "private",
"bucket_prefix": "profile-picture-storage",
"cors_rule": [],
"force_destroy": false,
"lifecycle_rule": [],
"logging": [],
"object_lock_configuration": [],
"policy": null,
"replication_configuration": [],
"server_side_encryption_configuration": [
{
"rule": [
{
"apply_server_side_encryption_by_default": [
{
"kms_master_key_id": null,
"sse_algorithm": "AES256"
}
]
}
]
}
],
"tags": null,
"versioning": [
{
"enabled": true,
"mfa_delete": false
}
],
"website": []
}
},
{
"address": "aws_s3_bucket_public_access_block.profile_picture_storage_access_rules",
"mode": "managed",
"type": "aws_s3_bucket_public_access_block",
"name": "profile_picture_storage_access_rules",
"provider_name": "aws",
"schema_version": 0,
"values": {
"block_public_acls": true,
"block_public_policy": true,
"ignore_public_acls": true,
"restrict_public_buckets": true
}
}
]
}
},
"resource_changes": [
{
"address": "aws_s3_bucket.profile_picture_storage",
"mode": "managed",
"type": "aws_s3_bucket",
"name": "profile_picture_storage",
"provider_name": "aws",
"change": {
"actions": [
"create"
],
"before": null,
"after": {
"acl": "private",
"bucket_prefix": "profile-picture-storage",
"cors_rule": [],
"force_destroy": false,
"lifecycle_rule": [],
"logging": [],
"object_lock_configuration": [],
"policy": null,
"replication_configuration": [],
"server_side_encryption_configuration": [
{
"rule": [
{
"apply_server_side_encryption_by_default": [
{
"kms_master_key_id": null,
"sse_algorithm": "AES256"
}
]
}
]
}
],
"tags": null,
"versioning": [
{
"enabled": true,
"mfa_delete": false
}
],
"website": []
},
"after_unknown": {
"acceleration_status": true,
"arn": true,
"bucket": true,
"bucket_domain_name": true,
"bucket_regional_domain_name": true,
"cors_rule": [],
"hosted_zone_id": true,
"id": true,
"lifecycle_rule": [],
"logging": [],
"object_lock_configuration": [],
"region": true,
"replication_configuration": [],
"request_payer": true,
"server_side_encryption_configuration": [
{
"rule": [
{
"apply_server_side_encryption_by_default": [
{}
]
}
]
}
],
"versioning": [
{}
],
"website": [],
"website_domain": true,
"website_endpoint": true
}
}
},
{
"address": "aws_s3_bucket_public_access_block.profile_picture_storage_access_rules",
"mode": "managed",
"type": "aws_s3_bucket_public_access_block",
"name": "profile_picture_storage_access_rules",
"provider_name": "aws",
"change": {
"actions": [
"create"
],
"before": null,
"after": {
"block_public_acls": true,
"block_public_policy": true,
"ignore_public_acls": true,
"restrict_public_buckets": true
},
"after_unknown": {
"bucket": true,
"id": true
}
}
}
],
"configuration": {
"root_module": {
"resources": [
{
"address": "aws_s3_bucket.profile_picture_storage",
"mode": "managed",
"type": "aws_s3_bucket",
"name": "profile_picture_storage",
"provider_config_key": "aws",
"expressions": {
"acl": {
"constant_value": "private"
},
"bucket_prefix": {
"constant_value": "profile-picture-storage"
},
"server_side_encryption_configuration": [
{
"rule": [
{
"apply_server_side_encryption_by_default": [
{
"sse_algorithm": {
"constant_value": "AES256"
}
}
]
}
]
}
],
"versioning": [
{
"enabled": {
"constant_value": true
}
}
]
},
"schema_version": 0
},
{
"address": "aws_s3_bucket_public_access_block.profile_picture_storage_access_rules",
"mode": "managed",
"type": "aws_s3_bucket_public_access_block",
"name": "profile_picture_storage_access_rules",
"provider_config_key": "aws",
"expressions": {
"block_public_acls": {
"constant_value": true
},
"block_public_policy": {
"constant_value": true
},
"bucket": {
"references": [
"aws_s3_bucket.profile_picture_storage"
]
},
"ignore_public_acls": {
"constant_value": true
},
"restrict_public_buckets": {
"constant_value": true
}
},
"schema_version": 0
}
]
}
}
}
execute the following command with the policies stored in a policy
dir:
conftest test input.json
When the policies are stored in the same directory this input will return an error. Otherwise it does not return anything (Not even an error).
The buildCompiler
method in the test
command should be updated to recursively search for subdirectories when compiling the rego policies.
I build a proof of concept kubectl
plugin for conftest, which can be found here https://github.com/instrumenta/conftest/tree/master/plugin. I'll let folks read the one liner in there to understand why I say proof of concept :)
It works as follows:
$ kubectl conftest -h
A Kubectl plugin for using Conftest to test objects in Kubernetes using Open Policy Agent
See https://github.com/instrumenta/conftest for more information
Usage:
kubectl conftest (TYPE[.VERSION][.GROUP] [NAME] | TYPE[.VERSION][.GROUP]/NAME)
You can use the same syntax as kubectl get
to grab lists or individual resources, and then have conftest apply policy against them.
$ kubectl conftest service
Found service hello-kubernetes but services are not allowed
Found service kubernetes but services are not allowed
I reason this is useful for auditing an existing cluster. Especially useful if you have opinions encoded in Rego, and someone has just given you a new cluster to manage. It's also useful when writing policies, as you can easily test it against your real cluster and not just mocked data.
I proposed this to the Krew index, kubernetes-sigs/krew-index#146, but that raised a larger conversation about what the Krew folks want in the index.
Opening this issue to track taking the PoC and making a more robust plugin. I'm imagining that as another standalone Go binary, reusing much of the Conftest code. This is a probably a good place to look at the public Conftest library interface too.
Hi, thanks for this awesome project :) I've been trying to use an OCI registry to push and pull the policies and hit a problem.
I have a conftest.toml
as follows:
policy = "policy"
[[policies]]
repository = "localhost:5000/conftest-circleci"
tag = "latest"
It's pointing to my local docker registry but fails to pull it on conftest update
.
$ conftest --debug update
INFO[0000] Downloading: localhost:5000/conftest-circleci
FATA[0000] Downloading policy failed: object required
It looks like it's missing the tag
because the localhost:5000
is containing a :
and conftest thinks the string already contains a tag.
https://github.com/instrumenta/conftest/blob/master/conftest.go#L323
Hello,
I saw that OPA was in the inspiration list and I am currently looking for tools that would help me test my terraform code.
I didn't try both tools a lot for the moment, but what would you think would be an advantage of contest compared to using open policy agent ?
Both seems very close in term of features for my use case.
Thanks
Hi! We've started using conftest
for evaluating our K8S YAML files last week, and it's been a nice way to encode certain conventions we expect our configurations to follow.
One thing that we immediately noticed we wanted but was not possible was to be able to verify fields across documents. For example, a GCE Ingress's service port needs to align with the backing ports in the service/deployment. Or, a deployment that pulls in values from a configmap needs to match the metadata names.
I took a quick skim of the code, and in general, it appears that it would require restructuring a bit to support this (the input
object may need to be more akin to a map or an array), but doing so would unlock some serious configuration checking super powers.
Thanks!
Brian
It looks like the Terraform 0.12 syntax isn't supported.
.tf
snippet:
resource "aws_vpc" "vpc" {
cidr_block = var.cidr
...
}
Error:
FATA[0000] Problem running evaluation: load terraform config failed: Error parsing JSON document as HCL: At 2:26: Unknown token: 2:26 IDENT var.cidr
Hey there ๐, loving conftest so far.
I'm interesting in picking up #5 and I also want to write tests for it. However, the functions are written in a way that a bulk of the action is within an anonymous function in the Command{}
struct.
Would y'all be open to the idea of separating it out as such:
func NewTestCommand() *cobra.Command {
cmd := &cobra.Command{
Use: "test <file> [file...]",
Short: "Test your configuration files using Open Policy Agent",
Version: fmt.Sprintf("Version: %s\nCommit: %s\nDate: %s\n", constants.Version, constants.Commit, constants.Date),
Run: TestFunction()
func TestFunction(ccmd *cobra.Command, args []string)
ctx := context.Background()
if len(args) < 1 {
cmd.SilenceErrors = true
log.G(ctx).Fatal("The first argument should be a file")
}
...
Right now we're re-defining a lot of the Cobra command definitions for each test so that we can single out functionality for testing. Changing each command to use a standalone function would reduce the amount of code duplication for testing. This would make testing easier for others, and as far as I'm aware not alter any user functionality.
Would you accept a PR into master with this change? Thanks :)
cc: @Salamandastron1
To expand the potential places where Conftest could be used, it would be good to support INIT as an input. Conftest currently supports JSON, YAML, HCL, TOML and CUE as input, but should work with any format we can unmarshall.
This is a matter of implementing the parser interface based on the existing examples.
Recently open policy agent gatekeeper has implemented changes in rego code. Now they use violation
instead of deny
. We use conftest to verify our PRs before merge. That simple change has caused the conftest to skip all the checks which worked before.
We kindly ask you to extend deny rule name to a list of (deny|rule|violation).
here you have the patch.
commit cf0c59d92802f5b53bdabc452295b296baef59b8
Author: Oleksandr DUDCHENKO <[email protected]>
Date: Fri Sep 6 17:21:30 2019 +0300
extended list for possible deny rule names
diff --git a/pkg/commands/test/test.go b/pkg/commands/test/test.go
index f5ed225..acf9d0d 100644
--- a/pkg/commands/test/test.go
+++ b/pkg/commands/test/test.go
@@ -26,7 +26,7 @@ import (
)
var (
- denyQ = regexp.MustCompile("^deny(_[a-zA-Z]+)*$")
+ denyQ = regexp.MustCompile("^(deny|rule|violation)(_[a-zA-Z]+)*$")
warnQ = regexp.MustCompile("^warn(_[a-zA-Z]+)*$")
)
Currently releasing a new version of conftest, and then building the Docker images and pushing to Docker Hub, are done manually offline.
Configuration is all done with goreleaser (GitHub release) and Make (Docker).
It should be possible to have CircleCI do this when new tags are added to master.
Basic introductions in https://goreleaser.com/ci/
To expand the potential places where Conftest could be used, it would be good to support CUE natively as an input. Conftest currently supports JSON or YAML only as input, but should work with any format we can unmarshall.
We have CUE examples, but would be useful to not require pipeline commands together.
This should just be about either using something more generic, or adding some logic around the current use of the YAML module around this line.
The current command lint output of conftest
is difficult to parse by CI or other command line tooling - mostly due to the fact that it does use show absolute file paths or distinguish between errors and warnings aside from color.
I propose changing the "output" to be of the form <msg_type>: <file> - <msg>
(still colorized)
For example:
WARNING: /Users/brendanjryan/projects/conftest/example/test.yaml - nginx-deployment should not be configured to live in the default namespace
FAILURE: /Users/brendanjryan/projects/conftest/example/test.yaml - nginx-deployment must not include any Horizontal Pod AutoScalers
Reference of existing format:
conftest test deployment.yaml
deployment.yaml
Containers must not run as root
Deployments are not allowed
Is this something we are interested in supporting? While this may break existing integrations - it will make future ones much easier to write.
When testing for two labels being required within a deployment, if either match passes the whole block is accepted.
Given a rule;
deny[msg] {
input.kind = "Deployment"
not input.spec.selector.matchLabels.app
not input.spec.selector.matchLabels.release
msg = sprintf("Deployment[%s] - Containers must provide app/release labls for pod selectors", [name])
}
See Gist for deployment.yaml test case with 4 deployments.
deployment.yaml
Deployment[app_only] - Containers must provide app/release labls for pod selectors
Deployment[release_only] - Containers must provide app/release labls for pod selectors
Deployment[no_labels] - Containers must provide app/release labls for pod selectors
$ conftest test deployment.yaml
deployment.yaml
Deployment[no_labels] - Containers must provide app/release labls for pod selectors
conftest --version
Version: 0.6.0
Commit: a27d0739a785fc52c421339d129267772a15662f
Date: 2019-05-20T07:40:07Z
See public Gist for configuration and test case:
https://gist.github.com/warmfusion/531aec5da54b1b248b02f40c48199e19
Error with some in rego:
tags_valid_pascal_case(tags) {
some key
# Foreach key and value, validate regex
val := tags[key]
re_match(`^([A-Z][a-z0-9]+)+`, key)
re_match(`^([A-Z][a-z0-9]+)+`, val)
}
results in the following error:
time="2019-06-19T13:13:16Z" level=fatal msg="Problem building rego compiler: 1 error occurred: main.rego:7: rego_parse_error: no match found\n\t some key\n\t ^"
Is this related to the interal OPA version?
Hi,
To expand the potential places where Conftest could be used, it would be good to support dockerfile as an input.
Thanks
When upgrading from 0.11 to 0.13, conftest went silent. No warnings or failed tests.
# previously
$ brew switch conftest 0.11.0
$ conftest test deployment.yaml
FAIL - prometheus-adapter is running as root
# now
$ brew switch conftest 0.13.0
$ conftest test deployment.yaml
# silent...
I expected at least some output, not just silently passing all tests.
$ conftest --version
Version: 0.13.0
Commit: 8340065
Date: 2019-09-27T22:40:00Z
The Docker API is not intended to be stable, so our dependency on it is going to cause dependency problems for us. We're already experiencing some.
It would probably be in our best interest to move buildkit's parser functionality to its own module, and own that functionality.
EDIT: Updating to include our go.mod
as a whole. Right now we have a lot of replace
directives due to our dependencies. I'd like to use this issue as a means to track upstream PRs to fix their go.mod
files, so ours becomes cleaner as well.
oras: oras-project/oras#132
Although Conftest can be used for testing any configuration, there is certainly lots of interest in testing Kubernetes configuration. I don't see this as much about Conftest, as much as about content. ie. it's about a standard library of Rego code which provides a useful DSL for common Kubernetes assertions.
Conftest has the push
and pull
commands that allow for saving OPA bundles to a OCI registry. This is somewhat experimental, but serves as a nice way of sharing Rego.
Opening an issue here to discuss, although my first instinct is to open a new repository for the project if folks like the idea.
I could imagine things like:
latest
tags, etc.)I'm thinking of starting with Kubernetes, but obviously the same pattern would hold for other common formats.
I'm a big fan of OPA's "table" output largely due to its readability, which you can see at work in this blog post that I wrote. It'd be nice to add that to the supported output options for Conftest.
The implementation seems pretty trivial so I'll attempt it myself.
This used to work - looks like the code in question has been re-worked lately to check for file-extension types yaml/toml.
Given a valid yaml file deploy.yaml
...
cat deploy.yaml | conftest test -
Unable to find available parser for extension
Where as the following works fine
conftest test deploy.yaml
Right now if I want to execute both conftest and evaluate the policies using Rego unit tests, I need to install both conftest and OPA. Ideally we would be able to use conftest for this purpose as well, as this would simplify my CI/CD pipeline.
I noticed in #48 that CI is not running on pull requests - although it is configured to run on master
.
Is this intentional? If not i'd be more than happy to get this integrated on every build!
To expand the potential places where Conftest could be used, it would be good to support TOML as an input. Conftest currently supports JSON or YAML only as input, but should work with any format we can unmarshall.
This should just be about either using something more generic, or adding some logic around the current use of the YAML module around this line.
I would like to have multiple policy directories being used in single run.
Use case :
I would like to have CI image with some common rules being there, but allow users to add their own rules to some other directory, and in CI run to use conftest
to apply all rego rules
While the help documentation shows that the -p
argument is meant to provide a directory path, the error message when giving an explicit filename is not clear;
$ conftest test -p specific_policy.rego file.yaml
FATA[0000] Problem building rego compiler: readdirent: invalid argument
Would it be possible either accept an explicit policy document as the argument, or make it clearer that the -p
argument expects a directory rather than a file (or other type)
MacOS - 10.13.6
$ conftest --version
Version: 0.6.0
Commit: a27d0739a785fc52c421339d129267772a15662f
Date: 2019-05-20T07:40:07Z
Conftest is silent, even on failing policy, with sprintf
errors. For example:
deny["unable to find a gke container resource"] {
not check_for_container_resources
}
check_for_container_resources {
containers := [
container |
some i
name := input.Resources[i].Name
type := input.Resources[i].Type
startswith(type, "google_container")
container := type
]
count(containers) == 2
}
results in exit 1
package main
deny[sprintf("unable to find a gke container resource %s %s", [input.Resources[i].Type, input.Resources[i].Name])] {
not check_for_container_resources
}
check_for_container_resources {
containers := [
container |
some i
name := input.Resources[i].Name
type := input.Resources[i].Type
startswith(type, "google_container")
container := type
]
count(containers) == 2
}
results in exit 0
To expand the usage of conftest with stdin or unknown file extensions, we need a flag for allowing injection of requested parser type.
After this feature enabled, we will support the commands below:
cat examples/ini/grafana.ini | ./conftest test -p examples/ini/policy/ --flag=ini -
./conftest test -p examples/some/policy/ --flag=toml examples/some/unknown.xyz
conftest should support ability to provide output formatting, which would help to integrate it better with other tools especially editors.
Most common linters provide two options:
<file-path>:<line>:<column>:<type>:<rule-id>:<message>
which allows it to be parsed by other tools, or output format in other standards like JUnit/xUnitRight now conftest:
Useful Links:
https://github.com/golangci/golangci-lint
Pycodestyle https://github.com/PyCQA/pycodestyle#example-usage-and-output
Vim quick formats https://vim.fandom.com/wiki/Errorformats
Yamllint https://yamllint.readthedocs.io/en/stable/quickstart.html#running-yamllint
Checkstyle https://checkstyle.sourceforge.io/
CodeClimate https://docs.codeclimate.com/docs/configuring-test-coverage
Junit format https://github.com/rodrigodiez/go-junit
I would like to test if the containerPort
in the Deployment config and targetPort
in service config are equal. For this scenario, is it possible to compare values between multiple inputs data(both service and deployment manifest are in the same file)
Hi,
could you provide an example with a policy to check Kubernetes recommanded labels ?
Thanks.
Here: https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/#labels
Reproduction steps:
docker run -it --rm -p 5000:5000 registry
conftest push localhost:5000/policies:latest
results:
conftest push localhost:5000/policies:latest
time="2019-08-10T14:19:31Z" level=fatal msg="failed to do request: Head http://localhost:5000/v2/policies/blobs/sha256:086ee78899cc6be63d638028055085630d199198424063098fee3c47383928e4: dial tcp 127.0.0.1:5000: connect: connection refused"
With ORAS it does succeed:
oras push localhost:5000/policies:latest ./policy
Uploading e05080fb451c policy
Pushed localhost:5000/policies:latest
Digest: sha256:28d7f27c0158b6df69ce40176ba9da081c6c49be9c450324e06e80d5bf3251c8
oras pull localhost:5000/policies:latest --output ./tmp
Downloaded e05080fb451c policy
Pulled localhost:5000/policies:latest
Digest: sha256:28d7f27c0158b6df69ce40176ba9da081c6c49be9c450324e06e80d5bf3251c8
First and foremost, thanks for putting this tool together. I saw your talk and got super excited about the possibilities of linting Kubernetes YML with OPA. I am still digging through everything, but I am really concerned about something, and hope I am just missing the whole picture.
Let's just take the really simple example of not allowing Services.
In a CI pipeline, someone is going to submit a PR with just their Kubernetes YML. I would like to use conftest on that YML against my policies. In this case, no services allowed. If that passed, the CI will pass.
Then, using (hopefully) the exact same policy, I'd want to have that policy running on my Kubernetes cluster, denying services.
Unfortunately I'm seeing a lot of tests, packages, etc that seem to treat these very differently. E.g. these tests are quite different than the examples you have here
https://www.openpolicyagent.org/docs/edge/guides-kubernetes-admission-control/
Hey y'all :)
I think a great feature extension to #5 would also be the ability to do this cross-referencing across actual files on a disk, as well. For example, Terraform files in a module. It would be cool to be able to write a policy that allows for comparing forwarding rule resources with their respective load balancers or firewall rules.
To expand the potential places where Conftest could be used, it would be good to support EDN as an input. Conftest currently supports JSON or YAML only as input, but should work with any format we can unmarshall.
This should just be about either using something more generic, or adding some logic around the current use of the YAML module around this line.
Hi,
All moderns CI tools seems to take the "modular path" where each job step is an OCI image and an overrided entrypoint to customize the command.
Personally I like it and I think Conftest, which as a lot of CI use-cases, make it difficult for remote policies.
Today you have to setup a 2 steps job:
conftest pull instrumenta.azurecr.io/test
then conftest test ...
echo "[[policies]]..." > conftest.toml
then conftest test --update ...
oras pull bundle.bar/opa/instrumenta/kubernetes:latest -a
then conftest test -p kubernetes.rego ...
I haven't UX suggestion atm but I think a one-liner like conftest test -p oci://bundle.bar/opa/instrumenta/kubernetes:latest ...
could be a great feature
This example here https://www.openpolicyagent.org/docs/latest/terraform/ shows how to write rego policy to produce some score out of the json file. Can conftest do something similar ?
While looking at Gatekeeper (https://github.com/open-policy-agent/gatekeeper) I noticed their policies allow you to attach structured data to any violation, like this snippet from the Gatekeeper Readme:
violation[{"msg": msg, "details": {"missing_labels": missing}}] {
provided := {label | input.review.object.metadata.labels[label]}
required := {label | label := input.parameters.labels[_]}
missing := required - provided
count(missing) > 0
msg := sprintf("you must provide labels: %v", [missing])
}
I think something like this could potentially be a really powerful addition to conftest, for example, right now when I try to write tests for conftest policies I often end up using something like:
test_some_policy {
deny[msg] with input as {...}
contains(msg, "Some error raised")
}
which feels somewhat fragile as changing the wording of the deny breaks the test. Having the ability to attach arbitrary data allows you to add things like error codes, urls with more information etc.
Of course implementing something like this would be a major change as it'd affect both how you write policies and the format of the output. It could theoretically be made backwards compatible by keeping the signature of warn
and deny
the same and adding a new third field (like for example the violation
from the Gatekeeper example) which has the new signature. It could even have a separate value indicating whether it is a warn
or fail
based on one of the structured outputs, something like:
violation[{"msg": msg, "level": "warn", "details": {}] {
}
The presence of a violation
record could affect the format of the output, or an explicit flag could yield the alternate output format.
Would a PR adding some or all of this functionality be welcome?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.