anchore / engine-db-preload

Some scripting to handling creation of preloaded anchore DB container

This repository is deprecated and no longer maintained.

If you're looking for a host-local container vulnerability scanner see our new projects:

Software Bill of Materials for Containers: Syft

Container Vulnerability Scanning: Grype

engine-db-preload's Issues

Q: Could I use this for airgap installs?

I'm curious if this set of scripts could be used to preload a DB in an internet connected environment that then dumps a container that is then moved into an air gap environment. This would be really convenient because the process of replacing a container is well defined where the process of regularly running a pg_dump/pg_restore between two somewhat isolated networks is much less defined.

Container updates would be something that would actually drop into our update automation process very easily.

Am I correctly understanding what this repo is doing to suggest this is a related task?

Improve db-preload build process

The current build process is prone to failure due to insufficient resources provided on CircleCi for large NVDv2 updates.

Feed script sub-commands print error message but return success

In using this tool (thank you it has been quite useful), I ran into a minor issue running 'feed_sync_wait.py 300 60":

c:\jdev\newpaas\src\main\resources\scripts\anchore-db-preload>scripts\feed_sync_wait.py 300 60
Starting Anchore feed sync
        Timeout: 300 minutes
        Sync Interval: 60.0
        Slim Build: False
got container IDs: engine=d07937d10360ba80dd5747e244124db5968c5f885ac5d02523187b4eaf6e5703 db=8f8a96629e04d4e4b63530cf7c709ce0ff4d33c7ae6a9c27aa7b8d9b767bb13d
failed to execute cmd: anchore-cli --u admin --p foobar --url http://localhost:8228/v1 system wait --timeout 18000 --interval 60.0 --feedsready "". Error - [Error 2] The system cannot find the file specified
verified that anchore-engine is up and ready
12/06/21_13:57:58 - 0 / 0 groups completed
        synced: []
        unsynced: []
...
12/06/21_15:49:04 - 73 / 74 groups completed
        synced: [...]
        unsynced: [...]

failed to execute cmd: anchore-cli --u admin --p foobar --url http://localhost:8228/v1 system wait --timeout 18000 --interval 60.0 --feedsready vulnerabilities. Error - [Error 2] The system cannot find the file specified
verified feed sync has completed
CMD: ['docker-compose', 'stop', 'anchore-engine']
Stopping anchore-engine ... done

So I went about figuring out why everything seemed to work - were the errors that were reported ("failed to execute cmd: anchore-cli...") this something I need to care about? I'm not sure that I do.

The problem was caused by the fact that I did not have Python3 module anchorecli installed and in my PATH - pip install anchorecli and adding scripts to my path cleared that up. But I still ask the following:

1. Please consider mentioning in the README that that anchorecli pip module must be installed and the Python scripts folder must be in your path. That may be obvious to Python wizards but I'm certainly not one of those. (I ran with Python 3, and despite the reported errors from not having the module, I appear to have a valid init script.)

2. The subroutines in feed_sync_wait.py "wait_for_feed_sync" (line 194) and "verify_anchore_engine_available" (line 203) printed an error message but still returned success, so execution continued even with errors in my output stream. In my case that appears to have been fortuitous. I think I got the right thing even without the anchore-cli module.
   a) If the errors from these functions are not fatal, please consider leading the messages in lines 200 and 209 with "WARNING:".
   b) If the errors were intended to be fatal, after the messages are printed there should perhaps be "return(False)" within the exception blocks?
   I suspect the expected behavior would be to fail immediately, even though in my use case I don't see any ill effects from these command failures - that might not be so in all or most other failure situations.

vulnerability feed data not available using docker image anchore/engine-db-preload:v0.10.1 in closed network

Hi,

Not sure if this is the right spot for this ticket, if its not point me in the right direction. I am trying to use the dockerhub image anchore/engine-db-preload:v0.10.1 with anchore-engine v0.10.1 OSS in a closed network (no internet) to perform a vulnerability scan.

I updated the postgres image name in our helm chart and confirmed the version is deployed. I also disabled the feed sync and confirmed the initial feed sync isn't running on the policy engine pod.

When I query the api for v1/system/feeds I get back an [] which I am assuming means there is no feed data. When using this preloaded image should this endpoint give me the feeds as of the last time the image was build which was ~8 hours ago? Any suggestions would be appreciated.

The only error I am seeing in the policy engine log is [ERROR] could no fetch CPE matches and an exception in policy_engine/engine/vulns/db.py line 181 image_cpe==cpe_cls.product AttributeError: type object 'CPEVulnerability' has no attribute 'product' I would copy and paste the full error but getting stuff off this network is a pain. not sure if that error has anything to do with no seeing the feed data. When I perform the scan I am getting Distro-specific feed data not found for distro namespace:Alpine:3.12.4. Cannot perform CVE Scan OS/distro packages

Include anchore_ci_tools.py ?

Is possible to include https://raw.githubusercontent.com/anchore/ci-tools/master/scripts/anchore_ci_tools.py into image?

There will be better usage of this container in CI. For example:
https://gitlab.com/anchore/gitlab-demo/blob/master/.gitlab-ci.yml