Coder Social home page Coder Social logo

kotnik / kepler Goto Github PK

View Code? Open in Web Editor NEW

This project forked from exein-io/kepler

0.0 1.0 0.0 45.71 MB

NIST-based CVE lookup store and API powered by Rust.

Home Page: https://exein.io/

License: Other

Dockerfile 0.72% Python 5.10% PLpgSQL 1.50% Rust 92.68%

kepler's Introduction


Kepler is a vulnerability database and lookup store and API currently utilising National Vulnerability Database and NPM Advisories as data sources; implementing CPE 2.3 tree expressions and version range evaluation in realtime.


Setup

Docker (recommended)

We provide a docker bundle with kepler, dedicated PostgreSQL database and Ofelia as job scheduler for continuous update

docker compose build
docker compose up

Database migration notes

When the application starts checks for pending database migrations and automatically applies them. Remove the --migrate option to stop when a pending migration is detected

Build from sources

Alternatively you can build kepler from sources. To build you need rust, cargo and libpg-dev (or equivalent PostgreSQL library for your Linux distribution)

cargo build --release

Data sources

The system will automatically fetch and import new records every 3 hours if you use our bundle, while historical data must be imported manually.

Kepler currently supports two data sources, National Vulnerability Database and NPM Advisories. You can import the data sources historically as follows.

NIST Data

To import NIST records from all available years (2002 to 2022):

for year in $(seq 2002 2022); do 
    docker run -v $(pwd)/data:/data \
        -e DATABASE_URL=postgres://kepler:kepler@localhost:5432/kepler \
	--network=kepler_default \
	kepler:dev import_nist $year -d /data; 
done

NPM Data

To import all available NPM records:

docker run -v $(pwd)/data:/data \
    -e DATABASE_URL=postgres://kepler:kepler@localhost:5432/kepler\
    --network=kepler_default \
    kepler:dev import_npm -d /data;

The system will automatically fetch and import new records records every 3 hours.

APIs

There are two primary APIs as of right now — the product API and the cve API detailed below.

Products API

Products can be listed:

curl http://localhost:8000/products

Grouped by vendor:

curl http://localhost:8000/products/by_vendor

Or searched:

curl http://localhost:8000/products/search/iphone

CVEs API

To use the vulnerabilities search API via cURL (prepend node- to the product name in order to search for NPM specific packages):

curl \
    --header "Content-Type: application/json" \
    --request POST \
    --data '{"product":"libxml2","version":"2.9.10"}' \
    http://localhost:8000/cve/search

Responses are cached in memory with a LRU limit of 4096 elements.

Python CLI

To get test and visualize the API results quickly you can use the Python utility wrappers.

pip install -r requirements-cli.txt
./kepler_cli.py linux_kernel $(uname -r)

It is also possible to use the CLI to see the difference in terms of CVEs between two revisions of the same software:

diff -Naur --color <(./kepler_cli.py chrome 93.0.4577.62) <(./kepler_cli.py chrome 93.0.4577.63)
JSON response for query: {"product":"libxml2","version":"2.9.10"} 
```json
[
    {
        "source": "NIST",
        "vendor": "xmlsoft",
        "product": "libxml2",
        "cve": "CVE-2021-3517",
        "summary": "There is a flaw in the xml entity encoding functionality of libxml2 in versions before 2.9.11. An attacker who is able to supply a crafted file to be processed by an application linked with the affected functionality of libxml2 could trigger an out-of-bounds read. The most likely impact of this flaw is to application availability, with some potential impact to confidentiality and integrity if an attacker is able to use memory information to further exploit the application.",
        "score": 7.5,
        "severity": "HIGH",
        "vector": "NETWORK",
        "references": [
            {
                "url": "https://bugzilla.redhat.com/show_bug.cgi?id=1954232",
                "tags": [
                    "Issue Tracking",
                    "Patch",
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://lists.fedoraproject.org/archives/list/[email protected]/message/QVM4UJ3376I6ZVOYMHBNX4GY3NIV52WV/",
                "tags": [
                    "Mailing List",
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://lists.debian.org/debian-lts-announce/2021/05/msg00008.html",
                "tags": [
                    "Mailing List",
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://lists.fedoraproject.org/archives/list/[email protected]/message/BZOMV5J4PMZAORVT64BKLV6YIZAFDGX6/",
                "tags": []
            },
            {
                "url": "https://security.netapp.com/advisory/ntap-20210625-0002/",
                "tags": []
            },
            {
                "url": "https://lists.apache.org/thread.html/rf4c02775860db415b4955778a131c2795223f61cb8c6a450893651e4@%3Cissues.bookkeeper.apache.org%3E",
                "tags": []
            },
            {
                "url": "https://lists.apache.org/thread.html/r58af02e294bd07f487e2c64ffc0a29b837db5600e33b6e698b9d696b@%3Cissues.bookkeeper.apache.org%3E",
                "tags": []
            },
            {
                "url": "https://security.gentoo.org/glsa/202107-05",
                "tags": []
            }
        ]
    },
    {
        "source": "NIST",
        "vendor": "xmlsoft",
        "product": "libxml2",
        "cve": "CVE-2021-3518",
        "summary": "There's a flaw in libxml2 in versions before 2.9.11. An attacker who is able to submit a crafted file to be processed by an application linked with libxml2 could trigger a use-after-free. The greatest impact from this flaw is to confidentiality, integrity, and availability.",
        "score": 6.8,
        "severity": "MEDIUM",
        "vector": "NETWORK",
        "references": [
            {
                "url": "https://lists.fedoraproject.org/archives/list/[email protected]/message/QVM4UJ3376I6ZVOYMHBNX4GY3NIV52WV/",
                "tags": [
                    "Mailing List",
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://lists.debian.org/debian-lts-announce/2021/05/msg00008.html",
                "tags": [
                    "Mailing List",
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://bugzilla.redhat.com/show_bug.cgi?id=1954242",
                "tags": [
                    "Issue Tracking",
                    "Patch",
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://lists.fedoraproject.org/archives/list/[email protected]/message/BZOMV5J4PMZAORVT64BKLV6YIZAFDGX6/",
                "tags": []
            },
            {
                "url": "https://security.netapp.com/advisory/ntap-20210625-0002/",
                "tags": []
            },
            {
                "url": "https://lists.apache.org/thread.html/rf4c02775860db415b4955778a131c2795223f61cb8c6a450893651e4@%3Cissues.bookkeeper.apache.org%3E",
                "tags": []
            },
            {
                "url": "https://lists.apache.org/thread.html/r58af02e294bd07f487e2c64ffc0a29b837db5600e33b6e698b9d696b@%3Cissues.bookkeeper.apache.org%3E",
                "tags": []
            },
            {
                "url": "https://security.gentoo.org/glsa/202107-05",
                "tags": []
            },
            {
                "url": "https://support.apple.com/kb/HT212604",
                "tags": []
            },
            {
                "url": "https://support.apple.com/kb/HT212605",
                "tags": []
            },
            {
                "url": "https://support.apple.com/kb/HT212602",
                "tags": []
            },
            {
                "url": "https://support.apple.com/kb/HT212601",
                "tags": []
            },
            {
                "url": "http://seclists.org/fulldisclosure/2021/Jul/55",
                "tags": []
            },
            {
                "url": "http://seclists.org/fulldisclosure/2021/Jul/54",
                "tags": []
            },
            {
                "url": "http://seclists.org/fulldisclosure/2021/Jul/58",
                "tags": []
            },
            {
                "url": "http://seclists.org/fulldisclosure/2021/Jul/59",
                "tags": []
            }
        ]
    },
    {
        "source": "NIST",
        "vendor": "xmlsoft",
        "product": "libxml2",
        "cve": "CVE-2021-3537",
        "summary": "A vulnerability found in libxml2 in versions before 2.9.11 shows that it did not propagate errors while parsing XML mixed content, causing a NULL dereference. If an untrusted XML document was parsed in recovery mode and post-validated, the flaw could be used to crash the application. The highest threat from this vulnerability is to system availability.",
        "score": 4.3,
        "severity": "MEDIUM",
        "vector": "NETWORK",
        "references": [
            {
                "url": "https://bugzilla.redhat.com/show_bug.cgi?id=1956522",
                "tags": [
                    "Issue Tracking",
                    "Patch",
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://lists.fedoraproject.org/archives/list/[email protected]/message/QVM4UJ3376I6ZVOYMHBNX4GY3NIV52WV/",
                "tags": [
                    "Mailing List",
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://lists.debian.org/debian-lts-announce/2021/05/msg00008.html",
                "tags": [
                    "Mailing List",
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://lists.fedoraproject.org/archives/list/[email protected]/message/BZOMV5J4PMZAORVT64BKLV6YIZAFDGX6/",
                "tags": []
            },
            {
                "url": "https://security.netapp.com/advisory/ntap-20210625-0002/",
                "tags": []
            },
            {
                "url": "https://security.gentoo.org/glsa/202107-05",
                "tags": []
            }
        ]
    },
    {
        "source": "NIST",
        "vendor": "xmlsoft",
        "product": "libxml2",
        "cve": "CVE-2021-3541",
        "summary": "A flaw was found in libxml2. Exponential entity expansion attack its possible bypassing all existing protection mechanisms and leading to denial of service.",
        "score": 4.0,
        "severity": "MEDIUM",
        "vector": "NETWORK",
        "references": [
            {
                "url": "https://bugzilla.redhat.com/show_bug.cgi?id=1950515",
                "tags": [
                    "Issue Tracking",
                    "Patch",
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://security.netapp.com/advisory/ntap-20210805-0007/",
                "tags": []
            }
        ]
    },
    {
        "source": "NIST",
        "vendor": "xmlsoft",
        "product": "libxml2",
        "cve": "CVE-2019-20388",
        "summary": "xmlSchemaPreRun in xmlschemas.c in libxml2 2.9.10 allows an xmlSchemaValidateStream memory leak.",
        "score": 5.0,
        "severity": "MEDIUM",
        "vector": "NETWORK",
        "references": [
            {
                "url": "https://gitlab.gnome.org/GNOME/libxml2/merge_requests/68",
                "tags": [
                    "Patch",
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://lists.fedoraproject.org/archives/list/[email protected]/message/545SPOI3ZPPNPX4TFRIVE4JVRTJRKULL/",
                "tags": []
            },
            {
                "url": "https://lists.fedoraproject.org/archives/list/[email protected]/message/5R55ZR52RMBX24TQTWHCIWKJVRV6YAWI/",
                "tags": []
            },
            {
                "url": "https://lists.fedoraproject.org/archives/list/[email protected]/message/JDPF3AAVKUAKDYFMFKSIQSVVS3EEFPQH/",
                "tags": []
            },
            {
                "url": "http://lists.opensuse.org/opensuse-security-announce/2020-05/msg00047.html",
                "tags": []
            },
            {
                "url": "https://security.netapp.com/advisory/ntap-20200702-0005/",
                "tags": []
            },
            {
                "url": "https://www.oracle.com/security-alerts/cpujul2020.html",
                "tags": []
            },
            {
                "url": "https://lists.debian.org/debian-lts-announce/2020/09/msg00009.html",
                "tags": []
            },
            {
                "url": "https://security.gentoo.org/glsa/202010-04",
                "tags": []
            }
        ]
    },
    {
        "source": "NIST",
        "vendor": "xmlsoft",
        "product": "libxml2",
        "cve": "CVE-2020-24977",
        "summary": "GNOME project libxml2 v2.9.10 has a global buffer over-read vulnerability in xmlEncodeEntitiesInternal at libxml2/entities.c. The issue has been fixed in commit 50f06b3e.",
        "score": 6.4,
        "severity": "MEDIUM",
        "vector": "NETWORK",
        "references": [
            {
                "url": "https://gitlab.gnome.org/GNOME/libxml2/-/issues/178",
                "tags": [
                    "Exploit",
                    "Patch",
                    "Vendor Advisory"
                ]
            },
            {
                "url": "https://lists.debian.org/debian-lts-announce/2020/09/msg00009.html",
                "tags": [
                    "Third Party Advisory"
                ]
            },
            {
                "url": "http://lists.opensuse.org/opensuse-security-announce/2020-09/msg00036.html",
                "tags": [
                    "Mailing List",
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://lists.fedoraproject.org/archives/list/[email protected]/message/2NQ5GTDYOVH26PBCPYXXMGW5ZZXWMGZC/",
                "tags": [
                    "Third Party Advisory"
                ]
            },
            {
                "url": "http://lists.opensuse.org/opensuse-security-announce/2020-09/msg00061.html",
                "tags": [
                    "Mailing List",
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://lists.fedoraproject.org/archives/list/[email protected]/message/O7MEWYKIKMV2SKMGH4IDWVU3ZGJXBCPQ/",
                "tags": [
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://lists.fedoraproject.org/archives/list/[email protected]/message/H3IQ7OQXBKWD3YP7HO6KCNOMLE5ZO2IR/",
                "tags": [
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://security.netapp.com/advisory/ntap-20200924-0001/",
                "tags": [
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://lists.fedoraproject.org/archives/list/[email protected]/message/7KQXOHIE3MNY3VQXEN7LDQUJNIHOVHAW/",
                "tags": [
                    "Mailing List",
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://lists.fedoraproject.org/archives/list/[email protected]/message/JCHXIWR5DHYO3RSO7RAHEC6VJKXD2EH2/",
                "tags": [
                    "Mailing List",
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://lists.fedoraproject.org/archives/list/[email protected]/message/J3ICASXZI2UQYFJAOQWHSTNWGED3VXOE/",
                "tags": [
                    "Mailing List",
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://gitlab.gnome.org/GNOME/libxml2/-/commit/50f06b3efb638efb0abd95dc62dca05ae67882c2",
                "tags": [
                    "Patch",
                    "Vendor Advisory"
                ]
            },
            {
                "url": "https://lists.fedoraproject.org/archives/list/[email protected]/message/ENEHQIBMSI6TZVS35Y6I4FCTYUQDLJVP/",
                "tags": [
                    "Mailing List",
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://lists.fedoraproject.org/archives/list/[email protected]/message/RIQAMBA2IJUTQG5VOP5LZVIZRNCKXHEQ/",
                "tags": [
                    "Mailing List",
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://lists.fedoraproject.org/archives/list/[email protected]/message/5KTUAGDLEHTH6HU66HBFAFTSQ3OKRAN3/",
                "tags": [
                    "Mailing List",
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://lists.fedoraproject.org/archives/list/[email protected]/message/674LQPJO2P2XTBTREFR5LOZMBTZ4PZAY/",
                "tags": [
                    "Mailing List",
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://lists.apache.org/thread.html/rf9fa47ab66495c78bb4120b0754dd9531ca2ff0430f6685ac9b07772@%3Cdev.mina.apache.org%3E",
                "tags": [
                    "Mailing List",
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://security.gentoo.org/glsa/202107-05",
                "tags": []
            }
        ]
    },
    {
        "source": "NIST",
        "vendor": "xmlsoft",
        "product": "libxml2",
        "cve": "CVE-2020-7595",
        "summary": "xmlStringLenDecodeEntities in parser.c in libxml2 2.9.10 has an infinite loop in a certain end-of-file situation.",
        "score": 5.0,
        "severity": "MEDIUM",
        "vector": "NETWORK",
        "references": [
            {
                "url": "https://gitlab.gnome.org/GNOME/libxml2/commit/0e1a49c89076",
                "tags": [
                    "Patch",
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://lists.fedoraproject.org/archives/list/[email protected]/message/545SPOI3ZPPNPX4TFRIVE4JVRTJRKULL/",
                "tags": [
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://usn.ubuntu.com/4274-1/",
                "tags": [
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://lists.fedoraproject.org/archives/list/[email protected]/message/5R55ZR52RMBX24TQTWHCIWKJVRV6YAWI/",
                "tags": [
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://lists.fedoraproject.org/archives/list/[email protected]/message/JDPF3AAVKUAKDYFMFKSIQSVVS3EEFPQH/",
                "tags": [
                    "Third Party Advisory"
                ]
            },
            {
                "url": "http://lists.opensuse.org/opensuse-security-announce/2020-05/msg00047.html",
                "tags": [
                    "Broken Link"
                ]
            },
            {
                "url": "https://security.netapp.com/advisory/ntap-20200702-0005/",
                "tags": [
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://www.oracle.com/security-alerts/cpujul2020.html",
                "tags": [
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://lists.debian.org/debian-lts-announce/2020/09/msg00009.html",
                "tags": [
                    "Mailing List",
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://security.gentoo.org/glsa/202010-04",
                "tags": [
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://cert-portal.siemens.com/productcert/pdf/ssa-292794.pdf",
                "tags": [
                    "Third Party Advisory"
                ]
            },
            {
                "url": "https://us-cert.cisa.gov/ics/advisories/icsa-21-103-08",
                "tags": [
                    "Third Party Advisory",
                    "US Government Resource"
                ]
            }
        ]
    }
]
```

kepler's People

Contributors

banditopazzo avatar juxhindb avatar

Watchers

avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.