fabric8-analytics / cvejob Goto Github PK
View Code? Open in Web Editor NEWA tool which tries to map CVEs from NVD to packages in supported ecosystems (Maven, NPM, PyPI).
License: Apache License 2.0
A tool which tries to map CVEs from NVD to packages in supported ecosystems (Maven, NPM, PyPI).
License: Apache License 2.0
Create MAINTAINERS
file with list of active maintainers.
Ideally two maintainers should be provided.
Code coverage so far:
Name Stmts Miss Cover Missing
---------------------------------------------------------------------
cvejob/__init__.py 0 0 100%
cvejob/config.py 52 11 79% 53, 57, 61, 65, 69, 73, 77, 81, 85, 89, 93
cvejob/cpe2pkg.py 39 1 97% 40
cvejob/filters/__init__.py 0 0 100%
cvejob/filters/input.py 128 12 91% 43, 49-51, 58, 83, 164, 177, 181, 195, 202, 206, 221
cvejob/identifiers/__init__.py 6 0 100%
cvejob/identifiers/naive.py 55 4 93% 75-78, 99
cvejob/identifiers/nvdtoolkit.py 19 19 0% 3-45
cvejob/outputs/__init__.py 0 0 100%
cvejob/outputs/victims.py 59 59 0% 3-140
cvejob/selectors/__init__.py 2 0 100%
cvejob/selectors/basic.py 34 3 91% 37, 43, 49
cvejob/utils.py 59 3 95% 28, 39, 46
cvejob/version.py 55 0 100%
cvejob/version_utils.py 170 5 97% 67, 70, 151, 171, 254
cvejob/versions/__init__.py 2 0 100%
cvejob/versions/version_identifier.py 8 0 100%
---------------------------------------------------------------------
TOTAL 688 117 83%
bitbucket, pagure, ...
We use Codecov.io to track code coverage changes. It would be good to document the usage of Codecov.io in the README.md as well.
INFO:cvejob:CVE-2014-7810 found
INFO:cvejob.filters.input:[('NotOlderThanCheck', True), ('NotUnsupportedFileExtensionCheck', True), ('NotUnderAnalysisCheck', True), ('IsSupportedGitHubLanguageCheck', True), ('AffectsApplicationCheck', True), ('NotUnexpectedSiteInReferencesCheck', True)]
INFO:cvejob.utils:product:( tomcat expression tomcat language securitymanager apache el ) AND vendor:( apache expression tomcat language securitymanager apache el )
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Traceback (most recent call last):
File "run.py", line 57, in <module>
run()
File "run.py", line 43, in run
winner = selector.pick_winner()
File "/home/jenkins/workspace/cve-job-maven/cvejob/selectors/basic.py", line 30, in pick_winner
cpe_dicts = self._get_cpe_dicts(self._cve.configurations)
File "/home/jenkins/workspace/cve-job-maven/cvejob/selectors/basic.py", line 83, in _get_cpe_dicts
cpe_dicts.append(self._get_cpe_dicts(node.children))
File "/home/jenkins/workspace/cve-job-maven/cvejob/selectors/basic.py", line 86, in _get_cpe_dicts
if cpe.is_application():
AttributeError: 'str' object has no attribute 'is_application'
We need to create template to be used to:
Please note:
We can use https://github.com/fabric8-analytics/poc-github-templates/blob/master/ISSUE_TEMPLATE.md after it will be discussed and updated.
Line 56 in c151eee
Line 80 in c151eee
WordPress (?)
Line 64 in 10d566e
Line 76 in 10d566e
Line 64 in c151eee
cvejob/cvejob/filters/input.py
Line 49 in b0bf928
Line 60 in 10d566e
We skip vulnerabilities which are still undergoing analysis, like for example: https://nvd.nist.gov/vuln/detail/CVE-2018-1000519
But it can take months sometimes. Should we try to ingest them sooner?
Line 88 in c151eee
Currently we guess up to 10 package name candidates and then try to pick the correct package name based on whether version mentioned in the CVE record exists for given candidate or not. If not, then we discard the candidate and move to the next one.
However, NVD often lists multiple affected/not affected versions in CVE records. All of them should exist for given candidate.
We should be able to improve accuracy by implementing this check as it will help us to filter out false positives and thus we will more likely pick the right package name.
Pre-work: #40
Line 84 in c151eee
The following line needs to be fixed:
Line 65 in b0bf928
Line 56 in 10d566e
We need to create a template to be used for each pull request.
An example of such template:
https://github.com/fabric8-analytics/poc-github-templates/blob/master/PULL_REQUEST_TEMPLATE.md
Line 80 in 10d566e
Line 72 in 10d566e
Line 60 in c151eee
Line 68 in 10d566e
Traceback (most recent call last):
File "run.py", line 57, in <module>
run()
File "run.py", line 43, in run
winner = selector.pick_winner()
File "/home/jenkins/workspace/cve-job-npm/cvejob/selectors/basic.py", line 40, in pick_winner
upstream_versions = self._get_upstream_versions(package)
File "/home/jenkins/workspace/cve-job-npm/cvejob/selectors/basic.py", line 73, in _get_upstream_versions
return get_javascript_versions(package)
File "/home/jenkins/workspace/cve-job-npm/cvejob/utils.py", line 66, in get_javascript_versions
versions = {x for x in response.json().get('versions')}
TypeError: 'NoneType' object is not iterable
cvejob/cvejob/filters/input.py
Line 56 in b0bf928
Check if Python version installed on CI slaves are at least 3.6 or newer.
EPEL repositories now contain proper Python 3.6 packages and at the same moment Python 3.4 is being deprecated [1] [2].
It means that we need to upgrade this repository to use Python 3.6 instead of Python 3.4.
What needs to be changed AND tested:
References:
[1] https://lists.fedoraproject.org/archives/list/[email protected]/thread/EGUMKAIMPK2UD5VSHXM53BH2MBDGDWMO/
[2] https://www.reddit.com/r/CentOS/comments/azetyy/python_34_to_be_deprecated_this_month/
Line 52 in c151eee
There are cases where it's clear that CVE affects a PHP module, for example:
Those should be easy to filter out early in the pipeline.
Line 68 in c151eee
Describe all linters and checkers we use on CI in README.md file
Line 76 in c151eee
Running CVEjob locally is pita. Having Dockerfile would make things much easier.
cvejob/cvejob/filters/input.py
Line 42 in b0bf928
We check if versions mentioned in CVE record exist in upstream repositories. Therefore we don't catch CVEs like https://nvd.nist.gov/vuln/detail/CVE-2017-16057.
Line 52 in 10d566e
Line 72 in c151eee
It seems like there are cases where CVEs don't belong to any currently supported ecosystem, but there is not enough information available to filter them out before we try to map them to packages name from supported ecosystems. And it happens that there is a package with similar name, but since ecosystem is incorrect, such mappings are always false positives.
We could remember which (vendor, product) pairs were marked as false positives by reviewers and automatically filter out new CVEs with the same pair based on feedback from previous reviews.
Marek also had an idea than we could remember (vendor, product) pairs which were previously successfully mapped to a package and next time when we encounter the same pair, we could with high-enough confidence say what the package name is (this should work nicely work ecosystems like Python and Node.JS).
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.