jeremylong / dependencycheck Goto Github PK

One of the NIST updates seems to be causing a number format exception

I am using version 1.0.4

INFO: Processing http://static.nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-modified.xml
Exception in thread "main" java.lang.NumberFormatException: For input string: "2013418100420"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:495)
at java.lang.Integer.parseInt(Integer.java:527)
at org.owasp.dependencycheck.dependency.VulnerableSoftware.compareTo(VulnerableSoftware.java:188)
at org.owasp.dependencycheck.dependency.VulnerableSoftware.compareTo(VulnerableSoftware.java:34)
at java.util.TreeMap.compare(TreeMap.java:1188)
at java.util.TreeMap.put(TreeMap.java:531)
at java.util.TreeSet.add(TreeSet.java:255)
at org.owasp.dependencycheck.dependency.Vulnerability.updateVulnerableSoftware(Vulnerability.java:187)
at org.owasp.dependencycheck.dependency.Vulnerability.addVulnerableSoftware(Vulnerability.java:174)
at org.owasp.dependencycheck.dependency.Vulnerability.addVulnerableSoftware(Vulnerability.java:157)
at org.owasp.dependencycheck.data.nvdcve.NvdCve20Handler.endElement(NvdCve20Handler.java:197)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1789)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2965)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:649)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(SAXParserImpl.java:333)
at javax.xml.parsers.SAXParser.parse(SAXParser.java:328)
at org.owasp.dependencycheck.data.update.AbstractUpdateTask.importXML(AbstractUpdateTask.java:286)
at org.owasp.dependencycheck.data.update.StandardUpdateTask.update(StandardUpdateTask.java:110)
at org.owasp.dependencycheck.data.update.DatabaseUpdater.update(DatabaseUpdater.java:77)
at org.owasp.dependencycheck.Engine.doUpdates(Engine.java:353)
at org.owasp.dependencycheck.Engine.(Engine.java:77)
at org.owasp.dependencycheck.App.runScan(App.java:119)
at org.owasp.dependencycheck.App.run(App.java:102)
at org.owasp.dependencycheck.App.main(App.java:71)

Due to a copy paste issue, setting a Nexus URL over-writes the connection string.

Hi,

I get the following SAXParseException while running the Maven plugin (via Jenkins). Both have the version 1.0.2:

18:30:20 [DependencyCheck] Parsing of file /var/lib/jenkins/jobs/build_project/workspace/target/DependencyCheck-Report.xml failed due to an exception:
18:30:20 
18:30:20 org.xml.sax.SAXParseException; lineNumber: 8262; columnNumber: 40; The entity "eacute" was referenced, but not declared.
18:30:20    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1236)
18:30:20    at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:628)
18:30:20    at org.apache.commons.digester.Digester.parse(Digester.java:1916)
18:30:20    at org.jenkinsci.plugins.DependencyCheck.parser.ReportParser.parse(ReportParser.java:97)
18:30:20    at hudson.plugins.analysis.core.AbstractAnnotationParser.parse(AbstractAnnotationParser.java:53)
18:30:20    at hudson.plugins.analysis.core.FilesParser.parseFile(FilesParser.java:306)
18:30:20    at hudson.plugins.analysis.core.FilesParser.parseFiles(FilesParser.java:264)
18:30:20    at hudson.plugins.analysis.core.FilesParser.parserCollectionOfFiles(FilesParser.java:215)
18:30:20    at hudson.plugins.analysis.core.FilesParser.invoke(FilesParser.java:184)
18:30:20    at hudson.plugins.analysis.core.FilesParser.invoke(FilesParser.java:31)
18:30:20    at hudson.FilePath.act(FilePath.java:905)
18:30:20    at hudson.FilePath.act(FilePath.java:878)
18:30:20    at org.jenkinsci.plugins.DependencyCheck.DependencyCheckPublisher.perform(DependencyCheckPublisher.java:128)
18:30:20    at hudson.plugins.analysis.core.HealthAwarePublisher.perform(HealthAwarePublisher.java:144)
18:30:20    at hudson.plugins.analysis.core.HealthAwareRecorder.perform(HealthAwareRecorder.java:333)
18:30:20    at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
18:30:20    at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:802)
18:30:20    at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:774)
18:30:20    at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.post2(MavenModuleSetBuild.java:974)
18:30:20    at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:724)
18:30:20    at hudson.model.Run.execute(Run.java:1617)
18:30:20    at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:499)
18:30:20    at hudson.model.ResourceController.execute(ResourceController.java:88)
18:30:20    at hudson.model.Executor.run(Executor.java:237)
18:30:20 
18:30:20 [DependencyCheck] Computing warning deltas based on reference build #466

The line where the exception occurs looks like (excerpt from the DependencyCheck-Report.xml):

<evidence>
    <source>Manifest</source>
    <name>Bundle-Vendor</name>
    <value>Andr&eacute; Rou&eacute;l</value>
</evidence>

The "problematic" line is the content of the value tag. The evidence is part of the UADetector library by André Rouél. So the parsing seems to have issues with (french) special characters...

I think the problem is, that escaping é to é is valid in HTML, but not valid in XML. Therefore, the SAX parser fails. So while creating the DependencyCheck-Report.xml file, an é should probably be encoded as é. See: numeric character reference...

after fixing issue #11 we need to look at improving the dependency-bundling analyzer to bundle an externalized "pom" back into the main jar if the pom came from the JAR and they have the same CPE identifiers.

Hello! Any idea on why I'm getting this issue?

Basically I cloned the repo and ran:

"C:\DependencyCheck\dependency-check-cli\target\release\bin\dependency-check.bat" -f ALL -a MY-PROJECT -s "C:\jenkins\workspace\MY-PROJECT" -o "C:\jenkins\workspace\MY-PROJECT"

The log:

Dez 16, 2013 11:27:56 AM org.owasp.dependencycheck.data.update.StandardUpdate up
date
INFO: NVD CVE requires several updates; this could take a couple of minutes.
Dez 16, 2013 11:27:56 AM org.owasp.dependencycheck.data.update.CallableDownloadT
ask call
INFO: Download Started for NVD CVE - 2002
Dez 16, 2013 11:27:56 AM org.owasp.dependencycheck.data.update.CallableDownloadT
ask call

[...]

(everything ok until here)

INFO: Processing Complete for NVD CVE - 2013

Exception in thread "main" java.lang.SecurityException: Invalid signature file d
igest for Manifest main attributes
at sun.security.util.SignatureFileVerifier.processImpl(SignatureFileVeri
fier.java:240)
at sun.security.util.SignatureFileVerifier.process(SignatureFileVerifier
.java:193)
at java.util.jar.JarVerifier.processEntry(JarVerifier.java:305)
at java.util.jar.JarVerifier.update(JarVerifier.java:216)
at java.util.jar.JarFile.initializeVerifier(JarFile.java:345)
at java.util.jar.JarFile.getInputStream(JarFile.java:412)
at org.owasp.dependencycheck.analyzer.JarAnalyzer.retrievePom(JarAnalyze
r.java:365)
at org.owasp.dependencycheck.analyzer.JarAnalyzer.analyzePOM(JarAnalyzer
.java:283)
at org.owasp.dependencycheck.analyzer.JarAnalyzer.analyze(JarAnalyzer.ja
va:225)
at org.owasp.dependencycheck.Engine.analyzeDependencies(Engine.java:336)

    at org.owasp.dependencycheck.App.runScan(App.java:126)
    at org.owasp.dependencycheck.App.run(App.java:103)
    at org.owasp.dependencycheck.App.main(App.java:71)

My java version:

C:\DependencyCheck> java -version
java version "1.7.0_45"
Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
Java HotSpot(TM) Client VM (build 24.45-b08, mixed mode, sharing)

Let me know if you have something in mind...

Thank you in advance,

• Fernando

Need to create a batch update mode so that the entire data archive can be downloaded from a URL specified rather then always pulling from the NVD.

package names of 1-2 characters should not be included as evidence.

like other false positives with similar protocol vendor/product CPEs ftp:ftp should be considered a false positive for JAR files.

The connection string and driver class name should be configurable options to allow an enterprise deployment to utilize a single central database instance.

The Spring Framework is bad about including vendor information in their manifest. There are a few "a priori" items of evidence added to aid in the detection of the Spring Framework. This is currently in the JAR Analyzer and should be moved to a PRE_IDENTIFIER_ANALYSIS Analyzer.

Additionally, the SpringCleaner Analyzer should be cleaned up to use regular expressions rather then the two hard-coded partial CPE Strings.

in the manifest, the specification-version should have a lower confidence. This is causing false positives in some JAR files.

Add an analyzer which will look up an artifact in an Artifactory instance by SHA-1.

This may present challenges as you can only search by checksum in the Pro version of Artifactory. You can get a 30-day eval license of it to test with, but if the API changes outside of that window, modifying the analyzer will be difficult. Somebody from the community is looking to see if we can possibly get a longer-lived license to test with.

It would be nice to add confidence scores to the matched CVE/CPE. This way, if one so chooses you can ignore anything that is a "low" quality match.

1-2, 1-2...this is just...a..test.

A tar file, provided by Steve, cannot be decompressed by dc's archive analyzer.

Currently, if more then one process attempts to access the data stores - one will fail due to the update functionality. The update opens a Lucene IndexWriter and this requires exclusive access.

Currently, uber-jars (also called a fat jar) are handled poorly and will result in a high false positive rate. This is because the evidence from each POM is bundled into a single EvidenceCollection (well, three actually: vendor, product, version). Because of this bundling a version from one pom may cause an older version of another pom to be identified.

These uber jars can be built using tools such as:

• Maven-Shade-Plugin: http://maven.apache.org/plugins/maven-shade-plugin/
• One-JAR: http://one-jar.sourceforge.net/
• etc.

There should be a way to recover from a source website (such as NIST) being down or the content being unavailable. At the time of this writing, http://static.nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-modified.xml returns a 503, so DependencyCheck fails. However, I think it would be good to attempt to use cached information (even if its not up to date) in an attempt to gracefully fail from this scenario. I would not want a build machine running a DependencyCheck job failing because of this.

Perhaps DatabaseUpdater.java:134 should be changed to not throw an UpdateException, but rather some other type of exception which, if DependencyCheck could recover from, only display a warning upon a failure and perhaps exit with a specific exit code.

Hi,

The scan of org.apache.ws.security:wss4j:jar:1.5.7 detects the vulnerability CVE-2012-5784.

I think it is a false positive:
During the information collection, the scanner detects that:

• there is an axis package name in the lib
• the manifest Bundle-version is 1.0.0 (don't sure that it is important in CPE search)

The project identifiers found are (wss4j is not present in CPE):

• cpe: cpe:/a:apache:axis:1.0
• cpe: cpe:/a:apache:xml_security_for_java:1.5.7
  and the CVE-2012-5784 is attached to cpe:/a:apache:axis:1.0

I've used the 1.0.5 version (and checked with current development version).

By the way, it is a nice project. Thanks.

The Nexus Analyzer configuration needs to be exposed to Maven, ANT, and CLI.

While I originally thought this was a good idea; its not. We need to remove the overall analysis exceptions from the dependencies and the resulting report. If there is an exception that should be reported we might put an overall message about exceptions have occurred, pleased see the log for more details.

The actual exceptions should be removed from the XML (and XSD) and HTML reports.

A bug has been reported where linux and mac users can't create the H2 database on first use in 1.0.8. This is likely due to the changes around how the H2 driver is loaded.

If there are any invalid zip, it crashes. I would prefer it ignored them with a warning

Exception in thread "main" java.lang.IllegalArgumentException: MALFORMED at java.util.zip.ZipCoder.toString(ZipCoder.java:58) at java.util.zip.ZipInputStream.readLOC(ZipInputStream.java:297) at java.util.zip.ZipInputStream.getNextEntry(ZipInputStream.java:121) at org.owasp.dependencycheck.analyzer.ArchiveAnalyzer.extractFiles(ArchiveAnalyzer.java:257) at org.owasp.dependencycheck.analyzer.ArchiveAnalyzer.analyze(ArchiveAnalyzer.java:180) at org.owasp.dependencycheck.Engine.analyzeDependencies(Engine.java:313) at org.owasp.dependencycheck.App.runScan(App.java:146) at org.owasp.dependencycheck.App.run(App.java:123) at org.owasp.dependencycheck.App.main(App.java:72)

There are times when a single JAR file may exist in multiple directories. The scan currently includes this JAR file multiple times as it exists in multiple locations. Going forward we should collapse these instances into a single dependency by doing the following:

1. Use the hash values of the dependency to determine if it is a duplicate.
2. Remove duplicates and add the removed instances file path to a collection of file paths in the remaining dependency.

Need to add aggregation capabilities to the Maven plugin.

hazelcast-2.5.jar is reported as vulnerable. It looks like a false positive given the exploit as a Joomla! and Mambo exploit:

CVE-2008-0816

Severity: High 
CVSS Score: 7.5 
CWE: CWE-89 Improper Neutralization of Special Elements used in an SQL Command ('SQL Injection')

SQL injection vulnerability in the com_sg component for Joomla! and Mambo allows remote attackers to execute arbitrary SQL commands via the pid parameter in an order task.

BID - 27821
BUGTRAQ - 20080215 joomla SQL Injection(com_sg)
SREASON - 3664
Vulnerably Software:

cpe:/a:com_sg:com_sg

We would like to scan a zip file which wraps a tar.gz file. inside it there is tomcat, and wars in it.

testAnalyze of org.codesecure.dependencycheck.analyzer.JarAnalyzerTest line 75 uses equals for string comparison which fails the "implementation-url" comparison. I've changed that to equalsIgnoreCase and compiled successtully.

        if (e.getName().equalsIgnoreCase("implementation-url") && e.getValue().equals("http://jetty.mortbay.org")) {
            found = true;
            break;
        }

CVE/CPE fetch for NIST are pretty huge and could take long to be downloaded.
For security and reproductibility reasons, we'd like to use an internal mirror.

Questions

• Is it possible to mirror using curl/wget/rsync, NIST CVE/CPE contents ?
• If so, would it be possible to told Dependency Check and its Jenkins Plugin to use local HTTP mirror ?

Thanks

in readme.md, link to Maven Plugin, http://jeremylong.github.io/DependencyCheck/dependency-check-maven/installation.html, is broken ;(

Added the plugin to maven build. Run mvn package, the build hangs at the dependency-check plugin and doesn't proceed (Waited 30 minutes).

mvn -X output:

[DEBUG] Configuring mojo org.owasp:dependency-check-maven:1.0.2:check from plugin realm ClassRealm[plugin>org.owasp:dependency-check-maven:1.0.2, parent: sun.misc.Launcher$AppClassLoader@affc70]
[DEBUG] Configuring mojo 'org.owasp:dependency-check-maven:1.0.2:check' with basic configurator -->
DEBUG autoUpdate = true
DEBUG description = A report providing details on any published vulnerabilities within project dependencies. This report is a best effort but may contain false positives and false negatives.
DEBUG externalReport = false
DEBUG failBuildOnCVSS = 8.0
DEBUG format = HTML
DEBUG name = Dependency-Check
DEBUG outputDirectory = /home/training/spring-secure-sample/target
DEBUG reportName = dependency-check-report
DEBUG reportOutputDirectory = /home/training/spring-secure-sample/target/site
DEBUG project = MavenProject: net.continuumsecurity:spring-secure-sample:1.0-SNAPSHOT @ /home/training/spring-secure-sample/pom.xml
[DEBUG] -- end configuration --

I'm running the check as:

mvn org.owasp:dependency-check-maven:1.1.1:check -DfailBuildOnCVSS=7

The resulting failure doesn't inform about the "root"-cause, other modules does inform about their causes of concerns, that feature would be great even for the failing module

We had discussion about CVE/CPE storage size on mailing list.
I did a fresh build in Jenkins, using DC Jenkins Plugin 1.0.3, cleaning workspace and noticed dependency-check-data database is still huge, more than 200MB.

Is it a normal behaviour ?

227M    dependency-check-data/cve
 1,6M   dependency-check-data/cpe  
 228M   dependency-check-data/

This could be a show stopper for us as we have tons of jobs running on many Jenkins slaves, some created dynamically and with limited storage allocated

Add the ability to detect if newer versions of a given dependency are available. For Java libraries, this may be possible using technology included in the Versions Maven Plugin.

Add an optional evidence-based analysis step to collect and analyze copyright, build, file stamp and other dates to determine when the library was published. Add a configurable threshold that would flag the library as 'old' if published prior to this date.

Add an optional and separate report that would output all dependencies that have newer versions and if possible, state what the current version is as well as the closest minor version not containing known vulnerabilities. Include another section containing possible out-of-date libraries.

I've been trying to compile DependencyCheck with Maven 3.0.4, Java 1.6.0_27 and got into errors like: generics are not supported, for-each loops are not supported, ...

To fix, please add the following lines within <properties></properties> in pom.xml

<maven.compiler.target>1.6</maven.compiler.target>
<maven.compiler.source>1.6</maven.compiler.source>

Thanks. Keep up the good work!

Cheers,
Luca

According to the below CVE entry spring-beans-2.5.5.jar has a security vulnerability which I couldn't get DependencyCheck to report. It might be because of CVE updates don't include it. I'm not sure, is it an expected result?

http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2010-1622

I checked our the source yesterday (0.3.2.5-SNAPSHOT) and when running through the step : java -jar dependency-check-0.3.2.5-SNAPSHOT.jar -a Testing -out . -scan ./test-classes -scan ./lib

Then I get :
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -1 at org.owasp.dependencycheck.utils.UrlStringUtils.extractImportantUrlData(UrlStringUtils.java:98) at org.owasp.dependencycheck.dependency.EvidenceCollection.urlCorrection(EvidenceCollection.java:300) at org.owasp.dependencycheck.dependency.EvidenceCollection.containsUsedString(EvidenceCollection.java:193) at org.owasp.dependencycheck.data.cpe.CPEAnalyzer.collectionContainsString(CPEAnalyzer.java:443) at org.owasp.dependencycheck.data.cpe.CPEAnalyzer.verifyEntry(CPEAnalyzer.java:424) at org.owasp.dependencycheck.data.cpe.CPEAnalyzer.determineCPE(CPEAnalyzer.java:145) at org.owasp.dependencycheck.data.cpe.CPEAnalyzer.analyze(CPEAnalyzer.java:459) at org.owasp.dependencycheck.Engine.analyzeDependencies(Engine.java:250) at org.owasp.dependencycheck.App.runScan(App.java:145) at org.owasp.dependencycheck.App.run(App.java:122) at org.owasp.dependencycheck.App.main(App.java:71)

The problem seems to be when it tries to parse "http://jsoup.org/"
when pathParts length is 0 and then this fails
if (!pathParts[pathParts.length - 1].isEmpty())

this made it work for us:
if (pathParts.length>0 && !pathParts[pathParts.length - 1].isEmpty())

The suppression file should be able to be specified as a URL (specifically HTTP or HTTPS).

The VulnerableSoftware class throws an exception on large version numbers.

I'm having a hard time determining exactly the circumstances, but it seems that any evidence added to the version collection is removed if it's unable to make a CPE match otherwise.

We created a new analyzer which attempts to identify the product and version of some internal files. If we do the following:

dependency.getVersionEvidence().addEvidence(source, name, version, Evidence.Confidence.HIGHEST);
dependency.getProductEvidence().addEvidence(source, name, product, Evidence.Confidence.HIGHEST);

Then no version evidence shows in the report. The only time version information shows up is if we add a vendor evidence, and that vendor is able to be matched. Even if the Version and Product together make an exact match, if the vendor is missing, the version information is removed.

I suspect this is because the CPE matching is done in an iterative fashion, and it stops working if it can't identify a product. This is challenging because the modules in question don't really have any vendor information in the files themselves.

We need an Archive Analyzer so that zip, ear, and war files can be scanned.

The lucene index grows on every update - even if there are no changes. This implies duplicates are somehow being added. This will affect performance and possibly results.

Hi,
I have problems with running the dependendency-check-maven
mvn org.owasp:dependency-check-maven:1.0.8:check

Jan 21, 2014 1:09:05 PM org.owasp.dependencycheck.Engine doUpdates
WARNING: Unable to update Cached Web DataSource, using local data instead. Results may not include recent vulnerabilities.
Jan 21, 2014 1:09:06 PM org.owasp.dependencycheck.Engine analyzeDependencies
SEVERE: Unable to load database

mvn -version
Apache Maven 3.0.5
Java version: 1.7.0_25, vendor: Oracle Corporation

mvn org.owasp:dependency-check-maven:1.0.7:check does the download of the database

What has changed, or what am I missing?

I need to run org.owasp:dependency-check-maven:1.0.7:check on a number of projects, and don't want to change the pom foreach project.

The setup would require the failBuildOnCVSS to be runtime configurable,

@parameter(property = "failBuildOnCVSS", defaultValue = "11", required = true)
private float failBuildOnCVSS = 11;

The reports should include information about the current age of the NVD/CVE data. This is to ensure someone looking at the report can validate that current data was used.

On occasion, a false positive may be consistently flagged by Dependency-Check as a vulnerability. For example, the MySQL JDBC driver.

This enhancement will provide a mechanism to which false positives can be suppressed and not reported if a specific condition is met.

Instead of trying to match Vendor + Product + Version; to try to match Vendor + Product. When match is found from Product:Vendor then try to figure out the version. This will help with the overall detection and would allow the creation of non-published/official CPEs that would result in valid vulnerabilities being detected (i.e. someone created there own point release with some minor code changes but it is still basically the same as version x.x - if we found the correct vendor:product pair; then we could tack on a version number from the manifest to generate a CPE that may not officially be in the CVE data). We could then update the CVE matching to account for these "non-official" CPEs.

This should reduce false positives.

The Archive Analyzer throws an exception due to a directory that ends with .zip... At least this has been privately reported; still need to create test case for this.

When "All Previous Versions" is indicated, all previous versions are currently listed. This seems a little duplicative and increases the size of the report. Instead, only those entries marked as "All Previous Versions" should be listed.