Comments (5)
What happened here was:
- OSS-Fuzz had an MSan regression. Annoying, but it happens.
- We got a ton of false reports in OSS-Fuzz.
- I spent 30 minutes of my day triaging one of them to confirm it was indeed a false positive.
- There is no way in OSS-Fuzz to reassign to maintainers, so we followed our usual process of ignoring false positives.
- A week later, the regression is finally fixed and OSS-Fuzz closes everything.
- Later, I get a bunch of alerts internally about unfixed vulnerabilities from this "OSV" thing I'd never heard of or registered with. I have to figure out how to close those out as being false, without the robot reopening them again.
- I spend about an hour searching and pinging people to figure out where this came from and where to file a bug about things going wrong.
- I eventually find the OSV site and see that it indeed is falsely claiming a bunch of vulnerabilities in BoringSSL
- I eventually get directed to an appropriate person, after having burned a ton of time, and learn that I need to manually edit some YAML files in a system I'd never heard of.
- I fork the GitHub repo and go to edit the files.
- Apparently one needs to manually make YAML timestamps and also manually fill in the last modified time. I honestly could not figure out how to do that effectively, which is why the seconds are zero in my timestamps.
- I put together google/oss-fuzz-vulns#37 and ping a human to go land it to stop the OSV-induced panic.
As for what would be better, basically every step in this process went wrong.
- Better testing in oss-fuzz might have reduced the frequency of MSan regressions: google/oss-fuzz#11941
- But we have to assume MSan regressions might come through, so oss-fuzz should have some way for project owners to triage things: google/oss-fuzz#11939, google/oss-fuzz#11925, and google/oss-fuzz#6974
- But project owners may be busy and may just not know to do this. After all, for the entire lifetime of the oss-fuzz project, false positives were not only safe to ignore, but ignoring them was the only means of triage. It's OSV that made assumptions on OSS-Fuzz that weren't tenable. When an OSS-Fuzz issue is imported to OSV, there should be a comment on OSS-Fuzz so project owners know that false data was imported.
- When OSV gets it wrong, there needs to be a lightweight flow to fix it. This could be fixing the oss-fuzz data and then having it import automatically (and quickly!), or it could be a direct flow. Whatever the flow, it must not involve any of:
- Pinging an OSV maintainer for human review
- Forking a GitHub repo to make a PR in an unfamiliar repo
- Manually writing YAML timestamps
from osv.dev.
Hey @davidben
Could you describe what your current user journey looks like, and what an appropriately lightweight one could look like?
from osv.dev.
Apparently one needs to manually make YAML timestamps and also manually fill in the last modified time
Timestamps alone make this process quite off-putting: #857 (comment). I'm still not sure how to get them right :-)
from osv.dev.
Brilliant, thanks for all this detail, it's very helpful.
4. When OSV gets it wrong
So based on google/oss-fuzz-vulns#37 being how to correct things, I think it's fair to say that this should read "When OSS-Fuzz gets it wrong"?
Based on your fourth point above, a wee bit of tooling to aid with manipulating that YAML sounds like it would go a long way to addressing the "lightweight" part of things, plus a modicum of documentation?
I would imagine some sort of GitHub Action could allow PRs from users identifiable as affiliated with the project the PR is the subject of to be automatically merged, thus removing the human component...
So, at a high level, perhaps:
- better documentation
- better signposting to that documentation from the OSS-Fuzz issues
- tooling to assist with record manipulation
- automation for record manipulation PR submission
I'm still on the fence about whether this issue belongs in the OSV.dev repo or the OSS-Fuzz repo...
from osv.dev.
When OSS-Fuzz gets it wrong
I wouldn't say OSS-Fuzz gets it wrong. It works as expected in the sense that it reports issues fuzz targets hit (regardless of whether they have anything to do with security or not). I think the wrong part here is that everything gets automatically imported into the OSV database with no vetting.
better documentation
better signposting to that documentation from the OSS-Fuzz issues
That would have certainly helped in google/oss-fuzz#7434.
I would imagine some sort of GitHub Action could allow PRs from users identifiable as affiliated with the project the PR is the subject of to be automatically merged, thus removing the human component
It should help but I'm not sure it can be (fully) automated. For example in google/oss-fuzz#11883 the OSS-Fuzz bot (which tries to do that) didn't recognize the maintainers. Other than that some maintainers aren't on GitHub so they can't open PRs. Some maintainers don't have access to Monorail because they don't have gmail accounts and bug reports just get sent to their mailing lists.
I have to admit I don't know how to fix that. It seems to me that one option would be turn off this import by default and let projects opt-in. If they decide to do that it should probably be safe to assume that they read the documentation, know what OSV is and are ready to vet bug reports. Another option would be to separate the OSS-Fuzz feed from OSV where it wouldn't be automatically implied that they are vulnerabilities.
from osv.dev.
Related Issues (20)
- Missing entries in the package field for GIT based ecosystem HOT 1
- Make documentation site style match main site. HOT 1
- combine-to-osv: withdraw rejected CVEs
- Support easy access to the json version of a vulnerability HOT 5
- Bioconductor enumeration code is fault-intolerant
- Request OSS-Fuzz project owner approval before importing OSS-Fuzz reports HOT 14
- Disable automatic OSS-Fuzz -> OSV import for BoringSSL
- CVEs getting reported for git hash, which have been resolved HOT 2
- Navigating away from a blog post attempts to get the blog's images from the new page
- There is a human readable authoritative definition of what the minimum quality bar looks like for an an OSV record that is acceptable for import by OSV.dev
- Tooling exists for OSV record creators to validate that they meet the minimum quality bar at record creation time HOT 4
- The minimum quality bar is sustainably enforced by OSV.dev
- OSV record providers have a machine-readable way to reason about existing published records that do not meet this quality bar
- OSV.dev downstream users have a way to reason about records that are absent from OSV.dev because of failure to meet the minimum quality bar
- OSV.dev downstream users have a clearly defined user journey to make corrections to OSV records served by OSV.dev with minimal overhead by all parties
- Data quality issue CVE-2023-24163 HOT 3
- Data quality issue with GHSA-4wrc-f8pq-fpqp HOT 1
- Handle very long fields on vulnerabilities pages HOT 1
- nvd-cve-osv: OpenSSL versions do not normalize correctly
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from osv.dev.