coreinfrastructure / best-practices-badge Goto Github PK

In /projects/new: Currently H1 is centered bold, H2 is italic underlined. Don't use underline (that implies links).

Don't spend a lot of time (20 min or less), since LF may customize the bootstrap CSS settings anyway.
Perhaps use "startbootstrap.com" or other pre-existing common CSS settings.

We should be able to dump an entire database - probably as SQL database. That would enable easy backup, easy transition, etc. That should probably require a login at least, and perhaps it would become privileged operation.

https://github.com/joecorcoran/judge
https://github.com/amatsuda/html5_validators
https://github.com/posabsolute/jQuery-Validation-Engine

Creating a wizard to save intermediate state
https://github.com/schneems/wicked

My specific suggestion is to create validations but have then all run on: :certify, not on :create or :update. Then, you create controller logic for create where if the object is valid, it runs certify, and otherwise it just saves (which keeps it in an intermediate state). This isn't fully thought through, though, because you would not want an :update to be able to move you to an invalid state.

The values for criteria are not simple booleans. We can eliminate "considered" and reduce them to three: MET, NOT MET, or UNKNOWN, but that still makes them non-booleans. As a result, we can't use traditional checkboxes. Here are some options:

• We can implement that as a select pull-down, but these are more work for users (for each selection you have to pull down, and select)... and that matters when there are this many questions.
• We could implement them as a radio box, but that takes a LOT of horizontal space (which is in short supply on mobile devices), due to having buttons adjacent to labels, and the selected button is separate from the text (which is slightly less clear).

We could, instead, try to implement these as tri-state values (e.g., tri-state checkboxes).

This mightn't be important in the first iteration of the badges (basic / bronze). We've now agreed upon using superscripted text in square brackets, and anchors to identify the criteria name. But how do we want to handle the criteria when we have different badge types?

If a criterion is a SHOULD in bronze, and then MUST in silver, will we use the same name across the badge criteria? And will app have the logic in it to determine whether criteria are mandatory or not for each badge?

Also semi-related is whether we'll eventually have all criteria on the same page or not. I'm raising this now as if we have them all on the same page we want to have anchors for each of the criteria name (and we use the same name between badges) then if someone provides the anchor link then it'll get confusing if the names aren't unique across the different badges.

Couple of rough options are:

1. We always have the different badge criteria on different pages.
2. We rename the criteria name marking and anchor to be prefixed by the badge colour (e.g., bronze-crypto-pfs, silver-crypto-pfs)

There needs to be a REST API for getting the badge status of a project, or getting the actual badge from a project. The REST API needs to be really simple.

The current intro page of BadgeApp needs to be rewritten. E.G., it says:

"The Best Practices Badge is a secure open source development maturity model. Projects having a CII badge will showcase the project's commitment to security...." But it's not at all clear that this is a maturity model, and 'showcase' is an odd phrasing.

How about this instead:

The CII Best Practices badge is a way for open source software (OSS) projects to show that they follow best practices. Projects can voluntarily self-certify, at no cost, by using this web application to explain how they follow each best practice. The CII Best Practices Badge is inspired by the many badges available to projects on Github. Consumers of the badge will be able to quickly assess which OSS projects are following best practices and as a result are more likely to produce higher-quality secure software.

Add a basic framework to automatically compute form values when we can (esp. for GitHub or git-supported projects).

See implementation.md for current proposed approach.

Once we have a framework, we can implement various automation routines within it.

Make it "Details" and "Hide Details" instead of ">>".

The "uses basic good cryptographic practices" section has requirements that often don't apply. Pull out delivery to a separate section, and then have a "does crypto apply" checkbox at the top - then if crypto doesn't apply, they don't need to do the rest.

The form doesn't always match criteria.md

Currently the justification text is hidden on reload, even if it's met or unmet.

Don't just change on blur.

Currently criteria are marked with names using square brackets.

In commit 7df076a there's discussion on whether there's a better way. Should names be visible to casual users? (David says yes, Dan is skeptical.) If they are visible, should they be displayed differently (e.g., as smaller superscripted text)? Should these be named anchors, enabling people to jump to specific locations?

Issue #55 implemented text hiding... now we need to USE it in the form.

People who aren't logged in should still be able to see the detailed answers of a project. For now, just make a stub with all the answers; we can pretty it up later.

Remove "Background" on Navigation bar; that's way too detailed. Perhaps replace it with "home" (the top level)?

"Please explain" is too generic - give more details specific to the question. In some cases, perhaps an explanation shouldn't be there.

Perhaps we should force users to enter the "identification" data first when entering a new project, and not even show the rest - that would give us a chance to automatically fill in a lot of the rest of the form before showing it to users.

We probably ought to talk about what is a user id.

For non-GitHub users, we could just use email addresses (Heroku does this). If we do, though, we ought to not display those usernames to the general public... perhaps they should only be displayed when you're viewing a project that you can also edit. We don't have to use email addresses, I just raise that as a possibility.

For GitHub users I don't see a problem displaying them, they're public anyway.

We'll need to make sure that the naming scheme is general, so that we can add systems other than GitHub in the future.

Currently the form shows whether or not each criteria meets the badge - show a progress bar showing the % meeting a badge.

Show something on each criteria (using Javascript) so users will know if they've done enough or not.

Don't make them different

I think we should require URLs in many of the fields that are entered by humans, with an error message something like "Please include one or more URLs that justify this claim."

Criteria may have an automatically-determined value when we have code that can determine it, but in many cases there's no way to automatically determine it, or the automated system can't figure it out. Thus, for most criteria we'll need to let people enter text to justify that they meet the criteria. Find, but we need to make it easy for other people to independently verify that the claim is true. If we require that there be a URL, then we can tell people they have to POINT to the justification.

BTW, I suspect the text entry might be easiest if we used markdown. It's widely known by software developers, and easier to enter than raw HTML (for example).

By requesting /badge/number, you should get the text to show your badge (or that you haven't achieved it currently).

Make it easy to search by name for a project. It should return a list of links to all project badge info with that name. In the longer term, we might supplement the list with Debian project names, Fedora project names, etc.

See issue #58 (related).

There's a lot of repetition in the Ruby code generating the Rails - simply that (e.g., a separate method for generating the radio buttons).

Put thumb on right, put buttons in table to improve organization.

Split this up: "Cryptographic protocols and algorithms used by default in the software AND the delivery mechanisms MUST be publicly published and reviewed by experts."

We want to let people fill in forms incrementally, but reject really bad data. E.G., limit the lengths of justifications to some plausible data, and limit license data to something that could be SPDX.

BadgeApp: Put criteria short names in superscript, not bold, and possibly add dagger and asterisk just like the criteria.md.

At least for crypto and warning flags, N/A should be allowed.

We want people to be able to see the information about a project without logging in. By having one form for both editing and for just viewing, it'll simplify things.

In "The project MUST include reference documentation that describes its interface. The project MAY use hypertext links to non-project material as documentation, as long as the linked-to information is available and meets the requirements." The justification box should be AFTER all that.

Once we get project and repo URLs, we should try to fill in the form and present it back with some automated results. We can then keep refining it.

A concept seem to be mssing: dependancies which are themselves "badged", would not need to be checked for vulnerabilities. This would render the badging process contagious.

Once a user logs in via github, they should be able to select their projects with a click instead of filling in the URL.

What are the groups thoughts on password hashing? At the moment we have a criterion that states they should stored as iterated hashes with a per user salt. Should we have another criterion that is Password hashes SHOULD be generated using a key stretching algorithm like PBKDF2, Bcrypt or Scrypt? Then for the more advanced badges this could become a MUST.

Greg KH suggests the following in pull request 1, #1 :

"Tests for major feature additions as without some kind of test being present, it's hard to verify if something new even works. This is now the "unofficial" rule for the kernel, as we have been burned by this in the past (feature additions that never even worked)."

I like the idea of adding this to the criteria set, but I think projects need to make it "official" in two senses: (1) their docs on making changes need to require it, and (2) there needs to be some evidence that they're doing it. If it's a super-secret rule, or one not actually used, there's not value. I don't want to add criteria to the "basic" list that few projects meet, though. If this the "unofficial" rule for the kernel, would they be willing to do that?

Make it possible to show/hide the recommendations and other detailed text. Also, make sure the recommendations aren't indented into the radio buttons.
Should hide or reveal be default?
Dan, Sam recommend "hide by default". Could also have "show all explanation" button.

There should be a way to drop-select common licenses - most projects only use MIT, BSD 3-clause, GPLv2, GPLv2+, GPLv3, LGPLv2+, and a few others. Make it easy to select. We should probably show the SPDX license tags at least.

Keep records of older versions of the badge data, and for now just show the current versions. We could do this by including an "edit date" in the record, and only show the most recent one. We could later add the ability to show older revisions.

If a project is using git or another revision control system that supports the equivalent of signed Git tags, it should require the release manager to sign tags.

I'd send a PR, but it seems like this could fit into different places - e.g. it could just be mentioned in the MitM section, but it might also be split off into its own section, etc. Let's discuss.

We should be able to see a project's data by asking for /project/NUMBER.json.

Make it possible to see the results of several selected projects on one screen. This is lower priority.

It may be a good idea for BadgeApp to automatically acquire its criteria text from criteria.md. That certainly would be DRY. We can tweak the criteria.md file so it's easier to process, e.g., if additional information needs to be added.

Add the ability to SHOW history (previous revisions) of badge info. This depends on issue #10 to capture the data.