dotnet / org-policy

This repo contains tools and tracks policy violations for the dotnet org.

Policy tooling for the dotnet org

This repo contains tools and tracks policy violations.

Policies and process

For details on policies, see the docs.

Usage

For the dotnet org, the policies are evaluated daily and violations are posted in the internal repo org-policy-violations. The repo is internal because it contains names of private repos and teams.

Running locally

You can run the tool locally by cloning this repo and running policop.cmd from the root.

Getting the org data

Before you can do anything useful, you need to get access to the org data, which includes repos, teams, users and their relationships. This also includes access to linking information between Microsoft user accounts and GitHub user accounts.

Due to performance and API rate limitations it's not practical to query this information from GitHub when you're experimenting and trying to analyze the org. So instead, you can download a cached version of the org that was computed and uploaded to a private Azure DevOps project during the nightly policy runs.

You do this by running:

$ .\policop cache-build

This will download the latest version of the org data and store it on your local machine. If you run this command for the first time, it will take you to a website where you'll need to create an access token that the tool will then store and use on future calls.

You can check how old your local cache is by running

$ .\policop cache-info

You can also clear the cache with

$ .\policop cache-clear -f

Evaluating policies

In order to check policies, you simply use this command:

$ .\policop check --excel

This will compute all policy violations and display the result in Excel. You can also write them to a file if you prefer that:

$ .\policop check -o D:\temp\test.csv

Querying org data

The primary command is policop list which you can use to query information from the org.

Using -r, -t, and -u you can list all components of the org:

• -r the list of repos
• -t the list of teams
• -u the list of users
• -r -t the list of repos and permissions teams are given
• -r -u the list of repos and permissions users are given
• -t -u the list of teams and their members
• -r -t -u the list of repos and permissions teams & users are given

Each of those options accept a list of terms you can use to filter, with basic wild card support, such as *core* or dotnet*.

So to list all teams whose name contains the text core you'd do this:

$ .\policop list -t *core*

To find all members of all teams named *core* you'd do this:

# List team members of teams whose name contains "core"
$ .\policop list -t *core* -u 

Using -f you can also filter:

# List all repos whose name contains dotnet and where a team
# grants admin access
$ .\policop list -r *dotnet* -t -f rt:permission=admin

For columns returning Yes/No you can also use the simple version:

# List all private repos
$ .\policop list -r -f r:private

And lastly, using -c you can create custom reports with specific columns:

# List all private repos and show their name, description and list of admins
$ .\policop list -r -f r:private -c r:name r:description r:admins

The available columns can be listed by running

$ .\policop list-columns

The naming convention indicates when the columns can be used:

• r:* when repos are included
• t:* when teams are included
• u:* when users are included
• rt:* when repos and teams are included
• ru:* when repos and users are included
• tu:* when teams and users are included
• rtu:* when repos, teams, and users are included

In general, policop list will print the results to the console but with -o you can write to a file and with --excel you can send it straight into Excel.

org-policy's Issues

Repo should be marked as either owned by Microsoft or not-owned by Microsoft

Explicit marking ensures we don't have repos accidentally marked.

Public repos shouldn't grant collaborators pull permissions

This used to be valuable so that we can assign issues, but that's no longer necessary (you can assign issues to anyone with permissions for the repo and anyone who commented on it). The only downside is the loss of the "collaborator" badge, but the downside for admins (having a huge list to deal with) is more important to fix IMHO.

Enforce repos to have a `SECURITY.md`

I think it's good to have all repos with SECURITY.md file. See docs.

@terrajobst Do you think this is useful?

Flag public wikis without "restrict to contributors" setting

Enforce that Microsoft-owned repos have GHAS enabled

See: https://docs.github.com/en/get-started/learning-about-github/about-github-advanced-security

Ensure repos have branch protections enabled

We should check that known branches have branch protection. The minimum we should be looking for is force push against known branches:

• master
• release/*

@jaredpar, anything else?

Link to usernames is broken

Instead of linking to https://github.com/orgs/dotnet/people/xxx, can we just link to https://github.com/xxx

Every time I'm assigned an issue, I need to manually change the link to see the person.

e.g. https://github.com/dotnet/org-policy-violations/issues/3351

OSPO API question

I was just browsing this code; do you still see upn/email swaps? Would love an example...

  // For some interesting reason, some people have their
            // email in the PreferredName field...

            var ms = link.MicrosoftInfo;

            if (ms.PreferredName != null && ms.PreferredName.Contains("@"))
            {
                if (string.IsNullOrEmpty(ms.EmailAddress))
                {
                    ms.EmailAddress = ms.PreferredName;
                    ms.PreferredName = null;
                }

org-policy/src/Microsoft.DotnetOrg.Ospo/OspoClient.cs

Line 59 in 705ab41

if (ms.PreferredName != null && ms.PreferredName.Contains("@"))

Consider checking profiles for names

This is a proposed rule for members in the Microsoft group: users should have their first and last name set as part of their GitHub profile. This helps with team maintenance, searches, and transparency in open source.

Web hooks should be audited

PR11 - JIT Admins

https://github.com/dotnet/org-policy/blob/main/doc/PR11.md doesn't take into account JIT admins.

org-policy/src/Microsoft.DotnetOrg.Policies/Rules/PR11_ReposShouldHaveSufficientNumberOfAdmins.cs

Line 20 in edf6221

var numberOfAdmins = repo.GetAdministrators(fallbackToOwners: false).Count();

In order to resolve:
https://github.com/dotnet/org-policy-violations/issues/5386

I would have to add admins directly instead of using the more secure JIT admins feature.

When reopening issues, the assignees should be updated