google / pullsheet

Generate spreadsheets based on GitHub contributions

pullsheet

pullsheet generates a CSV (comma separated values) & HTML output about GitHub activity across a series of repositories.

It currently supports CSV exports for:

• Merged Pull Requests: pullsheet prs [FLAGS]
• Pull Request Reviews: pullsheet reviews [FLAGS]
• Opening/Closing Issues: pullsheet issues [FLAGS]
• Issue Comments: pullsheet issue-comments [FLAGS]

As well as a new HTML leaderboard mode: pullsheet leaderboard [FLAGS]

This tool was created as a brain-tickler for what PR's to discuss when asking for that big promotion.

Usage

go run pullsheet [subcommand] --repos <repository> --since 2006-01-02 --token-path <github token path> [--users=<user>]

You will need a GitHub authentication token from https://github.com/settings/tokens

Example: Merged PRs for 1 person across repos

go run pullsheet.go prs --repos kubernetes/minikube,GoogleContainerTools/skaffold --since 2019-10-01 --token-path /path/to/github/token/file --users someone > someone.csv

Example: Merged PR Reviews for all users in a repo

go run pullsheet.go reviews --repos kubernetes/minikube --kind=reviews --since 2020-12-24 --token-path /path/to/github/token/file > reviews.csv

Example: Merged PRs for a user in all repos within an org

go run pullsheet.go prs --org google --since 2020-12-24 --token-path /path/to/github/token/file > reviews.csv

CSV fields

Merged Pull Requests

	URL         string
	Date        string
	User        string
	Project     string
	Type        string
	Title       string
	Delta       int
	Added       int
	Deleted     int
	FilesTotal  int
	Files       string // newline delimited
	Description string

Merged Pull Request Reviews

	URL            string
	Date           string
	Reviewer       string
	PRAuthor       string
	Project        string
	Title          string
	PRComments     int
	ReviewComments int
	Words          int

Closed/Opened Issues

	URL     string
	Date    string
	Author  string
	Closer  string
	Project string
	Type    string
	Title   string

Issue Comments

	URL         string
	Date        string
	Project     string
	Commenter   string
	IssueAuthor string
	IssueState  string
	Comments    int
	Words       int
	Title       string

pullsheet's Issues

Add server mode

Add web server mode that serves periodically leaderboard/CSV exports. Would use the command refactor introduced by #14

This could ideally support a multi-tenant YAML configuration approach, but happy to split that into a second issue.

Add GitLab as extra provider

Maybe we can add GitLab as well as an option to generate the same thing and in the command line can be passed something like

pullsheet --provider gitlab ...
or 
pullsheet --provider github ...

Most Supportive: Add Slack Chatroom Stats

example how may comments one did on slack chat in response to user's questions

Allow ignoreRe & truncRe to be set via flag

feature request: JSON output

For all current kinds

FR: Output a single leaderboard chart

Related to #5

Users should be able to say "I only want to generate the list of top reviewers".

Switch to klog

Currently we use logrus. But one of our dependencies is triage-party which uses klog. We should prefer to use klog anyway as it is more in line with the rest of our ecosystem.

Add CI

Add CI jobs to validate and test, can be github actions

if that sounds good I can help to setup that

HTML Output shows Data column(s) for axis #0 cannot be of type string in google visualization when there is no rows but the column is present

The output of the following command will yield in no data for some of the boards which is not handled by google visualizations.

go run pullsheet.go leaderboard --repos google/pullsheet --since 2021-05-06 --token-path gh_token.txt > test.html

The visualization in browser gives the error Data column(s) for axis #0 cannot be of type string

Line 102 in test.html should look like this [{label:'',type:'string'},{label: '# of Merged PRs reviewed', type: 'number'}, { role: 'annotation' }],
but curretnly looks like this ['', '# of Merged PRs reviewed', { role: 'annotation' }],

leaderboard in csv

wrong date range in leaderboard

in command line I am passing since 04/01/2021 until 08/23/2021

but I see

go run pullsheet.go leaderboard --token-path ~/token.txt --repos kubernetes/minikube,medyagh/gopogh,GoogleContainerTools/gcp-auth-webhook,GoogleContainerTools/minikube-image-benchmark since 04/01/2021 until 08/23/2021 --logtostderr=false --stderrthreshold=2 | sed -r -e "/Command\-line/,/pullsheet/d" >> ~/Desktop/perf_21_aug_2021.html

fix lint issues

Running [/home/runner/golangci-lint-1.57.2-linux-amd64/golangci-lint run --out-format=github-actions] in [] ...
  Error: emptyStringTest: replace `len(rootOpts.org) > 0` with `rootOpts.org != ""` (gocritic)
  Error: File is not `gofumpt`-ed (gofumpt)
  Error: File is not `gofumpt`-ed (gofumpt)
  Error: File is not `gofumpt`-ed (gofumpt)
  Error: unnecessary leading newline (whitespace)
  Error: import 'github.com/google/go-github/v33/github' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/google/triage-party/pkg/persist' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/google/go-github/v33/github' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/google/triage-party/pkg/persist' is not allowed from list 'Main' (depguard)
  Error: import 'k8s.io/klog/v2' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/google/go-github/v33/github' is not allowed from list 'Main' (depguard)
  Error: import 'k8s.io/klog/v2' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/google/pullsheet/pkg/client' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/google/pullsheet/pkg/ghcache' is not allowed from list 'Main' (depguard)
  Error: import 'k8s.io/klog/v2' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/google/pullsheet/pkg/client' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/google/pullsheet/pkg/ghcache' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/google/pullsheet/pkg/client' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/google/pullsheet/pkg/ghcache' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/blevesearch/segment' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/spf13/viper' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/google/pullsheet/pkg/repo' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/google/pullsheet/pkg/repo' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/google/pullsheet/pkg/repo' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/gocarina/gocsv' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/google/pullsheet/pkg/leaderboard' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/google/pullsheet/pkg/summary' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/google/pullsheet/pkg/server/job' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/google/pullsheet/pkg/server/job' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/google/pullsheet/pkg/server/site' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/karrick/tparse' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/google/pullsheet/pkg/print' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/google/pullsheet/pkg/summary' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/spf13/cobra' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/google/pullsheet/pkg/print' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/google/pullsheet/pkg/summary' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/spf13/cobra' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/karrick/tparse' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/spf13/cobra' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/google/pullsheet/pkg/leaderboard' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/google/pullsheet/pkg/print' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/karrick/tparse' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/pkg/errors' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/spf13/viper' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/google/pullsheet/pkg/server' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/google/pullsheet/pkg/server/job' is not allowed from list 'Main' (depguard)
  Error: import 'github.com/google/pullsheet/cmd' is not allowed from list 'Main' (depguard)
  Error: ineffectual assignment to err (ineffassign)
  Error: SA1019: "io/ioutil" has been deprecated since Go 1.19: As of Go 1.16, the same functionality is now provided by package [io] or package [os], and those implementations should be preferred in new code. See the specific function documentation for details.  (staticcheck)