google / triage-party

🎉 Triage Party: massively multi-player GitHub triage 🎉

Triage Party: massively multi-player GitHub triage 🎉

NOTE: This is not an officially supported Google product

Triage Party is a stateless web app to optimize issue and PR triage for large open-source projects using the GitHub API.

Triage Party focuses on reducing response latency for incoming GitHub issues and PR's, and ensure that conversations are not lost in the ether. It was built from the Google Container DevEx team's experience contributing to popular open-source projects, such as minikube, Skaffold, and Kaniko.

Triage Party is configured via YAML, and designed for projects to self-host on anything from a Raspberry Pi, Google Cloud Run, or Kubernetes.

Table of Contents

• Novel Features
• Triage Party in production
• Requirements
• Try it
• Usage Tips
• Multi-player mode
• Kanban mode (NEW)
• Data freshness
• Documentation

Novel Features

• Multiplayer mode: Up to 20 simultaneous players in group triage
• Conversation direction tracking and filtering
• Query across GitHub projects or repositories
• Duplicate issue detection
• Filters that are not possible on GitHub:
  • Follow-up latency by project members (responded: +15d)
  • Reactions (reactions: >=5)
  • Comment popularity (comments-per-month: >0.9)
• Code review state-tracking (v1.2.0+)
• Kanban dashboard (v1.2.0+)
• Easily open groups of issues into browser tabs
• YAML configuration for all pages, rules, and filters
• GitHub Enterprise support (via --github-api-url cli flag)
• Low latency (yet able to pull live data)

Triage Party in production

See these fine examples in the wild:

• kubernetes/minikube
• GoogleContainerTools/skaffold
• jetstack/cert-manager
• Thoth Station

Requirements

• GitHub API token, with public_repo permissions (read-only)

Try it

Store a GitHub token some place on disk:

echo YOUR_GENERATED_TOKEN > $HOME/.github-token

Run:

go run cmd/server/main.go \
  --github-token-file=$HOME/.github-token \
  --config config/examples/kubernetes.yaml \
  --repos kubernetes/sig-release

If you do not have Go installed, you can run Triage Party using Docker:

export GITHUB_TOKEN="$(cat $HOME/.github-token)"
docker build --tag=tp .
docker run \
  --rm \
  -e GITHUB_TOKEN \
  -v "$PWD/config/examples/kubernetes.yaml:/app/config/config.yaml" \
  -p 8080:8080 tp

You can also use Personal Access Token from the GitHub CLI to run the application:

export GITHUB_TOKEN=$(grep oauth_token ~/.config/gh/hosts.yml| cut -d ":" -f 2 | sed "s/ //g")
docker build --tag=tp .
docker run \
  --rm \
  -e GITHUB_TOKEN \
  -v "$PWD/config/examples/kubernetes.yaml:/app/config/config.yaml" \
  -p 8080:8080 \
  tp

You'll see logs emitted as Triage Party pulls content from GitHub. The first time a new repository is used, it will require some time (~45s in this case) to download the necessary data before minikube will render pages. Your new Triage Party site is now available at http://localhost:8080/, but will initially block page loads until the required content has been downloaded. After the first run, pages are rendered from memory within ~5ms.

Usage Tips

Triage Party can be configured to accept any triage workflow you can imagine. Here are some tips:

• Use the blue box-with-arrow icon to open issues/pull requests into a new tab
  • If nothing happens when clicked, your browser may be blocking pop-ups
  • The notification to allow-popups for Triage Party may be hidden in the URL bar.
• Rules work best when there is a documented resolution to remove it from the list
• Pages work best if the process is defined so that the page is empty when triage is complete
• If an non-actionable issue is shown as part of a daily or weekly triage, step back to tune your rules and/or define an appropriate resolution.

Multi-player mode

Use the drop-down labelled Solo on the top-right of any page to enable multi-player mode. In multi-player mode, the number of issues are split among the number of players you have configured. Since Triage Party is state-less, players are assigned via the remainder of the issue or PR divided by the total number of players. Here is a workflow that we have seen work well for triage parties:

1. Wait for attendees to show up
2. The meeting host selects the appropriate number of players, and shares the resulting Triage Party URL
3. If someone is showing up later, we may leave a slot open and re-shard later if they do not appear
4. The meeting host assigns each attendee a player number
5. Players move section by section, using the "open items in new tabs" feature to quickly work through issues
6. When a player does not have the context necessary to resolve an item, they present their screen and discuss it with the other players
7. When a player leaves, the meeting host "re-shards", and all players select the updated player count in the drop-down

NOTE: Multi-player works best if the "Resolution" field of each rule has a clear action to resolve the item and remove it from the list.

Kanban mode (NEW)

In v1.2.0, pages can now be displayed as a Kanban dashboard. The columns are based on the rule the issue was found in, and the rows are swim-lanes based on the assignee. To see a real-world example:

• minikube kanban dashboard
• minikube kanban config

Best practices for designing a useful Kanban dashboard:

• Rules should be designed and ordered in a way that represents progress: Not started -> Started -> Under Review -> Completed
• Rules work best when they are mutually excusive (no issue matches multiple rules)
• If a collection should be displayed in Kanban form by default, specify display: kanban in its configuration.
• For velocity measurements and time estimate support, create a rule named __velocity__ containing recently closed issues to include. See the example configuration.

Data freshness

With the default Dockerfile, Triage Party refreshes data at least every 8 minutes, settable using the --max-refresh flag. Triage Party will give popular pages a higher refresh rate, up to every 30 seconds by default (settable using --min-refresh flag). This default is conservative, allowing Triage Party to work with repositories containing 10,000 open issues without hitting GitHub API limits.

Live data can be requested at any time by using forcing a refresh in their browser, typically by holding the Shift button as you reload the page. See forced refresh for your browser.

You can see how fresh a pages data is by mousing-over the "unique items" text in the top-center of the page.

Documentation

Thirsting for more? See:

• Configuration guide
• Deployment guide
• Persistent cache configuration

triage-party's Issues

At head: items are cached much longer than intended

It seems like result.Time is up to date, but the underlying GitHub data can be very far out of date.

Scaffold example - broken link

The tinyurl (https://tinyurl.com/skaffold-teaparty ) is broken for the scaffold example.

kanban: Remove dependency on milestones

Somewhere down the line I broke kanban support for non-milestone based views. I assume this is due to the selector.

Add warning if token does not allow organization status to be queried

This should be both in the UI and the logs.

Triage Party still works in this case, but it is unable to track conversational state.

Support GitHub Enterprise

Thanks for sharing a great tool!

I would like to use triage-party against repositories on our internal GitHub Enterprise.

Fortunately, we can support this feature easily because go-github does support GitHub Enterprise via NewEnterpriseClient().

Remove TOKEN as a build-arg

It leaks credentials.

Reported by @munnerz

Add support for filtering out Draft PR's

Draft PR's should get a draft tag that can be filtered out.

Add TiKV support for cache persistence

It seems to be a popular Kubernetes option.

Persist to disk on a separate loop than content updates

This will allow us to support arbitrarily sized repositories.

Documentation: clarify token file

Hi there,

Looking to try this out with the Mautic community, however I fell at the first hurdle.

Can you explain how to set up a token file or point me at the appropriate documentation which explains how to create it? I am very familiar with creating the tokens, but have not seen that used in this way before, and could not find anything online.

Similarity: generate in background, and only when titles change

The logic is pretty basic right now:

triage-party/pkg/hubbub/similar.go

Line 83 in 083cec8

if h.similarCacheUpdated.After(h.lastItemUpdate) {

PR's don't seem to be getting the `commented` tag

As seen with #4126 at https://skaffold-triage-party-eyodkz2nea-uc.a.run.app/s/daily

Tune initial fresh run to overlap less with initial stale run

If there is no initcache, the stale run is effectively a duplicated fresh run (+ a few seconds).

If the fresh run and stale run were based on the same timestamp, they would not duplicate as much work.

Implement basic leaderboards

This is to allow users to define leaderboards to incentivize positive behavior, such as:

• of issues commented on

• of issues closed without follow-up comment

• of PR's merged

At HEAD: duplicate list shows two entries when an issue changes title

Our dupe checker should be smart enough to dedupe it's own entries.

Add persistence support for CockroachDB

Increase maximum player count to 12

Triage Party still doesn't reliably notice new PR's on issues

Some examples for Skaffold:

#4293
#4131

Add support for parsing OWNERS files

in miniukube's triage-party, I added a user to the reviwers in OWNERS file and I saw he had responded to an isssue and labeled too, but the triage party still was saying issues without comment, till I commented

example kubernetes/minikube#8460

Improve compatibility with very large repos (>3000 open PR's/issues)

Currently, these repositories are likely to run into the GitHub 5k daily rate limit, due to a combination of:

• Fetching comments on more PR's/issues than necessary.
• No persistent cache: #5

v1.2.0-beta: multi-player mode does not show filtered total

I don't think we are setting {{.TotalShown}} correctly any longer.

CloudSQL: Add Postgres support (fails with "driver: bad connection")

I've deployed an instance of triage-party to CloudRun without a persistent backend, and that worked fine.

Then, I deployed a Cloud SQL instance (postgres) and created a user and database. Then, based on minikube-deploy.sh example, I tried to deploy a new revision with CloudSQL by setting the environment variables PERSIST_BACKEND and PERSIST_PATH and that failed.

F0511 19:55:08.934229 1 main.go:94] unable to create persistence layer: new from cloudsql: USER:PASSWORD@tcp(PROJECT/us-central1/jx-teaparty)/tp: driver: bad connection

Advice/tips appreciated.

Add ability to hardcode repo org members

Basically requesting a workaround for the org-permissions issue (referenced here: #105) - If I can't get the permission, I'd still like the ability to filter out repo maintainers from outside contributors.

Show loading page for unavailable collections

It's a better user experience than flat-out showing a blank page until we are ready.

docker build: Replace cache-prefill with cache insertion

At the moment we regenerate the cache every build, which is slow.

Instead, we should insert local cache if available, and quickly bootstrap if not.

Replace expiry flags with refresh flags

• item_expiry was replaced by *refresh*
• org_expiry should be similarly replaced by org_member_refresh

`dedup: false` in collection doesn't work?

I set

collections:
  id: foo
  dedup: false
  rules:
  - rule-a

however, the output of rule rule-a in the collection foo looks like dedeup-ed

Is this behavior expected?

Add mouse-over support & documentation for tags

for example I wanna know what does send mean, I would like to hover over it and tell me in a hint that means...

is there a place we could see the definition of them?

Add support for GitLab

It would be great if the triage-party can also be used with other systems like gitlab, to get an overview of multiple repos and systems to get a better cross platform overview.
I am aware, that this is not a small feature, but it would be an awesome improvement for bigger communities or companies, who need to use multiple services.

Would love to know your opinion about it. No hard feelings, if the request is just closed ;)

Add optional MySQL support to initcache

Allow for faster restarts in environments which do not allow for persistent local cache, such as Cloud Run.

Only fetch closed items for collections require them

Doing this will allow us to defer the fetch until absolutely necessary.

Override cache expiration during bootstrap sequence to avoid requeries

TP currently respects the min/max refresh cache times during the bootstrapping sequence, which means that TP will end up duplicating search queries as they exceed the search cache expiry (~2 minutes).

This makes a slow situation even worse.

Save cache when SIGINT or SIGTERM is encountered

This will improve performance for iterative experiments.

Decrease unnecessary updates for popular collections

With the most recent changes, the updater will keep polling github during idleDuration (5 minutes):

triage-party/pkg/updater/updater.go

Line 124 in 59880a4

if resultAge > u.minRefreshAge && lastRequestAge < u.idleDuration {

So, a single request could spawn 60 loops to GitHub if minRefresh is set to 15 seconds.

Instead, we poll only as often as the popularity requires. Something like:

if resultAge > max(minRefresh, lastRequest) {
  return true
}

similarity detection: filter unimportant words out

We can get a higher hit-rate for similarity if we filter out a basic list of words which don't impact the nature of an issue or PR:

There are certain adjectives and prepositions that come to mind:

a
about
an
and
are
be
by
can
completely
does
extremely
for
has
have
how
in
is
maybe
of
on
or
should
some
something
still
than
the
then
to
use
very
via
when
why
with

However: we need to be careful not to skew things so badly that the min-similarity index is no longer helpful.

Closed PR's: timeline pull happens too often

Example:

I0617 11:16:34.275803   75669 timeline.go:36] cache miss for kubernetes-minikube-5452-timeline newer than 2020-06-17 11:15:29.210782 -0700 PDT m=+12.177519167
I0617 11:16:35.043964   75669 timeline.go:36] cache miss for kubernetes-minikube-6873-timeline newer than 2020-06-17 11:15:29.210782 -0700 PDT m=+12.177519167
I0617 11:16:35.792263   75669 timeline.go:36] cache miss for kubernetes-minikube-4752-timeline newer than 2020-06-17 11:15:29.210782 -0700 PDT m=+12.177519167
I0617 11:16:36.529062   75669 timeline.go:36] cache miss for kubernetes-minikube-8214-timeline newer than 2020-06-17 11:15:29.210782 -0700 PDT m=+12.177519167
I0617 11:16:37.244689   75669 timeline.go:36] cache miss for kubernetes-minikube-4805-timeline newer than 2020-06-17 11:15:29.210782 -0700 PDT m=+12.177519167
I0617 11:16:38.068430   75669 timeline.go:36] cache miss for kubernetes-minikube-7131-timeline newer than 2020-06-17 11:15:29.210782 -0700 PDT m=+12.177519167
I0617 11:16:38.541926   75669 timeline.go:36] cache miss for kubernetes-minikube-5188-timeline newer than 2020-06-17 11:15:29.210782 -0700 PDT m=+12.177519167

v1.2.0-beta: Use timelines/review data on first run if available

In v1.2.0-beta.3, we skip the timeline/review data on the first run as an optimization.

What would be better is to use the data if it's available, but not to fetch it (or fetch in the background).

Kanban dashboard does not reliably notice new PR's

If an issue is added to the dashboard...

Add result age to rendered pages

Implement "prioritized" filter (like created, but for when issue was prioritized)

This allows for us to accurately flag on older issues which were recently made a p0.

Periodically save state to initcache

At the moment, Triage Party only writes to the initcache at startup.

For larger repositories, it would be nice to have a periodic checkpoint to disk.

Support repo dependent rules

Hi, I would like to manage multiple repos in a single triage-party instance. Fortunately, triage party already support it:

settings:
...
  repos:
    - https://github.com/org1/repo1
    - https://github.com/org2/repo2

However, if I understood correctly, the current configuration scheme doesn't support repo dependent rules. For example, each repository has different SLOs.

Of course, we can prepare separate triage-party instances, But I would like to have only one triage-party site for efficiency (it would be easier for us to access one triage-party instance to perform triage for target multiple repos).

Perhaps, we can implement repo filter that accepts regex ([!]regex). Then user can define repo dependent rules like below:

p0-followup-overdue-repo1:
...
    filters:
      - repo: "org1/repo1"
      - label: "priority/p0"
      - tag: recv
      - responded: +3d
p0-followup-overdue-repo2:
...
    filters:
      - repo: "org2/repo2"
      - label: "priority/p0"
      - tag: recv
      - responded: +7d

Add filtering based on title regexp

404 Error for "moment-with-locales.min.js"

After deploying triage-party, I faced an error for fetching 'moment-with-locales.mn.js' in web browser's dev console.

And I found no moment assets in the third_party directory

Is 'moment-with-locales.min.js' really needed/used?

Skip timeline fetches for initial load

Starting up a kubernetes/kubernetes triage-party is too damned slow because of timeline fetches.

"Average age" summary is wrong - can go negative

As seen with kubernetes/kubernetes, Triage Party sometimes lists a negative number:

I believe I have also seen this in Skaffold.

FR: shard issues/PR's by label in multi-player mode

let say we have two users
one is windows expert
the other one is docker-driver expert

and we play triage party, we should be able to choose triage by "kind/docker" or "kind/hyperv" , "kind/multinode"

that we can users can play based on their expertise rather than get random set of issues.

a bug in recv state

This issue has an obvious response from the maintainers: quantumlib/Cirq#2799
But triage party determines it to be in recv state, as if I was the last one commenting.

Add /healthz endpoint

To make Kubernetes deployments easier.

Switching pages loses player information