Also set label when setting error status about app

[gr2m]

That’s not possible I’m afraid, it would need to be implemented. It’s fairly simple to build you own GitHub app though and use wip-bot as a template. You can deploy it to Glitch and edit your code right there, just follow instructions at https://probot.glitch.me :)

Note that you can also search PRs based on their commit status, maybe that helps already?
https://help.github.com/articles/searching-issues-and-pull-requests/#search-based-on-commit-status

[rhwood]

I know a PR would need to be submitted. I was wondering if such a PR would, if accepted, require that every user of the bot reauthorize the bot?

[gr2m]

yes, people will get notifications about a change of permission and they woudl need to reauthorize the bot. But I would not accept such a PR at this point, I’m sorry. But happy to help you deploy your own bot instead :)