jasonetco / ci-reporter

🚧📝 GitHub App to show CI failure logs in PR comments

ci-reporter

A GitHub App built with Probot that pastes the error output of a failing commit into the relevant PR

Usage

Simply install the app and watch the magic happen as your Pull Requests trigger failure statuses.

How it works

When a build fails, the CI provider will tell GitHub (via a status). GitHub then tells ci-reporter about a failed status, and it'll find the part of the build that failed, then comment back on the PR.

Configuration

You don't need any configuration for this to work in your project but you can customize a few things to fit your needs. You can create a .github/ci-reporter.yml file:

# Set to false to create a new comment instead of updating the app's first one
updateComment: true

# Use a custom string, or set to false to disable
before: "✨ Good work on this PR so far! ✨ Unfortunately, the [{{ provider }} build]({{ targetUrl }}) is failing as of {{ commit }}. Here's the output:"

# Use a custom string, or set to false to disable
after: "I'm sure you can fix it! If you need help, don't hesitate to ask a maintainer of the project!"

If you need more configuration, please let me know in a new issue.

Does it work with _____?

ci-reporter currently supports TravisCI and CircleCI. If you're interested in seeing support for another CI tool, please open an issue!

Contributing

So glad you want to contribute! If you're looking to help with a new feature, please open an issue to discuss it first. If you're helping fix a bug, let me know! Check out the contributing docs for more details.

ci-reporter's Issues

Support for AppVeyor

There are loads of github projects using https://www.appveyor.com/ so I'm surprised I seem to be the first one asking 😊

travis-ci report not working

The app doesn't seem to work with my project: I have made a PR where TravisCI failed but I got no message from ci-reporter.

This is the content of my ci-reporter.yml:

# Set to false to create a new comment instead of updating the app's first one
updateComment: true

# Use a custom string, or set to false to disable
before: "Unfortunately, the [{{ provider }} build]({{ targetUrl }}) is failing as of {{ commit }}. Here's the output:"

# Use a custom string, or set to false to disable
after: "If you need help with this issue, don't hesitate to ask a maintainer of the project!"

Support Scrutinizer-ci

Add support for shippable CI

Add support for https://app.shippable.com

Update previous comment

Rather than creating a new comment for every failing commit, it should update the original comment to show the new message.

Or, at the very least, update previous comments to hide the error log in a <details> element.

Please support bitrise

:)

Semaphore CI support

Semaphore is one of the top CI services, and I think it would make a great addition

Support for Jenkins

Hi,
We in our project are using Jenkins CI.
Can you please let me know , whether you are planning to add support for jenkins as well.
If you could you please suggest some other alternatives which can give ci-reporter functionality for Jenkins.

Thanks in advance.

Regards,
Vichar Sharma

Support Team City

Team City is heavily used and would be an excellent addition to the supported list for this tool

Support for Google Cloud Build

Docs https://cloud.google.com/cloud-build/docs/

Add config option for template

Parts of the comment template should be configurable.

The opening line, which is now:

✨ Good work on this PR so far! ✨ Unfortunately, the [{{ provider }} build]({{ targetUrl }}) is failing as of {{ commit }}. Here's the output:

And the closing line, which is now:'

I'm sure you can fix it! If you need help, don't hesitate to ask a maintainer of the project!

No CI reporter output on a PR

CI failed on this PR, but the reporter didn't comment on it: electron/apps#528

Not a big deal. Just wanted to report it in case it's useful for debugging.

Fold the comment if the build starts passing

If builds are failing, and ci-reporter is commenting, it'd be great to fold the original comment once the build starts passing again. That way there's no jarring "giant error message" if the PR is actually passing.

Want to publish the Automation Test Results executed in Azure Devops to team members through(Mail or Teams)

I have developed a testing framework for automating the UWP Application using appium.
Created the build pipeline to run the tests in VM in Azure Devops and able to see the Test Results and summary.

But wanted this results to publish to my entire team of 12 members as soon as the Test Automation in the VM completes through Mail or Microsoft Teams

Can anyone help me on this, if anyone feels any blog or reference would be useful for the above issue. I am reachable @ [email protected]

Do nothing in private repos

Because of CI permissions, if the repo is private then the ci-reporter won't be able to access the service's APIs for the builds. There should be a check in the status event handler to prevent acting on a private repo, which would incur errors.

Issue in setting up the bot

I tried installing the bot on my repo but its not posting any comment on a PR with failing builds. I am using CircleCI for the tests. I also created a config file but it ain't helping either. Am I missing any steps?
Thanks

support gitlab ci

Gitlab is a great ci tool to build

Seeing some weird output in snippet

Check out electron/apps#518 (comment)

It's got some stuff like �[92m �[0m�[32m 10180 passing�[0m�[90m (2s)�[0m in it.

One the bright side, this person was able to fix the issues on their own and get the build passing without intervention from our team! 💯

Travis message is sometimes empty

Not sure why, but here's an example of the Travis error log showing a blank message:

JasonEtco/failing-repo#8 (comment)

I've seen this a couple of times so I don't think it's a very specific edge-case. I'm guessing it's some RegEx issue, so I'm opening this to make a note to look into it.

docs/deploy.md not exist?

Cloud build support

Support AWS CodeBuild

Strip control codes

For test suites that use colorized output, comment on GitHub is a little painful on the eyes (e.g. probot/probot#441 (comment)).

It'd be nice if control codes we're automatically stripped: https://www.npmjs.com/package/strip-ansi

Add support for Codefresh CI

Add support for Codefresh CI.

Support Circle CI's wildcard contexts

Looking through some logs I'm seeing a lot of statuses from Circle CI with a context following this scheme:

ci/circleci: SOMETHING USER DEFINED

Right now, ci-reporter will ignore all of those. It'd be great if the check for known contexts was a little looser:

ci-reporter/src/providers/Circle.js

Lines 9 to 11 in 0911cef

	static get ctx () {
	return 'ci/circleci'
	}

ci-reporter/src/index.js

Lines 25 to 27 in 0911cef

	} else if (statusContext === Circle.ctx) {
	context.log(`Creating CircleCI instance for ${context.id}`)
	serializer = new Circle(context)

Would love to see support for CodeShip

Account for multiple jobs

ci-reporter/lib/providers/Travis.js

Line 58 in cbbc53e

module.exports = Travis

---

This issue was generated by todo based on a TODO: comment in cbbc53e. It's been assigned to @JasonEtco because they committed the code.

Codeship(basic) support

would love to use this app for fetching ci reports of codeship ci.
I'm mostly use basic plan

work with github actions?

An in-range update of webpack is breaking the build 🚨

Version 4.9.1 of webpack was just published.

Branch	Build failing 🚨
Dependency	webpack
Current Version	4.9.0
Type	devDependency

This version is covered by your current version range and after updating it in your project the build failed.

webpack is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Status Details

• ❌ continuous-integration/travis-ci/push The Travis CI build could not complete due to an error Details

---

Release Notes

v4.9.1

Bugfixes

• fix parameter references in default parameters

Internal changes

• change test cases to text format

Commits

The new version differs by 11 commits.

• 94cd709 4.9.1
• 5feb7e6 Merge pull request #7174 from JLHwung/revise-examples-build-readme
• 4cd0cf5 Merge pull request #7379 from xtuc/refactor-use-wast-in-tests
• c513cac Merge pull request #7403 from webpack/fix/7335
• f22fffd Evaluate arguments in function's scope
• 55ce143 Add test case
• 00eafa6 Update mem-access.wat
• acc45fd refactor: switch to wast
• 7184bb4 fix: keep decoding in wasm
• a9d9fee refactor: switch to tests to wast
• 64db306 docs(examples): add yarn add webpack-cli step

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.

---

Your Greenkeeper Bot 🌴

What is the behavior for numerous failing builds?

Hi! I think we should give this bot a go on electron/electron org, but would first like to understand what it does when it encounters multiple failing builds, like:

Will it comment on the failures for every build? If so, that could get pretty noisy.

Work for private repos

Currently, ci-reporter will not work for private repositories because the CI providers that private repos use require an API key for private builds. Here are the things that need to happen for private repo support to be possible:

• A UI (or some other method) to encrypt API keys
• Store encrypted API keys in the .github/ci-reporter.yml file
• Use those keys in API calls to CI providers

To be clear, I do not want to resort to using a database for these keys. There is too much overhead involved, and I'd like to nail down a good pattern for encrypting/storing keys in repositories since it'll have implications for many more apps.

Does it work with GitHub Actions?

Hi @JasonEtco! 👋

I see this in the README:

ci-reporter currently supports TravisCI and CircleCI.

I'm working on a project that is now using GitHub Actions to run its Jest test suite. Think this app will work for it?

cc @sarahs

Version 10 of node.js has been released

Version 10 of Node.js (code name Dubnium) has been released! 🎊

To see what happens to your code in Node.js 10, Greenkeeper has created a branch with the following changes:

• Added the new Node.js version to your .travis.yml
• The new Node.js version is in-range for the engines in 1 of your package.json files, so that was left alone

If you’re interested in upgrading this repo to Node.js 10, you can open a PR with these changes. Please note that this issue is just intended as a friendly reminder and the PR as a possible starting point for getting your code running on Node.js 10.

More information on this issue

Greenkeeper has checked the engines key in any package.json file, the .nvmrc file, and the .travis.yml file, if present.

• engines was only updated if it defined a single version, not a range.
• .nvmrc was updated to Node.js 10
• .travis.yml was only changed if there was a root-level node_js that didn’t already include Node.js 10, such as node or lts/*. In this case, the new version was appended to the list. We didn’t touch job or matrix configurations because these tend to be quite specific and complex, and it’s difficult to infer what the intentions were.

For many simpler .travis.yml configurations, this PR should suffice as-is, but depending on what you’re doing it may require additional work or may not be applicable at all. We’re also aware that you may have good reasons to not update to Node.js 10, which is why this was sent as an issue and not a pull request. Feel free to delete it without comment, I’m a humble robot and won’t feel rejected 🤖

---

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.

---

Your Greenkeeper Bot 🌴

hide comments when new commits are added to PR

As a PR progresses and the tests start to pass again, I often go back and manually hide old @ci-reporter comments. It would be cool if the bot could do this automatically.

Error logs are double-spaced

I've seen a couple of error logs coming back double-spaced; example: probot/probot#462 (comment)

Not sure why this is happening, but requires investigation to avoid spammy giant comments.

Azure Devops Support

Please add support for Azure Devops

TODO: Account for multiple jobs

ci-reporter/lib/providers/Travis.js

Line 64 in c935f05

module.exports = Travis

---

This issue was generated by todo based on a TODO: comment in c935f05. It's been assigned to @JasonEtco because they committed the code.

An in-range update of webpack is breaking the build 🚨

☝️ Greenkeeper’s updated Terms of Service will come into effect on April 6th, 2018.

Version 4.1.0 of webpack was just published.

Branch	Build failing 🚨
Dependency	webpack
Current Version	4.0.1
Type	devDependency

This version is covered by your current version range and after updating it in your project the build failed.

webpack is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Status Details

• ❌ continuous-integration/travis-ci/push The Travis CI build could not complete due to an error Details

---

Release Notes

v4.1.0

Features

• add filename option to optimization.splitChunks to modify the filename template for splitted chunks
• allow modules which doesn't emit code into the bundle

Bugfixes

• watchpack updated to 1.5.0
• performance fix for Module Concatenation (v8 bug)
• fix using this.xxx in ProvidePlugin

Commits

The new version differs by 73 commits.

• f916fc0 4.1.0
• 8eba694 Merge pull request #6650 from webpack/bump_watchpack
• 00f70fc Merge pull request #6645 from zacanger/fix/6644
• 2e3d319 Merge pull request #6648 from cheapsteak/patch-1
• 01c18cc Merge pull request #6651 from webpack/feature/split-chunks-filename
• 2e687d0 Merge pull request #6613 from brentwilton/improve-performance-of-module-concatenation-plugin
• 3c5b104 Merge pull request #6663 from webpack/feature/support-non-js
• 2c8ea60 expose stuff and all non-bundle modules
• b0e14df Set optimization.splitChunks to false in test webpack config.
• ecb65aa allow to configure filename for splitted chunks
• 325038d Update watchpack to 1.5.0
• a2fd80e Update downloads badge to point to graph of downloads
• 3a41ca4 Fix #6644
• 39095ef add todo for ModuleConcatenationPlugin for loop bugfix
• f6e366b Merge pull request #6611 from kvrlk/patch-1

There are 73 commits in total.

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.

---

Your Greenkeeper Bot 🌴

GitHub Actions

Does this work with GitHub actions?

Azure pipelines support

Please add support for Azure pipelines