Consider merging avr-100-pin branch into master about megacore

[per1234]

Sounds like a good idea to add Travis CI. I haven't found any new bug with my testing system for 5+ days now and have been using it a lot so I'm feeling more confident in it. I haven't really focused on using Travis with hardware packages other than the test I did on MCUdude_corefiles. I'm going to think about if there are any common tasks related to testing hardware packages that I should add to my script. I'm also planning to add a function to automate the Boards Manager JSON update process but that's a lower priority and would probably be specific to the gh-pages branch.

Here's my preliminary plan for testing:
Compile every example sketch in the libraries included with MegaCore for every board using every supported version of the Arduino IDE.

Do you have any sketches other than those that you want Travis to compile?

[MCUdude]

Do you have any sketches other than those that you want Travis to compile?

Nope, I think that's all. Ideally I want to get rid of the duplicate libraries and merge all MegaCore related dependencies into the official libraries. Haven't gone that path yet, but I think may be worth it in the long run. My only concern is that the official library almost seems to be abandoned. They all have lots of old unanswered issues and PRs.

[per1234]

I have now set up a Travis CI configuration for the MegaCore avr-100-pin branch:
https://github.com/per1234/MegaCore/blob/travis-ci/.travis.yml

Unfortunately the preliminary build revealed some problems:

• With Arduino IDE 1.6.3 (the oldest tested) to 1.6.5-r5 attempts to include the MegaCore library source files are instead either picking up the Arduino AVR Board library files or failing if the file doesn't exist in Arduino AVR Boards I can't reproduce this on Windows so it seems to be Linux specific (and maybe MacOS also). Are you using one of those OSs? I don't see a good solution for this. As far as I can tell it's an Arduino IDE bug, not a bug with the Travis system or MegaCore. The options I can see are:
  • Only test with Arduino IDE versions 1.6.6 and newer. This will also make the build time shorter.
  • Install the MegaCore libraries to the sketchbook/libraries folder or overwrite the Arduino AVR Boards libraries with the MegaCore libraries. I'm not crazy about this idea because this causes the test conditions to be different from what will occur on the user's system.
• Many function prototype generation failures with Arduino IDE 1.6.6. There are two options for solving this:
  • Don't test with Arduino IDE 1.6.6.
  • Make all the example sketches compatible with Arduino IDE 1.6.6 by adding the necessary function prototypes. Easy enough but it might make the examples more difficult for beginners to understand. Of course examples that don't compile are more confusing but I don't think many people are using 1.6.6. It seems more likely they would either have stayed with 1.6.5-r5 or older or jumped to a newer IDE version that doesn't have all the bugs caused by the newly introduced arduino-builder.
• Build exceeds the maximum time allowed by Travis CI for a job. The job timed out after 50 minutes while still testing and didn't even make it through the first MCU. Possible solutions:
  • Break the build into multiple jobs (each of which is allowed 50 minutes max). The downside to this is that each job has its own report so it's a bit more work to grab multiple reports and combine them if you want a unified report.
  • Simplify the test procedure. This could be done by testing with less IDE versions or less board configurations.

So let me know how I should proceed.

[MCUdude]

I just installed IDE v1.6.5r5 on my mac and renamed the arduino15 folder. It builds just fine, so I think it may be a linux related issue. Anyways I've clearly written in the readme file that IDE 1.6.12 or newer is recommended, so if someone complains about this issue they just have to update their IDE to a newer one.

I suggest only testing on IDE 1.6.7 and newer. What's important here is that it's compatible with the newest IDEs.

[per1234]

I think I'm close to having a working configuration. Unfortunately I had to break it into a bunch of jobs to keep the build from timing out. I wish I could figure out a good way to combine the reports.

I think it would be a good idea for you to enable Travis CI for your repository before I submit the pull request so the PR will be tested on your repository to make sure it's set up correctly. Travis CI is free for public repositories.

• Go to https://travis-ci.org
• Click "Sign Up"
• After you have an account you should see a list of your repositories, or maybe you need to click the "+" here:
  
• Then enable Travis CI for MegaCore by clicking the left side of the switch next to MegaCore on the repository list:
  
  
• I also like to click the gear icon and turn on the option "Build only if .travis.yml is present". Not sure if that matters.

After that, Travis will automatically build any commit or pull request to any branch of MegaCore as long as there is a .travis.yml file in the root folder. The current configuration in the .travis.yml file I wrote will cause you to get an email whenever a build succeeds or fails:
https://github.com/per1234/MegaCore/blob/travis-ci/.travis.yml#L166-L169
but you can change whether you get notifications and you have a variety of options for how to get them:
https://docs.travis-ci.com/user/notifications/

If you like you can add a build status badge to your README.md:
https://docs.travis-ci.com/user/status-images/
for example:

[per1234]

OK, 7 hours and 23000 compilations later the full build has completed!
https://travis-ci.org/per1234/MegaCore/builds/229800446
I had to break it into 52 jobs and some took up to 45 minutes, so they are getting close to the job time limit of 50 minutes.

The build failed but the errors were not caused problems with the testing system or the IDE. Most were legitimate bugs with MegaCore:

Blink_using_Timer0:33: error: invalid preprocessing directive #elseif
   #elseif defined(__AVR_ATmega1281__) || defined(__AVR_ATmega2561__)
    ^

/home/travis/Arduino/hardware/MegaCore/avr/libraries/SD/src/utility/Sd2PinMap.h:20:0: error: unterminated #elif
 #if defined(__arm__) // Arduino Due Board follows
 ^

/home/travis/Arduino/hardware/MegaCore/avr/libraries/SoftwareSerial/src/SoftwareSerial.cpp:317:36: error: 'digitalPinToPCICR' was not declared in this scope
   if (digitalPinToPCICR(_receivePin)) {
                                    ^

The only problem with my Travis configuration that was revealed is:

/home/travis/Arduino/hardware/MegaCore/avr/libraries/SoftwareSerial/src/SoftwareSerial.h:45:4: error: #error "ATmega64 and ATmega128 doesn't support SoftSerial!"
   #error "ATmega64 and ATmega128 doesn't support SoftSerial!"
    ^

So I need to change the configuration so that it doesn't compile the SoftwareSerial examples for ATmega64 and ATmega128.

Full report is attached:
megacore_build_14_report.zip

[MCUdude]

This is great, thank you!

I just created an account and followed you instructions of how to add MegaCore to travis. Do you have a .travis.yml file that I should merge, which contains the current build setup?

[per1234]

It's here:
https://github.com/per1234/MegaCore/blob/travis-ci/.travis.yml
but I still need to change it so that it doesn't compile the SoftwareSerial examples for ATmega64 and ATmega128 since that library states it's not compatible with those MCUs, which will cause the builds to fail even when there are no legitimate bugs. I'm a bit stumped on how to do this cleanly without adding a ton more jobs. The reason I don't like lots of jobs is because if you want a unified report it's a huge task to grab the report from every job and combine them. I find it useful to have the results of every compilation in a single spreadsheet so I can sort it any way I want, then use that information to determine if I need to look at the build logs for more details, such as exact error or warning message.

So I'd recommend waiting to merge until I've found some sort of solution to the SoftwareSerial on ATmega128/64 problem. The legitimate bugs found by the test can be fixed separately but I'd like the test configuration to be correct before it's merged. You could merge to a test branch if you want to play around with it. Maybe you can find a good solution to these issues I'm having. I have no prior experience with Travis CI, Linux and shell scripts so there may an obvious solution I'm overlooking.

[MCUdude]

Thanks, I'll wait until the issues are sorted out. Meanwhile I'll fix the bugs Travis found while building everything. This is really useful, as I'd never spot them myself.

[per1234]

Yeah, with a project like this, with so many different variables to test, doing so manually is impossible, automation is the key! I knew there were a lot of combinations but I was amazed that it ended up being over 23000 and that was after limiting the IDE versions to 1.6.7 and newer. I'm confident that these tests give reasonably good coverage of the code. The main thing it's missing is a check of whether the bootloader files for all boards.txt configurations are present. The IDE does print a message but it's not an error so the script doesn't currently fail the build if that happens. I originally had more tests of the individual boards configuration options but ended up combining them with other tests where possible because they were causing the builds to time out. It's a difficult balance between full coverage and a reasonable build time. As it is now having to wait over 7 hours to see the results of a change is quite inconvenient.

[per1234]

Ok, I'm back. Sorry for the delay. It's slow going with these long build times. Sometimes Travis CI is running slower than others so jobs that were just under the maximum time were going over and getting terminated. I finally have it broken into enough jobs where it doesn't time out but some of the jobs are still a bit close for comfort so I think I should split it up a little more to leave some room to grow.

It's currently at 149 jobs and will be around 170 after I break it up more. The problem this presents is that it would be extremely tedious to try to manually copy that many reports together into one spreadsheet after every build. I wanted a solution that didn't require the user to sign up for another account at a different website. I know they already have a GitHub account so I had to figure out how to do it here. Travis CI has a system for deploying the build to GitHub pages but it force pushes every job to your repository, which erases the report from the last job. So that's completely worthless. The solution I settled on is to create a gist for the reports and have the script automatically add each report to the gist. It also adds a link to the reports in a comment on the commit, this is optional. GitHub displays the tab separated report as a searchable table. The entire gist can be easily downloaded so it's a much easier to put together a unified report, though you still have to concatenate the files. One major limitation is that, for security reasons, Travis CI doesn't allow the use of secure environment variables for pull requests from forks. This makes sense because otherwise someone could change the .travis.yml to do whatever they like with your GitHub personal access token and then run it by submitting a pull request. So the only way for reports from builds of commits to forks to be published is if the user generates a token and adds it as a secure environment variable in their Travis CI settings. Once the PR is merged to your repository the Travis CI build is run again and that build is able to publish a report as usual as it has access to your token.

Here's the build:
https://travis-ci.org/per1234/MegaCore/builds/230812773
Here's the automated comment:
https://github.com/per1234/MegaCore/commit/c218f1673a543dfe991ce95e9a1f05fd08c42a83#commitcomment-22089074
Here's the gist:
https://gist.github.com/per1234/1b005ec33dc8cdbde14dd2dedfb083a5
The unified report is attached:
travis_ci_job_report_24.zip

I merged your recent commits into my fork so now it's just errors compiling the SoftwareSerial examples that are causing the build to fail:

/home/travis/Arduino/hardware/MegaCore/avr/libraries/SoftwareSerial/src/SoftwareSerial.cpp: In member function 'void SoftwareSerial::begin(long int)':

/home/travis/Arduino/hardware/MegaCore/avr/variants/100-pin-avr/pins_arduino.h:100:80: error: operands to ?: have different types 'volatile uint8_t* {aka volatile unsigned char*}' and 'bool'
                                 ((((p) >= 77) && ((p) <= 84)) ? (&PCMSK2) : 0))))
                                                                                ^

Unfortunately I've found the gist report solution has some problems that only showed after doing a large build:

• Gist doesn't want to display the later reports as tables, I guess it has a limit to how much data it will render.
• Gist only shows the first 99 files. All 149 are there though. That will be more of a problem with later builds when none of the report files for the build even show up. If you load an individual file from the gist it doesn't show in table form.

These problems make me think that automatically committing to a branch of the repository (or a dedicated repository) would be a better solution. This would allow each build to be put in a separate folder (folders aren't allowed in gists), The reports would be shown as a list of files and then when you open any file it displays as a table.

I'm going to add a function to my script for committing the report to a branch of a repository. I'll leave the one for the gist report, which might be a better option for smaller builds. So let me know what your preference is for the handling of the reports or if you have any ideas for another solution. I'm trying to make the script something that's generally useful for any Arduino project so any input from people who will be using it is helpful to me.

The other question I have for you is how you would like to handle the use of my script by MegaCore. I wrote a script called arduino-ci-script.sh that attempts for provide functions for any common task necessary for testing Arduino projects:
https://github.com/per1234/arduino-ci-script
This script is used in the MegaCore .travis.yml file. Currently I have Travis CI cloning the script from my repository and using the tip of the master branch. That's nice because it automatically picks up any changes I make to the script but it also presents some issues:

• Changes I make to the script could unintentionally break your build.
• If you're using the report publishing functions you are passing a GitHub personal access token to a script not under your control. This is a security issue because I could change the script to do nefarious things with your token without you noticing.

The possible solutions:

1. Add a copy of the script to the MegaCore repository. The downside of this is it's a little more difficult to keep the script current with any improvements I might make later.
2. Clone the script from my repository and checkout a specific commit. I believe this would solve the security issue because the commit is a hash so it would be extremely difficult if not impossible to modify the script in a useful manner without changing the hash.
3. Add the script repository as a subtree. This ensures you have a copy of the script at a specific point in the commit history but allows you to update it using git commands.

So let me know your preference for the handling of the script, or if you have another idea.

[per1234]

Just to make the question about your preferences for report publishing more clear:

Report publishing options:

1. Commit reports to a branch of the MegaCore repository dedicated to reports
2. Commit reports to a repository dedicated to reports (could have a branch for reports from each of your repositories).
3. Publish reports to gist, as done in this build.
4. Don't publish reports.

Report link comment options:

1. Automatically comment on each commit that is built by Travis CI with a link to the reports.
2. Don't do that.

[MCUdude]

Thanks for all the hard work trying figuing this out. I really appreciate it, and I'm sure lots of Github/Arduino devs will benefit from you effort here 🔢

I need to be honest and ask; what do you think is the best option here? I understand that passing my token to a script I don't have control over is not a good idea. In my simple head the subtree option seems like a good idea. Then I have a physical copy of your script in the MegaCore repo, and can easily update it to the latest version. I don't believe there will be any security issues here either. I still need to manually add the .travis.yml at the root, right?

Let's say we go for the subtree option. The procedure will go like this, right?

• Add the subtree (copy) of your ci-script
• add .travis.yml to the root (?)
• Change the settings over at travis-ci.org to enable the CI

Are there anything I need to be aware of or take into account when I'm pushing new code to the MegaCore repo? Will it take 7 hours+ before I know if I broke the build or not? Is there anything "Pull requesters" need to be aware of? Anything I need to be aware of when merging a PR, except from the travis-ci script and the .travis.yml file?

(Sorry if I've been a little slow with the messages lately. I've been quite busy, and it takes a little more time expressing myself in english rather than norwegian)

[per1234]

I hope it will be useful to others. I started by wanting to do it for one of my repos, then decided I should do it for all my repos while I was at it. I realized that I was going to end up with a lot of duplicate code to maintain so I came up with the idea of the centralized script. You're having to go through the growing pains with me of finding all the issues with scaling up my original concept to this more demanding repository but once the kinks are worked out I think it will make these tasks easier for any other repo it's added to.

I like the idea of the subtree. It makes sense that the script should be in this repository so it's completely self contained and not dependent on the existence of another repository. The clone option would make updating the script a little easier but you're already using sutrees in this repository so I don't think that's an issue.

• Add the subtree (copy) of your ci-script

Correct, just let me know what folder(s) you want it in and I'll set that up before I do the pull request.

• add .travis.yml to the root (?)

Yes, it must be in the root of the repository

Change the settings over at travis-ci.org to enable the CI

I think you already have done that following the instructions above but if you want the reports to be published you will also need to do the following:

1. Generate a GitHub personal access token
2. Create a secure environment variable that contains the token in your Travis CI settings for the repository.
3. Create a branch, repository, or gist for the reports to be published to.
4. If the reports are to be published to a gist or separate repository then it would make sense to configure that as an environment variable in your Travis CI settings for the repository so that forks can configure their own report location if they want. That doesn't need to be a secure variable because it's not secret information.

I have step by step instructions for doing the above processes in the readme for my script but I'm adding a function to allow publishing reports to a repository to the script now so the instructions will soon be updated to reflect that change.

Are there anything I need to be aware of or take into account when I'm pushing new code to the MegaCore repo?

In some cases you may want to add new tests to MegaCore's .travis.yml file to cover the new code:

• If you added a new board or a new board configuration option you would want to add more jobs
• Most of the boards will automatically test all sketches in the libraries folder but I had to specify each individual library folder to search for the ATmega128 and ATmega64 boards since they are not compatible with SoftwareSerial so if you add or remove a library you will need to update the configuration for those boards.
• If you add a new example to the libraries that will automatically be found so you don't need to do anything.
• If you discover a bug that was not covered by the current tests you can update the configuration to make sure that sort of thing will be found in the future.

Will it take 7 hours+ before I know if I broke the build or not?

Unfortunately, yes. I got the build down to "only" 15000 compilations (from 23000) but the build time didn't go down. It may be that there is some overhead to each job so the reduction in compilations was offset by the increase in jobs or maybe Travis CI was running slow or they've throttled my account since I''m running so many builds.

Possibilities for reducing the build time:

• Merge your changes to some of the libraries upstream and remove them from MegaCore
• Test a smaller range of IDE versions.
• Remove unnecessary jobs. I've tried to trim it down as much as possible already. The only thing I might have been excessive with is doing the full test for each of the clock speeds. I did this because it's possible there is an error that could occur with specific values of F_CPU. I don't do the full test for the 8 MHz internal option because that F_CPU value was already covered by the 8 MHz external test, I only do compilation of BareMinimum.ino for each IDE version and MCU for the 8 MHz internal option to check the boards.txt configuration so those jobs are much faster than the rest, which have to compile every library example.

Is there anything "Pull requesters" need to be aware of?

• If they want to run the Travis CI build on their fork before submitting the PR they will need to:
  • Get a Travis CI account if they don't already have one.
  • Turn Travis CI builds on for their MegaCore repository
• If they want report publishing for the tests in their fork and the PR they will need to:
  • Generate a GitHub personal access token
  • Configure a secure environment variable containing the token in their Travis CI settings for MegaCore.
  • Depending on which method of report publishing you chose they might also need to do some additional work:
    • Reports published to a branch of MegaCore: No additional work.
    • Reports published to a separate repository: Create the report repository.
    • Reports published to a gist:
      • Create a gist
      • Configure an environment variable containing the gist ID in their Travis CI settings for MegaCore.

Anything I need to be aware of when merging a PR, except from the travis-ci script and the .travis.yml file?

Depending on the changes made in the PR, additional jobs might need to be added to the .travis.yml file to ensure full testing coverage, as explained above. Ideally the PR would add any necessary tests but it's something that's easy to forget.

(Sorry if I've been a little slow with the messages lately. I've been quite busy, and it takes a little more time expressing myself in english rather than norwegian)

No worries! You're certainly better in English than I am in Norwegian! Sometimes I feel guilty that everyone else in computer land has to learn my native language.

[MCUdude]

Depending on which method of report publishing you chose they might also need to do some additional work:
Reports published to a branch of MegaCore: No additional work.
Reports published to a separate repository: Create the report repository.
Reports published to a gist:
Create a gist
Configure an environment variable containing the gist ID in their Travis CI settings for MegaCore.

You've used Gist for this, right? I've not used Gist before, but if understood correctly it lives its own life, and does not affect any of the "ordinary" Github repos, right? It seems like the most "clean" option, because it live on it's own site. I assume that when it's first set up correctly it just "works" and doesn't need to be tweaked/changed if I'm doing changes to .travis.yml or the CI script?

I'll take a closer look it it this evening

[per1234]

I used Gist to publish reports for the last build of MegaCore:
https://gist.github.com/per1234/1b005ec33dc8cdbde14dd2dedfb083a5

Gists are kind of like a piece of scratch paper. Say you needed to share some code with someone but it wasn't important enough to create a repository. Actually they're pretty powerful. They have a version history and can display markdown, csv, etc. They have comments. What I didn't realize before is they are actually repositories, you can clone them, commit, push, etc. but they don't support PRs or other collaboration, no issue tracker, etc. I've only used them for a couple things helping people on the forum but it's a useful tool sometimes. They even allow people to create anonymous gists without a GitHub account!

I agree a gist seems the most clean. It seemed the best option for publishing the reports when I started the process. I don't really like the idea of having another branch of the repository that Travis CI is automatically committing to. I like the idea of having a separate repository for reports even less.

Everything was looking very good for the gist approach until the MegaCore build of 150 jobs revealed some issues:

• gists only show the first 100 files. They're all there but the others are not shown on the gist page
• Not all the report tables are displayed in the gist. It's possible I have found a solution for this but I haven't tested it yet.
• It's not very convenient to navigate through a gist with lots of files. The file navigation system of a repository will make it much easier to browse through a lot of reports.

I think the gist feature of my script is still useful for some applications but MegaCore may not be a good match.

The easiest option for a fork to get report publishing to work is if you use a branch of MegaCore for the reports. That way they only need to generate a token and configure it. I'm not sure that's an important consideration though because a lot of people who submit a PR probably won't bother with the full testing (though they really should), and creating a gist and configuring Travis CI with the gist ID will only take a couple minutes extra and is very easy following the instructions I provide in my readme.

[MCUdude]

I understand that MegaCore might be to big for Gist. So, what do you think is the second best option? Maybe a dedicated branch Travis-build-outputs with a master branch with some info, and dedicated branches MegaCore, MightyCore, MiniCore and MajorCore? I don't like the idea of letting travis automatically commit to a branch either. How does the guys over at the esp8266 repo solve this?

[per1234]

The only downside I see with a repository dedicated to reports is that if some one wants to publish reports from Travis builds on their fork they will have to create that repository but I don't think that will be a very common occurrence and it's very fast and easy to do.

ESP8266 has a similar system in that they have a script that searches through the libraries folder and compiles every example sketch. They only compile for a single board with the hourly build of the Arduino IDE so it's a much less thorough test and it all fits in a single 17 minute job. They do have a report of the compile sizes of the sketches but from what I can tell it's only printed to the log. That's easy enough to work with when you have only one job per build. They are actually using arduino-builder to compile instead of the IDE. I had considered that because it might be a little faster but that means you can only test IDE versions 1.6.6 and newer and it doesn't reproduce the environment of the user as well. I have though about making the script using arduino-builder for IDE versions 1.6.6 and newer but I'd have to do some speed tests to see if it's worth all the extra complexity. I guess even a the difference of a second is huge when you're doing 15000 compilations per build!

ESP8266 is using a service called codecov.io to show the coverage of their tests:
https://codecov.io/gh/esp8266/Arduino/commit/7de81270a35e3867a5593e44c11f85a5c4b93e09/graphs/sunburst
It's a pretty cool thing, I don't fully understand it. I wanted to avoid the need for users of my script to sign up for any additional accounts but from what I can tell there's no need to integrate any part of my script with codecov at all, you just sign up for an account and add a line to your .travis.yml:
https://github.com/esp8266/Arduino/blob/master/.travis.yml#L47

 - bash <(curl -s https://codecov.io/bash) -X gcov

[MCUdude]

The only downside I see with a repository dedicated to reports is that if some one wants to publish reports from Travis builds on their fork they will have to create that repository but I don't think that will be a very common occurrence and it's very fast and easy to do.

If contributors want to take advantage of Travis CI, I think wouldn't be a problem. I think I might go for this option, if you're OK with that.

BTW i got the ATmega128 dev PCB in the mail the other day. Will sonder a few of them and ship one to you soon 🙂

[per1234]

Do you want the script to automatically comment a link to the report on each commit that is built with Travis CI?

BTW i got the ATmega128 dev PCB in the mail the other day. Will sonder a few of them and ship one to you soon 🙂

Awesome! Thanks so much. I really appreciate it.

[MCUdude]

I got some spare time today, so I'll dedicate some hours for this.

Do you want the script to automatically comment a link to the report on each commit that is built with Travis CI?

Like a link to the complete build output in the commit? I guess it's useful to have? If it isn't too much hassle though.

Add the subtree (copy) of your ci-script

Correct, just let me know what folder(s) you want it in and I'll set that up before I do the pull request.

Maybe a folder travis-ci on the root?

Should I start off by creating the subtree of your CI script, or can a subtree be added from a PR?

[MCUdude]

I've now added my personal Github token to the travis webpage using the guide over at your arduino-ci-script repo. I've also created a CI-reports repo with a branch for MegaCore. I guess what's left now is adding your script, adding the .travis.yml and a nice little build badge in the readme.

[per1234]

Like a link to the complete build output in the commit? I guess it's useful to have? If it isn't too much hassle though.

No trouble at all. The publish_report_to_repository function in the script has a parameter to turn that on or off so it's easy to turn that behavior on or off by only editing .travis.yml:
https://github.com/per1234/MegaCore/blob/travis-ci/.travis.yml#L416
It's turned off for the current build but I'll turn it on before I submit the PR.
I found it more useful with the gist. With the dedicated repository it's pretty easy to navigate to the report for the build but you do have to go to Travis CI to find the build number so this saves that trouble. Here's an example of the comment (on a commit in the arduino-ci-script repository):
per1234/arduino-ci-script@e6c1b9b#commitcomment-22126773
I created a GitHub "machine account" for commenting and committing and used the token from that account in my Travis CI settings, so that's why the comment shows as being made by "arduino-ci-script-bot". Since you used a token from your MCUdude account your comments will show as being made by MCUdude. The only thing I though was a bit questionable about the automated commenting is that it may be a bit more noisy for people following the repository. On the other hand I have thought it would be nice to get a notification for every commit on some repositories I watch, rather than only for PRs, issues, and comments because I don't notice when there's been new development.

OK, so the script location will be travis-ci/arduino-ci-script.sh. I currently have it at arduino-ci-script/arduino-ci-script.sh in my fork:
https://github.com/per1234/MegaCore/tree/travis-ci/arduino-ci-script
But will move it before I submit the PR.

I've also created a CI-reports repo with a branch for MegaCore

Oops. I had originally set it up to use the Travis-build-outputs repository as you had requested but I guess I accidentally changed it when I did a copy paste from one of my repositories, which use the CI-reports repository. If you prefer Travis-build-outputs I can change it. I decided on the more generic CI-reports for my repositories because I though I might eventually want to use an additional CI service, such as codecov.io, and maybe would also want to use the report repository to store files from those services..

Should I start off by creating the subtree of your CI script, or can a subtree be added from a PR?

what's left now is adding your script, adding the .travis.yml and a nice little build badge in the readme.

I'll submit a PR with everything ready to go. You've already done all the necessary configuration that it should correctly build your repository as soon as the PR is merged (the PR will build my fork using my token and report repository). I have a build running now of a .travis.yml configuration that I'm happy with. It hasn't revealed any fresh bugs in the script so I think I'm ready to submit the PR as soon as it finishes:
https://travis-ci.org/per1234/MegaCore/builds/231827228

[MCUdude]

I decided on the more generic CI-reports

I thought that too, so I decided to go for CI-reports myself.
I don't think anyone want to follow the CI-reports repo, as it will indeed get kinda "noisy" with all the commenting and such.

About the .travis.yml file. I'd like to get an email if the build fail, but I don't need to be notified if the build succeeded.

Again, thanks for doing this for me, and the rest of the community. I'll play around with this tonight while watching Eurovision 🇳🇴

[per1234]

I don't think anyone want to follow the CI-reports repo, as it will indeed get kinda "noisy" with all the commenting and such.

Comments would be made to the commits on the MegaCore repository. I believe this will cause anyone following MegaCore to get an email for every Travis CI build, which will happen every time you push commits (if you push multiple commits at once it only builds the most recent one).

About the .travis.yml file. I'd like to get an email if the build fail, but I don't need to be notified if the build succeeded.

OK, will do.

[MCUdude]

Comments would be made to the commits on the MegaCore repository. I believe this will cause anyone following MegaCore to get an email for every Travis CI build, which will happen every time you push commits (if you push multiple commits at once it only builds the most recent one).

Hmm. The author of the ArduinoJSON library uses multiple CI's. Travis or Codecov are commenting on every commit. Is is possible to disable these automatically generated messages? All the outputs can be found in the CI-reports repo anyway, right?

[per1234]

Travis or Codecov are commenting on every commit.

Followers won't get emails for those, they're not really comments. I don't know what those are called.

All the outputs can be found in the CI-reports repo anyway, right?

Correct. I set it up to put the reports from each build in a separate folder so it's easier to find the right report. Without the comment the process for finding the reports from the build would be:

1. Click on the check mark or X shown by the commit in the commit history, this will load the build log on Travis CI.
2. Note the build number.
3. Open the folder matching that build number in the MegaCore branch of the CI-reports repository.

The build just finished! You can see the reports here:
https://github.com/per1234/Travis-build-outputs/tree/MegaCore

So I'll go ahead and submit a pull request now.

[MCUdude]

So I'll go ahead and submit a pull request now

Great! Do the PR to the avr-100-pin branch, and I'll merge that one into the master very soon 🙂