Link time optimization doesn't work about minicore

[per1234]

In platform.txt change:

compiler.ar.cmd=avr-ar

to:

compiler.ar.cmd=avr-gcc-ar

[MCUdude]

Is this something that should be done with the other cores as well?

This fix works perfectly with IDE 1.6.12, but I get this error message when trying with 1.6.5 and 1.6.9:
sorry - this program has been built without plugin support

Any ideas on how to get it working with older versions as well?

[per1234]

This is how SpenceKonde did it in ATTinyCore:
https://github.com/SpenceKonde/ATTinyCore/blob/master/avr/boards.txt#L34
and
https://github.com/SpenceKonde/ATTinyCore/blob/master/avr/platform.txt#L29
That will cause compiler.ar.cmd to be avr-ar with LTO turned off and avr-gcc-ar with LTO turned on.

I would think it would be best to do this with all the cores that you add LTO support to since that's what Arduino AVR Boards did. I don't fully understand the difference between the two. There's some information here: arduino/Arduino#660 (comment)

[MCUdude]

hmm. I can make it work with IDE 1.6.5 and 1.6.12, but not at the same time. When I choose these boards.txt settings, and add compiler.ar.cmd=avr-{ltoarcmd}ar to the platform.txt file it works perfectly with IDE 1.6.12

# Compiler flags
328.menu.flags.Os_flto=-Os -flto (default)
328.menu.flags.Os_flto.compiler.c.extra_flags=-Wextra -flto
328.menu.flags.Os_flto.compiler.c.elf.extra_flags=-w -flto
328.menu.flags.Os_flto.compiler.cpp.extra_flags=-Wextra -flto
328.menu.flags.Os_flto.ltoarcmd=gcc-

328.menu.flags.Os=-Os
328.menu.flags.Os.compiler.c.extra_flags=
328.menu.flags.Os.compiler.c.elf.extra_flags=
328.menu.flags.Os.compiler.cpp.extra_flags=
328.menu.flags.Os.ltoarcmd=

however, with this boards.txt configuration I get this error with the LTO option enabled:
sorry - this program has been built without plugin support

If I change the LTO like to this (and just leave the platform.txt like it is):
328.menu.flags.Os_flto.ltoarcmd=

it compiles without errors in IDE 1.6.5, but the I get the huge error from the frist post with IDE 1.6.12. Is it really possible to make it work on both versions?

[per1234]

Using that modification to platform.txt and boards.txt I can successfully use either the Tools > GCC flags > -Os -flto(default) or the Tools > GCC flags > -Os options with IDE 1.6.12/Arduino AVR Boards 1.6.14. With IDE 1.6.5/AVR Boards 1.6.8 I can successfully use Tools > GCC flags > -Os but not Tools > GCC flags > -Os -flto(default). These are the results I'd expect as the AVR-GCC 4.8.1 included with AVR Boards 1.6.8 doesn't support LTO. I can successfully use either option with IDE 1.6.5 and AVR Boards 1.6.14. Are these the results you're getting?

On a side note, after thinking about it more I'm not actually crazy about the compiler.ar.cmd=avr-{ltoarcmd}ar thing because it just seems more confusing than necessary. I think it would be more clear to use compiler.ar.cmd={arcmd} and the full command name in the boards.txt menu options. Either one works just as well so I guess it really doesn't matter.

[MCUdude]

Using that modification to platform.txt and boards.txt I can successfully use either the Tools > GCC flags > -Os -flto(default) or the Tools > GCC flags > -Os options with IDE 1.6.12/Arduino AVR Boards 1.6.14. With IDE 1.6.5/AVR Boards 1.6.8 I can successfully use Tools > GCC flags > -Os but not Tools > GCC flags > -Os -flto(default).

That's the exact same result as I'm getting with IDE 1.6.5 and Boards 1.6.8. The problem is that boards 1.6.14 doesn't ship with IDE 1.6.5, so the user will have to manually replace the platform.txt file in order to use anything less than IDE 1.6.10.
I was thinking about making LTO default, since it reduce code size dramatically (fixing bad programming habits), while not making the code "slower". IIRC it's enabled by default since IDE 1.6.10.

These are the results I'd expect as the AVR-GCC 4.8.1 included with AVR Boards 1.6.8 doesn't support LTO.

Hmm. Is it true that GCC 4.8.1 doesn't support LTO? If I use avr-ar on both options using IDE 1.6.5, it seems like LTO is enabled. If I use -Os -flto a blank sketch compiles to 446b, while the -Os option compiles to 450b.
The main issue is that I need to use avr-ar when compiling with IDE 1.6.5 with LTO, and avr-gcc-ar when compiling with 1.6.12 with LTO.

On a side note, after thinking about it more I'm not actually crazy about the compiler.ar.cmd=avr-{ltoarcmd}ar thing because it just seems more confusing than necessary. I think it would be more clear to use compiler.ar.cmd={arcmd} and the full command name in the boards.txt menu options. Either one works just as well so I guess it really doesn't matter.

I agree. I just copied the line from ATTinyCore. I'll add the full command name to the boards.txt file instead. It doesn't need to be confusing just because it works!

[per1234]

Is it true that GCC 4.8.1 doesn't support LTO?

That was my understanding previously but after reading arduino/Arduino#660 more closely it sounds it does support it but possibly with some issues. It seems like there has to be some connection between the upgrade to AVR-GCC 4.9.2 and the LTO compiler flags being added to Arduino AVR Boards at the same time.

If I try to compile with the -Os -flto (default) selection with IDE 1.6.5/AVR Boards 1.6.8 with avr-gcc-ar I get:

...
C:\Program Files (x86)\arduino-1.6.5-r5\hardware\tools\avr/bin/avr-gcc-ar rcs C:\Users\per\AppData\Local\Temp\build7779310838770217580.tmp/core.a C:\Users\per\AppData\Local\Temp\build7779310838770217580.tmp\wiring_pulse.S.o 
C:\Program Files (x86)\arduino-1.6.5-r5\hardware\tools\avr\bin\avr-gcc-ar: Cannot find plugin 'liblto_plugin-0.dll'
Error compiling.

I don't find liblto_plugin-0.dll included with Arduino's avr-gcc 4.8.1. I do find it included with Arduino's avr-gcc 4.9.2. Maybe that's what's referred to at arduino/Arduino#660 (comment) (I'm using Windows):

I had to remove usage of LTO from Cosa for the Arduino IDE when the plugin was not correctly delivered with the Windows version of the AVR tool chain

If I copy liblto_plugin-0.dll from avr-gcc 4.9.2 to 4.8.1 then I get the sorry - this program has been built without plugin support error.

If I try to compile with the -Os -flto (default) selection with IDE 1.6.5/AVR Boards 1.6.8 with avr-ar I get:

...
C:\Program Files (x86)\arduino-1.6.5-r5\hardware\tools\avr/bin/avr-gcc -Wall -Wextra -Os -Wl,--gc-sections -mmcu=atmega328p -w -flto -o C:\Users\per\AppData\Local\Temp\build3787617155115921065.tmp/sketch_oct05a.cpp.elf C:\Users\per\AppData\Local\Temp\build3787617155115921065.tmp\sketch_oct05a.cpp.o C:\Users\per\AppData\Local\Temp\build3787617155115921065.tmp/core.a -LC:\Users\per\AppData\Local\Temp\build3787617155115921065.tmp -lm 
collect2.exe: fatal error: CreateProcess: No such file or directory
compilation terminated.
Error compiling.

I don't know what file or directory it's referring to.

So that means I'm getting a different result than you, probably because we're on different operating systems. I remember even with AVR-GCC 4.9.2 they had to try a couple times to build it correctly to support LTO on Windows. I'm happy to test out other configurations if you want, just let me know.

Another option would be to include an AVR-GCC tool with MiniCore so that you will always have control of which version is used, as discussed for AVRDUDE in MCUdude/MightyCore#32. Of course that requires the extra complication of including AVR-GCC with the manual installation and needing a different installation file for each operating system, etc.

You could also just explain in the readme that the LTO options require IDE 1.6.11 or newer or AVR Boards 1.6.13 or newer and make them non-default but that's unfortunate for people using older versions and may also lead to support requests from people who didn't read your documentation.

[MCUdude]

Backwards compatibility is a pain! I think the best option right now is to make -Os the default , and make the -flto another option. I'll document this well in the readme file, so nobody can blame me 😉

About the GCC compiler; the latest IDE is including 4.9.2. Why aren't we running GCC 5 or 6? Are there any reason why we shouldn't use newer versions?

[per1234]

Why aren't we running GCC 5 or 6

I think Arduino tends to prefer stable well tested over cutting edge. There are builds of 5.1.0 and 6.1 available from arduino/Arduino#660 so they certainly were considered. I've been using 6.1.0 for some bootloader builds lately, just because that's what the other developer is using and have experienced no problems. I just use the standard tool versions with the Arduino IDE so that I can better reproduce the experience of the standard user.

Another advantage of having your own avr-gcc tool would be to offer the users of your cores the latest tool versions but it does seem like some work to get the builds right for all operating systems.

[MCUdude]

How large will the core become (in MB) with GCC compiled for Windows, macOS and Linux? It's certainly an option, I'm tired of not having full control when a new IDE is released.

[per1234]

avr-gcc 4.9.2-atmel3.5.3-arduino2 for Windows is 154 MB unzipped, 42 MB zipped. If you use the option of providing operating specific manual installation files with each release then the user will only have to download/install/store the avr-gcc build for their OS, not all of them. The correct tool version for the users OS will be automatically installed by Boards Manager.

[MCUdude]

I think most users are using the boards manager option, but I want the manual installation to work as well. I'm no Github expert, but maybe there should exist three releases, one for each operating system. Can Github fix this automatically, or do I need to generate the ZIP file myself?

[per1234]

The only way I can think of to do three releases for each release would be to base each one on a different branch, one branch for each operating system. The hardware folders of all the branches would be synced and the tools folder of the branch would contain the operating system specific build of avr-gcc.

The other possibility, which I like better, would be to do a single release from the master branch and then use the release webhook((https://developer.github.com/v3/activity/events/types/#releaseevent)) to trigger a script that automatically generates the operating system manual installation zip files and then attaches them to the release(https://developer.github.com/v3/repos/releases/#upload-a-release-asset). That seems like it will make for a cleaner/easier to manage(because less branches) and much smaller(because it doesn't need to include avr-gcc) repository. That same script could also assemble the Boards Manager installation file and update the JSON file.

[MCUdude]

The other possibility, which I like better, would be to do a single release from the master branch and then use the release webhook((https://developer.github.com/v3/activity/events/types/#releaseevent)) to trigger a script that automatically generates the operating system manual installation zip files and then attaches them to the release(https://developer.github.com/v3/repos/releases/#upload-a-release-asset). That seems like it will make for a cleaner/easier to manage(because less branches) and much smaller(because it doesn't need to include avr-gcc) repository. That same script could also assemble the Boards Manager installation file and update the JSON file.

This definitely sounds like the best way. I think MiniCore might be the the best core to start with. It's not "mature", and isn't nearly as popular (yet) as the other two. I'd love some help with this, so you're welcome to help if you want to!

[per1234]

I don't have any experience with this sort of thing(beyond Windows batch files 😛) so if any script wizards end up reading this I'm sure we'd appreciate some help and would probably eventually be able to adapt it to the other cores. There's a good start at the Boards Manager part of the script here: https://github.com/Optiboot/optiboot/blob/master/optiboot/release.sh. I'll keep my eye out for other repositories that do something similar to the release script triggered by webhook that we could use as a reference.

Of course I'm happy to keep doing the Boards Manager thing manually. I can put together the manual installation release files too. I can't add files to releases via pull request but I could upload them to a branch of my fork and add links to them in the boards manager release pull request description. The script would be an improvement in the long term because it will less prone to mistakes once it's working correctly.

[MCUdude]

If it's easier for you to test and debug, I can always give you admin rights to this core!
I've a tiny bit of experience with bash scripting, so we may be able to create a working script on macOS and Linux

[per1234]

I appreciate your confidence in me. I think I could do just about anything from a fork but will let you know. The script would run on a server so it would only need to to be written for linux, the webhook causes a payload with the release information to be sent to a URL on that server. There's a general overview of GitHub webhooks here: https://developer.github.com/v3/repos/hooks