Enable WARNINGS_AS_ERRORS=1 about sv-benchmarks

[delcypher]

@dbeyer This issue shouldn't be closed as WARNINGS_AS_ERRORS is still set to 0 in the current build system due to problematic benchmarks. Enabling WARNINGS_AS_ERRORS is going to be very difficult because many benchmarks trigger warnings.

[dbeyer]

Sorry, I must have closed this issue incidentally by closing another one (#118).

[PhilippWendler]

I think it would be helpful to turn on WARNINGS_AS_ERRORS by default and turn it off for now only in those directories where it is necessary. This would serve as documentation about the current state, ensure that newly added tasks get added without warnings, and maybe motivate some people to remove the warnings from the tasks they submitted. At least we can then identify and ping the people more easily.

[PhilippWendler]

@delcypher I think if we want to get process on this we need to have strict checks as default and only loosen them specifically for some directories where this is (still) needed. This way we can always look at the exceptions and see where work is still necessary. I hope that this would improve working on fixing the benchmarks.

Currently nobody knows what kind of warnings need to be taken care of on which parts of the benchmark set, so nobody does anything.

Could you do this in the build system? Would be great.

[delcypher]

@PhilippWendler

The build system already supports enabling/disabling warnings on a per directory basis. Here is what you need to do.

1. Merge #149 to make debugging easier for yourself
2. Change WARNINGS_AS_ERRORS ?= 0 in Makefile.config to WARNINGS_AS_ERRORS ?= 1.
3. For directories where you need to disable WARNINGS_AS_ERRORS you can add this line

WARNINGS_AS_ERRORS ?= 0

to the Makefile for that directory before the include $(LEVEL)/Makefile.config line.

For example

LEVEL := ../

WARNINGS_AS_ERRORS ?= 0

include $(LEVEL)/Makefile.config

To check the change is having effect run make debug_vars in the directory where you changed the Makefile and check you see that WARNINGS_AS_ERRORS is set appropriately.

Note you can also specify WARNINGS_AS_ERRORS on the command line when building to override the behaviour

make WARNINGS_AS_ERRORS=1

There are a few things you need to be aware of

• SUPPRESS_WARNINGS and WARNINGS_AS_ERRORS cannot both be enabled at the same time.
• The TravisCI build has SUPPRESS_WARNINGS enabled because so many warnings fire that we hit TravisCI's log limit and the build fails. So if you make WARNINGS_AS_ERRORS default you will break the build.

As for doing the work of working out which warnings to have (and which compiler supports them) this is not my responsibility.

[PhilippWendler]

Can we disable individual warnings/errors per directory?

How can I do what you suggest without breaking the Travis build? Is it possible to make SUPPRESS_WARNINGS and WARNINGS_AS_ERRORS compatible? Should we simply redirect warnings to a file or /dev/null on Travis? Or pipe them through head to see at least some of them?