biddisco / pycicle Goto Github PK

When builds are started, currently, we use a fixed set of options for each 'machine'. A new config file with a different machine name can be used to create a second set of options for a the same machine, but it would be better to allow many options/choices to be setup and allow pycicle to pick some either at random - or according to some other rule.

Options would generally fall into categories

• Compiler choice
• Compiler flags (e.g. c++11/14 or sanitizer flags)
• CMake options (WITH_XXX)
• Library or dependency versions (boost/hwloc/other)

Recent builds on CSCS machine daint failed with an error

sbatch: error: You are unable to submit jobs because you have exceeded the quota of   
1 million off files/folders (inodes) on the scratch file system.  
You can find more details about the quotas on the CSCS User Portal ...

There should be a cleanup step added that will either

• cleanup any PRs that have been merged (requires iterating over all PRs since time began, aswe do not currently track them)
• or cleanup and build/ or src/ dirs that are older than some time limit (like a week or so), this just means that some PRs will be purged and rebuild from a new checkout and new build dir when they are next triggered.

The second option is currently preferred for simplicity.

When adding new features to pycicle, build/test fails that are caused by incorrect setup can make a project look bad by setting failed status on gihub PRs. There should be a --no-status or -n flag to disable status setting during testing of pycicle itself.

I think from readying this that for a regex like in our code that using

r"whatever" is the correct replacement for ur I think.

Using python 2.7.x

I find that only PYCICLE_GITHUB_TOKEN is all that works. I suspect this might be a unicode issue. Did some fooling with that, inconclusive.
Going to try pycicle in python 3, if that just works perhaps pycicle should just go python 3 only.

There is a bad smell around PYCICLE_GITHUB_BASE_BRANCH & PYCICLE_BASE
Just seems like a potentially disasterous alias for PICYCLE_BASE or vice versa
as far as I can tell they are set equal here and that's that. Could be for the others as well. Seems like the intention was to decouple the PYCICLE_GITHUB values from those passed to the CDash dashboard. But then GITHUB_ORG and GITHUB_PROJ are reused.

What about treating all variables/defines coming in from cmake configs as a dictionary with keys that are the cmake variable names. Most could just be carried straight across to the dashboard call.

In order to use pycicle for projects other than HPX, there are some HPX specific variables/links that need to be made into options or configurable elements.

Currently pycicle does not wipe binary dirs between builds. this can lead to invalid CMakeCache states that invalidate build/test results.

Since hpx is the build canary

Hello,

Your project was referred to me by Hannes Vogt, and I am considering using pycicle in our organization (Polish meteorological services, IMGW) for testing development of our new dynamic core for the weather forecast.

Our project is built and tested exclusively by CMake and CTest (with make and make test). Besides C++ and CUDA it does have Fortran code. The git repository containing the project itself contains git submodules. The project itself is hosted on our internal GitLab instance, and some submodules are on GitHub. We wish to build the project on our supercomputer managed by the Torque (not Slurm), where compute nodes do not have access to the internet. We don't use CDash (but may start using it, if it is recommended).

Do you think this kind of project is within reach of the pycicle (or at least pycicle can be easily expanded to cover it)?

This will include refactoring master => base MASTER => BASE where appropriate to avoid confusion. It should not break hpx.

When pycicle checks out a new branch to test a PR, it will not get the right SHA when the PR comes from a remote clone of the repo being tested.

The sequence used here assumes that the branch being tested comes from origin and therfore the SHA will not be fetched for PR from a clone

                       ${CTEST_GIT_COMMAND} checkout ${PYCICLE_MASTER};
                       ${CTEST_GIT_COMMAND} fetch origin;
                       ${CTEST_GIT_COMMAND} reset --hard origin/${PYCICLE_MASTER};
                       ${CTEST_GIT_COMMAND} branch -D ${GIT_BRANCH};
                       ${CTEST_GIT_COMMAND} checkout -b ${GIT_BRANCH};
                       ${CTEST_GIT_COMMAND} fetch origin ${PYCICLE_BRANCH};
                       ${CTEST_GIT_COMMAND} merge --no-edit FETCH_HEAD;
                       ${CTEST_GIT_COMMAND} checkout ${PYCICLE_MASTER};
                       ${CTEST_GIT_COMMAND} clean -fd;"

Allow pycicle to take its per project settings from the source project itself.

Each project that uses pycicle for testing can have a pycicle subdir that stores the testing/config settings, the main advantage being

• The settings for the project can be maintained by the project owners without modifiying pycicle
• Different branches could (in principle) have different test settings
• Changes to options can be smoothly updated by the project

and lots more reasons

When the test project is first configured, ctest submits results and pycicle is updating not just the config, but also build and test status with 'ok' before they have actually run. They are correctly set later to 'fail' if there are problems, but they should be marked as running rather than ok.

Currently, the user's home directory gets all the slurm-123456.out and slurm-123456.err files. It ought to be able to put them somewhere else. A better default would be $PYCICLE_ROOT/temp with the option to put them somewhere else as a command-line or config setting.

It would be useful to be able to pass a set of build options to the configs to easier run builds with combinations of options. These would be for example:

• compiler
• compiler version
• dependency (e.g. boost) version
• compiler/linker flags
• sanitizer
• and so on...

These can be generic though so that projects can define their own set of options. I'm not sure what the best syntax would be for this (could be command line options or a separate build options config file), but essentially one should be able to launch pycicle with:

• project=hpx
• machine=daint
• build_options={compiler: gcc49, clang5; generic_option_1: value_1, value_2; generic_option_2: value_1, value_2}

pycicle would then generate the cartesian product of the build options and launch a build for each combination with ctest -S config/hpx/daint.cmake ... -Dcompiler=gcc49 -Dgeneric_option_1=value_1 -Dgeneric_option_2=value_1.

The build name should be derived from the build options to make them unique.

Conflicts/dependencies between build options are project specific anyway, so this should be handled on a per project basis in the config files.

One may not want to run builds for all combinations of build options so one could additionally:

1. launch multiple instances of pycicle to piece together the combinations one wants (will this work now?), or
2. have a way to include or exclude specific combinations (probably easiest to put in a separate file which is passed to pycicle)

Ideally the build options and config (daint.cmake) would be taken from the source repository but this might require a bigger refactoring.