claw-project / claw-compiler

CLAW Compiler for Performance Portability

The CLAW Project is an open-source project including a directive language specification and a reference compiler targeting performance portability in climate and weather application written in Fortran.

CLAW Compiler

The CLAW Compiler is a source-to-source translator working on the XcodeML intermediate representation. It implements the necessary transformation to the CLAW Directive Language Specifications. Intent of this language is to achieve performance portability on weather and climate code, especially for column- or point-wise computation.

• CLAW Directive Language

  • Official repository
  • Specification v0.4.0
  • Specification v1.1.0
  • Specification v2.0.0
  • Specification v2.0.1

• CLAW Compiler code

  • Official repository
  • Releases

• CLAW Compiler - Developer's Guide

  • Guide version 1.0

CLAW X2T

CLAW X2T (XcodeML to XcodeML Translator) is the part in charge of the AST analysis and transformations based on the directives.

The Java documentation of the CLAW X2T libraries is available.

Contributing

The CLAW Project is welcoming contribution. More information can be found on the official repository. A Slack workspace is available for discussions.

Resources

Papers

• The CLAW DSL: Abstractions for Performance Portable Weather and Climate Models. In Proceedings of the Platform for Advanced Scientific Computing Conference (PASC '18) [doi]

Talks

• Plenary Talk at PASC'18: The CLAW DSL: Abstractions for Performance Portable Weather and Climate Models [Slides]

<iframe width="560" height="315" src="https://www.youtube.com/embed/zns7JcbuKB4?rel=0" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>

About

This work was initially funded by the ETH zürich and the PASC initiative under the ENIAC project.

---

CLAW logo by adrienbachmann.ch

claw-compiler's Issues

Pass _OPENMP or _OPENACC to claw compiler

When clawfc is called with --directive=openacc|openmp, the corresponding flag should be passed to the preprocessor.

Maybe the -D option now in place should be modified

Update OMNI to 1.0.0

New release of OMNI available. Update to new version.

http://omni-compiler.org/

Fusion clause cause problem when used with loop-extract

The fusion transformation throw a null pointer exception when used as a clause in the loop-extract directive.

Implement the array-transform transformation

The skeleton is already in place. Need to implement the required transformation.

CLAW language definition

Version is empty

The option --version guves an empty results

Copy from old repo to keep track and correct numbering.

[DEP SOLVER] simplify solver and add extension

--debug option should pass the debug flag to the translator

Currently, to have debug information from the translator, we have to pass the debug flag through the transflator specififc flag like this

clawfc --Wx-d

It would be nice to have this flag passed automatically when the "main" debug flag is on

clawfc --debug

PURE subprograms may not contain OpenMP directives

OpenMP is not allowed in a PURE function/subroutine.

clawfc --force-pure

Fix rebuild of Jar files

With the new Ant build process, the jar files get rebuild even without file changes. This should be fixed to save some time

Differenciate between target and accelerator

CLAW might have two different notion when talking about code generation.

There is the target (CPU vs. Accelerator) and there is the accelerator language used for generation.

clawfc --target=cpu --directive=openacc
clawfc --target=gpu --directive=openacc
clawfc --target=gpu --directive=openmp
clawfc --target=gpu --directive=none

Driver and translator must be updated to reflect this

List of target can be: cpu,gpu,mic
List of accelerator directive can be: openacc,openmp,none

[DEP SOLVER] Module filename and module name mismatch

When the module name in FORTRAN is not the same as the module filename, the driver fails to resolve the dependencies.

Automatic promotion of variable part of assignment

In some cases, promoted variables by the parallelize directive might be used in the rhs of an assignment statement. Like the z(k) = t(k) + q(k) in the following code. In this case, the variable zmust be promoted as well.

To be able to perform this, defined dimension information should be stored in the module file.

MODULE mo_column
  IMPLICIT NONE
CONTAINS

  SUBROUTINE compute_all(nz, q, t)
    INTEGER, INTENT(IN)   :: nz   ! Size of the array field
    REAL, INTENT(INOUT)   :: t(:) ! Field declared as one column only
    REAL, INTENT(INOUT)   :: q(:) ! Field declared as one column only
    REAL :: z(1:nz)
    INTEGER :: k

    DO k=1,nz
      z(k) = t(k) + q(k)
    END DO


    !$claw parallelize forward
    CALL compute_column(nz, q, t)


  END SUBROUTINE compute_all

  ! Compute only one column
  SUBROUTINE compute_column(nz, q, t)
    IMPLICIT NONE

    INTEGER, INTENT(IN)   :: nz   ! Size of the array field
    REAL, INTENT(INOUT)   :: t(:) ! Field declared as one column only
    REAL, INTENT(INOUT)   :: q(:) ! Field declared as one column only
    INTEGER :: k                  ! Loop index
    REAL :: c                     ! Coefficient

    ! CLAW definition

    ! Define one dimension that will be added to the variables defined in the
    ! data clause.
    ! Apply the parallelization transformation on this subroutine.

    !$claw define dimension proma(1:nproma) &
    !$claw parallelize

    c = 5.345
    DO k = 2, nz
      t(k) = c * k
      q(k) = t(k - 1)  + t(k) * c
    END DO
    q(nz) = q(nz) * c

  END SUBROUTINE compute_column
END MODULE mo_column

Option in parallelize forward (automatic promotion)

The automatic promotion resulting from a parallelize forward transformation will add the dimension at the beginning.
It might be useful to have an option to change this behavior and allow the user to add it at the end. It will also influence the placement of the induction variable in the resulting do statements.

Delete pragma after transformation (maybe add comment)

CLAW pragma directive must be deleted after the transformation (maybe option to keep it).

A comment can be placed to know which particular transformation was applied.

Separate XcodeML abstraction and Code transformation compilation

Everything related to Xelement abstraction should be separated with the CLAW code transformation.

Copy from old repo to keep track and correct numbering

Adapat mapped variable for loop-extract

there is now a function call/function definition separation for the mapping variables.

It should be done also for the mapped variables.

Copy from old repo to keep track and correct numbering

Move to array reference for promoted scalar

All var reference to a scalar promoted to an array must be transformed in an array reference.

In the given subroutine and possible inner subroutine/function.

[Code review] Avoid null

Avoid null initialization as much as possible.

for example here:
https://github.com/C2SM-RCM/claw-compiler/blob/master/omni-cx2x/src/cx2x/translator/transformation/loop/LoopFusion.java#L26

Use an empty array to avoid NullPointerException.

[Code review] (self) attach Xpragma directly

public static ClawLanguage analyze(String rawInput)

public static ClawLanguage analyze(Xpragma pragma)

https://github.com/C2SM-RCM/claw-compiler/blob/master/omni-cx2x/src/cx2x/translator/language/ClawLanguage.java#L95

Pass macro definition for OpenMP and OpenACC

When selecting the --target=<target> the corresponding marco definition should be passed to the preprocessor

--target=openacc --> would implicitly pass --Wp-D_OPENACC
--target=openmp --> would implicitly pass --Wp-D_OPENMP

A base macro for CLAW would be passed also _CLAW

CLAW directive for acc/omp

Would it make sense to have a directive like this:

OpenACC

!$claw acc parallel loop

OpenMP

!$claw omp parallel do

that transforms into this:
OpenACC

!$acc parallel loop

OpenMP

!$omp  parallel do

To hide OpenACC/OpenMP pragma until the transformation is done.

Implementation of loop-hoist transformation

Specifications

claw-language-specification

!$claw loop-hoist(induction_var[[, induction_var] ...]) [reshape(array_name(target_dimension[,kept_dimension]))] [interchange [(induction_var[[, induction_var] ...])]]

  ! structured block of code

!$claw end loop-hoist

The loop-hoist directive allows nested loops in a defined structured block to
be merged together and to hoist the beginning of those nested loop just after
the directive declaration. Loops with different lower-bound indexes
can also be merged with the addition of an IF statement. This feature works
only when the lower-bound are integer constant.

• induction_var: List of induction variables of the do statements to be hoisted.
• interchange: Allow the group of hoisted loops to be reordered.
  Options are identical with the \textbf{loop-interchange} directive.
• reshape: Reshape arrays to scalar or to array with fewer dimensions. The original declaration is replaced with the corresponding demoted declaration in the current block (function/module).

Options to specify the order of application of the transformations

Currently, the order is fixed in the class ClawTransformer.

It would be nice to be able to play on this order with a configuration file.

We could imagine a configuration file

<claw version="0.2">
  <target default="openacc"/>
  <groups>
    <group type="independent" name="remove"           class="cx2x.translator.transformation.utility.Remove"/>
    <group type="independent" name="array-transform"  class="cx2x.translator.transformation.loop.ArrayTransform"/>
    <group type="independent" name="loop-extract"     class="cx2x.translator.transformation.loop.LoopExtraction"/>
    <group type="dependent"   name="loop-fusion"      class="cx2x.translator.transformation.loop.LoopFusion"/>
    <group type="independent" name="loop-hoist"       class="cx2x.translator.transformation.loop.LoopHoist"/>
    <group type="independent" name="loop-interchange" class="cx2x.translator.transformation.loop.LoopInterchange"/>
    <group type="independent" name="on-the-fly"       class="cx2x.translator.transformation.claw.ArrayToFctCall"/>
    <group type="independent" name="kcache"           class="cx2x.translator.transformation.call.Kcaching"/>
  </groups>
</claw>

There should be a standard file following the specification advice.

A specific file can be passed at compile time.

clawfc --config=specific.xml ...

Also gives an option on the command line to output the current order.

clawfc --show-config
clawfc --config=specififc.xml

Target option

clawfc should have a target option to be able to specify the target architecture

For example

# Generate OpenACC for parallel and accelerator option
clawfc --target=openacc -o transformed_code.f90 original_code.f90

# Generate OpenMP for parallel and accelerator option
clawfc --target=openmp -o transformed_code.f90 original_code.f90

# Generate nothing for parallel and accelerator option
clawfc --target=none -o transformed_code.f90 original_code.f90

Automatic module dependency in the driver

OMNI Compiler front-end requires the module dependencies to be processed first and produces a .xmod file.

The driver should be able to process those dependencies automatically to some extend based on the current file location and the module search paths.

Add ability to generate vector_length/num_gangs/num_workers clauses

At the moment, accelerator directive generation is pretty simple. Only parallel, loop and private directives/clauses are generated.

It would be nice to be able to specify the corresponding vector_length, num_gangs and num_workers clauses to have a finer control over the accelerated region. These clauses are OpenACC clauses and must be abstracted in a way to let us generate also correct directives for OpenMP or other accelerator directive language.

Issue related to #22

Deferred array are not deferred anymore after transformation

There is a problem with OMNI Compiler (probably) that infers the size of the deferred dimension. This can lead to compilation error in the transformed code.

Original code:

SUBROUTINE dummy(myarray)
REAL(KIND=wp), INTENT(IN) :: myarray( : )
INTEGER :: nlay
nlay = 10

Transformed code:

SUBROUTINE dummy(myarray)
REAL(KIND=wp), INTENT(IN) :: myarray( 1 : nlay )
INTEGER :: nlay
nlay = 10

The second code snippet won't compile as nlay is not initialized yet. The deferred array shape should be maintained in the transformed code.

Add a force option to process file even they do not have CLAW directive

In some cases, especially for test purposes, it would be useful to have a force option to process files that do not have CLAW directives. At the moment, those files are just copied as is without any front-end/translator/back-end processing.

clawfc -f 
clawfc --force

Out of source build

When we perform an out of source build, there is an error in the build process

git clone [email protected]:C2SM-RCM/claw-compiler.git
cd claw-compiler
git submodule init
git submodule update --remote
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=<install_path> ..
make

Compiler error are not all displayed on the command line

Some errors are not display -> translator errors

Automatic generation of all accelerator directives

All accelerator directive must be generated by the CLAW translator

• Check mapping between OpenACC/OpenMP
• Check that all CLAW directive have appropriate clause to be able to generate accelerator directive
• Check AcceleratorGenerator interface if it fits with OpenACC/OpenMP

Generation of vector/gang/worker refinement #26

Multiple module paths

Looks like only the last path is taken into account by F_Front

Use OMNI Xname and Xcode

There is a file in the OMNI Compiler Project that reference all the Xcode and Xname so we can get rid of it in our code base

https://github.com/omni-compiler/omni-compiler/blob/master/XcodeML-Exc-Tools/src/exc/object/Xcode.java

https://github.com/omni-compiler/omni-compiler/blob/master/XcodeML-Exc-Tools/src/exc/xcodeml/XcodeMLNameTable_F.java

Always get latest dependency

It looks like that the dependency on omni-compiler is not always fetching the latest commit.

Move omni-compiler dependency

Move the dependency to MeteoSwiss-APN/claw branch instead of master. We will try to keep the master in sync with the official omni-compiler repo and the claw branch will follow our new dev.

Optimize dependency resolver

The dependency resolver in the driver is pretty dumb at the moment. A new one should be written to process each dependency only once and not multiple time like now.

Compile guard

The CLAW Fortran compiler is generating accelerator directive. The program might be wrong if the transformations are not applied. Therefore, it is useful to have a mechanism to make the compilation failed in such case.

For this, we introduce the CLAW compile guard.

This directive is defined with the target accelerator prefix and it is removed by the CLAW Fortran compiler to make the compilation succeed.

In case the CLAW Fortran compiler has not been applied, it will make the compilation with accelerator failed.

!$<accelerator language prefix> claw-guard 
!$acc claw-guard
!$omp claw-guard

loop-fusion group option is null if there is a space after it

The loop-fusion option is set to null in the translator if there is a space just after the option

!$claw loop-fusion group(j1)<space here>

Copy from old repo to keep track and correct numbering

Deal with OPTIONAL parameters

During the parallelization transformation, CLAW adds parameter for the lower and upper bound. This parameter must be added before any OPTIONAL parameter otherwise the compilation will fails.

Extra information could be inserted in the .claw.xmod file to know which parameters are added by CLAW.

Until now, CLAW relies on the parameter position in the list and this is not a good idea with OPTIONAL parameters in the way.

Insert attribute to tells which parameters were added by CLAW.

<FfunctionType result_name="r" return_type="Fint" type="F7fe84b508d40">
  <params>
    <name type="I7fe84b509c50">nz</name>
    <name type="I7fe84b50a340">b</name>
    <name type="A8dc9d740d58d">q</name>
    <name type="Abbd8df982469">t</name>
    <name type="A4894cc7a8f58">z</name>
    <name type="I248583c120cf" is_claw="true">nproma</name>
  </params>
</FfunctionType>

"Unable to locate module file for" if no module search path specified

If no module search path is specified, translator should look on pwd path.

Implement a way to avoid part of code (claw ignore)

It can be useful to work around bug or not implemented feature in OMNI compiler or CLAW to have a way to ignore portion of code.

MODULE xyz

CONTAINS
  SUBROUTINE test
    !$claw ignore
    FORALL (i=1:ni, j=1:nj, k=1:nk)
      ! other statements
    END FORALL
    !$claw end ignore
  END SUBROUTINE

END MODULE

The part between !$claw ignore and !$claw end ignore is then ignored during the parsing / transformation / decompilation phases but stay in the transformed code at the same place.

Separate the CLAW Compiler and the transformations set

It would be nice to be able to have several transformations set that are packaged in different jar file and being able to use the same CLAW Compiler infrastructure.

The configuration has been totally reviewed and the transformations are now belonging to a transformation set. The main configuration then describe which transformation set to use and specify the order of application.

The transformation set configuration file holds all information about the class, the type and the trigger of the transformation itself. This file also specify if an external JAR has to be loaded in order to load the transformation class.

The user can partially or totally overwrite the default configuration but not the transformation sets configurations. Those are distributed by the developers of the transformations.

Example of a transformation set configuration file:

<!--
This file describes CLAW internal transformation not triggered by the user.
-->
<transformations>
  <transformation name="internal-xcodeml"
    type="independent" trigger="translation_unit"
    class="cx2x.translator.transformation.utility.XcodeMLWorkaround" />

  <transformation name="openacc-continuation"
    type="independent" trigger="directive"
    class="cx2x.translator.transformation.openacc.OpenAccContinuation" />
</transformations>

Example of a the main default configuration file:

<!--
CLAW default configuration. This file should not be edited!
For more information about the configuration file and its extension, please
refer to the developer's guide
-->
<claw version="0.4">
  <!-- Global transformation parameters -->
  <global type="root">
    <!-- Define the transformer to be used. -->
    <parameter key="transformer" value="cx2x.translator.transformer.ClawTransformer" />

    <!-- Default general values -->
    <parameter key="default_target" value="gpu" />
    <parameter key="default_directive" value="openacc" />


    <!-- OpenACC default information -->
    <!-- Define the default values to be used for the clauses. If value set to 0, compiler default value used. -->
    <parameter key="openacc_vector_length" value="128" /> <!-- vector_length(128) -->
    <parameter key="openacc_num_gangs" value="8" />       <!-- num_gangs(8) -->
    <parameter key="openacc_num_workers" value="8" />     <!-- num_workers(8) -->
    <!-- Specify the order of generation of clauses when nested loops are generated -->
    <!-- possible values are vector_gang, gang_vector, vector, gang, none -->
    <parameter key="openacc_execution_mode" value="vector_gang" />
  </global>

  <!-- Transformation sets -->
  <sets>
    <set name="claw-internal-set" />
    <set name="claw-low-level-set" />
    <set name="claw-high-level-set" />
  </sets>

  <!-- Transformation groups and order -->
  <groups>
    <group name="internal-xcodeml" />
    <!-- Low-level transformations -->
    <group name="remove" />
    <group name="primitive" />
    <group name="array-transform" />
    <group name="loop-extract" />
    <group name="loop-hoist" />
    <group name="loop-fusion" />
    <group name="loop-interchange" />
    <group name="on-the-fly" />
    <group name="kcache" />
    <group name="if-extract" />
    <!-- High-level transformations -->
    <group name="parallelize" />
    <group name="parallelize-forward" />
  </groups>
</claw>

Example of a partial overwrite of the default configuration by a user defined configuration:

<claw version="0.4">
  <global type="extension"/>
  <groups>
    <group name="internal-xcodeml"/>
    <group name="remove"/>
    <group name="primitive"/>
    <group name="array-transform"/>
    <group name="loop-extract"/>
    <group name="loop-fusion"/>
    <group name="loop-hoist"/>
    <group name="loop-interchange"/>
    <group name="on-the-fly"/>
    <group name="kcache"/>
    <group name="if-extract"/>
    <group name="parallelize"/>
    <group name="parallelize-forward"/>
  </groups>
</claw>

In this case, only the application order is overwritten.

Better error message for the CLAW language parser

Enhance the error message for the CLAW parser written with ANTLR

Now they look like that

line 1:10 no viable alternative at input '('

They should be more precise with information like the expecting token.

line 1:10 expecting IDENTIFIER but found (

Test with PGI as main compiler (preprocessor)

Test are now performed with gfortran and Cray ftn. Test with PGI pgfortran should be made.

Preprocessor flags might changes for the driver and for test cases compilation as well.

Promotion from scalar to 1d of local variable

When promoting local scalar variable (local to a subroutine), the dimension specification cannot be of a deferred shape.

This doesn't work as compiler cannot find the deferred dimension:

REAL(KIND=wp) : : g0 ( : )

So the promoted variable should be declared like this:

REAL(KIND=wp) : : g0 ( 1 : ncol )

Enhance the possible OpenACC clause in the acc_optional parser rule

Currently only simple clause without parenthesis are allowed. We should allow any kind of OpenACC clauses inside the acc option.

Valid now

!$claw array-transform acc(loop gang vector)

Not valid at the moment but should be

!$claw array-transform acc(loop gang(16), vector(32))

Propagation of non parameter field with deferred shape

Clean up .bak file

fusion option in loop-extract must be the last option

There is a bug if the fusion option is not the last one, it is not considered.

Group option is not reachable

!$claw loop-extract range(xx) map(xx) fusion group(j1) acc(loop seq)

Ok

!$claw loop-extract range(xx) map(xx) acc(loop seq) fusion group(j1)

Should be fin in any order

Copy from old repo to keep track and correct numbering

Add revision + build timing in the version information

For the moment the version of the compiler is just a simple number.

$ clawfc --version
CLAW Fortran compiler 0.1

It is than sometimes hard to know if we are using the latest build especially in development phase. The version should include the revision and the build time.

$ clawfc --version
CLAW Fortran compiler 0.1 rev. f06990e build 20160322101104

For the releases, the version can be simpler with just the release number.

$ clawfc --version
CLAW Fortran compiler 0.1 release