nlesc-recruit / cudawrappers Goto Github PK

Let's describe the steps to compile the code and run it on various infrastructures (DAS5, DAS6, AWS, your local system). We can then use this information to create a developer documentation.

While making a release also update CITATION.cff file.

Blocked by:

• #8
• #12

The current code has wrappers for fft and blas.
https://github.com/nlesc-recruit/CUDA-wrappers/blob/ea0717593a010c1f3d596839ae46b50b1a43c1d5/cu/cufft.h
https://github.com/nlesc-recruit/CUDA-wrappers/blob/ea0717593a010c1f3d596839ae46b50b1a43c1d5/cu/cublas.h

@benvanwerkhoven @isazi is it fine with you if we keep these two wrappers out of scope and focus on cu.h and nvrtc.h?

In #43 we changed the folder structure which broke the build system. We need to adapt the buildsystem (CMakeLists.txt) and make sure we can build the code on system with CUDA.

We need a list of hardware and software used in development and testing. Please add the information below as a comment.

• Infrastructure details: < AWS, DAS5, DAS6 etc.>
• Operating system details:< Windows 11, Ubuntu 21.10, Amazon Linux, macOS>
• CUDA version and GPU device (paste output of nvidia-smi):

Following 10-new-project.md, we decided/noted on the following:

• Access
  • Everyone has access to the repo.
  • We may need an account for read-the-docs. In such case we will create an organization e-mail and add @bouweandela as admin
• Pull Request
  • PR template exists
• Debugging
  • We hope that the compiler will get most errors. We can use the guide.esciencecenter.nl/#/best_practices/language_guides/ccpp compiler flags in Static code analysis with GCC
  • We can use gdb and valgrind as tools for debugging during runtime.
• Documentation
  • guide.esciencecenter.nl/#/best_practices/language_guides/ccpp
  • CMake docs
  • https://www.doxygen.nl/index.html
  • Visual studio C++ plugin
  • Visual studio CMake plugin
• Virtual environments
  • We are not using virtual environments
• Building
  • See #7 for installation on AWS and DAS-5
  • Locally installing CMake
• Linting
  • We don't have any linters set up
  • We will ask C++ experts and TechLeads at the center how they do linting (rules, services)
• Running tests
  • We don't have any test
  • We need to setup the CI first
  • Writing unit test is out of the scope of the current sprint
  • We will skip this for now
• Enforcing consistent styling
  • We need to know which rules are already in place
  • We will use EditorConfig

Maybe use github search to look for similar projects, projects that already include a copy of "our" files, etc.

Add usage examples of the library in a separate repository.
Related issues:

• #2
• #9

Alternatives:

• https://www.coreinfrastructure.org/programs/best-practices-program/

This tutorial may help: https://github.com/ci-for-research/self-hosted-runners/blob/master/ubuntu-surf-hpc-cloud/README.md

add CONTRIBUTING.md

CONTRIBUTING.md from python-template can be reused but needs some changes.
https://github.com/NLeSC/python-template/blob/a59600f328d0eadad548893ac4822f37e54b677e/%7B%7Bcookiecutter.directory_name%7D%7D/CONTRIBUTING.md

After #59 or similar PR.

Perhaps as a section in the README like

## Used by

1. tool the first
2. tool the second

Having a Code of Conduct is a good idea for keeping a healthy community. See, for instance, Contributor Covenant.

about similar or complementary tools in this space

Decide if we want to have pre-commit hooks. If the answer yes, add a hook.
See: https://github.com/pocc/pre-commit-hooks

• build folder - or maybe explicitly
  - CMakeFiles
  - cmake_install.cmake
  - CMakeCache.txt
  - Makefile

Possibly other things.

We should rename the library and use the same name for the repo for clarity.
Some suggestions:

• cudacpp
• cudacxx
• cuda++
• cudawrappers
• your suggestion

Release 0.2.0

• Best practices issues: CITATION.cff, badges, PR and issues templates etc. https://github.com/nlesc-recruit/CUDA-wrappers/milestone/1

We need input about:

• new tree layout
• refactoring the code
• how they are running CI

Actions:

• @bouweandela sends email to stakeholder to ask their opinion
• We create an issue about refactoring

I have found these so far:

Direct users of this or very similar wrapper:
https://git.astron.nl/RD/idg
https://git.astron.nl/RD/dedisp/
https://git.astron.nl/RD/tensor-core-correlator

Similar code, possibly related:
https://git.astron.nl/ro/lofar/-/blob/master/RTCP/Cobalt/GPUProc/src/gpu_wrapper.cc

maybe add bram and steven to CITATION.cff

use correct file extensions: cpp, hpp
use include and src folder instead of cu
add example to tests folder

Make sure in CITATION.cff has correct name, ORCID etc of all the contributors.

Add step by step instructions to build the library in README.dev.md.

General

1. Find a better name for the repo #8
1. Capitalization of the repo name? Should be consistent with name of the library IMO #8
1. fix copyright issue in README.md #11
1. Write something about the added value of the wrapper. What thing is now easier? #12
1. Describe what this code does and why a user needs it #12
1. Is there value to having a sibling repo (say, cuda-wrappers-it) with a release of cuda-wrappers as a submodule? #9
1. Make an RSD entry #13
1. add editorconfig #14
1. pre-commit hooks for clang clang-tidy https://github.com/pocc/pre-commit-hooks #15
1. Add a Citation file #17
1. Add contributing guide (CONTRIBUTING.md) #18
1. add a PR template, an issue template #19
1. Add Code of conduct #20
1. Add badges: fair-software, repo badge, license badge, rsd badge, CI badges, zenodo badge, checklist badge #21

Code improvements

1. organize the files and think about folder structure #9
1. Clean up the code: delete commented out code #24

CI

1. look for options to do static analysis and add a GH action #23
1. Add integration test programs See #9
1. CI with self-hosted runners to run the integration tests #25
1. continuous integration for at least the same linters as precommit hooks #26
1. add cffconvert-github-action #27

1. check if PRs will be a problem for self-hosted runners

Documentation

1. look for alternatives for code documentation and build the documentation with CI #28
1. document how to build the documentation locally #29
1. notes on how to make a release #30
1. notes on precommit hooks #31
1. notes on running the linters locally #32
1. notes on how to build the library libcu.so #33
1. add list of software and hardware requirements #34
1. add list of supported CUDA versions and operating systems #34
1. add a section about 'alternatives' doing the same or similar thing #35
1. improve code documentation --> document functions #36

1. think about remote development/debugging using VSCode's remote containers extension (https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers)
2. Consider creating a conan package https://conan.io
3. Any IDE extensions or configuration needed?

Build system

1. Is libcu.so a good name? #8
1. Can we steal some ideas from https://github.com/nlesc-dirac/sagecal/blob/master/CMakeLists.txt ? #9

API

1.The API is not covering all the functionalities. Decide how to proceed after talking to Ben. #37

Dissemination

1. Add links to repos that use the wrappers maybe as a section in the README like "##Used by" #38
2. Find out who can benefit from this work #39
3. Think about events that this work can be advertised #40

• fair-software
• repo badge
• license badge
• rsd badge
• CI badges
• zenodo badge

Blocked by:

• #8
• #11

Maybe @benvanwerkhoven and @bouweandela can help.

We need a simple working example program that uses the library for testing the build system and as a test case.

The current plan is to construct a simple vector_add example program that uses both the CUDA Driver wrapper and NVRTC.

We need a tool that can analyse the code and generate a report.
https://sonarcloud.io/ can be one of the options.

Ask @benvanwerkhoven and @isazi for suggestions.

To make the library easy for others to include in their own work, we propose to make a few changes.

We'd like to make the file tree structure more or less as follows:

.
├── cudawrappers
│   ├── CMakeLists.txt
│   ├── cudawrappers-config.cmake
│   ├── include
│   │   └── cudawrappers.hpp
│   ├── README.md
│   ├── src
│   │   └── cudawrappers.cpp
│   └── tests
│       ├── CMakeLists.txt
│       ├── convolution
│       │   ├── convolution.cpp
│       │   └── README.md
│       ├── README.md
│       └── vector_add
│           ├── README.md
│           ├── vector_add.cpp
│           └── vector_add.cu
└── usage-examples
    ├── cmake-pull
    │   ├── CMakeLists.txt
    │   └── helloworld.cpp
    ├── git-submodule
    │   ├── [email protected]
    │   └── helloworld.cpp
    ├── locally-installed
    │   ├── CMakeLists.txt
    │   └── helloworld.cpp
    ├── plain-copy
    │   ├── CMakeLists.txt
    │   ├── cudawrappers
    │   └── helloworld.cpp
    └── README.md

1. We propose to name the library cudawrappers, this will be more consistent across various contexts in which the name will be used
2. cudawrappers/:
   1. contains the library files, with an include/, src/ and tests/ directory. src/ should contain the definitions part of what used to be in files cu/cu.cc, cu/cu.h, cu/nvrtc.c and cu/nvrth.h in release 0.1.0; include/ contains the corresponding header files with all declarations. Any definition code that's currrently in cu/cu.h or cu/nvrth.h should be moved to somewhere under src/.
   2. We propose to use filename extensions .cpp for implementations and .hpp for header files.
   3. tests/ will contain a few tests, I've put some placeholder directories (convolution/ and vector_add/) there to illustrate the general idea.
3. usage-examples/ contains some typical ways of how users of the library would access the functionality offered by the cudawrappers library, such as
   1. by including a plain copy of the library source code in their own source tree and building the complete project with CMake (plain-copy/);
   2. by including a git submodules copy of the library source code in their own source tree and building the complete project with CMake (git-submodule/);
   3. by locally installing the library to an arbitrary place on their system and building their project with CMake (locally-installed/);
   4. by having CMake pull in a copy of the library code from GitHub (cmake-pull/)

In README.md we should explain what problem this library solves.