Coder Social home page Coder Social logo

pharmaverse / admiralci Goto Github PK

View Code? Open in Web Editor NEW
13.0 13.0 4.0 10.35 MB

CI/CD workflows used across the admiral family of packages

Home Page: https://pharmaverse.github.io/admiralci

License: Other

R 83.07% Shell 16.93%
actions admiral cicd clinical-trials r-package workflows

admiralci's Introduction

pharmaverse website

pharmaverse.org

This website is maintained and governed by our website community. Further details of this are available here. The community lead is considered the product owner for the website and has final say on any decisions related to the website.

For any questions or enhancements for the site, please make issues at this repo.

Hosted sites:

This website is available at 2 locations:

Development / git flow

When you push to develop OR make a pull request onto develop a github action will run which will render the site, and deploy it to the test server. But the test server is different depending on which route you took.

The code on develop is hosted to: http://openpharma.s3-website.us-east-2.amazonaws.com/develop/

The intended workflows expects that develop is the UAT branch where you check things pre-prod. While the 'build from PR` allows you to work on something very experimental and check results, before pushing it do develop.

This git flow is predicated on it being unlikely to have more than one person actively working on the site at any one time.

You cannot work directly on main branch. The only way to push in is via a PR. And it will be denied if you have not come through develop. Any PR requires approval from one other Technology and Templates WG member.

Data flows

'Scraped' data

Passive data is collected via openpharma.github.io, a sister org that is completely un-opinionated and has
a wider remit spanning discovery to access. This site contains no curated data beyond names - and is instead focused on collecting and sharing data on open source health and related metadata (e.g. CRAN status and riskmetric scores).
Information on the data collected is here: https://openpharma.github.io/#data.

Data is stored in a pharmaverse AWS account.

Package info

Curated package info is stored in the folder data/. pharmaverse packages are in the folder data/packages. Non-pharma packages that get a special mention are in data/nonpharma. Each package is a unique .yaml file.

The general structure is:

name: admiral
repo: pharmaverse/admiral
repo_source: github.com
docs: https://pharmaverse.github.io/admiral/cran-release/
hex: https://github.com/insightsengineering/hex-stickers/raw/main/PNG/admiral.png
task: ADaM
details: (ADaM In R Asset Library) - Modular framework to generate ADaM via R functions relying on community contributions
splash: include # this controls what is shown on the main page. Plan is to deprecate and put all hex's up

It should be self explanatory. Please use the categories in the site for task:.

People

Info on people is in data/people. The yaml should be something like below, with the filename being the github handle.

name: James Black
pharmaverse_roles: tech
company: Roche
company_logo: roche.png
# Following is more a nice to have for profile pages [and optional]
linkedin: https://www.linkedin.com/in/epijim/
site: https://epijim.uk
title: People and Product Family Leader for Insights Engineering, Scientific Computing Environment Product Owner
bio: James is currently leading our pan-study codebase for insights

General content of the site

Site content is in content. It's normal markdown files. See markdown-guide.md.

Calling package info in the site

To call a package, we have two shortcuts. pharmaverse means it's 'pharmaverse' - e.g. in the folder data/packages. otherpackages is for non-pharmaverse packages, in data/nonpharma.

{{< pharmaverse pkg="rtables" >}}

{{< otherpackages pkg="gt" >}}

Council meeting minutes

From 2023 onwards, our monthly council meeting minutes will be published in static/council meeting minutes. Other static files such as images are stored in static also.

admiralci's People

Contributors

arkadiuszbeer avatar bms63 avatar bundfussr avatar cicdguy avatar dgrassellyb avatar galachad avatar pharmaverse-bot avatar rossfarrugia avatar walkowif avatar zdz2101 avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar

Forkers

arkadiuszbeer bms63 karmatarap dgrassellyb

admiralci's Issues

admiralvaccines requires metatools in their package

admiralvaccines requires metatools package to use combine_supp().

@galachad and @cicdguy do we just add this in manually to the lock file or will the propagation bot know to keep this in for us when a new propagation occurs. I do not believe any other admiral extension needs metatools.

@pharmaverse/admiralvaccines watch this issue for responses please.

General Issue: Automatically Add Project Board to Issue

Background Information

The Issue templates can auto add labels and I have seen other repos that can automatically add people and Project Board. I am assuming that we can also auto-add the Project Board with the backlog to the issue. This will cut down on us having to remember to add the project board to the issue.

Definition of Done

All Issue Templates automatically add the Project Board with Backlog assignment to them.

https://github.com/pharmaverse/admiral/tree/main/.github/ISSUE_TEMPLATE

Automate propagation to be set up as a push-button workflow

We need to allow propagation to happen as a workflow_dispatch event going forward, with the following proposed enhancements:

  • Any release manager, maintainer, developer, or admin will be able to propagate updated renv artifacts automatically
  • renv.lock preparation can take place automatically
  • Man pages will automatically be updated during propagation to keep the roxygen2 comments consistent

MultiVersion pkgdown fails for admiraldev

If I switch from devel to main I lose the ability to go back to devel. Also the search in main does not work

image

admiraldev devel search does not work - perhaps tied to multiversion not working

image

General Issue: Find Solution for Links in Documentation

Background Information

There are several issues regarding links in the admiral documentation:

  • Links to vignettes in function documentation do not work when viewed in RStudio help tab consider for example the links in the details section of ?derive_vars_query
  • We can not link to new vignettes on the NEWS page (using absolute links to the not yet release documentation) because our link check and the CRAN link check will fail.
  • When the documentation is build locally, e.g., for review, some links point to the release documentation and not the local one.
  • pkgdown::build_site() does not build the README correctly because README.md has to be created from README.Rmd first.

Definition of Done

Requirements

  • All links in the documentation should work in the HTML output as well as in the RStudio help tab. They should work in a local build and in a published build. They should work for multi versions docs.

  • The syntax for linking to vignettes within the package and to vignettes of other admiral packages should be the same.

  • Links to vignettes of other packages should link to the devel version for local builds and devel builds, they should link to the main version for main, pre-release, and patch build.

  • Create a proposal fulfilling the requirements

  • Get agreement from admiral team

  • Implement proposal

`admiral` Docker images

The objective is to make a common set of Docker images that will be used for all admiral CI/CD workflows and for local development.

The advantages of doing so are:

  1. Everyone has a consistent working environment (same version of R, same versions of packages) etc.
  2. Portability of the Docker images will allow for CI/CD workflows to be run by anyone, anywhere, and much faster
  3. Ease of maintenance of workspaces/working environments using a config-based approach

For execution:

  1. Host Dockerfile(s) and Docker images in this repo
  2. Leverage the widely used rocker RStudio images as the base, and populate these with packages from the renv.lock file in this repo
  3. Publish those images to GHCR (GitHub Container Registry) using a separate CI/CD workflow
  4. Update all CI/CD workflows to use the containers derived from the images in this step

Spell Check on `Rmd` files.

I've noticed some misspelling ins the vignettes and I think we don't have any spell checking running over them. Looking at the spell check we are excluding .Rmd. Could we stop excluding these checks or are the html files checked somewhere down the road??

admiralci/.github/workflows/spellcheck.yml

Line 94 in 4f0d6f7

exclude: data/*,**/*.Rd,**/*.Rmd,**/*.md,**/*.qmd

Link workflows documentation with title of PR Action

I was thinking that we should link these with the appropriate titles in the PR Actions. I don't think they line up exactly and new developers are often unsure which one is which and what they are doing. Some are super obvious, i.e. lint.yml with the Lint check.

image

image

Documentation: automate a check to throw an error if library() statement found in testthat

Please select a category the issue is focused on?

Other

Let us know where something needs a refresh or put your idea here!

While working on pharmaverse/admiral#1381, we decided that there should not be any library calls in any of the unit tests, e.g. in testthat. Instead, we should call a function directly if it is already available in the environment, i.e. mentioned in NAMESPACE importFrom() statement. Otherwise, we call it with the syntax pkgname::function().

General Issue: renv/activate.R – verify why was modified?

Background Information

When cloning admiral into RStudio, the renv/activate.R file shows up in the git panel as a file which has been changed. Should this file be listed in .gitignore?

image

Definition of Done

Knowledgeable review of this and determination of what should be done.

Automate `renv.lock` creation

The creation of renv.lock should be automated. A script should be implemented which creates renv.lock fulfilling the following requirements:

  • The lock file should contain all packages which are imported or suggested by any of the admiral packages. The version of the packages should be the one which was available when the R version we are using for development was released. At the moment we use R 4.0. I.e., we would store the version in the lock file which was available when R 4.0 was released.
  • There should be an option to use newer versions for some packages. For example, we use a newer version of pkgdown. The packages and versions should be stored in a separate file. It is read in by the script. Then the exceptions are clear and easy to maintain.
  • The admiral packages should not be in the lock file because these are handled by staged_dependencies.

pkgdown action erroring out

Hey @cicdguy @dgrassellyb @galachad

I merged in a PR for admiraldev into the devel branch and the page deployment is failing. I think some of this is the hook code that was jsut merged into admiralci. Any ideas?

image

https://github.com/pharmaverse/admiraldev/actions/runs/4245613987/jobs/7382218526

##[debug]Evaluating condition for step: 'Publish documentation'
##[debug]Evaluating: success()
##[debug]Evaluating success:
##[debug]=> true
##[debug]Result: true
##[debug]Starting: Publish documentation
##[debug]Loading inputs
##[debug]Evaluating: format('git config --local user.email "[email protected]"
##[debug]git config --local user.name "GitHub Actions"
##[debug]SUBDIR_OPTION=""
##[debug]if [ "{0}" != "true" ]
##[debug]then {{
##[debug] SUBDIR_OPTION="subdir = "${{GITHUB_REF##/}}","
##[debug]}}
##[debug]fi
##[debug]Rscript - <<EOF
##[debug]
##[debug]pkgdown_env <- asNamespace("pkgdown")
##[debug]rlang::env_unlock(env = pkgdown_env)
##[debug]rlang::env_binding_unlock(env = pkgdown_env)
##[debug]# Modify tweak_page
##[debug]pkgdown_env$call_hook <- function(hook_name, ...) {{
##[debug] # Get hooks from base::getHook
##[debug] hooks <- getHook(paste0("UserHook::admiralci::", hook_name))
##[debug] if (!is.list(hooks)) {{
##[debug] hooks <- list(hooks)
##[debug] }}
##[debug]
##[debug] # Evaluate hooks
##[debug] purrr::map(hooks, function(fun) {{
##[debug] fun(...)
##[debug] }}) %>%
##[debug] invisible()
##[debug]}}
##[debug]environment(pkgdown_env$call_hook) <- pkgdown_env
##[debug]
##[debug]tweak_page <- body(pkgdown_env$tweak_page)
##[debug]body(pkgdown_env$tweak_page) <-
##[debug] as.call(
##[debug] append(
##[debug] as.list(tweak_page),
##[debug] expression(call_hook("tweak_page", html, name, pkg)),
##[debug] after=length(tweak_page)
##[debug] )
##[debug] )
##[debug]
##[debug]rlang::env_binding_lock(env = pkgdown_env)
##[debug]rlang::env_lock(pkgdown_env)
##[debug]
##[debug]# Load package
##[debug]require(desc::desc_get("Package"), character.only = TRUE)
##[debug]
##[debug]pkgdown::deploy_to_branch(
##[debug] new_process = FALSE,
##[debug] ${{SUBDIR_OPTION}}
##[debug] clean = TRUE
##[debug])
##[debug]EOF
##[debug]', inputs.skip-multiversion-docs)
##[debug]Evaluating format:
##[debug]..Evaluating String:
##[debug]..=> 'git config --local user.email "[email protected]"
##[debug]git config --local user.name "GitHub Actions"
##[debug]SUBDIR_OPTION=""
##[debug]if [ "{0}" != "true" ]
##[debug]then {{
##[debug] SUBDIR_OPTION="subdir = "${{GITHUB_REF##/}}","
##[debug]}}
##[debug]fi
##[debug]Rscript - <<EOF
##[debug]
##[debug]pkgdown_env <- asNamespace("pkgdown")
##[debug]rlang::env_unlock(env = pkgdown_env)
##[debug]rlang::env_binding_unlock(env = pkgdown_env)
##[debug]# Modify tweak_page
##[debug]pkgdown_env$call_hook <- function(hook_name, ...) {{
##[debug] # Get hooks from base::getHook
##[debug] hooks <- getHook(paste0("UserHook::admiralci::", hook_name))
##[debug] if (!is.list(hooks)) {{
##[debug] hooks <- list(hooks)
##[debug] }}
##[debug]
##[debug] # Evaluate hooks
##[debug] purrr::map(hooks, function(fun) {{
##[debug] fun(...)
##[debug] }}) %>%
##[debug] invisible()
##[debug]}}
##[debug]environment(pkgdown_env$call_hook) <- pkgdown_env
##[debug]
##[debug]tweak_page <- body(pkgdown_env$tweak_page)
##[debug]body(pkgdown_env$tweak_page) <-
##[debug] as.call(
##[debug] append(
##[debug] as.list(tweak_page),
##[debug] expression(call_hook("tweak_page", html, name, pkg)),
##[debug] after=length(tweak_page)
##[debug] )
##[debug] )
##[debug]
##[debug]rlang::env_binding_lock(env = pkgdown_env)
##[debug]rlang::env_lock(pkgdown_env)
##[debug]
##[debug]# Load package
##[debug]require(desc::desc_get("Package"), character.only = TRUE)
##[debug]
##[debug]pkgdown::deploy_to_branch(
##[debug] new_process = FALSE,
##[debug] ${{SUBDIR_OPTION}}
##[debug] clean = TRUE
##[debug])
##[debug]EOF
##[debug]'
##[debug]..Evaluating Index:
##[debug]....Evaluating inputs:
##[debug]....=> Object
##[debug]....Evaluating String:
##[debug]....=> 'skip-multiversion-docs'
##[debug]..=> false
##[debug]=> 'git config --local user.email "[email protected]"
##[debug]git config --local user.name "GitHub Actions"
##[debug]SUBDIR_OPTION=""
##[debug]if [ "false" != "true" ]
##[debug]then {
##[debug] SUBDIR_OPTION="subdir = "${GITHUB_REF##/}","
##[debug]}
##[debug]fi
##[debug]Rscript - <<EOF
##[debug]
##[debug]pkgdown_env <- asNamespace("pkgdown")
##[debug]rlang::env_unlock(env = pkgdown_env)
##[debug]rlang::env_binding_unlock(env = pkgdown_env)
##[debug]# Modify tweak_page
##[debug]pkgdown_env$call_hook <- function(hook_name, ...) {
##[debug] # Get hooks from base::getHook
##[debug] hooks <- getHook(paste0("UserHook::admiralci::", hook_name))
##[debug] if (!is.list(hooks)) {
##[debug] hooks <- list(hooks)
##[debug] }
##[debug]
##[debug] # Evaluate hooks
##[debug] purrr::map(hooks, function(fun) {
##[debug] fun(...)
##[debug] }) %>%
##[debug] invisible()
##[debug]}
##[debug]environment(pkgdown_env$call_hook) <- pkgdown_env
##[debug]
##[debug]tweak_page <- body(pkgdown_env$tweak_page)
##[debug]body(pkgdown_env$tweak_page) <-
##[debug] as.call(
##[debug] append(
##[debug] as.list(tweak_page),
##[debug] expression(call_hook("tweak_page", html, name, pkg)),
##[debug] after=length(tweak_page)
##[debug] )
##[debug] )
##[debug]
##[debug]rlang::env_binding_lock(env = pkgdown_env)
##[debug]rlang::env_lock(pkgdown_env)
##[debug]
##[debug]# Load package
##[debug]require(desc::desc_get("Package"), character.only = TRUE)
##[debug]
##[debug]pkgdown::deploy_to_branch(
##[debug] new_process = FALSE,
##[debug] ${SUBDIR_OPTION}
##[debug] clean = TRUE
##[debug])
##[debug]EOF
##[debug]'
##[debug]Result: 'git config --local user.email "[email protected]"
##[debug]git config --local user.name "GitHub Actions"
##[debug]SUBDIR_OPTION=""
##[debug]if [ "false" != "true" ]
##[debug]then {
##[debug] SUBDIR_OPTION="subdir = "${GITHUB_REF##/}","
##[debug]}
##[debug]fi
##[debug]Rscript - <<EOF
##[debug]
##[debug]pkgdown_env <- asNamespace("pkgdown")
##[debug]rlang::env_unlock(env = pkgdown_env)
##[debug]rlang::env_binding_unlock(env = pkgdown_env)
##[debug]# Modify tweak_page
##[debug]pkgdown_env$call_hook <- function(hook_name, ...) {
##[debug] # Get hooks from base::getHook
##[debug] hooks <- getHook(paste0("UserHook::admiralci::", hook_name))
##[debug] if (!is.list(hooks)) {
##[debug] hooks <- list(hooks)
##[debug] }
##[debug]
##[debug] # Evaluate hooks
##[debug] purrr::map(hooks, function(fun) {
##[debug] fun(...)
##[debug] }) %>%
##[debug] invisible()
##[debug]}
##[debug]environment(pkgdown_env$call_hook) <- pkgdown_env
##[debug]
##[debug]tweak_page <- body(pkgdown_env$tweak_page)
##[debug]body(pkgdown_env$tweak_page) <-
##[debug] as.call(
##[debug] append(
##[debug] as.list(tweak_page),
##[debug] expression(call_hook("tweak_page", html, name, pkg)),
##[debug] after=length(tweak_page)
##[debug] )
##[debug] )
##[debug]
##[debug]rlang::env_binding_lock(env = pkgdown_env)
##[debug]rlang::env_lock(pkgdown_env)
##[debug]
##[debug]# Load package
##[debug]require(desc::desc_get("Package"), character.only = TRUE)
##[debug]
##[debug]pkgdown::deploy_to_branch(
##[debug] new_process = FALSE,
##[debug] ${SUBDIR_OPTION}
##[debug] clean = TRUE
##[debug])
##[debug]EOF
##[debug]'
##[debug]Loading env
Run git config --local user.email "[email protected]"
##[debug]/usr/bin/bash -e /home/runner/work/_temp/c48860c2-7565-40c4-a231-53c4307a7c52.sh
##[debug]Skipping logging an issue for the matched line because the message is empty.
Error in environment(pkgdown_env) <- pkgdown_env :
replacement object is not an environment
Execution halted
Error: Process completed with exit code 1.
##[debug]Finishing: Publish documentation
0s
##[debug]Evaluating condition for step: 'Post Restore cache'
##[debug]Evaluating: success()
##[debug]Evaluating success:
##[debug]=> false
##[debug]Result: false

r-cmd-check.yml: error_on = 'note'

I have a use case where r cmd check will always give a note. By design the package is using attach() to modify the search path. r cmd check does not like this and kicks out this note:

❯ checking R code for possible problems ... NOTE
  Found the following calls to attach():
  File ‘envsetup/R/autos.R’:
    attach(NULL, name = name_with_prefix)
    attach(NULL, name = name_with_prefix)
  File ‘envsetup/R/rprofile.R’:
    attach(config_minus_autos$paths, name = "envsetup:paths", pos = pos)
  See section ‘Good practice’ in ‘?attach’.

Is there any chance we can lighten this restriction or make it configurable without copying r-cmd-check.yml into my repo?

Keep as a script 

    For future reference, commands used to regenerate the `renv.lock`:

# within R --vanilla, in an empty directory, in rocker/verse:4.0.5 container
install.packages("renv", repos = c("CRAN" = "https://cran.microsoft.com/snapshot/2021-03-31/"))
renv::init(bare = TRUE)
options(repos = c(
    "CRAN" = "https://cran.microsoft.com/snapshot/2021-03-31/"
))
install.packages("devtools", repos = c("CRAN" = "https://cran.microsoft.com/snapshot/2021-03-31/"))
devtools::install_github("pharmaverse/admiral.test", dependencies = TRUE)
devtools::install_github("pharmaverse/admiraldev", dependencies = TRUE)
devtools::install_github("pharmaverse/admiral", dependencies = TRUE)
devtools::install_github("pharmaverse/admiralonco", dependencies = TRUE)
devtools::install_github("pharmaverse/admiraltemplate", dependencies = TRUE)
devtools::install_github("pharmaverse/admiralophtha", dependencies = TRUE)
devtools::install_github("openpharma/staged.dependencies", dependencies = TRUE)
options(repos = c(
    "UPDATES" = "https://cloud.r-project.org",
    "CRAN" = "https://cran.microsoft.com/snapshot/2021-03-31/"
))

# update packages which would otherwise be downgraded, compared to previous renv.lock file for R 3.6
install.packages("R.cache", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("R.utils", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("brio", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("bslib", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("callr", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("cli", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("cpp11", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("desc", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("downlit", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("ellipsis", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("evaluate", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("fs", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("gert", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("gh", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("glue", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("htmltools", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("jquerylib", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("knitr", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("lifecycle", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("magrittr", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("pkgdown", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("processx", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("ragg", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("rlang", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("rmarkdown", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("roxygen2", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("sass", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("styler", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("systemfonts", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("textshaping", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("tinytex", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("usethis", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("vctrs", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("withr", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("xfun", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("xml2", repos = c("UPDATES" = "cloud.r-project.org"))
install.packages("zip", repos = c("UPDATES" = "cloud.r-project.org"))

renv::snapshot(type = "all")
# manually remove admiral packages from renv.lock

Originally posted by @walkowif in #28 (comment)

Optimize Check Templates Workflow

Currently the Check templates workflows takes anywhere from 15 to 60 minutes to run, which can impact review times for Pull Requests.

https://github.com/pharmaverse/admiralci/blob/main/.github/workflows/check-templates.yml

There is talk of switching to containers , which I am all for, but if that talks awhile to set I would love to see something like devtools::install_deps("admiral") used rather then the installation of the renv.lock. Use of renv.lock means that every single dependency we use for developing admiral is installed, e.g. pkgdown, DT, etc. These are not needed for the templates.

General Issue: Consider use of the HEAD(default) branch as devel

Background Information

In most of the case when user install package from GitHub it's install HEAD(default) branch.

This issue is to consider setting a HEAD branch to devel.

Pros:

  • More easy start for currently development version.
  • For development it can simplify the start process
  • Easy to put integration with codespace as devel will be default branch
  • Less difficult to apply workflow changes (do not need to touch main branch)

Cons:

  • It can impact CI
  • Needs to verify how {stage.dependency} handle default branch.
  • Can cause incompatibility when non experience user install from GitHub on prd environment

I do not find any topic about that, but maybe it was already discussed in the past.

Definition of Done

No response

Create a PoC container image for `admiral` CI/CD and local development workflows

Leverage the widely used rocker RStudio images as the base, and populate these with packages from the renv.lock file in this repo.

Blueprint for creating the admiralci image:

FROM rocker/rstudio:4.1.3
COPY renv.lock renv.lock
RUN <commands to install sysdeps>
RUN R -s -e 'renv::restore()'
ENTRYPOINT ["/init"] # This starts a new RStudio Server session locally

You will find some reference implementations in https://github.com/insightsengineering/ci-images

Fix lintr issues after upgrade to 3.0 on renv propagation

The lintr v3 improve a lot of check and XPath, we need to address all new discovered issues.

  • Upgrade config

    • admiraldev
    • admiral.test
    • admiral
    • admiralci
    • admiralonco
    • ...

  • Fix lintr checks

    • admiraldev
    • admiral.test
    • admiral
    • admiralci
    • admiralonco
    • ...

Use renv.lock for one R-CMD check

On the other hand, in pharmaverse/admiral#1512 I used dplyr 1.0.0 functionality in one of the examples but forgot to update renv.lock. All R-CMD check passed (because they did not use renv.lock). But if we have merged the PR, local R-CMD check would fail for all developers and contributors who use renv.lock.
So maybe it would be better to use renv.lock for the lowest version we are testing.
@bms63 , @thomas-neitmann , what do you think?

Originally posted by @bundfussr in pharmaverse/admiral#1519 (comment)

Display past versions of package on pkgdown site

Using the versions tab - users can move to different versions of the package based on what they have installed. This is unique to Clinical Reporting as users might be locked into a version of the package for several years while using it on a study.

Use common `renv.lock` for all `{admiral}` packages

Proposal

Use a common renv.lock file for all {admiral} packages on GitHub. Keep a centralized renv.lock file in this repository and add all dependencies from every other admiral extension here, and propagate it across all other repositories so that they share the same environment for testing, development, and release.

This is what the proposed workflow would look like this:

graph LR
    Developer --> |Manages fa:fa-lock renv.lock| A[fa:fa-box admiralci]
    A -->|fa:fa-lock renv.lock| C{fa:fa-link Propagator}
    C -->|fa:fa-arrow-right propagate| D[fa:fa-box admiralonco]
    C -->|fa:fa-arrow-right propagate| E[fa:fa-box  admiraldev]
    C -->|fa:fa-arrow-right propagate| F[fa:fa-box admiraltemplate]
    C -->|fa:fa-arrow-right propagate| G[fa:fa-box admiralophtha]
    C -->|fa:fa-arrow-right propagate| H[fa:fa-box admiral]
    C -->|fa:fa-arrow-right propagate| I[fa:fa-box admiral.test]
    C -->|fa:fa-arrow-right propagate| J[fa:fa-box admiralXXX]
    subgraph CI/CD
    C
    end
    subgraph Extensions
    D
    E
    F
    G
    H
    I
   J
    end
Loading

Propagation steps will be as follows:

  1. A developer creates a PR against this repo to update the renv.lock file. Let's refer to this PR as "central PR".
  2. The renv.lock file is validated for consistency and the standard CI/CD workflows are run against the "central PR".
  3. Simultaneously, additional PRs are created automatically (by a bot) on the {admiral} extensions which will validate the changes on the renv.lock file by running the standard CI/CD workflows that the extensions currently use. Let's refer to these PRs as "extension PRs".
  4. If everything looks good on the extensions, the "extension PRs" are merged first, and the "central PR" is merged last.
  5. If things go awry, troubleshooting issues is imperative and the corresponding updates are made to the central "central PR", which propagate updates to the renv.lock to the "extension PRs" and automatically run tests there.
  6. The previous step is repeated until all issues are resolved. Eventually, step 4 is executed.

Justification

  • As a user, if I wanted to install the entire family of admiral packages on a given system, variations in dependencies will cause dependency conflicts and installation issues.
  • Even if installations succeed, results could vary as various versions of dependencies could result in different computed values.
  • From an operation toil perspective, developers will only need to maintain and update the renv.lock file in one location, saving time and effort.

Lint only files that changed.

Example from lintr package:

files <- gh::gh("GET https://api.github.com/repos/r-lib/lintr/pulls/1881/files")
changed_files <- purrr::map_chr(files, "filename")
all_files <- list.files(recursive = TRUE)
exclusions_list <- as.list(setdiff(all_files, changed_files))
lintr::lint_package(exclusions = exclusions_list)

This speed up the pipeline. It can detect change in lintr package or config file to check all files.

Create action to scrape CRAN checks website for errors

The code to get this working is suprisingly simple.

library(rvest)

pkg <- "admiral"
url <- sprintf("https://cran.r-project.org/web/checks/check_results_%s.html", pkg)

checks <- url %>%
  read_html() %>%
  html_element("table") %>%
  html_table()

if (any(checks$Status == "ERROR")) {
  stop("One or more CRAN checks resulted in an error")
}

@cicdguy We'd just need a way to dynamically populate the pkg variable inside the workflow and add some magic to open an issue if an error is thrown.

Originally posted by @thomas-neitmann in pharmaverse/admiral#1469 (comment)

Daily(?) check on CRAN to see if any ERROR is flagged for admiral and admiaralext packages 

    @bundfussr Yes, this should be part of admiralci!

The code to get this working is suprisingly simple.

library(rvest)

pkg <- "admiral"
url <- sprintf("https://cran.r-project.org/web/checks/check_results_%s.html", pkg)

checks <- url %>%
  read_html() %>%
  html_element("table") %>%
  html_table()

if (any(checks$Status == "ERROR")) {
  stop("One or more CRAN checks resulted in an error")
}

@cicdguy We'd just need a way to dynamically populate the pkg variable inside the workflow and add some magic to open an issue if an error is thrown.

Originally posted by @thomas-neitmann in pharmaverse/admiral#1469 (comment)

Bug: renv::restore throws a warning wrt 404 - not found

What happened?

I created a new RStudio Project with the admiral github http. I switched to the devel branch. I did renv::restore and received a warning.

Session Information

R version 4.0.5 (2021-03-31) -- "Shake and Throw"
Copyright (C) 2021 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

Bootstrapping renv 0.16.0 --------------------------------------------------

  • Downloading renv 0.16.0 ... OK
  • Installing renv 0.16.0 ... Done!
  • Successfully installed and loaded renv 0.16.0.
  • Project '~/rstudio/admiral_20230330' loaded. [renv 0.16.0]
  • The project library is out of sync with the lockfile.
  • Use renv::restore() to install packages recorded in the lockfile.

renv::restore()
The following package(s) will be updated:

CRAN ===============================

  • R.cache [* -> 0.15.0]
  • bslib [* -> 0.3.1]
  • callr [* -> 3.7.3]
  • cli [* -> 3.4.1]
  • cpp11 [* -> 0.4.3]
  • desc [* -> 1.4.2]
  • downlit [* -> 0.4.0]
  • evaluate [* -> 0.15]
  • htmltools [* -> 0.5.2]
  • knitr [* -> 1.40]
  • lintr [* -> 3.0.2]
  • pkgdown [* -> 2.0.7]
  • processx [* -> 3.6.1]
  • rlang [* -> 1.0.6]
  • rmarkdown [* -> 2.17]
  • roxygen2 [* -> 7.2.1]
  • sass [* -> 0.4.0]
  • styler [* -> 1.9.1]
  • testthat [* -> 3.1.7]
  • vctrs [* -> 0.4.1]
  • waldo [* -> 0.4.0]
  • withr [* -> 2.4.3]
  • xfun [* -> 0.34]
  • xml2 [* -> 1.3.3]

GitHub =============================

  • staged.dependencies [* -> openpharma/staged.dependencies@main]

RSPM ===============================

  • BH [* -> 1.75.0-0]
  • DT [* -> 0.17]
  • R.methodsS3 [* -> 1.8.1]
  • R.oo [* -> 1.24.0]
  • R.utils [* -> 2.10.1]
  • R6 [* -> 2.5.0]
  • Rcpp [* -> 1.0.6]
  • askpass [* -> 1.1]
  • assertthat [* -> 0.2.1]
  • backports [* -> 1.2.1]
  • base64enc [* -> 0.1-3]
  • brew [* -> 1.0-6]
  • brio [* -> 1.1.1]
  • cachem [* -> 1.0.4]
  • clipr [* -> 0.7.1]
  • commonmark [* -> 1.7]
  • covr [* -> 3.5.1]
  • crayon [* -> 1.4.1]
  • credentials [* -> 1.3.0]
  • crosstalk [* -> 1.1.1]
  • curl [* -> 4.3]
  • cyclocomp [* -> 1.1.0]
  • devtools [* -> 2.3.2]
  • diffdf [* -> 1.0.4]
  • diffobj [* -> 0.3.4]
  • digest [* -> 0.6.27]
  • dplyr [* -> 1.0.5]
  • ellipsis [* -> 0.3.1]
  • fansi [* -> 0.4.2]
  • fastmap [* -> 1.1.0]
  • fs [* -> 1.5.0]
  • generics [* -> 0.1.0]
  • gert [* -> 1.3.0]
  • gh [* -> 1.2.0]
  • git2r [* -> 0.28.0]
  • gitcreds [* -> 0.1.1]
  • glue [* -> 1.4.2]
  • highr [* -> 0.8]
  • hms [* -> 1.0.0]
  • htmlwidgets [* -> 1.5.3]
  • httpuv [* -> 1.5.5]
  • httr [* -> 1.4.2]
  • hunspell [* -> 3.0.1]
  • ini [* -> 0.3.1]
  • jquerylib [* -> 0.1.3]
  • jsonlite [* -> 1.7.2]
  • later [* -> 1.1.0.1]
  • lazyeval [* -> 0.2.2]
  • lifecycle [* -> 1.0.0]
  • lubridate [* -> 1.7.10]
  • magrittr [* -> 2.0.1]
  • memoise [* -> 2.0.0]
  • mime [* -> 0.10]
  • miniUI [* -> 0.1.1.1]
  • openssl [* -> 1.4.3]
  • pillar [* -> 1.5.1]
  • pkgbuild [* -> 1.2.0]
  • pkgconfig [* -> 2.0.3]
  • pkgload [* -> 1.2.0]
  • praise [* -> 1.0.0]
  • prettyunits [* -> 1.1.1]
  • promises [* -> 1.2.0.1]
  • ps [* -> 1.6.0]
  • purrr [* -> 0.3.4]
  • ragg [* -> 1.1.2]
  • rappdirs [* -> 0.3.3]
  • rcmdcheck [* -> 1.3.3]
  • rematch2 [* -> 2.1.2]
  • remotes [* -> 2.2.0]
  • rex [* -> 1.2.0]
  • rprojroot [* -> 2.0.2]
  • rstudioapi [* -> 0.13]
  • rversions [* -> 2.0.2]
  • sessioninfo [* -> 1.1.1]
  • shiny [* -> 1.6.0]
  • sourcetools [* -> 0.1.7]
  • spelling [* -> 2.2]
  • stringi [* -> 1.5.3]
  • stringr [* -> 1.4.0]
  • sys [* -> 3.4]
  • systemfonts [* -> 1.0.1]
  • textshaping [* -> 0.3.3]
  • tibble [* -> 3.1.0]
  • tidyr [* -> 1.1.3]
  • tidyselect [* -> 1.1.0]
  • tinytex [* -> 0.31]
  • usethis [* -> 2.0.1]
  • utf8 [* -> 1.2.1]
  • whisker [* -> 0.4]
  • xmlparsedata [* -> 1.0.5]
  • xopen [* -> 1.0.0]
  • xtable [* -> 1.8-4]
  • yaml [* -> 2.2.1]
  • zip [* -> 2.1.1]

Do you want to proceed? [y/N]: y

renv::restore()

  • The library is already synchronized with the lockfile.

Reproducible Example

No response

Check Template fail on both warnings and errors

When we deprecate a function it emits a warning in a release and later an error in the next release. Since we have quite a few templates it would be good to know if a deprecated function is being used before it starts getting an error.

Adopt versioning and changelog for admiralci

Going forward admiralci should up-version the description file every time there is an admiral release.

We should also start using a changelog to track changes of what has happened/updated/adopted into admiralci at each "release"

I think a simple PR template would be handy to remind us of a few simple updates to makes, e.g.

  • Changelog/News/md update with Issue # and short description of what was addressed
  • Check Description file has been up-versioned from last release
  • Documentation is up to date with your changes, e.g. Vignette, Readme.
  • Communication to wider team of upcoming CI changes

@cicdguy and @galachad what do you think?

Links broken on pkgdown site

Not completely sure how legit we are trying to make admiralci pkgdown site, but someone from gsk was taking a look at it and found that most of the links were broken.

image

Shall we fix these?

General: evaluate the use of GitHub Codespaces to provide a standard environment for admiral development

Background Information

GitHub has recently offered something called Codespaces. I think it may be very helpful in a number of different package development initiatives, including admiral.

My understanding: as part of a repository, you can attach a development environment which is perfectly tailored for that specific project. This could be useful for setting up a training class or for ensuring all development on a specific repo is done in an identical environment. Potential use cases:

  • We put effort (NAMESPACE, DESCRIPTION) into ensuring the USERS of admiral have all the packages necessary, and the expected versions of those packages, in their environment so that admiral will work correctly. But we do not do the same for the community of DEVELOPERS, who are left on their own to ensure they have R 4.0.5, the correct version/edition of devtools (2.3.2), testthat (3e), etc. set up in their environment. While some are fortunate to have a standard, company-provided environment available to them, many others are left to fend for themselves.
  • A workshop on Bookdown, for example, requires Tinytex, etc. to be available with some underlying OS software. This could be challenging for some attendees and tedious/time-consuming for others. And it all gets in the way of the goal of learning about Bookdown. Setting up the environment along with the course content in a repo allows attendees to work in an identical environment customized for the content, e.g. all of the Latex-related software and packages pre-installed. And, since this is accessed through a web browser, someone with limited hardware, e.g. a Google Chromebook, has the same experience as others.
    rstudio::conf(2022): "Zero-setup R workshops with GitHub Codespaces" (David Smith): "What if you could host a workshop using R that required no setup from the participants at all? With GitHub Codespaces, a GitHub repository becomes a cloud-based engine for running R in a container with a single click. Every participant, regardless of the power, configuration or operating system of their laptop will have the same experience, all with NO setup in advance." [GitHub repo]

Definition of Done

  • Determine costs involved, if any.
  • Determine if a standard admiral development codespace can be defined, created and used for development work. Does devtools, testthat, pkgdown all work as needed?
  • Determine the difficulty in modifying the codespace, e.g. if we decide to use a newer version of R or a newer version of packages.
  • Determine the difficulty in porting and modifying the codespace to admiraldev, admiralonco, etc.

CRAN Status Check (create only one issue)

IS there a way to only have this fire off when a new and unique error is detected? If it is the same error on CRAN for the same flavor then we don't need additional issues created.

For example, there is something wrong with the purrr package on CRAN for windows flavor and they are working on a fix, but due to the holiday break they haven't gotten to it yet. However, our CRAN Status Checker (which is awesome) has created quite a few issues over the last two weeks.

tidyverse/purrr#1017

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.