rconsortium / r-certification-wg

R Certification Working Group

Statistical programming for clinical trials is moving towards a multi-language environment where submissions to regulatory agencies may include code written in SAS, R and other languages (e.g. python, STATA, etc). The mission of this working group is to develop the content for a process to certify that statistical programmers have the basic skills to function effectively in a mixed SAS and R clinical trial programming environment.

Topics will include:

• FDA CDISC Standards
• Relevant SAS and R data structures
• The basics of managing SAS and R environments
• Reporting methods in SAS and R
• Developing tests to compare the results of SAS and R analyses.

r-certification-wg's Issues

Levels of Exam

-Non-Clinical Exam (R data structure, String manipulation, Data Handling, Simple functions, Basic R graphs, R Markdown) + (Data visualization, Ggplot, Ggplot extension, Intro to functions, writing function)
-Base R Certification (Data manipulation, modeling, visualization)
-Clinical Package Developer (dev-tools, base R functionality emphasis)
-Statistical Clinical Programmer (SAP, R Objects, Survival, Covariate Adjustment Adverse Events, Mixed-Modeling, Cochran Tests, etc.)

• There will be overlap for non-clinical vs clinical programming exam structure

10 questions for basic sas knowledge

packages to read SAS, SAS formats, properties (label/character length, formats)

Create Learning Resources Markdown

Survival Analysis Enhanced Analysis

-What packages might we want to have to suggest people have some exposure and competency for
-Visualization for output plots

yml test for certiverse platform

List of Packages for Exam

Create list of packages used in exam to keep tabs of user updates.

--> Pull from Github/CRAN if major updates and refactor exam

10 questions for reporting

tlf; includes plots

https://pharmaverse.org/e2eclinical/tlg/
https://github.com/RConsortium/rtrs-wg

For reference ^

Exam prep guide

Consolidate exam prep material for end users preparation.

10 questions for cdisc standards

-Review SAS Clinical Programming Exam Content Guide for some appropriate level of CDISC standard questions

• We want to broaden this to GCP and CDISC, covering the base terminology needed for Clinical Programming

assess package use in clinical trials

Whats an acceptable package https://www.pharmar.org/
https://cran.r-project.org/web/packages/riskmetric/index.html

https://pharmaverse.org/

https://www.r-consortium.org/blog/2021/12/08/successful-r-based-test-package-submitted-to-fda

Test

Domain Competencies

Core Competencies for exam aka Blueprint

Scope Statement and Exam Title

Certiverse Scope Statement and Exam Title

Related work from PHUSE Working Group

@kevin-procogia @kostbye
The PHUSE WG, Clinical Statistical Reporting in a Multilingual World, has made considerable progress investigating the differences in the results produced by SAS and R in four areas: Linear Models, Survival Analysis, Mixed Models, and Mantel-Haenszel Tests. Their test code is available in their repo here: https://github.com/phuse-org/CSRMLW/tree/main/cmh.

The video of their recent R Consortium webinar (https://www.youtube.com/watch?v=S7sHaakq0_o) provides a really nice overview.
I think we should reference, and maybe incorporate some of this work in our document.

R Consortium September 2024 newsletter: share updates of your working group

Hi @joseph-ProCogia and team,
I am Coline Zeballos, R Strategy Lead at Roche and member of the R Consortium's Marketing committee.
I am currently collecting working group updates for the RC's quarterly newsletter which will be published at the end of September 2024. Would you like to share a couple of updates for your WG? If so, please do so until Sept 16th. Thank you!
Coline, on behalf of the R Consortium Marketing Committee

10 questions for general syntax

Sub-Topics of Domains

• Create a list of topics for each domain (question categories) for users and exam

Exam Content Format Review

-Review existing questions for YAML structure
-Review existing questions for multiple choice options

Where to host the exam?

-Where to host, how to mark?
-Automated grading?

Reporting.yml

I'm not sure how to approach this. As written, it mostly focuses on r2rtf, but https://pharmaverse.org/e2eclinical/tlg/ has an example of 4 different frameworks which could instead be used for reporting. Also, when it comes to regulatory submissions, Rdata is not recommended; this still needs to be XPT for now (I also wouldnt recommend rdata in general, preferring either RDS or ideally a non language specific format such as parquet).

I think at this point in time we can't really ask questions on this topic without being too specific. I would recommend replacing this with a data manipulation set of questions, and maybe one on graphics using ggplot (which I think is uncontroversially the tool of choice for generating graphs in R)

Review Feedback for Exam Questions

Randomization of Exam Questions

-Create function for exam randomization
-Add weights for exam questions for Exam (ex: syntax needs more questions than CDISC for exam takers)

Environmental Factors for Analysts

"trustworthiness of the environment being used"

From a users perspective do they want to update the version of R package?

Standards around package updates, environments and dependency checks.

--> Knowing reliability of a package (GitHub vs CRAN)

Exam Beta Launch

-Test run for exam
-Provide feedback for exam takers

Suggestions to update foundational skills

• sas vs r: Sas is made up of parts (ilm, data step, sql)
• Add an intro to R packages: cran and bioconductor,
• core suite of packages. split by task
• proc sql as gateway to tidyverse

10 questions for exploratory analysis

summary statistics; data exploration

10 questions for output stats/modelling

focus on mle and survival analysis

core textbooks to go with learning

Exam Key

-Create Exam Key for Automated Review