marlycormar / riskmetric

Metrics to evaluate the risk of R packages

riskmetric

riskmetric is a collection of risk metrics to evaluate the quality of R packages.

This package is in experimentation. Final considerations about design are being considered, but core concepts are considered final.

Background

The risk of using an R package is evaluated based on a number of metrics meant to evaluate development best practices, code documentation, community engagement and development sustainability. We hope to provide a framework to quantify risk by assessing these metrics. This package serves as a starting point for exploring the heterogeneity of code quality, and begin a broader conversation about the validation of R packages. Primarily, this effort aims to provide some context for validation within regulated industries.

We sperate three steps in the workflow to assess the risk of an R package using riskmetric:

1. Finding a source for package information (installed package or CRAN/git source) pkg_ref()
2. Assessing the package under validation criteria assess()
3. Scoring assessment criteria score()
4. Summarize scores into an aggregate risk metric summarize_risk()

The results will be a datasets of validation criteria and its overall risk score for each package as showin in the example below.

Installation

riskmetric is not yet on CRAN. Until it is, install using devtools.

devtools::install_github("pharmaR/riskmetric")

Example

Scrape metadata locally or remotely, then assess that metadata and score it to estimate risk. For each package, derive a composite measure of risk, or a collection of individual scores which can be easily used to generate validation reports.

library(dplyr)
library(riskmetric)

pkg_ref(c("riskmetric", "utils", "tools")) %>%
  as_tibble() %>%
  assess() %>%
  score() %>%
  mutate(risk = summarize_risk(.))

Get Involved

We had a bi-weekly sprint meeting for developer to discuss the progress.

• Contact [email protected] to add into the meeting invitation.
• Date: 1st and 3rd Wednesday of the month.
• Meeting Time: 12:00PM - 12:30PM EST
• Project Planning Meeting Structure
• Milestone
• Meeting Room

riskmetric is centrally a community project. Comfort with a quantification of risk comes via consensus, and for that this project is dependent on close community engagement. There are plenty of ways to help:

• Share the package
• File issues when you encounter bugs
• Weigh in on proposed metrics, or suggest a new one
• Help us devise the best way to summarize risk into a single score
• Help us keep documentation up to date
• Contribute code to tackle the metric backlog