b-rodrigues / chronicler

My name is Bruno Rodrigues and hold a PhD in Economics from the University of Strasbourg.

I’m currently employed as a statistician for the Ministry of Research and Higher Education in Luxembourg. Before that I was senior data scientist and then manager in the data science team at PwC Luxembourg, and before that I was a research assistant at STATEC Research.

My hobbies are boxing, lifting weights, cycling, cooking and reading or listening to audiobooks, which is more compatible with the life of a young father and watching movies. I like blogging about R and statistics, you can read my blog here. I also teach a course about building reproducible analytical pipeline with R, and you can the course notes for free here.

Follow me on Mastodon, twitter, or check out my youtube channel.

chronicler's Issues

add as.data.frame.chronicle

would automatically call pick(.c, "value") or error if .c$value is not coercable to data.frame

Implicitly embellish the functions via pipe?

A colleague of mine pointed me towards this package. I think this is awesome.

A question to the authors:
If the fishbone pipe %>=% is already incorporating bind , why not make another (or the same) that also do map?
This way the functions do not have to be explicitly embellished.

This

r_group_by <- record(group_by)
r_select <- record(select, .g = dim)
r_summarise <- record(summarise, .g = dim)
r_filter <- record(filter, .g = dim)

output_pipe <- starwars %>%
  r_select(height, mass, species, sex) %>=%
  r_group_by(species, sex) %>=%
  r_filter(sex != "male") %>=%
  r_summarise(mass = mean(mass, na.rm = TRUE))

can become

output_pipe <- starwars %>=>%
  select(height, mass, species, sex) %>=>%
  group_by(species, sex) %>=>%
  filter(sex != "male") %>=>%
  summarise(mass = mean(mass, na.rm = TRUE))

Add zap_log()

zap_log() a function to delete the log of a chronicle object

Unit testing record ggplot

It'd be great to unit test record ggplot

but testing plots is not easy. Maybe a snapshot test? See https://testthat.r-lib.org/articles/snapshotting.html

Add write_chronicle_df

 write_chronicle <- function(.c, path, row.names = FALSE, sep = ",", ...){
                                                                          
  stopifnot("Only provide one path" = {length(path) == 1})                
                                                                          
  value <- chronicler::pick(.c, "value")                                  
                                                                          
  stopifnot("Value must be of class data.frame!" = is.data.frame(value))  
                                                                          
  ext <- stringr::str_extract(path,                                       
                              "\\.([0-9a-z]+)(?=[?#])|(\\.)(?:[\\w]+)$")  
                                                                          
  stopifnot("write_chronicle() can only save data as either .csv or .xlsx. Change the extension of the output." = (any(c(".csv", ".xlsx") %in% ext)))
                                 
  log <- chronicler::read_log(.c)
                                 
                                 
  if(ext == ".csv"){             
                                 
    logcsv <- c(paste0("This first ", length(log) + 2, " lines of this .csv file constitute a log."),
             paste0("Skip the first ", length(log) + 2, " lines to read in the data."),    
             log)                                                                          
                                                                                           
    write(logcsv, file = path)                                                             
    suppressWarnings(                                                                      
      write.table(value, file = path, sep = sep, append = TRUE, row.names = row.names, ...)
    )                                                                                      
                                                                                           
  } else {                                                                                 
                                                                                           
    logxlsx <- c("This sheet contains a log of the operations used to create the dataset in the 'value' sheet.",
             log)                                                                              
                                                                                               
    xlsx_output <- list("value" = value,                                                       
                        "log" = logxlsx)                                                       
                                                                                               
    openxlsx::write.xlsx(xlsx_output, file = path)                                             
                                                                                               
  }                                                                                            
                                                                                               
}   

Error messages for failed pipelines

Nice job with the package, the features are coming along well! One thing I noticed that could be improved is how the error messages are displayed when a pipeline fails. In the pipeline below, the second function fails with an informative error message but then the other 2 failure messages are just noise. It may be better to replace them with a generic message like "Pipeline failure upstream".

read_log(errord_output)
#> [1] "Complete log:"                                                                                                                                                                           
#> [2] "✔ select(height,mass,species,sex) ran successfully at 2022-04-15 23:04:20"                                                                                                               
#> [3] "✘ group_by(species,sx) ran unsuccessfully with following exception: Must group by variables found in `.data`.\n✖ Column `sx` is not found. at 2022-04-15 23:04:20"                       
#> [4] "✘ filter(sex != \"male\") ran unsuccessfully with following exception: no applicable method for 'filter' applied to an object of class \"logical\" at 2022-04-15 23:04:20"               
#> [5] "✘ summarise(mean(mass, na.rm = TRUE)) ran unsuccessfully with following exception: no applicable method for 'summarise' applied to an object of class \"logical\" at 2022-04-15 23:04:20"
#> [6] "Total running time: 0.0497806072235107 secs"

Signal to user that log was carried over

if object a has a log, and I build object b on top of a, it would be nice if read_log() would signal which part is from a and which from b

Self-document ggplot using this code?

see

chronicler/R/gg_chronicle.R

Line 132 in 1bab293

document_gg2 <- function(.c, layout = "

could this work with the current record_ggplot?

Check if supplying named arguments works with logged functions

Noticed that r_unique <- record(data.table:::unique.data.table)

then r_unique(a, by = c(x,y)) seems to ignore by argument