ropensci-books / targets Goto Github PK

Prework

• I understand and agree to this repository's code of conduct.
• I understand and agree to this repository's contributing guidelines.

Prework

• I understand and agree to this repository's code of conduct.
• I understand and agree to this repository's contributing guidelines.
• New features take time and effort to create, and they take even more effort to maintain. So if the purpose of the feature is to resolve a struggle you are encountering personally, please consider first posting a "trouble" or "other" issue so we can discuss your use case and search for existing solutions first.

Proposal

Even if the targetopia is mentioned twice in the book, I'd find it useful to see a sentence about it in the introduction, especially to put tarchetypes on readers' radar early on?

Prework

• I understand and agree to this repository's code of conduct.
• I understand and agree to this repository's contributing guidelines.
• New features take time and effort to create, and they take even more effort to maintain. So if the purpose of the feature is to resolve a struggle you are encountering personally, please consider first posting a "trouble" or "other" issue so we can discuss your use case and search for existing solutions first.

Proposal

The issue of managing sub-projects comes up a lot, and I think I should write a chapter to explain best practices. This chapter would also cover YAML-based configuration, e.g. _targets.yaml/tar_config_get()/tar_config_set(). I plan to write it sometime after I address these issues:

• ropensci/targets#622
• ropensci/targets#623
• ropensci/tarchetypes#68

Prework

• I understand and agree to this repository's code of conduct.
• I understand and agree to this repository's contributing guidelines.
• I am reasonably sure this is a genuine bug in this repository's code and most likely not a user error. (If you run into an error and do not know the cause, please submit a "Trouble" issue instead.)

Description

Chapter 2 mentions a targets-minimal repo that contains the source code and data for the walkthrough example, but the link fails.

Prework

• I understand and agree to this repository's code of conduct.
• I understand and agree to this repository's contributing guidelines.
• New features take time and effort to create, and they take even more effort to maintain. So if the purpose of the feature is to resolve a struggle you are encountering personally, please consider first posting a "trouble" or "other" issue so we can discuss your use case and search for existing solutions first.

Proposal

Please describe the potential feature.

To help us read any code you include (optional) please try to follow the tidyverse style guide. The style_text() and style_file() functions from the styler package make it easier.

Prework

• I understand and agree to this repository's code of conduct.
• I understand and agree to this repository's contributing guidelines.
• New features take time and effort to create, and they take even more effort to maintain. So if the purpose of the feature is to resolve a struggle you are encountering personally, please consider first posting a "trouble" or "other" issue so we can discuss your use case and search for existing solutions first.

Proposal

The current comparison with drake does not include the many arguments of drake::make that may or may not have been implemented elsewhere in targets. For example, I recently learned about tar_option_set(storage = "remote", retrieval = "remote"). The proposal is to include the arguments of make in the "What about drake?" section.

Prework

• I understand and agree to this repository's code of conduct.
• I understand and agree to this repository's contributing guidelines.

Description

After tarchetypes becomes open source.

I propose adding a new "Edge cases that fail" subsection to the walkthrough chapter (or elsewhere), and collating all code dependencies failure cases there.

This could include:

1. Namespaced function calls pkg::func() breaking things
2. The fact that targets does not correctly invalidate targets generated by S3 methods when those methods get updated.

For the second, consider

library(targets)

my_generic <- function(x) {
  UseMethod("my_generic")
}

my_generic.default <- function(x) {
  sum(x^3)
}

list(
  tar_target(
    vec,
    rnorm(100)
  ),
  tar_target(
    result,
    my_generic(vec)
  )
)

Run tar_make(), change my_generic.default() to anything you want, and note that running tar_make() again does not re-run the result target.

Prework

• Read and agree to the code of conduct and contributing guidelines.
• If there is already a relevant issue, whether open or closed, comment on the existing thread instead of posting a new issue.
• For any problems you identify, post a minimal reproducible example like this one so the maintainer can troubleshoot. A reproducible example is:
  • Runnable: post enough R code and data so any onlooker can create the error on their own computer.
  • Minimal: reduce runtime wherever possible and remove complicated details that are irrelevant to the issue at hand.
  • Readable: format your code according to the tidyverse style guide.

Description

• Configurable target script file path: ropensci/targets#480
• Target Markdown script path: ropensci/targets#478

Prework

• Read and agree to the code of conduct and contributing guidelines.
• If there is already a relevant issue, whether open or closed, comment on the existing thread instead of posting a new issue.

Description

I will be on vacation from September 13 through 20, and I will not have internet access most of the time. If you have experience with targets, it would be great if you would help answer new questions posted to issues and discussions. There are few new posts in a typical week. Thanks in advance.

Suggested changes in bold here:

targets is the successor of drake, an older pipeline tool. drake is superseded, which means there are no plans for new features or discretionary enhancements, but basic maintenance and support and will continue indefinitely. Existing projects that use drake can safely continue to use drake, and there is no need to retrofit targets. New projects should use targets because it is friendlier and more robust.

Prework

• I understand and agree to this repository's code of conduct.
• I understand and agree to this repository's contributing guidelines.
• I am reasonably sure this is a genuine bug in this repository's code and most likely not a user error. (If you run into an error and do not know the cause, please submit a "Trouble" issue instead.)

Description

Chapter 5 header in table of contents hyperlinks to Section 2.2.

Prework

• I understand and agree to this repository's code of conduct.
• I understand and agree to this repository's contributing guidelines.

Description

code of conduct issue templates, etc.

Prework

• Read and agree to the code of conduct and contributing guidelines.
• If there is already a relevant issue, whether open or closed, comment on the existing thread instead of posting a new issue.
• For any problems you identify, post a minimal reproducible example like this one so the maintainer can troubleshoot. A reproducible example is:
  • Runnable: post enough R code and data so any onlooker can create the error on their own computer.
  • Minimal: reduce runtime wherever possible and remove complicated details that are irrelevant to the issue at hand.
  • Readable: format your code according to the tidyverse style guide.

Description

The section "Target script file" from Ch. 2 suggests loading packages other than targets via tar_option_set instead of library.

https://books.ropensci.org/targets/walkthrough.html#target-script-file

However, the section "Workspace" from Ch. 4 is using library(tidyverse) in _targets.R.

https://books.ropensci.org/targets/debugging.html#workspaces

I'd suggest the change below.

atusy@04512f5

Please describe the issue.

To help us read any code you include (optional) please try to follow the tidyverse style guide. The style_text() and style_file() functions from the styler package make it easier.

Prework

• Read and agree to the code of conduct and contributing guidelines.
• If there is already a relevant issue, whether open or closed, comment on the existing thread instead of posting a new issue.
• For any problems you identify, post a minimal reproducible example like this one so the maintainer can troubleshoot. A reproducible example is:
  • Runnable: post enough R code and data so any onlooker can create the error on their own computer.
  • Minimal: reduce runtime wherever possible and remove complicated details that are irrelevant to the issue at hand.
  • Readable: format your code according to the tidyverse style guide.

Description

ropensci/targets#959

Prework

• I understand and agree to this repository's code of conduct.
• I understand and agree to this repository's contributing guidelines.
• I am reasonably sure this is a genuine bug in this repository's code and most likely not a user error. (If you run into an error and do not know the cause, please submit a "Trouble" issue instead.)

Description

Looks like there's just a typo here:

https://github.com/wlandau/targets-manual/blob/fbcfc9a3b1b7d9761a491b41cc3ae3672c32a34e/practice.Rmd#L65

I would even just say targets::tar_make(callr_function = NULL) there

Prework

• I understand and agree to this repository's code of conduct.
• I understand and agree to this repository's contributing guidelines.
• New features take time and effort to create, and they take even more effort to maintain. So if the purpose of the feature is to resolve a struggle you are encountering personally, please consider first posting a "trouble" or "other" issue so we can discuss your use case and search for existing solutions first.

Proposal

tar_rep(), tar_map(), and tar_combine() in tarchetypes.

c.f. ropensci/targets#1139

Prework

• Read and agree to the code of conduct and contributing guidelines.
• If there is already a relevant issue, whether open or closed, comment on the existing thread instead of posting a new issue.
• For any problems you identify, post a minimal reproducible example like this one so the maintainer can troubleshoot. A reproducible example is:
  • Runnable: post enough R code and data so any onlooker can create the error on their own computer.
  • Minimal: reduce runtime wherever possible and remove complicated details that are irrelevant to the issue at hand.
  • Readable: format your code according to the tidyverse style guide.

Prework

• I understand and agree to this repository's code of conduct.
• I understand and agree to this repository's contributing guidelines.
• New features take time and effort to create, and they take even more effort to maintain. So if the purpose of the feature is to resolve a struggle you are encountering personally, please consider first posting a "trouble" or "other" issue so we can discuss your use case and search for existing solutions first.

Proposal

There is a community misconception that targets (and drake) do not have HPC capabilities beyond parallel computing over the cores of a single local machine. On the contrary, both tools support distributed computing on clusters (guides here and here) and the workers do not necessarily need access to the file system of the master process. (In fact, I designed targets with an efficient dynamic branching model to go beyond the inherent limitations of map-reduce-like scheduling algorithms and conserve computing resources.) However, I do realize that data scientists from smaller institutions do not always have access to clusters, and an increasing number of folks use AWS. AWS ParallelCluster could be a way to deploy pipelines to the cloud without any need to modify targets itself. If it works, we should probably write a tutorial either in the existing HPC chapter or a chapter of its own.

ropensci/tarchetypes#8 could be an alternative way to deploy to AWS. The advantage of ropensci/tarchetypes#8 is that we should also get the data versioning capabilities of Metaflow for free, and Metaflow may take care of a lot of the AWS setup. However, each new tar_metaflow() will require its own local R worker in order to avoid blocking the master process, which is not ideal.

Prework

• I understand and agree to this repository's code of conduct.
• I understand and agree to this repository's contributing guidelines.
• New features take time and effort to create, and they take even more effort to maintain. So if the purpose of the feature is to resolve a struggle you are encountering personally, please consider first posting a "trouble" or "other" issue so we can discuss your use case and search for existing solutions first.

Proposal

I am getting started with targets (such a cool package / ecosystem of packages!) and started wondering how to create a target with a cue that'd be different from the ones in targets, then I discovered https://docs.ropensci.org/tarchetypes/reference/index.html#section-targets-with-custom-invalidation-rules which is exactly what I need. Should these functions be mentioned from

• https://docs.ropensci.org/targets/reference/tar_cue.html
• and/or somewhere in this book?

Prework

• I understand and agree to this repository's code of conduct.
• I understand and agree to this repository's contributing guidelines.

Description

• Reconfigure issue templates to offload to the discussions: https://github.com/ropensci/targets/tree/main/.github/ISSUE_TEMPLATE
• Edit the contributing guide, e.g. ropensci/targets@ae50ff0
• Customize issue categories: https://github.com/ropensci/targets/discussions

Prework

• I understand and agree to this repository's code of conduct.
• I understand and agree to this repository's contributing guidelines.

Description

Talk about https://stackoverflow.com/questions/63293094/how-do-i-read-dynamic-files-in-drake in branching.Rmd.

Prework

• Read and agree to the code of conduct and contributing guidelines.
• If there is already a relevant issue, whether open or closed, comment on the existing thread instead of posting a new issue.

Description

The hyperlink is faulty in Chapter 2.4 due to a missing "s" in the hyperlink.

Prework

• Read and agree to the code of conduct and contributing guidelines.
• If there is already a relevant issue, whether open or closed, comment on the existing thread instead of posting a new issue.

Description

The same sentence should be fixed in both lines, "workers do no have access" to "workers do not have access".

Prework

• Read and agree to the code of conduct and contributing guidelines.
• If there is already a relevant issue, whether open or closed, comment on the existing thread instead of posting a new issue.
• [n/a] For any problems you identify, post a minimal reproducible example like this one so the maintainer can troubleshoot. A reproducible example is:
  • [n/a] Runnable: post enough R code and data so any onlooker can create the error on their own computer.
  • [n/a] Minimal: reduce runtime wherever possible and remove complicated details that are irrelevant to the issue at hand.
  • [n/a] Readable: format your code according to the tidyverse style guide.

Description

• Shortcutting: ropensci/targets#523
• Branch selection helpers: ropensci/targets#525

Prework

• Read and agree to the code of conduct and contributing guidelines.
• If there is already a relevant issue, whether open or closed, comment on the existing thread instead of posting a new issue.
• For any problems you identify, post a minimal reproducible example like this one so the maintainer can troubleshoot. A reproducible example is:
  • Runnable: post enough R code and data so any onlooker can create the error on their own computer.
  • Minimal: reduce runtime wherever possible and remove complicated details that are irrelevant to the issue at hand.
  • Readable: format your code according to the tidyverse style guide.

Description

Explain how to debug tar_rep() and friends, especially after ropensci/tarchetypes#111. Also explain how to get help: GitHub discussions, reprexes, etc.

https://github.com/ropensci-books/targets/blob/203e6207cc16c5a24a0b84226fbbc623012150b0/literate-programming.qmd#L24C1-L24C1

The link to Appendix C (Target Markdown) is broken because the heading hash identifier (#markdown) is not defined in markdown.qmd. I will create a pull request; please review.

Prework

• I understand and agree to this repository's code of conduct.
• I understand and agree to this repository's contributing guidelines.
• New features take time and effort to create, and they take even more effort to maintain. So if the purpose of the feature is to resolve a struggle you are encountering personally, please consider first posting a "trouble" or "other" issue so we can discuss your use case and search for existing solutions first.

Proposal

In https://books.ropensci.org/targets/walkthrough.html#read-your-data it seems tar_read() is only for exploratory analysis, but it's actually recommended for R Markdown reports. Maybe this section should have a footnote/link to the section about literate programming?

Then, I've learnt (in rOpenSci slack) that using tar_read() in a helper function is an anti-pattern, that if a function uses a target the target should be an argument. Would this tip/warning have its place somewhere? (maybe in best practice; as well as in https://books.ropensci.org/targets/practices.html#dependencies?)

Prework

• I understand and agree to this repository's code of conduct.
• I understand and agree to this repository's contributing guidelines.

Description

https://github.com/ropensci-books/targets/blob/4ea7035d7c087e609ffb76928757777181c59c95/walkthrough.qmd#L87C61-L87C151

Hi,

I was just having a read through of this documentation. It seems to me like perhaps this link should point to https://books.ropensci.org/targets/functions.html and not https://books.ropensci.org/targets/debugging.html#functions

Please ignore if I'm mistaken about this!

Thanks 😊

Prework

• I understand and agree to this repository's code of conduct.
• I understand and agree to this repository's contributing guidelines.
• I am reasonably sure this is a genuine bug in this repository's code and most likely not a user error. (If you run into an error and do not know the cause, please submit a "Trouble" issue instead.)

Description

Please describe the bug.

The user manual is a great resource. Thanks for putting it together. Noticed this minor repeated word typo in the debugging chapter the other day.

• https://github.com/ropensci-books/targets/blob/main/debugging.qmd#L17
• "This chapter describes solutions to these challenges in terms of both best practices and and features in targets."

Reproducible example

Provide a minimal reproducible example with code and output that demonstrates the problem. The reprex() function from the reprex package is extremely helpful for this.

To help us read your code, please follow the tidyverse style guide. The style_text() and style_file() functions from the styler package make it easier.

Expected result

What should have happened? Please be as specific as possible.

• "This chapter describes solutions to these challenges in terms of both best practices and features in targets."

Diagnostic information

• A reproducible example.
• Session info, available through sessionInfo() or reprex(si = TRUE).
• A stack trace from traceback() or rlang::trace_back().
• The SHA-1 hash of the GitHub commit of targets currently installed. packageDescription("targets")$GithubSHA1 shows you this.