thinkr-open / engineering-shiny-book

Engineering Production-Grade Shiny Apps — Published in the R Series

A book about "Engineering Production-Grade Shiny Apps".

To be published in the R Series in 2020.

https://engineering-shiny.org/

Want to help?

Any feedback on the book is very welcome.
Please be sure that you comment on the WIP version of the book, which is at the wip branch.

Feel free to open an issue, or to make a PR if you spot a typo (We're not native English speakers, so there might be some issues waiting to be found ;) ).

engineering-shiny-book's Issues

Caveats — R does too much

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/common-app-caveats.Rmd#L13

Talk about when things could be replaced with JS (for example hide / showing things, generating menus, etc...)

The idea is to introduce to the last part about JS.

Short description of the workflow

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/workflow.Rmd

The idea there is to have a kind of one chapter cheatsheet about the whole workflow. So each subpart should summarize the steps of the workflow.

Complete the definition of complex & successful Shiny App

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/big-shiny.Rmd#L11

There is a list here, but it should be thought more and (maybe) illustrated.

Also, maybe it can be divided into several parts?

Contextualise the "Why"

We need to have a part in the book where we talk about why it's important to have stable foundation.

This part should explain that without solid foundations, the app is likely to fail when you need to ugrade / implement new things, because of a lack of control over what Shiny internally does, because reactivity is not handled well, because the business logic is not understood, and because we are trying to implement things that has not been imagined from the beginning of the project and/or which are not typical Shiny use cases so Shiny might not be natively made for what we do.

Rename the "secure" step

It's not about security, but about sustainability

Update the Reproducible environment with recent tools & content

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/step-by-step-secure.Rmd#L15

• Explain the why
• Show examples
• Define tooling ({renv}, {packrate}, Docker...)

Shiny Modules

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/structure.Rmd#L218

Write something about what are shiny modules and how they work. This should be the technical counterpart of the previous chapter about splitting app into little pieces (#12)

Populate the "Don't rush into coding" part

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/step-by-step-design.Rmd#L5

Maybe find examples, and explain in more depth with it's an anti-pattern to start with the code.

{shinipsum}

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/prototyping-fakir-shinipsum.Rmd#L5

There's nothing on {shinipsum} so far.

Add content about how to determine when we should optimize

The current chapter assumes one would need to optimize but there is nothing about determining if/where one need to optimize.

We should add a chapter about : don't optimize too quickly (#53) / detect bottleneck (profiling app from the server code base, and maybe set up tools to detect UI bottlenecks)

"Big"

Change the term "big" to sthg else like "production" / "successful", etc...

Optimizing - Keep it Simple

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/optimizing-shiny-code.Rmd#L11

Discussion about the fact that it might not be optimal to use certain widgets that can slow your app. People asking for an app want interactive things everywhere, it can be a bad idea, for example showing JS graphs with millions of points.

JS - Shiny JS

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/javascript.Rmd#L11

Some examples of pretty common things you can do with JS in Shiny.

Splitting your app into files

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/structure.Rmd#L222

I'm keeping it right now but I'm not sure it will stay there.

Update the test paragraph with recent tools

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/step-by-step-secure.Rmd#L7

and

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/secure.Rmd#L9

• Explain the philosophy
• Examples
• Details of the tools

Chapter about accessibility

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/challenges.Rmd#L7

Chapter about Web Accessibility

• definition
• what it is and how it can be applied
• example with Shiny
• testing some Shiny apps for accessibility

A (very) short introduction to Shiny

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/big-shiny.Rmd#L9

This part aims at introducing to Shiny. Remember that this book is aiming at (more or less) experienced Shiny users.

Project management

There should be some content about:

• time management
• project management
• testing the app, report bugs
• interaction with the client

JS - Client-side Optimization

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/javascript.Rmd#L3

This part will talk about doing some stuffs only from the client side (browser), without anything going through R.

Talk about how to insert JS code with {golem}.

Chapter about KISS (not the rock band)

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/challenges.Rmd#L17

• Keep It Simple, Stupid

• Don't Make Me Think

Git Workflow

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/challenges.Rmd#L50

Redo this part with our Git Workflow

JS - JS <-> Shiny Com

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/javascript.Rmd#L7

How to build Shiny JS Handler and how to create them with {golem}.

Create an example for each step

The Workflow shoud be exemplified somewhere with an app at each stage.

Caveats — Reactivity anti-patterns

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/common-app-caveats.Rmd#L5

The idea here is to list some weird things that can happen with Shiny's reactivity.

• intro to reactivity
• common anti-patterns (typically to much observe(), question about reactive())
• Solving this with "event based" reactivity

golem is introduced after the part of Shiny as a package

In the shiny app as a package part, we talk about golem, but it's introduced after that. These two parts might have to switch.

Caveats — Reading data

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/common-app-caveats.Rmd#L9

Talk here about loading too big datasets, and how they could be split / cached.

Deploy with golem

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/deploy-with-golem.Rmd

Presentation of the functions, and what to do with them.

The KISS principle, but applied to the code

Currently I've planned on covering the kiss principle in the UI/UX, but it seems important to add that it also applies to the code & package itself.

I'd say as a subpart of 2.

Probably as a way to make the app maintainable on the long run.

Populate the UI first approach

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/step-by-step-prototype.Rmd

• Explain the philosophy
• Find example in other languages / web app development methodologies, for example when you build a website you start with lorem ipsum
• References?

Rewrite the "intro to golem" chapter

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/golem.Rmd

The elements right now are copied and pasted from other elements. The introduction to {golem} here should be re-written.

add contributing.md to the github repo

we should have a contributing.md file on the Github repo

Dividing App into pieces

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/challenges.Rmd#L48

This part should quickly introduce Shiny Modules but this shouldn't be the center element. This part talks about dividing an app into pieces, but for organisational reasons (easier to maintain, to collaborate, to do code review...)

6.3 project planning

Add content about logging things

One important thing about successful Shiny apps is being able to log what happens in the app, while developing and while in production.

I think we should have a chapter about:

• Setting conventions for logging
• Tooling

Tools:

• {whereami} by @yonicd
• {logger} by @daroczig
• {ShinyEventLogger} by @kalimu

Change in Chapter 3

I think chapter 3 should be changed to:

• 3.1. app logic vs business logic #50
• 3.2. app as a package (current 3.1)
• 3.3. using modules #15

typo

Spotted a typo in Chapter 10 Using Golem while trying to install {golem}

remotes::install_github("Thinkr-open/golemn") --> remotes::install_github("Thinkr-open/golem")

Rule of Optimization: Prototype before polishing. Get it working before you optimize it.

For the prototyping chapter:

http://www.catb.org/~esr/writings/taoup/html/ch01s06.html#rule_of_optimization

Create authors bio

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/index.Rmd#L34

Change title

Some other ideas:

• "Engineering Big Shiny Apps"
• "Engineering Successful Shiny Apps"
• "Building Shiny Apps - A Workflow for Building and Maintaining Large Shiny Applications."
• " An Integrated Workflow for Successfully Building, Testing, Deploying, Maintaining, Upgrading and Optimizing Big Shiny Apps"

Scaling & Maintaining

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/challenges.Rmd#L56

This part should talk about the challenge behind scaling and maintaining big apps, with little concern about tools at that point.

About git

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/secure.Rmd#L3

I'm not sure how deep we should go on this one. Maybe introduction + Pointing resources.

CI

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/secure.Rmd#L9

Travis / GitHub Action / Gitlab CI...

And how to test Shiny Apps in that context

Separation of business logic / app logic

We need to stress somewhere how important (and why) it's important to split business and app logic, notably in time.

When implementing new things, it's hard to debug the app behaviour and the business logic at the same time.

If we have a client, they are the one to know what the business logic is, and it's easier to agree on business logic behaviour if the separation is clear.

List questions to ask before starting the app

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/step-by-step-design.Rmd#L9

This list should be thought through and restructured.

Rewording challenges?

Maybe switch to a more "positive" wording like "good practices"

Add info about "becoming out of date" content in the Disclaimer

I think this is important to state in the disclaimer that this book is a mix of practical advice, good practices, and R code, and that even if the R code can become out of date, a big part of the book is more or less "timeless".

Optimizing - Caching

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/optimizing-shiny-code.Rmd#L7

About renderCachedPlot and other cache methods.

Optimizing — Reading data

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/optimizing-shiny-code.Rmd#L3

• backing with database
• split into smaller pieces
• using faster readers like {data.table} or {vroom}

Don’t optimize too quickly

In intro to the optim part

get your design right with an un-optimized, slow, memory-intensive implementation before you try to tune. Then, tune systematically, looking for the places where you can buy big performance wins with the smallest possible increases in local complexity.

http://www.catb.org/~esr/writings/taoup/html/ch01s06.html

{fakir}

https://github.com/ThinkR-open/building-shiny-apps-workflow/blob/master/prototyping-fakir-shinipsum.Rmd#L11

There's nothing on {fakir} so far.