RFC: More Relaxed CodePush Usage for Folio about readme

[brainbicycle]

I am mostly onboard with being a little more aggressive with codepush in folio. With the caveat that I think it would be better to ramp up and see how it goes than to go all out continuous delivery immediately. Also if we did that would require some not insignificant planning and updates to our ci and release processes.

Two risks I am concerned about:

1. Surprise rejections + having to remove codepush:
The rfc quotes the codepush docs, but the docs of the tool are always going to try to assuage concerns like this. I am more interested in the guidelines of apple + google and if anyone has faced issues with this before. There are anecdotal rejections in the issues in the codepush repo: microsoft/react-native-code-push#2416
And historically there have been tools like rollout.io that promised similar and eventually got a blanket ban from the App Store.
That said they are pretty few and far between for a widely used tool, and the guidelines of apple do seem to allow for this. However the end of the day the review process can be a bit arbitrary and up to the interpretation of individual reviewers so I think we should consider that there is a non-zero possibility that we could end up having to remove codepush should we face that issue.

2. Releasing bad / incompatible updates:
@MounirDhahri raised the concern about incompatible updates that requires an update to the native code. We've seen this in practice as just a crash on startup. We have a basic script that checks if native code has been modified but there is a known gap, we don't have locked android native deps and there may be unknown gaps that could lead to an incompatible release. In the short term we could do a manual basic sanity check before releasing any updates into the wild but if we wanted truly continuous integration we need some way of confidently + automatically testing these releases before going out to prod.

For those 2 reasons would be in favor of folio being the vanguard for a more aggressive release process that might lead into more continuous delivery later. And if it goes well bringing it over to Eigen.

At a high level thinking something like this:

1. Any release that can be done via codepush we do via codepush with manual sanity check QAs before rolling out to prod (basically check if the app crashes)
2. When a native change is required do ad hoc standard releases with a standard regression QA
3. Keep track of how it goes, how we might adopt a more automated and continuous process, retro and plan next steps.

[MounirDhahri]

I am in for more ease releasing updates without going through the stores or waiting for the store update. The faster we release such updates, the happier we can keep our galleries. so 👍 for that

When it comes to rethinking the release process in favor of continuous integration in Folio, I think that's going to be a subject we need to be careful about and how often we do it. Some updates might bring some native code changes and can potentially break the app (speaking of palette-mobile deps). I believe such updates will fail thanks to @brainbicycle native code changes check, but when they do, we will need to get back to the normal process of releasing through the store.

Regardless of my worries here, I do though like the premise of it. I love CodePush canary builds in Eigen, they make my life so much easier and QA a ton faster. I would then also say yes to trying this out here, and figuring out a way to make sure we don't release an over-the-air update that breaks the app.

[damassi]

Yes, its crucial to mention here that Brian has already worked out significant safeguards on the native code side! Thank you for raising; I will update the RFC mentioning it.