(Journey) SJ14: Ensure repeatability of common tasks about emblem

[dinagraves]

What environment should run our "remote operations" that are not continuous?

There are three environments: ops, staging, prod

I propose to keep staging as close as possible to prod. Identical if possible, including some kind of data mirroring so our staging is as close to real as possible.

As such, all remote operations should be in the ops environment. This is currently used for building and storing artifacts, and I propose it also be used for nightly tests and other such tasks.

Should all operations be centralized, or should component-specific operations be nested inside the component for more independence?

Operations should be contained within the separate projects. Ie, any operations specific to the management of the prod services-- for instance database management or deployments-- would ideally happen inside the prod project. There will necessarily be some cross-project permissions, but we want to keep it as scoped down as possible. Cross-project permissions are required for things like reading from another project's artifact registry or promoting a revision to prod by publishing to another project's pubsub topic.

[grayside]

setup.sh has some interactive elements for auth setup, we may proof this further for headless running if we start spinning up temporary environments. The current implementation meets expectations.