delete old revisions keeping the most 5-10 recent about ember-cli-deploy-s3-index

[LevelbossMike]

We had a functionality for this in the original version of ember-deploy-s3-index in the past. We didn't implement it in this addon as it is basically free to host on s3 and there's not really a point in deleting a revision.in addition to that people where experiencing problems when only 5-10 revisions where available as this meant that you'd have to make sure that you don't delete the active revision if that was the "last" available one which complicates the upload process etc..

I am not totally against adding this if somebody actually has the need for this but I think in all honesty that this is not really a problem and fine the way it is due to the reasons I already mentioned.

The UI to handle activating revisions etc. is handled by the display-revisions plugin and that defaults to showing you the latest 10 revisions anyway. You can configure to only see the last 5 for example by specifying the amount-property of the display-revisions plugin.

I'm closing this for now but as I said I'd be happy to assist if that's really an issue for you.

[jrdn91]

I had read something that made me believe they do charge for storage. Is this not actually the case?

[LevelbossMike]

Yes but that's 3 cent per GB and the index.html that is stored there is about 5-6kb (i.e. you pay 3 cent per > 160_000 index files). So that's what I meant by "basically free".

[onsmith]

+1

I think the s3 storage argument misses the point.

I feel strongly that continuing to serve every deprecated revision of one's web app forever is a bad practice.

There should be a canonical way to remove old revisions.

[LevelbossMike]

@onsmith
Why do you feel that the s3-storage argument misses the point and keeping revisions is a bad practice?

As I stated earlier in this issue I'm happy to assist anybody that wants to build automatic deletion into the plugin I am just not convinced of the idea enough that I would implement that myself. We had this functionality in the project before and it lead to more headaches than it provided value to users.

Also it won't be done by deleting index files for a release alone. If you go deeper down the rabbit hole you will also need to figure out what other assets you can delete from other blob-storages if you want to get rid from a release. It isn't really trivial to figure out which app.js|css files and vendor.js|css files can be deleted given a release and one dumping ground for index-files and assets. This also gets complicated by the fact that you could have other arbitrary assets uploaded for a release as well.

If you really have the need delete releases because of compliancy regulations etc. I would suggest a different sort of deployment workflow that dynamically creates buckets before a release and uses these dynamically created buckets to store all relevant assets for a release. You could then be sure that all assets for one release would be contained in one bucket and delete those with a periodic cleanup task or by going through them manually. This of course complicates the release process quite a bit and I doubt that this is a common enough use case to bundle it up in this addon or ember-cli-deploy directly. You can always create your own plugin for use-cases like this of course.

I'm not saying I can't be convinced otherwise right now I just don't see the added value to the overall project.

Can you please me more specific in voicing your concerns why you feel that this needs to be implemented and the concrete value that you feel this would add to the project/plugin.

[onsmith]

Well, you're right that storing a few extra MBs on S3 won't matter in terms of pricing. So if my only concern was storage, it wouldn't matter.

My apprehension, however, is that I don't want old versions of my app to remain publicly accessible (even if the URL to the old versions is obfuscated by appending the hash to the html file name). This could be a security concern, as bugs in the client are patched, but it also seems unprofessional. Imagine if we could access early versions of Facebook, Twitter, or any other website by just hashing their contents.

I could be misguided about this (and if I am, please set me straight!). I'm new to ember-cli-deploy, and am currently trying to decide whether to implement ember-cli-deploy-s3/ember-cli-deploy-s3-index at all, or if we should just continue deploying by uploading our ember code manually to s3.

[LevelbossMike]

I'm not sure how this would be a security concern as you decide which version of your app you are serving to your users by default. It's actually pretty useful to keep older revisions around as this makes it much easier and faster to rollback to a known good revision in case of a severe problem occuring in a production setting (something that isn't fast as well by only uploading code manually to s3 btw). The revision key is not necessarily a hash of the index.html you can tag by other mechanism (e.g. the git-sha) as well. Also I can't follow why it would be unprofessional to keep older revisions around and I am not sure what threshold of revisions kept available would be considered to be professional from your standpoint.

I explained my reasoning why I don't think this is a problem and with a multitude of projects and in several years of using this approach I have never found this to be an issue on client projects.

If you really feel that this needs to be possible I'm happy to accept a pull request for this behind a option flag that makes it configurable how many revisions are kept available the default of this option being that no revisions are deleted.

[onsmith]

Maybe I don't understand the system correctly.

Doesn't ember-cli-deploy-s3-index continue to store every old index file in the s3 bucket? Doesn't this mean that every single deprecated index file is accessible to the public at all times, as long as you know the obfuscated URL?

[LevelbossMike]

Your understanding is correct.

To access the non-active index files you need know the revision-key. This can be any unique key you specify but per ember-cli-deploy's default this will be the file-hash of the index.html file.

The regular users of your application will not access those revisions because they will not access your bucket-files directly.

If you are concerned about some people finding a way to access your old revisions you are correct that people that hashed your index.html files would be able to access those old revisions if you used the default settings. If they wanted to be able to access those files they could also download your index.html files whenever you activate a new revision though.

This is not a security concern as these revisions would have been made available to the public anyway. As all other assets that you are serving in your ember application. Your ember application is always a public asset. People can download it and look at the source whenever they please.

So as I said multiple times. I don't agree that this is bad practice though I understand the idea that people want to be able to get rid of old revisions. I outlined an approach that makes sure you can delete all assets of a revision. I would also assist anybody who wants to create a PR that gets rid of old revisions automatically via an optional option on the plugin.

If you want the ability to delete revisions via a deploy-command this would need to be added to ember-cli-deploy directly. There is no way to delete revisions via a command right now.