jeremychone / rust-ss3 Goto Github PK

Implement the sync command. (see #5 for original request)

Definition work

• Look at the aws-cli sync command and see if all is sound.
• Must handle small to humongous syncs.
• Should look at etags and how the aw-cli does it.
• Perhaps a mode where if present, don't do anything, and another mod echeck etags/"equality".
• Command must be compatible when bucket-to-bucket sync will be supported.

Thanks for making ss3. I'm using it to download an S3 bucket to local files. The bucket objects were created over several years, but the downloaded files all show today's date. I wish ss3 would set the file creation and modification times to the times of the source S3 objects.

The tool could also set the file creation time to the time of the oldest object version.

Would be neat to use something like https://docs.rs/indicatif/ (or any of the other progress bar crates) for a file upload progress bar.

I'd really like to use this tool over the normal awscli Python based however, two operations that are missing are the sync and rm file operations.

It should be possible to list buckets with this command, rather than returning an error:

ss3 ls s3://

Implement the rm command. (see #5 origiinal request)

Spec

• All rm command will prompt the user to confirm the delete
• -f or --force flag (or something like it) to bypass the prompt

Happy paths:

# Full path, to remove a key (will prompt to confirm).
ss3 rm ss3://bucket-name/path/to/key.txt

# Same as above, but no prompt.
ss3 rm ss3://bucket-name/path/to/key.txt --force

# Prefix recursive delete (--recursive or -r).
ss3 rm ss3://bucket-name/path/to/key.txt --recursive

Fail paths:

# WILL FAIL - Cannot delete a full folder without the `-r` flag.
ss3 rm ss3://bucket-name/path/to

# WILL FAIL - Cannot delete bucket (but with -r will work and delete all content of the bucket)
ss3 rm ss3://bucket-name

# WILL FAIL - Cannot delete buckets
ss3 rm ss3://

For a multiple file upload - I'd be happy if this tool took advantage of Rust's concurrency properties to upload a file on each CPU available by default. A separate user option to specify the number of threads could of course also be available.

AWS do seem to impose limits, but the number are still quite high:

• https://docs.aws.amazon.com/AmazonS3/latest/userguide/optimizing-performance.html
• https://repost.aws/knowledge-center/http-5xx-errors-s3