Metadata lost when copying cross account about aws-lambda-copy-s3-objects

[mwhouser]

@gpip when copying S3 objects between S3 accounts, the object metadata will be lost if you "push" the file. Instead, you need to "pull" the object.

There are 2 ways to copy an object between AWS accounts:

1. "Push" the object to the target account's bucket.
   1. Account B has the target bucket. You apply a bucket policy allowing Account A to write to the bucket.
   2. In the Lambda function, you're executing as Account A, and you perform the copy.
2. "Pull" the object from the source account's bucket.
   1. Account A has the source bucket. You apply a bucket policy allowing Account B to read from the bucket.
   2. In the Lambda function, you're executing as Account B, and you perform the copy. This means you need to use credentials in your Lambda function aside from the role the lambda function is executing as.

In the "Pull" case, the metadata should be copied correctly. I've dealt with AWS support on this issue and they say that you need to "pull" the file rather than "push" it. I received no indication from them if this will ever change.

[gpip]

@mwhouser awesome! Indeed, I was using the push model. Do you remember how you configured the bucket on Account A for this? I have a policy in place for allowing a specific user to read the bucket on Account A from the Account B, but I probably need a different policy (or a different setup) in order to display that bucket when creating a S3 trigger for the lambda func.

[gpip]

I've configured the bucket on Account A to allow s3:Get* and s3:ListBucket for the lambda role in Account B. I also configured an event notification on the bucket on Account A to send it to lambda function, and this part is working. The lambda function is running on Account B and receiving the events, its role is configured to allow s3 operations but for some reason every execution fail due to access denied when trying to access the bucket on Account A.

I skipped the UI to configure the lambda function and instead used the aws cli for lambda add-permission --action "lambda:InvokeFunction" --principal s3.amazonaws.com --source-arn ... --source-account ... and this part seems to be working, as it's triggering the lambda func on Account B based on events happening on the bucket on Account A.

[gpip]

Ah, the bucket was configured to allow s3:Get* on bucketname/*, I missed that for just bucketname so get-bucket-tagging could run. Awesome, it's working now @mwhouser