What are the minimum permissions on S3 policy? about fluent-plugin-s3

[ijin]

Hi, the absolute minimal policy would look like this:

{
   "Version": "2012-10-17",
   "Statement":[
      {
         "Effect":"Allow",
         "Action":[
            "s3:ListBucket"
         ],
         "Resource":"arn:aws:s3:::myfluentdbucket"
      },
      {
         "Effect":"Allow",
         "Action":[
            "s3:PutObject"
         ],
         "Resource":"arn:aws:s3:::myfluentdbucket/*"
      }
   ]
}

provided you have 1. independently created "myfluentdbucket" beforehand and have auto_create_bucket set to false and 2. check_apikey_on_start set to false. Otherwise, you will need "s3:CreateBucket" and "s3:ListBucketVersions" respectively.

You might need other policies if you expect each file written to be greater than 16MB (i.e. by changing the buffer_chunk_limit parameter [default 8MB] ) since that is the default threshold that the aws-sdk uses to switch to multipart uploading.

Here are relevant permissions on multipart upload actions:
http://docs.aws.amazon.com/AmazonS3/latest/dev/mpuAndPermissions.html

Briefly looking through the aws-sdk code, it seems only "s3:PutObject" and "s3:AbortMultipartUpload" were sufficient in this case, although I'd recommend you to test it out.

[gaia]

thanks!

[ijin]

Oops, this plugin inherits from TimeSlicedOutput, which overrides the default buffer_chunk_limit to 256MB.

[bryanlarsen]

For the current version you also have to set check_object false or give the GET_OBJECT permission.