Coder Social home page Coder Social logo

carrierwave-aws's Introduction

Carrierwave AWS Storage

Test Code Climate Gem Version

Use the officially supported AWS-SDK library for S3 storage rather than relying on fog. There are several things going for it:

  • Full featured, it supports more of the API than Fog
  • Significantly smaller footprint
  • Fewer dependencies
  • Clear documentation

Here is a simple comparison table [07/17/2013]

Library Disk Space Lines of Code Boot Time Runtime Deps Develop Deps
fog 28.0M 133469 0.693 9 11
aws-sdk 5.4M 90290 0.098 3 8

Installation

Add this line to your application's Gemfile:

gem 'carrierwave-aws'

Run the bundle command from your shell to install it:

bundle install

Usage

Configure and use it just like you would Fog. The only notable difference is the use of aws_bucket instead of fog_directory, and aws_acl instead of fog_public.

CarrierWave.configure do |config|
  config.storage    = :aws
  config.aws_bucket = ENV.fetch('S3_BUCKET_NAME') # for AWS-side bucket access permissions config, see section below
  config.aws_acl    = 'private'

  # Optionally define an asset host for configurations that are fronted by a
  # content host, such as CloudFront.
  config.asset_host = 'http://example.com'
  # config.asset_host = proc { |file| ... } # or can be a proc

  # The maximum period for authenticated_urls is only 7 days.
  config.aws_authenticated_url_expiration = 60 * 60 * 24 * 7

  # Set custom options such as cache control to leverage browser caching.
  # You can use either a static Hash or a Proc.
  config.aws_attributes = -> { {
    expires: 1.week.from_now.httpdate,
    cache_control: 'max-age=604800'
  } }

  config.aws_credentials = {
    access_key_id:     ENV.fetch('AWS_ACCESS_KEY_ID'),
    secret_access_key: ENV.fetch('AWS_SECRET_ACCESS_KEY'),
    region:            ENV.fetch('AWS_REGION'), # Required
    stub_responses:    Rails.env.test? # Optional, avoid hitting S3 actual during tests
  }

  # Optional: Signing of download urls, e.g. for serving private content through
  # CloudFront. Be sure you have the `cloudfront-signer` gem installed and
  # configured:
  # config.aws_signer = -> (unsigned_url, options) do
  #   Aws::CF::Signer.sign_url(unsigned_url, options)
  # end
end

Custom options for S3 endpoint

If you are using a non-standard endpoint for S3 service (eg: Swiss-based Exoscale S3) you can override it like this

    config.aws_credentials[:endpoint] = 'my.custom.s3.service.com'

Custom options for AWS URLs

If you have a custom uploader that specifies additional headers for each URL, please try the following example:

class MyUploader < Carrierwave::Uploader::Base
  # Storage configuration within the uploader supercedes the global CarrierWave
  # config, so either comment out `storage :file`, or remove that line, otherwise
  # AWS will not be used.
  storage :aws

  # You can find a full list of custom headers in AWS SDK documentation on
  # AWS::S3::S3Object
  def download_url(filename)
    url(response_content_disposition: %Q{attachment; filename="#{filename}"})
  end
end

Configure the role for bucket access

The IAM role accessing the AWS bucket specified when configuring CarrierWave needs to be given access permissions to that bucket. Apart from the obvious permissions required depending on what you want to do (read, write, delete…), you need to grant the s3:PutObjectAcl permission (a permission to manipulate single objects´ access permissions) lest you receive an AccessDenied error. The policy for the role will look something like this:

PolicyDocument:
  Version: '2012-10-17'
  Statement:
  - Effect: Allow
    Action:
    - s3:ListBucket
    Resource: !Sub 'arn:aws:s3:::${BucketName}'
  - Effect: Allow
    Action:
    - s3:PutObject
    - s3:PutObjectAcl
    - s3:GetObject
    - s3:DeleteObject
    Resource: !Sub 'arn:aws:s3:::${BucketName}/*'

Remember to also unblock ACL changes in the bucket settings, in Permissions > Public access settings > Manage public access control lists (ACLs).

Migrating From Fog

If you migrate from fog your uploader may be configured as storage :fog, simply comment out that line, as in the following example, or remove that specific line.

class MyUploader < Carrierwave::Uploader::Base
  # Storage configuration within the uploader supercedes the global CarrierWave
  # config, so adjust accordingly...

  # Choose what kind of storage to use for this uploader:
  # storage :file
  # storage :fog
  storage :aws


  # More comments below in your file....
end

Another item particular to fog, you may have url(query: {'my-header': 'my-value'}). With carrierwave-aws the query part becomes obsolete, just use a hash of headers. Please read [usage][#Usage] for a more detailed explanation about configuration.

Contributing

In order to run the integration specs you will need to configure some environment variables. A sample file is provided as .env.sample. Copy it over and plug in the appropriate values.

cp .env.sample .env
  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request

carrierwave-aws's People

Contributors

artygus avatar bennypaulino avatar bowd avatar chrise86 avatar dapi avatar dlackty avatar felixbuenemann avatar filipegiusti avatar fschwahn avatar fusion2004 avatar gherry avatar hyb175 avatar ivanvanderbyl avatar johanneswuerbach avatar leehambley avatar lepfhty avatar meismann avatar milgner avatar mshibuya avatar netoneko avatar olivierlacan avatar pranas avatar robertcristian avatar ryanjohns avatar saicheg avatar sborsje avatar simahawk avatar sorentwo avatar tscholz avatar ylansegal avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

carrierwave-aws's Issues

Minimal rights policy ?

Hi,

What's the minimum rights policy should I give to my AWS user in order to use your gem ?
Maybe, you should document that specific point in your README. I will not give full access to my S3 buckets, and secrity aware people would do the same.

Regards,
Jules

Thanks!

I had been having an awful time using Fog/Carrierwave and S3. After 1.5 days of work with Fog, I was able to get everything up and running in 20 minutes with your gem. Cheers!

uninitialized constant Aws::CF

Hi, I´m getting this error when running my tests.

Failure/Error: get :shared, { format: :json, id: image.id, token: image.token }
     ActionView::Template::Error:
       uninitialized constant Aws::CF
     # ./config/initializers/carrierwave.rb:29:in `block (2 levels) in <top (required)>'
     #  /Users/nicolaiseerup/.rvm/gems/ruby-2.2.3/bundler/gems/carrierwave-aws-    900b3a2b2907/lib/carrierwave/storage/aws_file.rb:64:in `call'
      # /Users/nicolaiseerup/.rvm/gems/ruby-2.2.3/bundler/gems/carrierwave-aws-900b3a2b2907/lib/carrierwave/storage/aws_file.rb:64:in `signed_url'
     # /Users/nicolaiseerup/.rvm/gems/ruby-2.2.3/bundler/gems/carrierwave-aws-900b3a2b2907/lib/carrierwave/storage/aws_file.rb:81:in `url'

This is my config file:

CarrierWave.configure do |config|
  config.storage            = Rails.env.test? ? :file : :aws
  # config.storage            = :aws
  config.aws_bucket         = "#{ ENV['S3_DIRECTORY'] }#{ Rails.env }"
  config.aws_acl            = 'private'
  config.enable_processing  = false if Rails.env.test?
  # Optionally define an asset host for configurations that are fronted by a
  # content host, such as CloudFront.
  config.asset_host = ENV['CDN_ASSET_HOST']
  # The maximum period for authenticated_urls is only 7 days.
  config.aws_authenticated_url_expiration = ENV['ASSETS_EXP_TIME'].to_i + 60
  # Set custom options such as cache control to leverage browser caching
  config.aws_attributes = {
    expires: 1.week.from_now.httpdate,
    cache_control: 'max-age=604800'
  }
  config.aws_credentials = {
    access_key_id:     ENV['AWS_ACCESS_KEY_ID'],
    secret_access_key: ENV['AWS_SECRET_ACCESS_KEY'],
    region:            ENV['AWS_REGION']
  }

# Optional: Signing of download urls, e.g. for serving private
# content through CloudFront.
config.aws_signer = -> (unsigned_url, options) { Aws::CF::Signer.sign_url unsigned_url, options }

end

undefined method `download_url'

Somewhat new to Rails I had trouble making Fog work with S3 so I installed this gem. Uploading works fine but I am having trouble making my download links work. My model is Reports.

reports_controller.rb:

def download redirect_to download_url(@report.filename) end <./code>

reports_uploader.rb

def download_url(filename) url(response_content_disposition: %Q{attachment; filename="#{filename}"}) end <./code>

carrierwave.rb

CarrierWave.configure do |config| config.storage = :aws config.aws_bucket = ENV.fetch('AWS_BUCKET_NAME') config.aws_acl = :public_read config.asset_host = 'http://myproject.herokuapp.com' config.aws_authenticated_url_expiration = 600 config.aws_credentials = { access_key_id: ENV.fetch('AWS_ACCESS_KEY_ID')} config.aws_credentials = { secret_access_key: ENV.fetch('AWS_SECRET_ACCESS_KEY') } end <./code>

when I click the download link I get the error "undefined method `download_url' for #ReportsController:0xb30787f4"

Thanks!

Why does acl-public carry signature and and access key?

https://****.s3.amazonaws.com/vzbit/uploads/link/poster/18/4b4af9cc-3f4d-4ef1-878b-ad80a5866ca2.jpg?AWSAccessKeyId=AKIAJ43P3YSDPBMWFV5A&Expires=1441228284&Signature=k%2FDQ3IQjP8b1d5PkR87wc62MJug%3D

I dont see it being needed at all. How does signing the link affect performance?

Ruby AWS SDK Version 2

Is there currently any effort to migrate this to the new AWS SDK? If not, and this is something wanted, I'd be happy to spearhead that.

AWS::S3::Errors::NoSuchKey: No Such Key error on store

I'm attempting to use carrierwave_backgrounder with carrierwave-aws and am running into the following error: AWS::S3::Errors::NoSuchKey: No Such Key error on store

Here is a full stack trace:

AWS::S3::Errors::NoSuchKey: No Such Key
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/aws-sdk-1.49.0/lib/aws/core/client.rb:375:in `return_or_raise'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/aws-sdk-1.49.0/lib/aws/core/client.rb:476:in `client_request'
    (eval):3:in `head_object'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/aws-sdk-1.49.0/lib/aws/s3/s3_object.rb:293:in `head'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/aws-sdk-1.49.0/lib/aws/s3/s3_object.rb:325:in `content_type'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/carrierwave-aws-0.4.1/lib/carrierwave/storage/aws.rb:46:in `content_type'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/carrierwave-aws-0.4.1/lib/carrierwave/storage/aws.rb:113:in `uploader_write_options'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/carrierwave-aws-0.4.1/lib/carrierwave/storage/aws.rb:77:in `store'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/carrierwave-aws-0.4.1/lib/carrierwave/storage/aws.rb:16:in `block in store!'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/carrierwave-aws-0.4.1/lib/carrierwave/storage/aws.rb:15:in `tap'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/carrierwave-aws-0.4.1/lib/carrierwave/storage/aws.rb:15:in `store!'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/carrierwave-0.10.0/lib/carrierwave/uploader/store.rb:59:in `block in store!'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/carrierwave-0.10.0/lib/carrierwave/uploader/callbacks.rb:17:in `with_callbacks'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/carrierwave-0.10.0/lib/carrierwave/uploader/store.rb:58:in `store!'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/carrierwave-0.10.0/lib/carrierwave/mount.rb:375:in `store!'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/carrierwave-0.10.0/lib/carrierwave/mount.rb:207:in `store_file!'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/carrierwave_backgrounder-0.4.1/lib/backgrounder/orm/base.rb:89:in `store_file!'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:424:in `block in make_lambda'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:221:in `call'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:221:in `block in halting_and_conditional'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:215:in `call'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:215:in `block in halting_and_conditional'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:86:in `call'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/activesupport-4.1.5/lib/active_support/callbacks.rb:86:in `run_callbacks'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/activerecord-4.1.5/lib/active_record/callbacks.rb:302:in `create_or_update'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/activerecord-4.1.5/lib/active_record/persistence.rb:125:in `save!'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/activerecord-4.1.5/lib/active_record/validations.rb:57:in `save!'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/activerecord-4.1.5/lib/active_record/attribute_methods/dirty.rb:29:in `save!'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/activerecord-4.1.5/lib/active_record/transactions.rb:273:in `block in save!'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/activerecord-4.1.5/lib/active_record/transactions.rb:329:in `block in with_transaction_returning_status'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract/database_statements.rb:199:in `transaction'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/activerecord-4.1.5/lib/active_record/transactions.rb:208:in `transaction'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/activerecord-4.1.5/lib/active_record/transactions.rb:326:in `with_transaction_returning_status'
    /Users/donald/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/activerecord-4.1.5/lib/active_record/transactions.rb:273:in `save!'

FYI the value of new_file inside the content_type method inside storage/aws.rb:46 is as follows:

#<CarrierWave::Storage::AWS::File:0x007fc88e4a14c0 @uploader=#<DeviceIdentifierListUploader:0x007fc8969c9f08 @model=#<DeviceIdentifierList id: 18, name: "aertaeryaeryaertae", user_id: -1, latest_sha1: nil, created_at: "2014-08-21 13:39:41", updated_at: "2014-08-21 13:39:41", customer_id: 2, file: "ifa_list_example_file.csv", file_processing: false, file_tmp: nil, state: "processing">, @mounted_as=:file, @cache_id="1408628381-32549-8749", @filename="ifa_list_example_file.csv", @original_filename="ifa_list_example_file.csv", @file=#<CarrierWave::Storage::AWS::File:0x007fc88e4a14c0 ...>, @versions={}, @upload_directory="/Users/donald/Projects/poblano/public/uploads/tmp/1408628381-32549-8749", @storage=#<CarrierWave::Storage::AWS:0x007fc894e5a0d8 @uploader=#<DeviceIdentifierListUploader:0x007fc8969c9f08 ...>, @connection=<AWS::S3>>>, @connection=<AWS::S3>, @path="uploads/device_identifier_list/file/18/ifa_list_example_file.csv">

Any help would be greatly appreciated

carrierwave-aws doesn't append bucket name to the host URL

fog gem config:

config.fog_credentials = {
  :host => 's3-eu-west-1.amazonaws.com'
}
config.fog_directory  = 'mybucket'

carrierwave-aws config:

config.aws_bucket = 'mybucket'
config.asset_host = 'https://s3-eu-west-1.amazonaws.com/mybucket'

Shouldn't this be kept a bit more consistent?

aws private url

hi there,

just would like to double check, I get this kind of url by setting access to be private: https://xxx.s3-ap-southeast-1.amazonaws.com/uploads/xxx/xxx/3/xxx.jpeg?AWSAccessKeyId=xxxxx&Expires=xxx&Signature=xxxxx

Is it supposed/OK to expose AWSAccessKeyId in the url?

Config is like is:

config.aws_acl    = :private
config.aws_authenticated_url_expiration = 60

Thanks!

New gem version release

Hi Parker,
Please, release new gem version to rubygems.org.
We need to use new aws-sdk 2.x
Thanks

Version bump

Can we get a version bump? Would be nice to not have to track master to get commits.

aws_attributes not working as expected

I first searched for a solution which pointed me to this #12 and then I added that to my initializer:

config.aws_attributes = {
  'cache_control' => 'public,max-age=315576000',
  'expires' => 1.year.from_now.httpdate
}

But my uploaded images don't have these attributes. I'm using:

    carrierwave-aws (0.3.2)
      aws-sdk (>= 1.8.0)

Expose credentials to AWS.config

Adding credentials via

config.aws_credentials = {
    access_key_id:           ENV['s3_access_key_id'],
    secret_access_key:    ENV['s3_secret'],
 }

won't work outside carrierwave context. I tried calling the Elastic Transcoder api and got an AWS::Errors::MissingCredentialsError: Missing Credentials.

I tried adding the credentials once again inside the config option as the README suggested

config.aws_credentials = {
    access_key_id:     ENV['s3_access_key_id'],
    secret_access_key: ENV['s3_secret'],
    config: AWS.config({
      access_key_id:     ENV['s3_access_key_id'],
      secret_access_key: ENV['s3_secret']
    })
  }

This way it worked, so I guess those options aren't being exposed to the AWS object. Is this the intended behavior?

How can I change the s3 bucket for a given request?

I have multiple S3 buckets for different purposes (avatars, documents etc). In the docs it says that I must configure the s3 bucket in the config file. How do I override the aws_bucket parameter so that I can use different buckets for different models?

I tried something like this:

  mount_uploader :file, ProfilePictureUploader do
    def aws_bucket
      ENV['S3_AVATAR_BUCKET']
    end

but I got an error 'ArgumentError - bucket_name may not be blank'

Any help appreciated!

What storage method in uploader

For uploading to S3, the starting instructions are:
class LogoUploader < CarrierWave::Uploader::Base storage :fog
But installing the gem, and setting up an initializer CarrierWave.configure do |config| [...]
leads to the application not starting because:
Error message: uninitialized constant CarrierWave::Storage::Fog
Note: I disabled the fog gem on the basis of: "Significantly smaller footprint".

aws file link

Hi there,

I am generating some reports in my project, which needs to have links to point to the pdf files uploaded to the s3.

As the aws credentials I am using will expire the file link after a while(which means I cannot use those static links in the report), so I wonder is there any simple set up could support this:

  1. in the report, file hypelink behind looks like www.example/documents/1
  2. when user clicks the link, it goes to my server authentication.
  3. if server authentication passes, rails server will create a aws file link, like https://xxx.s3-ap-southeast-1.amazonaws.com/uploads/xxx/xxx/3/xxx.jpeg?AWSAccessKeyId=xxxxx&Expires=xxx&Signature=xxxxx

Thanks!

No route matches GET when uploading through seed script

Image uploads work when uploading through the rails app, but I have a seed file that attempts to create some records with images:

App.create( name: 'App Center', description: 'Find and install apps', url: '/app_center_/', category_id: cat_core.id, required: true, icon: File.open('./support/appicons/manage_apps.png') )

But I am getting 404 errors: No route matches [GET] "/development/images/app_icons/32/manage_apps.png"

This used to work when I used the fog gem. Any thoughts?

Aws::S3::Errors::InvalidArgument Error on Remote URL Upload

Here's the stack trace which should help point to the file/command causing the issue:

Aws::S3::Errors::InvalidArgument: 
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/aws-sdk-core-2.0.48/lib/seahorse/client/plugins/raise_response_errors.rb:15:in `call'
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/aws-sdk-core-2.0.48/lib/aws-sdk-core/plugins/s3_sse_cpk.rb:18:in `call'
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/aws-sdk-core-2.0.48/lib/seahorse/client/plugins/param_conversion.rb:22:in `call'
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/aws-sdk-core-2.0.48/lib/aws-sdk-core/plugins/response_paging.rb:10:in `call'
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/aws-sdk-core-2.0.48/lib/seahorse/client/plugins/response_target.rb:18:in `call'
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/aws-sdk-core-2.0.48/lib/seahorse/client/request.rb:70:in `send_request'
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/aws-sdk-core-2.0.48/lib/seahorse/client/base.rb:216:in `block (2 levels) in define_operation_methods'
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/aws-sdk-resources-2.0.48/lib/aws-sdk-resources/request.rb:24:in `call'
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/aws-sdk-resources-2.0.48/lib/aws-sdk-resources/operations.rb:41:in `call'
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/aws-sdk-resources-2.0.48/lib/aws-sdk-resources/operation_methods.rb:19:in `block in add_operation'
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/carrierwave-aws-0.6.0/lib/carrierwave/storage/aws_file.rb:56:in `store'
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/carrierwave-aws-0.6.0/lib/carrierwave/storage/aws.rb:16:in `block in store!'
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/carrierwave-aws-0.6.0/lib/carrierwave/storage/aws.rb:15:in `tap'
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/carrierwave-aws-0.6.0/lib/carrierwave/storage/aws.rb:15:in `store!'
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/carrierwave-0.10.0/lib/carrierwave/uploader/store.rb:59:in `block in store!'
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/carrierwave-0.10.0/lib/carrierwave/uploader/callbacks.rb:17:in `with_callbacks'
... 16 levels...
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/activerecord-4.2.3/lib/active_record/transactions.rb:351:in `block in with_transaction_returning_status'
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/activerecord-4.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:213:in `block in transaction'
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/activerecord-4.2.3/lib/active_record/connection_adapters/abstract/transaction.rb:184:in `within_new_transaction'
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/activerecord-4.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:213:in `transaction'
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/activerecord-4.2.3/lib/active_record/transactions.rb:220:in `transaction'
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/activerecord-4.2.3/lib/active_record/transactions.rb:348:in `with_transaction_returning_status'
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/activerecord-4.2.3/lib/active_record/transactions.rb:291:in `save!'
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/activerecord-4.2.3/lib/active_record/persistence.rb:51:in `create!'
    from (irb):3
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/railties-4.2.3/lib/rails/commands/console.rb:110:in `start'
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/railties-4.2.3/lib/rails/commands/console.rb:9:in `start'
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/railties-4.2.3/lib/rails/commands/commands_tasks.rb:68:in `console'
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/railties-4.2.3/lib/rails/commands/commands_tasks.rb:39:in `run_command!'
    from /Users/karanjain/.rvm/gems/ruby-2.2.0@tinygive/gems/railties-4.2.3/lib/rails/commands.rb:17:in `<top (required)>'
    from bin/rails:4:in `require'
    from bin/rails:4:in `<main>'

And here's the gem versions being used:

carrierwave-aws (0.6.0)
      aws-sdk (~> 2.0.47)
      carrierwave (~> 0.7)
 aws-sdk (2.0.48)
      aws-sdk-resources (= 2.0.48)
    aws-sdk-core (2.0.48)
      builder (~> 3.0)
      jmespath (~> 1.0)
      multi_json (~> 1.0)
    aws-sdk-resources (2.0.48)
      aws-sdk-core (= 2.0.48)

Does not seem to delete files in S3

When I call the delete method on my model, the model is deleted from the database but the files remain in S3.

Is deleting the file from S3 supported, or am I forgetting to do something?

Model

class HomeImage
  include Mongoid::Document
  include Mongoid::Timestamps

  mount_uploader :image, HomeImageUploader
end

uploader

class HomeImageUploader < CarrierWave::Uploader::Base
  include CarrierWave::MiniMagick

  # Process files as they are uploaded:
  process :custom_resize => [1200, 400]

  version :thumb do
    process :custom_resize => [100, 100]
  end

  def extension_white_list
    %w(jpg jpeg png)
  end

  def filename
    "app_#{Zlib.crc32(model.id).to_s(36)}.#{file.extension}" if original_filename
  end

  def custom_resize(width, height)
    manipulate! do |img|
      img.resize "#{width}x#{height}"
      img = yield(img) if block_given?
      img
    end
  end
end

Controller

def destroy
    if HomeImage.find(params[:id]).delete
      flash[:notice] = 'Success.'
    else
      flash[:error] = 'Error'
    end

    redirect_to home_images_path
end

AWS and SSLv3

Hey,

My company and I recently received an email about Amazon WS disabling SSL v3 on May, 20th. They are encouraging users to move to TLS instead. Do you have any clue about the compatibility of carrierwave-aws with this, or do we need to do some work in order to get things ready for this transition ?

Thanks for your answer, I'll keep on digging about this but hope you'll be able to provide me some insight.

question about configs

Hi there,

I configured my rails application as below:

CarrierWave.configure do |config|
    config.storage    = :aws
    config.aws_bucket = 'a-unique-name'
    config.aws_acl    = :public_read
    config.asset_host = 'http://localhost:3000' # not sure what to put here
    # config.aws_authenticated_url_expiration = 60 * 60 * 24 * 365

    config.aws_credentials = {
      access_key_id:    'xxxx',
      secret_access_key: 'xxxx',
      region: 'Singapore'
    }
  end

And my uploader is:

def store_dir
    "uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
  end

  def download_url(filename)
    url(response_content_disposition: %Q{attachment; filename="#{filename}"})
  end

So in this configuration, files are still uploaded to my local server, rather than S3. Is there anywhere I mis configured?

Another quick question, what is config.assert_url for?

Thanks a lot!

Compatibility with mobile browsers

Thanks for developing this gem. I've been using it to upload video files onto my Amazon S3 account. Recently, I've tried to upload videos from my iPhone browser (Chrome and Safari) to Amazon as well but for some reason, the size of the uploaded file in my S3 bucket is 0 bytes (yes, the name of the file is successfully created).

Can you tell me Carrierwave is capable of uploading files stored on iOS to Amazon S3? I'm wondering if the contents of the file aren't uploading because Carrierwave isn't able to upload the file in a temporary directory on iOS (the way it does for for desktop browsers) in order to validate the file prior to sending to Amazon. Thanks!

Cloudfront not working

Hi there,

Thanks for your gem, it's working fine!

However, after setting up
config.asset_host = 'https://[myid].cloudfront.net'

  • restart server.

The images are still served from the S3 bucket.
Is there another setting I forgot to do?

Thanks!

  • Vincent

Unexpected behaviour setting field to nil

Hi there,

with an uploader mounted on a field urls_file, setting this field to nil has confusing behavior.
model.urls_file = nil sets a weird path:

@file=#<CarrierWave::Storage::AWSFile:0x0000000be93210
...
 @path="uploads/model/urls_file/558c1d33af53dbc48000002f/_old_"
...

Moreover a model.urls_file.present? now returns true.
This differs from other carrierwave storages (like filestorage).

undefined method `aws_bucket=' for CarrierWave::Uploader::Base:Class

Hey,

I integrated the carrierwave-aws to our rails app and I its working on local, but I am getting weird errors when trying to deploy to AWS using EB CLI.
After trying to run rake db:migrate, rake is aborted with the following error:
NoMethodError: undefined method 'aws_bucket=' for CarrierWave::Uploader::Base:Class

This is how my carrierwave.rb file looks like:

CarrierWave.configure do |config|
config.storage = 'aws'
config.aws_bucket = ENV.fetch('S3_BUCKET_NAME')
config.aws_acl = 'private'

The maximum period for authenticated_urls is only 7 days.

config.aws_authenticated_url_expiration = 60 * 60 * 24 * 7

Set custom options such as cache control to leverage browser caching

config.aws_attributes = {
expires: 1.week.from_now.httpdate,
cache_control: 'max-age=315576000'
}

config.aws_credentials = {
access_key_id: ENV['AWS_ACCESS_KEY_ID'],
secret_access_key: ENV['AWS_SECRET_ACCESS_KEY'],
region: ENV['AWS_REGION'] # Required
}
end

Inside my gemfile I have tried to do it with both:
gem 'carrierwave'
gem 'carrierwave-aws', '~> 1.0.0'
as well as only with the gem 'carrierwave-aws', '~> 1.0.0' and error persists.

Any ideas on what might be causing this. I thought it might be some dependency issue but couldn't find anything.

AWS S3 V4

hello, just moved my bucket to Frankfurt which is supposed to run only V4 authentication. Does Carrierwave-aws does support V4? I specified the region as you can see below. I'm running the this gem at the (by now) latest commit '5efab1b1a5279dcb2b9e6e716ac40c6ed6f40655'.

...
config.aws_bucket = ENV['S3_BUCKET_NAME']
config.aws_acl    = :public_read
config.aws_authenticated_url_expiration = 60 * 60 * 24 * 365

config.aws_credentials = {
  access_key_id:     ENV['AWS_ACCESS_KEY_ID'],
  secret_access_key: ENV['AWS_SECRET_ACCESS_KEY'],
  region:            'eu-central-1'
}

Crash report

AWS::S3::Errors::InvalidRequest - The authorization mechanism you have provided is not supported.     Please use AWS4-HMAC-SHA256.:
16:54:08 puma.1          |   aws-sdk (1.41.0) lib/aws/core/client.rb:375:in `return_or_raise' 
16:54:08 puma.1          |   aws-sdk (1.41.0) lib/aws/core/client.rb:476:in `client_request'
16:54:08 puma.1          |   (eval):3:in `put_object'
16:54:08 puma.1          |   aws-sdk (1.41.0) lib/aws/s3/s3_object.rb:1752:in `write_with_put_object'
16:54:08 puma.1          |   aws-sdk (1.41.0) lib/aws/s3/s3_object.rb:607:in `write'

Is there a way to config private/public aws_acl based on different uploaders

Hi there,

In my project, there is an uploader for profile image which can be public read, and another uploader for files which is private.

Currently I just configure both of them to be private in the config/initializers/carrierwave.rb, I wonder is there a way to configure one to be private, the other one to be public?

Thanks a lot in advance!

uninitialized constant Aws::CF

When I try to show the uploaded image using <%= image_tag(@idea.image) %> inside my show.html.erb, I get uninitialized constant Aws::CF error.

screen shot 2016-04-29 at 1 58 57 pm

What I'm trying to do here:
Sign the image URL and display the image. I don't want to use CloudFront. I want to use S3 only.

Expected image tag HTML to something like this:

<img src="https://s3.amazonaws.com/bucket/uploads/idea/image/3/example.png?AWSAccessKeyId=AKIAJKOHTE4WTXCCXAMA&Signature=8PLq8WCkfrkthmfVGfXX9K6s5fc%3D&Expires=1354859553" />

My Gemfile:

source 'https://rubygems.org'

gem 'rails', '4.2.5.1'
gem 'sqlite3'
gem 'sass-rails', '~> 5.0'
gem 'uglifier', '>= 1.3.0'
gem 'jquery-rails'
gem 'carrierwave'
gem 'mini_magick'
gem 'aws-sdk'
gem 'aws_cf_signer'
gem 'carrierwave-aws'

group :development do
  gem 'byebug'
end

My Idea model:

class Idea < ActiveRecord::Base
    mount_uploader :image, ImageUploader
end

My ImageUploader:

class ImageUploader < CarrierWave::Uploader::Base
  storage :aws
  def store_dir
    "uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
  end
end

My CarrierWave initializer:

CarrierWave.configure do |config|
  config.storage    = :aws
  config.aws_bucket = '__________'
  config.aws_acl    = 'public-read'

  config.aws_authenticated_url_expiration = 60 * 60 * 24 * 7

  config.aws_attributes = {
    expires: 1.week.from_now.httpdate,
    cache_control: 'max-age=604800'
  }

  config.aws_credentials = {
    access_key_id:     '__________',
    secret_access_key: '__________',
    region:            'us-east-1'
  }

  config.aws_signer = -> (unsigned_url, options) { Aws::CF::Signer.sign_url unsigned_url, options }
end

My questions:

  1. What am I missing here?
  2. Do I need aws-sdk and aws_cf_signer gem?
  3. Do we have something like fog_public = false like what this guy mentioned here?

Please help. Thanks.

How not to store files in public folder?

I had carrierwave+fog and my uploads in development mode saves to "public/uploads/" folder.
All that I've changed are "config.fog_credentials" to "config.aws_credentials" and "config.fog_directory" to "config.aws_bucket". All other stays the same.

But as for now in my project all files saves to "public" directory. I would like to save these files to "public/uploads/", like it was with fog.
(I have "store_dir = nil" as well, so I'm expecting it would work like it should)

So, I need to save files to "public/uploads" in development mode and not to save files in production.

What else should I change to achieve this?

Custom AWS Headers (primarily for file encryption server-side) with bucket policy

I set up a bucket policy to require that files uploaded be server-side encrypted. I got it all to work without the bucket policy, but here I'm needing to pass header and value.

  CarrierWave.configure do |config|
   ...
    config.aws_attributes = {'x-amz-server-side​-encryption' => "AES256"}
  ...
  end

It doesn't seem to work and I'm not finding any documentation to the contrary. Is there something I'm missing?

Cloudfront configuration

How can I configure access to cloudfront using your plugin?

As I understand I have to set up host option, but I don't see this option.

The bucket you are attempting to access must be addressed using the specified endpoint

My bucket is in the us-west-1 region, and it is available via special endpoint. It seems, there is no way to set the custom endpoint in the configuration. I had to monkey patch the gem:

  if Settings.carrierwave.aws.endpoint.present?
    class CarrierWave::Storage::AWSFile
      def authenticated_url_with_endpoint(options = {})
        authenticated_url_without_endpoint(
          { endpoint: Settings.carrierwave.aws.endpoint }.merge(options)
        )
      end
      alias_method_chain :authenticated_url, :endpoint
    end
  end

But there is also problem with file uploads.

Private content and Cloudfront signed urls?

Thanks for this carrierwave adapter. I was already using the aws-sdk gem so this lightweight add-on to carrierwave is much nicer than having to use fog.

I'd like to use a cloudfront asset host to serve content, but it looks like this is currently only possible via the public_url method and a public_read acl. I have some private content I'd like to serve.

Is it possible to serve private content and also use cloudfront? I'm not super familiar with cloudfront but from what I've read it seems like it should be possible. I came across this post related to fog: http://stackoverflow.com/questions/19458004/cloudfront-carrierwave

Any help would be greatly appreciated!

Storage provider is not recognized

I'm trying to get up and running with carrierwave-aws, but I hit this problem early on.

    ArgumentError ( is not a recognized storage provider)

I'm pretty confused about what (if anything) needs to change in avatar_uploader.rb. Right now it still has:

    storage :fog
    include CarrierWave::MimeTypes
    process :set_content_type

It seems like having storage type :fog would be wrong, but I wasn't sure what else to use, and you said things ran the same way.

How can I get my program to recognize carrierwave-aws?

image_tag issues

I am only starting here because the issue cropped up after I switched from fog to carrierwave-aws. My image tags are rendering like this:

<%= image_tag(current_user.avatar.thumb.url, class: "img-circle") %>    
<img class="img-circle" src="https://xxxx.cloudfront.net/images/xxx.cloudfront.net/user/9/avatar_thumb.png" alt="Avatar thumb">

but current_user.avatar.thumb.url outputs https://xxxx.cloudfront.net/user/9/avatar_thumb.png.

S3 Download Url with attachment attribute (mp3 opens file instead of downloading)

Hi there,

I've been scratching my head for a while to be able to bring a download link (for a mp3 file) to work.

I tried the redirect method but it opens the file in the browser instead of downloading it.
I also tried with send data but it copies the file first which makes it useless (it takes too much processing time).

What I want to achieve is to stream the S3 file so that the user can download it directly.

I tried to add the attribute "content_disposition: "Attachment" in my carrier wave initializer without success.

I suspect this could be achieved with the S3 object

def download_url
  S3 = AWS::S3.new.buckets[ 'bucket_name' ] # This can be done elsewhere as well.

  S3.objects[ self.path ].url_for( :read,
    expires_in: 60.minutes, 
    use_ssl:    true, 
    response_content_disposition: "attachment; filename='#{attachment_file_name}'" ).to_s
end

But I don't know how to get access to the S3 object with carrierwave-aws (and I don't even know if the above code would work).

I'm sure this is pretty straight-forward to do but I'm stuck.

Any ideas?
Thanks!

  • Vincent

very very strange access issue

hi guys,

I have configuration like below:

CarrierWave.configure do |config|
  config.storage    = :aws
  config.aws_bucket = "vendorable-#{Rails.env}"
  config.aws_acl    = :private
  config.aws_authenticated_url_expiration = 60

  config.aws_credentials = {
    access_key_id:    'xxxxx',
    secret_access_key: 'xxxxx',
    region: 'ap-southeast-1'
  }
end

So it works properly with development and staging environment. But in production environment, every time a file is uploaded, the S3 request returns 403 forbidden on the page. I have checked S3 bucket, all files have been saved in the vendorable-production.

Any suggestions are much appreciated!!

unexpected value at params["Content-Disposition"]

Since trying to update to carrierwave-aws 0.6.0 from 0.5.0 these errors keep preventing anything from being uploaded:

unexpected value at params["Content-Disposition"]
ArgumentError (unexpected value at params["Content-Disposition"])

    aws-sdk (2.0.48)
      aws-sdk-resources (= 2.0.48)
    aws-sdk-core (2.0.48)
      builder (~> 3.0)
      jmespath (~> 1.0)
      multi_json (~> 1.0)
    aws-sdk-resources (2.0.48)
      aws-sdk-core (= 2.0.48)
    carrierwave (0.10.0)
      activemodel (>= 3.2.0)
      activesupport (>= 3.2.0)
      json (>= 1.7)
      mime-types (>= 1.16)
    carrierwave-aws (0.6.0)
      aws-sdk (~> 2.0.47)
      carrierwave (~> 0.7)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.