Coder Social home page Coder Social logo

kt-paperclip's Introduction

Paperclip

We plan to support and maintain paperclip, as well as clean it up.

Please feel free to contribute Issues and pull requests.

Existing documentation

Documentation valid for master branch

Please check the documentation for the paperclip version you are using: https://github.com/kreeti/kt-paperclip/releases


Build Status Code Climate Inline docs Security

Paperclip is intended as an easy file attachment library for ActiveRecord. The intent behind it was to keep setup as easy as possible and to treat files as much like other attributes as possible. This means they aren't saved to their final locations on disk, nor are they deleted if set to nil, until ActiveRecord::Base#save is called. It manages validations based on size and presence, if required. It can transform its assigned image into thumbnails if needed, and the prerequisites are as simple as installing ImageMagick (which, for most modern Unix-based systems, is as easy as installing the right packages). Attached files are saved to the filesystem and referenced in the browser by an easily understandable specification, which has sensible and useful defaults.

See the documentation for has_attached_file in Paperclip::ClassMethods for more detailed options.

The complete RDoc is online.


Requirements

Ruby and Rails

Paperclip now requires Ruby version >= 2.3 and Rails version >= 4.2 (only if you're going to use Paperclip with Ruby on Rails).

Image Processor

ImageMagick must be installed and Paperclip must have access to it. To ensure that it does, on your command line, run which convert (one of the ImageMagick utilities). This will give you the path where that utility is installed. For example, it might return /usr/local/bin/convert.

Then, in your environment config file, let Paperclip know to look there by adding that directory to its path.

In development mode, you might add this line to config/environments/development.rb):

Paperclip.options[:command_path] = "/usr/local/bin/"

If you're on Mac OS X, you'll want to run the following with Homebrew:

brew install imagemagick

If you are dealing with pdf uploads or running the test suite, you'll also need to install GhostScript. On Mac OS X, you can also install that using Homebrew:

brew install gs

If you are on Ubuntu (or any Debian base Linux distribution), you'll want to run the following with apt-get:

sudo apt-get install imagemagick -y

file

The Unix file command is required for content-type checking. This utility isn't available in Windows, but comes bundled with Ruby Devkit, so Windows users must make sure that the devkit is installed and added to the system PATH.

Manual Installation

If you're using Windows 7+ as a development environment, you may need to install the file.exe application manually. The file spoofing system in Paperclip 4+ relies on this; if you don't have it working, you'll receive Validation failed: Upload file has an extension that does not match its contents. errors.

To manually install, you should perform the following:

Download & install file from this URL

To test, you can use the image below: untitled

Next, you need to integrate with your environment - preferably through the PATH variable, or by changing your config/environments/development.rb file

PATH

1. Click "Start"
2. On "Computer", right-click and select "Properties"
3. In Properties, select "Advanced System Settings"
4. Click the "Environment Variables" button
5. Locate the "PATH" var - at the end, add the path to your newly installed `file.exe` (typically `C:\Program Files (x86)\GnuWin32\bin`)
6. Restart any CMD shells you have open & see if it works

OR

Environment

1. Open `config/environments/development.rb`
2. Add the following line: `Paperclip.options[:command_path] = 'C:\Program Files (x86)\GnuWin32\bin'`
3. Restart your Rails server

Either of these methods will give your Rails setup access to the file.exe functionality, thus providing the ability to check the contents of a file (fixing the spoofing problem)


Installation

Paperclip is distributed as a gem, which is how it should be used in your app.

Include the gem in your Gemfile:

gem "kt-paperclip", "~> 6.4", ">= 6.4.1"

Or, if you want to get the latest, you can get master from the main paperclip repository:

gem "kt-paperclip", git: "git://github.com/kreeti/kt-paperclip.git"

If you're trying to use features that don't seem to be in the latest released gem, but are mentioned in this README, then you probably need to specify the master branch if you want to use them. This README is probably ahead of the latest released version if you're reading it on GitHub.

For Non-Rails usage:

class ModuleName < ActiveRecord::Base
  include Paperclip::Glue
  ...
end

Quick Start

Models

class User < ActiveRecord::Base
  has_attached_file :avatar, styles: { medium: "300x300>", thumb: "100x100>" }, default_url: "/images/:style/missing.png"
  validates_attachment_content_type :avatar, content_type: /\Aimage\/.*\z/
end

Migrations

Assuming you have a users table, add an avatar column to the users table:

class AddAvatarColumnsToUsers < ActiveRecord::Migration
  def up
    add_attachment :users, :avatar
  end

  def down
    remove_attachment :users, :avatar
  end
end

(Or you can use the Rails migration generator: rails generate paperclip user avatar)

Edit and New Views

Make sure you have corresponding methods in your controller:

<%= form_for @user, url: users_path, html: { multipart: true } do |form| %>
  <%= form.file_field :avatar %>
  <%= form.submit %>
<% end %>

Edit and New Views with Simple Form

<%= simple_form_for @user, url: users_path do |form| %>
  <%= form.input :avatar, as: :file %>
  <%= form.submit %>
<% end %>

Controller

def create
  @user = User.create(user_params)
end

private

# Use strong_parameters for attribute whitelisting
# Be sure to update your create() and update() controller methods.

def user_params
  params.require(:user).permit(:avatar)
end

View Helpers

Add these to the view where you want your images displayed:

<%= image_tag @user.avatar.url %>
<%= image_tag @user.avatar.url(:medium) %>
<%= image_tag @user.avatar.url(:thumb) %>

Checking a File Exists

There are two methods for checking if a file exists:

  • file? and present? checks if the _file_name field is populated
  • exists? checks if the file exists (will perform a TCP connection if stored in the cloud)

Keep this in mind if you are checking if files are present in a loop. The first version is significantly more performant, but has different semantics.

Deleting an Attachment

Set the attribute to nil and save.

@user.avatar = nil
@user.save

Usage

The basics of Paperclip are quite simple: Declare that your model has an attachment with the has_attached_file method, and give it a name.

Paperclip will wrap up to four attributes (all prefixed with that attachment's name, so you can have multiple attachments per model if you wish) and give them a friendly front end. These attributes are:

  • <attachment>_file_name
  • <attachment>_file_size
  • <attachment>_content_type
  • <attachment>_updated_at

By default, only <attachment>_file_name is required for Paperclip to operate. You'll need to add <attachment>_content_type in case you want to use content type validation.

More information about the options passed to has_attached_file is available in the documentation of Paperclip::ClassMethods.

Validations

For validations, Paperclip introduces several validators to validate your attachment:

  • AttachmentContentTypeValidator
  • AttachmentPresenceValidator
  • AttachmentSizeValidator

Example Usage:

validates :avatar, attachment_presence: true
validates_with AttachmentPresenceValidator, attributes: :avatar
validates_with AttachmentSizeValidator, attributes: :avatar, less_than: 1.megabytes

Validators can also be defined using the old helper style:

  • validates_attachment_presence
  • validates_attachment_content_type
  • validates_attachment_size

Example Usage:

validates_attachment_presence :avatar

You can also define multiple validations on a single attachment using validates_attachment:

validates_attachment :avatar, presence: true,
  content_type: "image/jpeg",
  size: { in: 0..10.kilobytes }

NOTE: Post-processing will not even start if the attachment is not valid according to the validations. Your callbacks and processors will only be called with valid attachments.

class Message < ActiveRecord::Base
  has_attached_file :asset, styles: { thumb: "100x100#" }

  before_post_process :skip_for_audio

  def skip_for_audio
    ! %w(audio/ogg application/ogg).include?(asset_content_type)
  end
end

If you have other validations that depend on assignment order, the recommended course of action is to prevent the assignment of the attachment until afterwards, then assign manually:

class Book < ActiveRecord::Base
  has_attached_file :document, styles: { thumbnail: "60x60#" }
  validates_attachment :document, content_type: "application/pdf"
  validates_something_else # Other validations that conflict with Paperclip's
end

class BooksController < ApplicationController
  def create
    @book = Book.new(book_params)
    @book.document = params[:book][:document]
    @book.save
    respond_with @book
  end

  private

  def book_params
    params.require(:book).permit(:title, :author)
  end
end

A note on content_type validations and security

You should ensure that you validate files to be only those MIME types you explicitly want to support. If you don't, you could be open to XSS attacks if a user uploads a file with a malicious HTML payload.

If you're only interested in images, restrict your allowed content_types to image-y ones:

validates_attachment :avatar,
  content_type: ["image/jpeg", "image/gif", "image/png"]

Paperclip::ContentTypeDetector will attempt to match a file's extension to an inferred content_type, regardless of the actual contents of the file.

Duplicate error messages

By default Paperclip will copy validation errors from the attribute to the base of your model. Depending on how you display your validation errors, this can lead to confusing duplicate errors (one on the attribute and another referring to the base model).

You can override this behaviour with the add_validation_errors_to option. By default this is set to :both but can be set to either :attribute or :base.

  • :both creates errors on both the attribute and base model.
  • :attribute only creates an error on the attribute of the model.
  • :base only creates an error on the base model.

You can set this option globally:

Paperclip.options[:add_validation_errors_to] = :attribute

or pass it in to an individual validation declaration:

validates_attachment :document, content_type: { content_type: "application/pdf" }, add_validation_errors_to: :attribute


Internationalization (I18n)

For using or adding locale files in different languages, check the project https://github.com/kreeti/paperclip-i18n.

Security Validations

Thanks to a report from Egor Homakov we have taken steps to prevent people from spoofing Content-Types and getting data you weren't expecting onto your server.

NOTE: Starting at version 4.0.0, all attachments are required to include a content_type validation, a file_name validation, or to explicitly state that they're not going to have either. Paperclip will raise an error if you do not do this.

class ActiveRecord::Base
  has_attached_file :avatar
  # Validate content type
  validates_attachment_content_type :avatar, content_type: /\Aimage/
  # Validate filename
  validates_attachment_file_name :avatar, matches: [/png\z/, /jpe?g\z/]
  # Explicitly do not validate
  do_not_validate_attachment_file_type :avatar
end

This keeps Paperclip secure-by-default, and will prevent people trying to mess with your filesystem.

NOTE: Also starting at version 4.0.0, Paperclip has another validation that cannot be turned off. This validation will prevent content type spoofing. That is, uploading a PHP document (for example) as part of the EXIF tags of a well-formed JPEG. This check is limited to the media type (the first part of the MIME type, so, 'text' in text/plain). This will prevent HTML documents from being uploaded as JPEGs, but will not prevent GIFs from being uploaded with a .jpg extension. This validation will only add validation errors to the form. It will not cause errors to be raised.

This can sometimes cause false validation errors in applications that use custom file extensions. In these cases you may wish to add your custom extension to the list of content type mappings by creating config/initializers/paperclip.rb:

# Allow ".foo" as an extension for files with the MIME type "text/plain".
Paperclip.options[:content_type_mappings] = {
  foo: %w(text/plain)
}

Defaults

Global defaults for all your Paperclip attachments can be defined by changing the Paperclip::Attachment.default_options Hash. This can be useful for setting your default storage settings per example so you won't have to define them in every has_attached_file definition.

If you're using Rails, you can define a Hash with default options in config/application.rb or in any of the config/environments/*.rb files on config.paperclip_defaults. These will get merged into Paperclip::Attachment.default_options as your Rails app boots. An example:

module YourApp
  class Application < Rails::Application
    # Other code...

    config.paperclip_defaults = { storage: :fog, fog_credentials: { provider: "Local", local_root: "#{Rails.root}/public"}, fog_directory: "", fog_host: "localhost"}
  end
end

Another option is to directly modify the Paperclip::Attachment.default_options Hash - this method works for non-Rails applications or is an option if you prefer to place the Paperclip default settings in an initializer.

An example Rails initializer would look something like this:

Paperclip::Attachment.default_options[:storage] = :fog
Paperclip::Attachment.default_options[:fog_credentials] = { provider: "Local", local_root: "#{Rails.root}/public"}
Paperclip::Attachment.default_options[:fog_directory] = ""
Paperclip::Attachment.default_options[:fog_host] = "http://localhost:3000"

Migrations

Paperclip defines several migration methods which can be used to create the necessary columns in your model. There are two types of helper methods to aid in this, as follows:

Add Attachment Column To A Table

The attachment helper can be used when creating a table:

class CreateUsersWithAttachments < ActiveRecord::Migration
  def up
    create_table :users do |t|
      t.attachment :avatar
    end
  end

  # This is assuming you are only using the users table for Paperclip attachment. Drop with care!
  def down
    drop_table :users
  end
end

You can also use the change method, instead of the up/down combination above, as shown below:

class CreateUsersWithAttachments < ActiveRecord::Migration
  def change
    create_table :users do |t|
      t.attachment :avatar
    end
  end
end

Schema Definition

Alternatively, the add_attachment and remove_attachment methods can be used to add new Paperclip columns to an existing table:

class AddAttachmentColumnsToUsers < ActiveRecord::Migration
  def up
    add_attachment :users, :avatar
  end

  def down
    remove_attachment :users, :avatar
  end
end

Or you can do this with the change method:

class AddAttachmentColumnsToUsers < ActiveRecord::Migration
  def change
    add_attachment :users, :avatar
  end
end

Vintage Syntax

Vintage syntax (such as t.has_attached_file and drop_attached_file) is still supported in Paperclip 3.x, but you're advised to update those migration files to use this new syntax.


Storage

Paperclip ships with 3 storage adapters:

  • File Storage (storage: :filesystem)
  • S3 Storage (via aws-sdk-s3) (storage: :s3)
  • Fog Storage (storage: :fog)

If you would like to use Paperclip with another storage, you can install these gems along side with Paperclip:

Understanding Storage

The files that are assigned as attachments are, by default, placed in the directory specified by the :path option to has_attached_file. By default, this location is :rails_root/public/system/:class/:attachment/:id_partition/:style/:filename. This location was chosen because, on standard Capistrano deployments, the public/system directory can be symlinked to the app's shared directory, meaning it survives between deployments. For example, using that :path, you may have a file at

/data/myapp/releases/20081229172410/public/system/users/avatar/000/000/013/small/my_pic.png

NOTE: This is a change from previous versions of Paperclip, but is overall a safer choice for the default file store.

You may also choose to store your files using Amazon's S3 service. To do so, include the aws-sdk-s3 gem in your Gemfile:

gem 'aws-sdk-s3'

And then you can specify using S3 from has_attached_file. You can find more information about configuring and using S3 storage in the Paperclip::Storage::S3 documentation.

Files on the local filesystem (and in the Rails app's public directory) will be available to the internet at large. If you require access control, it's possible to place your files in a different location. You will need to change both the :path and :url options in order to make sure the files are unavailable to the public. Both :path and :url allow the same set of interpolated variables.


IO Adapters

When a file is uploaded or attached, it can be in one of a few different input forms, from Rails' UploadedFile object to a StringIO to a Tempfile or even a simple String that is a URL that points to an image.

Paperclip will accept, by default, many of these sources. It also is capable of handling even more with a little configuration. The IO Adapters that handle images from non-local sources are not enabled by default. They can be enabled by adding a line similar to the following into config/initializers/paperclip.rb:

Paperclip::DataUriAdapter.register

It's best to only enable a remote-loading adapter if you need it. Otherwise there's a chance that someone can gain insight into your internal network structure using it as a vector.

The following adapters are not loaded by default:

  • Paperclip::UriAdapter - which accepts a URI instance.
  • Paperclip::HttpUrlProxyAdapter - which accepts a http string.
  • Paperclip::DataUriAdapter - which accepts a Base64-encoded data: string.

Post Processing

Paperclip supports an extensible selection of post-processors. When you define a set of styles for an attachment, by default it is expected that those "styles" are actually "thumbnails." These are processed by Paperclip::Thumbnail. For backward compatibility reasons you can pass either a single geometry string, or an array containing a geometry and a format that the file will be converted to, like so:

has_attached_file :avatar, styles: { thumb: ["32x32#", :png] }

This will convert the "thumb" style to a 32x32 square in PNG format, regardless of what was uploaded. If the format is not specified, it is kept the same (e.g. JPGs will remain JPGs). Paperclip::Thumbnail uses ImageMagick to process images; ImageMagick's geometry documentation has more information on the accepted style formats.

For more fine-grained control of the conversion process, source_file_options and convert_options can be used to pass flags and settings directly to ImageMagick's powerful Convert tool, documented here. For example:

has_attached_file :image, styles: { regular: ['800x800>', :png]},
    source_file_options: { regular: "-density 96 -depth 8 -quality 85" },
    convert_options: { regular: "-posterize 3"}

ImageMagick supports a number of environment variables for controlling its resource limits. For example, you can enforce memory or execution time limits by setting the following variables in your application's process environment:

  • MAGICK_MEMORY_LIMIT=128MiB
  • MAGICK_MAP_LIMIT=64MiB
  • MAGICK_TIME_LIMIT=30

For a full list of variables and description, see ImageMagick's resources documentation.


Custom Attachment Processors

You can write your own custom attachment processors to carry out tasks like adding watermarks, compressing images, or encrypting files. Custom processors must be defined within the Paperclip module, inherit from Paperclip::Processor (see lib/paperclip/processor.rb), and implement a make method that returns a File. All files in your Rails app's lib/paperclip and lib/paperclip_processors directories will be automatically loaded by Paperclip. Processors are specified using the :processors option to has_attached_file:

has_attached_file :scan, styles: { text: { quality: :better } },
                         processors: [:ocr]

This would load the hypothetical class Paperclip::Ocr, and pass it the options hash { quality: :better }, along with the uploaded file.

Multiple processors can be specified, and they will be invoked in the order they are defined in the :processors array. Each successive processor is given the result from the previous processor. All processors receive the same parameters, which are defined in the :styles hash. For example, assuming we had this definition:

has_attached_file :scan, styles: { text: { quality: :better } },
                         processors: [:rotator, :ocr]

Both the :rotator processor and the :ocr processor would receive the options { quality: :better }. If a processor receives an option it doesn't recognise, it's expected to ignore it.

NOTE: Because processors operate by turning the original attachment into the styles, no processors will be run if there are no styles defined.

If you're interested in caching your thumbnail's width, height and size in the database, take a look at the paperclip-meta gem.

Also, if you're interested in generating the thumbnail on-the-fly, you might want to look into the attachment_on_the_fly gem.

Paperclip's thumbnail generator (see lib/paperclip/thumbnail.rb) is implemented as a processor, and may be a good reference for writing your own processors.


Events

Before and after the Post Processing step, Paperclip calls back to the model with a few callbacks, allowing the model to change or cancel the processing step. The callbacks are before_post_process and after_post_process (which are called before and after the processing of each attachment), and the attachment-specific before_<attachment>_post_process and after_<attachment>_post_process. The callbacks are intended to be as close to normal ActiveRecord callbacks as possible, so if you return false (specifically - returning nil is not the same) in a before_filter, the post processing step will halt. Returning false in an after_filter will not halt anything, but you can access the model and the attachment if necessary.

NOTE: Post processing will not even start if the attachment is not valid according to the validations. Your callbacks and processors will only be called with valid attachments.

class Message < ActiveRecord::Base
  has_attached_file :asset, styles: { thumb: "100x100#" }

  before_post_process :skip_for_audio

  def skip_for_audio
    ! %w(audio/ogg application/ogg).include?(asset_content_type)
  end
end

URI Obfuscation

Paperclip has an interpolation called :hash for obfuscating filenames of publicly-available files.

Example Usage:

has_attached_file :avatar, {
    url: "/system/:hash.:extension",
    hash_secret: "longSecretString"
}

The :hash interpolation will be replaced with a unique hash made up of whatever is specified in :hash_data. The default value for :hash_data is ":class/:attachment/:id/:style/:updated_at".

:hash_secret is required - an exception will be raised if :hash is used without :hash_secret present.

For more on this feature, read the author's own explanation

Checksum / Fingerprint

A checksum of the original file assigned will be placed in the model if it has an attribute named fingerprint. Following the user model migration example above, the migration would look like the following:

class AddAvatarFingerprintColumnToUser < ActiveRecord::Migration
  def up
    add_column :users, :avatar_fingerprint, :string
  end

  def down
    remove_column :users, :avatar_fingerprint
  end
end

The algorithm can be specified using a configuration option; it defaults to MD5 for backwards compatibility with Paperclip 5 and earlier.

has_attached_file :some_attachment, adapter_options: { hash_digest: Digest::SHA256 }

Run CLASS=User ATTACHMENT=avatar rake paperclip:refresh:fingerprints after changing the digest on existing attachments to update the fingerprints in the database.

File Preservation for Soft-Delete

An option is available to preserve attachments in order to play nicely with soft-deleted models. (acts_as_paranoid, paranoia, etc.)

has_attached_file :some_attachment, {
    preserve_files: true,
}

This will prevent some_attachment from being wiped out when the model gets destroyed, so it will still exist when the object is restored later.


Dynamic Configuration

Callable objects (lambdas, Procs) can be used in a number of places for dynamic configuration throughout Paperclip. This strategy exists in a number of components of the library but is most significant in the possibilities for allowing custom styles and processors to be applied for specific model instances, rather than applying defined styles and processors across all instances.

Dynamic Styles:

Imagine a user model that had different styles based on the role of the user. Perhaps some users are bosses (e.g. a User model instance responds to #boss?) and merit a bigger avatar thumbnail than regular users. The configuration to determine what style parameters are to be used based on the user role might look as follows where a boss will receive a 300x300 thumbnail otherwise a 100x100 thumbnail will be created.

class User < ActiveRecord::Base
  has_attached_file :avatar, styles: lambda { |attachment| { thumb: (attachment.instance.boss? ? "300x300>" : "100x100>") } }
end

Dynamic Processors:

Another contrived example is a user model that is aware of which file processors should be applied to it (beyond the implied thumbnail processor invoked when :styles are defined). Perhaps we have a watermark processor available and it is only used on the avatars of certain models. The configuration for this might be where the instance is queried for which processors should be applied to it. Presumably some users might return [:thumbnail, :watermark] for its processors, where a defined watermark processor is invoked after the thumbnail processor already defined by Paperclip.

class User < ActiveRecord::Base
  has_attached_file :avatar, processors: lambda { |instance| instance.processors }
  attr_accessor :processors
end

Logging

By default, Paperclip outputs logging according to your logger level. If you want to disable logging (e.g. during testing) add this into your environment's configuration:

Your::Application.configure do
...
  Paperclip.options[:log] = false
...
end

More information in the rdocs


Deployment

To make Capistrano symlink the public/system directory so that attachments survive new deployments, set the linked_dirs option in your config/deploy.rb file:

set :linked_dirs, fetch(:linked_dirs, []).push('public/system')

Attachment Styles

Paperclip is aware of new attachment styles you have added in previous deploys. The only thing you should do after each deployment is to call rake paperclip:refresh:missing_styles. It will store current attachment styles in RAILS_ROOT/public/system/paperclip_attachments.yml by default. You can change it by:

Paperclip.registered_attachments_styles_path = '/tmp/config/paperclip_attachments.yml'

Here is an example for Capistrano:

namespace :paperclip do
  desc "build missing paperclip styles"
  task :build_missing_styles do
    on roles(:app) do
      within release_path do
        with rails_env: fetch(:rails_env) do
          execute :rake, "paperclip:refresh:missing_styles"
        end
      end
    end
  end
end

after("deploy:compile_assets", "paperclip:build_missing_styles")

Now you don't have to remember to refresh thumbnails in production every time you add a new style. Unfortunately, it does not work with dynamic styles - it just ignores them.

If you already have a working app and don't want rake paperclip:refresh:missing_styles to refresh old pictures, you need to tell Paperclip about existing styles. Simply create a paperclip_attachments.yml file by hand. For example:

class User < ActiveRecord::Base
  has_attached_file :avatar, styles: { thumb: 'x100', croppable: '600x600>', big: '1000x1000>' }
end

class Book < ActiveRecord::Base
  has_attached_file :cover, styles: { small: 'x100', large: '1000x1000>' }
  has_attached_file :sample, styles: { thumb: 'x100' }
end

Then in RAILS_ROOT/public/system/paperclip_attachments.yml:

---
:User:
  :avatar:
  - :thumb
  - :croppable
  - :big
:Book:
  :cover:
  - :small
  - :large
  :sample:
  - :thumb

Testing

Paperclip provides rspec-compatible matchers for testing attachments. See the documentation on Paperclip::Shoulda::Matchers for more information.

Parallel Tests

Because of the default path for Paperclip storage, if you try to run tests in parallel, you may find that files get overwritten because the same path is being calculated for them in each test process. While this fix works for parallel_tests, a similar concept should be used for any other mechanism for running tests concurrently.

if ENV['PARALLEL_TEST_GROUPS']
  Paperclip::Attachment.default_options[:path] = ":rails_root/public/system/:rails_env/#{ENV['TEST_ENV_NUMBER'].to_i}/:class/:attachment/:id_partition/:filename"
else
  Paperclip::Attachment.default_options[:path] = ":rails_root/public/system/:rails_env/:class/:attachment/:id_partition/:filename"
end

The important part here being the inclusion of ENV['TEST_ENV_NUMBER'], or a similar mechanism for whichever parallel testing library you use.

Integration Tests

Using integration tests with FactoryBot may save multiple copies of your test files within the app. To avoid this, specify a custom path in the config/environments/test.rb like so:

Paperclip::Attachment.default_options[:path] = "#{Rails.root}/spec/test_files/:class/:id_partition/:style.:extension"

Then, make sure to delete that directory after the test suite runs by adding this to spec_helper.rb.

config.after(:suite) do
  FileUtils.rm_rf(Dir["#{Rails.root}/spec/test_files/"])
end

Example of test configuration with Factory Bot

FactoryBot.define do
  factory :user do
    avatar { File.new("#{Rails.root}/spec/support/fixtures/image.jpg") }
  end
end

Contributing

If you'd like to contribute a feature or bugfix: Thanks! To make sure your fix/feature has a high chance of being included, please read the following guidelines:

  1. Post a pull request.
  2. Make sure there are tests! We will not accept any patch that is not tested. It's a rare time when explicit tests aren't needed. If you have questions about writing tests for paperclip, please open a GitHub issue.

Please see CONTRIBUTING.md for more details on contributing and running test.

Thank you to all the contributors!

License

Copyright © 2020-2021 Kreeti Technologies Pvt. Ltd. Copyright © 2008-2017 thoughtbot, inc. It is free software, and may be redistributed under the terms specified in the MIT-LICENSE file.

kt-paperclip's People

Contributors

aboutqx avatar alexgodin avatar betesh avatar bjhess avatar dasch avatar dgynn avatar djcp avatar fxposter avatar henrik avatar joshuaclayton avatar maclover7 avatar mike-burns avatar monde avatar morgoth avatar mschulkind avatar murbanski avatar mvanholstyn avatar nickrivadeneira avatar princemaple avatar qrush avatar sbhawsingka avatar sidraval avatar sikachu avatar soramugi avatar sporky023 avatar ssinghi avatar tilsammans avatar tute avatar xuanxu avatar zmillman avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

kt-paperclip's Issues

NameError: uninitialized constant Paperclip::Storage::Fog::Excon (Zeitwerk?)

Describe the bug
First, thank you for keeping updating this cool gem.

I think we have found a bug, probably related to Rails Zeitwerk mode, where we get NameError, when we use Paperclip to store or update PDF files.

When we try to assign a document to a record with Paperclip attachment: bill.update document: pdf_document
We will see this error:

NameError: uninitialized constant Paperclip::Storage::Fog::Excon
from .bundle/ruby/3.0.0/gems/kt-paperclip-7.1.1/lib/paperclip/storage/fog.rb:120:in `rescue in block in flush_writes'
Caused by NameError: uninitialized constant Fog
from .bundle/ruby/3.0.0/gems/kt-paperclip-7.1.1/lib/paperclip/storage/fog.rb:232:in `connection'

See https://github.com/kreeti/kt-paperclip/blob/master/lib/paperclip/storage/fog.rb#L120

I also tried to activate config.autoloader = :classic mode, bud got the same error.

After adding gem "fog-aws", "~> 3.15.0", I got a new error:

TypeError: wrong argument type Class (expected Module)
from /home/nonroot/.bundle/ruby/3.0.0/gems/kt-paperclip-7.1.1/lib/paperclip/attachment.rb:425:in `extend'

See https://github.com/kreeti/kt-paperclip/blob/master/lib/paperclip/attachment.rb#L425

So, I have no idea, how to solve it.

Additional context
We are in process to upgrade old application from Rails 5 to 6.1 and Ruby 2.5.x to 3.0.4.
We followed Rails Guides, in order to upgrade, e.g.:

https://edgeguides.rubyonrails.org/upgrading_ruby_on_rails.html#autoloading
https://edgeguides.rubyonrails.org/classic_to_zeitwerk_howto.html
https://edgeguides.rubyonrails.org/autoloading_and_reloading_constants.html

Rails 6.1 tests fail

Describe the bug

The Rails 6.1 tests fail

To Reproduce

Look at the TravisCI results for 6.1, for example here.

Expected behavior

The tests should pass.

Additional context

I have created a PR#59 to fix the deprecation warnings that show up in that Travis run. Together with fixing these tests, it will make kt-paperclip ready to run on the newest versions of Rails (6.1 and the in-development 7.0) & Ruby (3.0).

wrong number of arguments (given 3, expected 1..2) with validations on Rails 6.1

Describe the bug

When using kt-paperclip on Rails 6.1, when using validations that will trigger errors, Rails will throw a wrong number of arguments (given 3, expected 1..2) error.

See rails/rails#41270 for details on that.

To Reproduce

class User < ApplicationRecord
  has_attached_file :avatar
  validates_attachment_presence :avatar, if: :confirmed?
end

This will trigger the error, if avatar is missing

Expected behavior

Should not trigger the error.

Additional context

Changing https://github.com/kreeti/kt-paperclip/blob/master/lib/paperclip/validators/attachment_presence_validator.rb#L7 to:

record.errors.add(attribute, :blank, **options) if record.send("#{attribute}_file_name").blank?

Fixes it for the presence validator. Not sure how compatible that would be to older versions though.

application/vnd.ms-excel uploaded file with content type validation of text/csv doesn't work

This is directly related to this issue on the original paperclip repo. thoughtbot#1924

I wasn't sure if issues like this should be addressed in this repo, but I was really hoping I could find some workaround than the one suggested there in turning off spoof checking for the files. My understanding is that the file -b --mime check on windows saved excel files produce the application/vnd.ms-excel mime-type. With the content-type text/csv specified in a model using paperclip this fails with has contents that are not what they are reported to be.

If issues like this aren't being addressed in this fork I understand, but figured it was worth a shot to ask about it. I would've used the issue template provided but honestly I'm not 100% familiar with the problem or how it happens. I don't have a windows machine or excel, but from what I've gathered from clients running into this issue, and then the linked GitHub issue I tried to give a description that hopefully helps. If it's too ambiguous no worries.

6.4.0 - Fix NoMethodError (private method `open' called for Paperclip::HttpUrlProxyAdapter)

Describe the bug
When using ruby-2.7.x with kt-paperclip, get the deprecation warning:

kt-paperclip-6.4.0/lib/paperclip/io_adapters/uri_adapter.rb:56: 
warning: calling URI.open via Kernel#open is deprecated, call URI.open directly or use URI#open

#35 fixed one part of the ruby-2.7 URI deprecation. This fix is needed for other parts.

However, 6.4 also broke the gem in rub 2.6

private method `open' called for Paperclip::HttpUrlProxyAdapter: :Paperclip::HttpUrlProxyAdapter
kt-paperclip-6.4.0/lib/paperclip/io_adapters/uri_adapter.rb:56:in `download_content'

To Reproduce
Run the gem in ruby-2.7. it will produce

warning: calling URI.open via Kernel#open is deprecated, call URI.open directly or use URI#open

Run 6.4.0 in ruby 2.6, it will error

NoMethodError (private method `open' called for Paperclip::HttpUrlProxyAdapter: :Paperclip::HttpUrlProxyAdapter)
kt-paperclip-6.4.0/lib/paperclip/io_adapters/uri_adapter.rb:56:in `download_content'

Expected behavior
No warnings are logged.

Additional context
See ruby-lang issue:15893.

Rubocop also has a check.

Note that the fix, to change from self.open to URI.open is compatible with ruby-2.5 and ruby-2.6.

If you still want to support ruby 2.4, the fix needs to be sensitive to ruby version.

i.,e

    if RUBY_VERSION < '2.5'
      def download_content
        options = { read_timeout: Paperclip.options[:read_timeout] }.compact

        open(@target, options)
      end
    else
      def download_content
        options = { read_timeout: Paperclip.options[:read_timeout] }.compact

        URI.open(@target, options)
      end
    end

I prefer changing the method itself based on ruby version rather than the open call as it makes it a gem load-time decision vs runtime.

Content Type Spoof: For XML file in v7.1

Describe the bug
After updating to the version v7.1 we get this 'error':

Content Type Spoof: Filename 534fa0c71f8a678c08e6fbc40e4e362d20220215-98201-1po9m4z.xml (application/xml from Headers, ["application/xml", "text/xml"] from Extension), content type discovered from file command: text/html. See documentation to allow this combination.

I think the detection for the content_type changed with #75

New behavior:

@filepath = "/var/folders/_6/l152jtnn025b5nfgcvss7v200000gp/T/534fa0c71f8a678c08e6fbc40e4e362d20220215-98201-1po9m4z.xml"
"/var/folders/_6/l152jtnn025b5nfgcvss7v200000gp/T/534fa0c71f8a678c08e6fbc40e4e362d20220215-98201-1po9m4z.xml"

Marcel::MimeType.for Pathname.new(@filepath), name: @filepath
"application/xml"

Old behavior:

Marcel::Magic.by_magic(@filepath)
nil

# fallback for marcel detection
FileCommandContentTypeDetector.new(@filepath).detect
"text/html"

Mime type from file (Mac OS)

$ file --version
file-5.39
magic file from /usr/share/file/magic

$ file -b --mime /var/folders/_6/l152jtnn025b5nfgcvss7v200000gp/T/534fa0c71f8a678c08e6fbc40e4e362d20220215-98201-1po9m4z.xml
text/html; charset=utf-8

Correct solution?

Paperclip.options[:content_type_mappings] = {
  xml: %w[application/xml text/html]
}

Is this the best way to handle this Problem? Are there any other known content_type changes?

Is there a best of for Paperclip.options[:content_type_mappings]?


Update I found out that it depends on the XML file what the file command returns:

text/xml; charset=us-ascii
text/html; charset=utf-8
text/plain; charset=us-ascii

So my updated solution would be:

Paperclip.options[:content_type_mappings] = {
  xml: %w[application/xml text/xml text/html text/plain]
}

Breaks After Rails 6 Upgrade

In Rails 6, with Ruby 3.0.2, all calls return:
NoMethodError - undefined method `escape' for URI:Module:
I believe I'm using the latest version of kt-paperclip:
gem "kt-paperclip", git: "git://github.com/kreeti/kt-paperclip.git"
Thanks

On paperclip-meta

Is your feature request related to a problem? Please describe.

gem 'kt-paperclip'
gem 'paperclip-meta'

still depends on paperclip. Do you have any plan to also fork paperclip-meta?

Describe the solution you'd like
kt-paperclip team maintain a fork of paperclip-meta

Describe alternatives you've considered

  • paperclip-meta team maintain their own fork for kt-paperclip
  • We maintain a form of paperclip-meta for kt-paperclip (privately)

Rails 6 and 7 break kt-paperclip due to lack of :on support in callbacks

Describe the bug
In Rails 6 and above, kt-paperlcip fails on line 98 in lib/paperclip/has_attached_file.rb where an :on condition is used for after_commit / destroy


 93     def add_active_record_callbacks
 94       name = @name
 95       @klass.send(:after_save) { send(name).send(:save) }
 96       @klass.send(:before_destroy) { send(name).send(:queue_all_for_delete) }
 97       if @klass.respond_to?(:after_commit)
 98         @klass.send(:after_commit, on: :destroy) do
 99           send(name).send(:flush_deletes)
100         end
101       else
102         @klass.send(:after_destroy) { send(name).send(:flush_deletes) }
103       end
104     end

To Reproduce
Steps to reproduce the behavior:

  1. Implement has_attached_file with any settings
  2. Attempt an rspec run, rails console, or start a puma server etc.
  3. Error will get thrown to the console
ArgumentError:
  Unknown key: :on. Valid keys are: :if, :unless, :prepend

Expected behavior
Should function normally.

Desktop (please complete the following information):

  • Amazon Linux2 and macos
  • All versions

Looks like someone else has hit this:
https://stackoverflow.com/questions/76966547/rails-7-argumenterror-unknown-key-on-valid-keys-are-if-unless-prepend

"Paperclip::Errors::NotIdentifiedByImageMagickError" on Debian Buster, fixed by altering the config of imageMagic

Describe the bug
On an up-to-date Debian Buster machine, when trying to insert a PDF file with Paperclip, the error: Paperclip::Errors::NotIdentifiedByImageMagickError
is returned.

To Reproduce
Steps to reproduce the behavior:

  1. Load a PDF file in a file-input field
  2. Save
  3. Look with horror at the error

Expected behavior
No error of course!

Screenshots
N/A

Additional context
The issue is likely linked to ImageMagick as here is the return of the convert command on a PDF file:

**convert 20F0001.pdf test.png**
convert-im6.q16: attempt to perform an operation not allowed by the security policy `PDF' @ error/constitute.c/IsCoderAuthorized/408.
convert-im6.q16: no images defined `test.png' @ error/convert.c/ConvertImageCommand/3258.
zsh: exit 1     convert 20F0001.pdf test.png

Fix `URI.escape is obsolete` warnings on Paperclip 4.x and 5.x

The problem

People that are not yet on Paperclip 6.x but on Ruby 2.7 have currently no alternative to upgrading to the latest major version in order to avoid seeing those deprecations, as KT Paperclip versions for 4.x and 5.x don't exist (yet?), so obviously they cannot incorporate the fix (which I think is included in #38).

Proposed solution

I was wondering if it would be possible to release patch versions that include the fix for older major versions of Paperclip. I offer my help, if this makes sense to you.

Additional context

As maintainers of Solidus, we have a non-current major version still supported of the framework that is still using the original Paperclip, so all installations using this version running on ruby 2.7.x are emitting the above-mentioned deprecation. Paperclip requirement on this version is >= 4.2, so we should not change dependencies to new major versions (i.e. to KT Paperclip 6.3) as this would automatically break semantic versioning. This is the PR that started the discussion on Solidus.

Pdf thumbnail not being generated when file is encoded in base64

Describe the issue
Hello everyone, thanks for the amazing job you've been doing with kt-paperclip.

We are currently working on a project that would store different types of files (images, videos, documents, etc) through an API call. We have 2 different projects working at the same time. On the first one ('Project Base'), we already have in place the system working and it's working as expected. The second project ('Project Client') would make an API call to Project Base, sending the desired files converted to base64 through a json call.

Everything works as expected when storing from Project Client to Project Base, except for pdf files. For this specific type, the thumbnail is never being generated, although the original file is stored as expected. Note that the same functionality on Project Base works as expected (if we upload from there, then the thumbnail is generated and stored on our s3 bucket).

Our controller for the upload action on Project Base is this one:

  def create
    @attachment = base_attachments.build(attachment_params.merge(user_id: current_user.id))

    respond_to do |format|
      if @attachment.save
        format.json
      else
        format.json { render json: { error: @attachment.errors.full_messages.to_sentence }, status: :bad_request }
      end
    end
  end

We wanted to verify if this could be an issue originating on Project Client so we modified this to store the file after converting it to a Base64 string.

    def create
      attachment = attachment_params[:attachment]
      base64 = "data:application/pdf;base64,#{Base64.strict_encode64(File.open(attachment).read)}"
      base_64 = attachment_params[:attachment]
      @attachment = base_attachments.build(attachment_params.merge(attachment: base64, user_id: current_user.id))

      respond_to do |format|
        if @attachment.save
          format.json
        else
          format.json { render json: { error: @attachment.errors.full_messages.to_sentence }, status: :bad_request }
        end
      end
    end

With this, same behaviour occurs: the original file is stored but the thumbnail is never generated.

The model is defined as follows:

module AttachmentType
  class Download < Attachment
    include AttachmentThumbnailConcern

    has_attached_file(
      :attachment,
      bucket: APP_CONFIG[:aws][:bucket],
      default_url: '/images/:style/missing.png',
      path: "/#{Rails.env}/downloads/:attachable_type/:attachable_id/:style/:basename.:extension",
      s3_region: APP_CONFIG[:aws][:region],
      styles: ->(a) { a.instance.check_file_type },
      processors: ->(a) { a.is_video? ? [:transcoder] : [:thumbnail] }
    )

    # Before applying the Imagemagick post processing to this record
    # check to see if we indeed wish to process the file. In the case
    # of audio files, we don't want to apply post processing
    before_post_process :apply_post_processing?

    validates_attachment_content_type :attachment, content_type: %w[
      image/png
      image/jpg
      image/jpeg
      application/pdf
      application/msword
      application/vnd.openxmlformats-officedocument.wordprocessingml.document
      application/vnd.ms-powerpoint
      application/vnd.openxmlformats-officedocument.presentationml.presentation
      application/vnd.ms-excel
      application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
    ]

    def microsoft_file?
      File.extname(attachment_file_name).match?(/xlsx?\Z|docx?\Z|pptx?\Z/)
    end

    def thumbnail_uri
      if microsoft_file?
        generic_icon_path
      else
        attachment.url(is_pdf? ? :pdf_thumbnail : :thumb)
      end
    end

    def generic_icon_path
      ext = File.extname(attachment_file_name).tr('.', '')
      ext = ext[0..-2] if ext.last == 'x'
      url = "/images/files/#{ext}.svg"

      if File.exists?("#{Rails.root}/public#{url}")
        url
      else
        "/images/files/default.svg"
      end
    end
  end
end

AttachmentThumbnailConcern is defined as follows:

require 'active_support/concern'
module AttachmentThumbnailConcern
  extend ActiveSupport::Concern

  included do
    # Helper method that uses the =~ regex method to see if
    # the current file_upload has a content_type
    # attribute that contains the string "image" / "video", or "audio"
    def is_image?
      attachment.content_type =~ %r(image)
    end

    def is_video?
      attachment.content_type =~ %r(video)
    end

    def is_audio?
      attachment.content_type =~ /\Aaudio\/.*\Z/
    end

    def is_plain_text?
      attachment_file_name =~ %r{\.(txt)$}i
    end

    def is_excel?
      attachment_file_name =~ %r{\.(xls|xlt|xla|xlsx|xlsm|xltx|xltm|xlsb|xlam|csv|tsv)$}i
    end

    def is_word_document?
      attachment_file_name =~ %r{\.(docx|doc|dotx|docm|dotm)$}i
    end

    def is_powerpoint?
      attachment_file_name =~ %r{\.(pptx|ppt|potx|pot|ppsx|pps|pptm|potm|ppsm|ppam)$}i
    end

    def is_pdf?
      attachment_file_name =~ %r{\.(pdf)$}i
    end

    def has_default_image?
      is_audio?
      is_plain_text?
      is_excel?
      is_word_document?
    end

    # If the uploaded content type is an audio file,
    # return false so that we'll skip audio post processing
    def apply_post_processing?
      if has_default_image?
        return false
      else
        return true
      end
    end

    # Method to be called in order to determine what styles we should
    # save of a file.
    def check_file_type
      if is_image?
        {
          thumb: "300x300>"
        }
      elsif is_pdf?
        {
          pdf_thumbnail: ["", :png]
        }

      elsif is_video?
        {
          thumb: {
            geometry: "432x243>",
            format: 'jpg',
            time: 2
          },
          # medium: {
          #   geometry: "500x500>",
          #   format: 'jpg',
          #   time: 0
          # }
        }
      elsif is_audio?
        {
          audio: {
            format: "mp3"
          }
        }
      else
        {}
      end
    end
  end
end

config/initializers/paperclip.rb is defined as follows:

Paperclip::UriAdapter.register
Paperclip::DataUriAdapter.register
Paperclip::HttpUrlProxyAdapter.register

Paperclip.interpolates :attachable_id do |attachment, style|
  attachment.instance.attachable_id
end

Paperclip.interpolates :attachable_type do |attachment, style|
  attachment.instance.attachable_type.pluralize.underscore
end

Currently using
kt-paperclip 7.0.1
Rails 6.1.6
Ruby 2.7.5

Expected behavior
We understand that handling base64 files should be handled the same as an UploadedFile object and that no other options should be passed to has_attached_file.

I've been through the code and the documentation and can't see anything related to any different option in the event of a base64 string file, so not sure if this could be an actual issue or maybe some parameter not being passed properly.

If you have any idea on this, I would really much appreciate your suggestions on this matter.

Compatibility issue with Ruby 3.1

Describe the bug
I'm getting an error when trying to run kt-paperclip on Ruby 3.1.

To Reproduce
Running current specs using ruby 3.1 the error occurs.
PS: I suppressed other test results because I had difficulty run the entire spec in my local environment.

$ rspec spec/paperclip/storage/s3_spec.rb:1506
Testing against version 7.0.3.1
Run options: include {:locations=>{"./spec/paperclip/storage/s3_spec.rb"=>[1506]}}
F

Failures:

  1) Paperclip::Storage::S3 with S3 credentials in a YAML file and aliases being set runs the file through ERB
     Failure/Error: YAML::safe_load(ERB.new(File.read(creds.path)).result, [], [], true)

     ArgumentError:
       wrong number of arguments (given 4, expected 1)
     # ./lib/paperclip/storage/s3.rb:430:in `find_credentials'
     # ./lib/paperclip/storage/s3.rb:316:in `parse_credentials'
     # ./lib/paperclip/storage/s3.rb:198:in `s3_credentials'
     # ./lib/paperclip/storage/s3.rb:232:in `bucket_name'
     # ./spec/paperclip/storage/s3_spec.rb:1523:in `block (3 levels) in <top (required)>'

Finished in 0.04137 seconds (files took 1.21 seconds to load)
1 example, 1 failure

Expected behavior
The entire spec work properly on Ruby 3.1

Desktop (please complete the following information):

  • OS: Darwin Kernel Version 21.6.0 / Ruby 3.1.2

Additional context
I found this bug when trying to migrate a rails app from Ruby 2.7.6 to Ruby 3.1.
Then I try to reproduce using the project specs but I'm getting some errors running the specs.
Regardless, I was able to reproduce the same error with the spec above.

Apparently the problem is related to the new version of Psych.
https://www.ctrl.blog/entry/ruby-psych4.html

s3_headers proc/lambda doesn't work

Describe the bug

To Reproduce
Steps to reproduce the behavior:

has_attached_file :asset,
  s3_headers: lambda { |attachment|
    puts "*********** attachment = #{attachment.to_json}"
    {
      'Cache-Control' => "max-age=#{1.year.to_i}",
      'Expires' => 1.year.from_now.httpdate,
      'Content-Disposition': "Attachment; file=#{attachment.asset_file_name}"
    }
  }

Expected behavior
When uploading a PDF file named "Test.pdf", it should print out

*********** attachment = {...., asset_file_name: "Test.pdf", asset_content_type: "application/pdf".... }

Desktop (please complete the following information):

  • OS: Docker ruby 2.6.6

Additional context
s3_headers doesn't get called with attachment therefore attachment.asset_file_name returns nil

Switch to mini_mime gem from mime-types

First of all thank you for continuing with paperclip. :)

Are there any planes to switch to mini_mime?

mini_mime is a more performant replacement for the mime-types gem. It uses the same database, but loads it in a more efficient manner, improving the number of allocations, memory, and speed.
The mail gem, which is a core component of Rails, has already switched to it.

mail
mikel/mail@634d911

Some more switches I could find:

capybara
teamcapybara/capybara#1884

rest-client
rest-client/rest-client#557

Prevent file name conflicts due to misconfiguration

Is your feature request related to a problem? Please describe.
I just fixed a problem caused by a misconfiguration with file path. Multiple styles were being generated (:original plus one custom one), but :style wasn't present in the path setting, so a filename conflict occurred.
This resulted in file uploads to a cloud service failing at first (due to a 404 from second delete), then succeeding on second try (no delete request made). It took a long time to figure out why it was "intermittently" failing!!!

Describe the solution you'd like
If there is more than one style (including :original), I think :style should be a required element in the path. Can this be validated upon initialisation?

Describe alternatives you've considered
Alternatively we could update the documentation to point this out
Actually, the path variables and :original style should be more documented anyway.

Additional context
Example configuration that was failing:

has_attached_file :image, path: "images/products/:id_:updated_at.:extension",# <-- no :style
                          default_style: :normal,
                          styles: { 
                            normal: { # <-- new style defined, without realising that :original also exists by default
                              geometry: '600x600#',
                              format: 'jpg',
                              quality: 80
                            }
                          }

Dev log from first try:

[paperclip] deleting images/products/3855_1637107719.jpg
[paperclip] deleting images/products/3855_1637107719.jpg
   (1.9ms)  ROLLBACK

Skipping work in a processor

Is your feature request related to a problem? Please describe.

We've been using Paperclip for a while in Mastodon, and we're chaining processors. Sometimes, a processor can decide it doesn't need to perform any change to the file. In this case, we just re-open the file to pass it to the next processor:

class LazyThumbnail < Paperclip::Thumbnail
  def make
    return File.open(@file.path) unless needs_convert?

    # … actual code
  end
end

This worked up until Paperclip 6.1, which caused the files to be closed after each processor in thoughtbot@90f9121, resulting in Errno::ENOENT errors.

This behavior remains unchanged in kt-paperclip so I was wondering how to best implement this “lazy” processor.

Looking at https://github.com/kreeti/kt-paperclip/blob/master/lib/paperclip/attachment.rb#L523-L530, there seem to be no clear way to handle that case.

Describe the solution you'd like

kt-paperclip should maybe unlink the original tempfile only once it has been fully processed. Pushing original into intermediate_files instead of immediately unlinking it should do the trick.

Describe alternatives you've considered

Our LazyThumbnail could create a copy of the file, but it seems wasteful.

Problem with `bundle install` never completing

I'm unable to get a bundle install for this project to complete, either on macOS (Big Sur), or on Linux (Ubuntu 18.04.1)

I've tried various versions of bundler, 2.1.x to a few different 2.2.x versions, on macOS, and under ruby 2.5.7 and 2.7.4.

On Linux, I've tried ruby 2.5.7 and bundler 2.2.25

On both platforms, the issue is that the "Resolving dependencies" process emits dots seemingly forever. I've allowed it to run for well over 24 hours, more than once, with no change.

For what it's worth, I see a similar behavior with thoughtbot's original paperclip repository.

I am able to successfully bundle install other open source ruby projects, including factory_bot and shoulda-matchers from thoughtbot, so my ruby installation may not be the issue.

I tried clearing the rubygems cache from ~/.bundle/cache/compact_index ... no change.

Are there any tips or tricks to bundling this project?

Thanks!

copy_to_local_file compatibility for storage who doesn't support multipart download

Aws::S3::Errors::BadRequest when copy_to_local_file cause my S3 storage provider doesn't support multipart download.
Bug discussion can be seen here. Conclusions below.

Some S3 compatible storage provider doesn't support multipart download, but in kt-paperclip, we happened to use an object.download_file method, which is multipart download by default.:

def copy_to_local_file(style, local_dest_path)
log("copying #{path(style)} to local file #{local_dest_path}")
s3_object(style).download_file(local_dest_path)
rescue Aws::Errors::ServiceError => e
warn("#{e} - cannot copy #{path(style)} to local file #{local_dest_path}")
false
end

In old paperclip, they use object.get method which is a single request:
https://github.com/thoughtbot/paperclip/blob/c769382c9b7078f3d1620b50ec2a70e91ba62ec4/lib/paperclip/storage/s3.rb#L413-L423

So can we consider provide an option of whether to use multipart download (s3_object(style).download_file(local_dest_path, { mode: 'single_request' }))?

New rubygems release?

Hi all,

Thanks for carrying on development of this gem. Would it be possible to get a new version released to rubygems incorporating the recent changes? Specifically it'd be really helpful to have Adjust version check to restore compatibility with non-rails projects available in a release version.
I'm aware we can set the gem's source to this commit with bundler, but this is cumbersome for gems that expose their dependency to kt-paperclip through a gemspec, where specifying a source isn't possible and must be done on all dependent applications.

If this is already planned or there's some other criteria pending for a release please feel free to close this. Thanks again.

undefined method `escape' for URI:Module (Rails 7 upgrade)

After upgrading to Rails 7.0.0 from 6.1.3 I get this exception. Running ruby 2.7.4 (which hasn't changed in this upgrade process)

My model has:

  has_attached_file :avatar, styles: {
    medium: "150x150>", thumb: "100x100>", small: "29x29>", large: "750x750>"
  }

This line, in my view, triggers the undefined method 'escape' for URI:Module exception:

...
<%= image_tag model.avatar.url(:thumb) %>
...

fog-google: Cannot insert legacy ACL

Describe the bug
Paperclip fog implementation sets fog_public to true if fog_public isn't specified (default value).
This causes fog-google to throw an exception when the corresponding bucket is using uniform access controls given predefined_acls (fine grain access control) is set (predefinedAcl=publicRead).
If fog_public were set to false then it also causes the same issue given predefinedAcl is set to private in fog-google.

Workaround: setting fog_public to a lambda that returns nil (seems hacky).

To Reproduce

  1. Create a GCS account.
  2. Create a bucket with uniform access control.
  3. Do not config fog_public
  4. Try saving an attachment.

Expected behavior
Not specifying fog_public leaves public nil.

Additional context
Issue on their end.
A PR providing a way to skip predefinedAcl if another gem sets it.

rails6 upgrade breaks has_attached_file, results in Unknown key: :on. Valid keys are: :if, :unless, :prepend

Steps to reproduce

During an upgrade from rails 5 to rails 6 on our app any models that use both an after_commit callback and have the has_attached_file method result in Unknown key: :on. Valid keys are: :if, :unless, :prepend

See this file:

@klass.send(:after_commit, on: :destroy) do

Expected behavior
No change from Rails 5 on after_commit callback.

Actual behavior
after_commit callbacks called using :on results in "Unknown key: :on. Valid keys are: :if, :unless, :prepend (ArgumentError)"

System configuration
Rails 6.1.4

Ruby version:
ruby 2.6.6

Notes:
Rails 6 adds strict argument checking for callbacks and now errors when using :on ,rails/rails#30919

Release new version

👋 I think with the new validator logic and Rails 7 support, it would be nice to get a new release 😄

Thanks!

Minitest::UnexpectedError: TypeError: no implicit conversion from nil to integer

Describe the bug

Model code

validates_attachment :logo, content_type: { content_type: %w[image/jpg image/jpeg image/gif image/png] }
  validates_attachment_size :logo, less_than: 5.megabytes, message: 'Max allowed attachment size is 5MB'
  validates_attachment_presence :logo

Test case:

FactoryGirl.create(:payment_provider, logo: File.new('test/fixtures/ofx-logo.png'))

To Reproduce
Steps to reproduce the behavior:

  1. I am upgrading my legacy Rails app which was originally built with Rails 4.2 to Rails 5.2.8
  2. We migrated our paperclip to kt-paperclip during rails 5 upgrade and been running it for years without any issue. It worked well till Rails 5.1.x.
  3. As soon as I upgraded Rails version to 5.2.8 I am getting given error Minitest::UnexpectedError: TypeError: no implicit conversion from nil to integer

Expected behavior
It should create uploads

Screenshots
If applicable, add screenshots to help explain your problem.

image

Desktop (please complete the following information):

  • OS: Ubuntu 20.20
  • Browser [e.g. chrome, safari]
  • Version kt-paperclip-6.4.1

Smartphone (please complete the following information):

  • Device: [e.g. iPhone6]
  • OS: [e.g. iOS8.1]
  • Browser [e.g. stock browser, safari]
  • Version [e.g. 22]

<<avatar>>_updated_at_changed? Not return value after save model

We are in process to upgrading the RoR and as a part of it upgraded paperclip 5.0 to 7.0.
After save model, not getting value true in <>_updated_at_changed? paperclip property.

I am not sure, why updated_at_changed? value not return true even model has file upload property.

I have taken two property in model data and data_local.
data = upload and link for s3 object
data_local = copy and converting data in local machine.

Code of model:

has_attached_file :data_local,
                       path: ":rails_root/public/system/:attachment/:id/:style/:basename.:extension",
                    url:  "/system/:attachment/:id/:style/:basename.:extension" 
has_attached_file :data,
                  processors: [:format_delegator],
                  path: ":configured_path",
                  whiny: false,
                  styles: {rectangle: ["1280x640#", :jpg],
                           narrow: ["640x", :jpg],
                           large: ["1980x", :jpg],
                           square: ["480x480#", :jpg],
                           thumbnail: ["1920x1080#", :jpg],
                           encoded: {output: "html5",
                                     processors: [:transcoder]}},
                  source_file_options:  { all: '-background white -alpha background'},
                  default_url: :set_default_url,
                  storage: :s3,
                  url: SETTINGS[:amazon][:assets_url_setting], 
                  s3_host_name: SETTINGS[:amazon][:host_name],
                  s3_host_alias: SETTINGS[:amazon][:cloudfront_host],
                  s3_permissions: :private,
                  s3_protocol: :https,
                  s3_region: SETTINGS[:amazon][:region],
                  s3_credentials: {
                      bucket: SETTINGS[:amazon][:bucket]
                  }
after_save do
    queue_upload_to_s3
    self.delay(queue: 'album', priority: 1).update_thumbnail if data_updated_at_changed? || data_local_updated_at_changed?
  end
def queue_upload_to_s3
      Delayed::Job.enqueue(FileModelJob.new(id), queue: 'upload', priority: 0) if data_local? && data_local_updated_at_changed?
  end
#queue_upload_to_s3 job will execute below function to move images to s3.
 def upload_to_s3
    #data_local moved to data and file will move to S3   
    self.data = data_local
    save!
  end

Request you to provide any views to solve the issue.
Is it paperclip bug? or need to change in code part?

paperclip-av-transcoder us

I am using paperclip-av-transcoder for video processing and it has a dependency on paperclip. Although I have uninstalled the paperclip and installed the kt-paperclip. rails is still using paperclip gem to file.

Is there any solution available for paperclip-av-transcoder, so I can fully migrate to kt-paperclip

Thanks in advance.

Webp image upload issue

webp images upload not supported it's giving this error message when trying to upload webp images has below

avatar_2_3_webp = open(image_ur)

at paperclip method "open" we are getting this error

ImageSpec::Error (#Paperclip::Tempfile:/tmp/18c5a2261d1d178068acb724a8a967b720221108-1242657-rpwwg520221108-1242657-n7zd3k.webp is not a supported image format. Sorry bub :():

we are using Rails 4.2 and ruby 2.3.0
paperclip (4.2.4)
paperclip-compression (0.3.11)
paperclip-optimizer (2.0.0)
ImageMagick 6.9.10-23

please help on this to how to fix this issue

Make `content_type_mappings` option more user-friendly

Is your feature request related to a problem? Please describe.
I spent quite some time debugging an issue in our project due to misconfiguration of content_type_mappings - keys were strings instead of symbols.

Describe the solution you'd like
I'd like for content_type_mappings option to be more user-friendly and/or fool-proof. Two options here:

  1. Raise an error when there are string keys provided;
  2. Don't raise any errors and make this option a "hash with indifferent access".

Additional context
This issue was discovered when we tried to upload .json files. Linux file utility returns text/plain content type while content type from extension is application/json. So in order to fix this we had to add a custom mapping:

Paperclip.options[:content_type_mappings] = {
  'json' => 'text/plain'
}

The above code didn't work, because the key is a string, not a symbol. The following code works as expected:

Paperclip.options[:content_type_mappings] = {
  json: 'text/plain'
}

This happens because of this code:

def filename_extension
File.extname(@name.to_s.downcase).sub(/^\./, "").to_sym
end

There was a related issue in the original repo: thoughtbot#2425

I'm happy to provide a PR once we settle on the exact solution: raise or accept both strings and symbols. Plus we could fix the case-sensitivity issue mentioned above as well.

Undefined Conversion Error in copy_to_local_file

Describe the bug
I'm getting an error when trying to copy a pdf file from a Paperclip::Attachment object (on S3) to a local tempfile.

To Reproduce
Steps to reproduce the behavior:

  1. Upload a PDF file to S3
  2. Create a tempfile
  3. call s3_file.copy_to_local_file
  4. See error below

Expected behavior
This error does not show up with paperclip 6.1.0, but several other errors and warnings do show up, which is why I am grateful you have made this gem available.

Screenshots

irb(main):010:0> s3_file.copy_to_local_file(:original, tmp_file)
[paperclip] copying /5270a9d4a5447bc32ad8557f380cb3eb3e12d997.pdf to local file #<File:0x000000010a05ec08>
Traceback (most recent call last):
        1: from (irb):10
Encoding::UndefinedConversionError ("\xFF" from ASCII-8BIT to UTF-8)

If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: MacOS 12.2.1
  • Ruby 2.7.5
  • Rails 6.1
  • kt-paperclip 6.4

Additional context
I found a similar problem with the paperclip gem from several years ago. Their solution involved adding the activesupport-json_encoder gem which is no longer supported. I don't have that gem in my Gemfile, so I presume they fixed it by 6.1.0.

Thanks again for all your work on this. Please let me know if there's anything else I can do to help.

Problem with your .github/FUNDING.yml

When I click the sponsor button I get an error from Github that informs me there are issues with the file .github/FUNDING.yml. You might want to fix that for possible sponsors.

Validation warnings

Describe the bug
Warnings associated with attachment validators that I have seen during testing:
/.rvm/gems/ruby-2.7.1/gems/kt-paperclip-6.4.1/lib/paperclip/validators/attachment_presence_validator.rb:7: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call

.rvm/gems/ruby-2.7.1/gems/kt-paperclip-6.4.1/lib/paperclip/validators/attachment_content_type_validator.rb:50: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call

.rvm/gems/ruby-2.7.1/gems/activemodel-6.1.1/lib/active_model/errors.rb:404: warning: The called method `add' is defined here

To Reproduce
Using validates_attachment and do_not_validate_attachment_file_type validators.

Expected behavior
No warnings.

I am using Rails 6.1 and kt-paperclip 6.4.1.

Thumbnail frame_index issue

Describe the bug
While migrating old paperclip 5.0 to new 7.0, execution throw error like

convert: unable to open image `/tmp/4d22f0c8f6b3e60d52c9633d4ad6b6e920211004-4396-hdequv.jpg[]': No such file or directory @ error/blob.c/OpenBlob/2712.
convert: no decode delegate for this image format `JPG[]' @ error/constitute.c/ReadImage/501.
convert: no images defined `/tmp/f5538c43af586218e55e52c89a854e5320211004-4396-ticqse.jpg' @ error/convert.c/ConvertImageCommand/3210.

Issue with, /tmp/ image should include [] frame_index blank at last.

Issue is with "def make" with below syntax
frame = animated? ? "" : "[#{@frame_index}]"
I have updated it with
frame = animated? ? "" : (@frame_index.present? "[#{@frame_index}]" : "")

I don't know if it's issue due to existing code or need to fix in paperclip.
Request you to fix the issue if it's bug in paperclip gems or provide valid approach to solve the issue.

Rails 7 compatibility: missing CHECKS constant in AttachmentSizeValidator

The CHECKS constant (as seen below) no longer exists in Rails 7. It seems to have been replaced by COMPARE_CHECKS
(there's also NUMBER_CHECKS & RANGE_CHECKS, though I don't think are relevant in this case)

unless value.send(CHECKS[option], option_value)

https://github.com/rails/rails/blob/ceb4b94baaf17f3a9f4ea795c83ec6c67211f737/activemodel/lib/active_model/validations/numericality.rb#L11-L14

Error during migration from Ruby 3.0.x to 3.1.x

Hi 👋

First thanks for this fork and the work you're doing!

Everything was running smoothly until I tried to upgrade my app from Ruby 3.0.x to Ruby 3.1.x. I changed nothing else than the ruby version and the method validates_attachment_content_type starts to fail with this error:

Screenshot 2022-12-06 at 09 33 12

I have no idea why, the data look correct but the record cannot be save

Any idea why?

Thanks in advance

Support SFTP storage

First of all, thank you for this gem! Since the gem paperclip was deprecated, we have been using this gem for our S3 storage type attachments along with a different gem - paperclip-sftp for our SFTP storage type attachments. The gem paperclip-sftp uses the deprecated paperclip gem but it is also pretty minimal. We cannot use that gem any longer due to how old it is and the other code it breaks.

Is your feature request related to a problem? Please describe.
It would be really nice if SFTP storage was a supported option and if we could use a DSL like this in our models:

  has_attached_file :csv, path: <path>, storage: :sftp,
    sftp_options: {
      host: <host>,
      user: <user>,
      password: <password>,
      port: <port>,
    }

Describe the solution you'd like
We could potentially mimic what the other gem was doing: https://github.com/spectator/paperclip-sftp.
Specifically we could add a new storage type that would look like this: https://github.com/spectator/paperclip-sftp/blob/master/lib/paperclip-sftp/storage/sftp.rb

Describe alternatives you've considered
Currently we are intending to initialize our app with this Paperclip::Storage::Sftp module

Additional context
I think this could be a useful feature for the community, but also a little surprised it was not requested before. Maybe SFTP attachments have other popular options nowadays?

Fix warning : URI.escape is obsolete

Describe the bug
When I use kt-paperclip on a Rails 6.0.3 application powered by Ruby 2.7.1 my logs are filled with this warning :

/home/pierre/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/kt-paperclip-6.3.0/lib/paperclip/url_generator.rb:68: warning: URI.escape is **obsolete**

To Reproduce
Run a rails application with kt-paperclip on Ruby 2.7+

Expected behavior
No warnings are logged

Additional context
The URI.escape documentation has this statement:

    # This method is obsolete and should not be used. Instead, use
    # CGI.escape, URI.encode_www_form or URI.encode_www_form_component
    # depending on your specific use case.

Several Ruby projects have fixed this error, example: contentful/contentful.rb@82dd6df#diff-606706a8b47f20495c7c206d12212665

undefined method `_read_attribute' for :read_attribute_for_validation:Symbol with Rails 7.0.4.1

Describe the bug
After upgrading to Rails 7.0.4.1 we're seeing frequent but not consistent failures across many branches related to a NoMethodError called in Paperclip.

To Reproduce

  1. an example failing model has a content type validation:
class MultimediaAsset < ApplicationRecord
  has_attached_file :upload
  validates_attachment_content_type :upload, content_type: /\Aimage\/.*\Z/
end
  1. We attempt to create one with a FactoryBot invocation:
multimedia_asset = create(:multimedia_asset, :as_image)

Expected behavior
It looks like some of the security fixes between Rails 7.0.4 and 7.0.4.1 might have changed some private APIs that Paperclip was using.

Screenshots
Here's an example spec error:



Failure/Error: multimedia_asset = create(:multimedia_asset, :as_image)
--
  |  
  | NoMethodError:
  | undefined method `_read_attribute' for :read_attribute_for_validation:Symbol
  | # /bundle/ruby/3.2.0/gems/kt-paperclip-7.1.1/lib/paperclip/validators/attachment_content_type_validator.rb:19:in `validate_each'
  | # /bundle/ruby/3.2.0/gems/kt-paperclip-7.1.1/lib/paperclip/validators.rb:68:in `block in create_validating_before_filter'
  | # /bundle/ruby/3.2.0/gems/kt-paperclip-7.1.1/lib/paperclip/callbacks.rb:28:in `block in hasta_la_vista_baby'
  | # /bundle/ruby/3.2.0/gems/kt-paperclip-7.1.1/lib/paperclip/callbacks.rb:38:in `run_paperclip_callbacks'
  | # /bundle/ruby/3.2.0/gems/kt-paperclip-7.1.1/lib/paperclip/attachment.rb:504:in `check_validity?'
  | # /bundle/ruby/3.2.0/gems/kt-paperclip-7.1.1/lib/paperclip/attachment.rb:494:in `post_process'
  | # /bundle/ruby/3.2.0/gems/kt-paperclip-7.1.1/lib/paperclip/attachment.rb:454:in `post_process_file'
  | # /bundle/ruby/3.2.0/gems/kt-paperclip-7.1.1/lib/paperclip/attachment.rb:112:in `assign'
  | # /bundle/ruby/3.2.0/gems/kt-paperclip-7.1.1/lib/paperclip/has_attached_file.rb:66:in `block in define_setter'
  | # /bundle/ruby/3.2.0/gems/factory_bot-6.2.1/lib/factory_bot/attribute_assigner.rb:16:in `public_send'
  | # /bundle/ruby/3.2.0/gems/factory_bot-6.2.1/lib/factory_bot/attribute_assigner.rb:16:in `block (2 levels) in object'
  | # /bundle/ruby/3.2.0/gems/factory_bot-6.2.1/lib/factory_bot/attribute_assigner.rb:15:in `each'
  | # /bundle/ruby/3.2.0/gems/factory_bot-6.2.1/lib/factory_bot/attribute_assigner.rb:15:in `block in object'
  | # /bundle/ruby/3.2.0/gems/factory_bot-6.2.1/lib/factory_bot/attribute_assigner.rb:14:in `object'
  | # /bundle/ruby/3.2.0/gems/factory_bot-6.2.1/lib/factory_bot/evaluation.rb:13:in `object'
  | # /bundle/ruby/3.2.0/gems/factory_bot-6.2.1/lib/factory_bot/strategy/create.rb:9:in `result'
  | # /bundle/ruby/3.2.0/gems/factory_bot-6.2.1/lib/factory_bot/factory.rb:43:in `run'
  | # /bundle/ruby/3.2.0/gems/factory_bot-6.2.1/lib/factory_bot/factory_runner.rb:29:in `block in run'
  | # /bundle/ruby/3.2.0/gems/factory_bot-6.2.1/lib/factory_bot/factory_runner.rb:28:in `run'
  | # /bundle/ruby/3.2.0/gems/factory_bot-6.2.1/lib/factory_bot/strategy_syntax_method_registrar.rb:28:in `block in define_singular_strategy_method'
  | # ./spec/models/permission_asset_spec.rb:423:in `block (4 levels) in <main>'

Additional context
Add any other context about the problem here.

Migration from paperclip 6.1.0 to latest release of kt-paperclip

Hi 👋
I don't see any docs that address migration from paperclip to kt-paperclip so I'm opening this issue.
I have huge app with this setup:

paperclip (6.1.0)
Ruby 2.7.2
Rails 6.1

Does replacing paperclip for kt-paperclip will break my app? I mean, will it work without having to re-upload all files?
Are there any changes that I need to apply for this to work?
Will I loose all my already uploaded attachments?

Thanks

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.