A pure Mongoid/Moped implementation of the MongoDB GridFS specification.
License: Other
This is the legacy fork which is no longer maintained. The official repository is now under the MongoDB organization here.
Hi folks, I just ran a bundle install on my project and I got the following message:
Using mongoid-grid_fs (2.0.0) from git://github.com/ahoward/mongoid-grid_fs.git (at master)
mongoid-grid_fs at .../ruby-1.9.3-p448@seaway/bundler/gems/mongoid-grid_fs-37287164eccc did not have a valid gemspec.
This prevents bundler from installing bins or native extensions, but that may not affect its functionality.
The validation message from Rubygems was:
["config.rb"] are not files
I want to find multiple files with given conditions, but "where" method is not public-accessible.
Running into this on Rails 4.1.4, Mongoid 4.0.0, Carrierwave 0.10.0, carrierwave-mongoid 0.7.1 and Mongoid-grid_fs 2.1.0 4dbecf1.
When uploading files via Carrierwave into GridFS, it's blowing up in lib/mongoid/grid_fs.rb in the "binary_for" method on line 172:
def binary_for(*buf)
if defined?(Moped::BSON)
Moped::BSON::Binary.new(:generic, buf.join)
else
BSON::Binary.new(buf.join, :generic)
end
endThe problem seems to be that Moped::BSON is still defined somewhere in the project, so it's using the old Moped BSON class even though that's been phased out. Actually, I found that if I change it to
if defined?(Moped::BSON::Binary)it works just fine, but I'm not exactly sure what the consequences of that are (I honestly don't know what the Moped::BSON constant is defined at all).
I'll fork and submit a pull request, and let you guys decide. Thanks.
I'm using mongoid-grid_fs on the following JRuby setup...
jruby 1.7.4 (1.9.3p392) 2013-05-16 2390d3b on Java HotSpot(TM) 64-Bit Server VM 1.7.0_07-b11 +indy [Windows 7-amd64]
I've got a Sinatra application that adds image files into the database for these to be later accessed from the client API. While I haven't noticed any problems with separate XML files that I am adding to the database, I've noticed the images (JPGs) are corrupted when served back to the client.
Example of the original file before being added to the database (apologies for the size):
And this is the result after being saved to Mongo and served back:
And another, same image but after being processed through a resizer job (worse visual corruption):
I've investigated this by taking the core logic out my application and distilling it down to a basic script that performs the same job.
require "mongoid-grid_fs"
ENV["RACK_ENV"] = "development"
Mongoid.load! "C:/Temp/database/config.yml"
file = File.open "C:/Temp/original.jpg"
# also tested with. no change to result
# file = java.nio.file.FileSystems.default.get_path "C:/Temp/original.jpg"
db_file = Mongoid::GridFs.put file
extracted_db_file = Mongoid::GridFs.get db_file.id
File.open("C:/Temp/jruby.jpg", "wb") { |file| file.write extracted_db_file.data }When run under jruby-1.7.4 this produces the corrupt image seen above. When run under ruby-1.9.3-p392 the image is reproduced perfectly.
I also broke down the put and get logic into separate scripts and run them on a combination of of JRuby vs MRI.
PUT | GET | Image OK?
JRuby | JRuby | No
JRuby | MRI | No
MRI | MRI | Yes
MRI | JRuby | Yes
So the result is that whenever JRuby performs the put operation the image is corrupted.
Taking a look at the binary diff of the two images (using diffnow.com) there are only two differences, hex values of FF have been deleted from the data at two points in the data. I've diffed other images and the same happens to these also.
I note you haven't stated any particular Ruby implementation is supported, I guess as this should Just Work™.
I'm no expert in the guts of JRuby (and I don't expect you to be either), rather I'd like someone more versed than me to take a quick look at the puts implementation and see what might be the issue here. Also this issue is useful to others thinking of using JRuby with mongoid-grid_fs
Mongoid seems to have small breaking changes that affect mongoid-grid_fs, if I have the time I'll make a PR for compatibility. Rails 5 doesn't seem to introduce any breaking changes here, but I may be wrong.
Changes in Mongoid 6: https://jira.mongodb.org/browse/MONGOID-4268
Update:
The only issue currently seems to be here: grid_fs.rb:154
Chunk/file associations are acting improperly, a validation error is raised upon saving chunks (i.e. "File can't be blank"). This can be hackishly fixed by forcing the association with chunk.file = file. I'll have to dig deeper, the issue might be caused by a breaking change but I've been unable to find any cause.
Finish #70
This method has the support of data_uri, how to get a picture of URL?
I can't find any indication in the source code or the Gem of what license this package is released under. Could you document that please?
Thanks.
Hey!
Great library, very useful in my application, easy to use etc!
But the documentation is lacking. I would have loved to see an usage example.
Either "how to use in rails", like so:
def stream
# here Item is my domain model
item = Item.find(params[:id])
grid_fs = Mongoid::GridFs
file = grid_fs.find(filename: item.gridname)
send_data file.data
end
or basically anything that mentions the ".data" method.
BR
I'm using the rmagick gem to split a PDF into images and store the resulting images into GridFS. However, this gem requires me to pass a file path to store a file to GridFS. Is there a way I could pass a blob and store it as a file?
In the earlier mongoid_grid gem, I could use a set_file method to set the contents of a file that I was uploading. I wish I could do the same thing here without having to first write the file to a temporary file.
Thanks a lot!
data_uri method returns the base64 encoded version of something else, but not the blob/binary data.
Instead of connect to the mongoid configured one (usually the used to store model documents).
This can be useful on cluster configurations with different number for files and documents hosts.
I had https://github.com/mongoid kindly donated to the community by the nice folks at MongoDB. The three people who have org level admin access are currently @durran, @estolfo and myself.
I want to invite this project to be moved to the org. I think it makes good sense to group mongoid projects together and have continuous support from MongoDB, the company, over the long term. All admins of this project will continue having admin access of course, but this would reduce the bus factor since currently only @ahoward can add maintainers here.
If you think it's a good idea, please transfer this project to me and I'll move it into the org and please add [email protected], [email protected] and [email protected] to rubygems.
We are currently blocked from upgrading to mongoid 8 as we use the carrierwave-mongoid to handle uploads, which in turn depends on mongoid-grid_fs.
Allowing mongoid 8 in this gem would in turn allow carrierwave-mongoid to allow the use of mongoid 8 as well.
$ RAILS_ENV=production bin/rails s --port 9292
ArgumentError: comparison of Class with Class failed
< at org/jruby/RubyComparable.java:168
descendants at /Users/kwando/.rbenv/versions/jruby-1.7.10/lib/ruby/gems/shared/gems/activesupport-4.0.2/lib/active_support/core_ext/class/subclasses.rb:19
each_object at org/jruby/RubyObjectSpace.java:173
descendants at /Users/kwando/.rbenv/versions/jruby-1.7.10/lib/ruby/gems/shared/gems/activesupport-4.0.2/lib/active_support/core_ext/class/subclasses.rb:18
field at /Users/kwando/.rbenv/versions/jruby-1.7.10/lib/ruby/gems/shared/gems/mongoid-4.0.0.alpha2/lib/mongoid/fields.rb:243
Fields at /Users/kwando/.rbenv/versions/jruby-1.7.10/lib/ruby/gems/shared/gems/mongoid-4.0.0.alpha2/lib/mongoid/fields.rb:26
module_eval at org/jruby/RubyModule.java:2304
append_features at /Users/kwando/.rbenv/versions/jruby-1.7.10/lib/ruby/gems/shared/gems/activesupport-4.0.2/lib/active_support/concern.rb:114
include at org/jruby/RubyModule.java:2085
append_features at /Users/kwando/.rbenv/versions/jruby-1.7.10/lib/ruby/gems/shared/gems/activesupport-4.0.2/lib/active_support/concern.rb:111
each at org/jruby/RubyArray.java:1613
append_features at /Users/kwando/.rbenv/versions/jruby-1.7.10/lib/ruby/gems/shared/gems/activesupport-4.0.2/lib/active_support/concern.rb:111
include at org/jruby/RubyModule.java:2085
append_features at /Users/kwando/.rbenv/versions/jruby-1.7.10/lib/ruby/gems/shared/gems/activesupport-4.0.2/lib/active_support/concern.rb:111
each at org/jruby/RubyArray.java:1613
append_features at /Users/kwando/.rbenv/versions/jruby-1.7.10/lib/ruby/gems/shared/gems/activesupport-4.0.2/lib/active_support/concern.rb:111
include at org/jruby/RubyModule.java:2085
build_file_model_for at /Users/kwando/.rbenv/versions/jruby-1.7.10/lib/ruby/gems/shared/bundler/gems/mongoid-grid_fs-059012a2c2a9/lib/mongoid-grid_fs.rb:296
module_exec at org/jruby/RubyModule.java:2335
initialize at org/jruby/RubyModule.java:1845
initialize at org/jruby/RubyClass.java:849
build_file_model_for at /Users/kwando/.rbenv/versions/jruby-1.7.10/lib/ruby/gems/shared/bundler/gems/mongoid-grid_fs-059012a2c2a9/lib/mongoid-grid_fs.rb:295
build_namespace_for at /Users/kwando/.rbenv/versions/jruby-1.7.10/lib/ruby/gems/shared/bundler/gems/mongoid-grid_fs-059012a2c2a9/lib/mongoid-grid_fs.rb:119
init! at /Users/kwando/.rbenv/versions/jruby-1.7.10/lib/ruby/gems/shared/bundler/gems/mongoid-grid_fs-059012a2c2a9/lib/mongoid-grid_fs.rb:67
Mongoid at /Users/kwando/.rbenv/versions/jruby-1.7.10/lib/ruby/gems/shared/bundler/gems/mongoid-grid_fs-059012a2c2a9/lib/mongoid-grid_fs.rb:580
(root) at /Users/kwando/.rbenv/versions/jruby-1.7.10/lib/ruby/gems/shared/bundler/gems/mongoid-grid_fs-059012a2c2a9/lib/mongoid-grid_fs.rb:59
require at org/jruby/RubyKernel.java:1083
(root) at /Users/kwando/.rbenv/versions/jruby-1.7.10/lib/ruby/gems/shared/gems/bundler-1.5.1/lib/bundler/runtime.rb:1
each at org/jruby/RubyArray.java:1613
require at /Users/kwando/.rbenv/versions/jruby-1.7.10/lib/ruby/gems/shared/gems/bundler-1.5.1/lib/bundler/runtime.rb:76
each at org/jruby/RubyArray.java:1613
require at /Users/kwando/.rbenv/versions/jruby-1.7.10/lib/ruby/gems/shared/gems/bundler-1.5.1/lib/bundler/runtime.rb:72
require at /Users/kwando/.rbenv/versions/jruby-1.7.10/lib/ruby/gems/shared/gems/bundler-1.5.1/lib/bundler/runtime.rb:61
require at /Users/kwando/.rbenv/versions/jruby-1.7.10/lib/ruby/gems/shared/gems/bundler-1.5.1/lib/bundler.rb:131
require at org/jruby/RubyKernel.java:1083
(root) at /Users/kwando/projects/work/abis/config/application.rb:14
tap at org/jruby/RubyKernel.java:1891
(root) at /Users/kwando/.rbenv/versions/jruby-1.7.10/lib/ruby/gems/shared/gems/railties-4.0.2/lib/rails/commands.rb:1
require at org/jruby/RubyKernel.java:1083
(root) at bin/rails:4
Gemfile with only relevant entries
source 'https://rubygems.org'
gem 'rails', '4.0.2'
gem 'mongoid', '4.0.0.alpha2'
gem 'carrierwave'
gem 'mongoid-grid_fs', github: 'ahoward/mongoid-grid_fs'
gem 'carrierwave-mongoid', '~> 0.6.3', :require => 'carrierwave/mongoid'It looks like there was an issue with the gem update, and none of the patches since 1.8 made it into 1.9. Can you push a change? Thanks
file= File.open 'image.jpg'
g = Mongoid::GridFs.put file, filename: "image.jpg"TypeError: no implicit conversion of Array into String
from /home/vagrant/local/ruby/gems/gems/bson-2.0.0.rc2/lib/bson/binary.rb:112:in `block in to_bson'
Hi,
I've checked out this repository and wrote the below simple test.rb, while using the below image:
require 'mongoid-grid_fs'
Mongoid.configure do |config|
config.connect_to('mongoid-grid_fs_test')
end
file = File.open('./signature.png')
grid_file = Mongoid::GridFs.put(file.path)If I execute this with bundle exec ruby test.rb, I'm getting the following exception.
Am I doing something wrong or overlooking something?
❯ bundle exec ruby test.rb
/usr/local/lib/ruby/gems/2.1.0/gems/bson-3.1.2/lib/bson/binary.rb:137:in `encode': "\x89" from ASCII-8BIT to UTF-8 (Encoding::UndefinedConversionError)
from /usr/local/lib/ruby/gems/2.1.0/gems/bson-3.1.2/lib/bson/binary.rb:137:in `block in to_bson'
from /usr/local/lib/ruby/gems/2.1.0/gems/bson-3.1.2/lib/bson/encodable.rb:81:in `encode_binary_data_with_placeholder'
from /usr/local/lib/ruby/gems/2.1.0/gems/bson-3.1.2/lib/bson/binary.rb:134:in `to_bson'
from /usr/local/lib/ruby/gems/2.1.0/gems/bson-3.1.2/lib/bson/hash.rb:46:in `block (2 levels) in to_bson'
from /usr/local/lib/ruby/gems/2.1.0/gems/bson-3.1.2/lib/bson/hash.rb:43:in `each'
from /usr/local/lib/ruby/gems/2.1.0/gems/bson-3.1.2/lib/bson/hash.rb:43:in `block in to_bson'
from /usr/local/lib/ruby/gems/2.1.0/gems/bson-3.1.2/lib/bson/encodable.rb:57:in `encode_with_placeholder_and_null'
from /usr/local/lib/ruby/gems/2.1.0/gems/bson-3.1.2/lib/bson/hash.rb:42:in `to_bson'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/protocol/message.rb:166:in `block in serialize_documents'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/protocol/message.rb:165:in `each'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/protocol/message.rb:165:in `serialize_documents'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/protocol/message.rb:323:in `serialize'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/connection.rb:171:in `block in write'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/connection.rb:169:in `each'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/connection.rb:169:in `write'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/node.rb:590:in `block (2 levels) in flush'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/node.rb:182:in `block in ensure_connected'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/node.rb:115:in `block in connection'
from /usr/local/lib/ruby/gems/2.1.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:64:in `block (2 levels) in with'
from /usr/local/lib/ruby/gems/2.1.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:63:in `handle_interrupt'
from /usr/local/lib/ruby/gems/2.1.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:63:in `block in with'
from /usr/local/lib/ruby/gems/2.1.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:60:in `handle_interrupt'
from /usr/local/lib/ruby/gems/2.1.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:60:in `with'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/node.rb:114:in `connection'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/node.rb:178:in `ensure_connected'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/node.rb:589:in `block in flush'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/node.rb:617:in `block in logging'
from /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.2.3/lib/active_support/notifications.rb:164:in `block in instrument'
from /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.2.3/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
from /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.2.3/lib/active_support/notifications.rb:164:in `instrument'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/instrumentable.rb:31:in `instrument'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/node.rb:616:in `logging'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/node.rb:587:in `flush'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/node.rb:358:in `pipeline'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/operation/write.rb:47:in `execute'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/node.rb:665:in `write'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/node.rb:273:in `insert'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/collection.rb:128:in `block (2 levels) in insert'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/cluster.rb:249:in `block in with_primary'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/node.rb:204:in `block in ensure_primary'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/executable.rb:25:in `execute'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/node.rb:203:in `ensure_primary'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/cluster.rb:248:in `with_primary'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/collection.rb:127:in `block in insert'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/retryable.rb:30:in `call'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/retryable.rb:30:in `with_retry'
from /usr/local/lib/ruby/gems/2.1.0/gems/moped-2.0.6/lib/moped/collection.rb:125:in `insert'
from /usr/local/lib/ruby/gems/2.1.0/gems/mongoid-4.0.2/lib/mongoid/query_cache.rb:117:in `insert_with_clear_cache'
from /usr/local/lib/ruby/gems/2.1.0/gems/mongoid-4.0.2/lib/mongoid/persistable/creatable.rb:79:in `insert_as_root'
from /usr/local/lib/ruby/gems/2.1.0/gems/mongoid-4.0.2/lib/mongoid/persistable/creatable.rb:27:in `block in insert'
from /usr/local/lib/ruby/gems/2.1.0/gems/mongoid-4.0.2/lib/mongoid/persistable/creatable.rb:118:in `block (2 levels) in prepare_insert'
from /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:84:in `run_callbacks'
from /usr/local/lib/ruby/gems/2.1.0/gems/mongoid-4.0.2/lib/mongoid/interceptable.rb:138:in `run_callbacks'
from /usr/local/lib/ruby/gems/2.1.0/gems/mongoid-4.0.2/lib/mongoid/persistable/creatable.rb:117:in `block in prepare_insert'
from /usr/local/lib/ruby/gems/2.1.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:84:in `run_callbacks'
from /usr/local/lib/ruby/gems/2.1.0/gems/mongoid-4.0.2/lib/mongoid/interceptable.rb:138:in `run_callbacks'
from /usr/local/lib/ruby/gems/2.1.0/gems/mongoid-4.0.2/lib/mongoid/persistable/creatable.rb:116:in `prepare_insert'
from /usr/local/lib/ruby/gems/2.1.0/gems/mongoid-4.0.2/lib/mongoid/persistable/creatable.rb:23:in `insert'
from /usr/local/lib/ruby/gems/2.1.0/gems/mongoid-4.0.2/lib/mongoid/persistable/savable.rb:23:in `save'
from /usr/local/lib/ruby/gems/2.1.0/gems/mongoid-4.0.2/lib/mongoid/persistable/savable.rb:44:in `save!'
from /Users/tdegrunt/Development/Personal/mongoid-grid_fs/lib/mongoid/grid_fs.rb:154:in `block (2 levels) in put'
from /Users/tdegrunt/Development/Personal/mongoid-grid_fs/lib/mongoid/grid_fs.rb:479:in `call'
from /Users/tdegrunt/Development/Personal/mongoid-grid_fs/lib/mongoid/grid_fs.rb:479:in `chunking'
from /Users/tdegrunt/Development/Personal/mongoid-grid_fs/lib/mongoid/grid_fs.rb:147:in `block in put'
from /Users/tdegrunt/Development/Personal/mongoid-grid_fs/lib/mongoid/grid_fs.rb:458:in `call'
from /Users/tdegrunt/Development/Personal/mongoid-grid_fs/lib/mongoid/grid_fs.rb:458:in `block in reading'
from /Users/tdegrunt/Development/Personal/mongoid-grid_fs/lib/mongoid/grid_fs.rb:457:in `open'
from /Users/tdegrunt/Development/Personal/mongoid-grid_fs/lib/mongoid/grid_fs.rb:457:in `reading'
from /Users/tdegrunt/Development/Personal/mongoid-grid_fs/lib/mongoid/grid_fs.rb:136:in `put'
from (eval):2:in `put'
from test.rb:8:in `<main>'
Here's the output of my bash locale:
❯ locale
LANG="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL=
The image used (via wikipedia):
For completeness here's the output of bundle list:
~/Development/Personal/mongoid-grid_fs master*
❯ bundle list
Gems included by the bundle:
* actionmailer (4.2.3)
* actionpack (4.2.3)
* actionview (4.2.3)
* activejob (4.2.3)
* activemodel (4.2.3)
* activerecord (4.2.3)
* activesupport (4.2.3)
* arel (6.0.2)
* bson (3.1.2)
* builder (3.2.2)
* bundler (1.10.5)
* coderay (1.1.0)
* connection_pool (2.2.0)
* erubis (2.7.0)
* globalid (0.3.5)
* i18n (0.7.0)
* json (1.8.3)
* loofah (2.0.2)
* mail (2.6.3)
* method_source (0.8.2)
* mime-types (2.6.1)
* mini_portile (0.6.2)
* minitest (5.7.0)
* mongoid (4.0.2)
* mongoid-grid_fs (2.1.0)
* moped (2.0.6)
* nokogiri (1.6.6.2)
* optionable (0.2.0)
* origin (2.1.1)
* pry (0.10.1)
* rack (1.6.4)
* rack-test (0.6.3)
* rails (4.2.3)
* rails-deprecated_sanitizer (1.0.3)
* rails-dom-testing (1.0.6)
* rails-html-sanitizer (1.0.2)
* railties (4.2.3)
* rake (10.4.2)
* slop (3.6.0)
* sprockets (3.2.0)
* sprockets-rails (2.3.2)
* thor (0.19.1)
* thread_safe (0.3.5)
* tzinfo (1.2.2)
Has notably final mongoid 6 support.
looking through your past commits looks like mongoid-grid_fs was overwritten by mongid-sequence?
I got this error:
Problem: Validation of Mongoid::GridFS::Fs::File failed. Summary: The following errors were found: Filename is already taken Resolution: Try persisting the document with valid data or remove the validations.
It seems that filename must be unique when saving file to grid
How to solve it? Please help.
Can you provide a way to bypass the filename validation?
Hi!
Is there a reason to change the default chunksize?
self.defaults.chunkSize = 4 * (mb = 2**20)
Sorry for my question, but I can't figure out how to change it.
For storing several document types I use carrierwave-mongoid and that depends on mongoid-grid_fs.
I need to minimize the consumption of db storage.
Taking the ::GridFS namespace when the gem's name is mongoid-gridfs seems like bad form. It's unconventional. I argue it really should be under the Mongoid namespace (i.e. Mongoid::GridFS)
Support for mongoid 4.0 (yes I know it is still in beta).
Bundler could not find compatible versions for gem "mongoid":
In Gemfile:
mongoid-grid_fs (~> 1.3) ruby depends on
mongoid (~> 3.0.1) ruby
mongoid (4.0.0.beta1)
This is the log
MOPED: 127.0.0.1:27017 COMMAND database=admin command={:ismaster=>1} (0.4761ms)
MOPED: 127.0.0.1:27017 QUERY database=linobank_development collection=users selector={"$query"=>{"_id"=>"50a9237a8b6915091b000001"}, "$orderby"=>{:_id=>1}} flags=[] limit=-1 skip=0 fields=nil (0.5140ms)
MOPED: 127.0.0.1:27017 QUERY database=linobank_development collection=users selector={"_id"=>"50a9237a8b6915091b000001"} flags=[] limit=0 skip=0 fields=nil (0.7141ms)
MOPED: 127.0.0.1:27017 QUERY database=linobank_development collection=users selector={"_id"=>"50a9237a8b6915091b000001"} flags=[] limit=0 skip=0 fields=nil (0.6011ms)
MOPED: 127.0.0.1:27017 QUERY database=linobank_development collection=fs.files selector={"$query"=>{"filename"=>"user/picture/50a9237a8b6915091b000001/9e90eee21e7511e2a03a22000a1fbd56_7.jpg"}, "$orderby"=>{:_id=>1}} flags=[] limit=-1 skip=0 fields=nil (0.6919ms)
MOPED: 127.0.0.1:27017 QUERY database=linobank_development collection=fs.chunks selector={"$query"=>{"files_id"=>"50b19e548b6915f86e000004"}, "$orderby"=>{"n"=>1}} flags=[] limit=0 skip=0 fields=nil (0.5009ms)
MOPED: 127.0.0.1:27017 DELETE database=linobank_development collection=fs.files selector={"_id"=>"50b19e548b6915f86e000004"} flags=[:remove_first] (0.1590ms)
MOPED: 127.0.0.1:27017 INSERT database=linobank_development collection=fs.chunks documents=[{"_id"=>"50b19fe68b6915f86e000005", "n"=>0, "files_id"=>"50b19fe68b6915f86e000006", "data"=>#<Moped::BSON::Binary type=:generic length=7616>}] flags=[] (0.1729ms)
MOPED: 127.0.0.1:27017 QUERY database=linobank_development collection=fs.files selector={"filename"=>"user/picture/50a9237a8b6915091b000001/9e90eee21e7511e2a03a22000a1fbd56_7.jpg"} flags=[] limit=1 skip=0 fields={:_id=>1} (0.4621ms)
MOPED: 127.0.0.1:27017 INSERT database=linobank_development collection=fs.files documents=[{"_id"=>"50b19fe68b6915f86e000006", "contentType"=>"image/jpeg", "length"=>7616, "chunkSize"=>268435456, "uploadDate"=>2012-11-25 00:00:00 UTC, "md5"=>"f4d56a5b2e67cd3288170741b8e3536e", "filename"=>"user/picture/50a9237a8b6915091b000001/9e90eee21e7511e2a03a22000a1fbd56_7.jpg"}] flags=[] (0.1988ms)
MOPED: 127.0.0.1:27017 DELETE database=linobank_development collection=fs.chunks selector={"_id"=>"50b19fe68b6915f86e000005"} flags=[:remove_first] (0.1180ms)
I tried to debug and it seems it came from this line
ensure
chunks.each{|chunk| chunk.destroy rescue nil} if $!
end
The content of $! was EOFError: end of file reached.
The problem only occurs when using carrierwave, it is fine when using pure mongoid-grid_fs.
I file similar issue in carrierwave-mongoid.
carrierwaveuploader/carrierwave-mongoid#73
With rails 3.2.17, carrierwave-mongoid 0.6.3, I see this:
Bundler could not find compatible versions for gem "mime-types":
In Gemfile:
rails (= 3.2.17) ruby depends on
mime-types (~> 1.16) ruby
carrierwave-mongoid (~> 0.6) ruby depends on
mime-types (2.1)
when I try to upgrade from 1.9.0 to 1.9.2
Seems like mongoid-grid_fs, delayed_job and thinking_sphinx are not getting along.
When I restart delayed_job, I get the following error:
Warning: Error loading /home/deploy/.bundler/our_project/ruby/1.9.1/gems/mongoid-grid_fs-1.9.0/lib/mongoid-grid_fs.rb:
uninitialized constant Home
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/activesupport-3.2.11/lib/active_support/inflector/methods.rb:230:in `block in constantize'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/activesupport-3.2.11/lib/active_support/inflector/methods.rb:229:in `each'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/activesupport-3.2.11/lib/active_support/inflector/methods.rb:229:in `constantize'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/activesupport-3.2.11/lib/active_support/core_ext/string/inflections.rb:54:in `constantize'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/thinking-sphinx-2.0.13/lib/thinking_sphinx/context.rb:64:in `block (2 levels) in load_models'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/thinking-sphinx-2.0.13/lib/thinking_sphinx/context.rb:54:in `each'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/thinking-sphinx-2.0.13/lib/thinking_sphinx/context.rb:54:in `block in load_models'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/thinking-sphinx-2.0.13/lib/thinking_sphinx/context.rb:53:in `each'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/thinking-sphinx-2.0.13/lib/thinking_sphinx/context.rb:53:in `load_models'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/thinking-sphinx-2.0.13/lib/thinking_sphinx/context.rb:15:in `prepare'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/thinking-sphinx-2.0.13/lib/thinking_sphinx.rb:81:in `block in context'
<internal:prelude>:10:in `synchronize'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/thinking-sphinx-2.0.13/lib/thinking_sphinx.rb:78:in `context'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/thinking-sphinx-2.0.13/lib/thinking_sphinx/active_record.rb:166:in `define_index'
/srv/www/our_project/releases/20131106143153/app/models/category.rb:12:in `<class:Category>'
/srv/www/our_project/releases/20131106143153/app/models/category.rb:1:in `<top (required)>'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/activesupport-3.2.11/lib/active_support/inflector/methods.rb:230:in `block in constantize'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/activesupport-3.2.11/lib/active_support/inflector/methods.rb:229:in `each'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/activesupport-3.2.11/lib/active_support/inflector/methods.rb:229:in `constantize'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/activesupport-3.2.11/lib/active_support/inflector/methods.rb:260:in `safe_constantize'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/activesupport-3.2.11/lib/active_support/core_ext/string/inflections.rb:66:in `safe_constantize'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/actionpack-3.2.11/lib/action_controller/metal/params_wrapper.rb:152:in `_default_wrap_model'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/actionpack-3.2.11/lib/action_controller/metal/params_wrapper.rb:169:in `_set_wrapper_defaults'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/actionpack-3.2.11/lib/action_controller/metal/params_wrapper.rb:133:in `inherited'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/actionpack-3.2.11/lib/abstract_controller/railties/routes_helpers.rb:7:in `block (2 levels) in with'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/actionpack-3.2.11/lib/action_controller/railties/paths.rb:7:in `block (2 levels) in with'
/srv/www/our_project/releases/20131106143153/app/controllers/categories_controller.rb:1:in `<top (required)>'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/railties-3.2.11/lib/rails/engine.rb:439:in `block (2 levels) in eager_load!'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/railties-3.2.11/lib/rails/engine.rb:438:in `each'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/railties-3.2.11/lib/rails/engine.rb:438:in `block in eager_load!'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/railties-3.2.11/lib/rails/engine.rb:436:in `each'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/railties-3.2.11/lib/rails/engine.rb:436:in `eager_load!'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/railties-3.2.11/lib/rails/application/finisher.rb:53:in `block in <module:Finisher>'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/railties-3.2.11/lib/rails/initializable.rb:30:in `instance_exec'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/railties-3.2.11/lib/rails/initializable.rb:30:in `run'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/railties-3.2.11/lib/rails/initializable.rb:55:in `block in run_initializers'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/railties-3.2.11/lib/rails/initializable.rb:54:in `each'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/railties-3.2.11/lib/rails/initializable.rb:54:in `run_initializers'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/railties-3.2.11/lib/rails/application.rb:136:in `initialize!'
/home/deploy/.bundler/our_project/ruby/1.9.1/gems/railties-3.2.11/lib/rails/railtie/configurable.rb:30:in `method_missing'
/srv/www/our_project/releases/20131106143153/config/environment.rb:5:in `<top (required)>'
/usr/local/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
/usr/local/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
script/delayed_job:17:in `<main>'
I did not experience this issue before I added grid_fs to my Gemfile.
I don't have a class Home by the way and Category's thinking_sphinx block reads:
define_index do # line 12
indexes :name
where "categories.name NOT LIKE 'Sonstige %'"
end
So no idea where "Home" come's from. Also unsure who's to blame here.
I believe this is the case.
I'm using this sweet gem via carrierwave-mongoid and I've run up into an interesting snag; I'd like to store some files in one database, and others in another database, ala this Mongoid syntax:
class Banana
include Mongoid::Document
store_in database: 'fruits'
endI'm looking through the code, and it doesn't seem like this is currently possible. I'd love to be wrong 😄 but I'd also love to try and add this functionality if you agree it might be useful to people. What do you think?
Ran into the following error when attempting to read file of more than 50mb (estimated):
[2013-04-05 15:29:36] ERROR Moped::Errors::QueryFailure: The operation: #<Moped::Protocol::Query
@length=144
@request_id=4
@response_to=0
@op_code=2004
@flags=[:slave_ok]
@full_collection_name="mayday_store_dev.fs.chunks"
@skip=0
@limit=0
@selector={"$query"=>{"files_id"=>"515e7d5fc5fe824fdb000001", "n"=>{"$lt"=>9, "$gte"=>0}}, "$orderby"=>{"n"=>1}}
@fields=nil>
failed with error 10128: "too much data for sort() with no index. add an index or specify a smaller limit"
See https://github.com/mongodb/mongo/blob/master/docs/errors.md
...
Sry, maybe I'm stupid, but how to I change the namespace the files are stored in?
I need attachments.files instead of fs.files for my app.
I have already opened the issue carrierwaveuploader/carrierwave-mongoid#205 where I explain that when deleting a model which has a mounter using the :grid_fs storage engine, the GridFs::File is well destroyed, but the belonging GridFs::Chunk aren't, even do I found the has_many :chunks, ..., dependent: :destroy, ... line in this gem.
I have checked the tests from this gem and I couldn't find one which ensures the chunks are cleaned.
Does anyone had that issue already ?
I tried creating a custom namespace as follows:
image = GridFs.namespace(:image)
f = image.put("photo.png")So f returns a GridFS::Image::File object the way I would expect. Now, when I go back to the mongo shell, I see that there are two new collections: image.chunks and image.files. However the file I uploaded is missing.
Is there anything else that I need to do to get the file to be stored in this custom namespace?
Whilst trying to upgrade a Mongoid 2 based application to Mongoid 3 I noticed that the mongoid-grid_fs gem requires file names of GridFS objects to be unique. The 10gen driver does not enforce this behavior and it's not documented in the GridFS specification either.
Would you consider changing the behavior of your gem to match the spec and (in my opinion) expected behavior?
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.