koanhealth / mongoid-encrypted-fields Goto Github PK
View Code? Open in Web Editor NEWA library for storing encrypted data in Mongo
License: MIT License
A library for storing encrypted data in Mongo
License: MIT License
Thanks for making this gem to support mongoid encryption!
I am having an issue implementing it in my Rails app. I made an initializer file containing:
require 'gibberish/aes'
class GibberishCipher
def initialize(password, salt)
@cipher = Gibberish::AES.new(password)
@salt = salt
end
def encrypt(data)
@cipher.encrypt(data, salt: @salt)
end
def decrypt(data)
@cipher.decrypt(data)
end
end
Mongoid::EncryptedFields.cipher = GibberishCipher.new(
Settings.site_key,
Settings.salt
)
My mongoid model contains:
field :oauth_token, Mongoid::EncryptedString
But when the model is created, I get this error:
NoMethodError - undefined method `keys' for Mongoid::EncryptedString:Class:
mongoid (3.1.4) lib/mongoid/fields/validators/macro.rb:74:in `validate_options'
mongoid (3.1.4) lib/mongoid/fields/validators/macro.rb:35:in `validate'
mongoid (3.1.4) lib/mongoid/fields.rb:243:in `field'
...
Any ideas?
Still investigating but it seems that Mongoid 3.1.7 breaks encrypted fields, specifically when I attempt to save an object, the validation fails, and then I see the encrypted chars in the input.
Rolling back to Mongoid 3.1.6 the issue goes away.
Not an issue per se, but I'm wondering what is the best way to query and/or sort encrypted fields? I have a "Person" object with name, email, and phone all encrypted.
To sort, I'm storing the first 3 letters of the last name as clear text (downcased).
To query, I'm truncating each field into 2, 4, 6-char versions and storing it as encrypted. Then, when I run the query I chop the string into the longest available query length (e.g. a 5-char query string gets truncated to 4 chars) and compare it.
Any thoughts on this? I'm considering extracting this into a gem if there's interest.
Hi, are you planning adding support for mongoid 4.0.0?
Bundler could not find compatible versions for gem "mongoid":
In Gemfile:
mongoid-encrypted-fields (>= 0) ruby depends on
mongoid (~> 3) ruby
mongoid (4.0.0)
aronlilland:backbone_cac_grape$ bundle exec rackup -p 3030
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.1/lib/bundler/runtime.rb:85:in `rescue in block (2 levels) in require': There was an error while trying to load the gem 'mongoid-encrypted-fields'. (Bundler::GemRequireError)
Gem Load Error is: "6.2.0" is not a class/module
Backtrace for gem load error is:
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/mongoid-encrypted-fields-1.3.4/lib/mongoid-encrypted-fields.rb:32:in `<top (required)>'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.1/lib/bundler/runtime.rb:82:in `require'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.1/lib/bundler/runtime.rb:82:in `block (2 levels) in require'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.1/lib/bundler/runtime.rb:77:in `each'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.1/lib/bundler/runtime.rb:77:in `block in require'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.1/lib/bundler/runtime.rb:66:in `each'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.1/lib/bundler/runtime.rb:66:in `require'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.1/lib/bundler.rb:108:in `require'
/Users/aronlilland/Documents/ruby/backbone_cac_grape/config.ru:5:in `block in <main>'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/builder.rb:55:in `instance_eval'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/builder.rb:55:in `initialize'
/Users/aronlilland/Documents/ruby/backbone_cac_grape/config.ru:in `new'
/Users/aronlilland/Documents/ruby/backbone_cac_grape/config.ru:in `<main>'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/builder.rb:49:in `eval'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/builder.rb:49:in `new_from_string'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/builder.rb:40:in `parse_file'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/server.rb:300:in `build_app_and_options_from_config'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/server.rb:209:in `app'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/server.rb:337:in `wrapped_app'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/server.rb:273:in `start'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/server.rb:148:in `start'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/bin/rackup:4:in `<top (required)>'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/bin/rackup:22:in `load'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/bin/rackup:22:in `<main>'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/bin/ruby_executable_hooks:15:in `eval'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/bin/ruby_executable_hooks:15:in `<main>'
Bundler Error Backtrace:
from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.1/lib/bundler/runtime.rb:81:in `block (2 levels) in require'
from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.1/lib/bundler/runtime.rb:77:in `each'
from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.1/lib/bundler/runtime.rb:77:in `block in require'
from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.1/lib/bundler/runtime.rb:66:in `each'
from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.1/lib/bundler/runtime.rb:66:in `require'
from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.1/lib/bundler.rb:108:in `require'
from /Users/aronlilland/Documents/ruby/backbone_cac_grape/config.ru:5:in `block in <main>'
from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/builder.rb:55:in `instance_eval'
from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/builder.rb:55:in `initialize'
from /Users/aronlilland/Documents/ruby/backbone_cac_grape/config.ru:in `new'
from /Users/aronlilland/Documents/ruby/backbone_cac_grape/config.ru:in `<main>'
from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/builder.rb:49:in `eval'
from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/builder.rb:49:in `new_from_string'
from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/builder.rb:40:in `parse_file'
from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/server.rb:300:in `build_app_and_options_from_config'
from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/server.rb:209:in `app'
from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/server.rb:337:in `wrapped_app'
from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/server.rb:273:in `start'
from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/server.rb:148:in `start'
from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/bin/rackup:4:in `<top (required)>'
from /Users/aronlilland/.rvm/gems/ruby-2.3.3/bin/rackup:22:in `load'
from /Users/aronlilland/.rvm/gems/ruby-2.3.3/bin/rackup:22:in `<main>'
from /Users/aronlilland/.rvm/gems/ruby-2.3.3/bin/ruby_executable_hooks:15:in `eval'
from /Users/aronlilland/.rvm/gems/ruby-2.3.3/bin/ruby_executable_hooks:15:in `<main>'
aronlilland:backbone_cac_grape$
source 'http://rubygems.org'
ruby '2.3.3'
gem 'grape'
gem 'grape-entity'
gem 'dotenv'
gem 'rack'
gem 'rack-contrib'
gem 'rack-cors'
gem 'puma'
gem 'sinatra'
gem 'mongoid'
gem "mongoid-encrypted-fields"
gem "sanitize"
gem 'symmetric-encryption'
gem "gibberish"
gem 'postmark'
## Dependencies
require 'csv'
require 'bundler'
Bundler.require :default
Dotenv.load
## Main application files
require './app.rb'
## CORS
require 'rack/cors'
use Rack::Cors do
allow do
origins '*'
resource '*', headers: :any, methods: [:get, :post, :options, :put, :delete]
end
end
## Run the Grape API
run Rack::Cascade.new [BACKBONE::API]
Hello, I wanted to know if the MARKER
used in EncryptedField
is an arbitrary string, and if not,
what's the reasoning behind it?
Thanks!
class Device
include Mongoid::Document
include Mongoid::Timestamps
field :password, type: Mongoid::EncryptedString
end
>> Device.create(password: 'password')
#<Device _id: 5953595480a6430b7e1f2e97, password: "XHgwMmB7fk1lRn59YFx4MDM={\"v\":1,\"adata\":\"\",\"ks\":256,\"ct\":\"wy1gSX8smct/axg8DGduvmNk4rvewKHEOoSUaLbUqOtS4hb3lkIQA2kwMhI=\",\"ts\":96,\"mode\":\"gcm\",\"cipher\":\"aes\",\"iter\":100000,\"iv\":\"FCTAJPnY0N1diC5v\",\"salt\":\"Z0z4eF4MaAc=\"}", username: "XHgwMmB7fk1lRn59YFx4MDM={\"v\":1,\"adata\":\"\",\"ks\":256,\"ct\":\"ztx/VG6GLw7QBXfFWaB+5UTIjFKsuEnMXSWel0z/a/ZDb6a20q/ZMH8qcJI=\",\"ts\":96,\"mode\":\"gcm\",\"cipher\":\"aes\",\"iter\":100000,\"iv\":\"ZBuY922Mdlfdso3t\",\"salt\":\"vzX2nF9JokQ=\"}">
And now
>> Device.where(password: 'password').count
0
But
>> Device.where(password: "XHgwMmB7fk1lRn59YFx4MDM={\"v\":1,\"adata\":\"\",\"ks\":256,\"ct\":\"wy1gSX8smct/axg8DGduvmNk4rvewKHEOoSUaLbUqOtS4hb3lkIQA2kwMhI=\",\"ts\":96,\"mode\":\"gcm\",\"cipher\":\"aes\",\"iter\":100000,\"iv\":\"FCTAJPnY0N1diC5v\",\"salt\":\"Z0z4eF4MaAc=\"}").count
1
Running Rails 4.1.7, Mongoid 4.0.0, and the latest mongoid-encrypted-fields.
I get:
DEPRECATION WARNING: The `Validator#setup` instance method is deprecated and will be removed on Rails 4.2. Do your setup in the constructor instead:
class MyValidator < ActiveModel::Validator
def initialize(options={})
super
options[:class].send :attr_accessor, :custom_attribute
end
end
. (called from initialize at /Users/jean/.rvm/gems/ruby-2.0.0-p247/bundler/gems/mongoid-encrypted-fields-87ad45cc36b0/lib/mongoid-encrypted-fields/mongoid4/validatable/uniqueness.rb:15)
It's not obvious to me how to refactor the setup
method to fit into what Rails is suggesting, because I'm worried about the klass
parameter being passed into the method.
The EncryptedHash is not a proper ducktype for the Mongoid Hash.
After retrieving a Hash from mongoid, all keys will be strings, even if the Hash is initially stored with symbol keys. However after retrieving a EncryptedHash type object from Mongoid the keys may still be symbols, as the YAML library can serialize and deserialize symbols.
To act as a proper ducktype replacement for the Mongoid Hash, the EncryptedHash should probably stringify the keys before serializing the hash and encrypting it.
Hey,
Thanks for the great gem guys.
I recently had to add encryption support to an existing database, which is kind of a pain if you're not familiar with all the mongoid methods and the mongoid-encrypted-fields source. I imagine its something that others have had to figure out before, and I wrote a quick guide on how to add encryption to an existing database.
You guys might want to add something about migrating an existing database in your wiki so that some other developer wont have to figure it all out for themselves.
http://blog.thesparktree.com/post/69538763994/transparently-adding-encrypted-fields-to-a-rails-app
A user asked about this enhancement in a comment on this blog post:
http://jerryclinesmith.me/blog/2013/03/29/storing-encrypted-data-in-mongodb/
This is driving me nuts.
I have a RoR app that was working on my local machine. The app works, except when I try to save an encrypted attribute, I get an error. This is working fine on Heroku with the exact same Gemfile and codebase.
`
class Testuser
include Mongoid::Document
field :email, type: String
field :encrypted_field, type: Mongoid::EncryptedString
end
`
[1] pry(main)> blah = Testuser.new => #<Testuser _id: 57046a3849c65a4bacc65561, email: nil, encrypted_field: nil> [2] pry(main)> blah.encrypted_field = "some stuff" TypeError: no implicit conversion of nil into String from /Users/edahl/.rvm/gems/ruby-2.3.0@deskulator/gems/gibberish-2.0.0/lib/gibberish/aes.rb:140:in
pbkdf2_hmac'`
Any idea whats happening?
Ruby 2.3.0
Rails 4.2.6
Hi
i m following http://jerryclinesmith.me/blog/2013/03/29/storing-encrypted-data-in-mongodb/
when i try to update ssn
person = Person.update_attributes(ssn: '123456789')
It throw following error
TypeError: no implicit conversion of Hash into String
from /home/test/.rvm/gems/ruby-2.1.2@nut/gems/gibberish-2.0.0/lib/gibberish/aes.rb:146:in `auth_data='
Rails 4.0.2, gem 'mongoid', github: 'mongoid/mongoid'
This monkey patch is causing an error on rails start due to mongoid removing the method: mongoid/mongoid@e9afb1e
/var/lib/gems/2.0.0/gems/mongoid-encrypted-fields-1.3.0/lib/mongoid-encrypted-fields/mongoid4/validations/uniqueness.rb:20:in alias_method': undefined methodsetup' for class Mongoid::Validatable::UniquenessValidator' (NameError)
from /var/lib/gems/2.0.0/gems/mongoid-encrypted-fields-1.3.0/lib/mongoid-encrypted-fields/mongoid4/validations/uniqueness.rb:20:inclass:UniquenessValidator'
from /var/lib/gems/2.0.0/gems/mongoid-encrypted-fields-1.3.0/lib/mongoid-encrypted-fields/mongoid4/validations/uniqueness.rb:9:in module:Validatable'
from /var/lib/gems/2.0.0/gems/mongoid-encrypted-fields-1.3.0/lib/mongoid-encrypted-fields/mongoid4/validations/uniqueness.rb:4:inmodule:Mongoid'
from /var/lib/gems/2.0.0/gems/mongoid-encrypted-fields-1.3.0/lib/mongoid-encrypted-fields/mongoid4/validations/uniqueness.rb:3:in <top (required)>'
from /var/lib/gems/2.0.0/gems/activesupport-4.0.2/lib/active_support/dependencies.rb:229:inrequire'
from /var/lib/gems/2.0.0/gems/activesupport-4.0.2/lib/active_support/dependencies.rb:229:in block in require'
from /var/lib/gems/2.0.0/gems/activesupport-4.0.2/lib/active_support/dependencies.rb:214:inload_dependency'
from /var/lib/gems/2.0.0/gems/activesupport-4.0.2/lib/active_support/dependencies.rb:229:in require'
from /var/lib/gems/2.0.0/gems/mongoid-encrypted-fields-1.3.0/lib/mongoid-encrypted-fields.rb:30:in'
from /var/lib/gems/2.0.0/gems/bundler-1.3.5/lib/bundler/runtime.rb:72:in require'
from /var/lib/gems/2.0.0/gems/bundler-1.3.5/lib/bundler/runtime.rb:72:inblock (2 levels) in require'
from /var/lib/gems/2.0.0/gems/bundler-1.3.5/lib/bundler/runtime.rb:70:in each'
from /var/lib/gems/2.0.0/gems/bundler-1.3.5/lib/bundler/runtime.rb:70:inblock in require'
from /var/lib/gems/2.0.0/gems/bundler-1.3.5/lib/bundler/runtime.rb:59:in each'
from /var/lib/gems/2.0.0/gems/bundler-1.3.5/lib/bundler/runtime.rb:59:inrequire'
from /var/lib/gems/2.0.0/gems/bundler-1.3.5/lib/bundler.rb:132:in require'
from /media/a/data/Dropbox/1234cast/cyclonecrm/config/application.rb:13:in'
from /var/lib/gems/2.0.0/gems/railties-4.0.2/lib/rails/commands.rb:74:in require'
from /var/lib/gems/2.0.0/gems/railties-4.0.2/lib/rails/commands.rb:74:inblock in '
from /var/lib/gems/2.0.0/gems/railties-4.0.2/lib/rails/commands.rb:71:in tap'
from /var/lib/gems/2.0.0/gems/railties-4.0.2/lib/rails/commands.rb:71:in'
from bin/rails:4:in require'
from bin/rails:4:in'
Hi!
First of all thanks for such useful gem!
I have one question about passing additional params to cipher being used, in particular - symmetric-encryption gem.
I set up it like
Mongoid::EncryptedFields.cipher = SymmetricEncryption
In that gem I can set up properties like
field :encrypted_life_history,type: String, encrypted: {random_iv: true}
But it is not possible to do so with mongoid-encrypted-fields.
Right now I just added few lines in my fork
module EncryptedFields
class << self
[...]
# Used with symmetric-encryption cipher
attr_accessor :random_iv
[...]
end
end
And using it like
def encrypt(plaintext)
cipher = EncryptedFields.cipher
if EncryptedFields.random_iv
encrypted = cipher.encrypt(plaintext, EncryptedFields.random_iv)
else
encrypted = cipher.encrypt(plaintext)
end
MARKER + encrypted.chomp
end
I don't think this is right or elegant solution of the problem. Do you have any idea or guidance how it can be implemented?
Thanks!
I run a few project in ruby that need encrypted data store.
This gem is useful, happy to take the development and support forward if its not actively being worked on.
Sandeep
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.