koanhealth / mongoid-encrypted-fields Goto Github PK

Thanks for making this gem to support mongoid encryption!

I am having an issue implementing it in my Rails app. I made an initializer file containing:

require 'gibberish/aes'

class GibberishCipher
  def initialize(password, salt)
    @cipher = Gibberish::AES.new(password)
    @salt = salt
  end
  def encrypt(data)
    @cipher.encrypt(data, salt: @salt)
  end
  def decrypt(data)
    @cipher.decrypt(data)
  end
end

Mongoid::EncryptedFields.cipher = GibberishCipher.new(
    Settings.site_key,
    Settings.salt
)

My mongoid model contains:

field :oauth_token, Mongoid::EncryptedString

But when the model is created, I get this error:

NoMethodError - undefined method `keys' for Mongoid::EncryptedString:Class:
  mongoid (3.1.4) lib/mongoid/fields/validators/macro.rb:74:in `validate_options'
  mongoid (3.1.4) lib/mongoid/fields/validators/macro.rb:35:in `validate'
  mongoid (3.1.4) lib/mongoid/fields.rb:243:in `field'
  ...

Any ideas?

Still investigating but it seems that Mongoid 3.1.7 breaks encrypted fields, specifically when I attempt to save an object, the validation fails, and then I see the encrypted chars in the input.

Rolling back to Mongoid 3.1.6 the issue goes away.

Not an issue per se, but I'm wondering what is the best way to query and/or sort encrypted fields? I have a "Person" object with name, email, and phone all encrypted.

To sort, I'm storing the first 3 letters of the last name as clear text (downcased).

To query, I'm truncating each field into 2, 4, 6-char versions and storing it as encrypted. Then, when I run the query I chop the string into the longest available query length (e.g. a 5-char query string gets truncated to 4 chars) and compare it.

Any thoughts on this? I'm considering extracting this into a gem if there's interest.

Hi, are you planning adding support for mongoid 4.0.0?

Bundler could not find compatible versions for gem "mongoid":
In Gemfile:
mongoid-encrypted-fields (>= 0) ruby depends on
mongoid (~> 3) ruby

mongoid (4.0.0)

Errors when launching app

aronlilland:backbone_cac_grape$ bundle exec rackup -p 3030
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.1/lib/bundler/runtime.rb:85:in `rescue in block (2 levels) in require': There was an error while trying to load the gem 'mongoid-encrypted-fields'. (Bundler::GemRequireError)
Gem Load Error is: "6.2.0" is not a class/module
Backtrace for gem load error is:
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/mongoid-encrypted-fields-1.3.4/lib/mongoid-encrypted-fields.rb:32:in `<top (required)>'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.1/lib/bundler/runtime.rb:82:in `require'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.1/lib/bundler/runtime.rb:82:in `block (2 levels) in require'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.1/lib/bundler/runtime.rb:77:in `each'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.1/lib/bundler/runtime.rb:77:in `block in require'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.1/lib/bundler/runtime.rb:66:in `each'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.1/lib/bundler/runtime.rb:66:in `require'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.1/lib/bundler.rb:108:in `require'
/Users/aronlilland/Documents/ruby/backbone_cac_grape/config.ru:5:in `block in <main>'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/builder.rb:55:in `instance_eval'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/builder.rb:55:in `initialize'
/Users/aronlilland/Documents/ruby/backbone_cac_grape/config.ru:in `new'
/Users/aronlilland/Documents/ruby/backbone_cac_grape/config.ru:in `<main>'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/builder.rb:49:in `eval'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/builder.rb:49:in `new_from_string'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/builder.rb:40:in `parse_file'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/server.rb:300:in `build_app_and_options_from_config'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/server.rb:209:in `app'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/server.rb:337:in `wrapped_app'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/server.rb:273:in `start'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/server.rb:148:in `start'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/bin/rackup:4:in `<top (required)>'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/bin/rackup:22:in `load'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/bin/rackup:22:in `<main>'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/bin/ruby_executable_hooks:15:in `eval'
/Users/aronlilland/.rvm/gems/ruby-2.3.3/bin/ruby_executable_hooks:15:in `<main>'
Bundler Error Backtrace:
	from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.1/lib/bundler/runtime.rb:81:in `block (2 levels) in require'
	from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.1/lib/bundler/runtime.rb:77:in `each'
	from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.1/lib/bundler/runtime.rb:77:in `block in require'
	from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.1/lib/bundler/runtime.rb:66:in `each'
	from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.1/lib/bundler/runtime.rb:66:in `require'
	from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/bundler-1.15.1/lib/bundler.rb:108:in `require'
	from /Users/aronlilland/Documents/ruby/backbone_cac_grape/config.ru:5:in `block in <main>'
	from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/builder.rb:55:in `instance_eval'
	from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/builder.rb:55:in `initialize'
	from /Users/aronlilland/Documents/ruby/backbone_cac_grape/config.ru:in `new'
	from /Users/aronlilland/Documents/ruby/backbone_cac_grape/config.ru:in `<main>'
	from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/builder.rb:49:in `eval'
	from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/builder.rb:49:in `new_from_string'
	from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/builder.rb:40:in `parse_file'
	from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/server.rb:300:in `build_app_and_options_from_config'
	from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/server.rb:209:in `app'
	from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/server.rb:337:in `wrapped_app'
	from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/server.rb:273:in `start'
	from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/lib/rack/server.rb:148:in `start'
	from /Users/aronlilland/.rvm/gems/ruby-2.3.3/gems/rack-1.6.8/bin/rackup:4:in `<top (required)>'
	from /Users/aronlilland/.rvm/gems/ruby-2.3.3/bin/rackup:22:in `load'
	from /Users/aronlilland/.rvm/gems/ruby-2.3.3/bin/rackup:22:in `<main>'
	from /Users/aronlilland/.rvm/gems/ruby-2.3.3/bin/ruby_executable_hooks:15:in `eval'
	from /Users/aronlilland/.rvm/gems/ruby-2.3.3/bin/ruby_executable_hooks:15:in `<main>'
aronlilland:backbone_cac_grape$ 

Gemfile

source 'http://rubygems.org'

ruby '2.3.3'

gem 'grape'
gem 'grape-entity'

gem 'dotenv'

gem 'rack'
gem 'rack-contrib'
gem 'rack-cors'
gem 'puma'

gem 'sinatra'

gem 'mongoid'
gem "mongoid-encrypted-fields"
gem "sanitize"

gem 'symmetric-encryption'
gem "gibberish"
gem 'postmark'

config.ru

## Dependencies
require 'csv'
require 'bundler'

Bundler.require :default
Dotenv.load

## Main application files
require './app.rb'

## CORS
require 'rack/cors'
use Rack::Cors do
	allow do
		origins '*'
		resource '*', headers: :any, methods: [:get, :post, :options, :put, :delete]
	end
end

## Run the Grape API
run Rack::Cascade.new [BACKBONE::API]

Hello, I wanted to know if the MARKER used in EncryptedField is an arbitrary string, and if not,
what's the reasoning behind it?

Thanks!

class Device
  include Mongoid::Document
  include Mongoid::Timestamps

  field :password,  type: Mongoid::EncryptedString
end

>> Device.create(password: 'password')
#<Device _id: 5953595480a6430b7e1f2e97, password: "XHgwMmB7fk1lRn59YFx4MDM={\"v\":1,\"adata\":\"\",\"ks\":256,\"ct\":\"wy1gSX8smct/axg8DGduvmNk4rvewKHEOoSUaLbUqOtS4hb3lkIQA2kwMhI=\",\"ts\":96,\"mode\":\"gcm\",\"cipher\":\"aes\",\"iter\":100000,\"iv\":\"FCTAJPnY0N1diC5v\",\"salt\":\"Z0z4eF4MaAc=\"}", username: "XHgwMmB7fk1lRn59YFx4MDM={\"v\":1,\"adata\":\"\",\"ks\":256,\"ct\":\"ztx/VG6GLw7QBXfFWaB+5UTIjFKsuEnMXSWel0z/a/ZDb6a20q/ZMH8qcJI=\",\"ts\":96,\"mode\":\"gcm\",\"cipher\":\"aes\",\"iter\":100000,\"iv\":\"ZBuY922Mdlfdso3t\",\"salt\":\"vzX2nF9JokQ=\"}">

And now

>> Device.where(password: 'password').count
0

But

>> Device.where(password: "XHgwMmB7fk1lRn59YFx4MDM={\"v\":1,\"adata\":\"\",\"ks\":256,\"ct\":\"wy1gSX8smct/axg8DGduvmNk4rvewKHEOoSUaLbUqOtS4hb3lkIQA2kwMhI=\",\"ts\":96,\"mode\":\"gcm\",\"cipher\":\"aes\",\"iter\":100000,\"iv\":\"FCTAJPnY0N1diC5v\",\"salt\":\"Z0z4eF4MaAc=\"}").count
1

Running Rails 4.1.7, Mongoid 4.0.0, and the latest mongoid-encrypted-fields.

I get:

DEPRECATION WARNING: The `Validator#setup` instance method is deprecated and will be removed on Rails 4.2. Do your setup in the constructor instead:

class MyValidator < ActiveModel::Validator
  def initialize(options={})
    super
    options[:class].send :attr_accessor, :custom_attribute
  end
end
. (called from initialize at /Users/jean/.rvm/gems/ruby-2.0.0-p247/bundler/gems/mongoid-encrypted-fields-87ad45cc36b0/lib/mongoid-encrypted-fields/mongoid4/validatable/uniqueness.rb:15)

It's not obvious to me how to refactor the setup method to fit into what Rails is suggesting, because I'm worried about the klass parameter being passed into the method.

The EncryptedHash is not a proper ducktype for the Mongoid Hash.

After retrieving a Hash from mongoid, all keys will be strings, even if the Hash is initially stored with symbol keys. However after retrieving a EncryptedHash type object from Mongoid the keys may still be symbols, as the YAML library can serialize and deserialize symbols.

To act as a proper ducktype replacement for the Mongoid Hash, the EncryptedHash should probably stringify the keys before serializing the hash and encrypting it.

Hey,
Thanks for the great gem guys.
I recently had to add encryption support to an existing database, which is kind of a pain if you're not familiar with all the mongoid methods and the mongoid-encrypted-fields source. I imagine its something that others have had to figure out before, and I wrote a quick guide on how to add encryption to an existing database.

You guys might want to add something about migrating an existing database in your wiki so that some other developer wont have to figure it all out for themselves.

http://blog.thesparktree.com/post/69538763994/transparently-adding-encrypted-fields-to-a-rails-app

A user asked about this enhancement in a comment on this blog post:
http://jerryclinesmith.me/blog/2013/03/29/storing-encrypted-data-in-mongodb/

This is driving me nuts.

I have a RoR app that was working on my local machine. The app works, except when I try to save an encrypted attribute, I get an error. This is working fine on Heroku with the exact same Gemfile and codebase.

`
class Testuser

include Mongoid::Document
field :email, type: String
field :encrypted_field, type: Mongoid::EncryptedString

end
`

[1] pry(main)> blah = Testuser.new => #<Testuser _id: 57046a3849c65a4bacc65561, email: nil, encrypted_field: nil> [2] pry(main)> blah.encrypted_field = "some stuff" TypeError: no implicit conversion of nil into String from /Users/edahl/.rvm/gems/ruby-2.3.0@deskulator/gems/gibberish-2.0.0/lib/gibberish/aes.rb:140:inpbkdf2_hmac'`

Any idea whats happening?

Ruby 2.3.0
Rails 4.2.6

Hi
i m following http://jerryclinesmith.me/blog/2013/03/29/storing-encrypted-data-in-mongodb/

when i try to update ssn
person = Person.update_attributes(ssn: '123456789')

It throw following error
TypeError: no implicit conversion of Hash into String
from /home/test/.rvm/gems/ruby-2.1.2@nut/gems/gibberish-2.0.0/lib/gibberish/aes.rb:146:in `auth_data='

Rails 4.0.2, gem 'mongoid', github: 'mongoid/mongoid'

This monkey patch is causing an error on rails start due to mongoid removing the method: mongoid/mongoid@e9afb1e

/var/lib/gems/2.0.0/gems/mongoid-encrypted-fields-1.3.0/lib/mongoid-encrypted-fields/mongoid4/validations/uniqueness.rb:20:in alias_method': undefined methodsetup' for class Mongoid::Validatable::UniquenessValidator' (NameError)
from /var/lib/gems/2.0.0/gems/mongoid-encrypted-fields-1.3.0/lib/mongoid-encrypted-fields/mongoid4/validations/uniqueness.rb:20:inclass:UniquenessValidator'
from /var/lib/gems/2.0.0/gems/mongoid-encrypted-fields-1.3.0/lib/mongoid-encrypted-fields/mongoid4/validations/uniqueness.rb:9:in module:Validatable'
from /var/lib/gems/2.0.0/gems/mongoid-encrypted-fields-1.3.0/lib/mongoid-encrypted-fields/mongoid4/validations/uniqueness.rb:4:inmodule:Mongoid'
from /var/lib/gems/2.0.0/gems/mongoid-encrypted-fields-1.3.0/lib/mongoid-encrypted-fields/mongoid4/validations/uniqueness.rb:3:in <top (required)>'
from /var/lib/gems/2.0.0/gems/activesupport-4.0.2/lib/active_support/dependencies.rb:229:inrequire'
from /var/lib/gems/2.0.0/gems/activesupport-4.0.2/lib/active_support/dependencies.rb:229:in block in require'
from /var/lib/gems/2.0.0/gems/activesupport-4.0.2/lib/active_support/dependencies.rb:214:inload_dependency'
from /var/lib/gems/2.0.0/gems/activesupport-4.0.2/lib/active_support/dependencies.rb:229:in require'
from /var/lib/gems/2.0.0/gems/mongoid-encrypted-fields-1.3.0/lib/mongoid-encrypted-fields.rb:30:in'
from /var/lib/gems/2.0.0/gems/bundler-1.3.5/lib/bundler/runtime.rb:72:in require'
from /var/lib/gems/2.0.0/gems/bundler-1.3.5/lib/bundler/runtime.rb:72:inblock (2 levels) in require'
from /var/lib/gems/2.0.0/gems/bundler-1.3.5/lib/bundler/runtime.rb:70:in each'
from /var/lib/gems/2.0.0/gems/bundler-1.3.5/lib/bundler/runtime.rb:70:inblock in require'
from /var/lib/gems/2.0.0/gems/bundler-1.3.5/lib/bundler/runtime.rb:59:in each'
from /var/lib/gems/2.0.0/gems/bundler-1.3.5/lib/bundler/runtime.rb:59:inrequire'
from /var/lib/gems/2.0.0/gems/bundler-1.3.5/lib/bundler.rb:132:in require'
from /media/a/data/Dropbox/1234cast/cyclonecrm/config/application.rb:13:in'
from /var/lib/gems/2.0.0/gems/railties-4.0.2/lib/rails/commands.rb:74:in require'
from /var/lib/gems/2.0.0/gems/railties-4.0.2/lib/rails/commands.rb:74:inblock in '
from /var/lib/gems/2.0.0/gems/railties-4.0.2/lib/rails/commands.rb:71:in tap'
from /var/lib/gems/2.0.0/gems/railties-4.0.2/lib/rails/commands.rb:71:in'
from bin/rails:4:in require'
from bin/rails:4:in'

Hi!

First of all thanks for such useful gem!
I have one question about passing additional params to cipher being used, in particular - symmetric-encryption gem.

I set up it like

Mongoid::EncryptedFields.cipher = SymmetricEncryption

In that gem I can set up properties like

field :encrypted_life_history,type: String,  encrypted: {random_iv: true}

But it is not possible to do so with mongoid-encrypted-fields.

Right now I just added few lines in my fork

module EncryptedFields
  class << self
    [...]
    # Used with symmetric-encryption cipher
    attr_accessor :random_iv
    [...]
  end
end

And using it like

def encrypt(plaintext)
  cipher = EncryptedFields.cipher
  if EncryptedFields.random_iv
    encrypted = cipher.encrypt(plaintext, EncryptedFields.random_iv)
  else
    encrypted = cipher.encrypt(plaintext)
  end
  MARKER + encrypted.chomp
end

I don't think this is right or elegant solution of the problem. Do you have any idea or guidance how it can be implemented?

Thanks!

I run a few project in ruby that need encrypted data store.

This gem is useful, happy to take the development and support forward if its not actively being worked on.

Sandeep