Dynamoid is an ORM for Amazon's DynamoDB for Ruby applications.
Dynamoid's home moved to http://github.com/Dynamoid/Dynamoid.
Ruby ORM for Amazon's DynamoDB
Home Page: http://joshsymonds.com/Dynamoid/
Dynamoid is an ORM for Amazon's DynamoDB for Ruby applications.
Dynamoid's home moved to http://github.com/Dynamoid/Dynamoid.
Dynamite? Where does it come from ? How is this possible ? I've basically only recreated the sample in the manual..
:001 > User.find(:all)
NameError: uninitialized constant Dynamoid::Adapter::AwsSdk::Dynamite
from /Users/kjetil/.rvm/gems/ruby-1.9.2-p290@rails31-zen/gems/dynamoid-0.4.1/lib/dynamoid/adapter/aws_sdk.rb:66:in create_table' from /Users/kjetil/.rvm/gems/ruby-1.9.2-p290@rails31-zen/gems/dynamoid-0.4.1/lib/dynamoid/adapter.rb:126:in
block (3 levels) in module:Adapter'
from /Users/kjetil/.rvm/gems/ruby-1.9.2-p290@rails31-zen/gems/dynamoid-0.4.1/lib/dynamoid/adapter.rb:39:in benchmark' from /Users/kjetil/.rvm/gems/ruby-1.9.2-p290@rails31-zen/gems/dynamoid-0.4.1/lib/dynamoid/adapter.rb:126:in
block (2 levels) in module:Adapter'
from /Users/kjetil/.rvm/gems/ruby-1.9.2-p290@rails31-zen/gems/dynamoid-0.4.1/lib/dynamoid/persistence.rb:50:in create_table' from /Users/kjetil/.rvm/gems/ruby-1.9.2-p290@rails31-zen/gems/dynamoid-0.4.1/lib/dynamoid/indexes.rb:45:in
block in create_indexes'
from /Users/kjetil/.rvm/gems/ruby-1.9.2-p290@rails31-zen/gems/dynamoid-0.4.1/lib/dynamoid/indexes.rb:42:in each' from /Users/kjetil/.rvm/gems/ruby-1.9.2-p290@rails31-zen/gems/dynamoid-0.4.1/lib/dynamoid/indexes.rb:42:in
create_indexes'
from /Users/kjetil/.rvm/gems/ruby-1.9.2-p290@rails31-zen/gems/dynamoid-0.4.1/lib/dynamoid/indexes.rb:28:in index' from /Users/kjetil/Dropbox/Development/railsprojects/DynTest/app/models/user.rb:11:in
class:User'
from /Users/kjetil/Dropbox/Development/railsprojects/DynTest/app/models/user.rb:1:in <top (required)>' from (irb):1 from /Users/kjetil/.rvm/gems/ruby-1.9.2-p290@rails31-zen/gems/railties-3.1.0/lib/rails/commands/console.rb:45:in
start'
from /Users/kjetil/.rvm/gems/ruby-1.9.2-p290@rails31-zen/gems/railties-3.1.0/lib/rails/commands/console.rb:8:in start' from /Users/kjetil/.rvm/gems/ruby-1.9.2-p290@rails31-zen/gems/railties-3.1.0/lib/rails/commands.rb:40:in
<top (required)>'
from script/rails:6:in require' from script/rails:6:in
/usr/lib/ruby/gems/1.8/gems/dynamoid-0.4.1/lib/dynamoid/adapter/aws_sdk.rb:66:in `create_table': uninitialized constant Dynamoid::Adapter::AwsSdk::Dynamite (NameError)
I am getting the above error when trying to use the gem. Anyone know why it is happening?
This is test result of my application .
Amazon DynamoDB does not allow ":" character in table name.
Dynamoid is trying to create a "rspec_index_pastehub::entry_usernames" table.
$ rake test_e
time ruby -I ./lib `which rspec` -b -t entries ./test/aws_spec.rb -r ./test/rspec_formatter_for_emacs.rb -f CustomFormatter
Amazon DynamoDB access_key_id: XXXXXXXXXXXXXXXXXXXX
Amazon DynamoDB secret_access_key: YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
/usr/local/stow/ruby-1.7.0.preview1-jruby/lib/ruby/gems/shared/gems/json-1.6.5-java/lib/json/version.rb:3 warning: already initialized constant version.rb:3
/usr/local/stow/ruby-1.7.0.preview1-jruby/lib/ruby/gems/shared/gems/json-1.6.5-java/lib/json/version.rb:4 warning: already initialized constant VERSION_ARRAY
/usr/local/stow/ruby-1.7.0.preview1-jruby/lib/ruby/gems/shared/gems/json-1.6.5-java/lib/json/version.rb:5 warning: already initialized constant VERSION_MAJOR
/usr/local/stow/ruby-1.7.0.preview1-jruby/lib/ruby/gems/shared/gems/json-1.6.5-java/lib/json/version.rb:6 warning: already initialized constant VERSION_MINOR
/usr/local/stow/ruby-1.7.0.preview1-jruby/lib/ruby/gems/shared/gems/json-1.6.5-java/lib/json/version.rb:7 warning: already initialized constant VERSION_BUILD
/usr/local/stow/ruby-1.7.0.preview1-jruby/lib/ruby/gems/shared/gems/json-1.6.5-java/lib/json/common.rb:98 warning: already initialized constant NaN
/usr/local/stow/ruby-1.7.0.preview1-jruby/lib/ruby/gems/shared/gems/json-1.6.5-java/lib/json/common.rb:100 warning: already initialized constant Infinity
/usr/local/stow/ruby-1.7.0.preview1-jruby/lib/ruby/gems/shared/gems/json-1.6.5-java/lib/json/common.rb:102 warning: already initialized constant MinusInfinity
/usr/local/stow/ruby-1.7.0.preview1-jruby/lib/ruby/gems/shared/gems/json-1.6.5-java/lib/json/common.rb:121 warning: already initialized constant UnparserError
(5472.0 ms) LIST TABLES - [[]]
(5485.0 ms) CACHE TABLESCreating rspec_index_pastehub::entry_usernames table. This could take a while.
#<Class:0x28c61629>: 1 validation error detected: Value 'rspec_index_pastehub::entry_usernames' at 'tableName' failed to satisfy constraint: Member must satisfy regular expression pattern: [a-zA-Z0-9_.-]+
How do I pass query options in a find_all_by_* query? Seems they are not passing through to the underlying query
Comment.find_all_by_user_id_and_blog_id "1234", "1", limit: 1, scan_index_forward: true
Comment.find_all_by_user_id "1234", limit: 1, scan_index_forward: true
If I use query directly, it works as expected:
query_options {hash_value: "12345", scan_index_forward: true, limit: 1}
hash_attrs = Dynamoid::Adapter.adapter.query Comment.table_name, query_options
hash_attrs.map{|attrs| self.from_database attrs}
Is it possible to create a table with a range key? Specifically, I'm looking to make an index on created_at. If I specify it as an index, then it creates a second table which is less than ideal. Shouldn't table and field take the :range option?
I think a set should be mapped differently as an index. If we have an object Book and we want to add Tags (as a set ["SF", "Drama"]) to it and it as index, then in the index table there should be created 2 items, one for each entry in the tags set. The first would have all the books with the "SF" tag and the other one all the books that have the "Drama" tag.
I have two models (ShoppingCart & Item) where ShoppingCart has_many :items and Item belongs_to :shopping_cart. Without validations on the Item everything works beautifully. But as soon as I add a validates_presence_of :quantity on the Item model it gives me the following error:
ArgumentError (unsupported attribute type NilClass in value for attribute items_ids):
app/controllers/item_controller.rb:17:in `create'
Removing the validates_presence_of makes it work correctly again.
I'm trying to work with a table with four fields, similar to the example below:
class Logging
include Dynamoid::Document
table :name => :api_calls
range :stamp, :integer
field :appId, :integer
field :function
field :stamp, :integer
field :version, :integer
index :appId
When I use the table without the index directive, everything works ok (but running a query against appId uses scanning, even if I create the table with the index in AWS console beforehand).
When running it with the index directive, Dynamoid creates a table called index_api_calls_appIds and the insert fails with ArgumentError: a range key value is required for this table.
What am I doing wrong? I'm running Rails 3.2.13 and aws-sdk 1.11.1 (same problem with 1.8.6) and the latest official rubygem.
Lets assume that we have a table with the following structure.
class Website
include Dynamoid::Document
range :page, :integer
field :subdomain, :string
index :subdomain
end
If we do Website.find_by_subdomain('test') we will get the following error: "ArgumentError: a range key value is required for this table". This happens because Dynamoid uses table.items.at
(https://github.com/Veraticus/Dynamoid/blob/master/lib/dynamoid/adapter/aws_sdk.rb#L162 ) which would expect to have a range key because the at
method returns only one item.
A better approach would be to use the query
method from the client which allows to search for an item by HASH even if it has a RANGE defined.
I have tried all kind of ways to do such a query (which is possible in the AWS console) but I haven't got nowhere so far. Please let me know what you think.
Sometimes I get different data (same object saved at different points in time) after reloading:
1.9.3p194 :009 > u = User.first ; u.first_name
=> "Aa"
1.9.3p194 :010 > u.reload ; u.first_name
=> "Aaron"
In the log, it shows that User.first uses SCAN and u.reload uses BATCH GET ITEM. I was thinking that might be a clue.
I also suspect I may not be using partitioning correctly. If I just turn it off, will it make everything simpler?
The u.reload version is the correct (latest) one. This has been maddening when trying to do a password change, because the password change completes successfully, but then when I try to log in with the new password, the system compares it to some older version of the password hash, and it fails.
You have to use find_all instead
of find
for tables with range keys because in dynamoid find
destructively calls flatten
and unique
on the array of ids. Why is this a problem? Because find
calls find_all
, which calls read
, which eventually calls AWS::DynamoDB::BatchGet
, which expects arguments in this form:
ids = [ ['hash_key1', 'range_key1'], ['hash_key2', 'range_key2'] ]
flattened becomes
ids.flatten.uniq
=> ["hash_key1", "range_key1", "hash_key2", "range_key2"]
which is wrong and fails. The solution is to change Dynamoid\lib\dynamoid\finders.rb#26 to only flatten the ids when the table has no range key, for example:
ids = Array(self.range_key ? ids.flatten.uniq : ids.uniq)
Seems that the change for 0.3.2 has caused the above error
Are there plans to support associations using join tables (like ActiveRecord's has and belongs to many)? For any large system, it doesn't appear to be feasible to use Dynamoid currently because the max record size of 64kb will quickly be filled up by storing the set of the join IDs in the row.
Hi, I successfully integrated the gem in my app, I can store and retrieve data from dynamoDB, no issue on it. But when i log into the console.aws.amazon.com/dynamodb i was aspecting to find the hash that i saved from my app but I still have the wizard used to create a new table and so on...it is normal?
Thank you
Index work fine only when the HASH KEY is id.
In my opinion, indexes should not map obj.id
but [HASH_KEY, RANGE_VALUE] and use it as an identifier for the row the index is added to. This will make it easy to retrieve the item from the original table this way.
I had a question about using Dynamoid to create a new table. I had a new table when I tried to start using it, but when I first tried to read from it, I got a ResourceNotFoundException. I couldn't figure out how to make Dynamoid create the table for me, so I had to create it manually, and hope that I got the table and key name correct (I did, yay!).
Is there a way to make Dynamoid create the table on my behalf? The documentation implies yes, but it doesn't say how.
Thanks a lot!
Hi, I have a model called StatDisplayDynamo
class StatDisplayDynamo
include Dynamoid::Document
table :name => :stat_display_dynamo, :key => :id, :read_capacity => 400, :write_capacity => 400
field :campaign_id, :integer
field :ip_address
field :ua_raw
field :ua_name
field :ua_version
field :ua_engine
field :ua_os
field :ua_engine_version
index :created_at, :range => true
belongs_to :company
end
with one entry in
StatDisplayDynamo.all
=> [#<StatDisplayDynamo:0x00000104dc2c38 @new_record=false, @attributes={:created_at=>Tue, 06 Aug 2013 20:55:21 +0200, :updated_at=>Tue, 06 Aug 2013 20:55:21 +0200, :id=>"b9b769e7-bdb7-4225-8215-842caaa1652c", :campaign_id=>114, :ip_address=>"127.0.0.1", :ua_raw=>"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:22.0) Gecko/20100101 Firefox/22.0", :ua_name=>"Unknown", :ua_version=>nil, :ua_engine=>"gecko", :ua_os=>"OS X 10.8", :ua_engine_version=>"20100101", :company_ids=>nil}, @Associations={}, @changed_attributes={}>]
but if I query in this way
StatDisplayDynamo.where("created_at.gt" => DateTime.now-2.hours).all
when
DateTime.now-2.hours => Tue, 06 Aug 2013 19:34:24 +0200
I get no result. Where am i wrong?
Hi,
I'd like to use DDB's standard hash+range key on a table without using Dynamoid's "index" function. Is this possible?
Is there any facility for Dynamoid to throw ActiveRecord::RecordNotFound errors? Am I missing it somewhere?
This functionality would help me clean up my controller code and make this a much cleaner ORM.
Specific issue: add performance benchmarks for Dynamoid::Config.partitioning? (provide a link to results in the readme).
Do you guys have performance benchmarks for Dynamoid::Config.partitioning? ? It seems at a glance that it does give a benefit of balanced writes (up to a 200x data volume), but it also requires 200x read capacity for any object that has been updated sufficient number of times.
Do you actively use this feature at scale? Is it actively supported or are you planning to pull it out?
Against the new API set (2012-08-10 - fake_dynamo 0.2+), Dynamoid cannot create tables:
$ rake
[snip]
/usr/local/opt/rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/aws-sdk-1.8.0/lib/aws/core/client.rb:318:in `return_or_raise': Validation error detected: KeySchema must be a Array (AWS::DynamoDB::Errors::ValidationException)
from /usr/local/opt/rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/aws-sdk-1.8.0/lib/aws/core/client.rb:419:in `client_request'
This is because it is using the old version of the CreateTable API:
operation CreateTable
data
{"TableName"=>"dynamoid_tests_index_user_names",
"KeySchema"=>
{"HashKeyElement"=>{"AttributeName"=>"id", "AttributeType"=>"S"}},
"ProvisionedThroughput"=>{"ReadCapacityUnits"=>100, "WriteCapacityUnits"=>20}}
response
{"__type"=>"com.amazon.coral.validate#ValidationException",
"message"=>"Validation error detected: KeySchema must be a Array"}
The newer API spec expects the KeySchema to be a list of hashes with key types on them.
Is there anything in progress to update Dynamoid for the 2012 API? Should there be a note showing which version of the API this library requires, if there are significant breaking changes in the API?
I'll help do some research, and start contributing fixes to a branch. I just wanted to bring it up while I work on it, and solicit any feedback as to how it could be approached. Thanks!
Maybe I am wrong, but shouldn't where (:id => '')
use QUERY instead of SCAN when id
is hash key?
I am using dynamoid which seems to work great, but how are people using dynamoid for testing? I want my rspec tests to be able to make nice calls, but not actually hit dynamodb.
With these classes
# app/models/vehicle.rb
class Vehicle
include Dynamoid::Document
table key: :vehicle_id
field :vehicle_id
field :type
end
# app/models/car.rb
class Car < Vehicle
end
# app/models/boat.rb
class Boat < Vehicle
end
Car.all
and Car.find(id)
return the same thing Vehicle.all
and Vehicle.find(id)
, which include Boat objects.
Hi,
It would be really nice to have a uniqueness validator.
validates :name, :uniqueness => true
That's all.
Thanks,
Mateusz
If we have the following table:
class Website
include Dynamoid::Document
range :created_at, :integer
field :subdomain, :string
index :subdomain
end
The created index table will have the RANGE key set to created_at, but when a new item is created, the range key is not set.
Website.create(:created_at => Time.now.to_i, :subdomain => 'test')
will fail.
I think such cases should have RANGE set to the original range.
DescribeTable return ItemCount without a full scan.
For pagination, or other stuff, it will be great to add a count method :
dynamodb = AWS::DynamoDB.new
dynamodb.client.describe_table(table_name: TABLE_NAME)["Table"]["ItemCount"]
Thanks
I was surprised to see an array type since DynamoDB only supports sets, but I figured you must be serializing the array to JSON and storing it as a string to get around that. Instead it's just treated as a set, which is broken! It's a cool idea to support an array, but it'd be better to just call it a set if it is going to remove uniques and probably futz with the order too
when :set, :array
if value.is_a?(Set) || value.is_a?(Array)
value
else
Set[value]
end
Hi,
I'd like to be able to create the dynamoid tables without actually saving a record. I can't seem to find this in the documentation somewhere. Is it possible?
It looks like there is an error on line 44 of Persistence. I continually get:
undefined method `symbolize_keys!' for {"name"=>"se", "email_addr"=>"[email protected]"}:ActiveSupport::HashWithIndifferentAccess
When doing a new or create.
Model.exists? returns true all the time.
I require to batch a collection of items together for batchPut call. Can't find an easy way to do this using Dynamoid. Am I missing something?
I may be missing something, but there is a problem with the default configuration: the default value in lib/dynamoid/config.rb sets Dynamoid::Config.adapter to be 'aws-sdk', but this causes a problem with how the reconnect! method is currently implemented:
# lib\dynamoid\adapter.rb
module Dynamoid
module Adapter
def reconnect!
require "dynamoid/adapter/#{Dynamoid::Config.adapter}" unless Dynamoid::Adapter.const_defined?(Dynamoid::Config.adapter.camelcase)
@adapter = Dynamoid::Adapter.const_get(Dynamoid::Config.adapter.camelcase)
@adapter.connect! if @adapter.respond_to?(:connect!)
self.tables = benchmark('Cache Tables') {list_tables}
end
end
end
require "dynamoid/adapter/aws-sdk" will fail, the file name is actually aws_sdk. Also on the 2nd line Dynamoid::Config.adapter.camelcase gives 'Aws-sdk' which I think is supposed to be Dynamoid::Adapter::AwsSdk.
I found this method to be too complex and monkey patched it for my application to something more simple:
# lib\dynamoid\adapter.rb
module Dynamoid
module Adapter
def reconnect!
@adapter = Dynamoid::Adapter::AwsSdk
@adapter.connect!
self.tables = benchmark('Cache Tables') {list_tables}
end
end
end
For a more general solution I would say, if you want to use the adapter in reconnect, leave the responsibility of the caller to require the file. I did not test this, but perhaps something like might do the trick:
# initializer
require 'some/custom_adapter'
Dynamoid.configure do |config|
config.adapter = Some::CustomAdapter
end
# lib\dynamoid\adapter.rb
module Dynamoid
module Adapter
def reconnect!
@adapter = Dynamoid::Config.adapter
@adapter.connect!
self.tables = benchmark('Cache Tables') {list_tables}
end
end
end
Even if you do not push the responsibility of requiring it to the caller, it should work with the defaults.
I believe the endpoint needs to be quoted in the Readme:
:dynamo_db_endpoint => dynamodb.ap-southeast-1.amazonaws.com
When running the server with Dynamoid, the following warning is generated:
/Library/Ruby/Gems/1.8/gems/dynamoid-0.3.2/lib/dynamoid/persistence.rb:181: warning: found = in conditional, should be ==
Thanks
I have the following models:
class User
include Dynamoid::Document
table :name => :users, :key => :id , :range_key => :created_at, :read_capacity => 10, :write_capacity => 5
field :imgurl
field :username
field :email
field :showemail #boolean
field :about
field :user_ids
has_many :star
end
class Star
include Dynamoid::Document
table :name => :stars, :key => :id , :range_key => :created_at, :read_capacity => 10, :write_capacity => 5
field :title
field :abstract
field :stop
field :think
field :act
field :review
belongs_to :user
end
In the a controller I have the following method:
def view
@user = User.find_by_id(params[:id])#session[:id])
begin
@stars = @user.star.all
rescue
@stars = nil
end
end
When the user has one star the request proceeds, when the user has more than one star the request returns the following error:
dynamo db Completed 500 Internal Server Error in ..
when I changed the gem version to:
gem "dynamoid", :git => 'git://github.com/indykish/Dynamoid.git'
works perfect for when the user has more than one star.
Hello,
I just checked out Dynamoid tip and rand the rspecs, the AWS SDK does not like the scan_index_forward option introduced in commit 67b4093. Before this commit, all tests pass, afterwards, many tests fail.
Hey guys, how is it going?
It is the first time I'm using Dynamoid and it looks like a found I bug. I'm not sure if I'm doing something wrong, but here is it goes:
The following model works fine:
class Dytest
include Dynamoid::Document
table name: "#{Rails.env}_preferences"
field :foo
field :bar
end
but if I set a custom hash key it breaks the query:
class Dytest
include Dynamoid::Document
table name: "#{Rails.env}_preferences", key: :entity_uuid
field :foo
field :bar
end
d = Dytest.new(foo: '42', bar: 'monkey')
d.save
Dytest.all #here it blows up
Backtrace:
NoMethodError: undefined method `split' for nil:NilClass
from /Users/esdrasmayrink/.rvm/gems/ruby-1.9.2-p320@organizze/gems/dynamoid-0.5.0/lib/dynamoid/adapter.rb:155:in `block (2 levels) in result_for_partition'
from /Users/esdrasmayrink/.rvm/gems/ruby-1.9.2-p320@organizze/gems/dynamoid-0.5.0/lib/dynamoid/adapter.rb:153:in `each'
from /Users/esdrasmayrink/.rvm/gems/ruby-1.9.2-p320@organizze/gems/dynamoid-0.5.0/lib/dynamoid/adapter.rb:153:in `block in result_for_partition'
from /Users/esdrasmayrink/.rvm/gems/ruby-1.9.2-p320@organizze/gems/dynamoid-0.5.0/lib/dynamoid/adapter.rb:152:in `tap'
from /Users/esdrasmayrink/.rvm/gems/ruby-1.9.2-p320@organizze/gems/dynamoid-0.5.0/lib/dynamoid/adapter.rb:152:in `result_for_partition'
from /Users/esdrasmayrink/.rvm/gems/ruby-1.9.2-p320@organizze/gems/dynamoid-0.5.0/lib/dynamoid/adapter.rb:115:in `scan'
from /Users/esdrasmayrink/.rvm/gems/ruby-1.9.2-p320@organizze/gems/dynamoid-0.5.0/lib/dynamoid/criteria/chain.rb:148:in `records_without_index'
from /Users/esdrasmayrink/.rvm/gems/ruby-1.9.2-p320@organizze/gems/dynamoid-0.5.0/lib/dynamoid/criteria/chain.rb:90:in `records'
from /Users/esdrasmayrink/.rvm/gems/ruby-1.9.2-p320@organizze/gems/dynamoid-0.5.0/lib/dynamoid/criteria/chain.rb:46:in `all'
from /Users/esdrasmayrink/.rvm/gems/ruby-1.9.2-p320@organizze/gems/dynamoid-0.5.0/lib/dynamoid/criteria.rb:20:in `block (2 levels) in <module:ClassMethods>'
from (irb):4
from /Users/esdrasmayrink/.rvm/gems/ruby-1.9.2-p320@organizze/gems/railties-3.1.4/lib/rails/commands/console.rb:45:in `start'
from /Users/esdrasmayrink/.rvm/gems/ruby-1.9.2-p320@organizze/gems/railties-3.1.4/lib/rails/commands/console.rb:8:in `start'
from /Users/esdrasmayrink/.rvm/gems/ruby-1.9.2-p320@organizze/gems/railties-3.1.4/lib/rails/commands.rb:40:in `<top (required)>'
from script/rails:6:in `require'
from script/rails:6:in `<main>'
It seems that this method is where it blows up.
# It seems that result[:id] is nil because I defined a custom hash key, am I right?
def result_for_partition(results)
{}.tap do |hash|
Array(results).each do |result|
next if result.nil?
id = result[:id].split('.').first
if !hash[id] || (result[:updated_at] > hash[id][:updated_at])
result[:id] = id
hash[id] = result
end
end
end.values
end
Please let me know if there is anything I can do to help.
Cheers, Esdras.
Hello guys I wrote a small program that basically just configures a Dynamoid object the code is the following:
require 'dynamoid'
Dynamoid.configure do |config|
config.adapter = 'aws_sdk' # This adapter establishes a connection to the DynamoDB servers using Amazon's own AWS gem.
config.access_key = 'lalal' # If connecting to DynamoDB, your access key is required.
config.secret_key = 'opaaaa' # So is your secret key.
config.endpoint = 'dynamodb.eu-west-1.amazonaws.com' # Set the regional endpoint for DynamoDB.
config.namespace = "dynamoid_app_development" # To namespace tables created by Dynamoid from other tables you might have.
config.warn_on_scan = true # Output a warning to the logger when you perform a scan rather than a query on a table.
config.partitioning = true # Spread writes randomly across the database. See "partitioning" below for more.
config.partition_size = 200 # Determine the key space size that writes are randomly spread across.
config.read_capacity = 100 # Read capacity for your tables
config.write_capacity = 20 # Write capacity for your tables
end
just running this I get the following error:
wrong constant name Aws-sdk (NameError)
P.S.:
If you could direct me to a more complete tutorial than your getting started guide I would be delighted
Hi,
How we can introduce the new LSI (Local Secondary Indexes) in Dynamoid ? It can be really useful in order to avoid create a table for index.
Is that in the roadmap ?
Hey there,
We're looking into using Unicorn's 'preload_app' option, which speeds up app startup and reduces memory usage by spawning a single instance of your application, and then forking that several times for each worker.
The only downside is that you need to make sure you re-initiate all external connections in an 'after_fork' block. We've set up ours to do the following:
after_fork do |server, worker|
Dynamoid::Adapter.reconnect!
end
Unfortunately, this doesn't seem to work as expected, as running under Unicorn with 'preload_app true' we sometimes get responses back that appear to be for a different request, which causes problems. If we set 'preload_app false' this issue goes away.
We're not 100% certain the issue is in Dynamoid itself, but figured you'd be a good person to ask :D
If you have any advice for using Dynamoid within an environment such as Unicorn preload, or Passenger when set to spawning mode, I welcome it!
Thanks!
Ryan
range_key for indexes is set to :number all the time. Why not allow other type of RANGE for indexes? An index basically creates another table which uses the index as hash primary key. I think the range should be allowed to be at least datetime too.
It would be great if dynamoid could support loose model, and valorize attributes based on the columns it finds in the dynamodb field
What do you think about adding some generators like mongoid has? If you think they would fit in, I am willing to implement them myself.
You give an example:
Address.limit(5).start(address)
To get this to work, I needed to do something like this:
Address.start(address).limit(5)
As the range key paragraph in the README suggests, I'm searching for past documents:
LogEntry.where("created_at.lt" => (DateTime.now - 1.week)).all
What I get in return is:
AWS::DynamoDB::Errors::ValidationException: One or more parameter values were invalid: An AttributeValue may not contain an empty string.
from ~/.rvm/gems/ruby-1.9.2-p180/gems/aws-sdk-1.5.2/lib/aws/core/client.rb:273:in `return_or_raise'
from ~/.rvm/gems/ruby-1.9.2-p180/gems/aws-sdk-1.5.2/lib/aws/core/client.rb:372:in `client_request'
from (eval):3:in `query'
from ~/.rvm/gems/ruby-1.9.2-p180/gems/aws-sdk-1.5.2/lib/aws/dynamo_db/item_collection.rb:788:in `_each_item'
from ~/.rvm/gems/ruby-1.9.2-p180/gems/aws-sdk-1.5.2/lib/aws/core/collection/limitable.rb:57:in `each_batch'
from ~/.rvm/gems/ruby-1.9.2-p180/gems/aws-sdk-1.5.2/lib/aws/core/collection.rb:64:in `each'
from ~/.rvm/gems/ruby-1.9.2-p180/gems/aws-sdk-1.5.2/lib/aws/dynamo_db/item_collection.rb:496:in `each'
from ~/.rvm/gems/ruby-1.9.2-p180/bundler/gems/Dynamoid-61de8e9ce41e/lib/dynamoid/adapter/aws_sdk.rb:180:in `each
My Dynamoid document looks like this:
class LogEntry
include Dynamoid::Document
field :query
index :created_at, range: true
end
The table was created implicitly by Dynamoid upon first access.
Is something wrong with the gem or am I missing something?
The #{attribute_name}_changed?
and similar methods from ActiveModel::Dirty do not work except for the created_at
attribute if you're using a version of ActiveModel older that 3.1.0.
I'm not sure if it's realistic to make it do that or not, but it's a bug at least to not require a newer version of ActiveModel.
Followed all the step but I'm getting following error
/.rvm/gems/ruby-1.9.3-p392/gems/activesupport-3.2.13.rc1/lib/active_support/dependencies.rb:251:in `require': cannot load such file -- dynamoid/adapter/aws (LoadError)
from /.rvm/gems/ruby-1.9.3-p392/gems/activesupport-3.2.13.rc1/lib/active_support/dependencies.rb:251:in `block in require'
from /.rvm/gems/ruby-1.9.3-p392/gems/activesupport-3.2.13.rc1/lib/active_support/dependencies.rb:236:in `load_dependency'
Here is an rspec test demonstrating the issue
address = Address.create(:city => 'Seattle', :deliverable => true)
found = Address.find(address.id)
found.deliverable.should eq true
# PASSES in head
found = Address.where(:deliverable => 't').all
found.should eq [address]
# FAILS
found = Address.where(:deliverable => true).all
found.should eq [address]
# PASSES in head
found = Address.where(:updated_at => address.updated_at.to_time.to_f).all
found.should eq [address]
# FAILS
found = Address.where(:updated_at => address.updated_at).all
found.should eq [address]
Dynamoid::Chain.where
is not transforming the values into their dumped versions. This makes for pretty ugly queries. Instead, it could call a dump_field in Dynamoid::Persistence, however it is private at the moment: https://github.com/Veraticus/Dynamoid/blob/master/lib/dynamoid/persistence.rb#L222
If I have the following model setup with partitioning turned on:
class Table
include Dynamoid::Document
table :name => :table
field :field1
field :start_time, :datetime
index :field1, :range_key => :start_time
end
Creates records fine. But when I try to Query the table with something like
Table.where(:field1 => "123", "start_time.lt" => Time.now)
This QUERY doesn't take into account the partitioning.
This makes sense as we can't really QUERY a set of partitioned ids, as dynamoDB needs a distinct HASH_KEY and RANGE_KEY to query with. But was wondering if you intended partitioning to work with indexes?
I am currently trying to find the best way to setup a schema where I can query on 2 fields and a datetime range key in one shot.
Our current solution involves combining 2 fields as the id as seen below:
class Table
include Dynamoid::Document
table :name => :table
range :start_time, :datetime
field :field1
field :field2
field :field3
end
and inserting like
Table.new( :id => field1 + '.' + field2, , :start_time => date, :field1 => field1, :field2 => field2, :field3 => field3)
Unfortunately, this doesn't spread the hash_keys out for uniform workload. but works!
Any insight would be fantastic.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.