fiksu / partitioned

Postgres database table partitioning support for Rails

Partitioned

Partitioned adds assistance to ActiveRecord for manipulating (reading, creating, updating) an activerecord model that represents data that may be in one of many database tables (determined by the Models data).

It also has features that support the creation and deleting of child tables and partitioning support infrastructure.

It supports Postgres partitioning and has specific features to overcome basic failings of Postgres's implementation of partitioning.

Basics: A parent table can be inherited by many child tables that inherit most of the attributes of the parent table including its columns. child tables typically (and for the uses of this plugin must) have a unique check constraint the defines which data should be located in that specific child table.

Such a constraint allows for the SQL planner to ignore most child tables and target the (hopefully) one child table that contains the records interested. This splits data, and meta-data (indexes) which provides streamlined targeted access to the desired data.

Support for bulk inserts and bulk updates is also provided via Partitioned::Base.create_many and Partitioned::Base.update_many.

Example

Given the following models:

class Company < ActiveRecord::Base
end

class ByCompanyId < Partitioned::ByForeignKey
  self.abstract_class = true

  belongs_to :company

  def self.partition_foreign_key
    return :company_id
  end

  partitioned do |partition|
    partition.index :id, :unique => true
  end
end

class Employee < ByCompanyId
end

and the following tables:

-- this is the referenced table
create table companies
(
    id               serial not null primary key,
    created_at       timestamp not null default now(),
    updated_at       timestamp,
    name             text null
);

-- add some companies
insert into companies (name) values
  ('company 1'),('company 2'),('company 2');

-- this is the parent table
create table employees
(
    id               serial not null primary key,
    created_at       timestamp not null default now(),
    updated_at       timestamp,
    name             text null,
    company_id       integer not null references companies
);

We now need to create some infrastructure for partitioned tables, in particular, we create a schema to hold the child partition tables of employees.

Employee.create_infrastructure

Which creates the employees_partitions schema using the following SQL:

create schema employees_partitions;

NOTE: We also install protections on the employees table so it isn't used as a data table (this SQL is not presented for simplicity but is apart of the create_infrastructure call).

You can create migration for this Employee in this case:

class CreatePartitionEmployee < ActiveRecord::Migration
  def up
    Employee.create_infrastructure
  end

  def down
    Employee.delete_infrastructure
  end
end

To add child tables we use the create_new_partitions_tables method:

company_ids = Company.all.map(&:id)
Employee.create_new_partition_tables(company_ids)

which results in the following SQL:

create table employees_partitions.p1
  ( CHECK ( company_id = 1 ) ) INHERITS (employees);
create table employees_partitions.p2
  ( CHECK ( company_id = 2 ) ) INHERITS (employees);
create table employees_partitions.p3
  ( CHECK ( company_id = 3 ) ) INHERITS (employees);

NOTE: Some other SQL is generated in the above example, specifically the reference to the companies table needs to be explicitly created for postgres child tables AND the unique index on 'id' is created. These are not shown for simplicity.

Now we can do operations involving the child partitions.

Since database records exist in a specific child table dependant on the field "company_id" we need to have creates that turn into database inserts of the EMPLOYEES table redirect the record insert into the specific child table determined by the value of COMPANY_ID

eg:

employee = Employee.create(:name => 'Keith', :company_id => 1)

this would normally produce the following:

INSERT INTO employees ('name', company_id) values ('Keith', 1);

but with Partitioned we see:

INSERT INTO employees_partitions.p1 ('name', company_id) values ('Keith', 1);

reads of such a table need some assistance to find the specific child table the record exists in.

Since we are partitioned by company_id the programmer needs to provide that information when fetching data, or the database will need to search all child table for the specific record we are looking for.

This is no longer valid (well, doesn't perform well):

employee = Employee.find(1)

instead, do one of the following:

employee = Employee.from_partition(1).find(1)
employee = Employee.find(:first,
                         :conditions => {:name => 'Keith', :company_id => 1})
employee = Employee.find(:first,
                         :conditions => {:id => 1, :company_id => 1})

an update (employee.save where the record already exists in the database) will take advantage of knowing which child table the record exists in so it can do some optimization.

so, the following works as expected:

employee.name = "Not Keith"
employee.save

turns into the following SQL:

update employees_partitions.p1 set name = 'Not Keith' where id = 1;

Contributing

1. Fork it
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create new Pull Request

Copyright 2010-2013 fiksu.com, inc, all rights reserved

partitioned's Issues

docs issue

To enable a model for bulk inserting the docs recommend doing this:

class Company < ActiveRecord::Base
  extend Partitioned::BulkMethodsMixin
end

This doesn't actually work, though, since BulkMethodsMixin was moved to its own gem (which is required by 'partitioned'). Instead I had to:

class Company < ActiveRecord::Base
  extend BulkMethodsMixin
end

errors from the DSL should be a pleasant surprise instead of a stab in the eye

the errors should not be obtuse (like when the partitioned key value is null and the normalize function can't do a divide on it -- it should catch these errors and, in this case if any key value is null, mention that key values are needed).

Unable to create partition.

I am unable to create partition. I am getting NoMethodError exception .rvm/gems/ruby-2.0.0-p247/gems/partitioned-1.3.4/lib/monkey_patch_postgres.rb:21:in check_constraint': undefined method<<' for nil:NilClass (NoMethodError), when calling create_new_partition_tables method. I also tried running script from the example.

Here is the full trace:
from /Users/sushant/.rvm/gems/ruby-2.0.0-p247/gems/partitioned-1.3.4/lib/partitioned/partitioned_base/sql_adapter.rb:166:in block in create_partition_table' from /Users/sushant/.rvm/gems/ruby-2.0.0-p247/gems/activerecord-4.1.1/lib/active_record/connection_adapters/abstract/schema_statements.rb:198:increate_table'
from /Users/sushant/.rvm/gems/ruby-2.0.0-p247/gems/partitioned-1.3.4/lib/partitioned/partitioned_base/sql_adapter.rb:161:in create_partition_table' from /Users/sushant/.rvm/gems/ruby-2.0.0-p247/gems/partitioned-1.3.4/lib/partitioned/partitioned_base/partition_manager.rb:135:increate_new_partition'
from /Users/sushant/.rvm/gems/ruby-2.0.0-p247/gems/partitioned-1.3.4/lib/partitioned/partitioned_base/partition_manager.rb:57:in block in create_new_partition_tables' from /Users/sushant/.rvm/gems/ruby-2.0.0-p247/gems/partitioned-1.3.4/lib/partitioned/partitioned_base/partition_manager.rb:56:ineach'
from /Users/sushant/.rvm/gems/ruby-2.0.0-p247/gems/partitioned-1.3.4/lib/partitioned/partitioned_base/partition_manager.rb:56:in create_new_partition_tables' from /Users/sushant/.rvm/gems/ruby-2.0.0-p247/gems/partitioned-1.3.4/lib/partitioned/partitioned_base.rb:419:increate_new_partition_tables'
from company_id.rb:344:in <top (required)>' from /Users/sushant/.rvm/gems/ruby-2.0.0-p247/gems/railties-4.1.1/lib/rails/commands/runner.rb:60:inload'
from /Users/sushant/.rvm/gems/ruby-2.0.0-p247/gems/railties-4.1.1/lib/rails/commands/runner.rb:60:in <top (required)>' from /Users/sushant/.rvm/gems/ruby-2.0.0-p247/gems/railties-4.1.1/lib/rails/commands/commands_tasks.rb:128:inrequire'
from /Users/sushant/.rvm/gems/ruby-2.0.0-p247/gems/railties-4.1.1/lib/rails/commands/commands_tasks.rb:128:in require_command!' from /Users/sushant/.rvm/gems/ruby-2.0.0-p247/gems/railties-4.1.1/lib/rails/commands/commands_tasks.rb:95:inrunner'
from /Users/sushant/.rvm/gems/ruby-2.0.0-p247/gems/railties-4.1.1/lib/rails/commands/commands_tasks.rb:40:in run_command!' from /Users/sushant/.rvm/gems/ruby-2.0.0-p247/gems/railties-4.1.1/lib/rails/commands.rb:17:in<top (required)>'
from bin/rails:4:in require' from bin/rails:4:in

'

Rails migration and monkey patch conflict

I was installing an existing Rail project and got an error while running a migration that creates a foreign key constraint using the Rails Helper

add_foreign_key :destination_model, :origin_model

It threw an exception regarding the number of argments:

add_foreign_key(:quota_trackings, :users)
rake aborted!
StandardError: An error has occurred, this and all later migrations canceled:

wrong number of arguments (2 for 3..4)

Using the call stack printed by Rails I found the error in this file of the Gem:

https://github.com/fiksu/partitioned/blob/master/lib/monkey_patch_postgres.rb

The add_foreign_key method is getting redefined, which creates a conflict with the original Rails AR method.

Is it possible to change the name in the monkey patch to prevent this error from happening and maintain full compatibility with Rails migrations?

support Model.delete_all and Model.update_all

specifically:

Foo.from_partition(1).delete_all

should delete from the specific child partition

Foo.from_partition(1).update_all("bar = true")

should update from the specific child partition

In a MultiLevel partition, indexes specified in the component partitions aren't applied

Note from Keith:

multi-level configurator is missing three function overrides
collect, collect_first, and collect_from_collection

separate existing rows by partitions

Is it possible to add this feature?
I have very big table with partitions, but I don't know how to separate it correctly.
Maybe someone will offer a solution?

document why this isn't a module included instead of a derived class

destroy on partitioned table fails

       using_arel_table = self.respond_to?(:dynamic_arel_table) ? dynamic_arel_table() : self.class.arel_table
        relation = self.class.unscoped.where(
          using_arel_table[pk].eq(substitute))

results in:

delete from employees where employees.p1.id = ?;

table aliasing please

figure out why activerecord sometimes aliases tables

the aliasing sometime causes fuss when the alias has to be transformed (like naf_jobs) and some of the accessors to the columns try to use the actual table name "naf.jobs"

Upgrade to Rails 5.1.7 and get ArgumentException on save

I have been running the Keymailer fork of partitioned (forked from AirHelp) on ruby 2.4.2, with rails 5.1.5.

I upgraded to 2.5.7 and 5.1.7 and I could no use ActiveRecord.save because of an ArgumentError in the Activerecord monkey patch.

I know that AirHelp has a partitioned branch for ruby 5.1 and 5.2.

Is anyone maintaining partitioned any more?

Identifying the issue in 5.1.7 and updating the monkey patch is fairly daunting.

If I move to rails 5.2.x, can anyone confirm which versions the AirHelp 5.2. branch will work with?

Many thanks in advance.

superclass mismatch for class ToSql

Hi, I'm trying to setup this gem on an existing proyect.

I have a Layer model that has_many Features, I'd like to partition the features by layer_id. Both table already exist and have rows stored.

I use PostgreSQL with PostGIS and the Postgis Active Record adapter.

After adding the partiotioned gem in my Gemfile and running bundle install, whenever I run any Rails command (console, generate migration, server, etc) I get the following error

/home/pankas87/.rvm/gems/ruby-2.2.0/gems/activerecord-redshift-adapter-0.9.12/lib/monkeypatch_arel.rb:73:in `<module:Visitors>': superclass mismatch for class ToSql (TypeError)
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/activerecord-redshift-adapter-0.9.12/lib/monkeypatch_arel.rb:72:in `<module:Arel>'
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/activerecord-redshift-adapter-0.9.12/lib/monkeypatch_arel.rb:71:in `<top (required)>'
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:274:in `require'
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:274:in `block in require'
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:240:in `load_dependency'
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:274:in `require'
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/activerecord-redshift-adapter-0.9.12/lib/activerecord_redshift_adapter.rb:4:in `<top (required)>'
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:274:in `require'
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:274:in `block in require'
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:240:in `load_dependency'
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:274:in `require'
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/partitioned-1.3.4/lib/monkey_patch_redshift.rb:5:in `<top (required)>'
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:274:in `require'
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:274:in `block in require'
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:240:in `load_dependency'
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:274:in `require'
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/partitioned-1.3.4/lib/partitioned.rb:3:in `<top (required)>'
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/bundler-1.10.6/lib/bundler/runtime.rb:76:in `require'
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/bundler-1.10.6/lib/bundler/runtime.rb:76:in `block (2 levels) in require'
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/bundler-1.10.6/lib/bundler/runtime.rb:72:in `each'
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/bundler-1.10.6/lib/bundler/runtime.rb:72:in `block in require'
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/bundler-1.10.6/lib/bundler/runtime.rb:61:in `each'
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/bundler-1.10.6/lib/bundler/runtime.rb:61:in `require'
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/bundler-1.10.6/lib/bundler.rb:134:in `require'
    from /home/pankas87/ruby-projects/wm20/config/application.rb:7:in `<top (required)>'
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/railties-4.2.0/lib/rails/commands/commands_tasks.rb:141:in `require'
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/railties-4.2.0/lib/rails/commands/commands_tasks.rb:141:in `require_application_and_environment!'
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/railties-4.2.0/lib/rails/commands/commands_tasks.rb:67:in `console'
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/railties-4.2.0/lib/rails/commands/commands_tasks.rb:39:in `run_command!'
    from /home/pankas87/.rvm/gems/ruby-2.2.0/gems/railties-4.2.0/lib/rails/commands.rb:17:in `<top (required)>'
    from bin/rails:4:in `require'
    from bin/rails:4:in `<main>'

Any clue to what might be wrong? Is there any way that I can help?

SQL Partition Question

I stumbled on your gem, and it looks great for complex partitioning cases. I am currently just exploring using Postgres inheritance. Primarily, I am looking at using a "virtual table" to define common fields with a uuid id, and then child tables with actual data. I have never wrote anything in Ruby, but is this the only trigger/component you use to prevent direct parent table inserts? Thanks, I appreciate it.

          CREATE OR REPLACE FUNCTION always_fail_on_insert(table_name text) RETURNS boolean
              LANGUAGE plpgsql
              AS $$
           BEGIN
             RAISE EXCEPTION 'partitioned table "%" does not support direct inserts, you should be inserting directly into child tables', table_name;
             RETURN false;
           END;
          $$;

Wrong number of arguments for partition_normalize_key_value in by_integer_field.rb

Hey there,

I'm trying to achieve multilevel partitioning. I examined the example titled company_id_and_created_at.rb. I'm working at rails console. I'm following the example step by step and when I write Mention.create_new_partition_tables(partition_key_values) and hit enter I get the following error:

(12.3ms) CREATE TABLE "mentions_partitions"."p1" (CHECK (( search_id = 1 ))) INHERITS (mentions)
(1.7ms) SELECT distinct i.relname, d.indisunique, d.indkey, pg_get_indexdef(d.indexrelid), t.oid
FROM pg_class t
INNER JOIN pg_index d ON t.oid = d.indrelid
INNER JOIN pg_class i ON d.indexrelid = i.oid
WHERE i.relkind = 'i'
AND d.indisprimary = 'f'
AND t.relname = 'mentions_partitions.p1'
AND i.relnamespace IN (SELECT oid FROM pg_namespace WHERE nspname = ANY (current_schemas(false)) )
ORDER BY i.relname

(7.7ms) CREATE UNIQUE INDEX "p1_id_udx" ON "mentions_partitions"."p1" ("id")
(8.3ms) ALTER TABLE mentions_partitions.p1 add foreign key (search_id) references searches(id)
ArgumentError: wrong number of arguments (2 for 1)
from /Users/ilker/.rvm/gems/ruby-1.9.2-p320/gems/partitioned-0.8.0/lib/partitioned/by_integer_field.rb:13:in partition_normalize_key_value' from /Users/ilker/.rvm/gems/ruby-1.9.2-p320/gems/partitioned-0.8.0/lib/partitioned/partitioned_base.rb:286:inblock (2 levels) in class:PartitionedBase'
from /Users/ilker/.rvm/gems/ruby-1.9.2-p320/gems/partitioned-0.8.0/lib/partitioned/partitioned_base/configurator/reader.rb:176:in call' from /Users/ilker/.rvm/gems/ruby-1.9.2-p320/gems/partitioned-0.8.0/lib/partitioned/partitioned_base/configurator/reader.rb:176:inblock in collect_first'
from /Users/ilker/.rvm/gems/ruby-1.9.2-p320/gems/partitioned-0.8.0/lib/partitioned/partitioned_base/configurator/reader.rb:172:in each' from /Users/ilker/.rvm/gems/ruby-1.9.2-p320/gems/partitioned-0.8.0/lib/partitioned/partitioned_base/configurator/reader.rb:172:incollect_first'
from /Users/ilker/.rvm/gems/ruby-1.9.2-p320/gems/partitioned-0.8.0/lib/partitioned/partitioned_base/configurator/reader.rb:104:in base_name' from /Users/ilker/.rvm/gems/ruby-1.9.2-p320/gems/partitioned-0.8.0/lib/partitioned/partitioned_base.rb:268:inblock (2 levels) in class:PartitionedBase'
from /Users/ilker/.rvm/gems/ruby-1.9.2-p320/gems/partitioned-0.8.0/lib/partitioned/partitioned_base/configurator/reader.rb:176:in call' from /Users/ilker/.rvm/gems/ruby-1.9.2-p320/gems/partitioned-0.8.0/lib/partitioned/partitioned_base/configurator/reader.rb:176:inblock in collect_first'
from /Users/ilker/.rvm/gems/ruby-1.9.2-p320/gems/partitioned-0.8.0/lib/partitioned/partitioned_base/configurator/reader.rb:172:in each' from /Users/ilker/.rvm/gems/ruby-1.9.2-p320/gems/partitioned-0.8.0/lib/partitioned/partitioned_base/configurator/reader.rb:172:incollect_first'
from /Users/ilker/.rvm/gems/ruby-1.9.2-p320/gems/partitioned-0.8.0/lib/partitioned/partitioned_base/configurator/reader.rb:121:in part_name' from /Users/ilker/.rvm/gems/ruby-1.9.2-p320/gems/partitioned-0.8.0/lib/partitioned/partitioned_base.rb:276:inblock (2 levels) in class:PartitionedBase'
from /Users/ilker/.rvm/gems/ruby-1.9.2-p320/gems/partitioned-0.8.0/lib/partitioned/partitioned_base/configurator/reader.rb:176:in call' from /Users/ilker/.rvm/gems/ruby-1.9.2-p320/gems/partitioned-0.8.0/lib/partitioned/partitioned_base/configurator/reader.rb:176:inblock in collect_first'
from /Users/ilker/.rvm/gems/ruby-1.9.2-p320/gems/partitioned-0.8.0/lib/partitioned/partitioned_base/configurator/reader.rb:172:in each' from /Users/ilker/.rvm/gems/ruby-1.9.2-p320/gems/partitioned-0.8.0/lib/partitioned/partitioned_base/configurator/reader.rb:172:incollect_first'
from /Users/ilker/.rvm/gems/ruby-1.9.2-p320/gems/partitioned-0.8.0/lib/partitioned/partitioned_base/configurator/reader.rb:97:in table_name' from /Users/ilker/.rvm/gems/ruby-1.9.2-p320/gems/partitioned-0.8.0/lib/partitioned/partitioned_base/sql_adapter.rb:131:increate_partition_table'
from /Users/ilker/.rvm/gems/ruby-1.9.2-p320/gems/partitioned-0.8.0/lib/partitioned/partitioned_base/partition_manager.rb:99:in create_new_partition' from /Users/ilker/.rvm/gems/ruby-1.9.2-p320/gems/partitioned-0.8.0/lib/partitioned/partitioned_base/partition_manager.rb:44:inblock in create_new_partition_tables'
from /Users/ilker/.rvm/gems/ruby-1.9.2-p320/gems/partitioned-0.8.0/lib/partitioned/partitioned_base/partition_manager.rb:43:in each' from /Users/ilker/.rvm/gems/ruby-1.9.2-p320/gems/partitioned-0.8.0/lib/partitioned/partitioned_base/partition_manager.rb:43:increate_new_partition_tables'
from (irb):134
from /Users/ilker/.rvm/gems/ruby-1.9.2-p320/gems/railties-3.2.6/lib/rails/commands/console.rb:47:in start' from /Users/ilker/.rvm/gems/ruby-1.9.2-p320/gems/railties-3.2.6/lib/rails/commands/console.rb:8:instart'
from /Users/ilker/.rvm/gems/ruby-1.9.2-p320/gems/railties-3.2.6/lib/rails/commands.rb:41:in <top (required)>' from script/rails:6:inrequire'
from script/rails:6:in...

partition_key_values has the following values:
[1, [1, Mon, 27 Dec 2010], [1, Mon, 03 Jan 2011], [1, Mon, 10 Jan 2011], [1, Mon, 17 Jan 2011], [1, Mon, 24 Jan 2011], [1, Mon, 31 Jan 2011], [1, Mon, 07 Feb 2011], [1, Mon, 14 Feb 2011], [1, Mon, 21 Feb 2011], [1, Mon, 28 Feb 2011], [1, Mon, 07 Mar 2011], [1, Mon, 14 Mar 2011], [1, Mon, 21 Mar 2011], [1, Mon, 28 Mar 2011], [1, Mon, 04 Apr 2011], [1, Mon, 11 Apr 2011], [1, Mon, 18 Apr 2011], [1, Mon, 25 Apr 2011], [1, Mon, 02 May 2011], [1, Mon, 09 May 2011], [1, Mon, 16 May 2011], [1, Mon, 23 May 2011], [1, Mon, 30 May 2011], [1, Mon, 06 Jun 2011], [1, Mon, 13 Jun 2011], [1, Mon, 20 Jun 2011], [1, Mon, 27 Jun 2011], [1, Mon, 04 Jul 2011], [1, Mon, 11 Jul 2011], [1, Mon, 18 Jul 2011], [1, Mon, 25 Jul 2011], [1, Mon, 01 Aug 2011], [1, Mon, 08 Aug 2011], [1, Mon, 15 Aug 2011], [1, Mon, 22 Aug 2011], [1, Mon, 29 Aug 2011], [1, Mon, 05 Sep 2011], [1, Mon, 12 Sep 2011], [1, Mon, 19 Sep 2011], [1, Mon, 26 Sep 2011], [1, Mon, 03 Oct 2011], [1, Mon, 10 Oct 2011], [1, Mon, 17 Oct 2011], [1, Mon, 24 Oct 2011], [1, Mon, 31 Oct 2011], [1, Mon, 07 Nov 2011], [1, Mon, 14 Nov 2011], [1, Mon, 21 Nov 2011], [1, Mon, 28 Nov 2011], [1, Mon, 05 Dec 2011], [1, Mon, 12 Dec 2011], [1, Mon, 19 Dec 2011], [1, Mon, 26 Dec 2011], [1, Mon, 02 Jan 2012], [1, Mon, 09 Jan 2012], [1, Mon, 16 Jan 2012], [1, Mon, 23 Jan 2012], [1, Mon, 30 Jan 2012], [1, Mon, 06 Feb 2012], [1, Mon, 13 Feb 2012], [1, Mon, 20 Feb 2012], [1, Mon, 27 Feb 2012], [1, Mon, 05 Mar 2012], [1, Mon, 12 Mar 2012], [1, Mon, 19 Mar 2012], [1, Mon, 26 Mar 2012], [1, Mon, 02 Apr 2012], [1, Mon, 09 Apr 2012], [1, Mon, 16 Apr 2012], [1, Mon, 23 Apr 2012], [1, Mon, 30 Apr 2012], [1, Mon, 07 May 2012], [1, Mon, 14 May 2012], [1, Mon, 21 May 2012], [1, Mon, 28 May 2012], [1, Mon, 04 Jun 2012], [1, Mon, 11 Jun 2012], [1, Mon, 18 Jun 2012], [1, Mon, 25 Jun 2012], [1, Mon, 02 Jul 2012], [1, Mon, 09 Jul 2012], [1, Mon, 16 Jul 2012], [1, Mon, 23 Jul 2012], [1, Mon, 30 Jul 2012], [1, Mon, 06 Aug 2012], [1, Mon, 13 Aug 2012], [1, Mon, 20 Aug 2012], [1, Mon, 27 Aug 2012], [1, Mon, 03 Sep 2012], [1, Mon, 10 Sep 2012], [1, Mon, 17 Sep 2012], [1, Mon, 24 Sep 2012], [1, Mon, 01 Oct 2012], [1, Mon, 08 Oct 2012], [1, Mon, 15 Oct 2012], [1, Mon, 22 Oct 2012], [1, Mon, 29 Oct 2012], [1, Mon, 05 Nov 2012], [1, Mon, 12 Nov 2012], [1, Mon, 19 Nov 2012], [1, Mon, 26 Nov 2012], [1, Mon, 03 Dec 2012], [1, Mon, 10 Dec 2012], [1, Mon, 17 Dec 2012], [1, Mon, 24 Dec 2012], [1, Mon, 31 Dec 2012], 3, [3, Mon, 27 Dec 2010], [3, Mon, 03 Jan 2011], [3, Mon, 10 Jan 2011], [3, Mon, 17 Jan 2011], [3, Mon, 24 Jan 2011], [3, Mon, 31 Jan 2011], [3, Mon, 07 Feb 2011], [3, Mon, 14 Feb 2011], [3, Mon, 21 Feb 2011], [3, Mon, 28 Feb 2011], [3, Mon, 07 Mar 2011], [3, Mon, 14 Mar 2011], [3, Mon, 21 Mar 2011], [3, Mon, 28 Mar 2011], [3, Mon, 04 Apr 2011], [3, Mon, 11 Apr 2011], [3, Mon, 18 Apr 2011], [3, Mon, 25 Apr 2011], [3, Mon, 02 May 2011], [3, Mon, 09 May 2011], [3, Mon, 16 May 2011], [3, Mon, 23 May 2011], [3, Mon, 30 May 2011], [3, Mon, 06 Jun 2011], [3, Mon, 13 Jun 2011], [3, Mon, 20 Jun 2011], [3, Mon, 27 Jun 2011], [3, Mon, 04 Jul 2011], [3, Mon, 11 Jul 2011], [3, Mon, 18 Jul 2011], [3, Mon, 25 Jul 2011], [3, Mon, 01 Aug 2011], [3, Mon, 08 Aug 2011], [3, Mon, 15 Aug 2011], [3, Mon, 22 Aug 2011], [3, Mon, 29 Aug 2011], [3, Mon, 05 Sep 2011], [3, Mon, 12 Sep 2011], [3, Mon, 19 Sep 2011], [3, Mon, 26 Sep 2011], [3, Mon, 03 Oct 2011], [3, Mon, 10 Oct 2011], [3, Mon, 17 Oct 2011], [3, Mon, 24 Oct 2011], [3, Mon, 31 Oct 2011], [3, Mon, 07 Nov 2011], [3, Mon, 14 Nov 2011], [3, Mon, 21 Nov 2011], [3, Mon, 28 Nov 2011], [3, Mon, 05 Dec 2011], [3, Mon, 12 Dec 2011], [3, Mon, 19 Dec 2011], [3, Mon, 26 Dec 2011], [3, Mon, 02 Jan 2012], [3, Mon, 09 Jan 2012], [3, Mon, 16 Jan 2012], [3, Mon, 23 Jan 2012], [3, Mon, 30 Jan 2012], [3, Mon, 06 Feb 2012], [3, Mon, 13 Feb 2012], [3, Mon, 20 Feb 2012], [3, Mon, 27 Feb 2012], [3, Mon, 05 Mar 2012], [3, Mon, 12 Mar 2012], [3, Mon, 19 Mar 2012], [3, Mon, 26 Mar 2012], [3, Mon, 02 Apr 2012], [3, Mon, 09 Apr 2012], [3, Mon, 16 Apr 2012], [3, Mon, 23 Apr 2012], [3, Mon, 30 Apr 2012], [3, Mon, 07 May 2012], [3, Mon, 14 May 2012], [3, Mon, 21 May 2012], [3, Mon, 28 May 2012], [3, Mon, 04 Jun 2012], [3, Mon, 11 Jun 2012], [3, Mon, 18 Jun 2012], [3, Mon, 25 Jun 2012], [3, Mon, 02 Jul 2012], [3, Mon, 09 Jul 2012], [3, Mon, 16 Jul 2012], [3, Mon, 23 Jul 2012], [3, Mon, 30 Jul 2012], [3, Mon, 06 Aug 2012], [3, Mon, 13 Aug 2012], [3, Mon, 20 Aug 2012], [3, Mon, 27 Aug 2012], [3, Mon, 03 Sep 2012], [3, Mon, 10 Sep 2012], [3, Mon, 17 Sep 2012], [3, Mon, 24 Sep 2012], [3, Mon, 01 Oct 2012], [3, Mon, 08 Oct 2012], [3, Mon, 15 Oct 2012], [3, Mon, 22 Oct 2012], [3, Mon, 29 Oct 2012], [3, Mon, 05 Nov 2012], [3, Mon, 12 Nov 2012], [3, Mon, 19 Nov 2012], [3, Mon, 26 Nov 2012], [3, Mon, 03 Dec 2012], [3, Mon, 10 Dec 2012], [3, Mon, 17 Dec 2012], [3, Mon, 24 Dec 2012], [3, Mon, 31 Dec 2012], 4, [4, Mon, 27 Dec 2010], [4, Mon, 03 Jan 2011], [4, Mon, 10 Jan 2011], [4, Mon, 17 Jan 2011], [4, Mon, 24 Jan 2011], [4, Mon, 31 Jan 2011], [4, Mon, 07 Feb 2011], [4, Mon, 14 Feb 2011], [4, Mon, 21 Feb 2011], [4, Mon, 28 Feb 2011], [4, Mon, 07 Mar 2011], [4, Mon, 14 Mar 2011], [4, Mon, 21 Mar 2011], [4, Mon, 28 Mar 2011], [4, Mon, 04 Apr 2011], [4, Mon, 11 Apr 2011], [4, Mon, 18 Apr 2011], [4, Mon, 25 Apr 2011], [4, Mon, 02 May 2011], [4, Mon, 09 May 2011], [4, Mon, 16 May 2011], [4, Mon, 23 May 2011], [4, Mon, 30 May 2011], [4, Mon, 06 Jun 2011], [4, Mon, 13 Jun 2011], [4, Mon, 20 Jun 2011], [4, Mon, 27 Jun 2011], [4, Mon, 04 Jul 2011], [4, Mon, 11 Jul 2011], [4, Mon, 18 Jul 2011], [4, Mon, 25 Jul 2011], [4, Mon, 01 Aug 2011], [4, Mon, 08 Aug 2011], [4, Mon, 15 Aug 2011], [4, Mon, 22 Aug 2011], [4, Mon, 29 Aug 2011], [4, Mon, 05 Sep 2011], [4, Mon, 12 Sep 2011], [4, Mon, 19 Sep 2011], [4, Mon, 26 Sep 2011], [4, Mon, 03 Oct 2011], [4, Mon, 10 Oct 2011], [4, Mon, 17 Oct 2011], [4, Mon, 24 Oct 2011], [4, Mon, 31 Oct 2011], [4, Mon, 07 Nov 2011], [4, Mon, 14 Nov 2011], [4, Mon, 21 Nov 2011], [4, Mon, 28 Nov 2011], [4, Mon, 05 Dec 2011], [4, Mon, 12 Dec 2011], [4, Mon, 19 Dec 2011], [4, Mon, 26 Dec 2011], [4, Mon, 02 Jan 2012], [4, Mon, 09 Jan 2012], [4, Mon, 16 Jan 2012], [4, Mon, 23 Jan 2012], [4, Mon, 30 Jan 2012], [4, Mon, 06 Feb 2012], [4, Mon, 13 Feb 2012], [4, Mon, 20 Feb 2012], [4, Mon, 27 Feb 2012], [4, Mon, 05 Mar 2012], [4, Mon, 12 Mar 2012], [4, Mon, 19 Mar 2012], [4, Mon, 26 Mar 2012], [4, Mon, 02 Apr 2012], [4, Mon, 09 Apr 2012], [4, Mon, 16 Apr 2012], [4, Mon, 23 Apr 2012], [4, Mon, 30 Apr 2012], [4, Mon, 07 May 2012], [4, Mon, 14 May 2012], [4, Mon, 21 May 2012], [4, Mon, 28 May 2012], [4, Mon, 04 Jun 2012], [4, Mon, 11 Jun 2012], [4, Mon, 18 Jun 2012], [4, Mon, 25 Jun 2012], [4, Mon, 02 Jul 2012], [4, Mon, 09 Jul 2012], [4, Mon, 16 Jul 2012], [4, Mon, 23 Jul 2012], [4, Mon, 30 Jul 2012], [4, Mon, 06 Aug 2012], [4, Mon, 13 Aug 2012], [4, Mon, 20 Aug 2012], [4, Mon, 27 Aug 2012], [4, Mon, 03 Sep 2012], [4, Mon, 10 Sep 2012], [4, Mon, 17 Sep 2012], [4, Mon, 24 Sep 2012], [4, Mon, 01 Oct 2012], [4, Mon, 08 Oct 2012], [4, Mon, 15 Oct 2012], [4, Mon, 22 Oct 2012], [4, Mon, 29 Oct 2012], [4, Mon, 05 Nov 2012], [4, Mon, 12 Nov 2012], [4, Mon, 19 Nov 2012], [4, Mon, 26 Nov 2012], [4, Mon, 03 Dec 2012], [4, Mon, 10 Dec 2012], [4, Mon, 17 Dec 2012], [4, Mon, 24 Dec 2012], [4, Mon, 31 Dec 2012], 5, [5, Mon, 27 Dec 2010], [5, Mon, 03 Jan 2011], [5, Mon, 10 Jan 2011], [5, Mon, 17 Jan 2011], [5, Mon, 24 Jan 2011], [5, Mon, 31 Jan 2011], [5, Mon, 07 Feb 2011], [5, Mon, 14 Feb 2011], [5, Mon, 21 Feb 2011], [5, Mon, 28 Feb 2011], [5, Mon, 07 Mar 2011], [5, Mon, 14 Mar 2011], [5, Mon, 21 Mar 2011], [5, Mon, 28 Mar 2011], [5, Mon, 04 Apr 2011], [5, Mon, 11 Apr 2011], [5, Mon, 18 Apr 2011], [5, Mon, 25 Apr 2011], [5, Mon, 02 May 2011], [5, Mon, 09 May 2011], [5, Mon, 16 May 2011], [5, Mon, 23 May 2011], [5, Mon, 30 May 2011], [5, Mon, 06 Jun 2011], [5, Mon, 13 Jun 2011], [5, Mon, 20 Jun 2011], [5, Mon, 27 Jun 2011], [5, Mon, 04 Jul 2011], [5, Mon, 11 Jul 2011], [5, Mon, 18 Jul 2011], [5, Mon, 25 Jul 2011], [5, Mon, 01 Aug 2011], [5, Mon, 08 Aug 2011], [5, Mon, 15 Aug 2011], [5, Mon, 22 Aug 2011], [5, Mon, 29 Aug 2011], [5, Mon, 05 Sep 2011], [5, Mon, 12 Sep 2011], [5, Mon, 19 Sep 2011], [5, Mon, 26 Sep 2011], [5, Mon, 03 Oct 2011], [5, Mon, 10 Oct 2011], [5, Mon, 17 Oct 2011], [5, Mon, 24 Oct 2011], [5, Mon, 31 Oct 2011], [5, Mon, 07 Nov 2011], [5, Mon, 14 Nov 2011], [5, Mon, 21 Nov 2011], [5, Mon, 28 Nov 2011], [5, Mon, 05 Dec 2011], [5, Mon, 12 Dec 2011], [5, Mon, 19 Dec 2011], [5, Mon, 26 Dec 2011], [5, Mon, 02 Jan 2012], [5, Mon, 09 Jan 2012], [5, Mon, 16 Jan 2012], [5, Mon, 23 Jan 2012], [5, Mon, 30 Jan 2012], [5, Mon, 06 Feb 2012], [5, Mon, 13 Feb 2012], [5, Mon, 20 Feb 2012], [5, Mon, 27 Feb 2012], [5, Mon, 05 Mar 2012], [5, Mon, 12 Mar 2012], [5, Mon, 19 Mar 2012], [5, Mon, 26 Mar 2012], [5, Mon, 02 Apr 2012], [5, Mon, 09 Apr 2012], [5, Mon, 16 Apr 2012], [5, Mon, 23 Apr 2012], [5, Mon, 30 Apr 2012], [5, Mon, 07 May 2012], [5, Mon, 14 May 2012], [5, Mon, 21 May 2012], [5, Mon, 28 May 2012], [5, Mon, 04 Jun 2012], [5, Mon, 11 Jun 2012], [5, Mon, 18 Jun 2012], [5, Mon, 25 Jun 2012], [5, Mon, 02 Jul 2012], [5, Mon, 09 Jul 2012], [5, Mon, 16 Jul 2012], [5, Mon, 23 Jul 2012], [5, Mon, 30 Jul 2012], [5, Mon, 06 Aug 2012], [5, Mon, 13 Aug 2012], [5, Mon, 20 Aug 2012], [5, Mon, 27 Aug 2012], [5, Mon, 03 Sep 2012], [5, Mon, 10 Sep 2012], [5, Mon, 17 Sep 2012], [5, Mon, 24 Sep 2012], [5, Mon, 01 Oct 2012], [5, Mon, 08 Oct 2012], [5, Mon, 15 Oct 2012], [5, Mon, 22 Oct 2012], [5, Mon, 29 Oct 2012], [5, Mon, 05 Nov 2012], [5, Mon, 12 Nov 2012], [5, Mon, 19 Nov 2012], [5, Mon, 26 Nov 2012], [5, Mon, 03 Dec 2012], [5, Mon, 10 Dec 2012], [5, Mon, 17 Dec 2012], [5, Mon, 24 Dec 2012], [5, Mon, 31 Dec 2012]]

What could be the problem?

table_alias_name in MultiLevel

table_alias_name returns the immediate parent table name in a multi-level partition, not the topmost parent table name. For example, a multi-level partition called employees_partitions.p1_20130128 will be aliased as 'employees_partitions_p1', instead of 'employees'.

partition.index doesn't support all index parameters needed

I'd like to be able to create an index over a modified field:

CREATE INDEX column_lower_idx ON partition_table (LOWER(column_name));

Tests don't appear to work from root of gem checkout

Documentation is insufficient to run tests on partitioned using the checkout.

Platform is ruby 2.3.0p0 (2015-12-25 revision 53290) [x86_64-darwin15]

Error 1:

$ rspec spec /Users/ivan/Projects/partitioned/spec/dummy/config/application.rb:7:in <top (required)>': uninitialized constant Bundler (NameError)
from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in require' from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in require'
from /Users/ivan/Projects/partitioned/spec/dummy/config/environment.rb:2:in <top (required)>' from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in require'
from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in require' from /Users/ivan/Projects/partitioned/spec/spec_helper.rb:3:in <top (required)>'
from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in require' from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in require'
from /Users/ivan/Projects/partitioned/spec/monkey_patch_postgres_spec.rb:1:in <top (required)>' from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.4/lib/rspec/core/configuration.rb:1361:in load'
from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.4/lib/rspec/core/configuration.rb:1361:in block in load_spec_files' from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.4/lib/rspec/core/configuration.rb:1359:in each'
from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.4/lib/rspec/core/configuration.rb:1359:in load_spec_files' from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.4/lib/rspec/core/runner.rb:106:in setup'
from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.4/lib/rspec/core/runner.rb:92:in run' from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.4/lib/rspec/core/runner.rb:78:in run'
from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.4/lib/rspec/core/runner.rb:45:in invoke' from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.4/exe/rspec:4:in <top (required)>'
from /Users/ivan/.rbenv/versions/2.3.0/bin/rspec:23:in load' from /Users/ivan/.rbenv/versions/2.3.0/bin/rspec:23:in

'`

Error 2:

$ bundle exec rspec spec /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/bundler-1.11.2/lib/bundler/runtime.rb:80:in rescue in block (2 levels) in require': There was an error while trying to load the gem 'partitioned'. (Bundler::GemRequireError)
from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/bundler-1.11.2/lib/bundler/runtime.rb:76:in block (2 levels) in require' from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/bundler-1.11.2/lib/bundler/runtime.rb:72:in each'
from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/bundler-1.11.2/lib/bundler/runtime.rb:72:in block in require' from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/bundler-1.11.2/lib/bundler/runtime.rb:61:in each'
from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/bundler-1.11.2/lib/bundler/runtime.rb:61:in require' from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/bundler-1.11.2/lib/bundler.rb:99:in require'
from /Users/ivan/Projects/partitioned/spec/dummy/config/application.rb:7:in <top (required)>' from /Users/ivan/Projects/partitioned/spec/dummy/config/environment.rb:2:in require'
from /Users/ivan/Projects/partitioned/spec/dummy/config/environment.rb:2:in <top (required)>' from /Users/ivan/Projects/partitioned/spec/spec_helper.rb:3:in require'
from /Users/ivan/Projects/partitioned/spec/spec_helper.rb:3:in <top (required)>' from /Users/ivan/Projects/partitioned/spec/monkey_patch_postgres_spec.rb:1:in require'
from /Users/ivan/Projects/partitioned/spec/monkey_patch_postgres_spec.rb:1:in <top (required)>' from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.4/lib/rspec/core/configuration.rb:1361:in load'
from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.4/lib/rspec/core/configuration.rb:1361:in block in load_spec_files' from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.4/lib/rspec/core/configuration.rb:1359:in each'
from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.4/lib/rspec/core/configuration.rb:1359:in load_spec_files' from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.4/lib/rspec/core/runner.rb:106:in setup'
from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.4/lib/rspec/core/runner.rb:92:in run' from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.4/lib/rspec/core/runner.rb:78:in run'
from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.4/lib/rspec/core/runner.rb:45:in invoke' from /Users/ivan/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.4/exe/rspec:4:in <top (required)>'
from /Users/ivan/.rbenv/versions/2.3.0/bin/rspec:23:in load' from /Users/ivan/.rbenv/versions/2.3.0/bin/rspec:23:in

'`

investigate and fix any issues with parent tables in non-public schema

Perhaps I've composed suboptimal config, here's what I have for my tables:

class PartitionedByReportDate < Partitioned::ByMonthlyTimeField
  def self.partition_time_field
    :report_date
  end
end

class PartitionedByIphoneApplicationId < Partitioned::ByForeignKey
  def self.partition_foreign_key
    :iphone_application_id
  end
end

partitioned do |partition|
  partition.using_classes PartitionedByIphoneApplicationId, PartitionedByReportDate

  # Postgres doesn't allow schema nesting, put partitions schema into public
  partition.schema_name "#{base.table_name.split('.').last}_partitions"
end

Maybe this syntax with defining Partitioned classes on user side gives greater flexibility, but my example would be more concise if I could write something like this:

partitioned do |partition|
  partition.by_foreign_key        :iphone_application_id
  partition.by_monthly_time_field :report_date

  # Postgres doesn't allow schema nesting, put partitions schema into public
  partition.schema_name "#{base.table_name.split('.').last}_partitions"
end

I have parent table 'my_data_sets' which resides in data_api schema. As postgres doesn't support schema nesting, I configured partitioning so that my_data_sets_partitions lives side by side with data_api:

data_api.my_data_sets           <- partitioned table
my_data_sets_partitions.*       <- all partitions go there

Other than configuring "partition.schema_name", I had to add these two overrides:

def arel_attributes_values(*args)
  attrs = super
  # This cuts table name from schema, so that we don't end up with invalid table aliases
  # like "data_api"."data_sets"
  actual_arel_table = dynamic_arel_table(self.class.table_name.split('.').last)
  return Hash[*attrs.map{|k,v| [actual_arel_table[k.name], v]}.flatten]
end

def update(attribute_names = @attributes.keys)
  attributes_with_values = arel_attributes_values(false, false, attribute_names)
  return 0 if attributes_with_values.empty?
  klass = self.class
  # .where should reference partition table
  stmt = klass.unscoped.where(dynamic_arel_table(self.class.table_name.split('.').last)[klass.primary_key].eq(id)).arel.compile_update(attributes_with_values)
  klass.connection.update stmt
end

Both are needed in order to make updates work. Without them I was getting the following SQL from AR:

UPDATE "my_data_sets_partitions"."pXXX_YYYYMM" "data_api"."my_data_sets"
^^^^^^^^^^ fixed by #arel_attributes_values
SET ...
WHERE "data_api"."my_data_sets"."id" = Y
^^^^^^^^^^ fixed by #update

help with associations

associations need to take partition keys and support accesses

I actually haven't thought this through at all, but we want belongs_to and has_on to pass the partition keys as needed.
collection associations probably can so something similar since generally the associations will reference the same target partition -- but we need to find some way of handling multi-partition many2many associations

Index generates names that exceed the length limit for Postgresql

For example, the following index will fail:

partition.index [:unharmonious_database_column_one, :unharmonious_database_column_two]

delegate method is not working when extending Partitioned::PartitionedBase

class Employee < PartitionedByCompanyId
  attr_accessible :first_name, :last_name, :company_id

  def personal_details
    { :test => true }
  end

  delegate :keys, :to => :personal_details
end

class PartitionedByCompanyId < Partitioned::ByForeignKey
  self.abstract_class = true

  belongs_to :company

  def self.partition_foreign_key
    return :company_id
  end

  partitioned do |partition|
    partition.index :id, :unique => true
    partition.foreign_key :company_id
  end
end

using delegate ends up with

/Users/matrushka/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/forwardable.rb:215:in `single_delegate': wrong number of arguments (2 for 1) (ArgumentError)

#reload does not take into account partition keys.

understand what happens when key values change

i've punted on this, i should at least know if postgres supports such activity.

create_many should not assign id

id should be left assigned automatically by the database.

Assigning it manually doesn't work correctly if the id is generated using a stored procedure.

Need a way to disable check constraint

I have a table that is partitioned by a foreign key id that is not actually in the table. This largely works, except that there is no easy way to prevent partitioned from trying to apply a check constraint. I've been doing something like this:

partition.check_constraint lambda { |model, id|
return "( 1 = 1 )"
}

But it would be better to not impose any constraint at all.

Activerecord 4 compatibility?

Is partitioned compatible with AR4? I couldn't find anything about it in docs or anywhere else.

Request for more background information

Could you guys elaborate on what you mean by "basic failings of Postgres's implementation of partitioning" and how you address them? I read the "partitioning explained" guide and from there I gather doing a select on a child table might be faster than select on parent table (i.e. select from employees_1 instead of select from employees where company_id = 1), are there other considerations? Many thanks!

PG::UndefinedTable: ERROR: missing FROM-clause entry for table "p54"

I'm using this gem to manage a huge table with pageviews of many sites. I divided table pageviews into partitions. Each site has own partition. Queries are being processed much faster now. But sometimes code, which usually works properly, produces an error. It looks like query is sometimes built wrongly. For example when I fire:

Pageview.from_partition(site_id).where('pageviews.distance <= ?', distance).select(:visitor_id).distinct.count

Query normally is built like this:

SELECT DISTINCT COUNT(DISTINCT "pageviews"."visitor_id") FROM "pageviews_partitions"."p54" "pageviews" WHERE (pageviews.distance <= 25)

But sometimes (even when previous line of code looks similarly and only distance variable has different value) I receive this error:

PG::UndefinedTable: ERROR: missing FROM-clause entry for table "p54" LINE 1: SELECT DISTINCT COUNT(DISTINCT "pageviews_partitions"."p54".... ^

It looks like wrong name of table is used as the name of parent table. I cannot paste complete sql query because this error appear only in Sidekiq and I see it in errbit. When I run this code manually the issue does not apear.

Possible memory leak from memoization of Partitioned::PartitionedBase.arel_table_from_key_values

Hi, I've noticed an issue when using partitioned classes inherit from Partitioned::ByTimeField. If we perform INSERT or UPDATE with many instances of the class, size of @arel_tables would grow indefinitely as key for the memoization would have type of DateTime whose resolution is 1 second.

    def self.arel_table_from_key_values(partition_key_values, as = nil)
      @arel_tables ||= {}
      new_arel_table = @arel_tables[[partition_key_values, as]]

      unless new_arel_table
        arel_engine_hash = {:engine => self.arel_engine, :as => as}
        new_arel_table = Arel::Table.new(self.partition_table_name(*partition_key_values), arel_engine_hash)
        @arel_tables[[partition_key_values, as]] = new_arel_table
      end

      return new_arel_table
    end

In the worst case, @arel_tables will grow by one entry for every second unless we normalize partition_key_values before memoization if it is needed.

last_n_partitions_order_by_clause / last_partitions_order_by_clause name discrepancy

In lib/partitioned/partitioned_base/sql_adapter.rb:95

def last_n_partitions_order_by_clause
  return configurator.last_n_partitions_order_by_clause
end

But in lib/partitioned/partitioned_base/configurator/reader.rb:143

def last_partitions_order_by_clause

Copy indexes from the parent table

If I have indexes on the parent table, it does seems to copy them to child partitions. Is this feature currently available?

From example:

class Client < Partitioned::ByMonthlyTimeField
  def self.partition_time_field
    return :started_at
  end
end

# with

add_index :clients, :portfolio_id
add_index :clients, :name

start_date = Date.parse('2005-01-01')
end_date   = Date.today + 2.years
dates = Client.partition_generate_range(start_date, end_date)
Client.create_new_partition_tables(dates)

The index for portfolio_id and name are not created on the client_partition tables. Are they supposed to be?

support janitorial work

partitioned should support some mechanism that allows partitioned tables to notify outside observers of their need for new child tables as well as the requirement to archive and/or drop stale partitions.

consider a table that is partitioned by year of created_at -- you would want the child table for next year to be created before it is needed. You'd may also want tables that are 10 years old to archived to s3 or simply dropped.

Multilevel versions of methods aren't available to component partitioned classes

For example, in a given lambda in one of the component partitioned classes:

lambda { |model, _partition_key_values|
model.partition_table_name(_partition_key_values)
}

The call to partition_table_name will fail, since the single-level partition represented by model doesn't know about the other single-level partitions in the multilevel.

add ability to update indexes and references

update_partition_table_index
update_references_to_partition_table
consider what it takes to handle indexes and references that have been removed or modified

A tiny plead: please tag the repository with each released version

Doing this makes it a lot easier to compare releases going forward. Maybe it's just a git push --tags that's missing?

Using a partition for an associated table (:includes => 'partition_name')

I was wondering if there was support to specify a partition table for an :includes eager loading so that I can do something like: (:includes => 'partition_name')

Thanks,
Steve

use attribute hooks to change partitioned name

if we can override these two methods and change the partitioned table name only when partition key values change, then we don't have to do much at all other sql build methods

active_record/attribute_methods/write.rb#write_attribute
active_record/attribute_methods/serialization.rb#initialize_attributes

Gem Abandoned?

Hi,

Looks like this gem is abandoned. There're 101 forks.

Anyone can point the most stable and maintained fork?

Maybe the author want another maintainer?

Thanks!

support for IntegerOrNull tables.

child table would be:

foos_partitions.p0
foos_partitions.p1
foos_partitions.p2
foos_partitions.pNULL

support for date time and other base types should be available.

SqlAdapter fails to add the insert redirector rule to intermediate level partitioned tables

If I created a multi-level partitioned table and call create_infrastructure/create_new_partitions, I get something like this:

employees
employees_partitions.p1
employees_partitions.p1_20130128
…

The employees table will have the rule, but the employees_partitions.p1 table will not.

Setting table_alias on an Arel::Table instance is persistant and impacts subsequent queries.

For example, if Employee is a partitioned model:

Employee.from_partition_without_alias(1).select('*').first
Employee.first

Will throw the following exception

PG::Error: ERROR:  syntax error at or near "." (ActiveRecord::StatementInvalid)

LINE 1: ...ns"."p1".* FROM "employees" "employees_partitions"."p1" LIM...
^
: SELECT "employees_partitions"."p1".* FROM "employees" "employees_partitions"."p1" LIMIT 1

I believe this can be fixed in the from_partition and from_partition_without_alias methods by returning a relation that is constructed from the partition's Arel::Table instead of the parent table's Arel::Table.

If we do this, we'll be creating a lot of Arel::Tables, so it might be wise to cache and re-use them. It looks like this was done in the past but no longer occurs?