Coder Social home page Coder Social logo

postamt's Introduction

Postamt

Gem Version

Postamt is a sane, solution for performing database reads against a hot standby server with Rails 4.1 and 4.2.

If you use Rails 3.2 or 4.0, use Postamt version 0.9.2.

Choose per model and/or controller&action whether a read-only query should be sent to master or a hot standby.
Inside a transaction reads always happen against master.

Care has been taken to avoid common performance pitfalls. It's been battle tested in production at sauspiel.de.

Monkey-patching is kept to an absolute minimum, the hard work happens through officially-supported Rails APIs. That's why there's so little code compared to similar gems.

Postamt requires Rails 3.2+ and works with Rails 4.

Installation

Add this line to your application's Gemfile:

gem 'postamt'

Example usage

# database.yml
development:
  adapter: postgresql
  database: app
  username: app
  password:
  host: master.db.internal
  encoding: utf8
  slave:
    host: slave.db.internal
    username: app_readonly
class UserController < ApplicationController
  use_db_connection :slave, for: ['User'], only: [:search]

  def search
    # SELECTs here are sent to slave
    # User#save and User.create would be sent to master anyways.
    # Everything in a transaction block too.
    @users = User.where(...) # sent to slave
    @something_else = SomethingElse.first # sent to master
  end

  def create
    @user = User.new(params[:user])
    @user.save! # sent to master
  end

  def invoice
    transaction do
      @user = User.where(...) # sent to master
      @invoices = Invoice.create(...) # sent to master
    end
  end
end
class ArchivedItem < ActiveRecord::Base
  # default_connection can be overwritten with
  # * Postamt.on(...) { ... },
  # * ActiveRecord::Base.transaction { ... }, and
  # * use_db_connection :other_connection, for: ['ArchivedItem'] in a controller.
  self.default_connection = :slave
end

User.where(...) # sent to master
item = ArchivedItem.where(...) # sent to slave
item.title = "changed title"
item.save! # sent to master
item.reload # sent to slave, beware of replication lag here!

ActiveRecord::Base.transaction do
  ArchivedItem.where(...) # sent to master, since we're in a transaction
  User.where(...) # sent to master
end

Postamt.on(:master) do
  ArchivedItem.where(...) # sent to master
  User.where(...) # sent to master
end
# If you don't want to test with a slave DB put this in config/environments/test.rb
Postamt.force_connection = :master

Tests

Create the DB postamt_test and ensure the users master and slave exist:

$ createdb postamt_test
$ createuser -s master # -s => superuser
$ createuser -s slave # better to restrict slave to be read-only

Migrate the DB in the Rails 4 app:

$ cd testapp
$ RAILS_ENV=test bundle exec rake db:migrate
$ bundle exec ruby -Itest test/integration/postamt_test.rb

You can't run the tests via a simple rake because Postamt deactivates itself when it detects that a task starting with 'db' is run (like db:test:prepare)

Contributing

  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request

postamt's People

Contributors

kommen avatar michaelglass avatar mk avatar msch avatar purintai avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

postamt's Issues

ActiveRecord::Relation.prepend Postamt::Relation results in recursion

I've experiencing problems while using Postamt and Grape together. For some reason, ActiveRecord::Relation.prepend Postamt::Relation results in a recursive call to delete_all method when destroying an object. I replaced the line with ActiveRecord::Relation.include Postamt::Relation and it seems to work fine. Can you please help me?

Destroying ActiveRecord objects results in SystemStackError when using newrelic_rpm

When having newrelic_rpm (version 3.15.0.314) as a dependency in a rails app together with postamt, destroying ActiveRecord records fails with an SystemStackError: stack level too deep.

The last stack frames look like:

from /Users/kommen/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/activerecord-4.2.6/lib/active_record/relation.rb:304:in `ensure in scoping'
    from /Users/kommen/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/activerecord-4.2.6/lib/active_record/relation.rb:304:in `scoping'
    from /Users/kommen/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/activerecord-4.2.6/lib/active_record/relation/delegation.rb:70:in `name'
    from /Users/kommen/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/newrelic_rpm-3.15.0.314/lib/new_relic/agent/instrumentation/active_record_helper.rb:53:in `delete_all'
    from /Users/kommen/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/postamt-0.9.7/lib/postamt.rb:121:in `block in delete_all'
    from /Users/kommen/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/postamt-0.9.7/lib/postamt.rb:19:in `on'
    from /Users/kommen/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/postamt-0.9.7/lib/postamt.rb:121:in `delete_all'
    from /Users/kommen/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/newrelic_rpm-3.15.0.314/lib/new_relic/agent/instrumentation/active_record_helper.rb:54:in `block in delete_all'
    from /Users/kommen/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/newrelic_rpm-3.15.0.314/lib/new_relic/agent.rb:574:in `with_database_metric_name'
    from /Users/kommen/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/newrelic_rpm-3.15.0.314/lib/new_relic/agent/instrumentation/active_record_helper.rb:53:in `delete_all'
    from /Users/kommen/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/postamt-0.9.7/lib/postamt.rb:121:in `block in delete_all'
    from /Users/kommen/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/postamt-0.9.7/lib/postamt.rb:19:in `on'
    from /Users/kommen/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/postamt-0.9.7/lib/postamt.rb:121:in `delete_all'
    from /Users/kommen/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/newrelic_rpm-3.15.0.314/lib/new_relic/agent/instrumentation/active_record_helper.rb:54:in `block in delete_all'
    from /Users/kommen/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/newrelic_rpm-3.15.0.314/lib/new_relic/agent.rb:574:in `with_database_metric_name'
    from /Users/kommen/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/newrelic_rpm-3.15.0.314/lib/new_relic/agent/instrumentation/active_record_helper.rb:53:in `delete_all'

The issue was not present with older newrelic_rpm versions (e.g. 3.9.0.229). This commit looks like it could be the cause: newrelic/newrelic-ruby-agent@9c267ff

Problem on Rails5 with 0.10.0

Rails versions:

  • rails: 5.0.7
  • rspec: 3.7
  • ruby: 2.3.0
  • postamt: 0.10.0

When I ran rspec for some models, it caused the following error.
Does anybody know postamt 0.10.0 can support Rails 5.0?

 Failure/Error: Postamt.force_connection || Postamt.connection_stack.last || Postamt.overwritten_default_connections[klass.name] || klass.default_connection || Postamt.default_connection
       
       NoMethodError:
         undefined method `name' for "primary":String
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/bundler/gems/postamt-d4ac81440aab/lib/postamt/connection_handler.rb:96:in `connection_for'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/bundler/gems/postamt-d4ac81440aab/lib/postamt/connection_handler.rb:104:in `pool_for'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/bundler/gems/postamt-d4ac81440aab/lib/postamt/connection_handler.rb:73:in `retrieve_connection_pool'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/bundler/gems/postamt-d4ac81440aab/lib/postamt/connection_handler.rb:51:in `retrieve_connection'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/activerecord-5.0.7/lib/active_record/connection_handling.rb:128:in `retrieve_connection'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/activerecord-5.0.7/lib/active_record/connection_handling.rb:91:in `connection'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/activerecord-5.0.7/lib/active_record/fixtures.rb:516:in `create_fixtures'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/activerecord-5.0.7/lib/active_record/fixtures.rb:1015:in `load_fixtures'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/activerecord-5.0.7/lib/active_record/fixtures.rb:988:in `setup_fixtures'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/activerecord-5.0.7/lib/active_record/fixtures.rb:852:in `before_setup'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-rails-3.7.2/lib/rspec/rails/adapters.rb:126:in `block (2 levels) in <module:MinitestLifecycleAdapter>'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.7.1/lib/rspec/core/example.rb:447:in `instance_exec'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.7.1/lib/rspec/core/example.rb:447:in `instance_exec'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.7.1/lib/rspec/core/hooks.rb:375:in `execute_with'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.7.1/lib/rspec/core/hooks.rb:608:in `block (2 levels) in run_around_example_hooks_for'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.7.1/lib/rspec/core/example.rb:342:in `call'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.7.1/lib/rspec/core/hooks.rb:609:in `run_around_example_hooks_for'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.7.1/lib/rspec/core/hooks.rb:466:in `run'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.7.1/lib/rspec/core/example.rb:457:in `with_around_example_hooks'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.7.1/lib/rspec/core/example.rb:500:in `with_around_and_singleton_context_hooks'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.7.1/lib/rspec/core/example.rb:251:in `run'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.7.1/lib/rspec/core/example_group.rb:628:in `block in run_examples'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.7.1/lib/rspec/core/example_group.rb:624:in `map'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.7.1/lib/rspec/core/example_group.rb:624:in `run_examples'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.7.1/lib/rspec/core/example_group.rb:590:in `run'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.7.1/lib/rspec/core/runner.rb:118:in `block (3 levels) in run_specs'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.7.1/lib/rspec/core/runner.rb:118:in `map'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.7.1/lib/rspec/core/runner.rb:118:in `block (2 levels) in run_specs'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.7.1/lib/rspec/core/configuration.rb:1926:in `with_suite_hooks'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.7.1/lib/rspec/core/runner.rb:113:in `block in run_specs'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.7.1/lib/rspec/core/reporter.rb:79:in `report'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.7.1/lib/rspec/core/runner.rb:112:in `run_specs'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.7.1/lib/rspec/core/runner.rb:87:in `run'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.7.1/lib/rspec/core/runner.rb:71:in `run'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.7.1/lib/rspec/core/runner.rb:45:in `invoke'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.7.1/exe/rspec:4:in `<top (required)>'
       # /usr/local/rbenv/versions/2.3.0/bin/rspec:23:in `load'
       # /usr/local/rbenv/versions/2.3.0/bin/rspec:23:in `<top (required)>'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/site_ruby/2.3.0/bundler/cli/exec.rb:74:in `load'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/site_ruby/2.3.0/bundler/cli/exec.rb:74:in `kernel_load'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/site_ruby/2.3.0/bundler/cli/exec.rb:28:in `run'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/site_ruby/2.3.0/bundler/cli.rb:424:in `exec'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/site_ruby/2.3.0/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/site_ruby/2.3.0/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/site_ruby/2.3.0/bundler/vendor/thor/lib/thor.rb:387:in `dispatch'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/site_ruby/2.3.0/bundler/cli.rb:27:in `dispatch'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/site_ruby/2.3.0/bundler/vendor/thor/lib/thor/base.rb:466:in `start'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/site_ruby/2.3.0/bundler/cli.rb:18:in `start'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/bundler-1.16.2/exe/bundle:30:in `block in <top (required)>'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/site_ruby/2.3.0/bundler/friendly_errors.rb:124:in `with_friendly_errors'
       # /usr/local/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/bundler-1.16.2/exe/bundle:22:in `<top (required)>'
       # /usr/local/rbenv/versions/2.3.0/bin/bundle:23:in `load'
       # /usr/local/rbenv/versions/2.3.0/bin/bundle:23:in `<main>'
       # 
       #   Showing full backtrace because every line was filtered out.
       #   See docs for RSpec::Configuration#backtrace_exclusion_patterns and
       #   RSpec::Configuration#backtrace_inclusion_patterns for more information.

I installed postamt 0.10.0 by like:

gem 'postamt', github: 'sauspiel/postamt', branch: 'rails5'

Breaks rake db:test:purge on rails4

Adding the postamt gem to a fresh rails4 rc1 project breaks rake db:test:purge with

PG::Error: ERROR:  cannot drop the currently open database
: DROP DATABASE IF EXISTS "testin_test"

Using multi-db by "establish_connection"

Hi,

I would like to use separate db for one model. Unfortunately when I use

establish_connection("another_db_#{Rails.env}"])

my app still use main db. When I commented out "development" settings in database.yml, Postamt return me:

ActiveRecord::ConnectionNotEstablished: ActiveRecord::ConnectionNotEstablished

in

gems/postamt-0.9.0/lib/postamt/connection_handler.rb:53:in `retrieve_connection'

Any ideas how to use "establish_connection" simultaneously with Postamt?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.