Kodama is a MySQL replication listener based on ruby-binlog. Kodama provides a simple DSL to easily write your own replication listener.
- Provides simple DSL for writing binlog event handlers
- Automatically restarts from the saved binlog position
- Attempts to reconnect to MySQL when the connection is somehow teminated
These features allow developers to focus on writing their own replication logic rather than having to spend time figuring things out.
Kodama can be used to replicate MySQL updates to other data stores, arbitrary software or even a flat file. The sole purpose of Kodama is to provide a convenient way to reflect the database updates to other components in your system.
- Replicate from MySQL to Postgres
- Replicate between tables with different schema
- Sync production data to development DB while masking privacy information
- Realtime full text index update
This gem links against MySQL's libreplication C shared library. You need to first install the mysql-replication-listener package.
But official repository has some bugs. It is recommended to use winebarrel's patched version (There are rpm package and homebrew formula).
Add this line to your application's Gemfile:
gem 'kodama'
And then execute:
$ bundle
Or install it yourself as:
$ gem install kodama
It is recommended to set the mysqld binlog_format option to ROW. This is because the ROW format allows Kodama to pickup every single updates made to the database.
SET GLOBAL binlog_format = 'ROW';require 'kodama'
Kodama::Client.start(:host => '127.0.0.1', :username => 'user') do |c|
c.binlog_position_file = 'position.log'
c.log_level = :info # [:debug|:info|:warn|:error|:fatal]
c.connection_retry_limit = 100 # times
c.connection_retry_wait = 3 # second
# Exit gracefully when kodama receives specified signals
c.gracefully_stop_on :QUIT, :INT
c.on_query_event do |event|
p event.query
end
c.on_row_event do |event|
p event.rows
end
endrequire 'rubygems'
require 'kodama'
require 'json'
require 'redis'
class Worker
def initialize
@redis = Redis.new
end
def perform(event)
record_id = get_row(event)[0] # first column is id
@redis.set "#{event.table_name}_#{record_id}", event.rows.to_json
end
def get_row(event)
case event.event_type
when /Write/, /Delete/
event.rows[0] # [row]
when /Update/
event.rows[0][1] # [[old_row, new_row]]
end
end
end
worker = Worker.new
Kodama::Client.start(:host => '127.0.0.1', :username => 'user') do |c|
c.binlog_position_file = 'position.log'
c.on_row_event do |event|
worker.perform(event)
end
endSets the filename to save the binlog position. Kodama will read this file and resume listening from the stored position.
Set logger's log level.
It accepts :debug, :info, :warn, :error, :fatal.
If for some reason the connection to MySQL is terminated, Kodama will attempt to reconnect connection_retry_limit times, while waiting connection_retry_wait seconds between attempts.
Kodama traps specified signals and stop gracefully. It accpets multiple signals like following.
Kodama::Client.start do |c|
c.gracefully_stop_on :INT, :QUIT
endYusuke Mito, Genki Sugawara
- Fork it
- Create your feature branch (
git checkout -b my-new-feature) - Commit your changes (
git commit -am 'Add some feature') - Push to the branch (
git push origin my-new-feature) - Create new Pull Request
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent