hokstadconsulting / purecdb

A Pure Ruby CDB reader/writer w/64 bit extensions

PureCDB

A Pure Ruby CDB reader/writer w/64 bit extensions

For information about CDB, see: http://cr.yp.to/cdb.html

The motivation for writing this was:

• Bernstein's CDB format can only handle files up to 4GB. For a past project we needed a simple CDB style file for datasets several times that.

• The C library is under a license that prevents us from releasing modified versions of it, but the format is so simple that writing our own reader and writer was easy.

• We don't like depending on C extensions for Ruby code if we don't have to.

Installation

Add this line to your application's Gemfile:

gem 'purecdb'

And then execute:

$ bundle

Or install it yourself as:

$ gem install purecdb

Basic Usage

To create a 32 bit (standard) CDB file:

    PureCDB::Writer.open("/tmp/somecdbfile.cdb") do |cdb| 
     cdb.store("key","value")
    end

To instead create a 64 bit file, pass {mode: 64} as the second argument to PureCDB::Writer#open .

To read a CDB file (auto-detecting standard 32-bit or extended 64-bit) CDB files:

    PureCDB::Reader.open("/tmp/somecdbfile.cdb") do |r|
       p r.values("key")
    end

To require a 32 or 64 bit file specifically, pass {mode: 32} or {mode: 64} as the second argument to PureCDB::Reader#open.

See PureCDB::Reader#new for additional usage.

64-bit Format

The 64 bit file format follows http://cr.yp.to/cdb/cdb.txt except that any reference to 32-bit should be replaced by 64-bit, and that a 64 bit file ends with the magic cookie "cdb64:01"

Development

After checking out the repo, run bin/setup to install dependencies. Then, run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release to create a git tag for the version, push git commits and tags, and push the .gem file to rubygems.org.

To run the Rspec tests, you need tinycdb or a command-line compatible implementation installed for interoperability tests.

Contributing

1. Fork it ( https://github.com/hokstadconsulting/purecdb/fork )
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create a new Pull Request

purecdb's Issues

CDB script

Hi

Do you have a copy of the mentioned perl script that populates the CDB database? I tried to follow the described format with Ruby, but I'm getting a segfault running rum:

require 'purecdb'

PureCDB::Writer.open("foo.cdb", mode: 64) do |cdb|
  cdb.store("myuser","*5C224C5BA2FC753C839B9C47CE8D71DFA597EBDD\x000tcp:127.0.0.1\x000")
end

$ sudo ./rum -s tcp:localhost:3307 -s sock:/var/run/mysqld/mysqld.sock -M ./foo.cdb
listening on tcp:localhost:3307
listening on sock:/var/run/mysqld/mysqld.sock
zsh: segmentation fault (core dumped)  sudo ./rum -s tcp:localhost:3307 -s sock:/var/run/mysqld/mysqld.sock -M 

strace output doesn't seem very useful:

open("./foo.cdb", O_RDONLY)             = 8
fstat(8, {st_mode=S_IFREG|0664, st_size=4342, ...}) = 0
mmap(NULL, 4342, PROT_READ, MAP_SHARED, 8, 0) = 0x7f37fd0f3000
rt_sigaction(SIGUSR1, {0x7f37fccacd50, ~[RTMIN RT_1], SA_RESTORER|SA_RESTART, 0x7f37fc6f14b0}, {SIG_DFL, [], 0}, 8) = 0
epoll_ctl(5, EPOLL_CTL_ADD, 7, {EPOLLIN, {u32=7, u64=7}}) = 0
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0} ---
+++ killed by SIGSEGV (core dumped) +++

Thanks.