Comments (3)
@gburd @jonmeredith @justinsheehy @jtuple Opinions?
Usage:
./this-script 1
./this-script 2
Run this script at top of compiled bitcask source (see line 3 for -pz path).
If you don't see the Merged info report before phase 1 is done, then alter
the sleep time beyond the hardcoded 7 seconds.
#!/usr/bin/env escript
%%
%%! -pz ./ebin
main(["1"]) ->
io:format("Deleting...\n"),
os:cmd("rm -rf data"),
io:format("Writing...\n"),
Reference = bitcask:open("data", [read_write | opts()]),
bitcask:put(Reference, <<"key_to_delete">>, <<"tr0ll">>),
[ bitcask:put(Reference, term_to_binary(X), <<1:(8 * 1024 * 100)>>) || X <- lists:seq(1, 3000)],
bitcask:delete(Reference, <<"key_to_delete">>),
[ bitcask:put(Reference, term_to_binary(X), <<1:(8 * 1024 * 100)>>) || X <- lists:seq(1, 3000)],
timer:sleep(1000 + 1000),
bitcask_merge_worker:merge("data", opts(), ["data/2.bitcask.data"]),
io:format("Sleeping...\n"),
timer:sleep(7*1000),
io:format("\nIf you see 'Merged' INFO REPORT, run this script with '2' arg\n");
main(["2"]) ->
Reference = bitcask:open("data", [read_write | opts()]),
io:format("Read zombie: ~p\n", [bitcask:get(Reference, <<"key_to_delete">>)]).
opts() ->
[{max_file_size, 268435456},
{dead_bytes_threshold, 89478485},
{dead_bytes_merge_trigger, 178956970}].
from bitcask.
gburd@carbon:/eng/bitcask$ ./this-script 1/eng/bitcask$ ./this-script 2
Deleting...
Writing...
Sleeping...
=INFO REPORT==== 12-Mar-2013::21:31:33 ===
Merged ["data",
[{max_file_size,268435456},
{dead_bytes_threshold,89478485},
{dead_bytes_merge_trigger,178956970}],
["data/2.bitcask.data"]] in 4.807972 seconds.
If you see 'Merged' INFO REPORT, run this script with '2' arg
gburd@carbon:
Read zombie: {ok,<<"tr0ll">>}
@gregburd | Basho Technologies | Riak | http://basho.com | @basho
On Tue, Mar 12, 2013 at 8:25 PM, Scott Lystig Fritchie <
[email protected]> wrote:
@gburd https://github.com/gburd @jonmeredithhttps://github.com/jonmeredith
@justinsheehy https://github.com/justinsheehy @jtuplehttps://github.com/jtupleOpinions?Usage:
./this-script 1
./this-script 2Run this script at top of compiled bitcask source (see line 3 for -pz
path).
If you don't see the Merged info report before phase 1 is done, then alter
the sleep time beyond the hardcoded 7 seconds.#!/usr/bin/env escript
%%
%%! -pz ./ebinmain(["1"]) ->
io:format("Deleting...\n"),
os:cmd("rm -rf data"),
io:format("Writing...\n"),
Reference = bitcask:open("data", [read_write | opts()]),
bitcask:put(Reference, <<"key_to_delete">>, <<"tr0ll">>),
[ bitcask:put(Reference, term_to_binary(X), <<1:(8 * 1024 * 100)>>) || X <- lists:seq(1, 3000)],
bitcask:delete(Reference, <<"key_to_delete">>),
[ bitcask:put(Reference, term_to_binary(X), <<1:(8 * 1024 * 100)>>) || X <- lists:seq(1, 3000)],
bitcask_merge_worker:merge("data", opts(), ["data/2.bitcask.data"]),
io:format("Sleeping...\n"),
timer:sleep(7*1000),
io:format("\nIf you see 'Merged' INFO REPORT, run this script with '2' arg\n");
main(["2"]) ->
Reference = bitcask:open("data", [read_write | opts()]),
io:format("Read zombie: ~p\n", [bitcask:get(Reference, <<"key_to_delete">>)]).opts() ->
[{max_file_size, 268435456},
{dead_bytes_threshold, 89478485},
{dead_bytes_merge_trigger, 178956970}].—
Reply to this email directly or view it on GitHubhttps://github.com//issues/82#issuecomment-14814640
.
from bitcask.
Possible fixes are brewing in https://github.com/basho/bitcask/compare/slf-merge-panopticon. Sorry about the delay, this is a thorny problem full of race conditions with merge operations, unfortunately. No guarantee yet about when this code might be ready.
from bitcask.
Related Issues (20)
- bitcask NIF doesn't compile cleanly on OS X 10.10 [JIRA: RIAK-1574] HOT 2
- Startup crash trying to read key from corrupted data file [JIRA: RIAK-1819] HOT 5
- Repeated crashes during merge operations [JIRA: RIAK-1903] HOT 4
- Can't manually force a merge, error - efile_driver_unavailable [JIRA: RIAK-2036] HOT 5
- Include Bitcask data file count in merge info messages. [JIRA: RIAK-2039]
- Potential Process Leak [JIRA: RIAK-2227] HOT 2
- Lock file staleness check could be more robust [JIRA: RIAK-2230]
- question about read concurrency [JIRA: RIAK-2330] HOT 1
- Operation aborts if any file contains a corrupt key [JIRA: RIAK-2371] HOT 1
- rebar abort [JIRA: RIAK-2485] HOT 5
- Errors in `merge_single_tombstone` and `open_file` `append` mode [JIRA: RIAK-2733] HOT 2
- bitcask:merge/1 keeps file descriptors of deleted files open [JIRA: RIAK-2814] HOT 2
- File Open behaves differently between NIF and Erlang modes [JIRA: RIAK-2895] HOT 1
- bitcask failed to merge.merge_locked,locked
- c_src/bitcask_nifs.c: pair of bad % specifiers ?
- Build broken with OTP 21: use of the deprecated get_stacktrace/0 in dependency meck HOT 3
- Call bitcask_merge_worker:merge(DirName) is error
- How to run YCSB workload for bitcask?
- In the bitcask intro paper we see it being mentioned that bitcask is able to support datasets larger than 10 X system memory. I am trying to understand how? HOT 4
- Recreate hintfile HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bitcask.