Comments (6)
Comment by spetrunia
Friday Sep 04, 2015 at 20:50 GMT
Possible solutions:
- Fix facebook/rocksdb#616. This might be complex.
- We only care about the scenario where SQL layer does Deletes/Updates on the fly. This means, SQL layer will request to delete the row that was just read. Rdb_transaction could catch the Delete operation and before the iterator is invalidated, advance it so it is no longer pointing at the row that's about to be deleted.
- A storage engine may implement Bulk Update/Delete API. This is not a full solution because there are cases where bulk update/delete is not used.
from mysql-5.6.
Comment by yoshinorim
Saturday Sep 05, 2015 at 02:09 GMT
@spetrunia : According to Siying, it is safe to mutable WriteBatchWithIndex while iterating it if using in a right way (the most important thing is to operate on a copy of the key when changing the state). Here is a unit test to verify it -- https://github.com/facebook/rocksdb/blob/3.13.fb/utilities/write_batch_with_index/write_batch_with_index_test.cc#L1247-L1366
from mysql-5.6.
Comment by siying
Tuesday Sep 08, 2015 at 17:40 GMT
@igorcanadi added a test case for mutating WriteBatchWithIndex: https://reviews.facebook.net/D39501
So you should be able to add/remove keys from it. You need to be careful about the way to do it. For example, the way will not work:
rocksdb::Slice rowkey= iter->key();
trx->Delete(rowkey); // (*)
iter->Prev(); // (**)
rowkey
is a reference to memory location. In our implementation issuing Delete() with it will cause problem as it changes in the middle of deletion and cause wrong results. But if you do:
std::string rowkey= iter->key();
trx->Delete(rowkey); // (*)
iter->Prev(); // (**)
It will work.
from mysql-5.6.
Comment by spetrunia
Tuesday Sep 08, 2015 at 17:58 GMT
@yoshinorim @siying thanks for clarification. My example was overly simplified, in the actual code MyRocks copies away the key/value it has got from the iterator.
from mysql-5.6.
Comment by spetrunia
Tuesday Sep 08, 2015 at 17:59 GMT
https://reviews.facebook.net/D45873 (Issue #86 patch) is now updated to make use of this
from mysql-5.6.
@spetrunia I think this is no longer an issue and #70 is tracking the rest performance issue with large data sets. Could you close this issue?
from mysql-5.6.
Related Issues (20)
- How to achieve ~20% replication throughput improvement using Read Free Replication (RFR) feature
- Keep long-running MyRocks mtr tests in their own suite HOT 1
- MyRocks 8.0.28 has poor performance of primary key range query HOT 1
- create secondary index needs attention
- issue while install HOT 1
- Error during create secondary index
- Update the Build Steps page HOT 1
- MyRocks engine should respect WITH_UNIT_TESTS
- Allow users to manually set the number of block cache shards HOT 1
- Provide a counter to show pending compaction bytes for RocksDB
- MyRocks does commit step non-durably under server group 2PC protocol HOT 1
- alter talbe add index optimization
- Range lock support HOT 2
- Cached RocksDB transaction object accessed after delete by XA COMMIT
- optimize table has no effect on HIDDEN_PK table
- Rdb_iterator_base::next_with_direction: too many compares for eof check HOT 2
- Determin secondary index value emptiness by datadic
- undefined reference to `sgemm_' When compiling vector DB
- -DWITH_LZ4=system doesn't find the lz4 library on the fb-mysql-8.0.28 branch
- is_thd_db_read_only_by_name CPU overhead can be reduce
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mysql-5.6.