Comments (15)
Comment by jkedgar
Wednesday Sep 09, 2015 at 17:03 GMT
Does this mean that you want a global variable in MyRocks so that you can configure this option by using "Set rocksdb_bytes_per_sync = "?
What other options should be this way?
from mysql-5.6.
Comment by yoshinorim
Wednesday Sep 09, 2015 at 17:27 GMT
I'd like to start from dynamically configurable DB options. Static options such as wal_dir, rocksdb_data_dir are not targets. For CF/Table options, I think we need more discussions.
from mysql-5.6.
Comment by jkedgar
Wednesday Sep 09, 2015 at 17:31 GMT
Is there a list of dynamically configurable options?
from mysql-5.6.
Comment by yoshinorim
Wednesday Sep 09, 2015 at 17:39 GMT
from mysql-5.6.
Comment by siying
Wednesday Sep 09, 2015 at 17:41 GMT
Search options.h and all dynamic configurable options will write in comments "dynamic changeable".
from mysql-5.6.
Comment by jkedgar
Wednesday Sep 09, 2015 at 18:07 GMT
The SetOptions() method appears to operate on a column family - the default column family if no column family is specified. Do I need to iterate through all the column families when an option is set via MySQL or do I just set it for the default column family?
The bytes_per_sync setting does not appear to be dynamically configurable. It is not listed as such in options.h and doesn't show up in util/options_helper.cc. Am I missing something?
from mysql-5.6.
I didn't receive any comments on my questions. The questions are again:
-
The request is for options like rocksdb_bytes_per_sync to be configurable via MySQL, but this option is not dynamically configurable (the only dynamically configurable options appear to be specified in rocksdb/util/options_helper.cc and 'bytes_per_sync' is not listed). Should I just ignore this and do all the other dynamically configured options?
-
The SetOptions() command in RocksDB is specific to a column family. Should the option change occur on just the default column family or should it occur on all column families associated with the database? I'm guessing that is should apply to all, but would like confirmation of that.
from mysql-5.6.
cc @yoshinorim
from mysql-5.6.
-
Some parameters like Rate Limiter is not configurable via options, but has an interface at util/rate_limiter.h::SetBytesPerSecond(). So some special handling is needed to configure Rate Limiter settings. I hope there aren't many parameters like it.
-
Is CF handle needed to change DB options too? I think we should start from dynamic DB options. We won't need dynamic CF options very much. You'd better talk with Siying and Venky if there is no reasonable api to change DB options.
from mysql-5.6.
- Maybe I'm not understanding, but the DB::SetOptions() method takes a column family as the first argument. There is a version that doesn't, but it just calls the other version with DefaultColumnFamily(). I'll also try to talk to Siying and Venkey.
from mysql-5.6.
DBOptions are not dynamically changeable. eg. bytes_per_sync and rate_limiter
Dynamic options per column family have to be changed for each column family.
from mysql-5.6.
So there are database-wide options that are not dynamically changeable and there are some column family options which are dynamically changeable. Here's a list of the options that I see that are dynamically configurable:
Memtable options:
- write_buffer_size
- arena_block_size
- memtable_prefix_bloom_bits
- memtable_prefix_bloom_probes
- memtable_prefix_bloom_huge_page_tlb_size
- max_successive_merges
- filter_deletes
- max_write_buffer_number
- inplace_update_num_locks
Compaction options:
- disable_auto_compactions
- soft_rate_limit
- hard_pending_compaction_bytes_limit
- hard_rate_limit (deprecated)
- level0_file_num_compaction_trigger
- level0_slowdown_writes_trigger
- level0_stop_writes_trigger
- max_grandparent_overlap_factor
- expanded_compaction_factor
- source_compaction_factor
- target_file_size_base
- target_file_size_multiplier
- max_bytes_for_level_base
- max_bytes_for_level_multiplier
- max_bytes_for_level_multiplier_additional (input is an array - difficult in MySQL)
- verify_checksums_in_compaction
Miscellaneous options:
- max_sequential_skip_in_iterations
- paranoid_file_checks
How many of these make sense to change via MySQL 'SET' statements? And should they be global across the entire database or should we have some way of setting them for different column families (indexes)?
from mysql-5.6.
After getting confirmation that I do have to update each column family to set any of these values as well as information that the rate limiter option is set-able using a different method, I've been thinking more about how this will work.
My assumption is that calling SET <variable> = <value>
would not persist beyond the life of the server (i.e. rebooting the mysqld process would cause this information to be forgotten). Does this sound correct?
Also, if I think I probably need to keep track of the current set options for the database so that if a new column family is created (via a new index), I can set the options for this column family to match the rest of them. Does that sound correct?
from mysql-5.6.
In MySQL-land SET options only exist for the life of the process. MySQL
operations is happy with that as they avoid a mysqld restart thanks to SET,
and in the background do their magic to update my.cnf files
On Wed, Sep 30, 2015 at 9:28 AM, Jay Edgar [email protected] wrote:
After getting confirmation that I do have to update each column family to
set any of these values as well as information that the rate limiter option
is set-able using a different method, I've been thinking more about how
this will work.My assumption is that calling SET = would not persist beyond the life of
the server (i.e. rebooting the mysqld process would cause this information
to be forgotten). Does this sound correct?Also, if I think I probably need to keep track of the current set options
for the database so that if a new column family is created (via a new
index), I can set the options for this column family to match the rest of
them. Does that sound correct?—
Reply to this email directly or view it on GitHub
#43 (comment).
Mark Callaghan
[email protected]
from mysql-5.6.
https://reviews.facebook.net/D48339
from mysql-5.6.
Related Issues (20)
- force_flush_memtable_and_lzero_now always compacts to physical L1
- Make compilation faster HOT 1
- How to achieve ~20% replication throughput improvement using Read Free Replication (RFR) feature
- Keep long-running MyRocks mtr tests in their own suite HOT 1
- MyRocks 8.0.28 has poor performance of primary key range query HOT 1
- create secondary index needs attention
- issue while install HOT 1
- Error during create secondary index
- Update the Build Steps page HOT 1
- MyRocks engine should respect WITH_UNIT_TESTS
- Allow users to manually set the number of block cache shards HOT 1
- Provide a counter to show pending compaction bytes for RocksDB
- MyRocks does commit step non-durably under server group 2PC protocol HOT 1
- alter talbe add index optimization
- Range lock support HOT 2
- Cached RocksDB transaction object accessed after delete by XA COMMIT
- optimize table has no effect on HIDDEN_PK table
- Rdb_iterator_base::next_with_direction: too many compares for eof check HOT 2
- Determin secondary index value emptiness by datadic
- undefined reference to `sgemm_' When compiling vector DB
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mysql-5.6.