sunbains / embedded-innodb Goto Github PK

It was supposed to cache non-unique secondary index inserts. It interferes with the buffer cache and complicates things and the value is moot.

Makes iterating over the UT_LIST similar to other STL containers.

The scheme in 8.x is to use a version number per page to check the staleness of a page. Maintain a version number in the table instance too. Don't delete the table instance in the data dictionary but move it to a deleted list. The version number is incremented when the tablespace is deleted. When selecting a victim from the LRU list or when flushing from the flush list check the staleness of the page for reuse or ignore during writes and them to the free list.

We could also add them to the free list when we find them during LRU or FLUSH list traversal for whatever reason.

Compile both Release and Debug with C++ 23.

This has some edge cases that took sometime to thrash out when it was implemented. The code idea is quite simple to implement.

The core work was done in v5.6 the later optimizations for RO were done in v5.7. The latter are fairly easy once the 5.6 changes are in place.

Remove all references to HOTBACKUP. Simplify the code layout.

I wrote this for FTS originally when the code was still in C and is past its use by date.

The original plan was to write to multiple redo log files or a group of files.. Where each fie can be on its own device to exploit parallel writes. The writing part is easy, the reading is a little bit more involved but doable. Parallel recovery should make that fairly straight forward .

The only downside I can think off is multiple fsyncs. Will it be a problem across multiple devices?

I think we should initially just remove it and then find a better way to do this after the new io_uring based model. Currently it can become a bottleneck under heavy IO load.

It depends on Parallel B+Tree read

Didn't run any recovery tests. Need write some tests, run a default suite before pushing.

They were a HUGE bottle neck after I split the kernel mutex in v5.6

Very useful in writing deterministic tests.

The modified LRU works well enough but there have been advances in this field since. There is ARC, LIRS and W TinyFLU to consider these days. With the simplified buffer pool code, we can experiment with these new schemes too.

Compact and later formats were a misguided attempt to reduce the size of the rows. They generates more redo, makes instant DDL and recovery difficult. The calculation of the offsets each time one has to access the row is a waste of CPU too. It makes the code unnecessarily complicated too. These were good ideas in 2004/5, better ways to solve this problem.

Better solution is to implement key prefix/suffix and better compression at the table-space level.

Currently all write go via the flush list and this causes mutex bottlenecks. Need to figure out a way to do write pages directly via the buffer pool especially during shutdown or bulk operations.

dullint was legacy code to implement 64 bit integers on 32 bit platforms.

The current scheme doesn't scale. We have to scan the entire log and there file headers. If we can recover the DD quickly then we don't have to scan the file headers for the tablespace ID to filename mapping etc.

Replace with a bloom filter or some other mechanism.

Build with GCOV

The truncate operation should be logged in the redo log and it should be possible to roll forward the operation on recovery if the redo log entry was committed to the redo log before the system crashed.

This is a tricky one. Not a priority.