pmem / pmem-rocksdb Goto Github PK

A version of RocksDB that uses persistent memory

License: GNU General Public License v2.0

CMake 0.42% Makefile 0.77% Python 2.03% Shell 1.21% PHP 0.05% Perl 1.44% PowerShell 0.09% C++ 80.98% C 1.83% Java 11.07% Dockerfile 0.01% JavaScript 0.02% Assembly 0.08% Starlark 0.01%

pmem-rocksdb's People

Contributors

Stargazers

Watchers

pmem-rocksdb's Issues

mapped region in DCPMMMmapFile never unmap

Note: Please use Issues only for bug reports. For questions, discussions, feature requests, etc. post to dev group: https://groups.google.com/forum/#!forum/rocksdb or https://www.facebook.com/groups/rocksdb.dev

In DCPMMMmapFile:
Memory regions mapped by MapNewRegion() are never unmapped.
UnmapCurrentRegion() is not implemented and not called when map new region or close the file.

pmem-rocksdb/env/env_dcpmm.cc

Lines 58 to 60 in b667ce5

    
           IOStatus DCPMMMmapFile::UnmapCurrentRegion() { 
        
             return IOStatus::OK(); 
        
           }

Actual behavior

When running pmem-rocksdb with large volumn of data, pmem_map_file() in MapNewRegion() failed because the mapped regions are too much.

pmem-rocksdb/env/env_dcpmm.cc

Lines 77 to 83 in b667ce5

    
           base_ = (char*)pmem_map_file(filename_.c_str(), file_offset_+map_size_, PMEM_FILE_EXCL, 0666, &mmap_len, &is_pmem); 
        
           if(base_ == nullptr){ 
        
             base_ = (char*)pmem_map_file(filename_.c_str(), file_offset_+map_size_, PMEM_FILE_CREATE, 0666, &mmap_len, &is_pmem); 
        
             if(base_ == nullptr){ 
        
               return IOStatus::IOError("dcpmm map file error"); 
        
             } 
        
           }

MAP_SYNC destroys performance

Hi!

I am running rocksdb on a ext4 dax system in kernel 5.1.
I am testing it with YCSB workloads, and I enable dcpmm wal.
I noticed that the MAP_SYNC flag destroys scalability due to ext4 journaling overheads, and the overall performance is worse than using default rocksdb.

Has anyone else experience this?
Is this expected?

Best Regards,
Chloe

Note: Please use Issues only for bug reports. For questions, discussions, feature requests, etc. post to dev group: https://groups.google.com/forum/#!forum/rocksdb or https://www.facebook.com/groups/rocksdb.dev

Expected behavior

Actual behavior

Steps to reproduce the behavior

the detailed design of pmem-rocksdb

I want to know the detailed design of pmem-rocksdb, but I can't find it on the Internet, How it optimizes rocksdb with nvm
? Is there any relevant documentation? Thank you very much.

pmem / pmem-rocksdb Goto Github PK

pmem-rocksdb's People

Contributors

Stargazers

Watchers

Forkers

pmem-rocksdb's Issues

mapped region in DCPMMMmapFile never unmap

Actual behavior

MAP_SYNC destroys performance

Expected behavior

Actual behavior

Steps to reproduce the behavior

the detailed design of pmem-rocksdb

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

	IOStatus DCPMMMmapFile::UnmapCurrentRegion() {
	return IOStatus::OK();
	}

	base_ = (char*)pmem_map_file(filename_.c_str(), file_offset_+map_size_, PMEM_FILE_EXCL, 0666, &mmap_len, &is_pmem);
	if(base_ == nullptr){
	base_ = (char*)pmem_map_file(filename_.c_str(), file_offset_+map_size_, PMEM_FILE_CREATE, 0666, &mmap_len, &is_pmem);
	if(base_ == nullptr){
	return IOStatus::IOError("dcpmm map file error");
	}
	}