Coder Social home page Coder Social logo

some questions about paper about kvell HOT 9 OPEN

suxiao111 avatar suxiao111 commented on August 20, 2024
some questions about paper

from kvell.

Comments (9)

BLepers avatar BLepers commented on August 20, 2024

Hi,

Thanks for your interest.

We modified the db_bench tool to run YCSB in RocksDB. The code of the bench is here: https://github.com/theoanab/SILK-USENIXATC2019 (tools/db_bench_tools.cc). We configured the number of injection/compaction threads to saturate the CPU (number of threads must be significantly higher than number of cores).

For the ratios, we used perf. Percentages are directly what's reported by perf report.

from kvell.

suxiao111 avatar suxiao111 commented on August 20, 2024

Hi,
ok, I've got it. Thank you very much for your reply!

from kvell.

suxiao111 avatar suxiao111 commented on August 20, 2024

Hi,
In your paper,Figure 3 shows the disk utilization and the CPU utilization on Config-Optane for RocksDB and Wired Tiger. Both systems saturate the CPU and do not take full advantage of the available bandwidth. And Figure 4 shows the throughput fluctuation over time for RocksDB and for WiredTiger.I have some questions:

  1. What is the relationship between the bandwidth in Figure 3 and the throughput in figure 4?How is the maximum IO bandwidth value obtained in Figure 3?
    2.Figure 4 shows that RocksDB sustains on average 63K requests/s ,and the KV-item size is1KB ,so RocksDB sustains on average 63KB/s.But Figure 3 shows that RocksDB sustains on average 600KB/s.

Thank you very much~

from kvell.

BLepers avatar BLepers commented on August 20, 2024

1/ We measure the IO bw using iostat, and throughput directly in the systems. When IO goes up, throughput goes down.
2/ The 63MB/s -> 600MB/s is due to compactions (write amplification is a good keyword to search for :) )

from kvell.

suxiao111 avatar suxiao111 commented on August 20, 2024

Yeah, thanks,write magnification is a serious matter, but the experimental setup and the benchmark in figure 3 and figure 4 is the same.They just have different experimental observation indexes(bw&throught).Because the KV-item size is1KB ,so I think the two values should be equal?

from kvell.

BLepers avatar BLepers commented on August 20, 2024

If RocksDB has a 10x write amplification, then 1KB translate into 10KB used BW. That's what we measure (disk level bandwidth). Or am I missing something in your question?

from kvell.

suxiao111 avatar suxiao111 commented on August 20, 2024

Yeah, thanks,Maybe I'm not making it clear.
Actually, The experimental setup in figure 3 and figure 4 is the same,including same system and same benchmark.
Take RocksDB for example, Figure 3 shows that RocksDB sustains on average 600KB/s.But Figure 4 shows that RocksDB sustains on average 600KB/s?

from kvell.

BLepers avatar BLepers commented on August 20, 2024

I am not sure to understand your question. One figure shows the throughput (requests/s), and the other one the disk usage (MB/s). The disk usage is due to RocksDB writing updates (commit log + flushing the memory component), and performing compaction. You cannot guess the throughput from the disk usage or vice versa, it is very dependant on the workloads and timings. For instance, if a key is updated twice, the disk usage will be lower than if two different keys are updated, but the throughput will be the same. Not sure what you are trying to measure here.

from kvell.

suxiao111 avatar suxiao111 commented on August 20, 2024

Good.! Your example is very specific. I understand what you mean.That's a good idea!
Thank you very much~

from kvell.

Related Issues (11)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.