Comments (6)
Hi @Alphacode18
In order to use concurrent IOs, you need to enable async multitasking at Navy layer (navy-async) by enabling NavyRequestScheduler
; refer to this and this for details. Note that qdepth will be set automatically if you enable navy-async (ref).
If you override the qdepth with some value >1 without enabling navy-async, you would have hit this assertion in debug build.
from cachelib.
Hi @jaesoo-fb
Thank you for your prompt response. I see. I am now able to set qDepth
automatically by varying the navyMaxNumReads
and navyMaxNumWrites
, with navyReaderThreads
and navyWriterThreads
set to 1.
Would you know if NavyRequestScheduler
is a parameter I can set using the config file? Maybe I am missing something, but I can't seem to set it correctly.
Thank you for all your help!
from cachelib.
@Alphacode18 NavyRequestScheduler
(async) as opposed to OrderedThreadPoolScheduler
is activated if you provide non-zero values for navyMaxNumReads
and navyMaxNumWrites
. See this
from cachelib.
Oh I see! Thanks. Unfortunately, in submitIo
, I still see numOutstanding_
oscillate between 0 and 1 (similar to the log above). Is that the correct place to log? If not, could you point me to the right place? I just want to verify that I am indeed seeing numOutstanding_
roughly similar to qDepth
.
Here's my updated config:
{
"cache_config" : {
"cacheSizeMB" : 100,
"numPools" : 1,
"nvmCacheSizeMB": 40960,
"nvmCachePaths": ["/mnt/nvme0n1/cachelib/testfile"],
"navyEnableIoUring": false,
"navyBlockSize": 4096,
"navyMaxNumReads": 16,
"navyMaxNumWrites": 16,
"navyReaderThreads": 1,
"navyWriterThreads": 1
},
"test_config" : {
"enableLookaside": true,
"numThreads" : 1,
"numKeys" : 1000000,
"numOps" : 5000000,
"distribution" : "range",
"generator": "workload",
"keySizeRange" : [16, 16],
"keySizeRangeProbability" : [1],
"valSizeRange" : [4096, 4096],
"valSizeRangeProbability" : [1],
"setRatio" : 0.0,
"delRatio" : 0.0,
"loneGetRatio" : 0.0,
"getRatio" : 1
}
}
from cachelib.
@Alphacode18 Navy configuration looks correct, but stressor configuration looks not; you are using only 1 thread (numThreads
), meaning there will only be at most one outstanding cachelib requests.
from cachelib.
Oh, I see. In this case, may I ask three questions:
- How do I set
qDepth
independent to the number of threads (and havecachelib
enforce that, i.e., maintain those many requests in-flight per thread?) - I observe now that
qDepth
is always upper-bounded by thenumThreads
(i.e., I can't have I/O depth 8 ifnumThreads
= 1), how can I get rid of this upper bound? - Also, If I launch 8 threads with my configuration, how can I explicitly allot them particular CPU cores?
Thank you for all your help!
from cachelib.
Related Issues (20)
- Build failure - cannot find python-six HOT 2
- build.sh: error: failed to build cachelib HOT 2
- build.sh: error: No build recipe for detected operating system 'fedora38' HOT 3
- New runtime error HOT 5
- cachelib fails to build HOT 3
- allocator-test-AllocationClassTest Failed On ARM
- Where is `cachelib/facebook/admin/CacheAdmin.h`? HOT 1
- cachelib fails to build HOT 3
- refCount alignment issue HOT 4
- The method config.enableChainedItems() cannot be found. HOT 1
- Some questions in resizing the cachelib pool size HOT 2
- Is there any plan to provide an Java SDK for this cachelib ? HOT 2
- Fail to build dependency fbthrift (with errors reported in fmt) HOT 4
- make clean option for contrib/build.sh HOT 1
- build error about fizz on ubuntu22.04 HOT 2
- CDN trace expected behavior HOT 2
- Enable FDP for CacheBench HOT 12
- Questions about trace files when running cachebench HOT 2
- Running simple-cache-example gives an error, flag 'v' was defined more than once HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cachelib.