Comments (21)
from libcachesim.
Memory on the server.
root@dbms:~# free -h
total used free shared buff/cache available
Mem: 62G 475M 62G 16K 148M 61G
Swap: 9.0G 102M 8.9G
from libcachesim.
from libcachesim.
I tried to figure out the stack with gdb, but it says I can't.
So I looked at the problem with dmeseg.
(gdb) p main
$1 = {int (int, char **)} 0x555555554c90
(gdb) r
Starting program: /root/libCacheSim/_build/c18.back.out
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
filename = /hdd/cluster22.000
Program terminated with signal SIGKILL, Killed.
The program no longer exists.
(gdb) bt
No stack.
(gdb)
[188077.776581] [14560] 0 14560 20101354 16210629 150704128 2311952 0 c18.back.out
[188077.776582] [14597] 0 14597 1546 16 53248 0 0 sleep
[188077.776584] Out of memory: Kill process 14560 (c18.back.out) score 956 or sacrifice child
[188077.777441] Killed process 14560 (c18.back.out) total-vm:80405416kB, anon-rss:64842516kB, file-rss:0kB, shmem-rss:0kB
I don't know why it holds 80gb...
Is the test.c file that I wrote wrong...?
from libcachesim.
is the problem solved?
from libcachesim.
No, some run for each trace and others don't. I haven't solved it yet.
from libcachesim.
How do you setup the reader?
from libcachesim.
I wrote the test in this way and opened the reader in various formats.
#include <libCacheSim.h>
#define NUM_TRACE_FILES 88
#define MEASUREMENT_FILES 72
int main(int argc, char *argv[]) {
unsigned long total = 0;
long lru_miss_cnt, fifo_miss_cnt;
int nfiles;
reader_t *reader;
request_t *req = new_request();
char filename[64];
common_cache_params_t cc_params = {.cache_size = 1024*1024*1024, .hashpower = 26};
cache_t *lru_cache = create_cache("LRU", cc_params, NULL);
// cache_t *fifo_cache = create_cache("FIFO", cc_params, NULL);
int64_t req_byte = 0, miss_byte = 0;
lru_miss_cnt = fifo_miss_cnt = 0;
for (nfiles = 0; nfiles < NUM_TRACE_FILES; nfiles++) {
sprintf(filename, "/hdd/cluster18.%03d", nfiles);
printf("filename = %s\n", filename);
reader_init_param_t init_params_csv = {.delimiter=',', .real_time_field=1, .obj_id_field=2, .ttl_field=7, .has_header=FALSE};
// reader = open_trace(filename, CSV_TRACE, OBJ_ID_STR, &init_params_csv);
// reader = open_trace(filename, PLAIN_TXT_TRACE, OBJ_ID_NUM, NULL);
while (read_one_req(reader, req) == 0) {
// printf("req_size = %u\n", req->obj_size);
if (nfiles > MEASUREMENT_FILES) total++;
if (lru_cache->get(lru_cache, req) == cache_ck_miss) {
if (nfiles > MEASUREMENT_FILES) lru_miss_cnt++;
}
// if (fifo_cache->get(fifo_cache, req) == cache_ck_miss) {
// if (nfiles > MEASUREMENT_FILES) fifo_miss_cnt++;
// }
}
close_trace(reader);
}
free_request(req);
lru_cache->cache_free(lru_cache);
// fifo_cache->cache_free(fifo_cache);
printf("========LRU=======\n");
printf("total:%lu\n", total);
printf("hit_rate:%f\n", ((float) total - (float) lru_miss_cnt) / (float) total * 100);
printf("miss_rate:%f\n", (float) lru_miss_cnt / (float) total * 100);
printf("======================\n");
printf("========FIFO=======\n");
printf("total:%lu\n", total);
printf("hit_rate:%f\n", ((float) total - (float) fifo_miss_cnt) / (float) total * 100);
printf("miss_rate:%f\n", (float) fifo_miss_cnt / (float) total * 100);
printf("======================\n");
return 0;
}
from libcachesim.
you did not specify obj_size_col, then each object will be size 1, and you give a cache size of 102410241024, which means the cache will have 102410241024 objects, each object will use 10s bytes, so it may use more than 64 GB DRAM.
You can try to 1. specify the obj_size_col 2. reduce the cache size
from libcachesim.
Thank you. I'll try again in the way you told me.
from libcachesim.
Can I know the location of obj_size_col? I can't find it.
from libcachesim.
reader_init_param_t init_params_csv = {.delimiter=',', .real_time_field=1, .obj_id_field=2, .ttl_field=7, .has_header=FALSE};
add .obj_size_field=v
from libcachesim.
Oh!
I was originally going to add this, but I couldn't because I didn't know how to specify it because the trace has cache_size_field with key and value.
How should I write the field when the trace is like this?
from libcachesim.
- you can write a new trace reader
- you can preprocess the trace and add a new column
- you can use the value size to approximate object size (not recommend)
from libcachesim.
Which reader format should I use to use Method 1? Originally, I tried to use txt, but it was not available due to a similar error, and there was an error in other formats.
from libcachesim.
you need to create a trace reader under libCacheSim/traceReader/
that can parse the data set you use
from libcachesim.
I'll give it a try.
from libcachesim.
from libcachesim.
At first, I tried to specify an object size field, but I didn't write it because I didn't know how to assign both sizes because the object size is key and value, as shown in the image above.
from libcachesim.
what image?
from libcachesim.
somehow the previous message was delayed
let me know if you have any question, if not I will close this.
from libcachesim.
Related Issues (20)
- Support Mithril
- S3Fifo article feedback
- Support dynamic window size in TinyLFU
- Documentation error HOT 2
- Building on MacOS BigSur HOT 2
- Understanding the MRC curves HOT 6
- Trace analyzer space usage HOT 8
- Unable to run csv traces that have alphabetical object IDs HOT 8
- a bug in LFU-DA
- Documentation bug HOT 1
- Error when running the cacheCluster example HOT 5
- Question: Understanding Units in Trace Data HOT 3
- how to read OracleGeneral trace,how to transfrom from csv to it HOT 1
- [ERROR] 11-26-2023 21:28:11 cache_init.h:160 (tid=140522033927552): do not support algorithm glcache HOT 1
- GLCache bug HOT 2
- Set but unused variable HOT 1
- Cloudphysic block trace origin format HOT 5
- v
- Question on Op in vscsi trace HOT 2
- Support Belady on traces without future information HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from libcachesim.