Comments (2)
遍历打印了一下sector的信息,发现大部分情况下一个sector并不会完全用满,也就不太好判断标记sector为GC的时机,绝大部分情况下即便一个sector里的kv全部删除也会剩下几十个bytes的空间,实际使用时很难完全写满一个sector
调整threshold也许可行,不过不同的应用场景需要权衡,空间大区块多的场景可以牺牲一些存储密度,空间小的场景就不太需要关注性能,但这个方法终究不太通用
换个思路,目前每次分配一个kv对象时,alloc_kv首先要做一次全sector遍历统计sector使用量信息,再做一次遍历找Using状态的sector,如果所有Using-sector空间都不足,需要再做一次sector遍历找到空sector。
static uint32_t alloc_kv(fdb_kvdb_t db, kv_sec_info_t sector, size_t kv_size)
{
uint32_t empty_kv = FAILED_ADDR;
size_t empty_sector = 0, using_sector = 0;
struct alloc_kv_cb_args arg = {db, kv_size, &empty_kv};
/* sector status statistics */
sector_iterator(db, sector, FDB_SECTOR_STORE_UNUSED, &empty_sector, &using_sector, sector_statistics_cb, false);
if (using_sector > 0) {
/* alloc the KV from the using status sector first */
sector_iterator(db, sector, FDB_SECTOR_STORE_USING, &arg, NULL, alloc_kv_cb, true);
}
if (empty_sector > 0 && empty_kv == FAILED_ADDR) {
if (empty_sector > FDB_GC_EMPTY_SEC_THRESHOLD || db->gc_request) {
sector_iterator(db, sector, FDB_SECTOR_STORE_EMPTY, &arg, NULL, alloc_kv_cb, true);
} else {
/* no space for new KV now will GC and retry */
FDB_DEBUG("Trigger a GC check after alloc KV failed.\n");
db->gc_request = true;
}
}
return empty_kv;
}
三次遍历耗时非常明显,感觉至少有两个改进点:
- sector的使用量统计可以在初始化加载的时候统计一次,随后对db进行增删改操作时 实时更新而不需要每次都重新遍历全部sector,占用RAM很少,不过需要配合修改的地方很多
- 寻找可用的sector应该合并成一次遍历,中间记录一下备选区块,找到合适的Using区块直接分配,找不到合适的Using区块就直接使用备选的Empty区块
其实第1条如果改动太麻烦的话,也可以把后两次的遍历合并到第一次的遍历里面去,目前的代码可能是为了代码简洁,在这个地方牺牲了不少性能。不过这个改进也只是常系数项的优化
from flashdb.
其实也不是每次都会执行三次遍历的,绝大多数情况下是两次遍历,第三次只有第二次分配不出来才会执行的
from flashdb.
Related Issues (20)
- 分区越大,插入速度越慢 HOT 2
- sfud_flash_def.h 部分有误 HOT 4
- KVDB Demo 运行出错 HOT 1
- linux的demo程序第一次运行都会卡住 HOT 2
- Error: The KV (@0x0000FF4F) CRC32 check failed!
- Support wear balance
- tsdb
- FR:tsdb读取最新一条数据内容 HOT 1
- TSDB能否调整为递增序列式。 HOT 1
- V2.1.0使用tsdb遇到一个问题,假如tsdb刚好全部写满的时候重启了,再次开机 初始化tsdb的时候,会出现oldest sectors is 0x00000FFF, current using sector is 0xFFFFFFFF, 导致后续无法写入 HOT 1
- 迭代器无法遍历所有键值对 HOT 2
- FDB_GC_EMPTY_SEC_THRESHOLD我想问这个能设置成0吗,我现在只想分配2个扇区给flashDB
- fdb_tsl_status_t end_info_stat[2]这里结束信息为啥需要2呢? HOT 1
- 如何主动进行一次GC
- tsdb 写数据的时候突然重启设备,重启后设备 db->last_time 变成0开始 HOT 10
- 希望更新一下RT-Thread 软件包 HOT 2
- FAL 组件里面的 device_table 和 device_table_len 用 const 声明编译器可能会分配 0 地址 HOT 2
- kvdb写满后,写入同名键值对无法写入无法覆盖,且kv大小是扇区的一半时,空间利用率最差,只有一半,fdb_kv_print打印的kv也不全但是get能读到
- [QUESTION] Can this work with ESP32 arduino?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from flashdb.