Comments (8)
Thanks for this nice suggestion. We will discuss on this point and update the response soon.
-Wayne Xin Zhao
from recbole.
To add a bit. The intent is not to creste a top performing benchmark in speed or accuracy. Rather, it would be a rough guide for users that provide parameters that work on a common platform (e.g. Colab K80) and and example of what to expect for runtimes.
Thank you for the consideration.
from recbole.
To add a bit. The intent is not to creste a top performing benchmark in speed or accuracy. Rather, it would be a rough guide for users that provide parameters that work on a common platform (e.g. Colab K80) and and example of what to expect for runtimes.
Thank you for the consideration.
Our team just had a discussion on this issue. We would arrange the test and give a rough time estimate of the implemented algorithms on some selected datasets with varying sizes. Hopefully, we would update these efficiency results on the main page or otherwhere before next Wednesday.
We would also inform you on this issue page.
BTW, your mentioned LightGCN issue is also important. I think if such a speed board was available, that issue might be clear. Our team also asked the implementer to locate the lines that are likely to yield the thrown memory exception. Will get back to you with the answer soon. A practical hint is that different algorithms may scale to varying-sized datasets. Graph based algorithms are likely to take up more space than other kinds of algorithms, which is likely to throw memory exception on large-scale datasets (e.g., Gowalla dataset). That is why we provide a series of data preprocessing functions in the library, e.g., K-core filtering. In the future, we would consider accelerating some competitive algorithms with slow speed (that would take some time, probably in 2021=) ).
Thanks again for your efforts with these suggestions!
from recbole.
@batmanfly (and @ShanleiMu )I saw this post today, which provides links to time and memory costs for general recommenders and sequential recommenders. Thank you.
I had a few questions/requests for those lists and figured this is a good Issue thread to post.
- I believe the times here are in seconds-per-epoch, corrrect? (sec/epoch). If so, adding that will help clarify for new users.
- I believe the memory is the GPU memory, correct? If so, adding that will help clarify.
- Would it be possible to run on the Context-Aware recommenders too? I tried some of those yesterday and realized that adding side-features can dramatically slow down training in some cases (depending on # features, feature structure, etc)
Thank you again!
from recbole.
@batmanfly (and @ShanleiMu )I saw this post today, which provides links to time and memory costs for general recommenders and sequential recommenders. Thank you.
I had a few questions/requests for those lists and figured this is a good Issue thread to post.
- I believe the times here are in seconds-per-epoch, corrrect? (sec/epoch). If so, adding that will help clarify for new users.
- I believe the memory is the GPU memory, correct? If so, adding that will help clarify.
- Would it be possible to run on the Context-Aware recommenders too? I tried some of those yesterday and realized that adding side-features can dramatically slow down training in some cases (depending on # features, feature structure, etc)
Thank you again!
@tszumowski Nice suggestions. We will add these details to clarity our results.
For context- and knowledge- aware algorithms, their results are on the way=) We do find that some context-aware algorithms run more slowly than general recommendation algorithms, so that we didn't obtain their results by now. Their results are expected to be ready on this weekend based on current intermediate results.
from recbole.
@batmanfly great! You're all so fast and responsive!
from recbole.
@tszumowski We have added more details to clarify our results and updated the time and memory costs of context-aware recommenders and knowledge-based recommenders.
from recbole.
@ShanleiMu this is great! Thank you. I'll close this issue given all the great docs!
from recbole.
Related Issues (20)
- 请问现在伯乐支持少样本学习吗?
- [🐛BUG] LightGCN在ml-100k数据集上性能不佳 HOT 6
- Parameters of HyperTuning
- [🐛BUG] Error running LightGCN
- [🐛BUG] 顺序推荐模型在使用带有label标签的数据集并且使用排序评价指标(如NDCG)的时候会发生报错。 HOT 2
- 数据集想加入除了user_id和item_id的特征,应该修改哪部份代码呢?
- [🐛BUG] Handling scores on training items when evaluating based on ranking
- [🐛BUG] recbole1.2.0与recbole-cdr兼容问题 HOT 2
- 尝试用General recommendation models进行个性化试题推荐发现效果不太好,求助 HOT 1
- 关于recbole中知识图谱数据集的问题请教 HOT 2
- run_hyper训练未完成 HOT 1
- [🐛BUG] Context-aware recommenders not properly embedding float sequences. HOT 1
- 在训练的每一轮结束后释放显存缓冲区
- 想请问如何取出数据集的一部分进行训练?(小白)
- Context-aware DeepFM not learning HOT 2
- 我想请教一个ml-1m知识图谱数据集的配置信息问题 HOT 1
- 请问序列推荐时如何实现每一个时间步都进行预测
- [🐛BUG] Migration errors in SASRec
- 使用recbole1.2.0自动下载知识图谱数据集ml-1m时发生错误
- 使用recbole1.2.0时发现ml-1m的数据数量对不上
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from recbole.