Comments (5)
Hello @drcege , thanks for asking. We followed this paper: https://arxiv.org/abs/1511.06233 for the F-measure calculation. It was modified from the original equation for better openset evaluation.
from openlongtailrecognition-oltr.
@zhmiao @drcege I also feel confused on this issue, I suggest authors @zhmiao can explain it clearly rather than re-direct readers to refer other literature since this criterion is so important for this manuscript.
from openlongtailrecognition-oltr.
@zhmiao I further seek related information from https://arxiv.org/abs/1511.06233,however, I got nothing explain, may you pay some attention to explain this issue?
from openlongtailrecognition-oltr.
@drcege hi, have you understand this issue, I have the same confusing feeling as you, may you share you newest understanding?
from openlongtailrecognition-oltr.
Searching for "open-set f-measure" I just found this thread. Here are my thoughts on this issue. Many works on open-set recognition use f-measure but the authors do not specify how the metric is calculated. Sometimes it seems to me that the conclusions presented in some works on open-set recognition are just non-reliable due to the metric employed. How to be sure that the employed metric is measuring a better open-set behavior? It also seems that some authors do not mind if the metric is really capturing a better behavior of the classifier. I do not know details of the work associated to this repository, anyhow, I agree with the concerns raised by @drcege. If any of you are interested in evaluation metrics specially proposed for open-set recognition, take a look in a paper in which we talk about this issue [1]. There in Section 4.1 we propose the open-set f-measure and the normalized accuracy, both to be employed on open-set problems.
[1] Mendes Júnior, P. R. and de Souza, R. M. and Werneck, R. de O. and Stein, B. V. and Pazinato, D. V. and de Almeida, W. R. and Penatti, O. A. B. and Torres, R. da S. and Rocha, A. de R. (2017). "Nearest Neighbors Distance Ratio Open-Set Classifier". Machine Learning, 106, 359–386. https://doi.org/10.1007/s10994-016-5610-8.
from openlongtailrecognition-oltr.
Related Issues (20)
- Reproducing OLTR results HOT 3
- Stage 2 multi GPU
- why fix all parameters except self attention parameters? HOT 4
- Table 2 results HOT 2
- Pretrained Weights for Places_LT?
- the use of fc layer HOT 2
- the accuracy of the train and val HOT 2
- how to compute centroids?
- Why the input dimension of the `fc_spatial` layer in `ModulatedAttLayer` is 7*7*in_channel? HOT 1
- Many_shot_accuracy_top1: nan on my own dataset HOT 1
- Revised F-measure results for other models in your paper
- Applications for face recognition
- Error when running stage_1.py under Places_LT
- Unable to reproduce baseline result on ImageNet-LT HOT 1
- BUG: stage1 test error!!
- Could you please give me an example of arranging ILSVRC2014 dataset? HOT 7
- Implementation on Inat-18
- About Class aware sampler
- The role of untrained FC(add_fc)
- The question about the version of Places_LT
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from openlongtailrecognition-oltr.