Comments (3)
I noticed that the number of examples within every age groups are balanced wrt gender and race for the images that have service_test=True
(but not for those that have service_test=False
)
For example, within the training dataset:
| service_test | age | Female | Male |
|:---------------|:-------------|---------:|-------:|
| False | 0-2 | 506 | 908 |
| False | 10-19 | 2583 | 1743 |
| False | 20-29 | 6809 | 3773 |
| False | 3-9 | 2819 | 3902 |
| False | 30-39 | 4228 | 6624 |
| False | 40-49 | 1810 | 4398 |
| False | 50-59 | 1125 | 2970 |
| False | 60-69 | 573 | 1202 |
| False | more than 70 | 243 | 276 |
| True | 0-2 | 186 | 192 |
| True | 10-19 | 2414 | 2363 |
| True | 20-29 | 7454 | 7562 |
| True | 3-9 | 1820 | 1867 |
| True | 30-39 | 4192 | 4206 |
| True | 40-49 | 2264 | 2272 |
| True | 50-59 | 1069 | 1064 |
| True | 60-69 | 467 | 537 |
| True | more than 70 | 196 | 127 |
| service_test | age | Black | East Asian | Indian | Latino_Hispanic | Middle Eastern | Southeast Asian | White |
|:---------------|:-------------|--------:|-------------:|---------:|------------------:|-----------------:|------------------:|--------:|
| False | 0-2 | 221 | 358 | 116 | 135 | 76 | 164 | 344 |
| False | 10-19 | 854 | 474 | 848 | 941 | 42 | 688 | 479 |
| False | 20-29 | 1013 | 2616 | 926 | 1387 | 204 | 1398 | 3038 |
| False | 3-9 | 1540 | 1215 | 991 | 1047 | 207 | 989 | 732 |
| False | 30-39 | 1437 | 1162 | 1579 | 1733 | 1195 | 810 | 2936 |
| False | 40-49 | 809 | 300 | 992 | 1306 | 876 | 410 | 1515 |
| False | 50-59 | 452 | 219 | 680 | 819 | 536 | 321 | 1068 |
| False | 60-69 | 115 | 132 | 321 | 201 | 293 | 188 | 525 |
| False | more than 70 | 38 | 39 | 112 | 42 | 73 | 102 | 113 |
| True | 0-2 | 58 | 50 | 53 | 54 | 58 | 50 | 55 |
| True | 10-19 | 664 | 702 | 681 | 668 | 693 | 700 | 669 |
| True | 20-29 | 2158 | 2149 | 2151 | 2144 | 2119 | 2112 | 2183 |
| True | 3-9 | 532 | 521 | 539 | 542 | 514 | 534 | 505 |
| True | 30-39 | 1182 | 1220 | 1192 | 1199 | 1203 | 1199 | 1203 |
| True | 40-49 | 650 | 639 | 643 | 651 | 641 | 648 | 664 |
| True | 50-59 | 312 | 305 | 304 | 305 | 301 | 295 | 311 |
| True | 60-69 | 164 | 138 | 136 | 154 | 142 | 119 | 151 |
| True | more than 70 | 34 | 48 | 55 | 39 | 43 | 68 | 36 |
| service_test | age | Female/Black | Female/East Asian | Female/Indian | Female/Latino_Hispanic | Female/Middle Eastern | Female/Southeast Asian | Female/White | Male/Black | Male/East Asian | Male/Indian | Male/Latino_Hispanic | Male/Middle Eastern | Male/Southeast Asian | Male/White |
|:---------------|:-------------|---------------:|--------------------:|----------------:|-------------------------:|------------------------:|-------------------------:|---------------:|-------------:|------------------:|--------------:|-----------------------:|----------------------:|-----------------------:|-------------:|
| False | 0-2 | 62 | 122 | 51 | 80 | 0 | 51 | 140 | 159 | 236 | 65 | 55 | 76 | 113 | 204 |
| False | 10-19 | 512 | 275 | 551 | 575 | 0 | 364 | 306 | 342 | 199 | 297 | 366 | 42 | 324 | 173 |
| False | 20-29 | 699 | 1820 | 648 | 946 | 0 | 823 | 1873 | 314 | 796 | 278 | 441 | 204 | 575 | 1165 |
| False | 3-9 | 584 | 414 | 436 | 570 | 0 | 430 | 385 | 956 | 801 | 555 | 477 | 207 | 559 | 347 |
| False | 30-39 | 733 | 509 | 577 | 778 | 0 | 328 | 1303 | 704 | 653 | 1002 | 955 | 1195 | 482 | 1633 |
| False | 40-49 | 353 | 26 | 343 | 516 | 0 | 88 | 484 | 456 | 274 | 649 | 790 | 876 | 322 | 1031 |
| False | 50-59 | 217 | 38 | 237 | 276 | 0 | 101 | 256 | 235 | 181 | 443 | 543 | 536 | 220 | 812 |
| False | 60-69 | 85 | 32 | 158 | 90 | 0 | 88 | 120 | 30 | 100 | 163 | 111 | 293 | 100 | 405 |
| False | more than 70 | 25 | 13 | 72 | 26 | 0 | 57 | 50 | 13 | 26 | 40 | 16 | 73 | 45 | 63 |
| True | 0-2 | 29 | 24 | 27 | 27 | 28 | 25 | 26 | 29 | 26 | 26 | 27 | 30 | 25 | 29 |
| True | 10-19 | 338 | 357 | 339 | 343 | 343 | 354 | 340 | 326 | 345 | 342 | 325 | 350 | 346 | 329 |
| True | 20-29 | 1070 | 1082 | 1056 | 1058 | 1041 | 1048 | 1099 | 1088 | 1067 | 1095 | 1086 | 1078 | 1064 | 1084 |
| True | 3-9 | 258 | 261 | 259 | 267 | 248 | 272 | 255 | 274 | 260 | 280 | 275 | 266 | 262 | 250 |
| True | 30-39 | 590 | 606 | 587 | 597 | 616 | 588 | 608 | 592 | 614 | 605 | 602 | 587 | 611 | 595 |
| True | 40-49 | 329 | 322 | 316 | 316 | 327 | 324 | 330 | 321 | 317 | 327 | 335 | 314 | 324 | 334 |
| True | 50-59 | 154 | 148 | 151 | 150 | 155 | 157 | 154 | 158 | 157 | 153 | 155 | 146 | 138 | 157 |
| True | 60-69 | 80 | 62 | 66 | 71 | 67 | 48 | 73 | 84 | 76 | 70 | 83 | 75 | 71 | 78 |
| True | more than 70 | 19 | 30 | 35 | 29 | 22 | 37 | 24 | 15 | 18 | 20 | 10 | 21 | 31 | 12 |
from fairface.
+1 for this issue - I would also like to know what the service_test
column means. Is it just an indicator that service_test == True
are the balanced data?
from fairface.
There are great statistics in the previous posts. That made me look into the label statistics too.
If filtered by service_test == True then both Train and Test datasets are pretty balanced for race and gender (and all combinations between the two categories). Without this filter the dataset is imbalanced towards White and Male.
So, if we use Race and Gender as combined filter (i.e. White vs Black or White+Male vs Black+Female) then the distribution within the age groups is also fairly balanced (though the population gets very small in some sub categories and the balance in such small numbers can be off). Good job by the authors.
The 2nd post in this thread nicely demonstrates this.
The balance between age groups is more tricky: the population in the full dataset is strongly imbalanced for age, the 20-29 and 30-39 groups together represent more than 50% of the population in the dataset, 20-29 group alone is more than 30% (if all age groups are equal it would give 11%).
Arguably, there was probably no objective for the authors to balance between age groups (it was deemed too difficult or to constraining to get similar number of images for i.e. the 0-3 and 20-23 groups especially in ALL race and gender combinations). This is reasonable.
Though, this raises a question for training: as race and gender are balanced but age is imbalanced the weighting (i.e. in loss calculation) might differ. Unfortunately the paper does not give any such detail on the training recipe.
from fairface.
Related Issues (20)
- Training configuration HOT 1
- Low validation accuracy 71% for race estimation HOT 6
- Many incorrectly labeled images HOT 4
- The pretrained model is not recognized HOT 5
- code to train HOT 5
- Process is automatically killed while processing a large dataset
- links to labels files are broken
- Can you tell me the specific pytorch version information?
- Obsolete HOT 1
- Could you upload the pretrained model? HOT 4
- 4 races definition and training HOT 2
- Hosting images on Google drive prevents programmatic access
- can we detect multiple faces ?
- What is the license of this repository HOT 1
- don't match bboxes per every iamge
- Pretrained model on different datasets
- Original Images HOT 1
- Accuracy difference between train and val datasets and paper
- Help in annotations for age
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fairface.