Comments (6)
The primary training shown some problems:
- The DL model is sensitive and might be stuck to the noises of labels.
- Due to both the convolution of DL and the defective labels, the predictions are significantly lack of details, even though they have high accuracy.
- The visualization of the predictions doesn't really match the high accuracy.
In order to solve these issues. We tried and will try:
- Try to cooperate other information. OSM is our choice. Using OSM vectors to clean the consensus labels.
- Run RF first as a guess model to get some pre-trained labels. Then select tiles with high certainty to check manually, and give a score to this tile. So finally we will get a sequence of tiles of labels with score tab.
- Then train a co-teaching DL model based on these scored pre-trained LC labels.
from hrlcm.
Some updates:
- The usage of OSM significantly improves the quality of ensemble labels.
- We used OSM as a mask to mask out some potential area of bad labels. For instance, roads and buildings with a buffer.
- We also used some OSM features to fill in the labels. For instance, buildings as new labels of urban, and roads as new labels of bareland. Waterbodies as well.
- RF as a guess model also significantly improve the quality of ensemble labels. The results of RF guess model are good enough as new LC product. However, it is very slow to predict on new tiles. And it can't be transferred as easily as DL.
So, next step we will manually check and grade the guess labels of RF, and then fit them into a co-teaching DL model to train a ready-to-use DL model.
from hrlcm.
Update Apr 20, 2021:
- Regular training seems work just fine. It is slow without a decent GPU instance though.
- Next step is to implement multiple methods of co-training which includes co-teaching, co-teaching-plus, JoCoR.
from hrlcm.
Update May 3, 2021:
Hyper-parameter tuning: optimizer, learning rate (includes using scheduler), batch size, and probably transformation methods.
from hrlcm.
Update May 9, 2021:
Use AdaBound optimizer (https://github.com/Luolc/AdaBound/blob/master/adabound/adabound.py), a learning rate scheduler, 32 batch size, and random flip sync transformation and normalization image transformation to train 200 epochs as a baseline.
And training progress:
Evaluation:
'AverageAcc': 0.8917790560347527
'weightedPrec': 0.9230187367229913
'weightedRec': 0.8719084136833852
'weightedF1': 0.8910451918369202
'weightedF2': 0.8769002324638219
'HammingLoss': 0.12809158631661477
Model performs poorly on class urban and barehand based on metrics. Because there are much less labels for these two classes.
Here is a prediction for a whole tile (4096 * 4096):
Based on the results so far, the method is a big success. Next step is to apply multiple methods to use noisy labels.
from hrlcm.
Update May 16, 2021:
Finish adding all co-teaching related loss functions. Will start to do experiments of how to use them to improve model performance.
from hrlcm.
Related Issues (5)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hrlcm.