open source 60th solution(code) for kaggle TGS salt identification challenge this is my first kaggle competition, I worked hard in last month competition times, thanks for all people shared their ideas, in especial Jack (Jiaxin) Shao, peter and heng, they give me a lot of encouragement, I have gained so much from this competition, and thanks for Eduardo Rocha de Andrade's code for our baseline model
Input: 101 -> resize to 202 -> pad to 256
Encoder: ResNet34 pretrained on ImageNet
Centerblock: FPA model for attention
Decoder: conv3x3 + GAU
- Optimizer: Lovasz Loss, SGD, Batch size: 32, Reduce LR on cosine annealing, 100 epochs each, LR starting from 0.01
- single model ResNet34 got 0.856 public LB (0.878 private LB)
- 5-fold Average Ensemble ResNet34 got 0.859 public LB (0.881 private LB)
- all threshold we used 0.45 for best score
- Transfer above best model for 5-fold Average Ensemble ResNet34 got 0.864 public LB (0.883 private LB)
we used heng's augmentations code, we first used on keras model but did not work , in pytorch, it worked perfect
do_horizontal_flip2
do_random_shift_scale_crop_pad2 0.2
do_horizontal_shear2 (-0.07, +0.07)
do_shift_scale_rotate2 rotate 15
do_elastic_transform2 (0, 0.15)
do_brightness_shift (-0.1, +0.1)
do_brightness_multiply (-0.08, +0.08)
do_gamma (-0.08, +0.08)
depth channels
Stratified in this kernel rather than simply cov_to_class, this give me 0.03 boost
pytorch >= 0.4.1
python = 3.6
[x] Implement ResNet34 encoded U-Net
[x] Use pretrain weights
[x] Compare pretrained encoder with full initialized
[x] Implement deeper encoders (ResNet50,101,152)[x] Compare transposed convolutions with bilinear upsampling
[x] No pooling in center block: 4 upsamplings
[x] Add pooling in center block: 5 upsamplings
[x] Try spatial dropout
[x] Try spatial pyramid pooling
[x] Replace sesc decode as GAU
[ ] Replace part of the decoder with dilated convs[x] Implement hyper columns on decoder
[ ] Increment hyper columns using adptation convolutions[x] Implement sSE and cSC on decoder
[x] Implement sSE and cSC on encoder
[x] Use cSE block as input to image depth[x] Implement PPM on U-Net's center block
[ ] Implement OC-ASP Module[x] Implement FPA on Unet center block\
[x] Implement FPNet
[ ] Add SE Blocks
[x] Implement RefineNet
[x] Add SE blocks
[ ] Implement dense connections in RefineBlocks
[ ] Use PPM or ASSP instead of ChainPoolingModule
[ ] Use hypercolumns
[ ] Implement DeepLabv3
[ ] Add SE blocks
[ ] Implement OC-ASP Module
[x] Implement learnig rate reduction on Plateau
[x] Implement learnig rate reduction on Milestones[x] Implement cosine annealing with warm restart
[x] Verify if getting predictions for the M last restarts can produce a good ensemble[x] Implement Binary Cross Entropy
[x] Implement Sørensen–Dice loss (IoU)
[x] Implement Hybrid BCE+Dice
[x] Implement Lovász-hinge loss
[x] Implement Hybrid BCE+Lovás loss
[ ] Add Regional Loss: Adaptive Affinity Field[x] Implement 5Fold cross-validation
[x] Implement stratified cross-validation by mask coverage
[x] Ensemble predictions (average) on 5 folds[x] Implement basic augmentations (flips, shift, scale, crop, elastic, rotate, shear, gamma, brightness)
[x] Use additional dataset made of masks with small salt coverage
[ ] Test other types of augmentations[x] Implement simple flip TTA
[ ] Implement other types of TTA