minerva-ml / open-solution-ship-detection

Open solution to the Airbus Ship Detection Challenge

Airbus Ship Detection Challenge

This is an open solution to the Airbus Ship Detection Challenge.

Our goals

We are building entirely open solution to this competition. Specifically:

1. Learning from the process - updates about new ideas, code and experiments is the best way to learn data science. Our activity is especially useful for people who wants to enter the competition, but lack appropriate experience.
2. Encourage more Kagglers to start working on this competition.
3. Deliver open source solution with no strings attached. Code is available on our GitHub repository 💻. This solution should establish solid benchmark, as well as provide good base for your custom ideas and experiments. We care about clean code 😃
4. We are opening our experiments as well: everybody can have live preview on our experiments, parameters, code, etc. Check: Airbus Ship Detection Challenge 📈 or screen below.

Train and validation monitor 📊

Disclaimer

In this open source solution you will find references to the neptune.ml. It is free platform for community Users, which we use daily to keep track of our experiments. Please note that using neptune.ml is not necessary to proceed with this solution. You may run it as plain Python script 🐍.

How to start?

Learn about our solutions

1. Check Kaggle forum and participate in the discussions.
2. See solutions below:

link to code	CV	LB
solution 1	0.541	0.573
solution 2	0.661	0.679
solution 3	0.694	0.696
solution 4	0.722	0.703
solution 5	0.719	0.725

Start experimenting with ready-to-use code

You can jump start your participation in the competition by using our starter pack. Installation instruction below will guide you through the setup.

Installation (fast track)

1. Clone repository and install requirements (use Python3.5) pip3 install -r requirements.txt
2. Register to the neptune.ml (if you wish to use it)
3. Run experiment based on U-Net:

Cloud

neptune account login

Create project say Ships (SHIP)

Go to neptune.yaml and change:

project: USERNAME/PROJECT_NAME

to your username and project name

Prepare metadata and overlayed target masks It only needs to be done once

neptune send --worker xs \
--environment base-cpu-py3 \
--config neptune.yaml \
prepare_metadata.py

They will be saved in the

  metadata_filepath: /output/metadata.csv
  masks_overlayed_dir: /output/masks_overlayed

From now on we will load the metadata by changing the neptune.yaml

  metadata_filepath: /input/metadata.csv
  masks_overlayed_dir: /input/masks_overlayed

and adding the path to the experiment that generated metadata say SHIP-1 to every command --input/metadata.csv

Let's train the model by running the main.py:

neptune send --worker m-2p100 \
--environment pytorch-0.3.1-gpu-py3 \
--config neptune.yaml \
--input /SHIP-1/output/metadata.csv \
--input /SHIP-1/output/masks_overlayed \
main.py 

The model will be saved in the:

  experiment_dir: /output/experiment

and the submission.csv will be saved in /output/experiment/submission.csv

You can easily use models trained during one experiment in other experiments. For example when running evaluation we need to use the previous model folder in our experiment. We do that by:

changing main.py

  CLONE_EXPERIMENT_DIR_FROM = '/SHIP-2/output/experiment'

and running the following command:

neptune send --worker m-2p100 \
--environment pytorch-0.3.1-gpu-py3 \
--config neptune.yaml \
--input /SHIP-1/output/metadata.csv \
--input /SHIP-1/output/masks_overlayed \
--input /SHIP-2 \
main.py

Local

Login to neptune if you want to use it

neptune account login

Prepare metadata by running:

neptune run --config neptune.yaml prepare_metadata.py

Training and inference by running main.py:

neptune run --config neptune.yaml main.py

You can always run it with pure python 🐍

python main.py 

Get involved

You are welcome to contribute your code and ideas to this open solution. To get started:

1. Check competition project on GitHub to see what we are working on right now.
2. Express your interest in particular task by writing comment in this task, or by creating new one with your fresh idea.
3. We will get back to you quickly in order to start working together.
4. Check CONTRIBUTING for some more information.

User support

There are several ways to seek help:

1. Kaggle discussion is our primary way of communication.
2. Submit an issue directly in this repo.

open-solution-ship-detection's Issues

rotated bbox appraoch

https://arxiv.org/pdf/1806.04331.pdf

modify lovasz

def symmetric_lovasz(outputs, targets):
    return (lovasz_hinge(outputs, targets) + lovasz_hinge(-outputs, 1 - targets)) / 2

Executable 'prepare_metadata.py' does not exist.

Hi, thank you for open your solution.

I am trying to use neptune but I an unable to generate metadata.csv as instructed.
The file prepare_metadata.py seems to be missing ?

enabling dev_mode

Hi

Using python 3.5 (without neptune), the following works.
python main.py -- train --pipeline_name unet

However I wish to debug using dev_mode and can't seem to pass through the flag.
python main.py -- train --pipeline_name unet --dev_mode

Error: No value provided for parameter 'dev_mode'

I think the click option is_flag for dev_mode means we shouldn't need to pass any values, so I'm doing something wrong?

Thanks

Local vs Cloud execution

Hello, thank you for putting together this excellent baseline.

I have two questions:

1. what are these two directories (the others are self explanatory):

  annotation_file:      /path/to/data
  masks_overlayed_dir:  /path/to/data

2. when I execute neptune run --config configs/neptune.yaml main.py prepare_masks I get

Calculated experiment snapshot size: 57.45 GB   
Sending sources to server:   0%|                                   | 31.8M/57.4G [00:25<8:14:17, 1.94MB/s

which means it is sending all the files to the cloud. Why is this happening? Isn't run intended to run everything locally?

Handle corrupted images

Kaggle topic: https://www.kaggle.com/c/airbus-ship-detection/discussion/62921

Image with examples: https://storage.googleapis.com/kaggle-forum-message-attachments/367968/10020/corrupted.png

There is no yaml config file.

There is no *.yaml file at the master branch.

Segmentation fault encountered in worker

I've successfully prepared the masks and the meta data. However when I run the training (as neptune run -x data --config configs/neptune.yaml main.py train --pipeline_name unet) I get the following error:

2018-08-27 14-54-31 ships-detection >>> training
/anaconda3/envs/neptune/lib/python3.5/site-packages/click/core.py:535: DtypeWarning: Columns (1) have mixed types. Specify dtype option on import or set low_memory=False.
  return callback(*args, **kwargs)
2018-08-27 14:54:31 steppy >>> initializing Step xy_train...
2018-08-27 14:54:31 steppy >>> initializing experiment directories under /Users/jonathan/devwork/open-solution-ship-detection/experiments
2018-08-27 14:54:31 steppy >>> done: initializing experiment directories
2018-08-27 14:54:31 steppy >>> Step xy_train initialized
2018-08-27 14:54:31 steppy >>> initializing Step xy_inference...
2018-08-27 14:54:31 steppy >>> initializing experiment directories under /Users/jonathan/devwork/open-solution-ship-detection/experiments
2018-08-27 14:54:31 steppy >>> done: initializing experiment directories
2018-08-27 14:54:31 steppy >>> Step xy_inference initialized
2018-08-27 14:54:31 steppy >>> initializing Step loader...
2018-08-27 14:54:31 steppy >>> initializing experiment directories under /Users/jonathan/devwork/open-solution-ship-detection/experiments
2018-08-27 14:54:31 steppy >>> done: initializing experiment directories
2018-08-27 14:54:31 steppy >>> Step loader initialized
/anaconda3/envs/neptune/lib/python3.5/site-packages/toolkit/pytorch_transformers/architectures/unet.py:22: UserWarning: Please make sure, that your input tensor's dimensions are divisible by (pool_stride ** repeat_blocks)
  warnings.warn("Please make sure, that your input tensor's dimensions are divisible by "
2018-08-27 14:54:32 steppy >>> initializing Step unet...
2018-08-27 14:54:32 steppy >>> initializing experiment directories under /Users/jonathan/devwork/open-solution-ship-detection/experiments
2018-08-27 14:54:32 steppy >>> done: initializing experiment directories
2018-08-27 14:54:32 steppy >>> Step unet initialized
2018-08-27 14:54:32 steppy >>> cleaning cache...
2018-08-27 14:54:32 steppy >>> cleaning cache done
2018-08-27 14:54:32 steppy >>> Step xy_train, adapting inputs...
2018-08-27 14:54:32 steppy >>> Step xy_train, transforming...
2018-08-27 14:54:32 steppy >>> Step xy_inference, adapting inputs...
2018-08-27 14:54:32 steppy >>> Step xy_inference, transforming...
2018-08-27 14:54:32 steppy >>> Step loader, adapting inputs...
2018-08-27 14:54:32 steppy >>> Step loader, transforming...
2018-08-27 14:54:32 steppy >>> Step unet, adapting inputs...
2018-08-27 14:54:32 steppy >>> Step unet, fitting and transforming...
2018-08-27 14:54:32 steppy >>> initializing model weights...
2018-08-27 14-54-32 ships-detection >>> starting training...
2018-08-27 14-54-32 ships-detection >>> initial lr: 0.0001
2018-08-27 14-54-32 ships-detection >>> epoch 0 ...
ERROR: Unexpected segmentation fault encountered in worker.
Traceback (most recent call last):
  File "/anaconda3/envs/neptune/lib/python3.5/site-packages/deepsense/neptune/job_wrapper.py", line 107, in <module>
    execute()
  File "/anaconda3/envs/neptune/lib/python3.5/site-packages/deepsense/neptune/job_wrapper.py", line 103, in execute
    execfile(job_filepath, job_globals)
  File "/anaconda3/envs/neptune/lib/python3.5/site-packages/past/builtins/misc.py", line 82, in execfile
    exec_(code, myglobals, mylocals)
  File "main.py", line 89, in <module>
    main()
  File "/anaconda3/envs/neptune/lib/python3.5/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/anaconda3/envs/neptune/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/anaconda3/envs/neptune/lib/python3.5/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/anaconda3/envs/neptune/lib/python3.5/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/anaconda3/envs/neptune/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "main.py", line 27, in train
    pipeline_manager.train(pipeline_name, dev_mode)
  File "/Users/jonathan/devwork/open-solution-ship-detection/src/pipeline_manager.py", line 28, in train
    train(pipeline_name, dev_mode)
  File "/Users/jonathan/devwork/open-solution-ship-detection/src/pipeline_manager.py", line 77, in train
    pipeline.fit_transform(data)
  File "/anaconda3/envs/neptune/lib/python3.5/site-packages/steppy/base.py", line 323, in fit_transform
    step_output_data = self._cached_fit_transform(step_inputs)
  File "/anaconda3/envs/neptune/lib/python3.5/site-packages/steppy/base.py", line 443, in _cached_fit_transform
    step_output_data = self.transformer.fit_transform(**step_inputs)
  File "/anaconda3/envs/neptune/lib/python3.5/site-packages/steppy/base.py", line 605, in fit_transform
    self.fit(*args, **kwargs)
  File "/Users/jonathan/devwork/open-solution-ship-detection/src/models.py", line 68, in fit
    for batch_id, data in enumerate(batch_gen):
  File "/anaconda3/envs/neptune/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 275, in __next__
    idx, batch = self._get_batch()
  File "/anaconda3/envs/neptune/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 254, in _get_batch
    return self.data_queue.get()
  File "/anaconda3/envs/neptune/lib/python3.5/multiprocessing/queues.py", line 335, in get
    res = self._reader.recv_bytes()
  File "/anaconda3/envs/neptune/lib/python3.5/multiprocessing/connection.py", line 216, in recv_bytes
    buf = self._recv_bytes(maxlength)
  File "/anaconda3/envs/neptune/lib/python3.5/multiprocessing/connection.py", line 407, in _recv_bytes
    buf = self._recv(4)
  File "/anaconda3/envs/neptune/lib/python3.5/multiprocessing/connection.py", line 379, in _recv
    chunk = read(handle, remaining)
  File "/anaconda3/envs/neptune/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 175, in handler
    _error_if_any_worker_fails()
RuntimeError: DataLoader worker (pid 33362) is killed by signal: Unknown signal: 0.
ERROR: Unexpected segmentation fault encountered in worker.
ERROR: Unexpected segmentation fault encountered in worker.

Do you have any ideas on this? Thank you very much.

Is the hyper parameters need to fine tune?

I just get the LB with 0.805, is the yaml file did not provide the best hyper parameters?