erfangithub / geo-deep-learning

Deep learning applied to georeferenced datasets

About

The Geo-Deep-Learning project stems from an initiative at NRCan's CCMEO. Its aim is to allow using Convolutional Neural Networks (CNN) with georeferenced datasets.

In Geo-Deep-Learning, the learning process comprises two broad stages: tiling and training, followed by inference, which makes use of a trained model to make new predictions on unseen imagery.

Requirement

This project comprises a set of commands to be run at a shell command prompt. Examples used here are for a bash shell in an Ubuntu GNU/Linux environment.

• Python 3.10, see the full list of dependencies in environment.yml
• hydra
• mlflow
• miniconda (highly recommended)
• nvidia GPU (highly recommended)

Installation

Miniconda is suggested as the package manager for GDL. However, users are advised to switch to libmamba as conda's default solver or to directly use mamba instead of conda if they are facing extended installation time or other issues. Additional problems are grouped in the troubleshooting section. If issues persist, users are encouraged to open a new issue for assistance.

Tested on Ubuntu 20.04, Windows 10 and WSL 2.

Quickstart with conda

To execute scripts in this project, first create and activate your python environment with the following commands:

$ conda env create -f environment.yml
$ conda activate geo_deep_env

Change conda's default solver for faster install (Optional)

conda install -n base conda-libmamba-solver
conda config --set solver libmamba

Troubleshooting

• ImportError: /lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found
  • Export path to library or set it permenantly in your .bashrc file (example with conda) :

    export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CONDA_PREFIX/lib/

How to use?

This is an example of how to run GDL with hydra in simple steps with the massachusetts buildings dataset in the tests/data/ folder, for segmentation on buildings:

1. Clone this github repo.

(geo_deep_env) $ git clone https://github.com/NRCan/geo-deep-learning.git
(geo_deep_env) $ cd geo-deep-learning

2. Run the wanted script (for segmentation).

# Creating the patches from the raw data
(geo_deep_env) $ python GDL.py mode=tiling
# Training the neural network
(geo_deep_env) $ python GDL.py mode=train
# Inference on the data
(geo_deep_env) $ python GDL.py mode=inference

This example runs with a default configuration ./config/gdl_config_template.yaml. For further examples on configuration options see the configuration documentation. To see the different mode and task available go see the documentation here.

New task

If you want to introduce a new task like object detection, you only need to add the code in the main folder and name it object_detection_tiling.py for example. The principle is to name the code like {task}_{mode}.py and the GDL.py will deal with the rest. To run it, you will need to add a new parameter in the command line python GDL.py mode=tiling task=object_detection or change the parameter inside the ./config/gdl_config_template.yaml.

Contributing

We welcome all forms of user contributions including feature requests, bug reports, code, documentation requests, and code. Simply open an issue in the tracker.

If you think you're not skilled or experienced enough to contribute, this is not TRUE! Don't be affraid to help us, every one start somewhere, and it will be our pleasure to help you to help us.

You can find more information on how to create a good issue on a GitHub project Here.

After creating an issue, you can start working on the solution. When you have finish working on your code, it's time for the PR. All the information on how to create a good PR on a GitHub project Here.

Citing Geo Deep Learning

Citations help us justify the effort that goes into building and maintaining this project. If you used Geo Deep Learning for your research, please consider citing us.

@misc{NRCAN:2020,
  Author = {Natural Resources Canada, Government of Canada},
  Title = {Geo Deep Learning},
  Year = {2020},
  Publisher = {GitHub},
  Journal = {GitHub repository},
  Howpublished = {\url{https://github.com/NRCan/geo-deep-learning}}
}

Or you can also use the CITATION.cff file to cite this project.

Contacting us

The best way to get in touch is to open an issue or comment on any open issue or pull request.

License

Project is distributed under MIT License.

---