Coder Social home page Coder Social logo

patball1 / detectree2 Goto Github PK

View Code? Open in Web Editor NEW
149.0 7.0 35.0 154.55 MB

Python package for automatic tree crown delineation based on the Detectron2 implementation of Mask R-CNN

Home Page: https://patball1.github.io/detectree2/

License: MIT License

Shell 0.01% Makefile 0.02% Python 0.55% Jupyter Notebook 99.15% R 0.27% Dockerfile 0.01%
detectron2 python pytorch deep-learning

detectree2's Introduction

predictions predictions

License: MIT Detectree CI PEP8 DOI

Python package for automatic tree crown delineation based on Mask R-CNN. Pre-trained models can be picked in the model_garden. A tutorial on how to prepare data, train models and make predictions is available here. For questions, collaboration proposals and requests for data email James Ball. Some example data is available for download here.

Detectree2是一个基于Mask R-CNN的自动树冠检测与分割的Python包。您可以在model_garden中选择预训练模型。这里提供了如何准备数据、训练模型和进行预测的教程。如果有任何问题,合作提案或者需要样例数据,可以邮件联系James Ball。一些示例数据可以在这里下载。

Code developed by James Ball, Seb Hickman, Thomas Koay, Oscar Jiang, Luran Wang, Panagiotis Ioannou, James Hinton and Matthew Archer in the Forest Ecology and Conservation Group at the University of Cambridge. The Forest Ecology and Conservation Group is led by Professor David Coomes and is part of the University of Cambridge Conservation Research Institute.



Note

To save bandwidth, trained models have been moved to Zenodo. Download models directly with wget or equivalent.

Citation

Please cite this article if you use detectree2 in your work:

Ball, J.G.C., Hickman, S.H.M., Jackson, T.D., Koay, X.J., Hirst, J., Jay, W., Archer, M., Aubry-Kientz, M., Vincent, G. and Coomes, D.A. (2023), Accurate delineation of individual tree crowns in tropical forests from aerial RGB imagery using Mask R-CNN. Remote Sens Ecol Conserv. 9(5):641-655. https://doi.org/10.1002/rse2.332

Independent validation

Independent validation has been performed on a temperate deciduous forest in Japan.

Detectree2 (F1 score: 0.57) outperformed DeepForest (F1 score: 0.52)

Detectree2 could estimate tree crown areas accurately, highlighting its potential and robustness for tree detection and delineation

Gan, Y., Wang, Q., and Iio, A. (2023). Tree Crown Detection and Delineation in a Temperate Deciduous Forest from UAV RGB Imagery Using Deep Learning Approaches: Effects of Spatial Resolution and Species Characteristics. Remote Sensing. 15(3):778. https://doi.org/10.3390/rs15030778

Requirements

e.g. pip3 install torch torchvision torchaudio

Installation

pip

pip install git+https://github.com/PatBall1/detectree2.git

Currently works on Google Colab (Pro version recommended). May struggle on clusters if geospatial libraries are not configured. See Installation Instructions if you are having trouble.

conda

Under development

Getting started

Detectree2, based on the Detectron2 Mask R-CNN architecture, locates trees in aerial images. It has been designed to delineate trees in challenging dense tropical forests for a range of ecological applications.

This tutorial takes you through the key steps. Example Colab notebooks are also available but are not updated frequently so functions and parameters may need to be adjusted to get things working properly.

The standard workflow includes:

  1. Tile the orthomosaics and crown data (for training, validation and testing)
  2. Train (and tune) a model on the training tiles
  3. Evaluate the model performance by predicting on the test tiles and comparing to manual crowns for the tiles
  4. Using the trained model to predict the crowns over the entire region of interest

Training crowns are used to teach the network to delineate tree crowns.

predictions predictions

Here is an example image of the predictions made by Detectree2.

predictions

Applications

Tracking tropical tree growth and mortality

predicting

Counting urban trees (Buffalo, NY)

predicting

Multi-temporal tree crown segmentation

predicting

Liana detection and infestation mapping

In development

predicting

Tree species identification and mapping

In development

To do

  • Functions for multiple labels vs single "tree" label

Project Organization

├── LICENSE
├── Makefile
├── README.md
├── detectree2
│   ├── data_loading
│   ├── models
│   ├── preprocessing
│   ├── R
│   └── tests
├── docs
│   └── source
├── model_garden
├── notebooks
│   ├── colab
│   ├── colabJB
│   ├── colabJH
│   ├── colabKoay
│   ├── colabPan
│   ├── colabSeb
│   ├── exploratory
│   ├── mask_rcnn
│   │   ├── testing
│   │   └── training
│   ├── reports
│   └── turing
├── report
│   ├── figures
│   └── sections
└── requirements

Code formatting

To automatically format your code, make sure you have black installed (pip install black) and call black . from within the project directory.


Copyright (c) 2022, James G. C. Ball

detectree2's People

Contributors

0scarj1ang avatar ancazugo avatar james-hirst-1998 avatar ma595 avatar patball1 avatar ptresson avatar shmh40 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

detectree2's Issues

`get_filenames` concatenates two absolute paths

The get_filenames() function of detectree2.models.train uses glob method to filter the files with a .png extension in the directory given as parameter. The glob method returns the absolute path of all the files. Then, the function uses os.path.join() to concatenate the directory and the glob result; hence the result is a non-existent path.

For instance, if the following chunk is run:

in_dir = Path("drive/Shareddrives/detectree2_Cambridge/data/Cambridge/tiles_0.25m_160_20_0_samp")
tiles_dir = in_dir / 'tiles'

files = get_filenames(str(tiles_dir) + '/')

This is the result:

[{'file_name': 'drive/Shareddrives/detectree2_Cambridge/data/Cambridge/tiles_0.25m_160_20_0_samp/drive/Shareddrives/detectree2_Cambridge/data/Cambridge/tiles_0.25m_160_20_0_samp/160_20_0CityCentre_2017_32630_712515_5789228_160_20_32630.png'},
 {'file_name': 'drive/Shareddrives/detectree2_Cambridge/data/Cambridge/tiles_0.25m_160_20_0_samp/drive/Shareddrives/detectree2_Cambridge/data/Cambridge/tiles_0.25m_160_20_0_samp/160_20_0CityCentre_2017_32630_713155_5788908_160_20_32630.png'}]

So the path is duplicated, which causes the predict_on_data() function to give NULL results because it couldn't read the png files.

Failure to install on 64-bit Windows 10 system, and Python version 3.8.10

Collecting rasterio==1.3a3
Using cached rasterio-1.3a3.tar.gz (401 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... error
ERROR: Command errored out with exit status 1:
command: 'C:\Users\xxxx\AppData\Local\Programs\Python\Python38\python.exe' 'C:\Users\xxxx\AppData\Local\Programs\Python\Python38\lib\site-packages\pip_vendor\pep517\in_process_in_process.py' get_requires_for_build_wheel 'C:\Users\xxxx\AppData\Local\Temp\tmprw8grws6'
cwd: C:\Users\xxxx\AppData\Local\Temp\pip-install-ozyk4_bv\rasterio_97d0b87c61a740218c42e09690312378
Complete output (2 lines):
INFO:root:Building on Windows requires extra options to setup.py to locate needed GDAL files. More information is available in the README.
ERROR: A GDAL API version must be specified. Provide a path to gdal-config using a GDAL_CONFIG environment variable or use a GDAL_VERSION environment variable.

Suggestions to improve clarity of tutorial

  • check for missing function calls - project_to_geojson, stitch_crowns, clean_crowns
  • clarify how to connect with personal google drive
  • clarify Confidence_scores and how filtering should be applied as a post-processing step
  • note to avoid redownloading models as this impacts bandwidth
  • add illustrations of training data, outputs etc
  • explain the early stopping mechanism in more detail

pip install detectree2

when I use pip install detectree2 ,it ERROR:

ERROR: Could not find a version that satisfies the requirement detectron2 (from detectree2) (from versions: none)
ERROR: No matching distribution found for detectron2

tutorial.html

"If you would just like to make predictions on an orthomosaic with a pre-trained model from the model_garden, skip to part 4."

2.3 and 2.4 are missing from this document. When will they be updated?

Do I need to run this file now if I want to use the pre-trained model provided by the author for image prediction ?

forest-modelling-detectree-v4.ipynb

Thanks again to the author's intention to answer, I wish you a happy life!

Environment, dependency management and packaging repo.

We need to choose a environment management tool (virtual env, conda, pipenv), package dependency resolver (conda, pipenv, poetry), and package repository (PyPI, anaconda, etc..).

Currently pip install git+https://github.com/PatBall1/detectree2.git does not work on clusters or systems without the GDAL headers installed (pip install GDAL). We can install GDAL headers using package managers (apt, yum etc) or load modules on clusters (module load GDAL), but there is no guarantee that the location is always in the same place. A typical ubuntu install looks like:

sudo add-apt-repository ppa:ubuntugis/ppa
sudo apt update
sudo apt install libgdal-dev
sudo apt install gdal-bin
export CPLUS_INCLUDE_PATH=/usr/include/gdal
export C_INCLUDE_PATH=/usr/include/gdal
pip install GDAL

Requiring users to manually specify the GDAL location; it might, however, be possible to install development headers from source and include as part of the pip build process?

Alternatively, we can obtain all detectree2 dependencies with Conda (detectron2, GDAL and openCV etc), and then package to conda forge. The environment is specified in an environment.yaml file. The environment is easily reproducible due to the conda-lock file which ensures that dependencies are transitively pinned. Packaging is done using conda-build (meta.yml file), i.e. https://github.com/conda-forge/staged-recipes with conda-build tutorial: https://docs.conda.io/projects/conda-build/en/latest/user-guide/tutorials/building-conda-packages.html. The distribution (archived package) (.whl / .tar) will sit on conda-forge. More on the merits of using Conda here: https://pythonspeed.com/articles/conda-dependency-management/

As another alternative, it is possible to combine Conda with Poetry as shown here: https://stackoverflow.com/questions/70851048/does-it-make-sense-to-use-conda-poetry

Poetry makes packaging easy, and dependency management (using pyproject.toml) is faster than Conda. https://www.youtube.com/watch?v=QX_Nhu1zhlg&t=676s

Neaten packaging by either adopting Conda or investigate Poetry as an alternative.

outputs.stitch_crowns is registering the same CRS as being different

Hi there, am trying to stitch predictions together and am getting the following error:

ValueError: Cannot determine common CRS for concatenation inputs, got ['WGS 84 / UTM zone 30N', 'WGS 84 / UTM zone 30N']. Use to_crs() to transform geometries to the same CRS before merging.

I have attempted to force a different crs onto the predictions using a for_loop (shown below but error persists). The two "different" crs's shown are identical!

specify the input and output folders

input_folder = tiles_path + "predictions_geo/"
output_folder = tiles_path + "predictions_geo_crs32630/"

specify the target CRS

target_crs = 'EPSG:32630'

loop through all files in the input folder

for filename in os.listdir(input_folder):
# construct the full file path for the input and output files
input_filepath = os.path.join(input_folder, filename)
output_filepath = os.path.join(output_folder, filename)

# read the file using geopandas
gdf = gpd.read_file(input_filepath)

# apply the to_crs() function to the file
gdf_reprojected = gdf.to_crs(target_crs)

# write the reprojected file to the output folder
gdf_reprojected.to_file(output_filepath)

Workflow dependencies

On ubuntu-latest (20.04)
Rasterio errors (fixed by updating to 1.3.0 (from 1.2.10)

GDAL errors
Workflow attempts to build wheel (latest version 3.5.1) for python3.10. Can't find a way to search for pre-existing GDAL wheels. There may be wheels available for older GDAL versions?
error: ‘GDT_UInt64’ undeclared (first use in this function); did you mean ‘GDT_UInt32’?
Have installed all development headers.

On ubuntu-22.04
Commands fail:
sudo add-apt-repository -y ppa:ubuntugis/ppa

Rasterio 1.3a3 unable to build wheel

I can install rasterio using conda install -c conda-forge rasterio.

The setup file does not recognize it as being installed and attempts to use rasterio 1.3a3 and cannot build the associated wheel.

Running windows using pip install git+url instructions

Collecting rasterio==1.3a3
Using cached rasterio-1.3a3.tar.gz (401 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [2 lines of output]
INFO:root:Building on Windows requires extra options to setup.py to locate needed GDAL files. More information is available in the README.
ERROR: A GDAL API version must be specified. Provide a path to gdal-config using a GDAL_CONFIG environment variable or use a GDAL_VERSION environment variable.
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

Understand poor generalisation across image scales

Models do not generalise well across image scales. Can this be fixed by adjusting the data augmentation parameters?

def build_train_loader(self, cls, cfg):
"""Summary_.
Args:
cfg (_type_): _description_
Returns:
_type_: _description_
"""
return build_detection_train_loader(
cfg,
mapper=DatasetMapper(
cfg,
is_train=True,
augmentations=[
T.Resize((800, 800)),
T.RandomBrightness(0.8, 1.8),
T.RandomContrast(0.6, 1.3),
T.RandomSaturation(0.8, 1.4),
T.RandomRotation(angle=[90, 90], expand=False),
T.RandomLighting(0.7),
T.RandomFlip(prob=0.4, horizontal=True, vertical=False),
T.RandomFlip(prob=0.4, horizontal=False, vertical=True),
],
),
)

Add tests

Create a suite of tests:

Unit testing:
Test small components of detectree2:

Regression testing:

CI using github actions:

  • linting / style etc.
  • build tests (conda environment, pip)
  • use pre-built wheel / tar.gz in github actions using artifacts. (mentioned in docs issue)
  • run prediction on CPU
  • run prediction on GPU (not currently supported on github actions - is it possible to set up a runner to use CSD3 resources instead?)

TypeError: 'float' object is not subscriptable during training

Hello,

I would like to include four locations for training my model.

Error

Here is the error code:

Traceback (most recent call last):
  File "driver.py", line 75, in <module>
    trainer = MyTrainer(cfg, patience=4)
  File "/home/nieding/detectree2/detectree2/models/train.py", line 182, in __init__
    super().__init__(cfg)
  File "/home/nieding/mambaforge/envs/detectree2/lib/python3.8/site-packages/detectron2/engine/defaults.py", line 378, in __init__
    data_loader = self.build_train_loader(cfg)
  File "/home/nieding/mambaforge/envs/detectree2/lib/python3.8/site-packages/detectron2/engine/defaults.py", line 547, in build_train_loader
    return build_detection_train_loader(cfg)
  File "/home/nieding/mambaforge/envs/detectree2/lib/python3.8/site-packages/detectron2/config/config.py", line 207, in wrapped
    explicit_args = _get_args_from_config(from_config, *args, **kwargs)
  File "/home/nieding/mambaforge/envs/detectree2/lib/python3.8/site-packages/detectron2/config/config.py", line 245, in _get_args_from_config
    ret = from_config_func(*args, **kwargs)
  File "/home/nieding/mambaforge/envs/detectree2/lib/python3.8/site-packages/detectron2/data/build.py", line 344, in _train_loader_from_config
    dataset = get_detection_dataset_dicts(
  File "/home/nieding/mambaforge/envs/detectree2/lib/python3.8/site-packages/detectron2/data/build.py", line 241, in get_detection_dataset_dicts
    dataset_dicts = [DatasetCatalog.get(dataset_name) for dataset_name in names]
  File "/home/nieding/mambaforge/envs/detectree2/lib/python3.8/site-packages/detectron2/data/build.py", line 241, in <listcomp>
    dataset_dicts = [DatasetCatalog.get(dataset_name) for dataset_name in names]
  File "/home/nieding/mambaforge/envs/detectree2/lib/python3.8/site-packages/detectron2/data/catalog.py", line 59, in get
    return f()
  File "/home/nieding/detectree2/detectree2/models/train.py", line 489, in <lambda>
    DatasetCatalog.register(name + "_" + d, lambda d=d: combine_dicts(train_location,
  File "/home/nieding/detectree2/detectree2/models/train.py", line 443, in combine_dicts
    tree_dicts += get_tree_dicts(d, classes=classes, classes_at=classes_at)
  File "/home/nieding/detectree2/detectree2/models/train.py", line 388, in get_tree_dicts
    px = [a[0] for a in anno["coordinates"][0]]
  File "/home/nieding/detectree2/detectree2/models/train.py", line 388, in <listcomp>
    px = [a[0] for a in anno["coordinates"][0]]
TypeError: 'float' object is not subscriptable

Code snippet

The following code describes how I register the datasets:

# register hain
train_location = '../data/Bamberg_Hain/training/training_tiles/train/'
register_train_data(train_location, 'Bamberg_Hain', 1) # registers train and val sets
print("Hain datasets registered")

# register stadtwald
train_location = '../data/Bamberg_Stadtwald/training/training_tiles/train/'
register_train_data(train_location, 'Bamberg_Stadtwald', 1) # registers train and val sets
print("Stadtwald datasets registered")

[...]

# set models
base_model = "COCO-InstanceSegmentation/mask_rcnn_R_101_FPN_3x.yaml" #with api
pre_trained_model = site_folder + 'models/220723_withParacouUAV.pth'

# registered sets
# registered sets
trains = ("Bamberg_Hain_train", "Bamberg_Stadtwald_train" "Tretzendorf_train", "Schiefer_train")
tests = ("Bamberg_Hain_val", "Bamberg_Stadtwald_val" "Tretzendorf_val", "Schiefer_val")

Update:

Ok with a fresh mind, I found that the error message was not misleading and that a sudden float object was the reason for the failure.

Rework tiling

Tiling function needs to be broken down into smaller functions. Introduce pathlib.

Error downloading model_garden

Following error on attempted install

Downloading model_garden/220723_withParacouUAV.pth (503 MB)
Error downloading object: model_garden/220723_withParacouUAV.pth (b2fc7ff): Smudge error: Error downloading model_garden/220723_withParacouUAV.pth (b2fc7ff3006b429ddbae947ead4149efb74370df7a5686de0af8573b13f62537): batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access

Geopackage not displaying geometry on QGIS

After stitching a set of predictions with the following lines of code, I create a geopackage file containing the polygons of the predictions. Still, when uploading the file in QGIS, the polygons are not present, even though they can be visualised as a geopandas data frame in Python. The attributes are present once uploaded in QGIS but cannot be seen on the map.

crowns_final = stitch_crowns(test_pred_geo_folder, 1)
crowns_final = crowns_final[crowns_final.is_valid]
crowns_final = clean_crowns(crowns_final, 0.6)
crowns_final = crowns_final[crowns_final["Confidence_score"] > 0.5]
crowns_final = crowns_final.set_geometry('geometry')
crowns_final.to_file(train_out_dir + f"test_crowns_out.gpkg")

test_pred_geo_folder and train_out_dir are predefined variables pointing to existing folders.

cannot import model_zoo


ModuleNotFoundError Traceback (most recent call last)
in
10 from PIL import Image
11 from pathlib import Path
---> 12 from detectron2 import model_zoo
13 from detectron2.engine import DefaultPredictor, DefaultTrainer
14 from detectron2.config import get_cfg

9 frames
/usr/local/lib/python3.7/dist-packages/detectron2/utils/tracing.py in
2 from typing import Union
3 import torch
----> 4 from torch.fx._symbolic_trace import _orig_module_call
5 from torch.fx._symbolic_trace import is_fx_tracing as is_fx_tracing_current
6

ModuleNotFoundError: No module named 'torch.fx._symbolic_trace'


NOTE: If your import is failing due to a missing package, you can
manually install dependencies using either !pip or !apt.

To view examples of installing some common dependencies, click the
"Open Examples" button below.

Transfer learning with detectree2

Hi guys,
I am considering whether we can use the pre-trained model like "220723_withParacouUAV.pth" to train with local datasets continuously.
I had tried the idea but got lots of bugs that I couldn't fix. Did anyone try and have better solutions?

Distribute on conda-forge or pypi

To enable easy use of detectree2 we should consider distributing on conda-forge or PyPI.

Packaging on PyPI is made difficult as detectron2 is not available there.

  • Consider maintaining detectron2 release for detectree2.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.