orthoseg / orthoseg

OrthoSeg makes it easy to train neural networks to segment orthophotos.

Orthophoto segmentation

This project provides a procedure and software to make it easy to:

1. create a training dataset for a topic of your choice
2. train a neural network to segment orthophotos
3. run the segmentation on a larger area + vectorize the result
4. apply some basic postprocessing like dissolve,...

Only open source software is needed, eg. QGIS and tensorflow.

Installation and usage instructions can be found in the orthoseg wiki

orthoseg's Issues

Add option to use topologic simplify

For multi-class predictions, often there different detected classes are located next to each other in the result. When simplification is applied to each resulting polygon separately, the result of the simplification will often result in gaps between both polygon borders.

If, prior to the simplification, the polygons would be converted to topologies, the simplification on the common line of the two polygons would only be applied once and hence will result in a border without gaps, even if the topologies are afterwards saved as seperate polygons again.

Revisit config of reclassify_to_neighbour

Now the query needs to be configured in predict and postprocess + it results in yet another output file.

Should be cleaned up.

references #86

ENH: Add support to train on different resolutions for one subject

Improve/add options to monitor training progress

• add support for one hot IOU + make default
• it is already possible to configure which (combination of) metrics is used to monitor training progress/selection of best model, but it is always a combination of training metrix and validation metrics. Make this configurable as well.

BUG: Topologic simplify doesn't respect topology in some cases

In some cases the topologic simplify still results in gaps and slivers being created, e.g.:

Input:

After topologic simplification:

Small test file that can be used to reproduce the issue:
bug_data_small6.zip

ENH: add feature to reclassify (small) features to the class of its neighbour

Configurable via the following parameter:

[predict]

# Reclassify all detected polygons that comply to the query provided to the class of
# the neighbour with the longest border with it.
# The query need to be in the form to be used by pandas.DataFrame.query() and the
# following columns are available to query on:
#   - area: the area of the polygon, as calculated by GeoSeries.area
#   - perimeter: the perimeter of the polygon, as calculated by GeoSeries.length
# Eg.: reclassify_to_neighbour_query = area <= 5
reclassify_to_neighbour_query

Change default columns name for class in label_polygons to classname

The default column name for the class in a training data input file is "label_name", but in the predicted output the column is called "classname". Because of this, the class name is lost when copying rows from the prediction to the polygons label file which is impractical.

So, the default name in labelpolygons.gpkg should be classname as well, but label_name should stay supported as well for backwards compatibility

Add option to specify the minimum probability in the result

Now, a hardcoded 50% probability is used.

BUG: train gives an error after training if there are no "test" locations defined

Not blocking, but obviously cleaner if this doesn't give an error.

Error:

Exception: Found no ['.png', '.tif', '.jpg'] images to predict in X:\Monitoring\OrthoSeg\animals\training\01\test\image

Traceback (most recent call last): File "C:\Tools\mambaforge\envs\orthoseg\lib\site-packages\orthoseg\train.py", line 437, in train predicter.predict_dir( File "C:\Tools\mambaforge\envs\orthoseg\lib\site-packages\orthoseg\lib\predicter.py", line 156, in predict_dir raise ValueError(f"Found no {input_ext} images to predict in {input_image_dir}") ValueError: Found no ['.png', '.tif', '.jpg'] images to predict in X:\Monitoring\OrthoSeg\animals\training\01\test\image 

Add early check in trainingsdata validation on whether the image layers exist

Automatically detect the axes order of the CRS used

Would be cleaner than #18

Maybe this can be used: https://pyproj4.github.io/pyproj/stable/examples.html#step-1-inspect-crs-definition-to-ensure-proper-area-of-use-and-axis-order

or if above doesn't work, make it configurable somewhere?

Add support for topological simplify in postprocess

Now it is only available in inline processing.

Depends on geofileops/geofileops#161

Improve test coverage...

At the moment of writing 25%... not really top-notch:
https://app.codecov.io/gh/orthoseg/orthoseg

BUG: support for WMS username/password broken

The support for username/password for WMS sources is broken since version 0.5.0.

Add check if location BBOXs are of the right size

Add support to load images from a WMTS service

Add option to limit CPU usage while predicting

Since version 0.3 it's possible to limit the amount of parallelisation used in postprocessing, but in some cases the CPU usage while predicting can become very high (up to 100%) as well.

Making the nb_parallel parameter general instead of specific for postprocessing seems the cleanest solution.

Add support to load images from an xyz service

Couldn't find "footballfields.ini" file under "/orthoseg/sample_projects/footballfields/"

Was trying to run the sample project football fields from sample_projects, but couldn't find the file name "footballfields.ini" to run this command
orthoseg_load_images --config ~/orthoseg/sample_projects/footballfields/footballfields.ini

Errors when predicting with topologic simplify

For some predicted tiles an error is thrown in the inline-post processing during the prediction.

These are the errors that seem to occur:

• File "C:\Users\pierog\projects_github\orthoseg\orthoseg\util\vector_util.py", line 151, in simplify_topolines       
      topoline_simpl = topolines_simpl.geoms[index].coords  # type: ignore
  AttributeError: 'LineString' object has no attribute 'geoms'
  

• fiona.errors.GeometryTypeValidationError: Record's geometry type does not match collection 
      schema's geometry type: 'LineString' != 'Polygon'
  

• File "C:\Users\pierog\projects_github\orthoseg\orthoseg\util\vector_util.py", line 183, in simplify_topo_orthoseg        
            topolines_simplified = simplify_topolines(
       File "C:\Users\pierog\projects_github\orthoseg\orthoseg\util\vector_util.py", line 147, in simplify_topolines
            assert topolines_simpl is not None
  

Change footballfields sample project to a multiclass segmentation project

This would make it a better example for the different configuration options.

eg. add tennis field detection

Update sample project footballfields + wiki documentation

The sample project + the description how to run it is outdated. Should be updated

Add support for a WMS layer with username/password authentication

Add traindata type todo

Now there are label types train, validation and test. A type todo would be practical to signify label locations that still need to be digitized.

Creation of spatial indexes on the output geopackage files fails on Windows

Wherever geopackage files are created in orthoseg on Windows, rtree creation errors are shown and the spatial index is not created.

This is a known issue. Probably there is an incompatibility between packages installed that results in a wrong version of sqlite being used... without support for rtree indexes.

As of now there is no known solution :-(.

ENH: Add support to overrule configuration parameters via command line arguments

Add support for training data from layers in different projections

Prediction pre-training not useful if prediction classes changed?

When training a network, orthoseg first runs a prediction on the training dataset based on the last previous version of the prediction network.

However, if the classes changed between both, this gives problems, so better skip prediction in this case?

Enable tests using github actions

Prediction throws error on Ubuntu 22.04

Hi all,

So I created the conda environment for orthoseg (resulting environment attached). I have followed the wiki entry and all is fine until I try to predict.

When I try to predict I get the following error (on Ubuntu 22.04 LTS).

$ orthoseg_predict --config ./work/arpa/orthoseg/sample_projects/orthoseg/sample_projects/footballfields/footballfields.ini 
2022-11-29 16:13:46.859616: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2022-11-29 16:13:46.859638: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
Segmentation Models: using keras framework.
16:13:48.804|INFO|root|Start predict for config footballfields
16:13:48.807|INFO|root|Best model found: /home/jp/work/arpa/orthoseg/sample_projects/orthoseg/sample_projects/footballfields/models/footballfields_01_0.92512_242.hdf5
16:13:48.807|INFO|root|Tensorrt is available, so use optimized model
16:13:48.807|ERROR|root|ERROR while running predict for task footballfields
Traceback (most recent call last):
  File "/opt/miniconda3/envs/orthoseg/lib/python3.9/site-packages/orthoseg/predict.py", line 166, in predict
    best_model["filepath"].parent / best_model["filepath"].stem + "_optim"
TypeError: unsupported operand type(s) for +: 'PosixPath' and 'str'
16:13:48.808|ERROR|root|Error: ERROR while running predict for task footballfields
Traceback (most recent call last):
  File "/opt/miniconda3/envs/orthoseg/lib/python3.9/site-packages/orthoseg/predict.py", line 166, in predict
    best_model["filepath"].parent / best_model["filepath"].stem + "_optim"
TypeError: unsupported operand type(s) for +: 'PosixPath' and 'str'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/miniconda3/envs/orthoseg/lib/python3.9/site-packages/orthoseg/predict.py", line 294, in main
    predict_args(sys.argv[1:])
  File "/opt/miniconda3/envs/orthoseg/lib/python3.9/site-packages/orthoseg/predict.py", line 71, in predict_args
    predict(config_path=Path(args.config))
  File "/opt/miniconda3/envs/orthoseg/lib/python3.9/site-packages/orthoseg/predict.py", line 289, in predict
    raise Exception(message) from ex
Exception: ERROR while running predict for task footballfields
Traceback (most recent call last):
  File "/opt/miniconda3/envs/orthoseg/lib/python3.9/site-packages/orthoseg/predict.py", line 166, in predict
    best_model["filepath"].parent / best_model["filepath"].stem + "_optim"
TypeError: unsupported operand type(s) for +: 'PosixPath' and 'str'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/miniconda3/envs/orthoseg/bin/orthoseg_predict", line 8, in <module>
    sys.exit(main())
  File "/opt/miniconda3/envs/orthoseg/lib/python3.9/site-packages/orthoseg/predict.py", line 294, in main
    predict_args(sys.argv[1:])
  File "/opt/miniconda3/envs/orthoseg/lib/python3.9/site-packages/orthoseg/predict.py", line 71, in predict_args
    predict(config_path=Path(args.config))
  File "/opt/miniconda3/envs/orthoseg/lib/python3.9/site-packages/orthoseg/predict.py", line 289, in predict
    raise Exception(message) from ex
Exception: ERROR while running predict for task footballfields

Any ideas why?
orthoseg_env.txt

Non blocking errors in prediction are ignored/not reported

If errors occur during the prediction that aren't immediately blocking, they end up being ignored instead of being reported at the end of the prediction run.

Add command to (only) validate a training dataset

This way a seperate pipeline can be setup that only checks this so you can do an immediate check on this instead of having to wait till a training job is started.

Add support to use more + UINT16/float32 bands

This has quite some impact.

Most important: replace use keras ImageDataGenerator for augmentation. Based on a quick scan, Albumentations seems like the best replacement. This also is a step towards supporting pytorch as well...

ENH: Support reading images directly from the datasource while predicting

For layers that are stored as local files the cache probably isn't needed for performance and definitely not for reducing server load, so it only takes unnecessary space.

Add pre-simplify with RDP when using (inline) LANG simplification

The LANG simplification algorithm seems to give a smoother output than RDP and VW, but the number of points removed is limited by the lookahead parameter. Because the inline simplification during prediction works on the polygonized vector data there are a lot of points, so the limit by the LANG lookahead parameter is an issue.

By pre-simplifying the polygons with RDP using a smaller tolerance (eg. 50%) this disadvantage can be largely countered.

Add option to disable ssl verification when downloading sample projects

model file for sample projet not being downloaded

When running:
orthoseg_load_sampleprojects
to download the sample projects, the model file (.hdf5) being downloaded consists of HTML content rather than the HDF5 itself.

This appears to be happening because the model file is being downloaded from a Google Drive location and when trying to get the file, Google is instead sending an HTML page, saying that the file is too big to be scanned.

As a workaround, I downloaded the file myself directly from the Google link: https://drive.google.com/file/d/1XmAenCW6K_RVwqC6xbkapJ5ws-f7-QgH

Add option(s) to do automatic cleanup of "old" models/predictions/...

Add support to use imagelayers using a local image in addition to WMS services

Add list of invalid geometries + coordinates to error mail...

Add option to use "k-fold" style seperation of train, validation and test instead of current "manual" way