Coder Social home page Coder Social logo

aeronetlib's People

Contributors

adeshkin avatar alexeytrekin avatar aliaksandr960 avatar godnik avatar hfmuzb avatar qubvel avatar sakharovgeoalert avatar vladimir-kozub avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

aeronetlib's Issues

Geojson CRS is not accepted in json format

For example, if it is written as
{'type': 'name', 'properties': {'name': 'urn:ogc:def:crs:OGC:1.3:CRS84'}}

Mast probably, we should replace string CRS with rasterio.crs.CRS object and make proper input from string and json.

Add exact vectorization

Current vectorization is based on OpenCV and produces a simplified output.
It can lead to errors due to self-intersection of the geometries, and can cause holes and intersections in case of two near objects.
Also, we want a topojson-like structure if the objects of two different classes have the same boundary, but currently their boundaries will be different due to separate vectorization.

So what we need is:

  • test if the current variant can make the exact vectorization if the parameters are correct ( probably not).
  • vectorization by the actual pixel boundaries (like rasterio.features.shapes(), but with contour hierarchy).
  • merge of the polygons that are connected by the corner, without self-intersection
  • simplification of the contours that will preserve correctness of the shapes

The next step will be a topojson extension based on the current result.

Screenshot from 2020-04-02 11-59-19

Illustration: current result of the vectorization of two objects with the shared boundary - both intersection and gap are present.

Fix requirenments.txt

No errors during installation via pip install aeronetlib/. But then error fall down after trying to import aeronetlib:

from aeronet.converters.split import split
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/.conda/envs/test_aeronetlib/lib/python3.7/site-packages/aeronet/converters/split.py", line 4, in <module>
    from ..dataset import BandCollection
  File "/home/user/.conda/envs/test_aeronetlib/lib/python3.7/site-packages/aeronet/dataset/__init__.py", line 8, in <module>
    from .vector import Feature
  File "/home/user/.conda/envs/test_aeronetlib/lib/python3.7/site-packages/aeronet/dataset/vector/__init__.py", line 1, in <module>
    from .feature import Feature, FeatureCollection
  File "/home/user/.conda/envs/test_aeronetlib/lib/python3.7/site-packages/aeronet/dataset/vector/feature.py", line 2, in <module>
    import rtree
ModuleNotFoundError: No module named 'rtree'

rtree Is imported from aeronet/dataset/vector/feature.py
But there no rtree package in requirenments.txt .
I tried to install rtree via pip https://pypi.org/project/Rtree/ but pip had fall following error.

WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
Collecting Rtree
  Using cached Rtree-0.9.3.tar.gz (520 kB)
    ERROR: Command errored out with exit status 1:
     command: home/user/.conda/envs/test_aeronetlib/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-dgw70zvn/Rtree/setup.py'"'"'; __file
__='"'"'/tmp/pip-install-dgw70zvn/Rtree/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __f
ile__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-dgw70zvn/Rtree/pip-egg-info
         cwd: /tmp/pip-install-dgw70zvn/Rtree/
    Complete output (15 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-dgw70zvn/Rtree/setup.py", line 3, in <module>
        import rtree
      File "/tmp/pip-install-dgw70zvn/Rtree/rtree/__init__.py", line 1, in <module>
        from .index import Rtree
      File "/tmp/pip-install-dgw70zvn/Rtree/rtree/index.py", line 6, in <module>
        from . import core
      File "/tmp/pip-install-dgw70zvn/Rtree/rtree/core.py", line 145, in <module>
        rt.Error_GetLastErrorNum.restype = ctypes.c_int
      File "/home/user/.conda/envs/test_aeronetlib/lib/python3.7/ctypes/__init__.py", line 377, in __getattr__
        func = self.__getitem__(name)
      File "/home/user/.conda/envs/test_aeronetlib/lib/python3.7/ctypes/__init__.py", line 382, in __getitem__
        func = self._FuncPtr((name_or_ordinal, self))
    AttributeError: /home/.conda/envs/test_aeronetlib/bin/python: undefined symbol: Error_GetLastErrorNum
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

I had success installing rtree via conda install -c conda-forge rtree.

Then another error occur:

from aeronet.converters.split import split
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/.conda/envs/test_aeronetlib/lib/python3.7/site-packages/aeronet/converters/split.py", line 4, in <module>
    from ..dataset import BandCollection
  File "/home/user/.conda/envs/test_aeronetlib/lib/python3.7/site-packages/aeronet/dataset/__init__.py", line 11, in <module>
    from .io import Predictor
  File "/home/user/.conda/envs/test_aeronetlib/lib/python3.7/site-packages/aeronet/dataset/io.py", line 3, in <module>
    from tqdm import tqdm
ModuleNotFoundError: No module named 'tqdm'

tqdm installs successfully using pip or conda.

Add functionality for generation of the sample by vector extent

Something like this

Find raster boundaries for the vector grid cells

def pix_from_coords(x, y, transform, size):
return int(round((x - transform[2])/transform[0])), int(round((y - transform[5])/transform[4] ))

def calc_raster_boundaries(xmin, xmax, ymin, ymax, transform, size):
# in given crs.
# Assume that xmin -> minimum column, ymax -> minimum row
#print(xmin,xmax, ymin, ymax)
x1, y1 = pix_from_coords(xmin, ymax, transform, size)
x2, y2 = pix_from_coords(xmax, ymin, transform, size)
#print(x1, x2, y1, y2)

x1 = min(max(0, x1), size[1])
y1 = min(max(0, y1), size[0])
x2 = min(max(0, x2), size[1])
y2 = min(max(0, y2), size[0])

if y1 == y2 or x1 == x2:
    return 0,0,0,0

return x1, y1, x2-x1, y2-y1

#save a spatial subset

def subsample(input_ds, output_ds, grid, cell):
bc = BandCollection(parse_directory(input_ds, RGB))
#print(bc.crs, bc.transform)
xmin, ymin, xmax, ymax = grid[cell].shape.bounds
x1, y1, width, height = calc_raster_boundaries(xmin, xmax, ymin, ymax, bc[0].transform, bc[0].shape)

if width == 0 or height == 0:
    print('Subset in {} cell is empty'.format(cell))
    return
else:
    print(width, height)

sample = bc.sample(x=x1, y=y1, width=width, height=height)
return sample

Allow all geometry types into Feature

Now the _valid function always returns a polygon, which results in POLYGON_EMPTY every time we try to create a Feature from Poing, LineString or any other shape with zero area.

It would be useful to support all types of the geometries, most probably apart from GeometryCollection, as it should be divided into several Features by geometry type.

Along with it, we would need a method to sort\filter a FeatureCollection by type

More options to vectorization

Now the opencv vectorization is invoked with fixed approximation parameters, and if we make epsilon=0.0 it will only affect the second stage of contour simplification.

It is better to allow cv2.APPROX_SIMPLE in find_contours, because it may give better results with small and jagged pixel boundaries, because it will not create sharp angles.

Topojson vectorization

There can be a situation when we want the vector data to represent all the extent divided into a set of classes/areas rather than separate objects.

Topojson https://github.com/osgn/python-topojson is a good way to represent it; it also allows to simplify shared boundaries without gaps.

What we need:

  • transform a featurecollection to topojson
  • simplify a featurecollection with shared boundaries by transformation to topojson, simplification and transform back

Remove all that is implemented in segmentation-models

All the neural network things belong to the neural networks libs. We can leave here some connected utils that are only for the remote sensing data, but should remove everything else and make a reference to the external lib if necessary.

Documentation

At least the automatically generated from the comments (ask @qubvel how to), and the wiki with the introductory remarks about all the satellite stuff.

Saving with drivers other than GTiff

We can use any supported by GDAL data type for reading, but only GTiff for writing.

In the same time, BandSample.save() has argument 'extension' that can imply that other formats are allowed, but no.

It would be convenient to save in the same format if needed.
Maybe also more saving options, like 'compression' etc.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.