geoalert / aeronetlib Goto Github PK
View Code? Open in Web Editor NEWA Remote Sensing data handling library for Deep Learning
License: MIT License
A Remote Sensing data handling library for Deep Learning
License: MIT License
For example, if it is written as
{'type': 'name', 'properties': {'name': 'urn:ogc:def:crs:OGC:1.3:CRS84'}}
Mast probably, we should replace string CRS with rasterio.crs.CRS object and make proper input from string and json.
And also to the BandCollection, BandSample and all the inheritors. Now it is only in Band which is not right.
Current vectorization is based on OpenCV and produces a simplified output.
It can lead to errors due to self-intersection of the geometries, and can cause holes and intersections in case of two near objects.
Also, we want a topojson-like structure if the objects of two different classes have the same boundary, but currently their boundaries will be different due to separate vectorization.
So what we need is:
The next step will be a topojson extension based on the current result.
Illustration: current result of the vectorization of two objects with the shared boundary - both intersection and gap are present.
No errors during installation via pip install aeronetlib/
. But then error fall down after trying to import aeronetlib:
from aeronet.converters.split import split
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/user/.conda/envs/test_aeronetlib/lib/python3.7/site-packages/aeronet/converters/split.py", line 4, in <module>
from ..dataset import BandCollection
File "/home/user/.conda/envs/test_aeronetlib/lib/python3.7/site-packages/aeronet/dataset/__init__.py", line 8, in <module>
from .vector import Feature
File "/home/user/.conda/envs/test_aeronetlib/lib/python3.7/site-packages/aeronet/dataset/vector/__init__.py", line 1, in <module>
from .feature import Feature, FeatureCollection
File "/home/user/.conda/envs/test_aeronetlib/lib/python3.7/site-packages/aeronet/dataset/vector/feature.py", line 2, in <module>
import rtree
ModuleNotFoundError: No module named 'rtree'
rtree Is imported from aeronet/dataset/vector/feature.py
But there no rtree package in requirenments.txt .
I tried to install rtree via pip https://pypi.org/project/Rtree/ but pip had fall following error.
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
Collecting Rtree
Using cached Rtree-0.9.3.tar.gz (520 kB)
ERROR: Command errored out with exit status 1:
command: home/user/.conda/envs/test_aeronetlib/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-dgw70zvn/Rtree/setup.py'"'"'; __file
__='"'"'/tmp/pip-install-dgw70zvn/Rtree/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __f
ile__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-dgw70zvn/Rtree/pip-egg-info
cwd: /tmp/pip-install-dgw70zvn/Rtree/
Complete output (15 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-dgw70zvn/Rtree/setup.py", line 3, in <module>
import rtree
File "/tmp/pip-install-dgw70zvn/Rtree/rtree/__init__.py", line 1, in <module>
from .index import Rtree
File "/tmp/pip-install-dgw70zvn/Rtree/rtree/index.py", line 6, in <module>
from . import core
File "/tmp/pip-install-dgw70zvn/Rtree/rtree/core.py", line 145, in <module>
rt.Error_GetLastErrorNum.restype = ctypes.c_int
File "/home/user/.conda/envs/test_aeronetlib/lib/python3.7/ctypes/__init__.py", line 377, in __getattr__
func = self.__getitem__(name)
File "/home/user/.conda/envs/test_aeronetlib/lib/python3.7/ctypes/__init__.py", line 382, in __getitem__
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: /home/.conda/envs/test_aeronetlib/bin/python: undefined symbol: Error_GetLastErrorNum
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
I had success installing rtree via conda install -c conda-forge rtree
.
Then another error occur:
from aeronet.converters.split import split
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/user/.conda/envs/test_aeronetlib/lib/python3.7/site-packages/aeronet/converters/split.py", line 4, in <module>
from ..dataset import BandCollection
File "/home/user/.conda/envs/test_aeronetlib/lib/python3.7/site-packages/aeronet/dataset/__init__.py", line 11, in <module>
from .io import Predictor
File "/home/user/.conda/envs/test_aeronetlib/lib/python3.7/site-packages/aeronet/dataset/io.py", line 3, in <module>
from tqdm import tqdm
ModuleNotFoundError: No module named 'tqdm'
tqdm installs successfully using pip or conda.
Something like this
def pix_from_coords(x, y, transform, size):
return int(round((x - transform[2])/transform[0])), int(round((y - transform[5])/transform[4] ))
def calc_raster_boundaries(xmin, xmax, ymin, ymax, transform, size):
# in given crs.
# Assume that xmin -> minimum column, ymax -> minimum row
#print(xmin,xmax, ymin, ymax)
x1, y1 = pix_from_coords(xmin, ymax, transform, size)
x2, y2 = pix_from_coords(xmax, ymin, transform, size)
#print(x1, x2, y1, y2)
x1 = min(max(0, x1), size[1])
y1 = min(max(0, y1), size[0])
x2 = min(max(0, x2), size[1])
y2 = min(max(0, y2), size[0])
if y1 == y2 or x1 == x2:
return 0,0,0,0
return x1, y1, x2-x1, y2-y1
#save a spatial subset
def subsample(input_ds, output_ds, grid, cell):
bc = BandCollection(parse_directory(input_ds, RGB))
#print(bc.crs, bc.transform)
xmin, ymin, xmax, ymax = grid[cell].shape.bounds
x1, y1, width, height = calc_raster_boundaries(xmin, xmax, ymin, ymax, bc[0].transform, bc[0].shape)
if width == 0 or height == 0:
print('Subset in {} cell is empty'.format(cell))
return
else:
print(width, height)
sample = bc.sample(x=x1, y=y1, width=width, height=height)
return sample
Now the _valid function always returns a polygon, which results in POLYGON_EMPTY every time we try to create a Feature from Poing, LineString or any other shape with zero area.
It would be useful to support all types of the geometries, most probably apart from GeometryCollection, as it should be divided into several Features by geometry type.
Along with it, we would need a method to sort\filter a FeatureCollection by type
The modules BandCOllection and FeatureCollection are not being imported even when all the dependencies are installed.
In resample, reproject, reproject_to - for efficiency
Now the opencv vectorization is invoked with fixed approximation parameters, and if we make epsilon=0.0 it will only affect the second stage of contour simplification.
It is better to allow cv2.APPROX_SIMPLE in find_contours, because it may give better results with small and jagged pixel boundaries, because it will not create sharp angles.
There can be a situation when we want the vector data to represent all the extent divided into a set of classes/areas rather than separate objects.
Topojson https://github.com/osgn/python-topojson is a good way to represent it; it also allows to simplify shared boundaries without gaps.
What we need:
All the neural network things belong to the neural networks libs. We can leave here some connected utils that are only for the remote sensing data, but should remove everything else and make a reference to the external lib if necessary.
File attached. Need to find a reasonable solution or make a bypass
900_pred.geojson.txt
(txt because github does not allow geojson)
At least the automatically generated from the comments (ask @qubvel how to), and the wiki with the introductory remarks about all the satellite stuff.
We can use any supported by GDAL data type for reading, but only GTiff for writing.
In the same time, BandSample.save() has argument 'extension' that can imply that other formats are allowed, but no.
It would be convenient to save in the same format if needed.
Maybe also more saving options, like 'compression' etc.
Current temp file creation is only linux-compatible. Maybe use of tempfile package can make it more general.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.