bopen / elevation Goto Github PK
View Code? Open in Web Editor NEWPython script to download global terrain digital elevation models, SRTM 30m DEM and SRTM 90m DEM.
Home Page: http://elevation.bopen.eu
License: Apache License 2.0
Python script to download global terrain digital elevation models, SRTM 30m DEM and SRTM 90m DEM.
Home Page: http://elevation.bopen.eu
License: Apache License 2.0
Hi @alexamici, I've run into an error when requesting a DEM through using elevation.clip() in a Python script on Windows. It appears make is running into some kind of problem entering an AppData subdirectory. Here's the output I'm getting the Visual Studio console:
make: Entering directory 'C:/Users/lmonn/AppData/Local/bopen/elevation/Cache/SRTM1'
The system cannot find the path specified.
The syntax of the command is incorrect.
make: *** [Makefile:17: spool/N41/N41E012.hgt.gz] Error 1
make: Leaving directory 'C:/Users/lmonn/AppData/Local/bopen/elevation/Cache/SRTM1'
Traceback (most recent call last):
File "test_one.py", line 338, in <module>
elevation.clip(bounds=(12.35, 41.8, 12.65, 42), output=output, product='SRTM1')
File "C:\Users\lmonn\AppData\Local\Programs\Python\Python37-32\lib\site-packages\elevation\datasource.py", line 181, in clip
datasource_root = seed(bounds=bounds, **kwargs)
File "C:\Users\lmonn\AppData\Local\Programs\Python\Python37-32\lib\site-packages\elevation\datasource.py", line 153, in seed
ensure_tiles(datasource_root, ensure_tiles_names, **kwargs)
File "C:\Users\lmonn\AppData\Local\Programs\Python\Python37-32\lib\site-packages\elevation\datasource.py", line 115, in ensure_tiles
return util.check_call_make(path, targets=['download'], variables=variables_items, **kwargs)
File "C:\Users\lmonn\AppData\Local\Programs\Python\Python37-32\lib\site-packages\elevation\util.py", line 93, in check_call_make
subprocess.check_call(cmd, shell=True)
File "C:\Users\lmonn\AppData\Local\Programs\Python\Python37-32\lib\subprocess.py", line 347, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'make -C C:\Users\lmonn\AppData\Local\bopen\elevation\Cache\SRTM1 download ENSURE_TILES="N41/N41E012.tif"' returned non-zero exit status 2.
What do you think the issue could be?
Note: I've tried this with 'SRTM3' as well.
When I try to download the SRTM1 30m DEM of Italy with
eio --make_flags "-j 8" --product SRTM1 seed --bounds 6.6 36.6 18.6 47.1
I have a crash.
It seems that I obligatorily have to insert -k
flag as follow:
eio --make_flags "-j 8 -k" --product SRTM1 seed --bounds 6.6 36.6 18.6 47.1
Hello,
I have been running into this issue with elevation:
CalledProcessError: Command 'make -C C:\Users\AppData\Local\bopen\elevation\Cache\SRTM1 clean ' returned non-zero exit status 1.
This error is reproduced when I use the elevation.clean()
function, and it has been the same error with the elevation.clip()
function. It seems to be a permissions or install issue?
running Python: 3.7.13
Anaconda 2.2.0
Nice to have:
$ eio selfcheck
Your system is ready.
$ eio clip -o Rome-30m-DEM.tif --bounds 12.35 41.8 12.65 42
curl -s -o spool/N41/N41E012.hgt.gz.temp https://s3.amazonaws.com/elevation-tiles-prod/skadi/N41/N41E012.hgt.gz && mv spool/N41/N41E012.hgt.gz.temp spool/N41/N41E012.hgt.gz
gunzip spool/N41/N41E012.hgt.gz 2>/dev/null || touch spool/N41/N41E012.hgt
gdal_translate -q -co TILED=YES -co COMPRESS=DEFLATE -co ZLEVEL=9 -co PREDICTOR=2 spool/N41/N41E012.hgt cache/N41/N41E012.tif 2>/dev/null || touch cache/N41/N41E012.tif
rm spool/N41/N41E012.hgt
gdalbuildvrt -q -overwrite SRTM1.vrt cache/N41/N41E012.tif
cp SRTM1.vrt SRTM1.26be21147c87403fbf2dffe59bafacfb.vrt
gdal_translate -q -co TILED=YES -co COMPRESS=DEFLATE -co ZLEVEL=9 -co PREDICTOR=2 -projwin 12.35 42.0 12.65 41.8 SRTM1.26be21147c87403fbf2dffe59bafacfb.vrt /Users/matt/Downloads/Rome-30m-DEM.tif
rm -f SRTM1.26be21147c87403fbf2dffe59bafacfb.vrt
$ open Rome-30m-DEM.tif
Hi, am getting the following error when trying to run the code:
The system cannot find the path specified. make: Entering directory C:/Users/Jonathan/AppData/Local/bopen/elevation/Cache/SRTM1 The syntax of the command is incorrect. make: *** [spool/N53/N53W010.hgt.gz] Error 1 make: Leaving directory C:/Users/Jonathan/AppData/Local/bopen/elevation/Cache/SRTM1 Traceback (most recent call last): File "c:/Users/Jonathan/Desktop/Projects/threedee.py", line 30, in <module> elevation.clip(bounds = bounds, output = output, product = 'SRTM1') File "C:\Users\Jonathan\Anaconda3\lib\site-packages\elevation\datasource.py", line 181, in clip datasource_root = seed(bounds=bounds, **kwargs) File "C:\Users\Jonathan\Anaconda3\lib\site-packages\elevation\datasource.py", line 153, in seed ensure_tiles(datasource_root, ensure_tiles_names, **kwargs) File "C:\Users\Jonathan\Anaconda3\lib\site-packages\elevation\datasource.py", line 115, in ensure_tiles return util.check_call_make(path, targets=['download'], variables=variables_items, **kwargs) File "C:\Users\Jonathan\Anaconda3\lib\site-packages\elevation\util.py", line 93, in check_call_make subprocess.check_call(cmd, shell=True) File "C:\Users\Jonathan\Anaconda3\lib\subprocess.py", line 291, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'make -C C:\Users\Jonathan\AppData\Local\bopen\elevation\Cache\SRTM1 download ENSURE_TILES="N53/N53W010.tif N53/N53W009.tif"' returned non-zero exit status 2.
All looks good when I run "eio systemcheck". Any help appreciated.
On my system, I have an old version of curl (7.15.5.) Running curl --version exits with a status of 2.
eio selfcheck
runs the code (line 38 of util.py)
try:
subprocess.check_output(check_cli, shell=True, stderr=subprocess.STDOUT)
except subprocess.CalledProcessError:
msg.append('%r not found or not usable.' % tool_name)
which fails then, even though curl is available.
Perhaps you could instead do something like
try:
subprocess.call(check_cli.split(), stderr=subprocess.STDOUT)
except OSError:
msg.append('%r not found or not usable.' % tool_name)
We are using your package to download SRTM data for kenya
Hopefully I can spare some of the details (the functions shown but not explained hopefully have somewhat intuitive sounding names) but this is where / how we are using your package:
def export(self, region_name: str = 'kenya',
product: str = 'SRTM3',
max_download_tiles: int = 15) -> None:
"""
Export SRTm topography data
Arguments
----------
region_name: str = 'kenya'
The region to download. Must be one of the regions in the
region_lookup dictionary
product: {'SRTM1', 'SRTM3'} = 'SRTM3'
The product to download the data from
max_download_tiles: int = 15
By default, the elevation package doesn't allow more than 9
tiles to be downloaded. Kenya is 12 tiles - this increases the
limit to allow Kenya to be downloaded
"""
region = region_lookup[region_name]
output_tif = self.output_folder / f'{region_name}.tif'
if not output_tif.exists():
print(f'Downloading tiles. Saving as tif to {output_tif}')
try:
elevation.clip(bounds=self._region_to_tuple(region), # type: ignore
output=output_tif.resolve().as_posix(),
product=product,
max_download_tiles=max_download_tiles)
except Exception as e:
print(e)
elevation.clean() # type: ignore
output_nc = self.output_folder / f'{region_name}.nc'
if not output_nc.exists():
print(f'Converting {output_tif} to NetCDF format ({output_nc})')
self._tif_to_nc(output_tif, output_nc)
We get the following error message on a remote cluster. Is this something you have seen before?
(crp) chri4118@linux8:/soge-home/projects/crop_yield/ml_drought/scripts$ python export.py
The SRTM exporter requires GDAL and the elevation python package. The mac conda environment contains them.In addition, a (finnicky) ubuntu environment contains them in environment.ubuntu.cpu.gdal.yml
Downloading tiles. Saving as tif to ../data/raw/srtm/kenya.tif
Unable to acquire lock on `b'/soge-home/users/chri4118/.cache/elevation/SRTM3/.folder_lock'` due to [Errno 38] Function not implemented
Traceback (most recent call last):
File "/soge-home/users/chri4118/.conda/envs/crp/lib/python3.7/site-packages/fasteners/process_lock.py", line 99, in _try_acquire
self.trylock()
File "/soge-home/users/chri4118/.conda/envs/crp/lib/python3.7/site-packages/fasteners/process_lock.py", line 217, in trylock
self._trylock(self.lockfile)
File "/soge-home/users/chri4118/.conda/envs/crp/lib/python3.7/site-packages/fasteners/process_lock.py", line 250, in _trylock
fcntl.lockf(lockfile, fcntl.LOCK_EX | fcntl.LOCK_NB)
OSError: [Errno 38] Function not implemented
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "export.py", line 83, in <module>
export_srtm()
File "export.py", line 74, in export_srtm
exporter.export()
File "../src/exporters/srtm.py", line 77, in export
elevation.clean() # type: ignore
File "/soge-home/users/chri4118/.conda/envs/crp/lib/python3.7/site-packages/elevation/datasource.py", line 201, in clean
datasource_root, _ = ensure_setup(cache_dir, product)
File "/soge-home/users/chri4118/.conda/envs/crp/lib/python3.7/site-packages/elevation/datasource.py", line 122, in ensure_setup
util.ensure_setup(datasource_root, product=product, force=force, **spec)
File "/soge-home/users/chri4118/.conda/envs/crp/lib/python3.7/site-packages/elevation/util.py", line 69, in ensure_setup
with fasteners.InterProcessLock(os.path.join(root, FOLDER_LOCKFILE_NAME)):
File "/soge-home/users/chri4118/.conda/envs/crp/lib/python3.7/site-packages/fasteners/process_lock.py", line 179, in __enter__
gotten = self.acquire()
File "/soge-home/users/chri4118/.conda/envs/crp/lib/python3.7/site-packages/fasteners/process_lock.py", line 161, in acquire
gotten = r(self._try_acquire, blocking, watch)
File "/soge-home/users/chri4118/.conda/envs/crp/lib/python3.7/site-packages/fasteners/_utils.py", line 121, in __call__
return fn(*args, **kwargs)
File "/soge-home/users/chri4118/.conda/envs/crp/lib/python3.7/site-packages/fasteners/process_lock.py", line 112, in _try_acquire
'exception': e,
RuntimeError: Unable to acquire lock on `b'/soge-home/users/chri4118/.cache/elevation/SRTM1/.folder_lock'` due to [Errno 38] Function not implemented
This command work perfectly:
eio clip -o TroutPass-30m-DEM.tif --bounds -78.92 38.95 -78.87 39.00
But the equivalent in Python (Anaconda, Python 3.7.7) returns no errors, but doesn't produce the saved file.
elevation.clip(bounds=(-78.92, 38.95, -78.87, 39.00), output='ExtractHeightField/Python-TroutPass-DEM.tif')
Since fixing #5 (implementing locking), running two eio instances at the same time doesn't corrupt the cache anymore. But as eio locks the datasource cache folder for everything (downloading tiles and clipping a DEM), it's not possible to concurrently run two eio instances, even if the tiles are already there and no cache writing would happen.
Is it possible to suppress the output? The library prints a lot of useless messages instead of some kind of the results indication:
make: Nothing to be done for `download'.
make: Nothing to be done for `all'.
cp SRTM3.vrt SRTM3.59bf58268fdd40ac9758f146322622f7.vrt
gdal_translate -q -co TILED=YES -co COMPRESS=DEFLATE -co ZLEVEL=9 -co PREDICTOR=2 -projwin 43.45734856112712 36.364031827389006 44.72374194164439 34.521751840885294 SRTM3.59bf58268fdd40ac9758f146322622f7.vrt /Users/mbg/Work/pygmtsar2/raw/DEM_EGM96.tif
rm -f SRTM3.59bf58268fdd40ac9758f146322622f7.vrt
find cache -size 0 -name "*.tif" -delete
rm -f SRTM1.*.vrt
rm -f -r spool/*
Hi,
I suspect that the SRTM3 download url has changed.
For instance, it calls:
curl -s -o spool/srtm_22_18.zip.temp http://srtm.csi.cgiar.org/wp-content/uploads/files/srtm_5x5/tiff/srtm_22_18.zip && mv spool/srtm_22_18.zip.temp spool/srtm_22_18.zip
The current URL is now with uppercase TIFF:
https://srtm.csi.cgiar.org/wp-content/uploads/files/srtm_5x5/TIFF/srtm_22_18.zip
Thanks
Getting the following when trying to run
The system cannot find the path specified.
make: Entering directory C:/Users/Micha/AppData/Local/bopen/elevation/Cache/SRTM3' make: Nothing to be done for
download'.
make: Leaving directory C:/Users/Micha/AppData/Local/bopen/elevation/Cache/SRTM3' The system cannot find the path specified. make: Entering directory
C:/Users/Micha/AppData/Local/bopen/elevation/Cache/SRTM3'
gdalbuildvrt -q -overwrite SRTM3
.vrt
ERROR 4: ' does not exist in the file system, and is not recognized as a supported dataset name. Warning 1: Can't open . Skipping it make: *** [SRTM3 .vrt] Error 1 make: Leaving directory
C:/Users/Micha/AppData/Local/bopen/elevation/Cache/SRTM3'
I've installed all the dependencies and the selfcheck says everything checks out.
When i enter : eio clip -o ROME-3Om-DEM.tif --bounds 12.35 41.8 12.65 42 i am getting this:
Makefile:17: recipe for target 'spool/N41/N41E012.hgt.gz' failed
make: *** [spool/N41/N41E012.hgt.gz] Error 60
Could somebody help me to fix this? I don't have any idea...
Thanks in advance
Hi, I'm using rPython to run the script from within R (on Ubuntu 14.04). That seems to work fine, but for some reason, the elevation.clean() procedure throws an error (breaking my iteration in R).
I get:
make: Entering directory `/home/user/.cache/elevation/SRTM1'
find cache -size 0 -name "*.tif" -delete
rm -f spool/*
rm: cannot remove ‘spool/N41’: Is a directory
make: *** [clean] Error 1
make: Leaving directory `/home/user/.cache/elevation/SRTM1'
I do not know whether this problem is related to the script or to my system setup, though..
Any thoughts?
Testing the python package on my OSX and I am getting the following error:
make: Nothing to be done for
download'.`
and
FAILURE: No input filenames specified.
make: *** [SRTM1.vrt] Error 1
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/larosa/dev/ext-libs/elevation/datasource.py", line 154, in clip
datasource_root = seed(bounds=bounds, **kwargs)
File "/Users/larosa/dev/ext-libs/elevation/datasource.py", line 137, in seed
util.check_call_make(datasource_root, targets=['all'])
File "/Users/larosa/dev/ext-libs/elevation/util.py", line 49, in wrapper
return wrapped(path, *args, **kwargs)
File "/Users/larosa/dev/ext-libs/elevation/util.py", line 80, in check_call_make
subprocess.check_call(cmd, shell=True)
File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 541, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'make -C /Users/larosa/Library/Caches/elevation/SRTM1 all ' returned non-zero exit status 2
Although the package works very well on Linux.
Updating the makefile template doesn't trigger an update of the Makefile on the filesystem during ensure_setup.
Maybe add and check a template version number/date.
I found that eio clip causes error when target AOI is located in too many srtm grid. Do you happen to know how we can solve this?
`root@yuend:~/eio_test# eio clip -o srtm.tif --bounds 12.79 40.53 15.42 42.47
Traceback (most recent call last):
File "/usr/local/bin/eio", line 8, in
sys.exit(eio())
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 722, in call
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/elevation/cli.py", line 59, in wrapper
return wrapped(**kwargs)
File "/usr/local/lib/python3.6/dist-packages/elevation/cli.py", line 93, in clip
elevation.clip(bounds, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/elevation/datasource.py", line 181, in clip
datasource_root = seed(bounds=bounds, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/elevation/datasource.py", line 150, in seed
"for how to bulk download tiles." % len(ensure_tiles_names))
RuntimeError: Too many tiles: 12. Please consult the providers' websites for how to bulk download tiles.`
I would like to get the elevation for a single point with the latitude and longitude.
I use darktable[1] to geotag photos. Darktable ha a nice map function to do this with drag and drop [2], bud sadly this function doesn't insert the elevation to the metadate. Now I would like to write a script that adds the elevation to all photos with latitude and longitude.
[1] https://www.darktable.org/
[2] https://www.darktable.org/usermanual/ch05.html.php
I was having trouble using the service to download elevation data I was interested in as the bounds I specified included more than 9 tiles. I see in the code that the value is assigned the value 9. I manually updated this to more than the number I required and the download preoceded without any issue.
datasource.py
def seed(cache_dir=CACHE_DIR, product=DEFAULT_PRODUCT, bounds=None, max_download_tiles=9, **kwargs)
Is there a process that already exists for configuring this value without editing the code? Perhaps a configuration file?
Hi @alexamici , I am trying to get the library to work in windows x64/x32. But I find that either the GNU make library is not working properly, or there is an internal error somewhere.
I managed to install the GNU library from conda -forge
make=3.82=0
These were the commands I am trying:
bounds = (float(lon_min) - .05, float(lat_min) - .05, float(lon_max) + .05, float(lat_max) + .05) elevation.clip(bounds=bounds, output=terrain_tiff_out_path, product='SRTM1') elevation.clean()
This is the error message i get:
I appreciate your advise, thanks!
I've done the bulk download outside your tool (as advertised) and manually generated a new SRTM1.vrt file with gdalbuildvrt:
<VRTDataset rasterXSize="1292301" rasterYSize="464402">
<SRS>GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0],UNIT["degree",0.0174532925199433],AUTHORITY["EPSG","4326"]]</SRS>
<GeoTransform> -1.7900041666666669e+02, 2.7777777777789753e-04, 0.0000000000000000e+00, 7.2000416666666666e+01, 0.0000000000000000e+00, -2.7777777777789753e-04</GeoTransform>
<VRTRasterBand dataType="Int16" band="1">
<NoDataValue>-32767</NoDataValue>
<ColorInterp>Gray</ColorInterp>
<ComplexSource>
<SourceFilename relativeToVRT="1">cache/N41/N41E000.tif</SourceFilename>
<SourceBand>1</SourceBand>
<SourceProperties RasterXSize="3602" RasterYSize="3602" DataType="Int16" BlockXSize="3602" BlockYSize="1" />
<SrcRect xOff="0" yOff="0" xSize="3602" ySize="3602" />
<DstRect xOff="644400" yOff="108000" xSize="3602" ySize="3602" />
<NODATA>-32767</NODATA>
</ComplexSource>
..
But when I e.g. execute eio clip -o Rome-30m-DEM.tif --bounds 12.35 41.8 12.65 42
elevation starts to download N41E012.hgt.gz despite ./cache/N41/N41E012.tif is definitively there and contains the right data (checked with QGIS).
What's my error?
Hello,
I have a question regarding the coverage of srtm 30m data that are downloaded from S3. From your README, it mentions that the 30m data comes from the global datasets SRTM 30m Global 1 arc second V003 elaborated by NASA and NGA. When looking at the link, it says that the spatial extent is "Global (60°N to 56°S, 180°W to 180°E)".
However, I downloaded a tile beyond the extent mentioned (skadi/N65/N65E015.hgt.gz) and I do have elevation data in that area that is supposed to be outside the coverage:
When looking at Mapzen websites, it says that they provide worldwide basemap coverage sourced from SRTM and other open data projects with several different data formats and varying levels of processing.
So my guess is that the elevation data downloaded from s3 bucket (skadi) are a mix of different sources and not only the original srtm data.
Could someone clarify this?
Thanks a lot,
Greg
In python 3:
import elevation
elevation.clip(bounds=(12.35, 41.8, 12.65, 42), output='Rome-DEM.tif')
Does not write the file in the working directory, but in /home/user/.cache/elevation/SRTM1/Rome-DEM.tif
Current fix is to use an explicit output path. E.g. output='/home/user/Rome-DEM.tif'
Suggested by Paolo Cavallini @pcav on qgis-user:
https://lists.osgeo.org/pipermail/qgis-user/2016-March/035871.html
Check what happens first!
Just curious why this is using old SRTM data and not newer ASTER GDEM Version 2 or NED Terrain/Elevation Data?
The easy way to download either data is via Global Mapper they provide access to their own Global Mapper Grid Tiles for the World and you can simply export the area you want or export tiles or any of the 30+ formats that it supports. It can be scripted out with a GMS with their scripting language.
According to the documentation, "Every command has a corresponding API function in the elevation module". However, the option to use a 'reference' image rather than the explicit 'bounds' only works in the CLI and not in the Python API.
This is not a serious problem because one can easily find the bounds using rasterio, which must be installed anyway. But this information needs to be explained in the documentation.
Proposed by Nicolas Cadieux here: https://lists.osgeo.org/pipermail/qgis-user/2016-March/035876.html
NASADEM ami to be a "state-of-the-art global digital elevation model (DEM) derived from a combination of Shuttle Radar Topography Mission (SRTM) processing improvements, elevation control, void-filling and merging with data unavailable at the time of the original SRTM production"
https://earthdata.nasa.gov/community/community-data-system-programs/measures-projects/nasadem
See also: http://www2.jpl.nasa.gov/srtm/
Add locking maybe?
Dependecies:
Hi,
I think this is not a software but a user problem. I tried to install elevation
pip install elevation
sudo add-apt-repository ppa:ubuntugis/ppa
sudo apt-get update
sudo apt-get install gdal-bin
But if I nor run
$ eio selfcheck
I just get
eio: Command not found
What is missing?
I need to acquire SRTM 3 DEM files for the following bounds: [25.50965297 58.40783034 27.67946847 60.06743426]. As the top right corner's latitude is 60, elevation downloads srtm_42_00.tif file, and as there is no such file on the datasource, an empty file is being created.
The problem lies in srtm3_tile_ilonlat(lon, lat) function in datasource.py:53
return (ilon + 180) // 5 + 1, (64 - ilat) // 5
One possible would be to check if the file exists, and only then download it.
Hi!
I'm using the iOS version of the GeoTIFF library (https://github.com/ngageoint/tiff-ios) to read and parse the resulting elevation models. Unfortunately, the iOS version of their library does not support tiff compression. Is there a way to generate DEMs w/o enabling tiff compression?
Bobby
I have cropped a tif file using latest elevation lib, e.g.:
eio clip -o dem.tif --bounds 35.079732 30.901217 35.845468 31.870474
Then inspected the CRS of the file using rasterio
, which returned "EPSG:4326", plus some point inside the area (lat,lng), which returned 69:
import rasterio
with rasterio.open('my.tif') as dataset:
print(f'CRS: {dataset.crs}') # output EPSG:4326
row, col = dataset.index(lng, lat)
elevation = dataset.read(1, window=((row, row+1), (col, col+1)))
print(elevation[0, 0]) # output 69
I'm new to this field, but looking up EPSG:4326 says it is "WGS 84, latitude/longitude coordinate system based on the Earth's center of mass, used by the Global Positioning System among others."
Does this mean the generated tif file contains heights above the WGS84 ellipsoid? I'm confused because we have an independent and highly-accurate measuring system that explicitly reports 69m above MSL and NOT above the ellipsoid. And that system also tells us they are ~20m apart at that point, so they can't both be right..
What am I missing here?
Data providers have download policies and often require bulk download to be authenticated or limited. We don't intend to make too easy for users to make bulk downloads, instead we want to make the need for bulk download be minimised to case when it is really needed (off-the-internet machines, real time applications, etc).
Right now we refuse to clip or seed more that 9 tiles (even if they are already cached, which is dumb!).
Let's define and implement a better strategy.
Trying to run the example shown on the readme. Getting an error that the module has no attribute clip. Command line utility works as expected.
It seems, srtm.csi.cgiar.org is no longer serving wp-content uploads over http, at least for me.
Changing the URI to https://srtm.csi.cgiar.org fixed the issue for me.
https://github.com/bopen/elevation/blob/master/elevation/datasource.py#L122
'datasource_url': 'https://srtm.csi.cgiar.org/wp-content/uploads/files/srtm_5x5/tiff',
The SRTM3 (90m resolution) url from CGIAR-CSI has changed recently. with the following patch on datasource.py, fetching tiles seems to work again:
diff datasource.py.org datasource.py
87c87,90
< 'datasource_url': 'http://srtm.csi.cgiar.org/SRT-ZIP/SRTM_V41/SRTM_Data_GeoTiff',
---
> # 'datasource_url': 'http://srtm.csi.cgiar.org/SRT-ZIP/SRTM_V41/SRTM_Data_GeoTiff',
> # new since Nov 2018:
> 'datasource_url': 'http://srtm.csi.cgiar.org/wp-content/uploads/files/srtm_5x5/tiff',
This is a common problem and it is particularly painful because it causes CI spurious crashes.
Distclean fails both under CLI and Python API
Ubuntu 22.10
Python API
>>> import elevation
>>> elevation.clip(bounds=(12.35, 41.8, 12.65, 42), output='/home/user/Rome-DEM.tif')
make: Entering directory '/home/user/.cache/elevation/SRTM1'
make: Nothing to be done for 'download'.
make: Leaving directory '/home/user/.cache/levation/SRTM1'
make: Entering directory '/home/user/.cache/elevation/SRTM1'
make: Nothing to be done for 'all'.
make: Leaving directory '/home/user/.cache/elevation/SRTM1'
make: Entering directory '/home/user/.cache/elevation/SRTM1'
cp SRTM1.vrt SRTM1.1013a80cf57a452aaec8ea3c149be78f.vrt
make: Leaving directory '/home/user/.cache/elevation/SRTM1'
make: Entering directory '/home/user/.cache/elevation/SRTM1'
gdal_translate -q -co TILED=YES -co COMPRESS=DEFLATE -co ZLEVEL=9 -co PREDICTOR=2 -projwin 12.35 42.0 12.65 41.8 SRTM1.1013a80cf57a452aaec8ea3c149be78f.vrt /home/user/Rome-DEM.tif
rm -f SRTM1.1013a80cf57a452aaec8ea3c149be78f.vrt
make: Leaving directory '/home/user/.cache/elevation/SRTM1'
>>> elevation.distclean()
make: Entering directory '/home/user/.cache/elevation/SRTM1'
find cache -size 0 -name "*.tif" -delete
rm -f SRTM1.*.vrt
rm -f -r spool/*
rm -f cache/* SRTM1.vrt Makefile
rm: cannot remove 'cache/N41': Is a directory
make: *** [Makefile:50: distclean] Error 1
make: Leaving directory '/home/user/.cache/elevation/SRTM1'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/user/.local/lib/python3.10/site-packages/elevation/datasource.py", line 247, in distclean
util.check_call_make(datasource_root, targets=['distclean'])
File "/home/user/.local/lib/python3.10/site-packages/elevation/util.py", line 88, in check_call_make
subprocess.check_call(cmd, shell=True)
File "/usr/lib/python3.10/subprocess.py", line 369, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'make -C /home/user/.cache/elevation/SRTM1 distclean ' returned non-zero exit status 2.
>>>
CLI
user@server:~> eio clip -o Rome-30m-DEM.tif --bounds 12.35 41.8 12.65 42
make: Entering directory '/home/user/.cache/elevation/SRTM1'
make: Nothing to be done for 'download'.
make: Leaving directory '/home/user/.cache/elevation/SRTM1'
make: Entering directory '/home/user/.cache/elevation/SRTM1'
gdalbuildvrt -q -overwrite SRTM1.vrt cache/N41/N41E012.tif
make: Leaving directory '/home/user/.cache/elevation/SRTM1'
make: Entering directory '/home/user/.cache/elevation/SRTM1'
cp SRTM1.vrt SRTM1.6183fda52aa745f094cc6d239ee81355.vrt
make: Leaving directory '/home/user/.cache/elevation/SRTM1'
make: Entering directory '/home/user/.cache/elevation/SRTM1'
gdal_translate -q -co TILED=YES -co COMPRESS=DEFLATE -co ZLEVEL=9 -co PREDICTOR=2 -projwin 12.35 42.0 12.65 41.8 SRTM1.6183fda52aa745f094cc6d239ee81355.vrt /home/user/Rome-30m-DEM.tif
rm -f SRTM1.6183fda52aa745f094cc6d239ee81355.vrt
make: Leaving directory '/home/user/.cache/elevation/SRTM1'
user@server:~> eio distclean
make: Entering directory '/home/user/.cache/elevation/SRTM1'
find cache -size 0 -name "*.tif" -delete
rm -f SRTM1.*.vrt
rm -f -r spool/*
rm -f cache/* SRTM1.vrt Makefile
rm: cannot remove 'cache/N41': Is a directory
make: *** [Makefile:50: distclean] Error 1
make: Leaving directory '/home/user/.cache/elevation/SRTM1'
Traceback (most recent call last):
File "/home/user/.local/bin/eio", line 8, in <module>
sys.exit(eio())
File "/usr/lib/python3/dist-packages/click/core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "/usr/lib/python3/dist-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/usr/lib/python3/dist-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/lib/python3/dist-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/lib/python3/dist-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/usr/lib/python3/dist-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/user/.local/lib/python3.10/site-packages/elevation/cli.py", line 63, in wrapper
return wrapped(**kwargs)
File "/home/user/.local/lib/python3.10/site-packages/elevation/cli.py", line 119, in distclean
elevation.distclean(**kwargs)
File "/home/user/.local/lib/python3.10/site-packages/elevation/datasource.py", line 247, in distclean
util.check_call_make(datasource_root, targets=['distclean'])
File "/home/user/.local/lib/python3.10/site-packages/elevation/util.py", line 88, in check_call_make
subprocess.check_call(cmd, shell=True)
File "/usr/lib/python3.10/subprocess.py", line 369, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'make -C /home/user/.cache/elevation/SRTM1 distclean ' returned non-zero exit status 2.
user@server:~>
Hi I really like this project and how usable it is.
I have a use case right now where I want to keep the .hgt files from a polygon rather than turn them into the geotiff files. I see in the makefile that it cleans them up the files in spool with a rm
after converting and storing in the cache directory.
Is there already an to stop this removal, or would I need to just edit the makefile?
The elevation package can create problems on Windows machine because the command 'mkdir -p' does not work. It should be replaced by 'mkdir'. Recursivity (flag p) is natively included in 'mkdir' on Windows.
One quick and dirty solution would be to modify 'datasource.mk' and check if the OS is Windows, in which case the code should use 'mkdir' , else 'mkdir -p'.
Could the developers please comment whether there is any argument speaking against such a change?
Does this support img files?
thanks
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.