sentinel-hub / sentinelhub-py

Download and process satellite imagery in Python using Sentinel Hub services.

Introduction

The sentinelhub Python package is the official Python interface for Sentinel Hub services. It supports most of the services described in the Sentinel Hub documentation and any type of satellite data collections, including Sentinel, Landsat, MODIS, DEM, and custom collections produced by users.

The package also provides a collection of basic tools and utilities for working with geospatial and satellite data. It builds on top of well known packages such as numpy, shapely, pyproj, etc. It is also a core dependency of eo-learn Python package for creating geospatial data-processing workflows.

The main package resources are GitHub repository, documentation page, and Sentinel Hub forum.

Installation

The package requires a Python version >= 3.8. The package is available at the PyPI package index and can be installed with

$ pip install sentinelhub

or with an extension tag for additional functionalities

$ pip install sentinelhub[AWS]  # extra dependencies for interacting with Amazon Web Services

Alternatively, the package can be installed with Conda from conda-forge channel

$ conda install -c conda-forge sentinelhub

To install the package manually, clone the repository and run

$ pip install .

Before installing sentinelhub on Windows it is recommended to install shapely package from Unofficial Windows wheels repository

Once installed the package can be configured according to configuration instructions in documentation.

Content

A high-level overview of the main functionalities:

• Sentinel Hub services

  • Process API,
  • Catalog API,
  • Batch Processing API,
  • BYOC API,
  • Statistical API,
  • OGC services (WMS/WCS/WFS),
  • FIS,
  • authentication and rate-limit handling,

• geospatial utilities

  • interface for geospatial objects and transformations,
  • large area splitting,
  • data collection objects,
  • IO tools,

• Geopedia WMS and REST API.

Documentation

For more information on the package and to access the documentation, visit readthedocs.

Examples

The package has a collection of Jupyter notebooks with examples. They are available in the examples folder on GitHub and converted into documentation under Examples section.

Additionally, some examples are explained in Sentinel Hub webinar videos:

• Process API in Python
• OGC API in Python

Blog posts

The package played a key role in many projects and use cases described at Sentinel Hub blog. The following blog posts are about the package itself:

• Upgrading the sentinelhub Python package
• Release of sentinelhub Python Package 2.0

Questions and Issues

Feel free to ask questions about the package and its use cases at Sentinel Hub forum or raise an issue on GitHub.

You are welcome to send your feedback to the package authors, Sentinel Hub research team, through any of Sentinel Hub communication channels.

License

See LICENSE.

sentinelhub-py's Issues

Decoding downloaded Jpeg2000 images without saving them first

So far we don't know how to correctly decode Jpeg2000 images the package downloads without saving them first. Therefore for example AwsTileRequest(...).get_data(save_data=False) raises an error.

The problem is that original Sentinel-2 L1C images provided by ESA and stored on AWS have a nonstandard 15-bit Jpeg2000 encryption. So most of the Python packages (e.g. OpenCV, Pillow) incorrectly read these images as 16-bit data and return values that are 2 times larger as they should be. The only package we know that reads the images correctly is Glymur. However it requires a filename of stored image and does not support reading from BytesIO, StringIO, numpy.array or any other structure. Therefore data always needs to be first saved locally.

Does anyone know a solution for this?
(Maybe some other package we don't know about, a clever "hack" in mentioned packages or even simulating a filename path without actually saving image on disk...)

MFA with sentinelhub.aws

Using sentinelhub.aws, I cannot download Sentinel imagery from S3 when using an AWS account that has a MFA (multi-factor authentication) requirement. I have verified that I can do so on an account that does not require MFA.

Failed to load resource: the server responded with a status of 503 (Service Unavailable)

Hi,

I have been trying to fetch data using WMS, however, I get 503 status every time. Can someone help?

EDIT 1: The image returned in a blank white image. Also, server still returns a 503 status

B04 Missing for specific tile.

While using the sentinel-2 bucket on S3 I noticed a missing band 4 for the following tile. The preview jpg looks fine, so maybe this slipped somehow? Just wanted to report in case its relevant.

aws s3 ls s3://sentinel-s2-l1c/tiles/11/S/NA/2016/11/22/0/
                           PRE auxiliary/
                           PRE preview/
                           PRE qi/
2016-11-24 14:34:20    1861642 B01.jp2
2016-11-24 14:34:21   53540367 B02.jp2
2016-11-24 14:34:21   55470612 B03.jp2
2016-11-24 14:34:20   16411474 B05.jp2
2016-11-24 14:34:20   16569386 B06.jp2
2016-11-24 14:34:20   16722422 B07.jp2
2016-11-24 14:34:21   59417098 B08.jp2
2016-11-24 14:34:20    1973499 B09.jp2
2016-11-24 14:34:20     964000 B10.jp2
2016-11-24 14:34:20   15903052 B11.jp2
2016-11-24 14:34:20   15617508 B12.jp2
2016-11-24 14:34:20   16781626 B8A.jp2
2016-11-24 14:34:20     285999 metadata.xml
2016-11-24 14:34:20     165856 preview.jp2
2017-07-10 11:34:23      74534 preview.jpg
2016-11-24 14:34:24       1111 productInfo.json
2016-11-24 14:34:20       1735 tileInfo.json

Sentinel-1 Image Download

Empty output images using WcsRequest to download SCL from L2A

Hi,

I also posted that question in the Sentinelhub Forum, but maybe the answer also helps people who do not know the forum yet.

I want to download an index (e.g. NDVI) and I also need to download the cloudmasks from L2A which is called SCL.

I considered the problem, that L1C seems to have more images and thus outputs than L2A and adapted the WMS configurator so that the NDVI and the SCL layer use L2A as input.

Using a time range of 10 days (2017-08-01 until 2017-08-10), I get 2 results for the NDVI layer.
However, for the SCL layer I get 5 results, only 2 with content (same time than the 2 NDVI results) and 3 empty images.

Hence, I think that your service did not recognize the invalidity of the empty output files.

Here is the code:

from sentinelhub.data_request import WcsRequest
from sentinelhub.constants import MimeType
from sentinelhub.common import BBox, CRS
import numpy as np

# FUNCTIONS ---------------------------------------------------------------------
def WCS_Download(bbox_coord):

    BBOX = BBox(bbox=bbox_coord, crs=CRS.WGS84)

    for Layer in Layers:

        wcs = WcsRequest(data_folder=outdir,
                        layer=Layer,
                        bbox=BBOX,
                        time=(start, end),
                        resx='{}m'.format(res), resy='{}m'.format(res),
                        image_format=MimeType.TIFF_d32f,
                        instance_id=INSTANCE_ID)

        wcs_img = wcs.get_data(save_data=True)

# CONFIG ------------------------------------------------------------------------

outdir = "PATH_TO_MY_OUTPUT_FOLDER"
Layers = ['NDVI', 'CLOUD']
INSTANCE_ID = '<instance_id>'
BBOX_coord = [16.53,47.83,16.84,48.31]
res = 20
start = '2017-08-01'
end = '2017-08-10'

# MAIN -------------------------------------------------------------------------
WCS_Download(BBOX_coord)

Single-digit UTM zone issue

When the UTM zone has a single digit, downloading in SAFE format fails.

import sentinelhub
sentinelhub.download_safe_format(tile=('4WEE', '2017-06-15'))

fails with:

[...]
DownloadError: Invalid url request http://sentinel-s2-l1c.s3.amazonaws.com/tiles/4W/E/E/2017/6/15/1/tileInfo.json

License?

Do you plan on adding a license to this project?

Currently it is without license and as such I am not sure where/how I can use this tool. Effectively I can't use or include it at all without asking your specific permission for each case.

Access denied on Sentinel-2 l1c

Whenever I run

# sentinelhub.aws --tile T33XWJ 2018-04-02

I get an access denied for sentinel-s2-l1c tiles/33/X/WJ/2018/4/2/4/qi/GEOMETRIC_QUALITY.xml:

I configured as described here:

https://sentinelhub-py.readthedocs.io/en/latest/configure.html#amazon-s3-capabilities

But did not add any permission policies in the AWS IAM console (should I?)

Traceback (most recent call last):
  File "/home/herbert/.virtualenvs/eye/bin/sentinelhub.aws", line 10, in <module>
    sys.exit(aws())
  File "/home/herbert/.virtualenvs/eye/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/herbert/.virtualenvs/eye/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/herbert/.virtualenvs/eye/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/herbert/.virtualenvs/eye/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/herbert/.virtualenvs/eye/lib/python3.6/site-packages/sentinelhub/commands.py", line 65, in aws
    bands=band_list, data_source=data_source)
  File "/home/herbert/.virtualenvs/eye/lib/python3.6/site-packages/sentinelhub/data_request.py", line 932, in download_safe_format
    safe_request.save_data(redownload=redownload)
  File "/home/herbert/.virtualenvs/eye/lib/python3.6/site-packages/sentinelhub/data_request.py", line 130, in save_data
    self._execute_data_download(data_filter, redownload, max_threads, raise_download_errors)
  File "/home/herbert/.virtualenvs/eye/lib/python3.6/site-packages/sentinelhub/data_request.py", line 165, in _execute_data_download
    data_list.append(future.result(timeout=SHConfig().download_timeout_seconds))
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 425, in result
    return self.__get_result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/herbert/.virtualenvs/eye/lib/python3.6/site-packages/sentinelhub/download.py", line 239, in execute_download_request
    response = _do_aws_request(request)
  File "/home/herbert/.virtualenvs/eye/lib/python3.6/site-packages/sentinelhub/download.py", line 313, in _do_aws_request
    return s3_client.get_object(Bucket=bucket_name, Key=url_key, RequestPayer='requester')
  File "/home/herbert/.virtualenvs/eye/lib/python3.6/site-packages/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/home/herbert/.virtualenvs/eye/lib/python3.6/site-packages/botocore/client.py", line 661, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the GetObject operation: Access Denied

Error on request when selecting SENTINEL2_L2A for a layer with SENTINEL2_L1C

Hi, both in the older version 1.1.0 and the new 2.0.0 I get the following error when I set the data_source argument to sentinelhub.constants.DataSource.SENTINEL2_L2A when the layer configured at sentinelhub is set to "Sentinel S2 - L1C"?

The error is as follow:

DownloadFailedException: Failed to download with HTTPError:
400 Client Error: Bad Request for url: https://services.sentinel-hub.com/ogc/wfs/<instance_id>?SERVICE=wfs&REQUEST=GetFeature&TYPENAMES=DSS2&BBOX=9.580444937668496%2C55.72393911802203%2C9.827918503903371%2C56.20413135969968&OUTPUTFORMAT=application%2Fjson&SRSNAME=EPSG%3A4326&TIME=2017-08-07T00%3A00%3A00%2F2017-08-12T23%3A59%3A59&MAXCC=100.0&MAXFEATURES=100&FEATURE_OFFSET=0
Server response: "TYPENAME=DSS2 not found!"

Sentinel-1 Image Download

Could you please give me an example of downloading Sentinel-1 images? I follow the example of Sentinel-2 but get_tiles() method doesnot work with Sentinel-1.

AttributeError: module 'requests' has no attribute 'ConnectTimeout'

Hi,

many thanks for providing this very useful tool.

When working through the tutorial (http://sentinelhub-py.readthedocs.io/en/latest/examples/ogc_request.html) I get the error message: "AttributeError: module 'requests' has no attribute 'ConnectTimeout'" when requesting the following:

wms_bands_request = WmsRequest(layer='ALL_BANDS',
                               bbox=betsiboka_bbox,
                               time='latest',
                               width=512, height=856,
                               image_format=MimeType.TIFF_d32f,
                               instance_id=INSTANCE_ID)

wms_bands_img = wms_bands_request.get_data()

What might be the reason for it? I have a trial account ID.

Many thanks
s.

AWSProductRequest Folder Structure

Would it be possible to add the HTML, rep_info, and AUX_DATA folders to the downloaded product by default? They will be empty folders, but appear to be necessary for Sen2Cor 2.4.0 to start. If any of the three folders are missing the program will not run.

Bug in aws.py File

Though the product is from L1C archive and generated from S2B instrument of Sentinel-2 Satellite.

I have faced an issue when trying to parse Product-Identifier of the Tile i.e, (S2B_OPER_MSI_L1C_TL_SGS__20180812T090522_A007481_T43QHV_N02.06) through a parser method i.e, AwsTile.tile_id_to_tile() and below is the error.

AwsTile.tile_id_to_tile(tile_id)

ValueError Traceback (most recent call last)
in ()
----> 1 AwsTile.tile_id_to_tile(tile_id)

~/.conda/envs/web-static/lib/python3.6/site-packages/sentinelhub/aws.py in tile_id_to_tile(tile_id)
680 """
681 if tile_id.split('_')[0] not in ['S2A', 'L1C','S2B']:
--> 682 raise ValueError('Transformation from tile_id to tile works currently only for Sentinel-2 L1C products')
683
684 tile_info = get_tile_info_id(tile_id)

ValueError: Transformation from tile_id to tile works currently only for Sentinel-2 L1C products

I have added 'S2B' in line 681 as on of the element in the list and works properly..

Problem with Downloading as .SAFE with Product ID Older than Jan 2016

I updated to the ver 2.0, trying to download an older product with multiple granules per product into .SAFE format, I get a FILE NOT FOUND ERROR for the first band .jp2 on the first granule. The .SAFE structure is created, and all the granule folders show up, but there is no actual data or metadata found. Regular download not using .SAFE format parameter works fine.

I was using a relative path name for the download_folder parameter in AwsProductRequest function, so I tried an absolute path. This did not work either. The product id was S2A_OPER_PRD_MSIL1C_PDMC_20160407T043302_R107_V20160405T085012_20160405T085012

Here's the full traceback:

Traceback (most recent call last):
File "D:\Python36\lib\multiprocessing\pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "D:\Winter2018\Sentinel2-DDPython\sentinel2dd.py", line 535, in download_task
job_status['download_status'] = download_product(product['title'], download_folder_path, product['platformname'])
File "D:\Winter2018\Sentinel2-DDPython\sentinel2dd.py", line 358, in download_product
download_result = product_request.save_data()
File "D:\Winter2018\Sentinel2-DDPython\venv\lib\site-packages\sentinelhub\data_request.py", line 100, in save_data
self._execute_data_download(redownload, max_threads, raise_all_errors=False)
File "D:\Winter2018\Sentinel2-DDPython\venv\lib\site-packages\sentinelhub\data_request.py", line 119, in _execute_data_download
data_list.append(future.result(timeout=SHConfig().download_timeout_seconds))
File "D:\Python36\lib\concurrent\futures_base.py", line 425, in result
return self.__get_result()
File "D:\Python36\lib\concurrent\futures_base.py", line 384, in __get_result
raise self._exception
File "D:\Python36\lib\concurrent\futures\thread.py", line 56, in run
result = self.fn(*self.args, **self.kwargs)
File "D:\Winter2018\Sentinel2-DDPython\venv\lib\site-packages\sentinelhub\download.py", line 236, in execute_download_request
_save_if_needed(request, response_content)
File "D:\Winter2018\Sentinel2-DDPython\venv\lib\site-packages\sentinelhub\download.py", line 333, in _save_if_needed
with open(request.file_location, 'wb') as file:
FileNotFoundError: [Errno 2] No such file or directory: 'D:\Winter2018\Sentinel2-DDPython\temp\S2A_OPER_PRD_MSIL1C_PDMC_20160407T043302_R107_V20160405T085012_201604
05T085012.SAFE\GRANULE\S2A_OPER_MSI_L1C_TL_SGS__20160405T124601_A004108_T35TQK_N02.01\IMG_DATA\S2A_OPER_MSI_L1C_TL_SGS__20160405T124601_A004108_T35TQK_B01.jp2'

Not sure what I am doing wrong here. Thanks!

Incorrect format string in log message causes TypeError

A log message in opensearch.get_tile_info_id uses %d to format a tile ID, which raises a TypeError because tile IDs are strings.

Workaround:

logging.getLogger('sentinelhub.opensearch').setLevel(logging.CRITICAL)

Incorrect folder structure with AwsProductRequest and data_folder

This is related to #12, but since it is a bug (or user error...) in the newly introduced functionality, I felt a new issue would be good.

Problem

I can't seem to get the new additional folders, AUX_DATA, HTML, and rep_info, to show up in the right place. On the face of it, it looks like the data_folder was left out somewhere. Here's a snippet that reproduces the issue:

from sentinelhub.data_request import AwsProductRequest

product_id = 'S2A_MSIL1C_20180110T000231_N0206_R030_T55GCM_20180110T030215'
data_folder = 'work'

product_request = AwsProductRequest(
    product_id=product_id,
    data_folder=data_folder,
    safe_format=True
)
product_request.save_data()

And here's the initial folder structure for completeness (the snippet is in product_test.py):

$ tree
.
└── product_test.py

The resulting folder structure looks like this:

$ tree -L 3
.
├── S2A_MSIL1C_20180110T000231_N0206_R030_T55GCM_20180110T030215.SAFE
│   ├── AUX_DATA
│   ├── DATASTRIP
│   │   └── DS_SGS__20180110T030215_S20180110T000815
│   ├── HTML
│   └── rep_info
├── product_test.py
└── work
    └── S2A_MSIL1C_20180110T000231_N0206_R030_T55GCM_20180110T030215.SAFE
        ├── DATASTRIP
        ├── GRANULE
        ├── INSPIRE.xml
        ├── MTD_MSIL1C.xml
        └── manifest.safe

I eventually found out that there are references to data_folder in places where it wouldn't actually have been set yet. This snippet is from AwsProductRequest:

    def create_request(self):
        if self.safe_format:
            self.aws_service = SafeProduct(self.product_id, tile_list=self.tile_list, bands=self.bands,
                                           metafiles=self.metafiles)
        else:
            self.aws_service = AwsProduct(self.product_id, tile_list=self.tile_list, bands=self.bands,
                                          metafiles=self.metafiles)

        self.download_list = self.aws_service.get_requests()

Note that self.data_folder is never passed to the SafeProduct constructor (and thus not the AwsProduct constructor either). This means that when the call to get_requests happens, the variable will be the default of '', which propagates down to the structure_recursion call in SafeProduct. As a result, when creating the folder structure, the data folder is ignored and everything is created in the parent folder. The odd thing is that the files are actually downloaded to the right place, even though the folder structure creation fails.

Solution (aka 'The Hacky Workaround')

As a temporary fix, I changed the create_request method in AwsProductRequest so that it passes the data folder as well, and I changed the structure_recursion method in AwsService so that it joins the data folder with the folder name that is being created:

    def create_request(self):
        if self.safe_format:
            self.aws_service = SafeProduct(self.product_id, tile_list=self.tile_list, bands=self.bands,
                                           #metafiles=self.metafiles)
                                           metafiles=self.metafiles, data_folder=self.data_folder)

    def structure_recursion(self, struct, folder):
        if not struct:
            #make_folder(folder)
            make_folder(os.path.join(self.data_folder, folder))
            return

This resulted in a nested work folder for some reason, but at least nothing is stored in the parent folder:

$ tree -L 3
.
├── product_test.py
└── work
    └── work
        └── S2A_MSIL1C_20180110T000231_N0206_R030_T55GCM_20180110T030215.SAFE

To fix the nesting, I changed the call to structure_recursion in SafeProduct so that it passes the empty string instead of self.data_folder:

class SafeProduct(AwsProduct):
    def get_requests(self):
        #self.structure_recursion(safe, self.data_folder)
        self.structure_recursion(safe, '')

And voila, the correct folder structure:

$ tree -L 3
.
├── product_test.py
└── work
    └── S2A_MSIL1C_20180110T000231_N0206_R030_T55GCM_20180110T030215.SAFE
        ├── AUX_DATA
        ├── DATASTRIP
        ├── GRANULE
        ├── HTML
        ├── INSPIRE.xml
        ├── MTD_MSIL1C.xml
        ├── manifest.safe
        └── rep_info

Conlusion

I haven't actually tested what kind of repercussions my hacks may have introduced, so I am not recommending that these changes be implemented as-is (that's why I didn't make a pull request), but I do hope that they can at least provide a hint as to where to start tearing stuff apart.

Thanks for your time.

Unable to import and run download_safe_format

Hi,

I am unable to import and run the download_safe_format function.

I have installed sentinelhub through pip.

import sentinelhub


def main():

    print('starting to download satellite data')
    res = sentinelhub.download_safe_format(tile=('T38TML', '2015-12-19'), entire_product=True)
    print('Finished downloading satellite data')

if __name__ == '__main__':
    main()

res = sentinelhub.download_safe_format(tile=('T38TML', '2015-12-19'), entire_product=True)

AttributeError: 'module' object has no attribute 'download_safe_format'

It works when i do it in the python console as suggested in your Readme file.

I also tried importing the functions like this, but it still doesn't work

from sentinelhub import download_safe_format, get_safe_format, download_data

ImportError: cannot import name download_safe_format

Thanks

Minor Feature Request : : Download only Metafiles, when bands aren't specified

Hi a small feature request, instead of downloading all the bands by default i.e, when bands are not explicitly passed as an argument. Download only Metafiles.

Example, for below code, download only the metafiles without including all the bands :

`metafiles = ['tileInfo', 'preview']

request = AwsTileRequest(tile=tile_name, time=time, aws_index=aws_index,
metafiles=metafiles, data_folder=data_folder,
data_source=DataSource.SENTINEL2_L1C)

request.save_data()`

Latest available tile

I raised an issues yesterday in regards to finding the most recent tile in sync sentinelhub.
Im not to proud of my coding skills so hence the location of posting this and not commiting.
I have managed to sort a way to find the latest available bands by tile and download using sentinelhub.download_data. (Band 4 and 8)
Here is a copy of the code.

Timestamp on image bands

The path to image files specified in MTD_MS1L1C.xml seems to be based on the datatake_sensing_start rather than the timestamp found in tileInfo.json. If the timestamp is used to name the files, the images can't open correctly in SNAP.

e.g. :
Path in MTD_MS1L1C.xml = GRANULE/L1C_T15RUQ_A009218_20170328T170619/IMG_DATA/T15RUQ_20170328T170301_B01
Downloaded path based on timestamp = GRANULE\L1C_T15RUQ_A009218_20170328T170619\IMG_DATA\T15RUQ_20170328T170619_B01

Tile name search

How does one search for all available tiles with the get_tile_info method, or a similar method? I see one can search for tile name and specific time, e.g.

get_tile_info('T10UEV', '2016-01-09')

But what if I don't know any specific dates?

Syntax error in sentinelhub functions

Hi,
I installed the last version of Sentinelhub. I have python 2.7 and Python3 3.6 versions.
When I try to import sentinelhub with python I get e syntax error message :

from .data_request import WmsRequest, WcsRequest, AwsTileRequest, AwsProductRequest, GeopediaWmsRequest,
File "/home/vpi/.local/lib/python2.7/site-packages/sentinelhub/data_request.py", line 36
def init(self,*, data_folder=None):
^
SyntaxError: invalid syntax

I don't know what is the problem and don't find any similar problems on Internet.

Thank you for your help.

Problems during Installation - failed to build tiffile

Sorry, I am relatively new to installing packages with pip and python. While installing Sentinelhub I could not get through the error related to installation of tiffle (see attachement). Any tips? Thanks.

sentinel chain.

(base) R:\Forest2020\ampp2\pyeo>Python sen2chain.py
Traceback (most recent call last):
File "sen2chain.py", line 28, in
main()
File "sen2chain.py", line 23, in main
save_query_output(products)
File "R:\Forest2020\ampp2\pyeo\Sen2Search.py", line 79, in save_query_output
pickle.dump(products, output_file)
TypeError: write() argument must be str, not bytes

Python 3.7 Support

Looking through the commit history, it looks like support for Python 3.7 was added in 92cbdb8 but then removed in ae69dfd.

I couldn't find any specific reasons for the reversion, so was wondering if/when explicit support for Python 3.7 might be re-added?

Error when `pip install sentinelhub`

Hi, I'm installing your model using pip in a Conda environment, like I did about a month ago where it worked.
However, today it failed with this error.

λ pip install sentinelhub --upgrade
Collecting sentinelhub
  Using cached sentinelhub-1.1.1.tar.gz
Requirement already up-to-date: click in c:\users\user\appdata\local\continuum\anaconda3\envs\data_science\lib\site-packages (from sentinelhub)
Collecting cython (from sentinelhub)
  Using cached Cython-0.28.1-cp36-cp36m-win_amd64.whl
Collecting matplotlib (from sentinelhub)
  Using cached matplotlib-2.2.2-cp36-cp36m-win_amd64.whl
Collecting numpy (from sentinelhub)
  Using cached numpy-1.14.2-cp36-none-win_amd64.whl
Collecting opencv-python (from sentinelhub)
  Using cached opencv_python-3.4.0.12-cp36-cp36m-win_amd64.whl
Collecting pip==9.0.1 (from sentinelhub)
  Using cached pip-9.0.1-py2.py3-none-any.whl
Collecting pyproj (from sentinelhub)
  Using cached pyproj-1.9.5.1.tar.gz
    Complete output from command python setup.py egg_info:
    nad2bin.c
    nad2bin.c(129): warning C4244: '=': conversion from 'double' to 'float', possible loss of data
    nad2bin.c(130): warning C4244: '=': conversion from 'double' to 'float', possible loss of data
    nad2bin.c(137): warning C4244: '=': conversion from 'double' to 'float', possible loss of data
    nad2bin.c(138): warning C4244: '=': conversion from 'double' to 'float', possible loss of data
    nad2bin.c(352): warning C4244: '=': conversion from 'double' to 'float', possible loss of data
    nad2bin.c(353): warning C4244: '=': conversion from 'double' to 'float', possible loss of data
    pj_malloc.c
    Generating code
    Finished generating code
    LINK : fatal error LNK1158: cannot run 'rc.exe'
    using bundled proj4..
    Traceback (most recent call last):
      File "C:\Users\user\AppData\Local\Continuum\anaconda3\envs\data_science\lib\distutils\_msvccompiler.py", line 519, in link
        self.spawn([self.linker] + ld_args)
      File "C:\Users\user\AppData\Local\Continuum\anaconda3\envs\data_science\lib\distutils\_msvccompiler.py", line 542, in spawn
        return super().spawn(cmd)
      File "C:\Users\user\AppData\Local\Continuum\anaconda3\envs\data_science\lib\distutils\ccompiler.py", line 909, in spawn
        spawn(cmd, dry_run=self.dry_run)
      File "C:\Users\user\AppData\Local\Continuum\anaconda3\envs\data_science\lib\distutils\spawn.py", line 38, in spawn
        _spawn_nt(cmd, search_path, dry_run=dry_run)
      File "C:\Users\user\AppData\Local\Continuum\anaconda3\envs\data_science\lib\distutils\spawn.py", line 81, in _spawn_nt
        "command %r failed with exit status %d" % (cmd, rc))
    distutils.errors.DistutilsExecError: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\x86_amd64\\link.exe' failed with exit status 1158

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\user\AppData\Local\Temp\pip-build-opj49wbm\pyproj\setup.py", line 74, in <module>
        cc.link_executable(objects, execname, extra_postargs = [ '/MANIFEST' ] if os.name == 'nt' else  None)
      File "C:\Users\user\AppData\Local\Continuum\anaconda3\envs\data_science\lib\distutils\ccompiler.py", line 734, in link_executable
        debug, extra_preargs, extra_postargs, None, target_lang)
      File "C:\Users\user\AppData\Local\Continuum\anaconda3\envs\data_science\lib\distutils\_msvccompiler.py", line 522, in link
        raise LinkError(msg)
    distutils.errors.LinkError: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\x86_amd64\\link.exe' failed with exit status 1158

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\user\AppData\Local\Temp\pip-build-opj49wbm\pyproj\

I asked Google about the error and found this fix, https://stackoverflow.com/a/44563421/7796217, which lets me install the model without any errors. However, this requires that I'm running the installation in a terminal with administrator permissions, else I get the following error:

Exception:
Traceback (most recent call last):
  File "c:\users\user\appdata\local\continuum\anaconda3\envs\data_science\lib\shutil.py", line 387, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\user\\AppData\\Local\\Temp\\pip-ba9_depc-uninstall\\users\\user\\appdata\\local\\continuum\\anaconda3\\envs\\data_science\\lib\\site-packages\\matplotlib\\backends\\_backend_agg.cp36-win_amd64.pyd
'```

I hope you can find a solution such that the installation only requires to run `pip install sentinelhub --upgrade`.

productInfo.json not found

We are using sentinelhub to download S2 Products to our AWS instance. Since a few hours, we cannot download any data anymore, since the file productInfo.json seems to be missing for all tiles I tried:

File "/usr/local/lib/python3.5/dist-packages/sentinelhub/data_request.py", line 630, in __init__
  super(AwsProductRequest, self).__init__(**kwargs)
File "/usr/local/lib/python3.5/dist-packages/sentinelhub/data_request.py", line 589, in __init__
  super(AwsRequest, self).__init__(**kwargs)
File "/usr/local/lib/python3.5/dist-packages/sentinelhub/data_request.py", line 41, in __init__
  self.create_request()
File "/usr/local/lib/python3.5/dist-packages/sentinelhub/data_request.py", line 635, in create_request
  metafiles=self.metafiles)
File "/usr/local/lib/python3.5/dist-packages/sentinelhub/aws.py", line 310, in __init__
  self.product_info = get_json(self.get_url(AwsConstants.PRODUCT_INFO))
File "/usr/local/lib/python3.5/dist-packages/sentinelhub/download.py", line 466, in get_json
  return execute_download_request(request)
File "/usr/local/lib/python3.5/dist-packages/sentinelhub/download.py", line 256, in execute_download_request
  raise AwsDownloadFailedException('File in location %s is missing' % request.url)
sentinelhub.download.AwsDownloadFailedException: File in location https://roda.sentinel-hub.com/sentinel-s2-l2a/products/2018/8/28/S2B_MSIL2A_20180828T052639_N0206_R105_T43QFD_20180828T092745/productInfo.json is missing

Opening the link in Browser shows me a blank file, but no error message; this happens with all the S2 Products I tried.

I am not sure if this issue is even related to sentinelhub...I saw that there is some maintenance on the ESA infrastructure today, but I don't know if thats related, or if it is an issue with my AWS credentials (although I thought the metadata should be still freely available).

How to search Sentinel-2 L2A imagery

Hi,
I am using the sentinelhub.aws command-line tool to download an atmospherically-corrected Sentinel-2 scene for my tile of interest. After configuring my AWS credentials, I can use the command sentinelhub.aws --tile T33UWR 2018-05-01 -e -f $HOME/downloaded_images --l2a to download the L2A product in .SAFE format. It works perfectly.

Now I'm looking for a method to search which dates have Sentinel-2 L2A data available on AWS for my tile of interest.

Right now I am running a command such as sentinelhub.aws --tile T33UWR {date} -i --l2a for each date in my time-period of interest and checking if the result throws an exception, but I'm sure there must be a better method of doing a search.

Is there a method or command line option for searching available S2 L2A imagery? If there is, it would be great if you can add an L2A search example in the documentation.

Thanks,
Jiri Kadlec
www.gisat.cz

PyInstaller Support

I'm having difficulty with using sentinelhub in a python app frozen with PyInstaller. Right now I am only using AwsProductRequest, so there might be more issues, but the main problem is that constants.py uses a file reference to load the version number from the init file. When frozen to a single .exe, that file reference doesn't work any more.

A possible change would make this work:
in constants.py

from . import __version__

mimetypes.add_type('application/json', '.json')

class PackageProps:
    """ Class for obtaining package properties. Currently it supports obtaining package version."""

    @staticmethod
    def get_version():
        version = __version__

and in __init__.py

"""
This module lists all externally useful classes and functions
"""
__version__ = "2.4.1"

from .data_request import WmsRequest, WcsRequest, AwsTileRequest, AwsProductRequest, GeopediaWmsRequest, \
    get_safe_format, download_safe_format

So you move the __version__ initialization to the top of __init__.py and then import the constant directly using from . import __version__ in the constants.py file.

This would allow PyInstaller to bundle sentinelhub successfully. Hope you can consider this change. Thanks.

ReferenceError: B10 is not defined"

Hi. Why does the following code produce the error, and why can I not use B10 in the data processing script?

bbox = BBox([start_min_latitude,
            start_min_longitude, 
            start_min_latitude + 500, 
            start_min_longitude - 500], 
            crs=CRS.UTM_32N)
all_bands_script = 'return [B01, B02, B03, B04, B05, B06, B07, B08, B8A, B09, B10, B11, B12]'
req = OgcRequest(data_source=DataSource.SENTINEL2_L2A,
                 service_type=ServiceType.WCS,
                 custom_url_params={CustomUrlParam.EVALSCRIPT: all_bands_script},
                 size_x='10m',
                 size_y='10m', 
                 bbox=bbox, 
                 time='2017-12-09',
                 instance_id=INSTANCE_ID,
                 layer=S2_LAYER,
                 image_format=FILE_TYPE)
req.get_data()

the returned error is:

Server response: "Failed to evaluate script! 
ReferenceError: B10 is not defined" 

opensearch : no tiles found

Hi,

I am using opensearch to get tile information but no tiles seem to be found.
Here is my command :
tile='T31TFJ'

infos = opensearch.get_tile_info_id(tile)
Traceback (most recent call last):
File "", line 1, in
File "/home/vpi/.local/lib/python3.6/site-packages/sentinelhub/opensearch.py", line 35, in get_tile_info_id
raise TileMissingException
sentinelhub.opensearch.TileMissingException

I tried with other tile names but have the same problem.

Thank you.

Convert pixel coordinates to lat, lng

Without downloading Sentinel images and reading them with the use of another library, eg Rasterio - I am struggling to find an effective way using sentinelhub-py of converting pixel coordinates to latitude and longitude. Can anyone help?

Error when making a 'OgcRequest' data request, not recognising service type?

Currently trying to make a OgcRequest, however not having any luck and getting an error:
ogc_tc_request = data_request.OgcRequest(layer='TRUE-COLOR-S2-L1C', service_type='WCS', data_folder=user_dir, bbox=farm_bbox, time=(start_date, end_date), size_x='20m', size_y='20m', image_format=MimeType.TIFF_d32f, custom_url_params={CustomUrlParam.SHOWLOGO: False})

The error I get is:
File "E:\xx_DJANGO_DEVELOPMENT\SentinelHub_June2018\lib\get_data_sentinelhub.py", line 117, in main true_color(farm_bbox, start_date, end_date, resolution, user_dir) File "E:\xx_DJANGO_DEVELOPMENT\SentinelHub_June2018\lib\get_data_sentinelhub.py", line 48, in true_color custom_url_params={CustomUrlParam.SHOWLOGO: False}) File "C:\Anaconda2\envs\sentinelhub\lib\site-packages\sentinelhub\data_request.py", line 296, in __init__ super(OgcRequest, self).__init__(**kwargs) File "C:\Anaconda2\envs\sentinelhub\lib\site-packages\sentinelhub\data_request.py", line 41, in __init__ self.create_request() File "C:\Anaconda2\envs\sentinelhub\lib\site-packages\sentinelhub\data_request.py", line 314, in create_request self.download_list = ogc_service.get_request(self) File "C:\Anaconda2\envs\sentinelhub\lib\site-packages\sentinelhub\ogc.py", line 154, in get_request size_x, size_y = self.get_image_dimensions(request) File "C:\Anaconda2\envs\sentinelhub\lib\site-packages\sentinelhub\ogc.py", line 319, in get_image_dimensions raise ValueError("At least one of parameters 'width' and 'height' must have an integer value") ValueError: At least one of parameters 'width' and 'height' must have an integer value

But when I look at the ogc.py file at the 'get_image_dimensions', it seems to be expecting integers related to 'WFS' (or other), and not 'WCS' as requested...

Am I missing something in my request?

AWS request payer

After 6 August AWS has implemented the requester pays to l1c bucket. Is there a way to still access sentinel l1c data by just adding the header of RequestPayer='requester' to a normal tile request as was done when requesting L2A data. If there is, in what python class would it be added and where. Sorry i am still new to using sentinel hub and python. At the moment access denied is shown when quering l1c but not l2a as i am when config option is set to true request contructer is called that doesn't include requester paid

Possibility to provide own bbox list

I would like the method implemented in sentinelhub-py/sentinelhub/areas.py to allow me to create a <sentinelhub.areas.BBoxSplitter object> based on a bbox list which is already existing.

The need stems from the fact that I have a reference tile grid (contiguous bboxes covering the entire globe) with uuid's, and I want to make sure the processing is performed consistently within these grid tiles, and not have the AOI split up into new tiles which do not correspond to this 'reference' grid.

Essentially, I just want to provide my list of rectangular geometries (e.g. shapely.geometry.box objects) covering my AOI to the AreaSplitter method, and have this list returned as an <sentinelhub.areas.BBoxSplitter object>, so that I can still leverage the other attributes from the method like get_info_list() or get_bbox_list().

Thanks in advance!

Inconsistent Tile Geometries

I wanted to report an inconsistency on the tile geometry of two images that belong to the same MGRS grid cell. The tile geometry is not the same in the two links below:

http://sentinel-s2-l1c.s3.amazonaws.com/tiles/22/J/BP/2015/11/28/0/tileInfo.json

http://sentinel-s2-l1c.s3.amazonaws.com/tiles/22/J/BP/2017/3/29/0/tileInfo.json

It looks like the 2015 tile geometry is set to a wrong MGRS tile.

Error in AwsTileRequest

Hi I'm getting error when I'm trying to create AwsTileRequest object. Even when I'm trying to run the sample code mentioned in the documentation I'm facing the below error.

`
from sentinelhub import AwsTile

tile_id = 'S2A_OPER_MSI_L1C_TL_MTI__20151219T100121_A002563_T38TML_N02.01'
tile_name, time, aws_index = AwsTile.tile_id_to_tile(tile_id)

from sentinelhub import AwsTileRequest

bands = ['B8A', 'B10']
metafiles = ['tileInfo', 'preview', 'qi/MSK_CLOUDS_B00']
data_folder = './AwsData'

request = AwsTileRequest(tile=tile_name, time=time, aws_index=aws_index,
bands=bands, metafiles=metafiles, data_folder=data_folder,
data_source=DataSource.SENTINEL2_L1C)

---

TypeError Traceback (most recent call last)
in ()
7 request = AwsTileRequest(tile=tile_name, time=time, aws_index=aws_index,
8 bands=bands, metafiles=metafiles, data_folder=data_folder,
----> 9 data_source=DataSource.SENTINEL2_L1C)

~/.conda/envs/web-static/lib/python3.6/site-packages/sentinelhub/data_request.py in init(self, tile, time, aws_index, data_source, **kwargs)
676 self.data_source = data_source
677
--> 678 super(AwsTileRequest, self).init(**kwargs)
679
680 def create_request(self):

~/.conda/envs/web-static/lib/python3.6/site-packages/sentinelhub/data_request.py in init(self, bands, metafiles, safe_format, **kwargs)
587
588 self.aws_service = None
--> 589 super(AwsRequest, self).init(**kwargs)
590
591 @AbstractMethod

~/.conda/envs/web-static/lib/python3.6/site-packages/sentinelhub/data_request.py in init(self, data_folder)
39 self.download_list = []
40 self.folder_list = []
---> 41 self.create_request()
42
43 @AbstractMethod

~/.conda/envs/web-static/lib/python3.6/site-packages/sentinelhub/data_request.py in create_request(self)
684 else:
685 self.aws_service = AwsTile(self.tile, self.time, self.aws_index, bands=self.bands,
--> 686 metafiles=self.metafiles, data_source=self.data_source)
687
688 self.download_list, self.folder_list = self.aws_service.get_requests()

~/.conda/envs/web-static/lib/python3.6/site-packages/sentinelhub/aws.py in init(self, tile_name, time, aws_index, data_source, **kwargs)
467 self.data_source = data_source
468
--> 469 super(AwsTile, self).init(**kwargs)
470 self.tile_url = None
471

TypeError: super(type, obj): obj must be an instance or subtype of type
`

More data on failed request?

Hello, I'm a new user of sentinelhub-py, and I'm currently trying out different calls and custom configuration layers.
However, at times I find that my requests fail, and using sentinelhub-py the only message I get is Failed to download from <URL>. Calling the HTTP GET request in the browser I found the ServiceException tag which is much more descriptive of the error. In my example it says:

<ServiceExceptionReport xmlns="http://www.opengis.net/ogc" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="1.2.0" xsi:schemaLocation="http://www.opengis.net/ogc http://schemas.opengis.net/wcs/1.0.0/OGC-exception.xsd">
<ServiceException>
<![CDATA[ Failed to evaluate script! ]]>
</ServiceException>
</ServiceExceptionReport>

This way it can be seen that the error is due to the script for the respective configuration layer.

Could someone extend the raise of DownloadFailedException at line 210 in sentinelhub/download.py with this more descriptive information?

Is there a way to filter by cloud cover ?

I'm looking through your functions, is there a way to filter by cloud cover ?
Trying to get the latest tile with cloud cover between 0-5%

Thank you

Polygon (replace bounding boxes)

Is there a parameter for generating areas of interest as polygons?

Like described for the http-request: https://www.sentinel-hub.com/faq/how-can-i-clip-image-specific-polygon

from sentinelhub import Polygon --> does not work, which indicates that it will not be as easy as with BBoxes

WCS request with time range writes empty images

The following code example shows a WCS request, which should automatically save tiff-images of a specific bounding box (mybbox) and time range. The Layer is called 'NDVI' in my Configurater Utility.

mybbox = [47.24, 11.05, 47.61, 11.55]    
BBOX = BBox(bbox=mybbox, crs=CRS.WGS84)

wcs = WcsRequest(data_folder=outdir,
                              layer='NDVI',
                              bbox=BBOX,
                              time=('2017-08-15', '2017-08-25'),
                              resx='20m', resy='20m',
                              image_format=MimeType.TIFF_d32f,
                              instance_id=INSTANCE_ID)

wcs_img = wcs.get_data(save_data=True)

The NDVI images are completely downloaded. However, only 3 from 5 images have a content, while the other 2 are empty. Because it is a time range and I get no error message, what is the most convenient way to avoid that issue. I also tried wcs.get_data(redownload=True), but this did not solve the problem.

AttributeError: UTM_*

In a following script AtributeError for UTM_33 appears not able to read this atribute. Am I using a correct CRS name? I adjusted it according to: http://sentinelhub-py.readthedocs.io/en/latest/common.html#sentinelhub.common.BBox.

betsiboka_coords_utm = [545461.852, 5223143.25, 548510.849, 5225341.7800]
betsiboka_bbox = BBox(bbox=betsiboka_coords_utm,crs=CRS.UTM_33)

wcs_true_color_request = WcsRequest(data_folder='C:\DATA_raw\Sentinel2_WCScalls\data\Mitterdorf_32633_atm_testSentinelhubpython',
source=DataSource.WCS,
layer='B04',
bbox=betsiboka_bbox,
time=('2017-12-01','2017-12-31'),
image_format=MimeType.TIFF_d32f,
instance_id=userID)

wcs_true_color_img = wcs_true_color_request.get_data(save_data=True)

Swap opencv-python for opencv-python-headless

With the release of opencv-python 3.3.0.10, new dependencies were added to support Qt. This has made it harder to use opencv-python in some server environments were certain libraries might not be available, and/or not possible to install. See opencv/opencv-python#44 for one such example.

In response opencv-python now provides headless packages.

We are using sentinelhub-py in a server environment where installing the GUI related dependancies might not be possible, so for now we've just had to pin our opencv-python package to version 3.3.0.9.

Would it be possible switch sentinelhub-py's requirement on opencv-python to opencv-python-headless? I've had a look through the code here, and done some testing locally that would seem to suggest the two could be swapped directly.

Interested in any thoughts on this. Thanks.

Date filter

Download functions work great..no problem there....thanks!
Is there a way to return only the most recent tile? eg under /55/H/CV/
How do I access by tile the max year YYYY, max month M and max day DD available? Is this possible?

Additional Processing for Sentinel-1 Data?

I've successfully downloaded sentinel-1 thanks to the really helpful jupyter notebook examples but I'm curious what kind of post-processing needs to go into the data now?

I've ingesting this into a series of Keras deep learning and SKlearn machine learning models for land cover classification and I'm just not sure what state the data is in and what additional processing might be necessary. Are there any good examples of this or somewhere in the docs I can read to learn more?

Thanks for all the work thus far!

Basic question: Recording frequencies

I downloaded images using a defined time range ('2017-03-01','2017-10-31). I recognized that the recording frequency is only five days for my investigated area because I only got images every five days. Is that the normal recording rate/frequency?

I used:

wcs_true_color_request = WcsRequest(data_folder='/home/Test/',
                                    layer='TRUE_COLOR',
                                    bbox=test_bbox,
                                    time=('2017-03-01','2017-10-31'),
                                    resx='10m', resy='10m',
                                    image_format=MimeType.TIFF,
                                    instance_id=INSTANCE_ID)
wcs_bands_img = wcs_true_color_request.save_data()

AwsProductRequest().get_data () causing python to crash

Hi,

I am following through the aws_request examples but keep getting python crashes with request.get_data(). Request.save_data() runs successfully and the expected data is downloaded but upon calling request.get_data() python crashes:

from sentinelhub.data_request import AwsProductRequest
product_id = 'S2A_MSIL1C_20171010T003621_N0205_R002_T01WCV_20171010T003615'
data_folder = 'C:/path/to/folder'

request = AwsProductRequest(product_id=product_id, data_folder=data_folder)
data_list = request.get_data(save_data=False,redownload=True)

Issue is different when specifying the data_folder relatively, python does not crash but request.get_data() errors out:
data_folder = '.'

conda 4.3.3
python 3.6

Any ideas where I am going wrong or might this be a bug?

AwsTileRequest.get_data() doesn't work

Hi there,
I'm jsut getting into your python tool but I am not able to reproduce your Aws example.

Support querying for L2A products

Currently the function OgcRequest.get_dates queries dates for which L1C products are available. It does so regardless of whether source in Sentinel Hub Configuration Utility is set to L1C or something else (such as L2A).

Although there is a plan to have an L2A for each L1C product, we currently provide few L2A products. Consequently, when one has source set to L2A in the Configuration Utility, some queries may result in empty images (see issue #20 ).

We should fix this in the upcoming releases.