esa-philab / major-tom Goto Github PK

I downloaded some data and noticed that some S2 data is completely black, e.g., grid cell 207D_1378R or 438U_1009R. The S1 data looks fine.

I used the filter_download function that is provided in this repo, I tested with and without by_row. I also tested Image.open(BytesIO(table[col][0].as_py())).show() with the same result.

The tif files do not include a FillValue. I assume 0 is used for NaN values?

Is it possible that some data got corrupted during the download or upload to HF?

Not sure how I did this, but I was going between L2A-1C:

Traceback (most recent call last):
  File "/home/user/.local/lib/python3.11/site-packages/reacton/ipyvue.py", line 23, in handler
    callback_ref.current(*args)
  File "/home/user/.local/lib/python3.11/site-packages/solara/components/button.py", line 84, in <lambda>
    ipyvue.use_event(btn, click_event, lambda *_ignore: on_click and on_click())
                                                                     ^^^^^^^^^^
  File "/app.py", line 54, in update_image
    ret = map_to_image(m, return_centre=True, return_gridcell=True, l2a=(s2_level.value=='L2A'))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/helpers/functional.py", line 72, in map_to_image
    bbox = map.get_bbox()
           ^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/leafmap/leafmap.py", line 4324, in get_bbox
    bbox = [bounds[0][1], bounds[0][0], bounds[1][1], bounds[1][0]]
            ~~~~~~^^^
IndexError: tuple index out of range

Hi! Thanks a lot for releasing the dataset! I have a question about matching S1 and S2 data.

Based on the current processing, we estimate around 75% of the grid cells which have Sentinel-2 data will be successfully sampled with Sentinel-1 data.

I wonder how this number is calculated. Per my calculation, around 50% of the S2 grids can be matched with an S1 grid within an offset of four days. Here is how I did it

gdf_s2.sort_values("timestamp", inplace=True)
gdf_s1.sort_values("timestamp", inplace=True)

gdf_matched = pd.merge_asof(left=gdf_s2, right=gdf_s1, suffixes=("_s2", "_s1"), by="grid_cell", direction='nearest', on="timestamp", tolerance=pd.Timedelta(pd.offsets.Day(4)), allow_exact_matches=False).dropna(subset="parquet_url_s1", axis=0)

Any clarification would be much appreciated!

Please include python code to recompute thumbnails and also how to compute full scale True Color Images from the three RGB bands, perhaps along the lines of https://sentinels.copernicus.eu/web/sentinel/user-guides/sentinel-2-msi/definitions. In practice, the details of the algorithm matter a great deal.

Do you think you could provide a demo of filtering/query speed at different levels of image/request granularity?