adugnag / gee_s1_ard

Creates an analysis ready sentinel-1 SAR image collection in Google Earth Engine by applying additional border noise correction, speckle filtering and radiometric terrain normalization.

Sentinel-1 SAR Backscatter Analysis Ready Data Preparation in Google Earth Engine

Introduction

The Sentinel-1 satellites provide temporally dense and high spatial resolution synthetic aperture radar (SAR) imagery. The open data policy and global coverage of Sentinel-1 make it a valuable data source for a wide range of SAR-based applications. In this regard, Google Earth Engine (GEE) is a key platform for large area analysis with preprocessed Sentinel-1 backscatter images being available within few days after acquisition. In this implementation, we present a framework for preparing Sentinel-1 SAR backscatter Analysis-Ready-Data (ARD) in GEE that implements additional border noise correction, speckle filtering and radiometric terrain normalization. The proposed framework can be used to generate Sentinel-1 ARD suitable for a wide range of land and inland water mapping/monitoring applications. The ARD preparation framework is implemented in GEE JavaScript and Python API's.

This framework is intended for researchers and non-experts in microwave remote sensing. It is intended to provide flexibility for a wide variety of large area land and inland water monitoring applications.

Features

This framework generates a Sentinel-1 SAR ARD by applying three processing modules.

1. Addtional Border noise correction
2. Speckle Filtering
   • Mono-temporal
   • Multi-temporal
3. Radiometric Terrain Normalization

The framework processes single (VV or VH) or dual (VV and VH) polarization data in either ascending, descending or both orbits at the same time. Results can be displayed and exported in the linear or dB scale.

Usage

The details about parameter setting and their associated methods is described in the main script and accompanying technical note published in MDPI Remote sensing.

To use the framework in GEE code editor, go to the gee_s1_ard public repo and copy the contents of s1_ard.js to your own repository. The path to the preprocessing functions i.e. ('users/adugnagirma/gee_s1_ard') is a public so you don't need to have the preprocessing functions copied to your repository.

When using the Python API, the user should adjust the script path and GEE id to their own path and id before processing.

RGB visualization of a dual polarized (VV and VH) Sentinel-1 SAR backscatter image of central Borneo, Indonesia (Lat: -0.35, Lon: 112.15) (a) as ingested into Google Earth Engine; and (b) after applying additional boarder noise removal, a 9×9 multi-temporal Gamma MAP specklefilter and radiometric terrain normalization with a volume scattering model. Here VV is in red,VH is in green and VV/VH ratio is in blue.

Dependencies

The JavaScript code runs in the GEE code editor with out installing additional packages. However, the python code requires the installation of Google Earth Engine API

Citation

Mullissa, A.; Vollrath, A.; Odongo-Braun, C.; Slagter, B.; Balling, J.; Gou, Y.; Gorelick, N.; Reiche, J. Sentinel-1 SAR Backscatter Analysis Ready Data Preparation in Google Earth Engine. Remote Sens. 2021, 13, 1954. https://doi.org/10.3390/rs13101954

gee_s1_ard's Issues

Is the mixing of DESCENDING and ASCENDING data allowed?

Hi,
I'm sorry to bother you again.
Will the mixed use of DESCENDING and ASCENDING data cause any problems?
I've seen a lot of research clearly marked the type of orbital used.
And heard that they can not be mixed due to the difference of backscattering coefficient.
Is that true?
Thanks!!!

Bug in S1 ARD slope correction

There was an error in the calculation of the aspect and heading angle. The original implementation in Hoekman and Reiche 2015 expects angles from -180 to 180, while the aspect computed in GEE is from 0 to 360. In addition, the aspect angle on GEE seems downhill directed, while it should be uphill.

As a result, the correction factor needs to be multiplied by gamma0 instead of being divided. All this is corrected in the updated version.

Multitemporal filter doesn't do anything (except singletemporal filtering)

I did multitemporal lee filter on S1 collection (whole year) and it does nothing else than Lee filter itself.

Speckle filters creating artifact at scene edges

The speckle filters create artifacts at scene gaps when SPECKLE_FILTER_FRAMEWORK: is set to 'MULTI'

MULTI mode example

MONO mode example

Note: These are collection means, not the first image of the collections.

s1_preprocess does not show on python folium map.

Nothing is ploted when FORMAT='DB'.

#Parameters
parameter = {  'START_DATE': '2016-06-23',
            'STOP_DATE': '2020-02-01',
            'POLARIZATION': 'VVVH',
            'ORBIT' : 'BOTH',
            'ROI': train_aoi,
            'APPLY_BORDER_NOISE_CORRECTION': True,
            'APPLY_SPECKLE_FILTERING': True,
            'SPECKLE_FILTER_FRAMEWORK':'MULTI',
            'SPECKLE_FILTER': 'BOXCAR',
            'SPECKLE_FILTER_KERNEL_SIZE': 10,
            'SPECKLE_FILTER_NR_OF_IMAGES':15,
            'APPLY_TERRAIN_FLATTENING': True,
            'DEM': ee.Image('USGS/SRTMGL1_003'),
            'TERRAIN_FLATTENING_MODEL': 'VOLUME',
            'TERRAIN_FLATTENING_ADDITIONAL_LAYOVER_SHADOW_BUFFER':0,
            'FORMAT': 'LINEAR',
            'CLIP_TO_ROI': True,
            'SAVE_ASSET': False,
            'ASSET_ID': "",
            'FOLDER_NAME': "s1_data"
            }
#processed s1 collection
s1_processed = s1_preproc(parameter)

s1 = s1_processed[0]
s1_processed = s1_processed[1]

visparam = {}
if (parameter["POLARIZATION"]=='VVVH'):
    if (parameter["FORMAT"]=='DB'):
        s1_processed_view = s1_processed.map(add_ratio_lin).map(lin_to_db2)
        s1_view = s1.map(add_ratio_lin).map(lin_to_db2)
        visparam = {"bands":['VV','VH','VVVH_ratio'],"min": [-20, -25, 1],"max": [0, -5, 15], "gamma": 1.1}

    else:
        s1_processed_view = s1_processed.map(add_ratio_lin)
        s1_view = s1.map(add_ratio_lin)
        visparam = {"bands":['VV','VH','VVVH_ratio'], "min": [0.01, 0.0032, 1.25],"max": [1, 0.31, 31.62], "gamma": 1.1}

else:
    if (parameter["FORMAT"]=='DB'):
        s1_processed_view = s1_processed.map(lin_to_db)
        s1_view = s1.map(lin_to_db)
        visparam = {"bands":[parameter["POLARIZATION"]],"min": -25,"max": 0}

    else:
        s1_processed_view = s1_processed
        s1_view = s1
        visparam = {"bands":[parameter["POLARIZATION"]],"min": 0,"max": 0.2}

print(visparam)
center = train_aoi.centroid(10).coordinates().reverse().getInfo()
m = folium.Map(location=center, zoom_start = 12)

m.add_ee_layer(
    s1_view.first(), visparam, "First image in the input S1 collection", False
)
"""
m.add_ee_layer(
    s1_processed_view.first(), visparam, "First image in the processed S1 collection", True
)

m.add_ee_layer(
    train_s2_col_list[0].mosaic().clip(train_aoi),
                    {'bands': ['B4', 'B3', 'B2'], 'min': 0, 'max': 2500, 'gamma': 1.1},
                    f'S2 image', False, 1, 9
)

m.add_child(folium.LayerControl())

display(m)

Problem with helper.py function

Tank you for your work!

I want to use the python api, but imagecollection.map(helper.lin_to_db) or imagecollection.map(helper.lin_to_db2) got the non valve in VV and VH.
here is my code:

parameter = {'START_DATE': startdate,
'STOP_DATE': enddate,
'POLARIZATION': 'VVVH',
'ORBIT' : 'BOTH',
#'ORBIT_NUM': None,
'ROI': geometry,
'APPLY_BORDER_NOISE_CORRECTION': True,
'APPLY_SPECKLE_FILTERING': True,
'SPECKLE_FILTER_FRAMEWORK':'MULTI',
'SPECKLE_FILTER': 'BOXCAR',
'SPECKLE_FILTER_KERNEL_SIZE': 50,
'SPECKLE_FILTER_NR_OF_IMAGES':50,
'APPLY_TERRAIN_FLATTENING': True,
'DEM': ee.Image('USGS/SRTMGL1_003'),
'TERRAIN_FLATTENING_MODEL': 'VOLUME',
'TERRAIN_FLATTENING_ADDITIONAL_LAYOVER_SHADOW_BUFFER':0,
'FORMAT': 'DB',
'CLIP_TO_ROI': True,
'SAVE_ASSET': False,
'ASSET_ID': "users/mraj"
}
#processed s1 collection
s1_processed = wp.s1_preproc(parameter)

visparam = {}
if parameter.get('POLARIZATION')=='VVVH':
if parameter.get('FORMAT')=='DB':
s1_preprocces_view = s1_processed.map(helper.add_ratio_lin).map(helper.lin_to_db2)
visparam = {'bands':['VV', 'VH', 'VVVH_ratio'], 'min': [-20, -25, 0], 'max': [0, -5, 0.7]}
else:
s1_preprocces_view = s1_processed.map(helper.add_ratio_lin)
visparam = {'bands':['VV', 'VH', 'VVVH_ratio'], 'min': [0.01, 0.0032, 0], 'max': [1, 0.31, 0.7]}
else:
if parameter.get('FORMAT')=='DB':
s1_preprocces_view = s1_processed.map(helper.lin_to_db)
visparam = {'bands':[parameter.POLARIZATION], 'min': -25, 'max': 0}
else:
s1_preprocces_view = s1_processed
visparam = {'bands':[parameter.POLARIZATION], 'min': 0, 'max': 0.2}

Map.addLayer(s1_preprocces_view.first().clip(geometry), visparam, 'First image in the processed S1 collection');

#######
error:

EEException: Collection.first: Error in map(ID=S1A_IW_GRDH_1SDV_20200604T100319_20200604T100344_032866_03CE8D_F0CE):
Image.select: Pattern 'VV' did not match any bands.

Question about availability of VVVH

Hello,

I was trying out the code as written, and noticed I was getting the following error message below, which to me indicates that either

• there is no VH available, which is odd considering the parameter VVVH means the ImageCollection is filtered to contain both bands, or
• the bands are being renamed incorrectly in the underlying scripts.

I tried changing the wrapper script such that it contained the following code but this did not solve the issue. Possibly this might be because there is a period of time for this point (Myanmar) where some data was not obtained, but I had thought adding this filtering process would mean dates were analyzed only if they contained observations for both polarizations.

if (params.POLARIZATION=='VV') { s1 = s1.select(['VV','angle'])}
  else if (params.POLARIZATION=='VH') {s1 = s1.select(['VH','angle'])}
  else if (params.POLARIZATION=='VVVH') {
    s1 = s1.filter(ee.Filter.listContains('transmitterReceiverPolarisation', 'VV'))
            .filter(ee.Filter.listContains('transmitterReceiverPolarisation', 'VH'))
            .select(['VV', 'VH', 'angle'])}

Can you please advise? Here is the set of parameters with my geometry.

var pt = ee.Geometry.Point([95.219623, 23.029892])
  
var wrapper = require('users/adugnagirma/gee_s1_ard:wrapper');
var helper = require('users/adugnagirma/gee_s1_ard:utilities');

//---------------------------------------------------------------------------//
// DEFINE PARAMETERS
//---------------------------------------------------------------------------//

var parameter = {//1. Data Selection
              START_DATE: "2015-06-23",
              STOP_DATE: "2020-02-01",
              POLARIZATION:'VVVH',
              ORBIT : 'BOTH',
              GEOMETRY: pt, //uncomment if interactively selecting a region of interest
              //GEOMETRY: ee.Geometry.Polygon([[[104.80, 11.61],[104.80, 11.36],[105.16, 11.36],[105.16, 11.61]]], null, false), //Uncomment if providing coordinates
              //GEOMETRY: ee.Geometry.Polygon([[[112.05, -0.25],[112.05, -0.45],[112.25, -0.45],[112.25, -0.25]]], null, false),
              //2. Additional Border noise correction
              APPLY_ADDITIONAL_BORDER_NOISE_CORRECTION: true,
              //3.Speckle filter
              APPLY_SPECKLE_FILTERING: true,
              SPECKLE_FILTER_FRAMEWORK: 'MULTI',
              SPECKLE_FILTER: 'REFINED LEE',
              SPECKLE_FILTER_KERNEL_SIZE: 7,
              SPECKLE_FILTER_NR_OF_IMAGES: 10,
              //4. Radiometric terrain normalization
              APPLY_TERRAIN_FLATTENING: true,
              DEM: ee.Image('USGS/SRTMGL1_003'),
              TERRAIN_FLATTENING_MODEL: 'VOLUME',
              TERRAIN_FLATTENING_ADDITIONAL_LAYOVER_SHADOW_BUFFER: 0,
              //5. Output
              FORMAT : 'DB',
              CLIP_TO_ROI: false,
              SAVE_ASSETS: false
}

//---------------------------------------------------------------------------//
// DO THE JOB
//---------------------------------------------------------------------------//
     
//Preprocess the S1 collection
var s1_preprocces = wrapper.s1_preproc(parameter);

var s1 = s1_preprocces[0]
s1_preprocces = s1_preprocces[1]


//---------------------------------------------------------------------------//
// EXPORT
//---------------------------------------------------------------------------//

//Convert format for export
if (parameter.FORMAT=='DB'){
  s1_preprocces = s1_preprocces.map(helper.lin_to_db);
}

print(s1_preprocces.first())
print(s1_preprocces)

//Save processed collection to asset
if(parameter.SAVE_ASSETS) {
helper.Download.ImageCollection.toAsset(s1_preprocces, '', 
               {scale: 10, 
               region: s1_preprocces.geometry(),
                type: 'float'})
}

bands not compatible when exporting

Hi, in GEE I tried to export S1 ARD images to my drive in configuration VVVH with DB values for a specific region. Unfortunately, I got for most of them (not all) the following error cause bands are not of the same type.

VV and VH bands are in double, while angle and VVVH-ratio are float types. Where in the scripts would be the implementation of a band transformation most appropriate?

Thanks for your help.
Felix

Problems downloading ImageCollection to asset

So...am such a noob. I totally forgot to go to Tasks and run the downloads.......

Hi, your code is working well, unless I set download to true, at which point the page becomes unresponsive.
I wondered whether I am alone in this experience, or whether you could assist a relative newbie in overcoming this challenge.
I m trying to map locations of small scale illegal artisanal mining in the Lower Zambezi region of Zambia where I live. I am hoping the SAR will be able to detect the locations of the mining sites. Much appreciated.

issues with .py functions

Hi there,
I just downloaded the python version of your module and tried to get it to run. the javascript functions run well, and thank you for sharing this really wonderful work.

But the python versions have a few issues. First, you refer to two parameters in the wrapper.py function that do not exist in the s1_ard.py function, specifically ORBIT_NUM and PLATFORM_NUMBER. And I want to note for others who might have issues going forward that one parameter is named differently in javascript and python, and that's the ADDITIONAL BORDER CORRECTION one. Finally, and I think I am right here, the following parameters are not optional as your comments say in python, so I had to specify them: 'CLIP_TO_ROI': False, 'SAVE_ASSET': False, 'ASSET_ID': 'Bob'

Maybe all the above is because I got an odd version in my recent download. But I wanted to post one more issue here, for some other person who might be trying to adapt your python code. In your javascript code, you export s1 and s1_1 from wrapper.js and don't do a linear to db transform. So that way, your code for calculating ratios and visualization in s1_ard.js doesn't throw an error. But in your s1_ard.py code, you do the DB conversion (if the parameter is set) and only export the s1_1, and that entire visualization section is missing. So for those looking to add that visualization code back in Python, don't forget that you must have your parameter 'FORMAT' : 'LINEAR', not 'DB', to quickly adapt that visualization code. then if you run this code in geemap it will output a map:

s1_preprocces_view = s1_processed.map(helper.add_ratio_lin).map(helper.lin_to_db2)
visparam = {'bands': ['VV', 'VH', 'VVVH_ratio'], 'min': -25, 'max':10}
Map.addLayer(s1_preprocces_view , visparam, 'Median proc BOTH input S1 collection', True)
Map

Cheers, and thanks again for the awesome work!
Matt

Correct link to the repo on GEE

Hi.

Unless I'm missing something, I'm clicking on the link to access the repo on Earth Engine (both of them in the README of this repo on GitHub), but it is taking me to a separate repo that does not have the function s1_ard.js. I instead see the repo below. Can you please advise?

extra parameters when using python api: issues

Dear adugnag, thanks for creating this fantastic package!

I have no problem running the javascript api and incorporating it into my workflow on the GEE code editor.

I am having issues running the same workflow over the same study area using the python api: I am running into issues around the extra parameters of orbit_num and platform number specifically. I noticed these are not required for the JS code but are required for the python. Can I just comment these out? I noticed there are no default values for these two?

Thanks!

Same parameters, different results in Code Editor and Python

Hi there! I am attempting to implement the gee_s1_ard script on Google Colab.

• While the script works correctly on Google Earth Engine Code Editor (returning 57 available images for my ROI), when trying to run it on Google Colab with the same parameters, it indicates no images available.
• In Code editor I've created a polygon while In python, I've imported a shapefile and then I've converted it into a ee.Geometry

Result in Code editor:

Result in Python:

Geometry in python:

Tt would help me a lot to know what mistake I am making.

Thanks in advance!

Problems in _correct function in terrain_flattening.py

Hi! I am using your library the python-API version. I find it very useful, thank you very much!

I think I found some indentation problems in wrapper.py and in terrain_flattening.py. Also, the function _correct in the terrain_flattening.py script was having some trouble. It seems that the reproject function was not calling the arguments properly. I fix them and made you a PR: #6
Thanks again!

How to implement speckle filtering only？

Dear author:
I want to use LEE filter on GEE, Sentinel-1's most commonly used filter.
And I noticed you seem to use another filtering method.
So how about this filter's performance?(compare with LEE filter)
And how can I use a filter only?
Thanks for sharing! Looking forward to your repeat! :)

Code Editor Repo not accessible

I tried running the https://github.com/adugnag/gee_s1_ard/blob/main/javascript/s1_ard.js script in the Code Editor and got a message:

Cannot find required repo: users/adugnagirma/grs_ard:utilities

Loading the helper is fine:

var helper = require('users/adugnagirma/gee_s1_ard:utilities');
print(helper);

The problem occurs when loading the wrapper:

var wrapper = require('users/adugnagirma/gee_s1_ard:wrapper');
print(wrapper);

The wrapper must be importing the grs_ard:utilities module, which is either not shared or is maybe a typo?

Absence of IDAN filter for speckle filter

Thank you for wrting these scriots,
I was looking to IDAN speckle filter in your codes but I think you did not implemented this filter.
Could you add this filter also in the future version?
Thaks

Problem with converting to decibels

Hi! Thank you very much for your work! Today I used JavaScript modules and found a strange result.

I’ve calculated sigma-nought in linear scale (‘LINEAR’) and converted it to decibels: db_scale = 10*log10(lin_scale). Then I’ve calculated sigma-nought in decibels (‘DB’) and the results didn't match. My script: https://code.earthengine.google.com/?scriptPath=users%2Fdakhramov%2Fpublic%3Atest_db

How to install the python packages in Colab

A simple question!
I want to use Python packages in Colab. When I installed the software package, the code indicated an error（As shown follows）.
How should I correct it?
Thank you first