marcyin / siac_gee

SIAC GEE version

SIAC on GEE

Feng Yin

Department of Geography, UCL

[email protected]

This is the GEE version of SIAC, under the intention of utilizing the power of GEE servers and wide variety of data. It has a simple UI based on the GEE UI APIs, which has the ability to search available Sentinel 2 images based on point clicked on the map, defined date range and cloud coverage, and correct for selected one image tile or do AC for all of them.

Usage:

1. Click here to open the GEE APP;

2. Click show code to show the code for the UI, and click Run to start the UI;

3. Click over the map top set AOI, using datesliders to set time period and using slider to set cloud cover threshold

4. Click search to get the desired Sentinel 2 images, then choose to Do AC for one selected image or for all the results;

5. Cick Do LAI to get LAI products for all the results

Click RUN buttons to submit jobs to GEE server:

It is hard to click RUN button for every single image, so a piece of code to do it. To use it, you need to press F12 in your keyboard to open browser console and paste the following code:

Great thanks to Dongdong Kong for sharing it on stackexchange!

/**
 * Copyright (c) 2017 Dongdong Kong. All rights reserved.
 * This work is licensed under the terms of the MIT license.  
 * For a copy, see <https://opensource.org/licenses/MIT>.
 *
 * Batch execute GEE Export task
 *
 * First of all, You need to generate export tasks. And run button was shown.
 *   
 * Then press F12 get into console, then paste those scripts in it, and press 
 * enter. All the task will be start automatically. 
 * (Firefox and Chrome are supported. Other Browsers I didn't test.)
 * 
 * @Author: 
 *  Dongdong Kong , 28 Aug' 2017 
 *      Sun Yat-sen University
 */

function runTaskList(){
    var tasklist = document.getElementsByClassName('task local type-EXPORT_IMAGE awaiting-user-config');
    for (var i = 0; i < tasklist.length; i++)
            tasklist[i].children[2].click();
}
// confirmAll();
function confirmAll() {
    var ok = document.getElementsByClassName('goog-buttonset-default goog-buttonset-action');
    for (var i = 0; i < ok.length; i++)
        ok[i].click();
}
runTaskList();
confirmAll();

Reference:

Yin, F., Lewis, P. E., Gomez-Dans, J., & Wu, Q. (2019, February 21). A sensor-invariant atmospheric correction method: application to Sentinel-2/MSI and Landsat 8/OLI. https://doi.org/10.31223/osf.io/ps957

An example UI:

LICENSE

GNU GENERAL PUBLIC LICENSE V3

siac_gee's Issues

SIAC not working for S2 imagery since March 2023

Is there any issue with SIAC_GEE? it seems it has stopped working for Sentinel-2 MSI images from March 2023 because there is no output.
Please, let us know

Could not load script

After clicking the GEE link in ‘1. Click here
to open the GEE APP’, the following error messages pop up:
‘Could not load required script from: users/marcyinfeng/SIAC:SIAC_AC_ALL’
‘Could not load required script at: users/marcyinfeng/SIAC:SIAC_AC_ALL’

gee_monkey script license

Why copy my gee_monkey script and remove the author name?

https://github.com/kongdd/gee_monkey
https://gis.stackexchange.com/questions/290771/batch-task-execution-in-google-earth-engine

/**
 * Copyright (c) 2017 Dongdong Kong. All rights reserved.
 * This work is licensed under the terms of the MIT license.  
 * For a copy, see <https://opensource.org/licenses/MIT>.
 *
 * Batch execute GEE Export task
 *
 * First of all, You need to generate export tasks. And run button was shown.
 *   
 * Then press F12 get into console, then paste those scripts in it, and press 
 * enter. All the task will be start automatically. 
 * (Firefox and Chrome are supported. Other Browsers I didn't test.)
 * 
 * @Author: 
 *  Dongdong Kong , 28 Aug' 2017 
 *      Sun Yat-sen University
 */
function runTaskList(){
    var tasklist = document.getElementsByClassName('task local type-EXPORT_IMAGE awaiting-user-config');
    for (var i = 0; i < tasklist.length; i++)
            tasklist[i].getElementsByClassName('run-button')[0].click();
}

function confirmAll() {
    var ok = document.getElementsByClassName('goog-buttonset-default goog-buttonset-action');
    for (var i = 0; i < ok.length; i++)
        ok[i].click();
}

runTaskList();
confirmAll();

addBands error

Hi,

First of all, really excited to see this code available as I can't get Sentinel BoA reflectance from the level 2 products in Sentinel_S2/SR for 2017 for my study site.

I'm getting the below error using the SIAC code, but probably more a problem with my code structure.

Error:
In users/marcyinfeng/utils:SIAC
Line 152: image.addBands is not a function

Can you advise please?

My code:
var site = table.geometry();
Map.centerObject(site);
Map.addLayer(site, {color: 'red'}, 'site');

var label = ee.String('Trangie');
// Map.addLayer(roi);
// Map.addLayer(pad)

// This field contains UNIX time in milliseconds.
var timeField = 'system:time_start';

var addVariables = function(image) {
// Compute time in fractional years since the epoch.
var date = ee.Date(image.get(timeField));
var years = date.difference(ee.Date('1970-01-01'), 'year');
// Return the image with the added bands.
return image
// Add a time band.
.addBands(ee.Image(years).rename('t').float())
// Add a constant band.
.addBands(ee.Image.constant(1))
};

//function for masking clouds
var maskClouds= function(img) {
var qa = img.select('QA60');
var mask = qa.bitwiseAnd(0x400).eq(0).and( // no clouds
qa.bitwiseAnd(0x800).eq(0)); // no cirrus
return img.updateMask(mask).resample('bicubic');
};

//call sentinel2 galery for t1: May 2020
var im_toa = ee.ImageCollection('COPERNICUS/S2').select(['B2','B3','B4', 'B5', 'B8', 'B7', 'B6','B9','B11','QA60'])
.filterBounds(site)
.filterDate('2017-05-30','2017-11-30')
.map(maskClouds);

var siac = require('users/marcyinfeng/utils:SIAC');

//var im = siac.get_sur(im_toa);
var im = im_toa.map(function(image){
return siac.get_sur(im_toa);
})

var Blue = im.select('B2');
var Green = im.select('B3');
var Red = im.select('B4');
var NIR = im.select('B8');
var REP1 = im.select('B5');
var REP2 = im.select('B6');
var REP3 = im.select('B7');
var SWIR = im.select('B11');

var max=17

//List Blue
var list=Blue.toList(150);
for (var i=0;i<max;i++){
var image=ee.Image(list.get(i));
//var image_toa=ee.Image(list.get(i));
//var image = siac.get_sur(image_toa);
var date = image.date().format('yyyy-MM-dd').getInfo();
// Export Blue
Export.image.toDrive({
image: image,
description: 'Blue_'+i.toString()+''+date,
fileNamePrefix: 'Blue'+i.toString()+'_'+date,
scale: 20,
region:site,
crs : 'EPSG:3857'
});
}

//List Green
var list=Green.toList(150);
for (var i=0;i<max;i++){
var image=ee.Image(list.get(i));
//var image_toa=ee.Image(list.get(i));
//var image = siac.get_sur(image_toa);
var date = image.date().format('yyyy-MM-dd').getInfo();
// Export Blue
Export.image.toDrive({
image: image,
description: 'Green_'+i.toString()+''+date,
fileNamePrefix: 'Green'+i.toString()+'_'+date,
scale: 20,
region:site,
crs : 'EPSG:3857'
});
}

//List Red
var list=Red.toList(150);
for (var i=0;i<max;i++){
var image=ee.Image(list.get(i));
//var image_toa=ee.Image(list.get(i));
//var image = siac.get_sur(image_toa);
var date = image.date().format('yyyy-MM-dd').getInfo();
// Export Blue
Export.image.toDrive({
image: image,
description: 'Red_'+i.toString()+''+date,
fileNamePrefix: 'Red'+i.toString()+'_'+date,
scale: 20,
region:site,
crs : 'EPSG:3857'
});
}

//List REP1
var list=REP1.toList(150);
for (var i=0;i<max;i++){
var image=ee.Image(list.get(i));
//var image_toa=ee.Image(list.get(i));
//var image = siac.get_sur(image_toa);
var date = image.date().format('yyyy-MM-dd').getInfo();
// Export Blue
Export.image.toDrive({
image: image,
description: 'REP1_'+i.toString()+''+date,
fileNamePrefix: 'REP1'+i.toString()+'_'+date,
scale: 20,
region:site,
crs : 'EPSG:3857'
});
}

//List REP2
var list=REP2.toList(150);
for (var i=0;i<max;i++){
var image=ee.Image(list.get(i));
//var image_toa=ee.Image(list.get(i));
//var image = siac.get_sur(image_toa);
var date = image.date().format('yyyy-MM-dd').getInfo();
// Export Blue
Export.image.toDrive({
image: image,
description: 'REP2_'+i.toString()+''+date,
fileNamePrefix: 'REP2'+i.toString()+'_'+date,
scale: 20,
region:site,
crs : 'EPSG:3857'
});
}

//List REP3
var list=REP3.toList(150);
for (var i=0;i<max;i++){
var image=ee.Image(list.get(i));
//var image_toa=ee.Image(list.get(i));
//var image = siac.get_sur(image_toa);
var date = image.date().format('yyyy-MM-dd').getInfo();
// Export REP3
Export.image.toDrive({
image: image,
description: 'REP3_'+i.toString()+''+date,
fileNamePrefix: 'REP3'+i.toString()+'_'+date,
scale: 20,
region:site,
crs : 'EPSG:3857'
});
}

//List NIR
var list=NIR.toList(150);
for (var i=0;i<max;i++){
var image=ee.Image(list.get(i));
//var image_toa=ee.Image(list.get(i));
//var image = siac.get_sur(image_toa);
var date = image.date().format('yyyy-MM-dd').getInfo();
// Export NIR
Export.image.toDrive({
image: image,
description: 'NIR_'+i.toString()+''+date,
fileNamePrefix: 'NIR'+i.toString()+'_'+date,
scale: 20,
region:site,
crs : 'EPSG:3857'
});
}

//List SWIR
var list=SWIR.toList(150);
for (var i=0;i<max;i++){
var image=ee.Image(list.get(i));
//var image_toa=ee.Image(list.get(i));
//var image = siac.get_sur(image_toa);
var date = image.date().format('yyyy-MM-dd').getInfo();
// Export Blue
Export.image.toDrive({
image: image,
description: 'SWIR_'+i.toString()+''+date,
fileNamePrefix: 'SWIR'+i.toString()+'_'+date,
scale: 20,
region:site,
crs : 'EPSG:3857'
});
}

Nodata for B2, B3, B4 and B8 on specific dates after SIAC_S2

Hello Marc!

I noticed that my time series for vegetation index was missing some values in a specific period of time (october-december 2021, see image below)

I sampled some images before/after applying the SIAC and I got no data after the correction for all the bands I am working with.

The weird thing is that I could get the values for that gap last week, but got this problem a few days ago after running my code again.

Any idea what is causing this?

Can the SIAC be uploaded to gee Reader so that it can be used in geemap with python?

Operational extent at high latitudes

Greetings.

I have a question regarding atmospheric correction module and the data used.
It seems that 'USGS/SRTMGL1_003' dataset is used by SIAC_GEE for elevation information.
If one imports the SIAC atmospheric correction module to GEE ('users/marcyinfeng/utils:SIAC'),
get_sur -function seems to produce outputs for Sentinel-2 data at latitudes >60N, beyond the coverage of the elevation data used. Is use of SIAC_GEE possible at latitudes beyond SRTM Digital Elevation data, or is unwanted behaviour expected at these areas?

I did not find any documentation regarding possible limitations in operational extent.

Thank you for you time.