bluegreen-labs / gee_subset Goto Github PK

Error on this code, add trapping routine loading an Image instead.

   # define the collection from which to sample
   col = ee.ImageCollection(product).\
    select(tuple(bands)).\
filterDate(start_date, end_date)

Hello,

I am using the following command:

./gee_subset.py -p "LANDSAT/LC08/C01/T1" -s 2013-01-01 -e 2018-12-31 -b "B4" "B5" -l 38.17619870263358 -91.7266592510949

to generate a time series, of B4 and B5 for Landsat 8, but I noticed that it is generating ~ 400 values, with the first 111 values between 2013-2018, and the next 100 or so values for different dates between 2013-2018. Is this expected behavior? I do not think so many values should be generated for a single location in the given time range, given the temporal frequency of Landsat 8.

Appreciate your help.

To make the install easier, put things on pip.

Hi @khufkens,

I was trying to import gee_subset as a module but for some reason, ee has problem resolving the geometry part (when it reads the area as a region not as a point).

The snippet is:

`import ee, gee_subset, sys
ee.Initialize()

df = gee_subset.gee_subset(product = "LANDSAT/LC08/C01/T1",
bands = "B4 B5",
start_date = "2015-01-01",
end_date = "2016-12-31",
latitude = 44.0646,
longitude = -71.2875,
scale = 0,
pad = 0)
It returns this error:---------------------------------------------------------------------------
EEException Traceback (most recent call last)
in ()
6 longitude = -71.2875,
7 scale = 0,
----> 8 pad = 0)

/home/hilagavm/gee_subset/gee_subset/gee_subset.pyc in gee_subset(product, bands, start_date, end_date, latitude, longitude, scale, pad)
130
131 # region values as generated by getRegion
--> 132 region = col.getRegion(geometry, int(scale)).getInfo()
133
134 # stuff the values in a dataframe for convenience

/home/hilagavm/.local/lib/python2.7/site-packages/earthengine_api-0.1.144-py2.7.egg/ee/computedobject.pyc in getInfo(self)
93 The object can evaluate to anything.
94 """
---> 95 return data.getValue({'json': self.serialize()})
96
97 def encode(self, encoder):

/home/hilagavm/.local/lib/python2.7/site-packages/earthengine_api-0.1.144-py2.7.egg/ee/data.pyc in getValue(params)
267 """
268 params['json_format'] = 'v2'
--> 269 return send_('/value', params)
270
271

/home/hilagavm/.local/lib/python2.7/site-packages/earthengine_api-0.1.144-py2.7.egg/ee/data.pyc in send_(path, params, opt_method, opt_raw)
826 raise ee_exception.EEException('Invalid JSON: %s' % content)
827 if 'error' in json_content:
--> 828 raise ee_exception.EEException(json_content['error']['message'])
829 if 'data' not in content:
830 raise ee_exception.EEException('Malformed response: ' + str(content))

EEException: ImageCollection.getRegion: Error in map(ID=LC08_001004_20150511):
Image.select: Pattern 'B' did not match any bands.`

Following the example R script R_query_plot.R, the Python script gee_subset.py writes data into a file, named <directory>/site_gee_subset.csv. However, Python adds a string for the product name, T1 in this case, from os.path.basename(args.product) (see l.208 in gee_subset.py). Thus, R doesn't find the respective file. This is resolved by the following (R_query_plot.R, l. 54):

df = read.table( paste0( directory, "site_", tail( unlist( strsplit( product, "[/]" ) ), n=1 ), "_gee_subset.csv" ), sep = ",", header = TRUE )

Great library. I have used it recently on Colab. I tried to run it today but this error appears.

While I still cannot run the example in Winsows from R due to authentication error (credentials file is not found although when running the script directly in python it is found) I tried to run the script directly in python.

if using a csv-file for -f (site, latitude longitude) I got an error:

  File "C:/Users/user/Documents/google_earth_engine_subsets/gee_subset/gee_subset.py", line 255, in <module>
    print("processing: " + loc[1] + " at " + "%s / %s" % (loc[3],loc[2]))
TypeError: cannot concatenate 'str' and 'long' objects

My sites are numeric (1,2,3,... ).

Changing this line in the gee_subset.py script to

      print("processing: " + str(loc[1]) + " at " + "%s / %s" % (loc[3],loc[2]))

and also he following

       # depending on output options write to file
       # or just print to console   
        if args.directory and not args.image:
          df.to_csv(args.directory + "/" + str(loc[1]) + "_" + os.path.basename(args.product) + "_gee_subset.csv", index = False)
        else:
          print(df)

helped for me.

sat_name = 's1'
path = "D:/" + sat_name
if not os.path.exists(path):
os.makedirs(path)

for index, row in tqdm(gdf.iterrows(), total=gdf.shape[0]):
df1 = gee_subset.gee_subset(product="COPERNICUS/S1_GRD",
bands=["VV", "VH"], #
instrument="IW",
orbit="ASCENDING",
start_date="2023-05-01",
end_date="2023-06-30",
latitude=row['Latitude'],
longitude=row['Longitude'],
scale=20 )

sid = str(row['fid'])
df1["SiteID"] = sid

df1.to_csv(os.path.join(path, "site_" + str(sid) + ".csv"))   

报错提示 --> 203 region = col.getRegion(geometry, int(scale)).getInfo()
204 except:
...
402 return call.execute(num_retries=num_retries)
403 except googleapiclient.errors.HttpError as e:
--> 404 raise _translate_cloud_exception(e)

EEException: Image.load: Asset 'COPERNICUS/S1_GRD' is not an Image.
Output is truncated. View as a scrollable element or open in a text editor. Adjust cell output settings...

Query start and end date from location file in addition to pixel locations.

Consider adding extent as well.

Should considerably speed up downloads as multiple authentication handshakes are avoided.