awesome-spectral-indices / spyndex

Awesome Spectral Indices in Python.

Awesome Spectral Indices in Python:

Numpy | Pandas | GeoPandas | Xarray | Earth Engine | Planetary Computer | Dask

---

GitHub: https://github.com/davemlz/spyndex

Documentation: https://spyndex.readthedocs.io/

Paper: https://doi.org/10.1038/s41597-023-02096-0

PyPI: https://pypi.org/project/spyndex/

Conda-forge: https://anaconda.org/conda-forge/spyndex

Tutorials: https://spyndex.readthedocs.io/en/latest/tutorials.html

---

Citation

If you use this work, please consider citing the following paper:

@article{montero2023standardized,
  title={A standardized catalogue of spectral indices to advance the use of remote sensing in Earth system research},
  author={Montero, David and Aybar, C{\'e}sar and Mahecha, Miguel D and Martinuzzi, Francesco and S{\"o}chting, Maximilian and Wieneke, Sebastian},
  journal={Scientific Data},
  volume={10},
  number={1},
  pages={197},
  year={2023},
  publisher={Nature Publishing Group UK London}
}

Overview

The Awesome Spectral Indices is a standardized ready-to-use curated list of spectral indices that can be used as expressions for computing spectral indices in remote sensing applications. The list was born initially to supply spectral indices for Google Earth Engine through eemont and spectral, but given the necessity to compute spectral indices for other object classes outside the Earth Engine ecosystem, a new package was required.

Spyndex is a python package that uses the spectral indices from the Awesome Spectral Indices list and creates an expression evaluation method that is compatible with python object classes that support overloaded operators (e.g. numpy.ndarray, pandas.Series, xarray.DataArray).

Some of the spyndex features are listed here:

• Access to Spectral Indices from the Awesome Spectral Indices list.
• Multiple Spectral Indices computation.
• Kernel Indices computation.
• Parallel processing.
• Compatibility with a lot of python objects!

Check the simple usage of spyndex here:

import spyndex
import numpy as np
import xarray as xr

N = np.random.normal(0.6,0.10,10000)
R = np.random.normal(0.1,0.05,10000)

da = xr.DataArray(
    np.array([N,R]).reshape(2,100,100),
    dims = ("band","x","y"),
    coords = {"band": ["NIR","Red"]}
)

idx = spyndex.computeIndex(
    index = ["NDVI","SAVI"],
    params = {
        "N": da.sel(band = "NIR"),
        "R": da.sel(band = "Red"),
        "L": 0.5
    }
)

Bands can also be passed as keywords arguments:

idx = spyndex.computeIndex(
    index = ["NDVI","SAVI"],
    N = da.sel(band = "NIR"),
    R = da.sel(band = "Red"),
    L = 0.5
)

And indices can be computed from their class:

idx = spyndex.indices.NDVI.compute(
    N = da.sel(band = "NIR"),
    R = da.sel(band = "Red"),
)

How does it work?

Any python object class that supports overloaded operators can be used with spyndex methods.

---

"Hey... what do you mean by 'overloaded operators'?"

---

That's the million dollars' question! An object class that supports overloaded operators is the one that allows you to compute mathematical operations using common operators (+, -, /, *, **) like a + b, a + b * c or (a - b) / (a + b). You know the last one, right? That's the formula of the famous NDVI.

So, if you can use the overloaded operators with an object class, you can use that class with spyndex!

BE CAREFUL! Not all overloaded operators work as mathematical operators. In a list object class, the addition operator (+) concatenates two objects instead of performing an addition operation! So you must convert the list into a numpy.ndarray before using spyndex!

Here is a little list of object classes that support mathematical overloaded operators:

• float (Python Built-in type) or numpy.float* (with numpy)
• int (Python Built-in type) or numpy.int* (with numpy)
• numpy.ndarray (with numpy)
• pandas.Series (with pandas or geopandas)
• xarray.DataArray (with xarray)
• ee.Image (with earthengine-api and eemont)
• ee.Number (with earthengine-api and eemont)

And wait, there is more! If objects that support overloaded operatores can be used in spyndex, that means that you can work in parallel with dask!

Here is the list of the dask objects that you can use with spyndex:

• dask.Array (with dask)
• dask.Series (with dask)

This means that you can actually use spyndex in a lot of processes! For example, you can download a Sentinel-2 image with sentinelsat, open and read it with rasterio and then compute the desired spectral indices with spyndex. Or you can search through the Landsat-8 STAC in the Planetary Computer ecosystem using pystac-client, convert it to an xarray.DataArray with stackstac and then compute spectral indices using spyndex in parallel with dask! Amazing, right!?

Installation

Install the latest version from PyPI:

pip install spyndex

Upgrade spyndex by running:

pip install -U spyndex

Install the latest version from conda-forge:

conda install -c conda-forge spyndex

Install the latest dev version from GitHub by running:

pip install git+https://github.com/davemlz/spyndex

Features

Exploring Spectral Indices

Spectral Indices from the Awesome Spectral Indices list can be accessed through spyndex.indices. This is a Box object where each one of the indices in the list can be accessed as well as their attributes:

import spyndex

# All indices
spyndex.indices

# NDVI index
spyndex.indices["NDVI"]

# Or with dot notation
spyndex.indices.NDVI

# Formula of the NDVI
spyndex.indices["NDVI"]["formula"]

# Or with dot notation
spyndex.indices.NDVI.formula

# Reference of the NDVI
spyndex.indices["NDVI"]["reference"]

# Or with dot notation
spyndex.indices.NDVI.reference

Default Values

Some Spectral Indices require constant values in order to be computed. Default values can be accessed through spyndex.constants. This is a Box object where each one of the constants can be accessed:

import spyndex

# All constants
spyndex.constants

# Canopy Background Adjustment
spyndex.constants["L"]

# Or with dot notation
spyndex.constants.L

# Default value
spyndex.constants["L"]["default"]

# Or with dot notation
spyndex.constants.L.default

Band Parameters

The standard band parameters description can be accessed through spyndex.bands. This is a Box object where each one of the bands can be accessed:

import spyndex

# All bands
spyndex.bands

# Blue band
spyndex.bands["B"]

# Or with dot notation
spyndex.bands.B

One (or more) Spectral Indices Computation

Use the computeIndex() method to compute as many spectral indices as you want! The index parameter receives the spectral index or a list of spectral indices to compute, while the params parameter receives a dictionary with the required parameters for the spectral indices computation.

import spyndex
import xarray as xr
import matplotlib.pyplot as plt
from rasterio import plot

# Open a dataset (in this case a xarray.DataArray)
snt = spyndex.datasets.open("sentinel")

# Scale the data (remember that the valid domain for reflectance is [0,1])
snt = snt / 10000

# Compute the desired spectral indices
idx = spyndex.computeIndex(
    index = ["NDVI","GNDVI","SAVI"],
    params = {
        "N": snt.sel(band = "B08"),
        "R": snt.sel(band = "B04"),
        "G": snt.sel(band = "B03"),
        "L": 0.5
    }
)

# Plot the indices (and the RGB image for comparison)
fig, ax = plt.subplots(2,2,figsize = (10,10))
plot.show(snt.sel(band = ["B04","B03","B02"]).data / 0.3,ax = ax[0,0],title = "RGB")
plot.show(idx.sel(index = "NDVI").data,ax = ax[0,1],title = "NDVI")
plot.show(idx.sel(index = "GNDVI").data,ax = ax[1,0],title = "GNDVI")
plot.show(idx.sel(index = "SAVI").data,ax = ax[1,1],title = "SAVI")

Kernel Indices Computation

Use the computeKernel() method to compute the required kernel for kernel indices like the kNDVI! The kernel parameter receives the kernel to compute, while the params parameter receives a dictionary with the required parameters for the kernel computation (e.g., a, b and sigma for the RBF kernel).

import spyndex
import xarray as xr
import matplotlib.pyplot as plt
from rasterio import plot

# Open a dataset (in this case a xarray.DataArray)
snt = spyndex.datasets.open("sentinel")

# Scale the data (remember that the valid domain for reflectance is [0,1])
snt = snt / 10000

# Compute the kNDVI and the NDVI for comparison
idx = spyndex.computeIndex(
    index = ["NDVI","kNDVI"],
    params = {
        # Parameters required for NDVI
        "N": snt.sel(band = "B08"),
        "R": snt.sel(band = "B04"),
        # Parameters required for kNDVI
        "kNN" : 1.0,
        "kNR" : spyndex.computeKernel(
            kernel = "RBF",
            params = {
                "a": snt.sel(band = "B08"),
                "b": snt.sel(band = "B04"),
                "sigma": snt.sel(band = ["B08","B04"]).mean("band")
            }),
    }
)

# Plot the indices (and the RGB image for comparison)
fig, ax = plt.subplots(1,3,figsize = (15,15))
plot.show(snt.sel(band = ["B04","B03","B02"]).data / 0.3,ax = ax[0],title = "RGB")
plot.show(idx.sel(index = "NDVI").data,ax = ax[1],title = "NDVI")
plot.show(idx.sel(index = "kNDVI").data,ax = ax[2],title = "kNDVI")

A pandas.DataFrame? Sure!

No matter what kind of python object you're working with, it can be used with spyndex as long as it supports mathematical overloaded operators!

import spyndex
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt

# Open a dataset (in this case a pandas.DataFrame)
df = spyndex.datasets.open("spectral")

# Compute the desired spectral indices
idx = spyndex.computeIndex(
    index = ["NDVI","NDWI","NDBI"],
    params = {
        "N": df["SR_B5"],
        "R": df["SR_B4"],
        "G": df["SR_B3"],
        "S1": df["SR_B6"]
    }
)

# Add the land cover column to the result
idx["Land Cover"] = df["class"]

# Create a color palette for plotting
colors = ["#E33F62","#3FDDE3","#4CBA4B"]

# Plot a pairplot to check the indices behaviour
plt.figure(figsize = (15,15))
g = sns.PairGrid(idx,hue = "Land Cover",palette = sns.color_palette(colors))
g.map_lower(sns.scatterplot)
g.map_upper(sns.kdeplot,fill = True,alpha = .5)
g.map_diag(sns.kdeplot,fill = True)
g.add_legend()
plt.show()

Parallel Processing

Parallel processing is possible with spyndex and dask! You can use dask.array or dask.dataframe objects to compute spectral indices with spyndex! If you're using xarray, you can also define a chunk size and work in parallel!

import spyndex
import numpy as np
import dask.array as da

# Define the array shape
array_shape = (10000,10000)

# Define the chunk size
chunk_size = (1000,1000)

# Create a dask.array object
dask_array = da.array([
    da.random.normal(0.6,0.10,array_shape,chunks = chunk_size),
    da.random.normal(0.1,0.05,array_shape,chunks = chunk_size)
])

# "Compute" the desired spectral indices
idx = spyndex.computeIndex(
    index = ["NDVI","SAVI"],
    params = {
        "N": dask_array[0],
        "R": dask_array[1],
        "L": 0.5
    }
)

# Since dask works in lazy mode,
# you have to tell it that you want to compute the indices!
idx.compute()

Plotting Spectral Indices

All posible values of a spectral index can be visualized using spyndex.plot.heatmap()! This is a module that doesn't require data, just specify the index, the bands, and BOOM! Heatmap of all the possible values of the index!

import spyndex
import matplotlib.pyplot as plt
import seaborn as sns

# Define subplots grid
fig, ax = plt.subplots(1,2,figsize = (20,8))

# Plot the NDVI with the Red values on the x-axis and the NIR on the y-axis
ax[0].set_title("NDVI heatmap with default parameters")
spyndex.plot.heatmap("NDVI","R","N",ax = ax[0])

# Keywords arguments can be passed for sns.heatmap()
ax[1].set_title("NDVI heatmap with seaborn keywords arguments")
spyndex.plot.heatmap("NDVI","R","N",annot = True,cmap = "Spectral",ax = ax[1])

plt.show()

License

The project is licensed under the MIT license.

Contributing

Check the contributing page.

spyndex's Issues

[Suggestion] Pin requirement versions (specifically python-box)

Hello, I am the developer of python-box and see that it is a requirement in this repo and has not been version pinned.
I suggest that you pin it to the max known compatible version in your requirements.txt and/or setup.py file(s):

python-box[all]~=5.4  

Or without extra dependencies

python-box~=5.4

Using ~=5.0 (or any minor version) will lock it to the major version of 5 and minimum of minor version specified.
If you add a bugfix space for 5.4.0 it would lock it to the minor version 5.4.*.

The next major release of Box is right around the corner, and while it has many improvements,
I want to ensure you have a smooth transition by being able to test at your own leisure to ensure your standard user cases do not run into any issues. I am keeping track of major changes, so please check there as a quick overview of any differences.

To test new changes, try out the release candidate:

pip install python-box[all]~=6.0.0rc4

QST: Maturity level of spyndex

Hello,

I would like to use your library in eoreader, to replace my own way of computing spectral indices.
I see that in setup.py you still are in pre-alpha mode, but according to your code, documentation and README, you seems pretty well advanced.

So, should I wait an API stabilization ? Or am I good to go ? 😄

Add datasets module

Create a datasets module where users can load datasets with different types to try spyndex.

Add `omega`

Add omega parameter for MBWI. Default = 2.

Support index-specific constant defaults

Has there already been discussion and/or decisions about providing support for index specific constant defaults, e.g L=1.0 for EVI and L=0.5 for SAVI? I think having index specific defaults could make computing indices easier for users, and could avoid user errors of assuming the constant's default applies to all indices which use that constant.

Happy to discuss and to help with any related PR work, if you think this could be a potential enhancement!

issue in calculating some of the vegetation indices

Hi,
I have used this library to calculate some vegetation indices, but the "EVI", "GBNDVI", "GLI", "GRNDVI", "MSAVI", "MTVI2", and "VARI" could not calculate and I got this error:
MergeError: conflicting values for variable 'band' on objects to be combined. You can skip this check by specifying compat='override'.

Missing gamma parameter for ARVI index.

@davemlz is the gamma parameter a fixed constant or does it need to always be specified by the user ? It is not added in the constant class. Thanks for looking into this.

`globals` to `locals`

When using a global dictionary, avoid using the globals argument.

Something wrong with NDWI

Hello, i've been trying to use the computeIndex for NDWI but apart from all other indexes working well, NDWI has been presenting issues so i've tested every way to compute it correctly but it seems something's wrong.

The image below shows the test i've made using the same variables but computeIndex returning the wrong range of values:

Add `kwargs` to `computeIndex` and `computeKernel`

Add kwargs so users don't have to pass a dict if they don't want to.

Example:

spyndex.computeIndex("NDVI",N = 0.67,R = 0.12)

instead of:

spyndex.computeIndex("NDVI",{"N": 0.67,"R": 0.12})

Add plots module

Create a plots module where the user can visualize the behaviour of a spectral index value according to the change in the spectral inputs with anotated heatmaps.

Add `SpectralIndex` class

Add a SpectralIndex class that allows to compute indices directly from spyndex.indices.

Example:

spyndex.indices.NDVI.compute(args)

Error while running the toy example

I am trying to run the toy example and it gives me the error below.

Here is the example I am trying to run.

spyndex.plot.heatmap not working

Hi, I'm trying to replicate the plot heatmap example but it doesn't work

QST: Compute custom spectral indices

Hello,

Is it possible to compute custom indices that are not registered in Awesome Spectral Indices ?

My usecase is that I have maybe too specific indices that wouldn't be useful to the community.
Or indices using satellites not handled currently like WorldViews/PlanetScope with the Yellow band.

If not I would be happy to share them all 😄

`online` argument is not working

Check the _check_params: It is not catching the indices dict when using online=True

migrate from setup.py to pyproject.toml

Hey,

I noticed that this project use setup.py. But this format is not the preferred format anymore for a python project : https://peps.python.org/pep-0518/ .
I can open a pull request if you agree to migrate to pyproject.toml and it will work in the same way it worked before.

I also notice that there is no wheel, we can in a second step generate wheels.

Bastien

Default values for constants in spectral indices

Hello,

Would it be possible (if useful) to have default values for constants in specified index (ie. L for SAVI) ? 😃
I am trying to have the minimum required intervention from the user, so it would be helpful!

Extras require for `spyndex`

Hello,

The current dependencies management includes dev packages like earthengine-api. earthengine-api is heavy and users of spyndex don't necessary need it.
So it's better to mark this dependency as a dev dependency. You can for example use the extras_require key in setup.py file. It would looks like this :

import io
import os
import re

from setuptools import find_packages, setup


def read(filename):
    ...

setup(
    name="spyndex",
    version="0.3.0",
    [...]
    install_requires=[
        "dask>=2021.9.1",
        "matplotlib",
        "numpy",
        "pandas",
        "python-box>=6.0",
        "requests",
        "seaborn",
        "xarray",
    ],
  extras_require={
        "dev": ["earthengin-api", "eemont>=0.3.6"],
    },
)

You can then install the regular version of this package with pip install . or the dev version with pip install .[dev]

Thanks !

Separate bands and constants in `SpectralIndex` objects

Hi @davemlz,

I appreciate the work you have put into Awesome Spectral Indices and this package!

While working on this short guide for my students, I thought that it might be a good idea to separate bands and constants in SpectralIndex objects. Meaning that when someone calls spyndex.indices.EVI.bands, for example, it should not return ['g', 'N', 'R', 'C1', 'C2', 'B', 'L'] but only ['N', 'R', 'B']. The constants could be returned separately with spyndex.indices.EVI.constants.

I think it might otherwise lead to confusion, especially for inexperienced users. In the guide I've mentioned, I tried to clear this up by using list comprehension to separate them.

`spyndex` breaks with `earthengine-api==0.1.365`

I'm struggling to get spyndex to work on my machine.
The installation with pip install spyndex works like a charm, but when I run import spyndex in python, I invariably get errors relating to ee.

So I'm wondering what version of earthengine-api you're based on, as all the versions I've tried so far result in various errors. Here what I get with the latest (earthengine-api==0.1.365

Traceback (most recent call last):
  File "python\helpers\pydev\pydevconsole.py", line 364, in runcode
    coro = func()
  File "<input>", line 1, in <module>
  File "python\helpers\pydev\_pydev_bundle\pydev_import_hook.py", line 21, in do_import
    module = self._system_import(name, *args, **kwargs)
  File "venv\lib\site-packages\spyndex\__init__.py", line 8, in <module>
    from .axioms import bands, constants, indices
  File "python\helpers\pydev\_pydev_bundle\pydev_import_hook.py", line 21, in do_import
    module = self._system_import(name, *args, **kwargs)
  File "venv\lib\site-packages\spyndex\axioms.py", line 3, in <module>
    from .spyndex import computeIndex
  File "python\helpers\pydev\_pydev_bundle\pydev_import_hook.py", line 21, in do_import
    module = self._system_import(name, *args, **kwargs)
  File "venv\lib\site-packages\spyndex\spyndex.py", line 7, in <module>
    import ee
  File "python\helpers\pydev\_pydev_bundle\pydev_import_hook.py", line 21, in do_import
    module = self._system_import(name, *args, **kwargs)
  File "venv\lib\site-packages\ee\__init__.py", line 15, in <module>
    from ee import batch
  File "python\helpers\pydev\_pydev_bundle\pydev_import_hook.py", line 21, in do_import
    module = self._system_import(name, *args, **kwargs)
  File "venv\lib\site-packages\ee\batch.py", line 15, in <module>
    from ee import data
  File "python\helpers\pydev\_pydev_bundle\pydev_import_hook.py", line 21, in do_import
    module = self._system_import(name, *args, **kwargs)
  File "venv\lib\site-packages\ee\data.py", line 542, in <module>
    def listAssets(params: dict[str, Any]) -> dict[str, List[Any]]:
TypeError: 'type' object is not subscriptable

I'm working in Python 3.8.10.

Can't load the package in google colab

Hi, thank you for developing the package.
I tried to use it in fresh google colab session but it keeps giving me dask error.

The installation is successful but loading the package gives me this error:

thank you..

Add parallel computing support

Add parallel computing support: dask