gokriznastic / hybridsn

A keras based implementation of Hybrid-Spectral-Net as in IEEE GRSL paper "HybridSN: Exploring 3D-2D CNN Feature Hierarchy for Hyperspectral Image Classification".

Hybrid-Spectral-Net for Hyperspectral Image Classification.

PyTorch Implimentation of HybridSN

PyTorch version of the HybridSN is available: https://github.com/Pancakerr/HybridSN

Description

The HybridSN is spectral-spatial 3D-CNN followed by spatial 2D-CNN. The 3D-CNN facilitates the joint spatial-spectral feature representation from a stack of spectral bands. The 2D-CNN on top of the 3D-CNN further learns more abstract level spatial representation.

Model

Fig: Proposed HybridSpectralNet (HybridSN) Model with 3D and 2D convolutions for hyperspectral image (HSI) classification.

Prerequisites

• Anaconda 2.7
• Tensorflow 1.3
• Keras 2.0

Results

Indian Pines (IP) dataset

Fig.2 The IN dataset classification result (Overall Accuracy 99.81%) of Hybrid-SN using 30% samples for training. (a) False color image. (b) Ground truth labels. (c) Classification map. (d) Class legend.

University of Pavia (UP) dataset

Fig.3 The UP dataset classification result (Overall Accuracy 99.99%) of Hybrid-SN using 30% samples for training. (a) False color image. (b) Ground truth labels. (c) Classification map. (d) Class legend.

Salinas Scene (SS) dataset

Fig.4 The UP dataset classification result (Overall Accuracy 100%) of Hybrid-SN using 30% samples for training. (a) False color image. (b) Ground truth labels. (c) Classification map.

Detailed results can be found in the Supplementary Material

Citation

If you use HybridSN and A2S2K-ResNet and HSI-Survey code in your research, we would appreciate a citation to both the original paper:

@article{roy2019hybridsn,
    	title={HybridSN: Exploring 3D-2D CNN Feature Hierarchy for Hyperspectral Image Classification},
	author={Roy, Swalpa Kumar and Krishna, Gopal and Dubey, Shiv Ram and Chaudhuri, Bidyut B},
	journal={IEEE Geoscience and Remote Sensing Letters},
	volume={17},
	no.={2},
	pp.={277-281},
	year={2020}
	}
@article{roy2020attention,
	title={Attention-based adaptive spectral-spatial kernel resnet for hyperspectral image classification},
	author={Swalpa Kumar Roy, and Suvojit Manna, and Tiecheng Song, and Lorenzo Bruzzone},
	journal={IEEE Transactions on Geoscience and Remote Sensing},
	volume={59},
	no.={9},
	pp.={7831-7843},
	year={2021},
	publisher={IEEE}
	}	
@article{ahmad2021hyperspectral,
	title={Hyperspectral Image Classification--Traditional to Deep Models: A Survey for Future Prospects},
	author={Muhammad Ahmad, and Sidrah Shabbir, and Swalpa Kumar Roy, and Danfeng Hong, and Xin Wu, and Jing Yao, and Adil Mehmood Khan,
	and Manuel Mazzara, and Salvatore Distefano, and Jocelyn Chanussot},
	journal={IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing},
	year={2022},
	volume={15},
	pages={968-999},
	doi={10.1109/JSTARS.2021.3133021},
	publisher={IEEE}
	}

Acknowledgement

Part of this code is from a implementation of Classification of HSI using CNN by Konstantinos Fokeas.

License

Copyright (c) 2019 Gopal Krishna. Released under the MIT License. See LICENSE for details.

hybridsn's Issues

PCA & Imagecube before train test split cause data leak.

Hi, thank you for sharing the code. That helps alot.
I applied the code on my own hyperspectral image, the test result is very good with almost 100%. However, the model doesn't predict well on other similar images. One thing I can think of is because of PCA on the whole image, and different images will cause different PCA loadings. However, I tried train the model without PCA, i.e. put all spectral in, the test result is less than 40% accuarcy rate (haven't figured out why). Another thing that will compromise the model integrity is create image cube before train test split. That will mix training pixels and testing pixels all together. For any 25x25 image cube, it will definatly contain training and testing spectra.

history = model.fit(x=Xtrain, y=ytrain, batch_size=256, epochs=100, callbacks=callbacks_list)

crashing google colab

Solved

,,,,

Salinas dataset 25x25 windowsize problem

Could you share salinas code for image cubes function.I'm doing in colab it is crashing with that function

ValueError: Error when checking : expected input_1 to have shape (25, 25, 30, 1) but got array with shape (25, 24, 30, 1)

Dataset is IP, the windowSize is set to 25, when i valiation this model, the error is reported . The error is lie in line 64：prediction = (model.predict(X_test_image))

Error while create model

Negative dimension size caused by subtracting 3 from 1 for 'conv3d_6/convolution' (op: 'Conv3D') with input shapes: [?,1,1,20,16], [3,3,3,16,32].

difference between windowSize=5 and windowSize=25

Why you have taken different windowSize=5 in function createImageCubes() and windowSize=25. What is the difference it would make in program. I am new to this field. please explain..

How to show every class map?

Hi. I want to show every class color map like the picture, how I can do it? Thanks.

Code for Multispectral data

@gokriznastic Can we use this code for multispectral data i.e., data of 5 channels ?