pengyu-gis / remote-sensing-chatgpt

Chat with RS-ChatGPT and get the remote sensing interpretation results and the response!

RS-ChatGPT: Solving Remote Sensing Tasks with ChatGPT and Visual Models

Introduction

Remote Sensing ChatGPT(RS-ChatGPT) is an open source tool for solving remote sensing tasks with ChatGPT in an interactive way.ChatGPT acts as an expert to response to users' linguistic resquests based on the input remote sensing image. It supports various interpretation tasks that are trained on remote sensing datasets. To help ChatGPT better understand remote sensing knowledge, remote sensing image captioning is set as cue to help ChatGPT understand remote sensing image.We will keep updating RS-CHatGPT~ Please comments with Issues or send me a email if you have any suggestions! Thanks in advance!

Generally, RS-ChatGPT includes four steps in implementation:

• Prompt Template Generation
• Task Planning
• Task Execution
• Response Genration

Our article is now available in arXiv! See you in IGARSS204, Greece!
My personal website is also avalable:www.poleguo98.top

Updates

• TODO1: An online website is comming! Let's use RS-ChatGPT Online!
  
• TODO2: What about asking RS-CHatGPT to choose different models within the given task?
  
• TODO3: Put all the models available in Huggingface
  

2024.01.30:

• Remote Sensing GPT is updated, with bugs fixed, more readable, and more scalable~
  

2023.09.19:

• Remote Sensing GPT now supports GPT-4 and multi-round chating! An interactive interface is now available(see interface.py)~
  

2023.08.23:

• Initial release

The code

Requirements

Please Refer to requirements.txt

Usage

->Clone the repository:git clone https://github.com/HaonanGuo/Remote-Sensing-ChatGPT
->Download the below models and place them in the checkpoints folder
->Run RSChatGPT-shell.py

Supported Function

Function	Description	Method	Pretrain Dataset	Model Weights
Image Captioning	Describe the remote sensing image	BLIP	BLIP Dataset	weight(github)
Scene Classification	Classify the type of scene	ResNet	AID Dataset	weight(Google) weight(Baidu)
Object Detection	Detect RS object from image	YOLO v5	DOTA	weight(Google) weight(Baidu)
Instance Segmentation	Extract Instance Mask of certain object	SwinTransformer+UperNet	iSAID	weight(Google) weight(Baidu)
Landuse Classification	Extract Pixel-wise Landuse Classification	HRNet	LoveDA	weight(Google) weight(Baidu)
Object Counting	Count the number of certain object in an image	YOLO v5	DOTA	Same as Object Detection
Edge Detection	Extract edge of remote sensing image	Canny	None	None

More funtions to be updated~

Citation

Please cite the repo if you use the data or code in this repo.

@article{RS ChatGPT,
	title = {Remote Sensing ChatGPT: Solving Remote Sensing Tasks with ChatGPT and Visual Models},
	shorttitle = {Remote Sensing ChatGPT},
	doi = {10.48550/ARXIV.2401.09083},
	author = {Guo, Haonan and Su, Xin and Wu, Chen and Du, Bo and Zhang, Liangpei and Li, Deren},
	year = {2024},
}

Acknowledgments

• Visual ChatGPT
• YOLOv5
• BLIP

Help

Remote Sensing ChatGPT is an open source project that welcome any contribution and feedback. Please contact us with: [email protected]