metaslam / gpr_competition

Dataset for MetaSLAM Challenge

Table of Contents

• 1. Overview
  • System Structure
• 2. Software
  • MrExplorer
  • AutoMerge
  • BioSLAM
• 3. Multi-Agent Exploration
  • Run MrExplorer
  • FakeMerge
• 4. Multi-Agent Localization
  • Visualization
• 5. Lifelong Localization
  • Configure Datasets
• 6. TODO List

1. Overview

In traditional Multi-Agent Exploration (MAE) methods, all agents start in the same place and start with a shared coordinate system; and once the relative transformation missed due to localization failures, the MAE system will be very fregial and easily crash. This framework provide a novel Autonomous Exploration paradigm, all the agents can start in the darkness, without knowing anything about where they are, where each other are, and what the environment looks like. As the MAE system explorer individually, MetaUnion helps to find associated overlaps among different agents and solve the above three challenges gradually; and most importantly, MetaUnion can incrementally update the feature learning ability for new observed areas. In a summary, our key contributions includes:

• Multi-Agent Localiztion without Initial Transformation;

• Multi-Agent Hierarchical Exploration;

• Incremental Lifelong Localization.

System Structure

MetaUnion mainly contains three key modules within the framework: AutoMerge, MrExplorer, and BioSLAM. In the begining, each agent is randomly assigned at a new envionrment, and has neither information about other agents' localization, nor place recognition ability for the new areas, and conduct single-agent exploration individually. When the trajectries of different agents have overlaps, AutoMerge module can automatically detect the data association between agents, and estimate the relative positions for them. Based on the relative transformation, MrExplorer can divide the agent into different groups based on their connections, and apply hierarichal exploration for individual agent. Parallel with the above procedure, BioSLAM constructs a dual-memory system to memorize the large-scale and long-duration place features. As the MAS system explorer to bigger area, their cooperation ability and place recognition ability are improved gradually.

2. Software

To quickly install all the necessary softwares for MetaUnion, please following the instructions,

git clone --recurse-submodules -j8 https://github.com/MetaSLAM/MetaUnion.git
# On Server
sh scripts/install_server.sh
## On Client
sh scripts/install_client.sh
source ~/.bashrc

Please keep in mind, the scripts will automatically generate PATH into the bashrc, and you can use

METASLAM=/home/maxtom/codespace/MetaUnion
source /home/maxtom/codespace/MetaUnion/stack/devel/setup.bash
# on client
alias enclient='sh /home/maxtom/codespace/MetaUnion/scripts/start_client.sh'
# on server
alias enserver='sh /home/maxtom/codespace/MetaUnion/scripts/start_server.sh'
# general command
alias cdw='cd /home/maxtom/codespace/MetaUnion'
alias killmeta='sh /home/maxtom/codespace/MetaUnion/scripts/killall.sh'

In the following sections, we will investigate the details of each module.

MrExplorer

Since all agents begin from unknown darkness area within an given map, each agent are starting from a local exploration policy based on its observations. And based on the AutoMerge module, small group agents will find their relative transformations, then such agents are grouped into one set, and the single agent exploration policy are transfermed into multi-agent exploration policy. Gradually, all the agents are converted into one group. This hierarchical strategy help crowd agents quickly understand their coopeartion status with each others, and can balance the exploration efficiency and robustness without any initial position and relative transformation information.

Note	API: Input and Output Odometry for each agent Grouping status from AutoMerge Waypoints for each agent

AutoMerge

AutoMerge can provide an automatic data-association for agents without any knowledge about initial position and each others. When agents upload the observations to the cloud, the agents can detect their associated overlaps (if exists), and send back the grouping results among different agent groups.

Note	API: Input and Output Odometry from individual agents SubMaps from individual agents Grouping Indexes Relative Transformations

Note	TODO List Onine SubMap Generation Onine Feature Evaluation Onine Cache Checking Onine Position Estimation

BioSLAM

We present BioSLAM, a lifelong SLAM framework for learning various new appearances incrementally and maintaining accurate place recognition for previously visited areas.Unlike humans, artificial neural networks suffer from catastrophic forgetting and may forget the previously visited areas when trained with new arrivals. For humans, researchers discover that there exists a memory replay mechanism in the brain to keep the neuron active for previous events. Inspired by this discovery, BioSLAM designs a gated generative replay to control the robot’s learning behavior based on the feedback rewards.Specifically, BioSLAM provides a novel dual-memory mechanism for maintenance: 1) a dynamic memory to efficiently learn new observations and 2) a static memory to balance new-old knowledge. When combined with a visual-/LiDAR- based SLAM system, the complete processing pipeline can help the agent incrementally update the place recognition ability, robust to the increasing complexity of long-term place recognition.

Note	API: Input and Output Dual Memory Assistance Long-short Memorization Regular Updated Models

Note	TODO List Online Feature Updating Strategy Selection Group Waypoints Estimation

3. Multi-Agent Exploration

Run MrExplorer

cdw
enserver #on server
enclient #on client

Note	TODO List Change the name of the project Integrate the configuration of the client into the config file Debug the global planning Integrate the multi-master into this

FakeMerge

git clone https://github.com/MetaSLAM/fakemerge.git fakemerge
cd fakemerge
catkin_make
source devel/setup.bash
roslaunch fakemerge transformPose.launch

4. Multi-Agent Localization

DATA:
    OFFLINE_LENGTH: 100 # Set for desire testing length

Use OFFLINE_LENGTH to set the length for each agent.

python src/offline_merging.py

In the current global_rough_align step, we will use spectral clustering method to divide agents into different groups based on their connections.

Visualization

Outside the docker, subscibe /global_map with world frame with rviz. Different un-merged maps will be visualized along the z-axis (index*30).

5. Lifelong Localization

See the demo tutorial in the jupyter and python version.

Configure Datasets

Download Pittsburgh datasets for online map merging. https://drive.google.com/drive/folders/19AK8jc6yZpKN6Ub_ILGJaceZixpbwKcV?usp=sharing

Download pre-trained models and set path

cd data && sh download.sh
echo "export ROS_IP='172.17.0.1' " >> ~/.bashrc
echo "export MASTER_IP='172.17.0.2' " >> ~/.bashrc
echo "export ROS_MASTER_URI=http://$MASTER_IP:11311/ " >> ~/.bashrc
echo "export BAG_PATH='$PATH_TO_PITT_ROS_BAGS' " >> ~/.bashrc
source ~/.bashrc

And the following difference matrix in the data/results/. Then set ROS_IP to enable communication between docker and host computer. Finally, export dataset path, and replace PATH_TO_PITT_ROS_BAGS to the Pittusbrugh rosbags.

6. TODO List

gpr_competition's Issues

Cannot download from Baidu

Hi,

I'm currently trying to download the dataset in Australia - I cannot register an account for Baidu or navigate the interface, and the dropbox link still appears to be broken. Is there a timeline on when this issue will be fixed?

Thank you,

Joshua

Error in visualize cloud

I try to use the pose in npy file to visualize database and query clouds. In round 1, the paired point clouds are successfully displayed in picture above. However, in round 2, the point clouds show a clear deviation in picture below. I guess there is a mismatch between pose and point cloud

Question about meter/pixel information for Visual Terrain Relative Navigation Dataset(Helicopter dataset)

Thank you for sharing great dataset.
Now I'm trying to recover approximate metric unit translation from consequtive two images using Homography.
For example, if translation btw two images(query 00001.png, query 00004.png) is (10pixel, 30pixel), i want to recover approximate metric unit translation using meter/pixel scale information.

So, I would like to request meter/pixel information for the duration of the flight path if that information exists. (for both aerial / satellite caes.)
Thank you in advance.

Submission

Thank for this dataset.

We have a pipeline and are able to achieve good results on the validation set. However, our method does not result in a descriptor for every point cloud instead for a given point pair of point clouds we directly return the distance between them in the latent space.

As I understand the current submission format does not support this, can you please provide a few instructions to upload results of our new method

Still can not download the dataset Pittsburgh City-scale Dataset and CMU Cross-Domain Dataset

Dear authors, I still can not download the dataset, please provide downloadable resources, thank you very much.

AutoMerge

Cannot download the dataset

Dear authors, please provide downloadable resources, thank you very much.

The PR curve

@maxtomCMU Hi,the work is excellent!But I wan to know the package gpr.evaluation how to get PR curve,the code only can get the Topk-recall.Thanks!

Pitts Helicopter Dataset

When will datasets be available for download？

CMU Pittsburgh Testing Datsets

When will datasets be available for download？

I am willing to take part in the competition. Could you tell me when will datasets be available for download？
Thanks！

Terrain UAV Datasets

Dataset download link not working

The links for the dataset download are not working.

how to download dataset for round 3?

CMU UAV Datasets

Cant Download UAV Round 2 Training Set

Somehow I can't download the training dataset for UAV Round 2. I use this dropbox link. I could click the download, it progressed, but at the end of the download, it was suddenly failing. I tried multiple times using Firefox and Chrome. I have no problem with the validation and testing dataset.

Inconsistent datasets description?

I've noticed that on competion site (https://sites.google.com/andrew.cmu.edu/gpr-competition/datasets) you've described "Pitts Helicopter Dataset" as a dataset for second task, but in repository's readme you've described another dataset "CMU Cross-Domain Dataset". More of that, in dataset_description.md there is again description of "Pitts Helicopter Dataset".

Could you clarify what information is relevant? Thank you!

UAV round 2 dataset - gt_matches.csv in train subset is missing

There is no gt_matches.csv for Train subset of UAV dataset for round 2:

$ ls Train/
query.csv  query_images  reference.csv  reference_images

$ ls Val/
gt_matches.csv  query.csv  query_images  reference.csv  reference_images

Can you provide this file? Thank you!

CMU Day/Night Testing Datasets

Unable to extract the dataset

I am unable to extract the files TRAIN.tar.gz.00 , TRAIN.tar.gz.01 , TRAIN.tar.gz.02 for the 3D-3D LiDAR scan dataset given in this link, can you please help me with this.

Thank You

Dataset lacks the absolute location

Thanks for your excellent dataset.
I want to do some experiments on your excellent dataset for Place Recognition but I have a question: the submap in the dataset doesn't have absolute location because only relative poses of submaps are given, long-term information of multiple trajectories can't be recognized. So I ask that if you can release the GPS or absolute location of every submaps in every track? Thank you!

About get_recall

Why does "true_neighbors" use indice of queries_feature instead of reference_feature? Isn't "retrieved_indices" from "reference_feature"?

GPR_Competition/src/gpr/evaluation/recall.py

Lines 39 to 40 in ad19377

	for i in range(true_threshold, len(queries_feature) - true_threshold):
	true_neighbors = i + np.arange(-true_threshold, true_threshold + 1)