rpfey / activeinr

[ICCV 2023] Active Neural Mapping

Active Neural Mapping

Zike Yan, Haoxiang Yang, Hongbin Zha

           

Installation

Our environment has been tested on Ubuntu 18.04 (CUDA 10.8 with RTX2080Ti) & 20.04 (CUDA 11.8 with RTX4080).

Clone the repo and create conda environment

git clone --recurse-submodules [email protected]:ZikeYan/activeINR.git && cd activeINR

# create conda env
conda env create -f environment.yml
conda activate activeINR

Install pytorch by following the instructions. For torch 2.0.1 with CUDA version 11.8:

pip install torch==2.0.1+cu118 torchvision==0.15.2+cu118 torchaudio==2.0.2+cu118 --extra-index-url https://download.pytorch.org/whl/cu118

pip install -e .

Preparation

Simulated environment

Habitat-lab and habitat-sim need to be installed for simulation. We use v0.1.7 (git checkout tags/v0.1.7) and install the habitat-sim with the flag --with-cuda.

pip install -e habitat-lab
python habitat-sim/setup.py install --with-cuda

Data

To run the active mapping in the simulated environment, Gibson dataset for Habitat-sim and the Matterport3D dataset should be downloaded. The directory for the downloaded data should be specified in the config file of activeINR/train/configs/gibson.json via the key root.

Trained models

We adopt the DDPPO for point-goal navigation. All pre-trained models can be found here. The model should be placed in activeINR/local_policy_models and specified in the config file of activeINR/train/configs/gibson.json via the key planner.

Run

To run Active Neural Mapping on the Denmark scene of Gibson dataset, run the following command.

python activeINR/train/vis_exploration.py --config activeINR/train/configs/mp3d_big.json --scene_id 2azQ1b91cZZ

The logs will be saved in the ./activeINR/train/logs/ folder with actions, mesh file, checkpoints of the neural map, etc.

The mesh quality and the exploration coverage can be evaluated through the following manuscript:

python activeINR/eval/eval_action.py --config activeINR/train/configs/mp3d_big.json --scene_id 2azQ1b91cZZ

python eval/eval_mesh.py

TODO

The repo is still under construction, thanks for your patience.

• Running with a live camera in ROS.
• BALD implementation.
• Loss landscape visualization.
• [ ]

Acknowledgement

Our code is partially based on iSDF and UPEN. We thank the authors for making these codes publicly available.

Citation

@inproceedings{Yan2023iccv,
  title={Active Neural Mapping},
  author={Yan, Zike and Yang, Haoxiang and Zha, Hongbin},
  booktitle={Intl. Conf. on Computer Vision (ICCV)},
  year={2023}
}