evenhax / awesome-scene-understanding

A resource repository for scene understanding

Awesome Scene Understanding

A resource repository for scene understanding, inspired by 3D-Machine-Learning.

Contributing

Please feel free to pull requests to add papers.

Table of Contents

• Workshops and Tutorials
• Survey
• Dataset
  • Realistic Datasets
  • Synthetic Datasets
• Holistic Scene Understanding
  • Perspective Image
  • Panoramic Image
• Room Layout Estimation
  • Perspective Image
  • Panoramic Image
• Floorplan
• Primitive Detection
  • Junction
  • Line Segment
  • Wireframe
  • Plane
  • Cuboid
  • Others
• Object Reconstruction

Workshops and Tutorials

• Holistic 3D Reconstruction: Learning to Reconstruct Holistic 3D Structures from Sensorial Data (ICCV'19) [Webpage] [Resources]

• Holistic Scene Structures for 3D Vision (ECCV'20) [Webpage]

Survey

• RGBD Datasets: Past, Present and Future (CVPRW'16) [Project] [Paper]

• Indoor Scene Understanding in 2.5/3D for Autonomous Agents: A Survey (IEEE Access'19) [Paper]

Dataset

Realistic Dataset

• [NYUv2] Indoor Segmentation and Support Inference from RGBD Images (ECCV'12) [Project] [Paper]

• SUN3D: A Database of Big Spaces Reconstructed using SfM and Object Labels (ICCV'13) [Project] [Paper]

• SUN RGB-D: A RGB-D Scene Understanding Benchmark Suite (CVPR'15) [Project] [Paper]

• SceneNN: a Scene Meshes Dataset with aNNotations (3DV'16) [Project] [Paper]

• ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes (CVPR'17) [Project] [Paper]

• [2D-3D-S] Joint 2D-3D-Semantic Data for Indoor Scene Understanding (CoRR'17) [Project] [Paper]

• Matterport3D: Learning from RGB-D Data in Indoor Environments (3DV'17) [Project] [Paper] [Code]

• The Replica Dataset: A Digital Replica of Indoor Spaces (CoRR'19) [Paper] [Code]

• 3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camera (ICCV'19) [Project] [Paper]

Synthetic Dataset

• The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes (CVPR'16) [Project] [Paper]

• SceneNet: Understanding Real World Indoor Scenes With Synthetic Data (CVPR'16) [Project] [Paper]

• [SUNCG] Semantic Scene Completion from a Single Depth Image (CVPR'17) [Paper]

• SceneNet RGB-D: Can 5M Synthetic Images Beat Generic ImageNet Pre-training on Indoor Segmentation? (ICCV'17) [Project] [Paper]

• InteriorNet: Mega-scale Multi-sensor Photo-realistic Indoor Scenes Dataset (BMVC'18) [Project] [Paper]

• Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling (CoRR'19) [Project] [Paper] [Code]

Holistic Scene Understanding

Perspective Image

• Thinking Inside the Box: Using Appearance Models and Context Based on Room Geometry (ECCV'10) [Paper]

• Estimating Spatial Layout of Rooms using Volumetric Reasoning about Objects and Surfaces (NeurIPS'10) [Paper]

• Efficient Structured Prediction for 3D Indoor Scene Understanding (CVPR'12) [Paper]

• Efficient Exact Inference for 3D Indoor Scene Understanding (ECCV'12) [Paper]

• Recovering Free Space of Indoor Scenes from a Single Image (CVPR'12) [Paper]

• Understanding Indoor Scenes using 3D Geometric Phrases (CVPR'13) [Paper]

• Scene Parsing by Integrating Function, Geometry and Appearance Models (CVPR'13) [Project] [Paper]

• Emptying, Refurnishing, and Relighting Indoor Spaces (SIGGRAPH Asia'16) [Project] [Paper]

• DeepContext: Context-Encoding Neural Pathways for 3D Holistic Scene Understanding (ICCV'17) [Project] [Paper]

• Im2CAD (CVPR'18) [Project] [Paper]

• Factoring Shape, Pose, and Layout from the 2D Image of a 3D Scene (CVPR'18) [Project] [Paper] [Code]

• Holistic 3D Scene Parsing and Reconstruction from a Single RGB Image (ECCV'18) [Project] [Paper] [Code]

• Cooperative Holistic Scene Understanding: Unifying 3D Object, Layout, and Camera Pose Estimation (NeurIPS'18) [Project] [Paper] [Code]

• Complete 3D Scene Parsing from an RGBD Image (IJCV'18) [Paper]

• Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Image (CVPR'20) [Paper]

Panoramic Image

• PanoContext: A Whole-room 3D Context Model for Panoramic Scene Understanding (ECCV'14) [Project] [Paper]

• Pano2CAD: Room Layout From A Single Panorama Image (WACV'17) [Paper]

• Automatic 3D Indoor Scene Modeling from Single Panorama (CVPR'18) [Paper]

Room Layout Estimation

Perspective Image

• Recovering the Spatial Layout of Cluttered Rooms (ICCV'09) [Paper]

• Estimating the 3D Layout of Indoor Scenes and its Clutter from Depth Sensors (ICCV'13) [Project] [Paper]

• Box In the Box: Joint 3D Layout and Object Reasoning from Single Images (CVPR'13) [Paper]

• Rent3D: Floor-Plan Priors for Monocular Layout Estimation (CVPR'15) [Project] [Paper]

• Learning Informative Edge Maps for Indoor Scene Layout Prediction (ICCV'15) [Homepage] [Paper]

• DeLay: Robust Spatial Layout Estimation for Cluttered Indoor Scenes (CVPR'16) [Paper]

• A Coarse-to-Fine Indoor Layout Estimation (CFILE) Method (ACCV'16) [Paper]

• Physics Inspired Optimization on Semantic Transfer Features: An Alternative Method for Room Layout Estimation (CVPR'17) [Project] [Paper]

• RoomNet: End-to-End Room Layout Estimation (ICCV'17) [Paper]

• Thinking Outside the Box: Generation of Unconstrained 3D Room Layouts (ACCV'18) [Paper]

• Flat2Layout: Flat Representation for Estimating Layout of General Room Types (CoRR'19) [Paper]

• Smart Hypothesis Generation for Efficient and Robust Room Layout Estimation (WACV'20) [Paper]

• General 3D Room Layout from a Single View by Render-and-Compare (CoRR'19) [Paper]

Panoramic Image

• Efficient 3D Room Shape Recovery From a Single Panorama (CVPR'16) [Project] [Paper] [Code]

• LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image (CVPR'18) [Paper] [Code]

• Layouts from Panoramic Images with Geometry and Deep Learning (IROS'18) [Paper] [Code]

• HorizonNet: Learning Room Layout with 1D Representation and Pano Stretch Data Augmentation (CVPR'19) [Paper] [Code]

• DuLa-Net: A Dual-Projection Network for Estimating Room Layouts from a Single RGB Panorama (CVPR'19) [Project] [Paper]

• Corners for Layout: End-to-End Layout Recovery from 360 Images (CoRR'19) [Project] [Paper] [Code]

• 3D Manhattan Room Layout Reconstruction from a Single 360 Image (CoRR'19) [Paper] [Code]

Floorplan

• Raster-to-Vector: Revisiting Floorplan Transformation (ICCV'17) [Project] [Paper] [Code]

• FloorNet: A unified framework for floorplan reconstruction from 3D scans (ECCV'18) [Project] [Paper] [Code]

• Floorplan Priors for Joint Camera Pose and Room Layout Estimation (CoRR'18) [Paper]

• HouseExpo: A Large-scale 2D Indoor Layout Dataset for Learning-based Algorithms on Mobile Robots (CoRR'19) [Paper] [Code]

• CubiCasa5K: A Dataset and an Improved Multi-Task Model for Floorplan Image Analysis (CoRR'19) [Paper] [Code]

• DeepPerimeter: Indoor Boundary Estimation from Posed Monocular Sequences (CoRR'19) [Paper]

• Floor-SP: Inverse CAD for Floorplans by Sequential Room-wise Shortest Path (ICCV'19) [Project] [Paper] [Code]

• Deep Floor Plan Recognition using a Multi-task Network with Room-boundary-Guided Attention (ICCV'19) [Project] [Paper]

• Floorplan-Jigsaw: Jointly Estimating Scene Layout and Aligning Partial Scans (ICCV'19) [Project] [Paper]

• Scan2Plan: Efficient Floorplan Generation from 3D Scans of Indoor Scenes (CoRR'20) [Paper]

Primitive Detection

Junction

• Manhattan Junction Catalogue for Spatial Reasoning of Indoor Scenes (CVPR'13) [Paper]

Line Segment

• LSD: A Fast Line Segment Detector with a False Detection Control (TPAMI'10) [Paper]

• Lifting 3D Manhattan Lines from a Single Image (ICCV'15) [Paper]

• MCMLSD: A Dynamic Programming Approach to Line Segment Detection (CVPR'17) [Paper]

• A Novel Linelet-Based Representation for Line Segment Detection (TPAMI'18) [Paper]

• Novel Single View Constraints for Manhattan 3D Line Reconstruction (3DV'18) [Paper]

• Learning Attraction Field Representation for Robust Line Segment Detection (CVPR'19) [Paper] [Code]

Wireframe

• Learning to Parse Wireframes in Images of Man-Made Environments (CVPR'18) [Paper] [Code]

• PPGNet: Learning Point-Pair Graph for Line Segment Detection (CVPR'19) [Paper] [Code]

• End-to-End Wireframe Parsing (ICCV'19) [Paper] [Code]

• Learning to Reconstruct 3D Manhattan Wireframes from a Single Image (ICCV'19) [Paper]

• Holistically-Attracted Wireframe Parsing (CVPR'20) [Paper]

Plane

• PlaneNet: Piece-wise Planar Reconstruction from a Single RGB Image (CVPR'18) [Project] [Paper] [Code]

• LS3D: Single-View Gestalt 3D Surface Reconstruction from Manhattan Line Segment (ACCV'18) [Paper]

• Recovering 3D Planes from a Single Image via Convolutional Neural Networks (ECCV'18) [Paper] [Code]

• PlaneRCNN: 3D Plane Detection and Reconstruction from a Single Image (CVPR'19) [Project] [Paper] [Code]

• Single-Image Piece-wise Planar 3D Reconstruction via Associative Embedding (CVPR'19) [Paper] [Code]

Cuboid

• Localizing 3D Cuboids in Single-view Images (NIPS'12) [Paper]

• A Linear Approach to Matching Cuboids in RGBD Images (CVPR'13) [Project] [Paper]

• Deep Cuboid Detection: Beyond 2D Bounding Boxes (CoRR'16) [Paper]

Others

• Detection and Matching of Rectilinear Structures (CVPR'08) [Paper]

• Bottom-Up/Top-Down Image Parsing with Attribute Grammar (TPAMI'09) [Paper]

Object Reconstruction

• Symmetry Hierarchy of Man-Made Objects (Computer Graphics Forum'11) [Project] [Paper]

• GRASS: Generative Recursive Autoencoders for Shape Structures (SIGGRAPH'17) [Project] [Paper] [Code]

• Learning Shape Abstractions by Assembling Volumetric Primitives (CVPR'17) [Project] [Paper] [Code]

• 3D-PRNN: Generating Shape Primitives with Recurrent Neural Networks (ICCV'17) [Paper] [Code]

• Im2Struct: Recovering 3D Shape Structure from a Single RGB Image（CVPR'18) [Paper] [Code]

• 3D Interpreter Networks for Viewer-Centered Wireframe Modeling [Project] [Paper]

• Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboids (CVPR'19) [Paper] [Code]

• PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding (CVPR'19) [Project] [Paper] [Code] [PartNet-Symh]

• PartNet: A Recursive Part Decomposition Network for Fine-grained and Hierarchical Shape Segmentation (CVPR'19) [Project] [Paper] [Code]

• StructureNet : Hierarchical Graph Networks for 3D Shape Generation (SIGGRAPH Asia'19) [Project] [Paper] [Code]

• Learning Adaptive Hierarchical Cuboid Abstractions of 3D Shape Collections (SIGGRAPH Asia'19) [Project] [Paper] [Code]