bruinxiong / transpr

TRANSPR: Transparency Ray-Accumulating Neural 3D Scene Point Renderer

3DV 2020

Project page   Paper

TRANSPR: Transparency Ray-Accumulating Neural 3D Scene Point Renderer
Maria Kolos*¹   Artem Sevastopolsky*^1,2   Victor Lempitsky^1,2
1Samsung AI Center   ¹Skolkovo Institute of Science and Technology

* indicates equal contribution

Code: Coming soon... (planned release month: November '20)

About

This is a PyTorch implementation of TRANSPR, a new method for realtime photo-realistic rendering of 3D scenes with semi-transparent parts and complex geometry. This method is an extension of Neural Point-Based Graphics (NPBG) which uses a raw point cloud as the geometric representation of a scene, and augments each point with a learnable neural descriptor that encodes local geometry and appearance. In our method, we expand the descriptor with a learned transparency value, use ray accumulation to account all points perceived by the camera. The points along the rays are fused into an opacity-aware 2D representation processed by the rendering network. Several scenes can be rendered in conjunction, opacity of objects can be edited, and the non-transparent objects can be combined with the introduced transparency. The repository extends NPBG with the additional features.