Why not use Keras or Tensorflow instead of PyTorch? about redner

[shyamalschandra]

Why use C++? Why not Python? Okay, how long should I wait before I skim through the code and implement something similar? For the physically-based rendering being an computational expensive problem, couldn't you just parameter sweep and then just have a LUT or DHT? Right? Colab is perfect if you have a data store on the Amazon datastore (as you mentioned that you use lambda functions) that you can access through an API service functionless call? By the way, could you follow the scientific method and include your training and testing data for a small images and compare the performance with the SOTA in the same arena along with the gold-standard, "perfect" ray tracing of the object and number of cm^3 off of the "real" thing? Just a thought! Thanks for the prompt reply!

[shyamalschandra]

Also, is there a Knuthian or Tarjanian-like algorithm in pseudo code like CLRS textbook but without all the abstraction? Is that possible in the future instead of commenting the code line-by-line so I can get the zest as I just looked through the code and the paper along with the demo videos?

[BachiLi]

We use PyTorch because it's the one we are more familiar with. It shouldn't be too hard to port the interface to Tensorflow, since most of the C++ functions do not depend on any deep learning frameworks and only takes pointers to the arrays. Once we have the Tensorflow interface, you should be able to use this in Keras through the lambda layer.
Physically-based rendering is an inherently computational expensive problem, so if your scene has billions of triangles with complex occlusion and difficult lighting, it is going to take a while to render. We will work hard to make the renderer more efficient though.
Colab can be tricky since you would need to compile the C++ code and dependencies (Embree, Optix) on the server.

[BachiLi]

After a year of development, I am happy to announce that we have Tensorflow support AND you can use redner on Google Colab now (more examples will come later)! Also Keras is part of Tensorflow now. So all the requests are fulfilled!

Rendering in general is a long-studied problem in the field of computer science. It is only a little bit younger than perceptron! (Ivan Sutherland built sketchpad at MIT at around 1962, while the perceptron algorithm was invented at 1958.) People have studied the acceleration of rendering with tons of different tricks since then, and it is still an active research topic. If you are interested in working on this topic with a "Knuthian" style reference, I recommend pbrt, which is written in a literate programming style. Note that pbrt only touches a very thin surface of rendering. For a full treatment, you will have to read research papers, unfortunately. SIGGRAPH is a good place to find them.

Over the course of development since redner is released, I realized that one of the biggest obstacles of practical adoption of redner in research software is the gap of knowledge between graphics and vision/ML researchers (and some misunderstanding due to the way this paper is presented in related works. redner doesn't need any training data nor did it make any approximation, and it handles primary visibility just fine and is not slower than any rasterizers in theory). In the future, I plan to have more detailed and hands-on tutorial and examples, so that people can use redner more easily without having intensive knowledge of physically-based rendering.

[shyamalschandra]

Thank you, @BachiLi !