Support for bitmap texture materials about three.js-pathtracing-renderer

[erichlof]

Hi @marcusstenbeck

I'm glad to announce that the path tracer now supports texture materials! Check out the new Billiard Table Demo

I would like to add support for normal maps and emissive maps, I just need to study those more and learn how to efficiently trace rays on them inside the path tracing loop. Also, as mentioned near the end of this project's Readme.md, my implementation is still a little hacky and requires some setup on each and every texture that you want to load and apply. In the future it will be nice to have a dedicated, generalized material texture loader.

I am still trying to figure out how to incorporate the concept of a three.js 'material' which has it's own color, texture, etc., and my path tracing 'material' which has additional components such as reflectivity, roughness, index of refraction for clear-coat plastic and glass, water, etc. So it's not a simple of a transition as I had hoped. But it's still on the TODO list. In the meantime, enjoy using image textures on materials!
-Erich

[ericek111]

Wow, that looks really good! Thanks for this awesome pathtracer, it truly is an amazing piece of software. So fast, effective and beautiful, it's hard to believe it works realtime in a browser. I hope you'll keep the updates up!

[CNCaddict]

Really fantastic work!! With a little bit of deep learning based denoising I think full realtime path tracing with 99% of reference quality is within reach with current hardware. Erich, have you looked into this? I think Octane Render is working in a similar direction right now.
https://twitter.com/OTOY/status/870507332555251712
http://cvc.ucsb.edu/graphics/Papers/SIGGRAPH2015_LBF/

[erichlof]

Hi @marcusstenbeck

Sorry for the late reply - I live in Houston, TX, the city that was hit by hurricane Harvey. Luckily I didn't have damage but it's been a crazy couple of weeks! In answer to your question, yes I would like to have support for textures as materials (or material layers) instead of only colored flat materials as I have now. Picking up the pace after the storm, I actually have been experimenting with using jpg and png images as textures, but in a slightly different capacity. I am currently working with a 256x256 rgb noise texture in png format to help create and raymarch/pathtrace non-geometric shapes such as mountain terrain, water, clouds, etc. It's going well now finally, after many trials and errors all summer long.

Soon I will be adding a more realistic cloud rendering demo to the Ocean scene, as proof of concept. Then maybe I can work on a small demo using textures for materials as you suggested. I would also like to use textures for terrain building, so I might delve into that first. But what you suggest would be a useful feature indeed.

Thanks for the nice comment and suggestion!
-Erich

[marcusstenbeck]

Wow, I hope your luck keeps you safe!

[erichlof]

Hello @ericek111
Thank you for the kind words! I'm glad you are enjoying it. Yes part of the reason I chose Javascript, three.js, and WebGL as the platform for this project is that it would run on the browser, and as we all know, browsers are everywhere! Some other path tracers out there use native OpenGL on a single target platform, usually Windows desktop with NVIDIA cards (access to CUDA). Although these can get higher FPS than mine (maybe 100+ fps instead of WebGL capped at 60 fps), they can only be experienced and enjoyed on that higher-end setup.

Since I only have a lowly 2014 laptop with integrated graphics card, and a slightly newer Samsung S7 cell phone (which admittedly has a really good GPU) I figured I might was well make the path tracer run on a browser so it could be experienced everywhere, even on the go!

I will keep the updates coming: I am currently experimenting ray tracing a heightfield loaded in as a texture image. It works but initially I was getting 18-20 fps. I want at least 30 before I'm happy with it. I'll keep you posted, thanks again!

[erichlof]

Hi @CNCaddict
Thank you for the compliment! Wow that OTOY de-noiser looks promising, and thanks for the link to the machine-learning paper - I will check it out right after I type this!

It's interesting that you brought up Octane renderer because it evolved from, and kind of branched off from Brigade 1 and Brigade 2. Brigade 2 is my inspiration for this entire project, because when I first saw videos like this I had to pick my jaw up off of the floor!

Brigade 1 used to be open source, but when it got acquired by OTOY, unfortunately they took all references and snapshots of the source code off the internet and made it proprietary. I think they used that technology to build and enhance the GPU preview window inside the Octane Renderer. As mentioned in the Readme.md, without the Brigade source code to go on, I searched around and found resources like Kevin Beason's smallpt (which was not real time, but digestible in only 100 lines of C++ source code!) so at least I could start somewhere. Also sites like ShaderToy are great for learning how to handle this tracing stuff on the GPU and make it go as fast and tight as possible in the constraints of WebGL. Having the source code to Brigade would have saved me many hours though!

I'm also a little disappointed that OTOY is really pushing their Octane renderer in the direction of Film/TV rendering, because there are already a lot of software choices if you are a team doing offline production rendering. Octane has the ability to go real time on a desktop with a 1080 card, and possibly into VR with multiple cards, bringing us closer to a true Holodeck. But they are not even considering that from the tone of their latest presentation from Jules Urbach (CEO) at Unity's Unite conference.

Having said that, they still own Brigade and are 'working' on Brigade 3, which IS meant to be consumed for realtime entertainment by the masses on high-end PC's with possible multiple NVIDIA graphics cards. But rather than wait around for them, I decided to make my path tracer run as fast as possible within the constraints of browsers, javascript, and WebGL, as a kind of proof-of-concept that yes, this is possible on mass consumer hardware, and yes, this is a much more realistic experience than rasterizing projected polygons with so many hacks, which is what the game industry has been stuck with doing for the past 30 years. I am hopeful, as you are, that in the near future we will see realtime path tracing of scenes with detailed high-poly triangle meshes (the industry standard) that everyone can enjoy - we viewers deserve better! :-)

[CNCaddict]

I think it may be in your interest to contact Jacco Bikker and see what he is up to!? I'm not sure how involved he was in the sale of Brigade and if there are any non competes, but it would be a shame if he never works on path tracing again. Food for thought!

I do worry that OTOY is trying to do too many things and brigade has moved to the backburner. It may come back now though with the ability to clean up noise using neural networks. In my mind it's a no brainer.

Also, I check github at least once a week to see if you've posted any new updates. I get really excited every time I've seen that you've added a new feature!

https://jbikker.github.io/index.html#about
https://jbikker.github.io/literature/The%20Brigade%20Renderer%20-%20A%20Path%20Tracer%20for%20Real-Time%20Games%20-%202012.pdf
https://www.youtube.com/watch?v=Znr1JJLI5uY

[erichlof]

@CNCaddict ,
That YouTube video of the robot lab in his Aurora path tracer is awesome! As you know, for that level of scene detail and triangle count, it is necessary to have some sort of acceleration structure, usually a BVH or KD-tree. I hope to catapult my tracer into that realm once three.js has built in support for WebGL 2.0 which allows bit manipulations in the shaders. This lets us traverse really quickly through a binary tree using bit masks and manipulations, greatly speeding up triangle mesh rendering. As soon as mrdoob and his three.js colleagues get WebGL 2.0 support going, I will start updating my tracer to handle larger triangle meshes. I'm looking into LBVH which can rebuild the entire data acceleration structure every frame in milliseconds on the gpu!

Yes I will contact Jacco about his recent activity and interests in the path tracing realm - thanks for the suggestion! He has been active as recently as a year ago. Looks like he is experimenting with different light transport algorithms. His recent efforts look promising, although his projects use CUDA and native OpenGL, so users must have that pc setup to run those: https://youtu.be/W87zWTuF0k0

So honored and happy that you regularly check my github repo for updates! I have 2 updates I'm ironing out at the moment, a difficult lighting classic scene from Eric Veach's famous Metropolis Light Transport paper, although instead of his MLT (which I don't understand, ha), I just put my bi-directional algo to the ultimate test, with pretty fair initial results! Also I mentioned elsewhere I'm working on terrain rendering from a heightmap texture, which is fun and interesting. Lastly, be on the lookout for the LBVH when WebGL 2.0 support lands in three.js, but that might be farther down the road since I have to wait for that to happen first.
Thanks again!

[CNCaddict]

So excited about the new updates. You mentioned earlier that you are running on an old laptop with integrated graphics. I have a few quad core PC's with slightly aged Nvidia graphics lying around that might be an upgrade from your current setup. I can send one over if you are interested?

[erichlof]

@CNCaddict
Oh wow thank you for offering. I'm ok for the moment with the current setup, it forces me to stay creative on making the code run fast and sometimes thinking slightly outside the box. However I might contact you when I get the LBVH going because I don't know if WebGL alone (even though I love working in that environment) will be able to do all that at a decent frame rate. I might want to give CUDA a try, but that would be way down the line, because I haven't studied CUDA programming in depth yet.
Thanks again! Also, I'll let you know if I get in touch with Jacco and what he's up to :-)