Comments (7)
Looks like the network expects 2250 edges (1500 faces) [or fewer?]. Segmentation works great once I decimate the mesh to the target density.
from meshcnn.
Awesome! Thanks for letting me know. Yeah, I trained that network on 2250 edges, so deviating from that too much probably won't work well.
It is possible to retrain the network with more edges, but you should probably modify it a bit (the pooling resolution). Also it is possible to segment the original mesh from the lower resolution version. I have some plans to add code to do this :)
from meshcnn.
@obisker I am curious -- where is this mesh from? It looks like it may have been scanned or something, which is pretty cool.
from meshcnn.
That's right, the mesh is from a total body capture system.
An additional observation about the accuracy is that the currently trained network does not like mesh boundary edges. Presumably, because it was trained with watertight meshes it does not seem to generalize to edges which fail to collapse as an impl limitation as opposed to a learned collapse.
def __pool_edge(self, mesh, edge_id, mask, edge_groups):
if self.has_boundaries(mesh, edge_id):
return False
from meshcnn.
Cool.
The mesh that you sent is indeed watertight manifold. Are you referring to a different mesh you did not send?
So the network pooling can technically handle meshes with boundaries (although, I don't think human seg dataset has boundaries). If the edge is a boundary, we do not collapse it (a "boundary-preserving edge-collapse"). So it will just move to the next edge in the queue.
It is possible to synthetically add boundaries in the humans training data (like "break" the mesh) and re-train, to see if that helps with generalizing better with the boundaries on your mesh (didn't try it!).
I think the best option is to try to close the hole in your mesh / meshes. Have you tried using this: https://github.com/hjwdzh/Manifold ? Out of curiosity, if these are human meshes, why are there boundaries ?
from meshcnn.
In a typical 3D capture system, achieving complete coverage of every visible body area can be prohibitive either in number of cameras or time (if using sweeping scanner). Small holes in armpits, ears, bottoms of feet, and around hair are quite common. Automatically filling these holes is the typical solution (what I did on the mesh I linked), but requires additional processing time. The other scenario with edges is partial body capture - suppose you’ve only captured the front of the body.
This type of dataset: http://www.dais.unive.it/~shrec2016/
I believe that your model should be able to cope with holes or missing portions if you can figure out how to do pooling on the boundaries and padded edges. For example, if I pass in a half body with 1125 edges, the code will pad an additional 1125 edges. Now correct me if I am wrong, during pooling it will not collapse any of the 1125 padded edges and will also not be able to collapse any of the more than 100 boundary edges (in the valid 1125 half body set). Instead, it will skip the padded and boundary edges and collapse edges out of the remaining pool of 1000 edges. In my tests, things bomb out trying to meet the 1800-1350-600 edge pooling reductions... basically the queue doesn't have enough valid edges to collapse in order to meet the target edge counts.
You can see in the attached image how the boundary edges persist through the pooling stages (green marks boundary edges).
At the very least, boundary edges are displacing more informative (higher weighted) edges from surviving to the bottom of the U in the network.
from meshcnn.
Nice, it sounds like an interesting application.
So about the padded edges, they are place holders which are removed in the pooling layers. Take your example, a mesh with 1125 edges which is padded to 2250 edges - and pooled to: 1800, 1350, 600 target edges. In the first and second pooling layers (1800 and 1350) the pooling will remove the padded edges. Only in the last pooling layer (to 600) will real edges be removed. But you are correct, if there are too many boundary edges, it is possible that there will be no more valid edges to collapse, and the target edge count (of 600) will not be met.
One option is to retrain the network on different pooling resolutions: download the human seg data, and just modify the flag which is currently --pool_res 1800 1350 600
to something like: --pool_res 1800 1350 800
. For example, you can check what the smallest number of edges the network can collapse your data to (if it is 800, so you can make that the last pooling layer target number of edges).
About pooling boundary edges, it requires a bit of extra checks, since we don't want to pool boundary edges which would make the mesh non-manifold (for example, have a dangling edge). I will see what I can do about adding it to the mesh pooling (we will do it anyway at some point), but it might take some time
Let me know if the re-train works 😊
from meshcnn.
Related Issues (20)
- I'm trying to apply your model to my project, but i found out that I cannot create the *.eseg and *.seseg formats! HOT 2
- What is the purposes of soft_labels in segmentation data?
- How to input a mesh file to know the classification result? HOT 2
- IndexError: index out of range
- AssertionError
- What is the relationship between your meshes and original coseg meshes?
- Segmentation of objects with different characteristic
- assert no zero face area
- Does the class "Mesh" have an attribute about face?
- Undestanding .eseg file
- Proper TensorBoard Usage
- How to prepare data for a custom dataset?
- cannot install pytorch=1.2.0 HOT 1
- How to save a segmented (colored) mesh in .obj format?
- IndexError: index out of range
- Using my own obj files for training
- The problem when I run meshcnn classification HOT 1
- Problems with date preprocessing.
- help in figuring out how human_seg dataset was created
- Cannot load pre-trained weights
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from meshcnn.