Animating on MacOS but not on iOS, maybe still "Armature/" related about gltf2usd

[wave-electron]

When you do have chance later... take a look at this example by Apple on Skel animation with weights.

soccerBall.usda.txt

[wave-electron]

Can you reload that zip again. I don't think you gave it enough time to upload

[wave-electron]

@ox I have now successfully changed several doing @enzyme69 suggestion. The animation after the edit in all cases work in IO12. I'll try out your patch above and see what results I get. Cheers.

I created another animation this morning which I posted on my fusionar.app. gallery. Tt could benefit from the Blend shape support. How are you progressing on that front?

[wave-electron]

We might be able to request example from Pixar in the usd-interest group. Those guys seem fairly accommodating to requests. In respect to my animations with weights. I’m creating the animation in Cheetah 3D, exporting as a collada, importing into Blender to export as glTF 2. Then running the gltf2usd command line tool. Finally manually editing the file to fix the joint name. Then using usd_converter tool for final conversion. That’s it!

[wave-electron]

@ox I tried out your patch above on the bananaMan.usdz above. I had to make the following edits manually to get it to work. If i didn't do this the animation was all distorted and broken!

Are you able to re-jig a patch to replace Armature/Armature_Joint with skel?

The edits in red marker

Edited usda - animates in IOS 12 correctly

[enzyme69]

Aha... I got it, I am an idiot! 👻

[wave-electron]

@enzyme69

I was having a look at this myself... and while the animation runs in xcode beta 6, it wouldn't start in safari ios12 (beta 12). Did it run for you after making those changes to Armature/?

[enzyme69]

@wave-electron Yes it works! Ensure the "first name of the joint or bone" matches those for root. That's another gotcha, but @kcoley your Python GLTF 2 USD converter is so handy!

[wave-electron]

@enzyme69 Jimmy I'm not following what you mean by this. I've tried replacing "Armature/" references to say "Armature_Bone/" in the same spots as your screenshot above... but the animation still does not run in IOS12 beta 12. It works however in xcode 10 beta 6. Can you be more explicit with what you mean by.... ensure the first name of the joint or bone matches those for root?

Yes it works! Ensure the "first name of the joint or bone" matches those for root. That's another gotcha, but @kcoley your Python GLTF 2 USD converter is so handy!

[wave-electron]

When I make those changes the animation will work in xcode beta6, but it previews in IOS12 Beta 12 but the animation does not run. I've uploaded the files for you to check if we're both on the same page. Note: the usdz may work on MAC, I'm still running on High Sierra OS, so cannot check that.
quicky.zip

[wave-electron]

@enzyme69 Jimmy did you get a chance to look at this? I'd love some feedback.

[enzyme69]

@wave-electron You cannot have same name bone within the hierarchy. On MacOS Mojave, the USDZ animation works fine, but on iOS, it's more strict.

Let me try to fix yours, I've been using Sublime to quickly replace same name bones ... one sec..

Believe it or not, I have been asked the same questions on this weird hierarchy name issue, no "Armature/" ... :D

[wave-electron]

ok. Thats what I thought it was... but I've been finding it hard to fix that hierachy from within the usda file. Do you fix that inside Blender? If so could you screen shot how it looks. Thanks

[enzyme69]

I am fixing it using Sublime Text.

Basically "Find All" : "Armature/", whilst all highlighted, I deleted it.

quicky_fixed.zip

[wave-electron]

Jimmy got it. It works for me now on ios 12 as well. Appreciate that. I think that will help me fix many other bone animations I've been having problems with. :)

[enzyme69]

It works, yay!! Great, alright can start making some animation now :D

[wave-electron]

@enzyme69 @ox @kcoley Shouldn't this issue still be open?

Unfortunately, haven't had time myself to explore a coding solution, but it really requires one.

The only way I know to currently build a bone animation using gltf2usd tool required me to manually edit the usda file to make the animation run on IOS12 beta 12. Whilst the animation does work in Xcode without the edit.... I would think most folks would ultimately require it just work on IOS as well.

I've also found nearly every bone animation I've tried suffers from this joint name problem, except a couple of models I found in KhronosGroup Samples glTF

[kcoley]

Yeah I am not sure how this was closed (might have been auto-closed by one of the other merged PRs). I'll reopen this.

[ox]

@wave-electron yes this issue should still open. I'm digging through it right now. So far, the only real problem I see in @enzyme69's model is the lack of a root skeletal node (without '/' in it's path) in the "wrong" screenshot. Looking at the difference between the "quicky" models, I noticed that in quickyX, the "Armature_Bone" skeleton has a root joint "Armature_Bone" (not to be confused with the skeleton name), whereas the broken one has an invalid root joint "Armature/Armature_Bone".

I think the solution is to remove the root skeleton name from the hierarchy building code. I made the following changes in _convert_skin_to_usd and it did give a root node, which is a start:

-        skeleton = UsdSkel.Skeleton.Define(self.stage, '{0}/{1}'.format(parent_path, gltf_root_node_name))
+        skeleton_path = '{0}/skel'.format(parent_path)
+        skeleton = UsdSkel.Skeleton.Define(self.stage, skeleton_path)
+        print("skeleton path %s, root %s" % (skeleton_path, gltf_root_node_name))
         skel_binding_api_skel_root.CreateSkeletonRel().AddTarget(skeleton.GetPath())
         if len(bind_matrices) > 0:
             skeleton.CreateBindTransformsAttr().Set(bind_matrices)
 
-        joint_paths = []
+        joint_paths = [Sdf.Path(gltf_root_node_name)]

[ox]

@wave-electron BlendShape support was merged however I haven’t gotten animations to play. To be honest I haven’t seen any BlendShape USD models out on the Internet. I think that part of the schema is very new so we might be on the frontier on this one. I have successfully generated USD models from gltf models which has an animation channel with “path”: “weights”, but iOS refuses to render it.

What exactly are you doing to get the skinning to work?

[enzyme69]

Exciting development in regards to blendshapes!!

But you are right, it does not work as USDZ yet, maybe on Apple side. I don't know how Maya/Houdini/Katana handles this USD however, does it simply load and can render for Renderman or something?

Is there example by Apple, a USDZ with blendshapes that work, apart from Memoji?

OUTPUT.zip

[enzyme69]

Much simpler animation to test this "shapekeys" or "blendshapes".
simplevirus.zip

The GLTF with blendshapes seems to work, I tested:
https://gltf-viewer.donmccurdy.com

[wave-electron]

@ox @enzyme69 @kcoley From Pixar via usd-interest group 1st August 2018

On Blend Shapes
UsdSkel provides a fully featured encoding for Skeletons, geometry bindings, and Joint Animations for USD (with blend shape support anticipated in the coming year)

From Pixar via usd-interest group 8th Sept 2018

I'm not sure what iOS 12 supports, but I can tell you we haven't implemented Maya blendshape import/export yet. The schema has been written as part of UsdSkel, and we did some verification that the behavior matches Maya's conception of blendshapes, but it's still on our to-do list for get import/export working. Also, UsdImaging doesn't visualize blenshapes yet, but that's true for UsdSkel as well at the moment. We're hoping to work on that soon.

[ox]

@wave-electron I’m away from my computers this weekend but I’m glad that we are narrowing down the issue. I have so far avoided wading into the skinning and skeleton building code in gltf2usd, as it’s the most intricate, so my patches are still just exploratory. Would it be possible for you to share the bananaman gltf model so I can build a fix against it?

It seems that first it’s important that the number of elements in all timesamples for each skeleton animation is the same as the number of tokens in that animstion’s joints array. Then that there is a root node and that it is the skeleton node? That last part doesn’t quite make sense since the UsdSkel docs have skeleton animations where the joints array doesn’t make any mention of the skeleton as the root node. Maybe extra intermediate joints are being computed and added? I think @kcoley is way more familiar with the Skelton code at this point.

[ox]

@wave-electron thanks for that model. I came across it earlier but I didn’t know USD then. Kind of bummed that blendshapes are in limbo. I was hoping to use it in place of collada files to play with face meshes, but I guess we have to wait for iOS 12 Gold?

[wave-electron]

Yeah its a pity. Anyway here's that BananaMan - with all the files you need to explore.
BananaMan.zip

[kcoley]

@ox

It seems that first it’s important that the number of elements in all timesamples for each skeleton animation is the same as the number of tokens in that animstion’s joints array.

I made some progress on fixing this in the refactorskeleton branch. I added logic to interpolate animation data based on a given time sample, which should fill in the missing frames. I am getting way smoother results, though some models are stuttery. I will double check this logic to make sure I did not miss something.

Then that there is a root node and that it is the skeleton node?

The skeleton root seems to be a parent to the joints in the joints array. This may be counted as the "true" root node of the joints, since it is possible to have multiple joints defined at the root level without a top level parent.

I

[ox]

@kcoley I interpreted the presence of the SkelRoot to just mean that there is a skeleton somewhere further down this stack. I’m still confused about why the skeleton name needs to be specified as the root joint even though the joints array is just to establish hierarchy using arbitrary but consistent naming. I really think it’s a mismatch of array lengths between the joints tokens and the timesample arrays. That should fix bone animation, but now I’m not so sure.

[kcoley]

@ox ah, I learned at SIGGRAPH that this is a bug in the iOS implementation of USD. I believe the convention is based on Maya's USD exporter, which names the skeleton root the same name as the root joint. Renaming the joints is a workaround to get it to animate on iOS until the implementation is updated.

[kcoley]

@ox

I really think it’s a mismatch of array lengths between the joints tokens and the timesample arrays. That should fix bone animation, but now I’m not so sure

Yes, this is correct. With the skeleton refactoring branch, the array lengths are now the same for all the joint tokens since the animation samples are now being interpolated based on sampling with the fps parameter. This seems to be working with the models from Sketchfab, though I am chasing down an animation hierarchy issue, which causes some parts of an animated model to be detached and floating in the wrong location. I plan to make a PR this weekend with the update so others can try it out,

[ox]

@kcoley awesome, can’t wait to see it. Thank you for putting in the time to refactor and clean!

[kcoley]

@ox no problem! I spent a good chunk of time working on this over the weekend, though I will probably need one more day or two to finish integrating all my changes. It's definitely a big refactor so it's probably safer to test out the PR once I create it to make sure everything is ironed out.

[kcoley]

I made a recent update on iOS to the next Beta (12.1) and it seems to resolve the root name issue. Is anyone else able to confirm?

[wave-electron]

@kcoley Yes, that seems to be the case.

[kcoley]

Just merged a fix for this (#94)