microsoft / gltf-toolkit

A collection of tools for modifying and optimizing glTF assets

Microsoft glTF Toolkit

This project contains a collection of tools and libraries to modify and optimize glTF assets.

Additionally the repository includes a command line tool that uses these steps in sequence in order to convert a glTF 2.0 core asset for use in the Windows Mixed Reality home, following the published documentation. The latest release of the Windows Mixed Reality Asset converter is available on the releases tab.

Features

The current release includes code for:

• Packing PBR material textures using DirectXTex for use with the MSFT_packing_occlusionRoughnessMetallic and MSFT_packing_normalRoughnessMetallic extensions.
• Compressing textures as BC3, BC5 or BC7 and generate mip maps using DirectXTex for use with the MSFT_texture_dds extension.
• Removing KHR_materials_pbrSpecularGlossiness by converting material prameters to metallic-roughness.
• Mesh compression using KHR_draco_mesh_compression extension; this can only be used on 1809 and later and should only be used for assets that are transmitted over the network as load time is increased with compression.
• Merging multiple glTF assets into a asset with multiple levels of detail using the MSFT_lod extension.
• A command line tool that combines these components to create optimized glTF assets for the Windows Mixed Reality Home
• A UWP compatible Windows Runtime component to perform conversions between GLTF and GLB, as well as optimize assets for Windows Mixed Reality at runtime

Dependencies

This project consumes the following projects through NuGet packages:

• Microsoft.glTF.CPP, licensed under the MIT license
• DirectXTex, licensed under the MIT license
• RapidJSON, licensed under the MIT license
• Draco, licensed under Apache License 2.0

Building

This project can be built using Visual Studio 2017 Update 4 on Windows 10 Fall Creators Update (16299.0).

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

License

Copyright (c) Microsoft Corporation. All rights reserved.

Licensed under the MIT License.

gltf-toolkit's Issues

NDEBUG missing from Release configuration?

Hi,

Shouldn't NDEBUG be set for Release builds?

https://github.com/Microsoft/glTF-Toolkit/blob/ee37a890c2cdcc20656d2ff3517332a815279965/glTF-Toolkit/glTF-Toolkit.vcxproj#L123

We're getting artifacts in our output when running in software mode otherwise (there seem to be some issues using a WARP device)

Sample glTF files fail to load in Microsoft's own software

The files "glTF-Toolkit/glTF-Toolkit.Test/Resources/gltf/CubeAsset3D.gltf" and "glTF-Toolkit/glTF-Toolkit.Test/Resources/gltf/CubeWithLOD.gltf" fail to load in both Windows explorer preview pane and the Windows 10 3D Viewer program by Microsoft.

Link to documentation dead

The link provided to documentation is no longer valid

Pipeline features

Hi! Awesome toolkit you've started here, thanks for doing this. 🙂 Wanted to share some ideas on things that could make this more useful to WebVR devs, and so, (apologies in advance for all the feature requests!) here are some thoughts. In no particular order:

• Enable building source on Unix systems, or provide binaries. This would allow others to host this as a web UI tool for convenience, see discussion here.
• Option to merge meshes with shared materials. I see this issue all the time on models from various sources, and many WebVR devs don't realize it's an issue. I use obj-simplify to solve this in OBJ files, and a similar tool for glTF could be helpful.
• Option to strip specific attributes. For example many models include tangents by default, but environments like three.js and A-Frame currently ignore them, so it's just added weight.
• Option to merge multiple animated glTF assets into a single asset with multiple animations. If authoring tools begin to support multiple animations they may not be necessary, but currently virtually none do, and it's unclear to me how/if to handle that in Autodesk workflows.

Best,
Don

Redundant textures in output after specular gloss -> metal rough conversion

I noticed that after the conversion of the spec gloss textures to metal rough that they still are present in the output

Here's the sheets compared between 1.5.0 and 1.6.0 when running glTF-Toolkit/glTF-Toolkit.Test/Resources/gltf/WaterBottle/WaterBottle.gltf. The spec gloss textures are still present in the textures/images arrays in 1.6.0 (indices: 0, 1, 5, 6).

Need help with MSFT_LOD generation for Threejs

Hello,
With threejs, we use https://github.com/takahirox/three-gltf-extensions to successfully load GLTF models that already include MSFT_LOD.
(working GLTF model : https://github.com/takahirox/three-gltf-extensions/tree/main/examples/assets/gltf/Torus/glTF-lod)

But when we try making our own with WindowsMRAssetConverter, no luck doing so, with multiple scenarios:

case 1 : multiple levels are active at the same time and with wild scales
(red and green are 2 separate LODs)

Command used on Windows : .\scripts\WindowsMRAssetConverter.exe ./src/models/LOD0Arbre2.gltf -o ./dist/models/arbre.glb -lod ./src/models/LOD1Arbre2.gltf ./src/models/LOD2Arbre2.gltf ./src/models/LOD3Arbre2.gltf ./src/models/LOD4Arbre2.gltf -screen-coverage 0.5 0.45 0.25 0.05 0.001

case 2 : we get Vector errors

Can you help us ?
I can share the assets if needed for testing.

Cant work with GLTF

Hello,

I want to use the gltf-Toolkit in my projekt to convert gltf-Files to glb-Files for the Mixed Reality Viewer.
First I tried directly to use
var converted = await WindowsMRConversion.ConvertAssetForWindowsMR(tempGltfFile, outputFolder, 512, TexturePacking.OcclusionRoughnessMetallic);
but the result is always a empty ( 0 byte) glb file (without the "_converted" extention on the filename).

next I tried to first serialize the gltf-File to glb an then use the Conversion Methode.

StorageFile sourceGlbFile = await GLTFSerialization.PackGLTFAsync(gltfFile, ApplicationData.Current.LocalFolder, glbfileName);

Here I got direct an exeption "System.Runtime.InteropServices.COMException" in System.Private.CoreLib.ni.dll"

For my tests I use the BoomBox.gltf and Avocado.gltf from the Khronos Repod
https://github.com/KhronosGroup/glTF-Sample-Models/tree/master/sourceModels

Here my full Methode

` public async Task GLBConvertToWindowsMR()
{
const string gltfFileName = "Avocado.gltf";

        // Pack the gltf back into a glb file
        var gltfFile = await StorageFile.GetFileFromPathAsync(Path.Combine(ApplicationData.Current.LocalFolder.Path, gltfFileName));
        var glbfileName = Path.ChangeExtension(gltfFileName, ".glb");
        StorageFile sourceGlbFile = await GLTFSerialization.PackGLTFAsync(gltfFile, ApplicationData.Current.LocalFolder, glbfileName);


        StorageFile tempGlbFile = await CopyFileToTempFolderAsync(sourceGlbFile.Path);
        StorageFolder outputFolder = await CreateTemporaryOutputFolderAsync("Out_" + Path.GetFileNameWithoutExtension(gltfFileName));


        var converted = await WindowsMRConversion.ConvertAssetForWindowsMR(tempGlbFile, outputFolder, 512, TexturePacking.OcclusionRoughnessMetallic);

    }`

NormalRoughnessMetallic packing needs to renormalize normals and compute roughness from normal variance

Incorrect results are being produced from the NRM packing. The current algorithm does a simple value copy but to maintain comparative visual equality we need to renormalize the normal values and also compute roughness from normal variance.

Repro steps:
Input Normal texture:

Input Metallic Roughness texture:

Expected NRM texture:

Actual NRM texture:

Improve handling of duplicate resources

Given a set of gltf input LODs that have the same materials for each LOD:

1. the images get packed multiple times (once per GLTF file), even though they could be packed just once if they're identical
2. the merged GLTF json will contain redundant material information
3. the exported GLB will as a result embed the same data (textures/images etc) once per LOD level, whereas it should really all share the one set of materials/textures/images

Some possible solutions:

1. Add a flag to WindowsMRAssetConverter to specify whether the materials are shared between LODs, and if so, make the merger avoid adding new resources.
2. Detect duplicate resources in the merger by binary comparision, and then resolve/remap all necessary places in the resulting GLTF json. Note that this wouldn't fix the issue with doing packing once per GLTF input file. Pack the images/textures after merging, assuming the merge step has eliminated all duplicates.

Basic models from Pain3D: Can't place hologram

I have tried to follow the great post here:

https://mtaulty.com/2018/06/19/placing-3d-models-in-the-mixed-reality-home/

I also used the same tortoise model but even after conversion, the hololens is unable to place the model - even with the platform set to holographic.

Multiples of 4 Texture resizing flag for WinMRAssetConverter

Block compression expects textures to be multiples of 4 to compress correctly. Adding a step to resize textures automatically and toggle this behavior on and off in the converter would provide a fallback when assets aren't authored with this requirement in mind.

-replace-textures flag does not work

When enabled the -replace-textures flag should remove all original textures and references from the optimized file leaving only DDS textures in the final .glb. The flag currently no-ops resulting in bloated files.

Windows MR Converter should report file not found errors

Using the tool this am. I thought it was broken, but it I just had a typo on the file name that I was entering. Would it be possible to report file not file not found or similar text.

Project does not build in VS 2019

The project does not build in Visual Studio 2019. I added the /Wv:18 option to the compiler command line and replaced all instances of "std::experimental::filesystem" with "std::filesystem" and that eliminated some errors but more remain and I am giving up.

LOD merge with tangents fails

LOD merge causes the tangents from the highest LOD to be used where it really should use the tangents from the appropriate LOD instead.

I think all that's needed is a line in GLTFLODUtils.cpp::AddGLTFNodeLOD:
...
AddIndexOffset(primitive.tangentsAccessorId, accessorOffset);
...

crash

Crash on GLTFDocument document = DeserializeJson(*stream);

Unhandled exception at 0x76A308F2 in WindowsMRAssetConverter.exe: Microsoft C++ exception: Microsoft::glTF::InvalidGLTFException at memory location 0x00AFC2F4. occurred

This when loading a gltf file

WindowsMRAssetConverter Could not parse accessor 0

Hello,

I am trying to use the MSFT_LOD extension for LOD with gltf files and came to this toolkit. I've downloaded the WindowsMRAssetConverter.exe. On running the command I'm getting the following output:

WindowsMRAssetConverter T2H.glb -o T2.glb -lod T2L.glb T1L.glb -screen-coverage 0.5 0.2 0.01

This will generate an asset compatible with Desktop (version 1709+)

Loading input document: T2H.glb...
Could not parse accessor 0
The member bufferView was not found

I had a look at this documentation (https://docs.microsoft.com/en-us/windows/mixed-reality/creating-3d-models-for-use-in-the-windows-mixed-reality-home) but couldn't figure out the issue.

Is it possible to run the converter from some non-Windows system as well and is there a documentation for it? If not, I could help in creating one with a little guidance.

WindowsMRAsset Converter -replace-textures flag causes conversion to fail

When converting an asset with the -replace-textures flag set the converter silently fails during the compression step. Removing the flag causes the same asset to convert successfully.

Need help with creating draco mesh straight from gltf-sdk

Hi,

I've been trying to use the toolkit with the gltf-SDK to generate a .glb file with Draco mesh compression. (without the file being saved by the SDK first.)

I keep getting the error Buffer.uri was not specified

If I try the .gltf file there is heap corruption at

glTF-Toolkit/glTF-Toolkit/src/GLTFMeshCompressionUtils.cpp

Line 250 in e478643

auto bufferView = builder->AddBufferView(buffer.data(), buffer.size());

This is what buffer.data() prints for a buffer of any size :
DRACO☻☻☺☺

Any help would be appreciated, thanks!

Packing crashes if the packed textures don't have the same size

For example, if you're packing ORM and the Occlusion and MetallicRoughness textures have different sizes.

Existing mesh serializing

Hi,
Do you have any documentation on how to create a gltf/glb mesh with more than 1 triangle and probably how to convert an existing mesh (I have mine in STL format) with facets and nodes correspondences.
Thanks!

Texture compression fails if GPU not found (Release only)

When using the 'Release' target, the glTF-Toolkit will throw an error "No Direct3D hardware device found" on trying to compress textures. In 'Debug' mode, a software fallback is used instead.

"Cannot read the magic number"

Trying to compress the file attached, I'm getting the "Cannot read the magic number" message and no results.
Thorus.zip
Does it mean compression is applicable only on gltf files?
Also, can I create Draco compressed glb with MSFT_lods extension?

Tests actually execute DDS compression, which is slow and unnecessary

Tests should use a mock of DirectXTex to return compressed textures immediately instead of slowly compressing them on the test machines. As a stopgap, we should disable slow tests on AppVeyor / CI.

This repo is missing important files

There are important files that Microsoft projects should all have that are not present in this repository. A pull request has been opened to add the missing file(s). When the pr is merged this issue will be closed automatically.

Microsoft teams can learn more about this effort and share feedback within the open source guidance available internally.

Merge this pull request

We couldn't load the selected model, Please try agian later

After conversion for the holographic plattform with the following command without any options:

• WindowsMRAssetConverter.exe original.glb -o ConvertedFile.glb -platform holographic

I get the aboved error, any sugesstion is appreciated!

(Btw, beofore conversion the error is "Sorry, this model is not optimized for Windows Mixed Reality")