natmlx / videokit Goto Github PK

View Code? Open in Web Editor NEW

65.0 5.0 12.0 32.58 MB

Low-code, cross-platform user-generated content for Unity Engine. Register at https://hub.natml.ai

Home Page: https://docs.videokit.ai/videokit

License: Apache License 2.0

C# 72.66% C 25.71% Objective-C 1.62%

computer-vision natml unity3d user-generated-content video-editing video-effects video-filter video-recording

videokit's People

Contributors

Stargazers

Watchers

Forkers

kiranjodhani carscan alinccc safdher dwtcourses game-challenge aboutunity dalong1225 shigeoyamada imanirajdoost reddefine2018 test-jar-labs

videokit's Issues

iOS build Parent is called on an empty path

Hi,
When building for iOS with NatML Videokit and Meet Segmentation, I receive below error, please help.

Unhandled exception: System.InvalidOperationException: Parent is called on an empty path

   at NiceIO.NPath.get_Parent() in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP.Common35/NiceIO.cs:line 144

   at NiceIO.NPath.EnsureParentDirectoryExists() in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP.Common35/NiceIO.cs:line 767

   at NiceIO.NPath.EnsureDirectoryExists(NPath append) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP.Common35/NiceIO.cs:line 762

   at NiceIO.NPath.EnsureParentDirectoryExists() in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP.Common35/NiceIO.cs:line 767

   at NiceIO.NPath.EnsureDirectoryExists(NPath append) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP.Common35/NiceIO.cs:line 762

   at NiceIO.NPath.EnsureParentDirectoryExists() in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP.Common35/NiceIO.cs:line 767

   at NiceIO.NPath.EnsureDirectoryExists(NPath append) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP.Common35/NiceIO.cs:line 762

   at NiceIO.NPath.EnsureParentDirectoryExists() in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP.Common35/NiceIO.cs:line 767

   at NiceIO.NPath.EnsureDirectoryExists(NPath append) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP.Common35/NiceIO.cs:line 762

   at NiceIO.NPath.EnsureParentDirectoryExists() in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP.Common35/NiceIO.cs:line 767

   at NiceIO.NPath.EnsureDirectoryExists(NPath append) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP.Common35/NiceIO.cs:line 762

   at NiceIO.NPath.EnsureParentDirectoryExists() in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP.Common35/NiceIO.cs:line 767

   at NiceIO.NPath.EnsureDirectoryExists(NPath append) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP.Common35/NiceIO.cs:line 762

   at NiceIO.NPath.EnsureParentDirectoryExists() in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP.Common35/NiceIO.cs:line 767

   at NiceIO.NPath.EnsureDirectoryExists(NPath append) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP.Common35/NiceIO.cs:line 762

   at NiceIO.NPath.EnsureParentDirectoryExists() in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP.Common35/NiceIO.cs:line 767

   at NiceIO.NPath.EnsureDirectoryExists(NPath append) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP.Common35/NiceIO.cs:line 762

   at NiceIO.NPath.EnsureParentDirectoryExists() in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP.Common35/NiceIO.cs:line 767

   at NiceIO.NPath.EnsureDirectoryExists(NPath append) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP.Common35/NiceIO.cs:line 762

   at NiceIO.NPath.EnsureParentDirectoryExists() in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP.Common35/NiceIO.cs:line 767

   at NiceIO.NPath.EnsureDirectoryExists(NPath append) in /Users/bokken/build/output/unity/il2cpp/Unity.IL2CPP.Common35/NiceIO.cs:line 762

   at il2cpp.Il2CppOptionParser.SetupOtherArguments(Il2CppCommandLineArguments il2CppCommandLineArguments, List`1 foundAssemblies) in /Users/bokken/build/output/unity/il2cpp/il2cpp/Il2CppOptionParser.cs:line 133

   at il2cpp.Il2CppOptionParser.ParseArguments(String[] args, Boolean& continueToRun, ExitCode& exitCode, RuntimePlatform& platform, Il2CppCommandLineArguments& il2cppCommandLineArguments, BuildingOptions& buildingOptions) in /Users/bokken/build/output/unity/il2cpp/il2cpp/Il2CppOptionParser.cs:line 67

   at il2cpp.Program.Run(String[] args, Boolean setInvariantCulture, Boolean throwExceptions) in /Users/bokken/build/output/unity/il2cpp/il2cpp/Program.cs:line 61

   at il2cpp.Program.Main(String[] args) in /Users/bokken/build/output/unity/il2cpp/il2cpp/Program.cs:line 36

Command PhaseScriptExecution failed with a nonzero exit code`

PrepareEncoder doesn't respect video format chosen.

It looks like preparencoder will always use MP4 even though webm might have been defined.

private static async Task PrepareEncoder () {
            // Check platform
            if (Application.platform == RuntimePlatform.WebGLPlayer)
                return;
            // Create recorder
            var width = 1280;
            var height = 720;
            var clock = new FixedIntervalClock(30);
            var recorder = new MP4Recorder(width, height, 30);
            // Commit empty frames
            var pixelBuffer = new byte[width * height * 4];
            for (var i = 0; i < 3; ++i)
                recorder.CommitFrame(pixelBuffer, clock.timestamp);
            // Finis and delete
            var path = await recorder.FinishWriting();
            File.Delete(path);
        }

WebGL crash with Meet VideoKit 0.0.10 - MediaDeviceQuery_DiscoverDevices

Hi,
In WebGL build sometimes (very often) when loading scene with NatML Meet Segmentation, there is a crash.
Please take a look in the Profiler screenshot - seems like MLModelData.OnEnable() drops FPS suddenly, and maybe this is a cause of crash? If not - could you suggest some solution for this? Is there a way to improve memory size for WebGL?
Or maybe it is directly connected with MediaDeviceQuery_DiscoverDevices?

This error occurs:
testtest.framework.js:3 Uncaught RuntimeError: memory access out of bounds at dlfree (testtest.wasm:0x9b73e) at MemoryManager::LowLevelFree(void*, unsigned long) (testtest.wasm:0x2c6f13) at UnityDefaultAllocator<LowLevelAllocator>::Deallocate(void*) (testtest.wasm:0x18a6591) at ThreadsafeLinearAllocator<false>::TryDeallocate(void*) (testtest.wasm:0x699528) at MemoryManager::Deallocate(void*, MemLabelId, char const*, int) (testtest.wasm:0x4a2737) at operator delete(void*) (testtest.wasm:0x9125e) at NDMediaDeviceGetName (testtest.wasm:0xf193bd) at NatDevice_Name_m7BE4A206300A690F90B92697C3E540277F177766 (testtest.wasm:0x7438fc) at AudioDevice__ctor_mE8A1C592C648F889936191E95E8E557A845B30E6 (testtest.wasm:0x117f137) at U3CDiscoverAudioDevicesU3Ed__20_MoveNext_m3947F1662EDDA93971217F8C7E67172E63FDE931 (testtest.wasm:0x1979979) at InterfaceFuncInvoker0<Toggle_tBF13F3EBA485E06826FD8A38F4B4C1380DF21A1F*>::Invoke(unsigned short, Il2CppClass*, Il2CppObject*) (testtest.wasm:0x918b5) at dynCall_iiii (testtest.wasm:0x1749226) at invoke_iiii (testtest.framework.js:3:525248) at List_1_AddEnumerable_mFD4E2919839A27C7CEACADF62222C63701302D02_gshared (testtest.wasm:0x1e81a36) at RuntimeInvoker_TrueVoid_t4861ACF8F4594C3437BB48B6E56783494B843915_VoidU2A_t104EAEFBD2D237A8C29618913DA9B4D99355E965(void (*)(), MethodInfo const*, void*, void**, void*) (testtest.wasm:0xdf43cc) at InvokerActionInvoker1<RtcStats_tB82CEAB4F971E9D092B96B90C0C3BFFB035D1BE8>::Invoke(void (*)(), MethodInfo const*, void*, RtcStats_tB82CEAB4F971E9D092B96B90C0C3BFFB035D1BE8) (testtest.wasm:0x9292a) at List_1_InsertRange_m729639E2C8D1AA7579AC78D259085921E7DC72F2_gshared (testtest.wasm:0xf5b303) at RuntimeInvoker_TrueVoid_t4861ACF8F4594C3437BB48B6E56783494B843915_CancellationToken_t51142D9C6D7C02D314DA34A6A7988C528992FFED_RuntimeObject(void (*)(), MethodInfo const*, void*, void**, void*) (testtest.wasm:0x1c3d9d) at InvokerActionInvoker2<int, String_t*>::Invoke(void (*)(), MethodInfo const*, void*, int, String_t*) (testtest.wasm:0xa0b5a) at List_1_AddRange_m44DED39FFEBC8F2ACF6162DB065B649E888B9C57_gshared (testtest.wasm:0xf5ba9a) at List_1_AddRange_m66D40F60D60CF3FA653432ABEBEBF703840689DB(List_1_t437B6C3879E969156A381BDC3C459CF809D39DDF*, Il2CppObject*, MethodInfo const*) (testtest.wasm:0xa11cd) at MediaDeviceQuery_DiscoverDevices_m0520738B6315FF77EFB342F12AB3CF55102140C5 (testtest.wasm:0xf450aa) at MediaDeviceQuery__ctor_mE2C86815B53C8771C6F5BD82319E56505C59FF82 (testtest.wasm:0x2c74bc) at U252EU252DU252DU252EU252EU252EU252DU252DU252DU252EU252DU252DU252DU252EU252DU252DU252DU252EU252EU252EU252EU252DU252DU252EU252DU252EU252DU252DU252EU252EU252EU252EU252EU252DU252EU252EU252EU252DU252EU252EU252EU252EU252EU252DU252DU252EU252E_U252EU252EU252DU252DU252DU252EU252DU252EU252DU252EU252DU252EU252EU252EU252DU252DU252EU252EU252EU252EU252DU252EU252EU252EU252DU252DU252EU252EU252EU252EU252DU252EU252DU252DU252DU252DU252EU252DU252DU252EU252DU252DU252EU252DU252DU252EU252E_mA165F6D2E8CE5BDA1F792073168AB4F5C78B81EB (testtest.wasm:0x682594) at NatMlMeetBackgroundRemover_SwitchCameraDeviceToTarget_mF727B58824BDA93FF90F16C8943B75A8A71D35A5 (testtest.wasm:0x186543) at HallwayViewModel_U252DU252DU252EU252EU252EU252EU252EU252DU252EU252EU252DU252DU252EU252DU252EU252DU252EU252EU252EU252DU252DU252DU252DU252EU252DU252EU252DU252DU252EU252EU252EU252DU252DU252DU252DU252EU252DU252EU252EU252DU252EU252DU252DU252DU252EU252DU252E_mBED46317D9DAB642EACBDF579B8CC8D473313E97 (testtest.wasm:0x783e11) at HallwayViewModel_U252EU252EU252EU252EU252EU252DU252DU252EU252DU252DU252DU252EU252DU252DU252DU252DU252EU252DU252DU252DU252DU252EU252EU252EU252EU252DU252EU252EU252EU252DU252DU252EU252EU252DU252EU252EU252EU252DU252DU252DU252EU252EU252DU252EU252EU252DU252E_mE7C7F9E1E9446C4ECB510F50DDA0C22E8FE59917 (testtest.wasm:0xfe70f4) at HallwayViewModel_Start_mD1BD1C85DC59E11053F9FD98C15301F462C6A8B8 (testtest.wasm:0x1f49417) at RuntimeInvoker_TrueVoid_t4861ACF8F4594C3437BB48B6E56783494B843915(void (*)(), MethodInfo const*, void*, void**, void*) (testtest.wasm:0x1c82f76) at il2cpp::vm::Runtime::InvokeWithThrow(MethodInfo const*, void*, void**) (testtest.wasm:0x40d8b2) at dynCall_iiii (testtest.wasm:0x1749226) at invoke_iiii (testtest.framework.js:3:525248) at il2cpp::vm::Runtime::Invoke(MethodInfo const*, void*, void**, Il2CppException**) (testtest.wasm:0xf56fe) at il2cpp_runtime_invoke (testtest.wasm:0xaef264) at scripting_method_invoke(ScriptingMethodPtr, ScriptingObjectPtr, ScriptingArguments&, ScriptingExceptionPtr*, bool) (testtest.wasm:0x3c948a) at ScriptingInvocation::Invoke(ScriptingExceptionPtr*, bool) (testtest.wasm:0x1c30a1) at ScriptingInvocation::Invoke(ScriptingExceptionPtr*) (testtest.wasm:0xf60f9) at MonoBehaviour::InvokeMethodOrCoroutineChecked(ScriptingMethodPtr, ScriptingObjectPtr) (testtest.wasm:0x7473aa) at MonoBehaviour::Start() (testtest.wasm:0x36b6c3) at MonoBehaviour::DelayedStartCall(Object*, void*) (testtest.wasm:0x1e2128e) at DelayedCallManager::Update(int) (testtest.wasm:0x2059dc9) at InitPlayerLoopCallbacks()::EarlyUpdateScriptRunDelayedStartupFrameRegistrator::Forward() (testtest.wasm:0x1f0da91) at ExecutePlayerLoop(NativePlayerLoopSystem*) (testtest.wasm:0x103f5a0) at ExecutePlayerLoop(NativePlayerLoopSystem*) (testtest.wasm:0x103f614) at MainLoop() (testtest.wasm:0xf50fda) at dynCall_v (testtest.wasm:0x1748a79) at browserIterationFunc (testtest.framework.js:3:257137) at callUserCallback (testtest.framework.js:3:210100) at Object.runIter (testtest.framework.js:3:211360) at Browser_mainLoop_runner (testtest.framework.js:3:209635)

Need ability to switch between front and back cameras

Feature: Add the ability to manually define recording width x height.

You can currently only choose resolutions. It would be nice to be able to input a custom width x height.

Crash after switching camera on WebGL build

Hi
I have error on WebGL build in Chrome and Firefox browser, doesn't happen in Unity Editor:

It happens when I try to quickly change camera, twice or more, meshing button.
I use this code for switching camera:

        public static CameraDevice GetNextCameraDevice(this VideoKitCameraManager videoKitCameraManager, CameraDevice currentCamera)
        {
            MediaDeviceQuery.ConfigureAudioSession = false;
            var query = new MediaDeviceQuery(MediaDeviceCriteria.CameraDevice);
            var devices = query.OrderBy(device => device is CameraDevice camera).ToList();
            var currentDeviceIndex = devices.IndexOf(currentCamera);

            if (currentDeviceIndex < devices.Count - 1)
            {
                currentDeviceIndex++;
            }
            else
            {
                currentDeviceIndex = 0;
            }
            var nextDevice = devices[currentDeviceIndex];
            return (CameraDevice)nextDevice;
        }

Using RGB stream causes Logitech webcam startup delay

We have HD Pro Webcam C920 and B525 HD Webcam. For them, it takes around one minute to start receiving frames. Same delay when changing resolution.

It happens only on laptops. Tested on two different laptops with clean Win11
No issues on Win11 PCs and other platforms
Other types of webcams is OK
Other apps like OBS, Skype etc. has no such big delay

`Screen` recording is washed out when using linear rendering

Editor freezes on shutdown after camera fails to render

Editor is freezing when starting multiple times, using VideoKitCameraManager. It always works the first time. Then there is about a 25% chance of freezing when stopping play mode every subsequent run. I know in advance when it will freeze because the camera doesn't initialize.

I thought the problem was domain reload, but after turning off domain reload it still happened.

In the inspector window, I disabled the game object that held VideoKitCameraManager and the editor continued to work. However, with it disabled, I then removed the component in the editor window the the editor froze up.

This happens with the sample from your website

Recording resolution is incorrect when the orientation is portrait

I am using VideoKit on an iPhone with the following settings

But the video I got was in 2796 * 1290, which was not a portrait one.

Regarding to Lanre from Discord:

"When the Resolution is Screen or HalfScreen and the Orientation is not Landscape, then the output resolution will actually be landscape. The fix is to either change the Resolution to not be Screen or HalfScreen; or set the Orientation to Landscape when using either of these modes."

Option to fetch mic data

Feature request:
In my music visualization app I use NatDevice to access the microphone and process the audio data to finally visualize it (FFT …). I also use the same audio data to feed NatCorder, so it can record the microphone audio and include it in the video. Is the same possible using VideoKit (commit audio data from NatDevice)?

Your response on discord: “ Not yet; the current API doesn't expose a way to extract the audio data while recording. I recommend opening an issue for this; we can pretty easily expose an API to allow you to receive mic audio while it's being streamed.https://github.com/natmlx/VideoKit”

It would be great to have this feature - thanks!

Camera Orientation Incorrect on Pixel 6

On the Pixel 6, we are seeing the front facing camera image rotated 90 degrees counter clockwise.

This previously worked as expected on this device, but the behavior changed when we upgraded to the latest version of Android available on the device. Other devices we have tested (see below) have not exhibited this issue, but do not seem to have the same OS updates available to them yet.

When we start the camera, we are seeing this spamming the console in logcat on the Pixel 6:
2023/07/05 13:48:46.260 31104 31278 Error ACameraMetadata getConstEntry: cannot find metadata tag 65568
This log message does not seem to appear on devices without the issue.

Consistent across all below tests:

Unity 2021.3.24f1
`Default Orientation` set to Landscape Left in Unity Resolution and Presentation Player Settings
NatDevice 1.3.1 and 1.3.4 were both tested with the same results
Minimum Android API Level 24
IL2CPP
API Compatibility Level: .NET Framework
Graphics APIs: OpenGLES3, Vulkan
Linear Color Space
URP

Configurations tested, where the issue does occur:

Pixel 6
Android 13
March 5, 2023 security update
Google play system update May 1, 2023
Kernel 5.10.149-android13-4-00004-g0e5c0e91bbee-ab947504
Issue present with 'Auto Rotate screen' both enabled and disabled in device's Settings/Display menu

Configurations tested, where the issue does not occur:

Pixel 6
Android 13
Dec 5, 2022 security update
Google play system update May 1, 2023
Kernel 5.10.107-android13-4-00038-ga6f610f1f1e6-ab9083768

Samsung galaxy S7
Android 13
July 1, 2023 security update
Google play system update: May 1, 2023
Kernel 4.19.113-26430179, natmlx/natdevice#1 wed apr 19 14:13:54 +07 2023

Samsung s22 ultra
Android 13
June 1, 2023 security update
Google play system update: may 1, 2023
kernel version: 5.10.136-android12-9-26203684-abs908usqu2cwe6

Mac & Windows in editor working as expected

iPhone 13 Pro & other iOS devices working as expected

Embedded unity With IOS using video kit 0.0.13

Hi I Embedded my unity with IOS. App crash with the error
Helped for embed unity With IOS : https://docs.unity3d.com/2021.3/Documentation/Manual/UnityasaLibrary-iOS.html
Repo that provided by unity : https://github.com/Unity-Technologies/uaal-example/blob/master/docs/ios.md
Library not loaded: @rpath/NatDevice.framework/NatDevice

dyld[5945]: Library not loaded: @rpath/NatDevice.framework/NatDevice
Referenced from: <88AF24F6-A3C6-3B42-880B-232B72490096> /private/var/containers/Bundle/Application/7EA307FF-438C-483F-90AF-9C57B5FB774D/NativeiOSApp.app/Frameworks/UnityFramework.framework/UnityFramework
Reason: tried: '/private/var/containers/Bundle/Application/7EA307FF-438C-483F-90AF-9C57B5FB774D/NativeiOSApp.app/Frameworks/NatDevice.framework/NatDevice' (no such file), '/private/var/containers/Bundle/Application/7EA307FF-438C-483F-90AF-9C57B5FB774D/NativeiOSApp.app/Frameworks/NatDevice.framework/NatDevice' (no such file), '/private/var/containers/Bundle/Application/7EA307FF-438C-483F-90AF-9C57B5FB774D/NativeiOSApp.app/Frameworks/NatDevice.framework/NatDevice' (no such file), '/System/Library/Frameworks/NatDevice.framework/NatDevice' (no such file, not in dyld cache)

Thanks in advance

Edit Socket connection problems

I come from China, I have purchased VideoKit, but my editor still often prompts Token error, which causes me to not develop and run normally. Then every time you want to reconnect to the socket it takes a long time, during which time you will be prompted with this

Out of memory

Hi,
When using NatMl VideoKit 0.0.12 I get this error when switching camera devices. Maybe it is connected with #23, but this one happens only sometimes, mainly after the first run.
Browser version: Chrome 111.0.5563.147
WebGL Unity 2021.3.20f1.

myapplication.cloudfront.net/:1 [.WebGL-000068A406CBB800] GL_INVALID_OPERATION: Feedback loop formed between Framebuffer and active Texture.
myapplication.cloudfront.net/:1 [.WebGL-000068A406CBB800] GL_INVALID_OPERATION: Error: 0x00000502, in ..\..\third_party\angle\src\libANGLE\renderer\d3d\VertexDataManager.cpp, reserveSpaceForAttrib:520. Internal error: 0x00000502: Vertex buffer is not big enough for the draw call.
myapplication.cloudfront.net/:1 [.WebGL-000068A406CBB800] GL_INVALID_OPERATION: Error: 0x00000502, in ..\..\third_party\angle\src\libANGLE\renderer\d3d\VertexDataManager.cpp, reserveSpaceForAttrib:520. Internal error: 0x00000502: Vertex buffer is not big enough for the draw call.
Build007.loader.js:80 exception thrown: RuntimeError: memory access out of bounds,RuntimeError: memory access out of bounds
    at dlfree (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[134051]:0x29391ff)
    at il2cpp_codegen_marshal_free(void*) (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[2895]:0x13f7ed)
    at NatDevice_Name_m7BE4A206300A690F90B92697C3E540277F177766 (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[31499]:0xc64f19)
    at AudioDevice__ctor_mE8A1C592C648F889936191E95E8E557A845B30E6 (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[31497]:0xc64e91)
    at U3CDiscoverAudioDevicesU3Ed__20_MoveNext_m3947F1662EDDA93971217F8C7E67172E63FDE931 (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[31611]:0xc68152)
    at InterfaceFuncInvoker0<XmlQualifiedName_t2794880B373257E4108CC3F36D7373A343ACC5B9*>::Invoke(unsigned short, Il2CppClass*, Il2CppObject*) (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[19032]:0x9b57df)
    at dynCall_iiii (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[134309]:0x29495a7)
    at https://myapplication.cloudfront.net/Build/Build007.framework.js:1142:20
    at invoke_iiii (https://myapplication.cloudfront.net/Build/Build007.framework.js:22666:10)
    at List_1_AddEnumerable_m76C85811BC9C905E7224C054AD480C3062FA7700_gshared (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[3066]:0x147c37)
    at List_1_InsertRange_m386ABA9B2C4950D31E787C87CDEF3D3E103F1B1A_gshared (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[3060]:0x1477f6)
    at MediaDeviceQuery_DiscoverDevices_m0520738B6315FF77EFB342F12AB3CF55102140C5 (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[31596]:0xc67cfb)
    at MediaDeviceQuery__ctor_mE2C86815B53C8771C6F5BD82319E56505C59FF82 (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[31595]:0xc67b7c)
    at VideoKitCameraManagerCustom_GetNextTargetCameraDevice_m9380574BD557FF61F6CAE1A521899518DE5399D5 (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[32593]:0xcb543a)
    at NatMlMeetBackgroundRemover_SwitchCameraDeviceToTarget_mF727B58824BDA93FF90F16C8943B75A8A71D35A5 (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[32592]:0xcb5355)
    at HallwayViewModel_SwitchCameraToTarget_m04ABAD5FFB58C9FA8399D4233B2C97E367BE5417 (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[32653]:0xcb8514)
    at HallwayViewModel_SetupBindings_mFE85A06D529A292A245905F764C0FE7CB02F498E (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[32649]:0xcb8291)
    at HallwayViewModel_Start_mD1BD1C85DC59E11053F9FD98C15301F462C6A8B8 (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[32647]:0xcb7f1d)
    at RuntimeInvoker_TrueVoid_t4861ACF8F4594C3437BB48B6E56783494B843915(void (*)(), MethodInfo const*, void*, void**, void*) (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[106995]:0x1b8b794)
    at il2cpp::vm::Runtime::InvokeWithThrow(MethodInfo const*, void*, void**) (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[109073]:0x1bb3470)
    at dynCall_iiii (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[134309]:0x29495a7)
    at https://myapplication.cloudfront.net/Build/Build007.framework.js:1142:20
    at invoke_iiii (https://myapplication.cloudfront.net/Build/Build007.framework.js:22666:10)
    at il2cpp::vm::Runtime::Invoke(MethodInfo const*, void*, void**, Il2CppException**) (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[109047]:0x1bb292e)
    at il2cpp_runtime_invoke (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[109117]:0x1bb4f99)
    at scripting_method_invoke(ScriptingMethodPtr, ScriptingObjectPtr, ScriptingArguments&, ScriptingExceptionPtr*, bool) (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[133406]:0x28fe5eb)
    at ScriptingInvocation::Invoke(ScriptingExceptionPtr*, bool) (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[131381]:0x2868a96)
    at MonoBehaviour::InvokeMethodOrCoroutineChecked(ScriptingMethodPtr, ScriptingObjectPtr) (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[131321]:0x286386f)
    at MonoBehaviour::DelayedStartCall(Object*, void*) (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[131335]:0x2864fba)
    at DelayedCallManager::Update(int) (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[111940]:0x1c4eab4)
    at InitPlayerLoopCallbacks()::EarlyUpdateScriptRunDelayedStartupFrameRegistrator::Forward() (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[131072]:0x283ee3d)
    at ExecutePlayerLoop(NativePlayerLoopSystem*) (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[111761]:0x1c3df9d)
    at ExecutePlayerLoop(NativePlayerLoopSystem*) (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[111761]:0x1c3e039)
    at MainLoop() (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[128891]:0x26f8d62)
    at dynCall_v (https://myapplication.cloudfront.net/Build/Build007.wasm:wasm-function[134312]:0x29495c7)
    at https://myapplication.cloudfront.net/Build/Build007.framework.js:1142:20
    at browserIterationFunc (https://myapplication.cloudfront.net/Build/Build007.framework.js:15075:13)
    at callUserCallback (https://myapplication.cloudfront.net/Build/Build007.framework.js:13254:3)
    at Object.runIter (https://myapplication.cloudfront.net/Build/Build007.framework.js:13315:4)
    at Browser_mainLoop_runner (https://myapplication.cloudfront.net/Build/Build007.framework.js:13229:20)
printErr @ Build007.loader.js:80

Android StopRecording fails if scene with VideoKitRecorder in get's unloaded and reloaded.

I have Scene with a VideoKitRecorder gameobject in it (see screenshot).
The first time I run and load this scene on Android it works fine, and saves the recording correctly. I then unload the Scene, go to another scene and reload the scene with the VideoKitRecorder gameobject in it again, the second time this scene get's loaded and I call StopRecording, Android starts spitting out a bunch of errors (see screenshot).

Note, the VideoKitRecorder gameobject is disabled by default, and I call SetActive(true) on it with a small delay so I can control when the prepareRecorder logic starts running, as it stalls the scene for a sec or two.

When recording from texture with camera running, the camera turns black and stutters in the first few frames

It seems like the camera device is being reset at the start of recording, along with some stuttering

Microphone audio crackling on WebGL

Request fast mobile-optimized face mesh

Custom resolution mode swaps width and height in final recording

Safari with NatML

Hi @lanreolokoba @olokobayusuf

I know there is no support for Safari, but maybe there is some solution for working with that browser?
When building sample project for Meet Segmentation I get this error in Safari:

abort({}) at jsStackTrace@https://v6p9d9t4.ssl.hwcdn.net/html/7579938/Build/Build.framework.js:3:25512
stackTrace@https://v6p9d9t4.ssl.hwcdn.net/html/7579938/Build/Build.framework.js:3:25797
abort@https://v6p9d9t4.ssl.hwcdn.net/html/7579938/Build/Build.framework.js:3:1002
https://v6p9d9t4.ssl.hwcdn.net/html/7579938/Build/Build.framework.js:3:20377
promiseReactionJob@[native code]

Thanks in advance for any help

[Android] Missing permission requests for photo/video saving in VideoKit

Some (possibly newer) android devices/OS version seem to require a permission-check before allowing to save video/photo.
See here:
https://developer.android.com/about/versions/14/changes/partial-photo-video-access
https://developer.android.com/training/permissions/requesting

Currently, the user has to manually go to the settings and grant permissions.

Events for VideoKitCameraView

Due to a delay between calling VideoKitCameraManager.StartRunning() and the camera actually being rendered to VideoKitCameraView it would be nice to have a set of events that would allow for performing actions when the camera feed starts/finishes being rendered onto the texture.

Request - Best practice for recording front camera device as well as screen/in-game camera

Expose frame skipping when recording with `VideoKitRecorder`

This allows us control the compression and flow of time when recording animated GIF images.

Enhancement: Add LateUpdate-based ScreenInput Option for Capturing VR Footage

Issue:
I have been using the NatCorder library in combination with the Google Cardboard XR Plugin to capture VR footages. However, I encountered an issue where the screen recorder was capturing single-view footages instead of capturing the VR black mask as desired.

Proposed Enhancement:
I have implemented a new class, LateScreenInput, that captures frames using LateUpdate, allowing me to capture the VR black mask successfully. This addition provides users with more flexibility when recording VR content, as it captures the VR mask, while the ScreenInput can still capture non-VR views. I believe this enhancement could be valuable for users who need to capture VR content with the Cardboard XR Plugin.

Code Changes:
I've made the following changes in the LateScreenInput class:

Removed the WaitForEndOfFrame yield and replaced it with LateUpdate-based frame capturing.
Added a resolution parameter to customize the recording resolution.
Resized the captured frame to match the desired resolution.

Code Comparison:
Below is a comparison of the original ScreenInput class and my modified LateScreenInput class.

Original ScreenInput class:

/* 
*   NatCorder
*   Copyright (c) 2022 NatML Inc. All Rights Reserved.
*/

namespace NatML.Recorders.Inputs {

    using System;
    using System.Collections;
    using UnityEngine;
    using Clocks;

    /// <summary>
    /// Recorder input for recording video frames from the screen.
    /// Unlike the `CameraInput`, this recorder input is able to record overlay UI canvases.
    /// </summary>
    public sealed class ScreenInput : IDisposable {

        #region --Client API--
        /// <summary>
        /// Control number of successive camera frames to skip while recording.
        /// This is very useful for GIF recording, which typically has a lower framerate appearance.
        /// </summary>
        public int frameSkip;

        /// <summary>
        /// Create a video recording input from the screen.
        /// </summary>
        /// <param name="recorder">Media recorder to receive video frames.</param>
        /// <param name="clock">Recording clock for generating timestamps.</param>
        public ScreenInput (IMediaRecorder recorder, IClock clock = default) : this(TextureInput.CreateDefault(recorder), clock) { }

        /// <summary>
        /// Create a video recording input from the screen.
        /// </summary>
        /// <param name="input">Texture input to receive video frames.</param>
        /// <param name="clock">Recording clock for generating timestamps.</param>
        public ScreenInput (TextureInput input, IClock clock = default) {
            this.input = input;
            this.clock = clock;
            this.frameDescriptor = new RenderTextureDescriptor(input.frameSize.width, input.frameSize.height, RenderTextureFormat.ARGB32, 0);
            // Start recording
            attachment = new GameObject("NatCorder ScreenInputAttachment").AddComponent<ScreenInputAttachment>();
            attachment.StartCoroutine(CommitFrames());
        }

        /// <summary>
        /// Stop recorder input and release resources.
        /// </summary>
        public void Dispose () {
            GameObject.Destroy(attachment.gameObject);
            input.Dispose();
        }
        #endregion


        #region --Operations--
        private readonly TextureInput input;
        private readonly IClock clock;
        private readonly RenderTextureDescriptor frameDescriptor;
        private readonly ScreenInputAttachment attachment;
        private int frameCount;

        private IEnumerator CommitFrames () {
            var yielder = new WaitForEndOfFrame();
            for (;;) {
                // Check frame index
                yield return yielder;
                if (frameCount++ % (frameSkip + 1) != 0)
                    continue;
                // Capture screen
                var frameBuffer = RenderTexture.GetTemporary(frameDescriptor);
                if (SystemInfo.graphicsUVStartsAtTop) {
                    var tempBuffer = RenderTexture.GetTemporary(frameDescriptor);
                    ScreenCapture.CaptureScreenshotIntoRenderTexture(tempBuffer);
                    Graphics.Blit(tempBuffer, frameBuffer, new Vector2(1, -1), Vector2.up);
                    RenderTexture.ReleaseTemporary(tempBuffer);
                } else
                    ScreenCapture.CaptureScreenshotIntoRenderTexture(frameBuffer);
                // Commit
                input.CommitFrame(frameBuffer, clock?.timestamp ?? 0L);
                RenderTexture.ReleaseTemporary(frameBuffer);
            }
        }

        private sealed class ScreenInputAttachment : MonoBehaviour { }
        #endregion
    }
}

Modified LateScreenInput class:

namespace NatML.Recorders.Inputs
{
    using System;
    using System.Collections;
    using UnityEngine;
    using Clocks;

    /// <summary>
    /// Recorder input for recording video frames from the screen.
    /// Unlike the `CameraInput`, this recorder input is able to record overlay UI canvases.
    /// </summary>
    public sealed class LateScreenInput : IDisposable
    {
        #region --Client API--

        /// <summary>
        /// Control number of successive camera frames to skip while recording.
        /// This is very useful for GIF recording, which typically has a lower framerate appearance.
        /// </summary>
        public int frameSkip;

        /// <summary>
        /// Create a video recording input from the screen.
        /// </summary>
        /// <param name="recorder">Media recorder to receive video frames.</param>
        /// <param name="clock">Recording clock for generating timestamps.</param>
        /// <param name="resolution"></param>
        public LateScreenInput(IMediaRecorder recorder, IClock clock = default) : this(
            TextureInput.CreateDefault(recorder),
            clock)
        {
        }

        /// <summary>
        /// Create a video recording input from the screen.
        /// </summary>
        /// <param name="input">Texture input to receive video frames.</param>
        /// <param name="clock">Recording clock for generating timestamps.</param>
        public LateScreenInput(TextureInput input, IClock clock = default)
        {
            this.input = input;
            this.clock = clock;
            screenDescriptor = new RenderTextureDescriptor(Screen.width, Screen.height,
                RenderTextureFormat.ARGBHalf, 0);
            resizeDescriptor = new RenderTextureDescriptor(input.frameSize.width, input.frameSize.height,
                RenderTextureFormat.ARGBHalf, 0);
            // Start recording
            attachment = new GameObject(@"NatCorder ScreenInputAttachment").AddComponent<ScreenInputAttachment>();
            //attachment.StartCoroutine(CommitFrames());
            attachment.CommitFrames = CommitFrames;
        }

        /// <summary>
        /// Stop recorder input and release resources.
        /// </summary>
        public void Dispose()
        {
            GameObject.DestroyImmediate(attachment.gameObject);
            input.Dispose();
        }

        #endregion


        #region --Operations--

        private readonly TextureInput input;
        private readonly IClock clock;
        private readonly RenderTextureDescriptor screenDescriptor;
        private readonly RenderTextureDescriptor resizeDescriptor;
        private readonly ScreenInputAttachment attachment;
        private int frameCount;

        public void CommitFrames()
        {
            // Check frame index
            if (frameCount++ % (frameSkip + 1) != 0)
                return;
            // Capture screen
            var frameBuffer = RenderTexture.GetTemporary(screenDescriptor);
            if (SystemInfo.graphicsUVStartsAtTop)
            {
                var tempBuffer = RenderTexture.GetTemporary(screenDescriptor);
                ScreenCapture.CaptureScreenshotIntoRenderTexture(tempBuffer);
                Graphics.Blit(tempBuffer, frameBuffer, new Vector2(1, -1), Vector2.up);
                RenderTexture.ReleaseTemporary(tempBuffer);
            }
            else
                ScreenCapture.CaptureScreenshotIntoRenderTexture(frameBuffer);

            // Resizing
            var resizedFrameBuffer = RenderTexture.GetTemporary(resizeDescriptor);
            Graphics.Blit(frameBuffer, resizedFrameBuffer);
            Debug.LogError(resizedFrameBuffer.width + "   " + resizedFrameBuffer.height);
            // Commit
            input.CommitFrame(resizedFrameBuffer, clock?.timestamp ?? 0L);
            RenderTexture.ReleaseTemporary(frameBuffer);
            RenderTexture.ReleaseTemporary(resizedFrameBuffer);
        }

        private sealed class ScreenInputAttachment : MonoBehaviour
        {
            public Action CommitFrames;

            private void LateUpdate()
            {
                CommitFrames();
            }
        }

        #endregion
    }
}

How to Use:
To use the LateScreenInput class for capturing VR content with the Cardboard XR Plugin, follow these steps:

Instantiate a LateScreenInput object, passing your media recorder and desired resolution as parameters. Additionally, make sure to create a RealtimeClock for timestamp generation.

Example Usage:

// Create a RealtimeClock for timestamp generation
clock = new RealtimeClock();

// Calculate the recording resolution based on your aspect ratio
float aspectRatio = (float)Screen.width / Screen.height;
int recordWidth = (int)(Resolution * aspectRatio);
recordWidth = recordWidth % 2 == 0 ? recordWidth : recordWidth - 1;

// Instantiate an MP4Recorder with your resolution and desired frame rate (e.g., 30 FPS)
recorder = new MP4Recorder(recordWidth, Resolution, 30);

// Instantiate a LateScreenInput with the recorder, resolution, and clock
lateScreenInput = new LateScreenInput(recorder, clock);

Additional Information:

Operating System: Windows 10 x64
Unity Editor Version: 2020.3.30f1
NatCorder Version: 1.9.1
Cardboard XR Plugin Version: 1.21.0

Expected Behavior:

With this enhancement, users should be able to capture VR footages that include the VR black mask when using the Cardboard XR Plugin, in addition to the existing non-VR view capture functionality.

Android Build Fails when using AVPro plugin

Hello I am getting this error when trying to build APk that is using VideoKit

When I remove VideoKit it builds with no errors.
It happens when I import the AVPro plugin.
Here is a test project.
https://drive.google.com/file/d/1HxO0c3zUfv5a439E_dg0cRH5HmsGhJ-h/view?usp=sharing

Expose `frameDuration` when using GifRecorder

Is it possible to expose the frameDuration so that one can manually define the GifRecorder's frame delay? It's private right now.

Add functionality to "warm-up" recorder, to avoid stall on first recording.

As per discord discussion there's a stutter / lag happening at the beginning of the first recording. The suggested workaround from discord thread is: "The workaround is pretty simple: create a recorder, commit exactly one video frame (empty array), then finish recording and delete the file.
Best done in a loading screen" - would be nice to have some helper function that could be called that does this.

Allow for sharing a recording sometime after recording is finished

Need way to separate and then combine recording of audio and video

I'm working on a project which performs post processing on a video. Each video frame needs to be processed and committed to back to the recording at the original frame's timestamp. In order for the audio to sync up, the audio needs to be transplanted to the new recording as well.

Prepare on awake setting for the recorder still causes stutter

Recorder in free tier causes image to be zoomed in

Unity freezes during unstable network connections

SocketException: No such host is known
System.Net.Http.ConnectHelper.ConnectAsync (System.String host, System.Int32 port, System.Threading.CancellationToken cancellationToken) (at <29bbdedb06b44b60bf32f833977d7bd1>:0)
Rethrow as HttpRequestException: No such host is known
System.Net.Http.ConnectHelper.ConnectAsync (System.String host, System.Int32 port, System.Threading.CancellationToken cancellationToken) (at <29bbdedb06b44b60bf32f833977d7bd1>:0)
System.Net.Http.HttpConnectionPool.CreateConnectionAsync (System.Net.Http.HttpRequestMessage request, System.Threading.CancellationToken cancellationToken) (at <29bbdedb06b44b60bf32f833977d7bd1>:0)
System.Net.Http.HttpConnectionPool.WaitForCreatedConnectionAsync (System.Threading.Tasks.ValueTask1[TResult] creationTask) (at <29bbdedb06b44b60bf32f833977d7bd1>:0) System.Net.Http.HttpConnectionPool.SendWithRetryAsync (System.Net.Http.HttpRequestMessage request, System.Boolean doRequestAuth, System.Threading.CancellationToken cancellationToken) (at <29bbdedb06b44b60bf32f833977d7bd1>:0) System.Net.Http.RedirectHandler.SendAsync (System.Net.Http.HttpRequestMessage request, System.Threading.CancellationToken cancellationToken) (at <29bbdedb06b44b60bf32f833977d7bd1>:0) System.Net.Http.HttpClient.FinishSendAsyncBuffered (System.Threading.Tasks.Task1[TResult] sendTask, System.Net.Http.HttpRequestMessage request, System.Threading.CancellationTokenSource cts, System.Boolean disposeCts) (at <29bbdedb06b44b60bf32f833977d7bd1>:0)
NatML.Hub.NatMLHub.RequestNet[TRequest,TResponse] (TRequest request, System.String accessKey) (at ./Library/PackageCache/[email protected]/Runtime/NatMLHub.cs:196)
NatML.Hub.NatMLHub.GetUser (System.String accessKey) (at ./Library/PackageCache/[email protected]/Runtime/NatMLHub.cs:77)
Rethrow as AggregateException: One or more errors occurred. (No such host is known)
System.Threading.Tasks.Task.ThrowIfExceptional (System.Boolean includeTaskCanceledExceptions) (at <434571214be741f48f7262e1e0f039a1>:0)
System.Threading.Tasks.Task1[TResult].GetResultCore (System.Boolean waitCompletionNotification) (at <434571214be741f48f7262e1e0f039a1>:0) System.Threading.Tasks.Task1[TResult].get_Result () (at <434571214be741f48f7262e1e0f039a1>:0)
NatML.Hub.Editor.HubProjectSettings.CreateSettings (System.Boolean refresh) (at ./Library/PackageCache/[email protected]/Editor/HubProjectSettings.cs:46)
NatML.Hub.Editor.HubProjectSettings.OnLoad () (at ./Library/PackageCache/[email protected]/Editor/HubProjectSettings.cs:64)
System.Runtime.CompilerServices.AsyncMethodBuilderCore+<>c.b__7_0 (System.Object state) (at <434571214be741f48f7262e1e0f039a1>:0)
UnityEngine.UnitySynchronizationContext+WorkRequest.Invoke () (at /Users/bokken/build/output/unity/unity/Runtime/Export/Scripting/UnitySynchronizationContext.cs:153)
UnityEngine.UnitySynchronizationContext.Exec () (at /Users/bokken/build/output/unity/unity/Runtime/Export/Scripting/UnitySynchronizationContext.cs:83)
UnityEngine.UnitySynchronizationContext.ExecuteTasks () (at /Users/bokken/build/output/unity/unity/Runtime/Export/Scripting/UnitySynchronizationContext.cs:107)

CameraFrame.image.width and CameraFrame.image.height are set to 0 intermittently

How to / Add feature for: Read out ISO value of camera

We are currently digging deeper into the SDK and we realized that it's possible to get the ISO-range, but not the current ISO-value. Getting the exposure value works, just not the ISO.
We need to get the actual value to display in a slider (current value, and min/max as the ends of the slider)

Is it possible to get/access the current ISO value somehow already / could this feature be added?

UnityPlayer crashes when using Japanese paths(User Name) when recording starts on Windows.

About

the UnityPlayer crashes when using Japanese paths(User Name) on Windows.
Similar problems have been reported on the forum and I have the same fix to avoid the Japanese pass, but UnityPlayer crashes when I try to use the recording feature.
a possible cause is that NatCorder.Dll does not support multibyte characters.

How to reproduce the Crush

Create two local users (one is the Japanese name and the other is the English name)
Users with Japanese names will crash when recording starts
Users with English names are normal when recording starts

Crush when the following code is executed

Recorder = new MP4Recorder(256, 144, Framerate, _SampleRate, _ChannelCount, audioBitRate: 96_000);

System Information

CPU : Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz 3.20 GHz
RAM : 32GB
OS : Windows 11 Pro(22621.1992)
UnityVersion : 2020.3.48f1
NatCoderVersion : 1.8.8(I tried v1.9.4 and it still crushed.)

Other

.\[email protected]\Runtime\RecorderExtensions.cs:57 was rewritten with the following code, new MP4Recorder(...) worked, ~~but it crushed at the start of recording.~~

        [RuntimeInitializeOnLoadMethod]
        private static void OnInitialize () {
            var editor = Application.platform == RuntimePlatform.OSXEditor || Application.platform == RuntimePlatform.WindowsEditor;

            const string workingDirectoryPath = @"c:\tmp\";
            Directory.CreateDirectory(workingDirectoryPath);
            foreach (var filePath in Directory.GetFiles(workingDirectoryPath)) { File.Delete(filePath);}

            directory = editor ? Directory.GetCurrentDirectory() : workingDirectoryPath; //Application.persistentDataPath;
        }

Camera settings like framerate and resolution don't update between restarts

The settings don't take effect after stopping and restarting the camera

Access Key Works in editor but not in IOS build

Everything works great in the editor, including video and photo recording. BUT when I build a mobile IOS device I get the following errors:
"InvalidOperationExeption: NatCorder session token invalid. Check your NatML access key and plan"
"InvalidOperationExeption: NatDevice session token invalid. Check your NatML access key and plan"
"InvalidOperationExeption: NatDevice session token invalid. Check your NatML access key and plan"

I've been trouble shooting this bug for two days now any idea is welcome!

Safari 16.4 StopRunning() camera doesn't stops camera.

We have found that on Safari 16.4 after StopRunning() camera, the camera device is still running. (We see camera icon in browser and diode is on)
WebGL, Unity 2021.3.20.

Capturing a single image

Hi!

I was wondering if there was an easy way to capture a single image with VideoKit. I would like my recorder to support both video recording and image recording.
I have seen the JPEG recorder but it feels a bit hacky to force it to take a single frame.

Is there any way to achieve this ?

Thank you very much.

Regards,

Add a 'pause recording' functionality

Currently, with NatDevice, frames can be skipped (by not committing them) to simulate a pause while recording a video. Since VideoKit is out now, it would be cool to add it as a function to the library.

Add option to record VideoKitCameraView as one of VideoKitRecorder's Video Modes

Currently VideoKitRecorder allows for recording of Camera, Screen or Texture.

With VideoKitCameraView allowing us to display the camera feed on a RawImage i think it would be a common use case to record the RawImage's Texture itself.

Currently it is possible to achieve this effect with a workaround involving another VideoKitCameraView rendered on a Canvas with Render Mode set to Screen Space - Camera then setting VideoKitRecorder's Video Mode to Camera and setting the camera to the one on which the Canvas is overlayed.

Add watermark to recordings with WatermarkTextureInput

We can add watermark with the use of WatermarkTextureInput in NatCoder.

Is that possible now to add watermark functionality with WatermarkTextureInput into VideoKitRecorder?

Thank you very much.

Requests the ability to save a movie with a specified aspect ratio (area).

hello.
I would like to upload a game screen video in conjunction with social media.
However, all parts are not required. For Instagram, an aspect ratio of 4:3 is best. I'd like to make this automatically adjustable.
I tried to use the Texture Input part, but it seems that it is not possible to modify the script manually.

Are there any plans to create a screen recording function for each area?

Request the ability of trimming videos using VideoKit

Hi. I need to trim or cut videos before exporting/sharing. If it's not too big of a hassle, It would be great to be able to do it using VideoKit and not adding another video api/sdk specifically for this. Thanks.

Camera recording creates a mirrored output

Ideally this would be exposed as a togglable setting

NDSetSessionToken error

Hi,
I have 3 testers on the team; one is testing our application daily. Two other testers were testing last time our application about a week ago. In the meantime, we updated nat VifeoKit to 0.0.13. Those two testers get error as shown below. Both of the testers cleared the browser cache and only for one it was a solution.
We are using NatML VideoKit and Natml Meet Segmentation for WebGL Unity.

TypeError: navigator.mediaDevices is undefined
7565410@http://myapplication.cloudfront.net/Build/Build013.framework.js:1304:21
_emscripten_asm_const_int@http://myapplication.cloudfront.net/Build/Build013.framework.js:13143:26
NDSetSessionToken@http://myapplication.cloudfront.net/Build/Build013.wasm:wasm-function[16881]:0x42be0a
NatDevice_SetSessionToken_mF84DD814AB106B6E48603805BA3973F6F00ADCF7@http://myapplication.cloudfront.net/Build/Build013.wasm:wasm-function[17321]:0x440075
MediaDeviceQuery_OnInitialize_m37E36832EFD39CC3C4951D728FBEA07BFEB1020F@http://myapplication.cloudfront.net/Build/Build013.wasm:wasm-function[17320]:0x440066
RuntimeInvoker_FalseVoid_t4861ACF8F4594C3437BB48B6E56783494B843915(void (*)(), MethodInfo const*, void*, void**, void*)@http://myapplication.cloudfront.net/Build/Build013.wasm:wasm-function[108837]:0x1bb8511
il2cpp::vm::Runtime::InvokeWithThrow(MethodInfo const*, void*, void**)@http://myapplication.cloudfront.net/Build/Build013.wasm:wasm-function[109321]:0x1bc479e
dynCall_iiii@http://myapplication.cloudfront.net/Build/Build013.wasm:wasm-function[134557]:0x295ae14
unityFramework/createExportWrapper/<@http://myapplication.cloudfront.net/Build/Build013.framework.js:1142:20
invoke_iiii@http://myapplication.cloudfront.net/Build/Build013.framework.js:22798:10
il2cpp::vm::Runtime::Invoke(MethodInfo const*, void*, void**, Il2CppException**)@http://myapplication.cloudfront.net/Build/Build013.wasm:wasm-function[109295]:0x1bc3c5c
il2cpp_runtime_invoke@http://myapplication.cloudfront.net/Build/Build013.wasm:wasm-function[109365]:0x1bc62cb
scripting_method_invoke(ScriptingMethodPtr, ScriptingObjectPtr, ScriptingArguments&, ScriptingExceptionPtr*, bool)@http://myapplication.cloudfront.net/Build/Build013.wasm:wasm-function[133654]:0x290fe58
ScriptingInvocation::Invoke(ScriptingExceptionPtr*, bool)@http://myapplication.cloudfront.net/Build/Build013.wasm:wasm-function[131629]:0x287a2fb
RuntimeInitializeOnLoadManager::ExecuteInitializeOnLoad(RuntimeInitializeLoadType)@http://myapplication.cloudfront.net/Build/Build013.wasm:wasm-function[134083]:0x29363d7
RuntimeInitializeAfterAssembliesLoaded()@http://myapplication.cloudfront.net/Build/Build013.wasm:wasm-function[134081]:0x29360fc
CallbackArray::Invoke()@http://myapplication.cloudfront.net/Build/Build013.wasm:wasm-function[134095]:0x29376f2
main@http://myapplication.cloudfront.net/Build/Build013.wasm:wasm-function[129141]:0x2728f15
unityFramework/createExportWrapper/<@http://myapplication.cloudfront.net/Build/Build013.framework.js:1142:20
callMain@http://myapplication.cloudfront.net/Build/Build013.framework.js:25692:26
doRun@http://myapplication.cloudfront.net/Build/Build013.framework.js:25735:29
run@http://myapplication.cloudfront.net/Build/Build013.framework.js:25747:3
runCaller@http://myapplication.cloudfront.net/Build/Build013.framework.js:25675:18
removeRunDependency@http://myapplication.cloudfront.net/Build/Build013.framework.js:1097:4
createUnityInstance/loadBuild/</<@http://myapplication.cloudfront.net/Build/Build013.loader.js:1104:16

Higher preview resolution looks blurry on windows

I was playing the demo project with a stereo cameras, with resolution support like 640x376(single view),1280x376(dual views), 2560x800, 3840x1080.
However, not matter how high I config the previewResolution and MP4Recorder.videoWidth, the result preview resolution I get on screen looks like a very low resolution.
Then I updated natdevice to 1.3.1 and natcorder to 1.9.1, still the same results: Even I set the preview resolution to 640x376, the result is a blurry dual view.

I just set screen resolution to : 640,376 and added two prints as:
cameraDevice = query.FirstOrDefault(device => device is CameraDevice) as CameraDevice;
cameraDevice.previewResolution = (playWidth, playHeight);
//cameraDevice.photoResolution = (1920, 1080);
print(" resolutions preview: " + cameraDevice.previewResolution.width + " : " + cameraDevice.previewResolution.height);
audioDevice = query.FirstOrDefault(device => device is AudioDevice) as AudioDevice;
// Start the camera preview
previewTextureOutput = new TextureOutput();
cameraDevice.StartRunning(previewTextureOutput);
// Display the preview texture
var previewTexture = await previewTextureOutput;
previewImage.texture = previewTexture;
previewAspectFitter.aspectRatio = (float)previewTexture.width / previewTexture.height;
print(" previewTexture "+ previewTexture.width + " : " + previewTexture.height);

The result is :
resolutions preview: 640 : 376
previewTexture 640 : 376

https://user-images.githubusercontent.com/11953709/200452187-b7a53ff9-7eb1-44ab-b80d-19f2e9acd1ed.png

FileNotFoundException in the combination of the format 'JPEG' and the destination 'Documents'.

In case of JPEG output format, recording result is not file, but directory and this causes problems (FileNotFoundException) in SaveToDocuments method, where you are trying to move file instead of directory on line 709 by call: File.Move(path, result)

I didnt do any further investigation, but it looks like this will be global issue no matter which platform.