norihiro / obs-face-tracker Goto Github PK
View Code? Open in Web Editor NEWFace tracking plugin for OBS Studio
License: GNU General Public License v2.0
Face tracking plugin for OBS Studio
License: GNU General Public License v2.0
I hope the face will be centered in the screen, even including the size after cropping.
Amazing OBS Plugin! Thank you for your development!
Switch scene by these triggers.
Getting the following error with selfcompiled obs-face-tracker & obs-studio installed via flatpak
13:50:39.080: os_dlopen(/home/user/.var/app/com.obsproject.Studio/config/obs-studio/plugins/obs-face-tracker/bin/64bit/obs-face-tracker.so->/home/user/.var/app/com.obsproject.Studio/config/obs-studio/plugins/obs-face-tracker/bin/64bit/obs-face-tracker.so): libflexiblas.so.3: Kann die Shared-Object-Datei nicht öffnen: Datei oder Verzeichnis nicht gefunden
13:50:39.080:
13:50:39.080: Module '/home/user/.var/app/com.obsproject.Studio/config/obs-studio/plugins/obs-face-tracker/bin/64bit/obs-face-tracker.so' not loaded
A library openblas
need to be added target_link_libraries
.
When face detection starts, I got this error.
obs: symbol lookup error: /usr/local//lib/obs-plugins/obs-face-tracker.so: undefined symbol: dgesvd_
OS: CentOS8
Add openblas
to target_link_libraries(${CMAKE_PROJECT_NAME} ...)
.
It would be better performance to enable openblas for Windows and MacOS build.
I don't know about Ubuntu behavior. Maybe it's better to add.
These code should be configurable from a property so that user can adjust the trade-off between speed and accuracy.
Especially those who use HD or 4K, the scale factor should be larger to reduce CPU usage.
As the main idea, after pressing the RESET button, the KPI value is scaled back to its original size.
The idea is that if the face is not tracked, or if there is a mistake, it can be elegantly returned to the original size, instead of directly switching to the original size.
I think it might be possible to make a checkbox option so that "directly RESET" and "elegant RESET" can coexist.
My English is not very good, so if there is anything I can't understand, please talk to me.
Thanks to the developer for creating such a great plug-in!
Thinking the user experiences, this plugin should send commands to the camera through the other plugin obs-ptz
so that a user can install both plugin and use at the same time. Especially VISCA over serial port cannot be controlled from individual program.
glikely/obs-ptz#17
Having these states and switch between them so that the behavior is closer to real camera operator
It gives the option to create a facetracker ptz under the audio tab in filters but when ever you click on it nothing happens.
Is it possible to add swap x y z option, support camera vertical installation, or directly support face recognition after image rotation.
Not just detecting faces, but also recognize faces.
It would be useful when there are a lot of people on the scene but want to track just main person.
This feature was requested by several users such as
https://obsproject.com/forum/threads/face-tracker.144419/post-541643
Conditions
Duplicate Sources
and Duplicate Scene
are both enabled.Since the internal state of Face Tracker is not copied, Face Tracker start from the initial location.
I found this issue when looking into #14 but this is a different issue originally reported.
Hi @norihiro ,
Thanks for the great plugin!
Played around with it for the first time today and it was a lot of fun and we will use it a lot in the future.
What I couldn't find is a basic explanation what each attribute or setting actually does?
A couple are explained in the wiki but not all of them.
The best way would be little hover infos next to the settings, but of course a basic web page would do the trick too.
Thanks again for your work!
20:08:43.202: video device: FaceTime HD Camera (Built-in)
20:08:43.202: video path: \\?\usb#vid_05ac&pid_8514&mi_00#7&26c477d1&0&0000#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global
20:08:43.202: resolution: 640x480
20:08:43.202: flip: 0
20:08:43.202: fps: 30.00 (interval: 333333)
20:08:43.202: format: NV12
20:08:45.013: effect_setval_inline: invalid param
20:08:47.013: effect_setval_inline: invalid param
20:08:49.013: effect_setval_inline: invalid param
20:08:49.296: [obs-face-tracker] ftf_preset_load: loading preset
20:08:49.297: [obs-face-tracker] ftf_preset_load: preset does not exist
20:08:50.540: [obs-face-tracker] ftf_preset_save: deleting preset
20:08:51.013: effect_setval_inline: invalid param
20:08:53.013: effect_setval_inline: invalid param
20:08:55.013: effect_setval_inline: invalid param
20:08:57.013: effect_setval_inline: invalid param
20:08:59.013: effect_setval_inline: invalid param
20:09:01.013: effect_setval_inline: invalid param
20:09:03.013: effect_setval_inline: invalid param
20:09:05.013: effect_setval_inline: invalid param
20:09:07.013: effect_setval_inline: invalid param
20:09:09.013: effect_setval_inline: invalid param
20:09:11.013: effect_setval_inline: invalid param
20:09:13.013: effect_setval_inline: invalid param
20:09:15.013: effect_setval_inline: invalid param
20:09:17.013: effect_setval_inline: invalid param
20:09:19.013: effect_setval_inline: invalid param
20:09:21.013: effect_setval_inline: invalid param
20:09:23.013: effect_setval_inline: invalid param
20:09:25.013: effect_setval_inline: invalid param
20:09:27.013: effect_setval_inline: invalid param
20:09:29.013: effect_setval_inline: invalid param
20:09:31.013: effect_setval_inline: invalid param
20:09:33.013: effect_setval_inline: invalid param
20:09:35.013: effect_setval_inline: invalid param
20:09:37.013: effect_setval_inline: invalid param
20:09:39.013: effect_setval_inline: invalid param
20:09:41.013: effect_setval_inline: invalid param
20:09:43.013: effect_setval_inline: invalid param
20:09:45.013: effect_setval_inline: invalid param
20:09:47.013: effect_setval_inline: invalid param
20:09:49.013: effect_setval_inline: invalid param
20:09:51.013: effect_setval_inline: invalid param
21:02:21.779: Last log entry repeated for 1575 more lines
Hi,
Just a quick info: Apparently the obs-face-tracker-0.5.4-Windows.zip file is damaged. I can't unzip it on my Win10 computer.
Hello,
sorry if I ask a totally stupid question, but for the hell I cannot find the download files for obs-face-tracker. I wanted to install it on macOS according to the Wiki instructions, but I have no idea where to get the dmg from.
Could you please point out, where the files reside?
Thanks a lot.
I'd like to try and compare with another face detection like using OpenCV.
Track object instead of automatically detected face.
https://obsproject.com/forum/threads/face-tracker.144419/post-548452
It just zooms in on where it last tracked my face and doesn't track when I come back. "Reset Tracking" doesn't work, the only solution is to delete filter and add it again, which resets the preferences set.
If my webcam is on its side, with the capture rotated 90°, the Face Tracker no longer works. Is it possible to make the tracker function if the webcam is rotated?
The idea comes from a discussion on the forum.
https://obsproject.com/forum/threads/face-tracker.144419/post-543304
THX , great plugin! & sorry for abusing issue tracking for feature requests:
I am building a multicam virtual studio in OBS. After some tweaking the tracker works quite well on 4K
live input, yet - as expected - I run into problems as soon as there are multiple faces in the picture, and it uses about 50% CPU.
I wish I could run the tracker on separate layer with just the raw live input - possibly down scaled to save CPU -
then apply translation on the group that contains my complete scene the way the "Move Source" plugin that comes with
move transition works: https://obsproject.com/forum/resources/move-transition.913/
We would need X/Y scale and offset to make this work I on scenes, where live input is not same size as the scene itself.
This might be useful for additional smoothing:
https://cristal.univ-lille.fr/~casiez/1euro/
The idea comes from a discussion on the forum.
https://obsproject.com/forum/threads/face-tracker.144419/post-543304
Hello there!
I am planning to use this plugin to convert the 16:9 live video to 9:16 for streaming to Instagram. (I am also streaming at 16:9 on another computer to other platforms.) So it would be great to add a 9:16 aspect ratio to the existing list of the final video dimensions. Now the closest aspect ratio to portrait is 4:3. It would be also great to add a possibility not to zoom at all (now the minium zoom is 0,10). With these 2 little improvements, I guess this plugin could serve very well on purposes when wanting to track a face in landscape video to convert to portrait video.
Thank you very much for developing this plugin!
Jeremias
When zooming from the most wide shot, it start to zoom toward the center at first, then gradually move to the face.
Expected behavior is zoom toward the face from the beginning.
OBS 27.01 installed on Mint 20.01 (Mate)
deb install puts .so in: /usr/lib/x86_64-linux-gnu/obs-plugins/obs-face-tracker.so
Needed to add link to the rest of the plugins for it to appear in OBS.
sudo ln -s /usr/lib/x86_64-linux-gnu/obs-plugins/obs-face-tracker.so /usr/lib/obs-plugins/
obs-face-tracker/src/texture-object.cpp
Lines 135 to 137 in f1d8464
This code should use video scaler (video_scaler_t
) from obs-studio so that any format can be converted.
Hi,
Great plugin detecting the face, it would be nice if rather than a zoomed out and digital zoom to the area of interest that the software would control a PTZ Camera to track and optimize the image resolution.
That is, the PTZ control would move the camera left/right/up/down and zoom in or out to help maximize the persons face.
Certainly it would be nice to extend this to bodies as people might not be facing the camera and if there are multiple bodies to include as many as possible.
Thanks for the work on this!
The face tracker is not working, I just compiled using the same instructions as the azure pipline for linux
(though I am using manjaro, not ubuntu)
The regular face tracking filter does not work, I tried the PTZ version for fun to just see what it does, and when enabling the detection it detects me nicely
I tried both latest main and v0.2.2
I am using a NDI capture source for my webcam
logs for the regular tracker
17:06:23.122: [obs-face-tracker] face_detector_base: starting the thread.
17:06:23.122: [obs-face-tracker] Cannot load '(null)' (face-tracker.effect)
logs for the ptz tracker
17:06:44.669: [obs-face-tracker] face_detector_base: starting the thread.
17:06:44.696: [obs-face-tracker] received frame format=5
I hope this is enough, If you need me to collect more information, please let me know
If the resolution is too small to detect a face, the face tracker does not work.
I'd like to add an FAQ entry to describe how to apply face tracker filter for the low resolution sources.
Screenshot would be helpful to understand how to operate on OBS Studio.
The idea comes from a discussion on the forum.
https://obsproject.com/forum/threads/face-tracker.144419/post-543304
If the face is lost, it should gradually zoom-out or go back to the home or the preset position.
Depends #52.
I'd like to see the ability to use the tracker to apply an image or mask to the video source. The tracking is quite effective with appropriate lighting, and there is a lack of effective, straightforward plugins for OBS to use face coverings that track tightly.
To avoid OBS Studio from potential crashing, main algorithm should be separated from the OBS Studio.
Drawback will be slow data transfer if using pipe, complicated implementation if using shared memory.
Hi. I'm just a fan of obs-face-tracker.
Here is the most likely branch of OBS for Apple silicon and obs-face-tracker doesn't work because of architecture maybe.
obsproject/obs-studio#5155
Export detection and/or tracking results so that a user can control something through Lua script.
This idea could be an alternative of #57.
It will lso resolve this use case: https://obsproject.com/forum/threads/face-tracker.144419/post-541686
Even if the face is lost from the frame, the tracker is still remaining for a long time.
It should be cleaned up in a reasonable time.
#55 could be one of the solutions.
May not be supported but I'm running Monterey on a MBP M1. Everything looks good as far as installation goes but the plugin doesn't show up in the filters list for my camera.
This is a feature request to have a preset of parameters.
track_x/y/z
, scale_max
, and PID control parameters at once. Or, should I separate PID?Hello I have installed this plugin on both of my PCs but it will not show up in the filters list, Is there something I am missing?
Error log shows noting is wrong.
Windows 10
Latest OBS
In recent update, the number of the tracker threads keeps increasing. I should check it with bisect.
Can you provide a smoother description of the win 10 compilation process? It seems a bit tricky. Thank you.
I should suppress logs that is shown for each iteration
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.