Comments (5)
Whilst at an ML level Face Detection is 'just another segmentation problem', from the user's point of view it is somewhat more personal than detecting an orange - especially since the bulk use-case of WebRTC is video conferences and good background blur is an egalitarian feature.
I think that some exceptionalism for this use case is justified.
from mediacapture-extensions.
@aboba, can you clarify whether this issue is a blocker for the CFC?
AIUI, your suggestion seems like a request for API change, not a blocker for the API.
from mediacapture-extensions.
It's a request for a metadata change, so that we don't have to define metadata for segmentation in addition to metadata specific to face detection. If the encoder wants to utilize segmentation information to figure out where to spend its effort, it shouldn't have to be able to understand multiple metadata formats, each optimized for a particular use.
from mediacapture-extensions.
@aboba I agree that it would be better to define a generic segmentation metadata. We're happy to change the spec proposal once agreed on directions. What do you think of this:
partial dictionary VideoFrameMetadata {
sequence<Segment> segment;
};
dictionary Segment {
DOMString type; // One of enum SegmentType
long id;
long partOf; // References the parent segment id
float probability; // or confidence
Point2D? centerPoint;
DOMRectReadOnly? boundingBox;
// sequence<Point2D>? contour; // Possible future extension
};
enum SegmentType {
"human-face",
"left-eye",
"right-eye",
"mouth",
// To be extended later with other types of segments
};
from mediacapture-extensions.
This issue was mentioned in WEBRTCWG-2023-02-21 (Page 44)
from mediacapture-extensions.
Related Issues (20)
- [Track Stats API] Support all types of MediaStreamTrack HOT 3
- [Track Stats API] Remove `timestamp` HOT 1
- [Track Stats API] The stats API should be synchronous HOT 1
- [Track Stats API] Rename "videoStats" to "stats" HOT 1
- [Track Stats API] Rephrase sentence on when to update internal slots HOT 14
- [Track Stats API] Rename deliveredFrames to deliverableFrames HOT 2
- [Track Stats API] SameObject is a confusing API shape HOT 23
- [Track Stats API] When to initialize frame counting HOT 3
- Should web applications be aware of reaction effects added by OS to camera feeds? HOT 17
- [Audio Stats] Add current latency HOT 2
- Background Blur: Unprocessed video should be mandatory to support HOT 8
- [Track Stats API] Make stats attribute nullable instead of throwing when unsupported HOT 2
- [Stats] Example uses ratio, not percentage
- [Audio Stats] Add average, min and max latency HOT 5
- [Audio Stats] Disagreement about audio dropped counters HOT 19
- volume is not working HOT 1
- Move MediaStreamTrack stats in its own spec? HOT 10
- Clarify how `track.stats.resetLatency()` relates to run-to-completion semantics HOT 1
- https://w3c.github.io/mediacapture-extensions/#transferable-mediastreamtrack should talk about the MediaStreamTrack's application-set content hint
- Moving the source of a track HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mediacapture-extensions.