Comments (15)
I think the "scale-only" makes sense and solves the problem. Especially since the number one thing shown in video from these cameras is probably a person and vertical crop is only going to make it zoom more. When turning native 16:9 into 4:3 then horizontal crop makes sense.
There are similar uncontrollable behaviors in params for screen sharing which might be solved by the same change. When I share my 4k screen when I know it will eventually be displayed on a 720p canvas, I may ask the browser for 720p in advance to spare the intense bandwidth of sending the native 4k over the peer connection which I know will be eventually downscaled anyway. This used to work and the share was letterboxed into the requested res was sent but at some point, the same interpretation of the spec results in scale and cropped screen instead of letterbox despite what combo of params are given.
from mediacapture-extensions.
Re-reading the thread, I think I agree that "scale-only" makes sense.
The issue here is that we have a camera capable of 2 modes, one close to the desired size, and the other having the desired aspect ratio.
of course, specifying "scale-only" on a browser that does not support it will be ignored (if ideal) or cause failure (if exact).
from mediacapture-extensions.
also a "letterbox" option so you can choose to downscale shared applications locally via screen share.
from mediacapture-extensions.
Despite the resizeMode "none" or "crop-and-scale" Chrome for instance will take the 4:3 640x480 and crop it to 16:9 ratio which effectively zooms the image leaving barely enough room for someone's head to fit.
I'd say this is a bug in Chrome. none
says: "This resolution is offered by the camera, its driver, or the OS."
There is not a way using the spec to insist that you want to scale down the nearest 16:9 res (1280x720) to 640x360
Even if you say you want an exact aspect of 16:9 it still chooses the crop technique.
Good point. I think I like the idea of standardizing a "scale-only"
mode better than user agents trying to infer a client's preference from whether they used exact
or not with aspectRatio
.
Note, if this is for web conferencing, you may be able to work around this problem by using scaleResolutionDownBy
.
from mediacapture-extensions.
Options:
- Work around it using
track.clone
and set one to a high-res 16:9 mode usingresizeMode: none
and the other to the resolution you want. This should force the implementation to open the higher res (in theory, may not work on all browsers yet) - File a bug on browsers to stop cropping 4:3 to 16:9 (though there's a performance cost I hear)
- Since user agents are ultimately allowed to choose which mode to pick, ask them to look at
aspectRatio
as a hint (might have poor web compat) - Define a new
scale-only
value in the spec specifically to disallow cropping.
from mediacapture-extensions.
@guidou What is a sensible way forward here?
from mediacapture-extensions.
I think the best way to achieve what @anthmFS wants is to define a "scale-only" resize mode.
It shouldn't be very difficult to implement if you already implement "crop-and-scale", since it's basically a "crop-and-scale" where you maintain the original aspect ratio (so that you don't have to crop).
from mediacapture-extensions.
Given a source of 640x480 (only):
If constraints are exact 640x360, then "scale-only" doesn't make any sense.
If constraints are ideal 640x360, then "crop-and-scale" will give you 640x360 cropped; "none" will give you 640x480.
If constraints are ideal w=640, then "crop-and-scale" shouldn't give you a reason to crop; aspect ratio can be preserved.
The middle case is the only one where "scale-only" makes a difference, I think.
from mediacapture-extensions.
Consider @anthmFS's case.
He wants 640x360 res, but the camera supports 640x480, 1280x720, and 1920x1080 and uses "crop-and-scale".
He wants the camera to be opened in 1280x720 with the track producing 640x360 using only scaling.
However Chromium opens it in 640x480 and the track produces 640x360 using only cropping.
Based on the spec, both strategies are perfectly valid. Chromium chooses the latter because it always tries to use the closest (and thus smallest) resolution (in size) that can be cropped-scaled to the ideal resolution. This is 100% spec compliant, however it is not what the user in this case would prefer, and I think preferring only scaling is a legitimate thing to want and there is no way to tell the browser to apply just scaling to the source frames.
from mediacapture-extensions.
Thanks, what would the next steps be?
from mediacapture-extensions.
Bug opened in Chrome
https://bugs.chromium.org/p/chromium/issues/detail?id=1380442
from mediacapture-extensions.
Any particular steps to make progress on this?
from mediacapture-extensions.
@guidou, What are good next steps? The folks in chromium are waiting for the spec to be clarified before they engage.
I opened this bug 3 years ago and everyone seems to think I have a reasonable point. I am a n00b at getting specs passed.
from mediacapture-extensions.
- Get someone to write a pull request on the spec adding "scale-only" mode with a description of what it does.
- Raise a Chromium bug asking them to implement what's (now) in the spec.
Because of our desire to get mediacapture-main pushed to REC, we are unlikely to add new features there at the moment, which is why this thread is on the mediacapture-extensions repo.
from mediacapture-extensions.
How about now, a year later? Who would that someone be btw?
from mediacapture-extensions.
Related Issues (20)
- [Track Stats API] Rename deliveredFrames to deliverableFrames HOT 2
- [Track Stats API] SameObject is a confusing API shape HOT 23
- [Track Stats API] When to initialize frame counting HOT 3
- Should web applications be aware of reaction effects added by OS to camera feeds? HOT 17
- [Audio Stats] Add current latency HOT 2
- Background Blur: Unprocessed video should be mandatory to support HOT 8
- [Track Stats API] Make stats attribute nullable instead of throwing when unsupported HOT 2
- [Stats] Example uses ratio, not percentage
- [Audio Stats] Add average, min and max latency HOT 5
- [Audio Stats] Disagreement about audio dropped counters HOT 19
- volume is not working HOT 1
- Move MediaStreamTrack stats in its own spec? HOT 10
- Clarify how `track.stats.resetLatency()` relates to run-to-completion semantics HOT 1
- https://w3c.github.io/mediacapture-extensions/#transferable-mediastreamtrack should talk about the MediaStreamTrack's application-set content hint
- Moving the source of a track HOT 1
- Should we add reasons to MediaStreamTrack.onended HOT 4
- Add a blackFrames counter to MediaStreamTrackVideoStats HOT 7
- Consider adding onVoiceActivity event on MediaStreamTrack for audio HOT 3
- How to select camera presets that have better power efficiency at the expense of quality? HOT 2
- Clarifying Transferable MediaStreamTrack Types HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mediacapture-extensions.