Comments (18)
I've started a PR at GoogleChrome/web.dev#7248 to update our documentation. Note that I've updated the snippet to wait for last remaining transfers. Let me know if that works better for you @demonguy .
It's really honored that i can contribute to webUSB in this way. And your example codes hint me that i forget promise.all
I found this by trying to develop an internal android image flash tool like https://flash.android.com/welcome
Very glad that i can help with this:)
from webusb.
Some operating systems impose limitations on how much data can be part of pending USB transactions. If you check about://device-log you should see the low-level operating system error which triggered the DOMException
. I recommend splitting your data into smaller transactions and only submitting a few at a time. In addition to avoiding operating system limitations it will also reduce the amount of memory used and allow your application to report progress as the transfers complete.
from webusb.
Is this limitation for chrome? If I transfer 500 MB of files from the command line, it will work.
from webusb.
I think he means that Google fastboot is sending 500MB buffer to usb everytime.
Here is the code
You can see it uses static constexpr uint32_t MAX_MAP_SIZE = 512 * 1024 * 1024;
But after detailed review, the real usb_linux.cpp uses 16KB buffer to send it. So case closed
from webusb.
thanks all.
from webusb.
but is there any webAPI to check os limit?
The thing is, if we use less size buffer, the total transfer time is 2 times of linux native fastboot command. I don't get why?
from webusb.
Here is the experiment
- Google native linux fastboot, usb buffer is 16KB, and takes 6 seconds to send around 100MB data
- WebUSB fastboot, usb buffer is also 16KB, and takes 13 seconds to send around 100MB data
- WebUSB fastboot, usb buffer is 15MB, and takes only 5.5 seconds to send around 100MB data.
sample code like this
console.time("transfer")
await this.device.transferOut(this.epOut, chunk);
console.timeEnd("transfer")
if chunk is 16KB buffer, the time is about 0.88ms ~ 1.00 ms
and if the chunk is 15MB buffer, the time is about 522ms ~ 536 ms.
you may notice that 15MB is 960 time of 16KB, but 522ms is only 593 times of 0.88 ms. almost doubled.
Is there any potential low efficient implementation in usbDevice.TransferOut
?
from webusb.
The additional time is likely due to the overhead of submitting and asynchronously waiting for all those USB transfers to complete. The native fastboot utility uses the Linux USB API in synchronous mode and so likely has lower latency, A good trick for reducing the latency of USB transfers is to have multiple in flight so instead of sending only one 16KB transfer at a time send two or three and replace them with the next transfer when each completes. That's obviously more complicated but could easily be encapsulated into a helper function.
from webusb.
I didn't get it.
If i send multiple packet in one time, is there anyway to make sure the packet is sent in order? Is there any code snippet?
Do you mean like this
p1 = this.device.transferOut(this.epOut, chunk1);
p2 = this.device.transferOut(this.epOut, chunk2);
p3 = this.device.transferOut(this.epOut, chunk3);
// wait p1 p2 p3 to be resolved
p4 = this.device.transferOut(this.epOut, chunk4);
p5 = this.device.transferOut(this.epOut, chunk5);
p6 = this.device.transferOut(this.epOut, chunk6);
from webusb.
When you submit multiple transfers to an endpoint they will execute in order. It builds up a queue of data for the USB host controller to transmit to the device. Giving the host controller multiple queued chunks improves performance because it doesn't have to involve your code (or even the operating system) to find the next chunk. Each time a chunk is fully transmitted it will notify your code that it should provide more data.
These tests provide a couple examples of submitting multiple transfers like this. The purpose of the tests is in fact to check that they execute in the correct order. These examples exercise transferIn()
but the principle is the same for transferOut()
.
In the first example there are only 3 transfers and so we wait for completion explicitly. The second example is more dynamic and keeps a configurable number of transfers in flight, submitting a new one whenever one completes in order to keep the same number going all the time. In the case of transferOut()
you'd stop submitting new transfers once you reach the end of the buffer.
from webusb.
I see you mentioned about "limit" 3 transfer and wait the first one to finish, and then push the fourth one.
If i have 1000 transfers to complete, this kind of implementation sounds hard.
Is it possible i just call 1000 times transferout and directly use promise.all?
I'm not sure if there any sideeffect of calling a big amoung of transferout
from webusb.
I believe that for platforms which limit transfer size the limitation applies to the total number of bytes across all pending transfers and so submitting 1000 transfers all at once wouldn't help you stay under the limit. The tests linked above have an example of the necessary code and it's not all that complicated.
from webusb.
i tried like this. and it works perfectly. Thanks
async _sendRawPayload(buffer) {
let i = 0;
let transferList = []
let maxTransfers = 3
let remainingBytes = buffer.byteLength;
while (remainingBytes > 0) {
let chunk = buffer.subarray(i * BULK_TRANSFER_SIZE, (i + 1) * BULK_TRANSFER_SIZE);
if (transferList.length == maxTransfers) {
await transferList[0]
transferList.shift()
}
transferList.push(this.device.transferOut(this.epOut, chunk))
remainingBytes -= chunk.byteLength;
i += 1;
}
}
from webusb.
Just to satisfy my curiosity, how does this version compare to your earlier benchmarks?
from webusb.
I flashed 7.3 GB file to android device. (since transfer and flash process are both involved in the benchmarks. the real benefit should be more than 30%)
- 16KB buffer one by one: 9 minutes 8 second
- 16KB buffer for 3 parrallel transfers: 6 minutes 2 second
- 16KB buffer of native fastboot command: 7 minutes 55 seconds
This makes me amazed that it's even faster than native fastboot. I think this really should be in the document.
from webusb.
CC @beaufortfrancois to get this "one weird trick" into our WebUSB developer documentation.
from webusb.
I've started a PR at GoogleChrome/web.dev#7248 to update our documentation. Note that I've updated the snippet to wait for last remaining transfers. Let me know if that works better for you @demonguy .
from webusb.
And here it is: https://web.dev/usb/#limits-on-transfer-size
from webusb.
Related Issues (20)
- bScheme = 0xff do not show notification HOT 2
- BYOB support for controlTransferIn, transferIn and isochronousTransferIn HOT 2
- Why is the exception type of claiminterface() is DOMException? HOT 6
- Can't detect device disconnection when `transferIn` is waiting incoming data HOT 1
- SerialPort.open() bufferSize option specifies the upper limit of the transfer size? HOT 1
- A transfer error has occurred. HOT 3
- Support Landing Page for Any Device HOT 2
- Feature request. On device webpage support.
- Feature request: transferIn listener HOT 2
- device serial number exposes a fixed, global ID HOT 6
- Web demo cannot connect device
- Exclusive filter in `requestDevice` HOT 5
- Unify the behaviour between `requestDevice()` and the Permissions API HOT 2
- [Feature request] Expose device speed HOT 6
- Clarify wording on object equality HOT 2
- Connecting to 2 usb devices in the same web app HOT 1
- Allow shared array buffers
- WinUSB driver HOT 1
- USB connected but cannot transferIn or transferOUt HOT 1
- Detaching a (Linux) kernel driver HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from webusb.