anthumchris / fetch-progress-indicators

Progress indicators/bars using Streams, Service Workers, and Fetch APIs

Examples for showing progress bars and progress indicators for fetch(). Uses the Fetch API, Streams API, and Service Worker API.

Demo

https://fetch-progress.anthum.com/

Examples

• Fetch: A ReadableStream is used to show download progress during a fetch() download.
• Fetch - Enhanced: Same as above with robust code for preventing multiple downloads and handling other real-world UI interactions and edge cases.
• Service Worker: A ReadableStream is used in a Service Worker to simulatenously show download progress for the FetchEvent of an inline <img> tag.

Gzip & Content-Encoding Support

• #13

Browser Support

The aforementioned APIs are new/expermiental and do not currently work on all browsers. Testing was done with the browsers below:

Browser	Test Results
Chrome 64	Full support
Firefox 58/59	Full support (requires activation of experimental flags)
iOS Safari 8	Unsupported
iOs Safari 11	Fetch support only. Service Workers unsupported
Mac Safari 11.0	Fetch support only. Service Workers unsupported
Mac Safari 11.1	Full Support
IE/Edge	Not tested (no device available)

Background

Prior to the recent addition of fetch(), the XMLHttpRequest.onprogress callback handler was traditionally used to show progress bars and progress indicators. The Fetch API is great and we should use it, but "onprogress" handlers are not currently implemented with Fetch. These examples allow us to leverage Fetch while providing users with immediate feedback during certain "loading" states. Progress indicators could be especially useful to users on slow networks.

Lessons & Conclusions

This repository began as a proof of concept for showing progress indicators from Service Workers. Everything seemed to work out and a few important lessons and caveats were discovered:

1. Firefox successfully stops network reading and cancels downloads on fetch events that implement custom ReadableStream readers when the user signals the cancelation/abort of a page load (e.g. pressing ESC, clicking stop button, invoking window.stop())
2. Chrome and Safari don't stop network reading and files continue to download when a page load cancel/abort occurs.
3. The abort event does not seem to be firing on Chrome, Firefox, or Safari as defined in the HTML spec 7.8.12 Aborting a document load.
   1. <img onabort> callbacks are not called.
   2. window.onabort callbacks are not called.
   3. see Abort Event Detection Test
4. A Firefox bug was discovered when using hash fragments in URLs: https://bugzilla.mozilla.org/show_bug.cgi?id=1443850

Back-End Image Server

To properly exemplify progress indicators for slow downloads or large files, a small (100kb) JPEG is being served from a remote HTTP/2 Nginx server that limits download speeds. The buffer/packet size is also reduced to show smoother, or more frequent progress updates (more iterations of ReadableStreamDefaultReader.read()). Otherwise, read() would send fewer progress updates that result in choppy progress indicators. Caching is disabled to force network requests for repeated tests.

Both Baseline and Progressive JPEG files are available for testing with other speeds:

https://fetch-progress.anthum.com/10kbps/images/sunrise-baseline.jpg
https://fetch-progress.anthum.com/20kbps/images/sunrise-baseline.jpg
https://fetch-progress.anthum.com/30kbps/images/sunrise-baseline.jpg
https://fetch-progress.anthum.com/60kbps/images/sunrise-baseline.jpg
https://fetch-progress.anthum.com/120kbps/images/sunrise-baseline.jpg

https://fetch-progress.anthum.com/10kbps/images/sunrise-progressive.jpg
https://fetch-progress.anthum.com/20kbps/images/sunrise-progressive.jpg
https://fetch-progress.anthum.com/30kbps/images/sunrise-progressive.jpg
https://fetch-progress.anthum.com/60kbps/images/sunrise-progressive.jpg
https://fetch-progress.anthum.com/120kbps/images/sunrise-progressive.jpg

fetch-progress-indicators's Issues

SSL expired

Progress incorrect when content is gzip encoded

The content length reported by the HTTP header is the encoded length, whereas the length accumulated in the code is decompressed:

https://github.com/AnthumChris/fetch-progress-indicators/blob/master/fetch-basic/supported-browser.js#L36

As a result, progress extends beyond 100%

Use Proper Restart feature

A "Reload" button currently exists to start the test over by reloading the entire page. This was a fast solution, and ideally a "Restart" or "Run Again" button should be implemented to re-start the test without full page reload.

ReadableStream.cancel() can be used, which stops the network transfer and also currently shows partially loaded images, which is kind of cool for testing/observation load differences between Progressive and Baseline JPEG images

sw-basic image doesn't display in Firefox 59.0+

As reported by @chrisdavidmills, image does not display in Firefox 59+, yet the image loads in the Service Worker as indicated in the progress bar.

Add Aggregate Download Progress for Multiple Assets

A SW could be used to show aggregate/total progress indicators for multiple assets that download individually as part dependencies for a PWA. Video games loading screens are good examples of this. For large downloads that would exceed a few seconds, it would make sense to load the SW first, refresh, and then show progress for multiple assets in transit.

This example should leverage HTTP2 to ensure that totals for all assets are gathered from the onset. Assumption is that multiple responses will come in before downloads complete, and that assumption should be tested with various "total asset" values (i.e. 5, 10, 20, 30). Various browsers and back-end Nginx server may effect the total files that could start to download, hence affecting the "total bytes" aggregate.

This recent originated from a recent StackOverflow question "Show progress while caching files using a service worker"

Add Example for inline `<img>` loading

Implement a Service Worker to intercept fetch events to show progress bars for inline <img> loading when src attribute is set. All existing fetch() calls should be removed and download progress monitoring should occur in the Service Worker

Make Mobile-Friendly

Development was lazily done on desktop, and these examples should be mobile-friendly/responsive and testable from phones

Add Example for File Upload

Progress indicators for file uploads (specifically indicators for Request objects) would be a great example. At the time of writing, this doesn't seem possible yet, and the Request docs do indicate ReadableStream may be used as a request body

String body: sent

// Post "hello" body.  Browser sends and server responds with what it receives.
fetch(
  new Request('https://dev.anthum.com/post-test/', {
    body: 'hello',
    headers: {
      'content-type': 'text/plain'
    },
    method: 'POST'
  })
)
.then(response => response.text())
.then(data => console.log(data))
.catch(error => console.error(error));

ReadableStream body: not sent

This may just be bad experimental code on my part or the browsers do not yet support it.

const stream = new ReadableStream({
  start(controller) {
    controller.enqueue(new TextEncoder().encode('hello'));
    controller.close();
  }
});

// Post ReadableStream body. Browser doesn't seem to send, and server responds with nothing received
fetch(
  new Request('https://dev.anthum.com/post-test/', {
    body: stream,
    headers: {
      'content-type': 'text/plain'
    },
    method: 'POST'
  })
)
.then(response => response.text())
.then(data => console.log(data))
.catch(error => console.error(error));

Mobile Safari Does Not Show Errors

Demos do not work on Mobile Safari (versions 8 tested) and no errors show. Ensure that proper errors show if certain JS APIs are not yet implemented

Possibility to pause the downlod

Hi thanks alot for the great work.Is it possible to puse the downlod ?

Simpler solution for fetch-basic

In your fetch-basic example you read the response then "pipe" it to a new response:

...
return new Response(
    new ReadableStream({
      start(controller) {
        const reader = response.body.getReader();

        read();
        function read() {
          reader.read().then(({done, value}) => {
            if (done) {
              controller.close();
              return; 
            }
            loaded += value.byteLength;
            progress({loaded, total})
            controller.enqueue(value);
            read();
          }).catch(error => {
            console.error(error);
            controller.error(error)                  
          })
        }
      }
    })
  );
})
.then(response => response.blob())
.then(data => {
  status('download completed')
  document.getElementById('img').src = URL.createObjectURL(data);
})
...

There is no need to perform this extra work:

const content = []; // <===
let bytesReceived = 0;

while (true) {
  const { done, value } = await reader.read();
  if (done) break;

  content.push(value); // <===
  bytesReceived += value.byteLength;
}

document.getElementById('img').src = URL.createObjectURL(new Blob(content));

Full example:

const progressIndicator = document.getElementById('progress');

const { headers, body } = await fetch(
  'https://fetch-progress.anthum.com/30kbps/images/sunrise-baseline.jpg'
);

const contentLength = headers.get('Content-Length');
if (contentLength === null) throw new Error('No Content-Length response header');
const totalBytes = parseInt(contentLength, 10);
progressIndicator.max = totalBytes;

if (body === null) throw new Error('No response body');
const reader = body.getReader();

const content = []; // <===
let bytesReceived = 0;

while (true) {
  const { done, value } = await reader.read();
  if (done) break;

  content.push(value); // <===
  bytesReceived += value.byteLength;

  progressIndicator.value = bytesReceived;
}

document.getElementById('img').src = URL.createObjectURL(new Blob(content));

<progress id="progress" value="0"></progress>
<img id="img" alt="download-progress-img" src="data:,">

Use TransformStream in Examples

Refactor with TransformStream as described by @ricea:

The examples would be cleaner with TransformStream, but unfortunately no browser is shipping it yet. You could be really concise, something like (totally untested)

fetchEvent.respondWith(new Response(response.body.pipeThrough(new TransformStream({
  transform(chunk, controller) {
    loaded += chunk.length;
    progress({loaded, total});
    controller.enqueue(chunk);
  }
})), response));

Network Error does not show in UI

1. Start Download
2. Disable network (turn off WiFi)
3. Error below occurs and does not show message to user in HTML.

Headers from original request not forwarded

Hi there -- thanks for these examples. One improvement I suggest is to show how to forward the headers from the original request:

return new Response(stream, { headers: response.headers });