Add ability to not remove finished downloads from panel about download_panel_tweaker

[Infocatcher]

Test version: download_panel_tweaker-0.1.1pre5-fx.xpi (source).

[Infocatcher]

Doesn't work in Firefox 27.0a1 (2013-09-19), but some downloads (at least saved pages) aren't saved in Library without any extensions.
And seems like browser.download.useJSTransfer = false can be used as a workaround.

[Infocatcher]

Unfortunately doesn't work since Firefox 26.0. :(

[Infocatcher]

resource://gre/modules/DownloadStore.jsm

this.DownloadStore.prototype = {
  ...
  /**
   * This function is called with a Download object as its first argument, and
   * should return true if the item should be saved.
   */
  onsaveitem: () => true,
  ...
  /**
   * Saves persistent downloads from the list to the file.
   *
   * If an error occurs, the previous file is not deleted.
   *
   * @return {Promise}
   * @resolves When the operation finished successfully.
   * @rejects JavaScript exception.
   */
  save: function DS_save()
  {
    return Task.spawn(function task_DS_save() {
      let downloads = yield this.list.getAll();
      ...
      for (let download of downloads) {
        try {
          if (!this.onsaveitem(download)) {
            continue;
          }

resource://gre/modules/DownloadIntegration.jsm

this.DownloadIntegration = {
  ...
  initializePublicDownloadList: function(aList) {
    return Task.spawn(function task_DI_initializePublicDownloadList() {
      ...
      this._store.onsaveitem = this.shouldPersistDownload.bind(this);
  ...
  /**
   * Determines if a Download object from the list of persistent downloads
   * should be saved into a file, so that it can be restored across sessions.
   *
   * This function allows filtering out downloads that the host application is
   * not interested in persisting across sessions, for example downloads that
   * finished successfully.
   *
   * @param aDownload
   *        The Download object to be inspected.  This is originally taken from
   *        the global DownloadList object for downloads that were not started
   *        from a private browsing window.  The item may have been removed
   *        from the list since the save operation started, though in this case
   *        the save operation will be repeated later.
   *
   * @return True to save the download, false otherwise.
   */
  shouldPersistDownload: function (aDownload)
  {
    // In the default implementation, we save all the downloads currently in
    // progress, as well as stopped downloads for which we retained partially
    // downloaded data.  Stopped downloads for which we don't need to track the
    // presence of a ".part" file are only retained in the browser history.
    // On b2g, we keep a few days of history.
//@line 319 "c:\builds\moz2_slave\m-cen-w32-ntly-000000000000000\build\toolkit\components\jsdownloads\src\DownloadIntegration.jsm"
    return aDownload.hasPartialData || !aDownload.stopped;
//@line 321 "c:\builds\moz2_slave\m-cen-w32-ntly-000000000000000\build\toolkit\components\jsdownloads\src\DownloadIntegration.jsm"
  },

http://mxr.mozilla.org/mozilla-central/source/toolkit/components/jsdownloads/src/DownloadIntegration.jsm#305

305   shouldPersistDownload: function (aDownload)
306   {
307     // In the default implementation, we save all the downloads currently in
308     // progress, as well as stopped downloads for which we retained partially
309     // downloaded data.  Stopped downloads for which we don't need to track the
310     // presence of a ".part" file are only retained in the browser history.
311     // On b2g, we keep a few days of history.
312 #ifdef MOZ_B2G
313     let maxTime = Date.now() -
314       Services.prefs.getIntPref("dom.downloads.max_retention_days") * 24 * 60 * 60 * 1000;
315     return (aDownload.startTime > maxTime) ||
316            aDownload.hasPartialData ||
317            !aDownload.stopped;
318 #else
319     return aDownload.hasPartialData || !aDownload.stopped;
320 #endif
321   },

So, we can override DownloadIntegration.shouldPersistDownload like in #ifdef MOZ_B2G case.

[Infocatcher]

Test version: download_panel_tweaker-0.1.1pre8-fx.xpi (source).

[Infocatcher]

Bug: for all downloads from downloads.json displayed "0 bytes" after restart (data saved, like "totalBytes": 953186, but not displayed).

[Infocatcher]

Bug: for all downloads from downloads.json displayed "0 bytes" after restart

Also displayed startup time instead of download time (and unfortunately we have only "startTime" field in downloads.json).
See resource://app/modules/DownloadsCommon.jsm:

/**
 * Represents a single item in the list of downloads.
 *
 * The endTime property is initialized to the current date and time.
 *
 * @param aDownload
 *        The Download object with the current state.
 */
function DownloadsDataItem(aDownload)
{
  this._download = aDownload;
  ...
  this.endTime = Date.now();

  this.updateFromDownload();
}

This should be fixed in 8b0bc4e commit.

[Infocatcher]

Test version: download_panel_tweaker-0.1.1pre9-fx.xpi (source).
File was updated, see below.

[Infocatcher]

Oops, this works only in Firefox 28+. :(
In Firefox 26:
resource://app/modules/DownloadsCommon.jsm

/**
 * Represents a single item in the list of downloads.  This object either wraps
 * an existing nsIDownload from the Download Manager, or provides the same
 * information read directly from the downloads database, with the possibility
 * of querying the nsIDownload lazily, for performance reasons.
 *
 * @param aSource
 *        Object containing the data with which the item should be initialized.
 *        This should implement either nsIDownload or mozIStorageRow.  If the
 *        JavaScript API for downloads is enabled, this is a Download object.
 */
function DownloadsDataItem(aSource)
{
  if (DownloadsCommon.useJSTransfer) {
    this._initFromJSDownload(aSource);
  } else if (aSource instanceof Ci.nsIDownload) {
    this._initFromDownload(aSource);
  } else {
    this._initFromDataRow(aSource);
  }
}

  /**
   * Initializes this object from the JavaScript API for downloads.
   *
   * The endTime property is initialized to the current date and time.
   *
   * @param aDownload
   *        The Download object with the current state.
   */
  _initFromJSDownload: function (aDownload)
  {
    this._download = aDownload;
    ...
    this.endTime = Date.now();

    this.updateFromJSDownload();
  },

[Infocatcher]

Test version: download_panel_tweaker-0.1.1pre9-fx.xpi (source).

[Infocatcher]

Something went wrong with many entries in downloads.json: list becomes empty after restart + no download notifications for new downloads. But all works fine with the same downloads.json with less entries count. E.g. bugs with 35 entries, but all OK with 34.

[Infocatcher]

To cleanup downloads.json manually:

var {DownloadIntegration} = Components.utils.import("resource://gre/modules/DownloadIntegration.jsm", {});
DownloadIntegration._store.save();

(devtools.chrome.enabled = true, Web Developer – Scratchpad – Environment – Browser)

[Infocatcher]

Test version: download_panel_tweaker-0.1.1pre10-fx.xpi (source).

[Infocatcher]

Also downloads.json doesn't saved (sometimes?) after removing of finished download item from panel. And sometimes doesn't saved after removing of browsing and download history (using History – Clear Recent History…).
Probably due to some internal optimizations...
So, we can call

var {DownloadIntegration} = Components.utils.import("resource://gre/modules/DownloadIntegration.jsm", {});
DownloadIntegration._store.save();

manually in some cases.

[Infocatcher]

Test version: download_panel_tweaker-0.1.1pre11-fx.xpi (source).

[Infocatcher]

Test version: download_panel_tweaker-0.1.1pre12-fx.xpi (source).

[Infocatcher]

Something went wrong with many entries in downloads.json

Testcase: https://gist.github.com/Infocatcher/9066069

[Infocatcher]

Something went wrong with many entries in downloads.json

resource://gre/modules/DownloadStore.jsm, DownloadStore.prototype.load:

  load: function DS_load()
  {
    return Task.spawn(function task_DS_load() {
      let bytes;
      try {
        bytes = yield OS.File.read(this.path);
      } catch (ex if ex instanceof OS.File.Error && ex.becauseNoSuchFile) {
        // If the file does not exist, there are no downloads to load.
        return;
      }

      let storeData = JSON.parse(gTextDecoder.decode(bytes));

      // Create live downloads based on the static snapshot.
      for (let downloadData of storeData.list) {
        try {
          let download = yield Downloads.createDownload(downloadData);

=> resource://gre/modules/Downloads.jsm, Downloads.createDownload:

  createDownload: function D_createDownload(aProperties)
  {
    try {
      return Promise.resolve(Download.fromSerializable(aProperties));
    } catch (ex) {
      return Promise.reject(ex);
    }
  },

Call stack from Downloads.createDownload() wrapper (366 lines!):
https://gist.github.com/Infocatcher/9091608

Possible workaround, resource://gre/modules/DownloadStore.jsm, DownloadStore.prototype.load:

-          let download = yield Downloads.createDownload(downloadData);
+          let {Download} = Components.utils.import("resource://gre/modules/DownloadCore.jsm", {});
+          let download = Download.fromSerializable(downloadData);

[Infocatcher]

Possible workaround

Works better, but we still get "too much recursion" error for a bit larger data. :(

[Infocatcher]

Also added some fixes for startup performance.

resource://gre/modules/Downloads.jsm

  getList: function (aType)
  {
    if (!this._promiseListsInitialized) {
      this._promiseListsInitialized = Task.spawn(function () {
        let publicList = new DownloadList();
        let privateList = new DownloadList();
        let combinedList = new DownloadCombinedList(publicList, privateList);

        try {
          yield DownloadIntegration.addListObservers(publicList, false);
          yield DownloadIntegration.addListObservers(privateList, true);
          yield DownloadIntegration.initializePublicDownloadList(publicList);
        } catch (ex) {
          Cu.reportError(ex);
        }

=> resource://gre/modules/DownloadIntegration.jsm:

  initializePublicDownloadList: function(aList) {
    return Task.spawn(function task_DI_initializePublicDownloadList() {
      ...
      if (this._importedFromSqlite) {
        try {
          yield this._store.load();

=> resource://gre/modules/DownloadStore.jsm:

  load: function DS_load()
  {
    return Task.spawn(function task_DS_load() {
      ...
      // Create live downloads based on the static snapshot.
      for (let downloadData of storeData.list) {
        try {
          let download = yield Downloads.createDownload(downloadData);
          try {
            ...
          } finally {
            // Add the download to the list if we succeeded in creating it,
            // after we have updated its initial state.
            yield this.list.add(download);

=> resource://gre/modules/DownloadList.jsm:

  add: function DL_add(aDownload) {
    this._downloads.push(aDownload);
    aDownload.onchange = this._change.bind(this, aDownload);
    this._notifyAllViews("onDownloadAdded", aDownload);

    return Promise.resolve();
  },

And finally we (somehow... just thanks to built-in profiler) have many synchronous
processPendingRequests() (chrome://browser/content/downloads/downloads.js)
DownloadsIndicatorDataCtor.prototype.onDataItemAdded() -> _updateViews() (resource://app/modules/DownloadsCommon.jsm)
DownloadsDataCtor.prototype.onDownloadAdded() (resource://app/modules/DownloadsCommon.jsm)