isaacs / inflight

Add callbacks to requests in flight to avoid async duplication

inflight

Add callbacks to requests in flight to avoid async duplication

USAGE

var inflight = require('inflight')

// some request that does some stuff
function req(key, callback) {
  // key is any random string.  like a url or filename or whatever.
  //
  // will return either a falsey value, indicating that the
  // request for this key is already in flight, or a new callback
  // which when called will call all callbacks passed to inflightk
  // with the same key
  callback = inflight(key, callback)

  // If we got a falsey value back, then there's already a req going
  if (!callback) return

  // this is where you'd fetch the url or whatever
  // callback is also once()-ified, so it can safely be assigned
  // to multiple events etc.  First call wins.
  setTimeout(function() {
    callback(null, key)
  }, 100)
}

// only assigns a single setTimeout
// when it dings, all cbs get called
req('foo', cb1)
req('foo', cb2)
req('foo', cb3)
req('foo', cb4)

inflight's Issues

Memory leak detected

Preliminary note. I didn't installed inflight directly, it's just a package that glob uses.

$ npm list inflight
[email protected] path/to/myLibrary
`-- [email protected]
  `-- [email protected]

I'm developing a CLI apps that scans a set of files in the user's disk. Sometimes, it stops working throwing the following log:

<--- Last few GCs --->

[7828:000001CAE9906BF0]      729 ms: Mark-sweep 27.1 (94.3) -> [...] allocation failure GC in old space requested
[7828:000001CAE9906BF0]      754 ms: Mark-sweep 27.1 (94.3) -> [...]  last resort
[7828:000001CAE9906BF0]      780 ms: Mark-sweep 27.1 (32.3) -> [...] last resort


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 000002CEAE928799 <JSObject>
    1: _readdirEntries [path\to\myLibrary\node_modules\glob\glob.js:~559] [pc=000002F43C32198C](this=0000001978260B49 <Glob map = 000000777A7EB871>,abs=0000027C3514ED71 [...]

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

So, I've inspected the Node's process' heap usage (node --inspect myLibrary.js) and got the following result, in which the inflight module is the most expensive package in terms of retained size.

probably an npm thing but missing dep error

I saw this error after an npm install:

Error: Cannot find module 'wrappy'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:581:15)
    at Function.Module._load (internal/modules/cjs/loader.js:507:25)
    at Module.require (internal/modules/cjs/loader.js:637:17)
    at require (internal/modules/cjs/helpers.js:20:18)
    at Object.<anonymous> (/home/oleg/codes/sce/suman-chrome-extension/node_modules/inflight/inflight.js:1:14)
    at Module._compile (internal/modules/cjs/loader.js:689:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:700:10)
    at Module.load (internal/modules/cjs/loader.js:599:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
    at Function.Module._load (internal/modules/cjs/loader.js:530:3)

Here is the package.json for inflight:

{
  "_args": [
    [
      "[email protected]",
      "/home/oleg/codes/sce/suman-chrome-extension"
    ]
  ],
  "_development": true,
  "_from": "[email protected]",
  "_id": "[email protected]",
  "_inBundle": false,
  "_integrity": "sha1-Sb1jMdfQLQwJvJEKEHW6gWW1bfk=",
  "_location": "/inflight",
  "_phantomChildren": {},
  "_requested": {
    "type": "version",
    "registry": true,
    "raw": "[email protected]",
    "name": "inflight",
    "escapedName": "inflight",
    "rawSpec": "1.0.6",
    "saveSpec": null,
    "fetchSpec": "1.0.6"
  },
  "_requiredBy": [
    "/glob",
    "/glob-stream/glob",
    "/stylus/glob",
    "/true-case-path/glob"
  ],
  "_resolved": "https://registry.npmjs.org/inflight/-/inflight-1.0.6.tgz",
  "_spec": "1.0.6",
  "_where": "/home/oleg/codes/sce/suman-chrome-extension",
  "author": {
    "name": "Isaac Z. Schlueter",
    "email": "[email protected]",
    "url": "http://blog.izs.me/"
  },
  "bugs": {
    "url": "https://github.com/isaacs/inflight/issues"
  },
  "dependencies": {
    "once": "^1.3.0",
    "wrappy": "1"
  },
  "description": "Add callbacks to requests in flight to avoid async duplication",
  "devDependencies": {
    "tap": "^7.1.2"
  },
  "files": [
    "inflight.js"
  ],
  "homepage": "https://github.com/isaacs/inflight",
  "license": "ISC",
  "main": "inflight.js",
  "name": "inflight",
  "repository": {
    "type": "git",
    "url": "git+https://github.com/npm/inflight.git"
  },
  "scripts": {
    "test": "tap test.js --100"
  },
  "version": "1.0.6"
}

Memory leak detected in version 1.0.6 by veracode

Veracode has detected a memory leak vulnerability

sometimes callbacks not called

When writing tests for libraries that use node-glob, I find that sometimes matches aren't emitted because I'm using the same file path in a bunch of tests. When I remove the inflight code from node-glob or run the tests individually, they all pass.

I saw that you made this commit about a year ago to handle when new callbacks are added. I'm not sure what a fix for this would be (or if it even belongs here).

CWE-772: Missing Release of Resource after Effective Lifetime

It seems like the most recent version of inflight has a vulnerability https://cwe.mitre.org/data/definitions/772.html