Coder Social home page Coder Social logo

async-local-storage's Introduction

async-local-storage

!!!Please use AsyncLocalStorage instead of async-local-storage.

Build Status Coverage Status npm Github Releases

I want something like thread-local storage in threaded programming and async_hooks is usable in node.js 8.0, so there is an easy way to use thread-local.

API

const als = require('async-local-storage');
als.enable();
setTimeout(() => {
  als.scope();
  const id = randomBytes(8);
  als.set('id', id);
  delay().then(() => {
    assert.equal(als.get('id'), id);
    return readfilePromise(__filename);
  }).then(() => {
    assert.equal(als.get('id'), id);
    return superagent.get('http://www.baidu.com/');
  }).then(() => {
    assert.equal(als.get('id'), id);
  });
}, 100);

enable

enable the async hooks

const als = require('async-local-storage');
als.enable();

disable

disable the async hooks

const als = require('async-local-storage');
als.enable();
setTimeout(() => {
  als.disable();
}, 100);

size

get the size of storage

const als = require('async-local-storage');
als.enable();
setTimeout(() => {
  console.info(als.size());
}, 100);

scope

change the scope of call chain, it will be the call chain top (remove the parent of itself)

const als = require('async-local-storage');
const Koa = require('koa');
const assert = require('assert');

const app = new Koa();
app.use(async (ctx, next) => {
  const id = ctx.get('X-Request-Id');
  als.scope();
  als.set('id', id);
  await next();
});

app.use(async (ctx, next) => {
  const id = ctx.get('X-Request-Id');
  assert.equal(als.get('id'), id);
  await next();
});

app.use((ctx) => {
  ctx.body = 'OK';
});

set

set the value by key for the current id

  • key the key
  • value the value
  • linkedTop set the value linked to top
als.enable()
setTimeout(() => {
  als.scope();
  const id = randomBytes();
  setTimeout(() => {
    als.set('id', id, true);
  }, 1);
  setTimeout(() => {
    assert.equal(als.get('id'), id);
  }, 10);
}, 10);

get

get the value by key, if will find from parent, self --> parent --> parent, until the value is not undefined

  • key the key
als.enable();
setTimeout(() => {
  als.scope();
  const id = randomBytes();
  setTimeout(() => {
    als.set('id', id, true);
  }, 1);
  setTimeout(() => {
    assert.equal(als.get('id'), id);
  }, 10);
}, 10);

enableLinkedTop

enable linked top for default (default is disabled)

als.enable();
als.enableLinkedTop();
setTimeout(() => {
  als.scope();
  setTimeout(() => {
    // the same as als.set('id', 'a', true)
    als.set('id', 'a');
  }, 10);
}, 10);

disableLinkedTop

disable linked top for default

als.enable();
als.enableLinkedTop();
setTimeout(() => {
  als.disableLinkedTop();
  als.scope();
  setTimeout(() => {
    // the same as als.set('id', 'a', false)
    als.set('id', 'a');
  }, 10);
}, 10);

currentId

get the current id

const assert = require('assert');
als.enable();
setTimeout(() => {
  console.info(als.currentId());
}, 10);

use

get the use time of id

  • id The tigger id, default is als.currentId()
als.enable()
setTimeout(() => {
  const id = als.currentId();
  console.info(als.use(id));
}, 10);

enableCreateTime

enable create time of data, default is enabled.

als.enableCreateTime();

disableCreateTime

disable create time of data, it can save memory.

als.disableCreateTime();

async-local-storage's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

async-local-storage's Issues

How to delete some variables

i want delete each items immediately after use (clear after first use) but in read me, i did not see a way. disable() not useful because i want delete in multi step

Unexpected behaviour with Node version 12.1.0

Hey @vicanso
I came across a strange behaviour using als and NodeJS v12.1.0 It seems like the triggerIds are messed up a bit. I am not assuming your library is working bad or anything, I am just here to get some help as you probably know async_hooks better then me.

I created a sample application to reproduce the behaviour we have in our real codebase.

const app = require('connect')();
const request = require('request');
const als = require('async-local-storage');
const hooks = require('async_hooks');

als.enable();

app.use((req, res, next) => {
    als.set('foo', { b: 'bar' });

    next();
});

app.use((req, res) => {
    request.get('https://google.com', (err, resp, body) => {
        console.log('Response from google in ' + hooks.executionAsyncId() + ' triggered by ' + hooks.triggerAsyncId());
        const asd = als.get('foo');
        console.log(asd);
        return res.end(JSON.stringify(asd));
    });
});

//create node.js http server and listen on port
http.createServer(app).listen(3000, function() {
    // eslint-disable-next-line no-console
    console.log('Your server is listening on port 3000 (http://localhost:3000)');
});

I piped the logs to a file. It can be found here: https://pastebin.com/xyC5AhH4
I refreshed localhost:3000 once using chrome browser. (in the logs it looks like 2 requests arrived, but I guess it is not related to als at all)

Some part of the logs I think might be important:

4(TCPSERVERWRAP) init by 1
...
7(TCPWRAP) init by 4
8(Timeout) init by 7
9(HTTPINCOMINGMESSAGE) init by 7
10(Timeout) init by 7
11(TickObject) init by 7
destroy 11
set foo:[object Object] to 9
...
12(Immediate) init by 9
...
19(TickObject) init by 12
20(HTTPINCOMINGMESSAGE) init by 19
...
destroy 20
...
251(TickObject) init by 20
Response from google in 251 triggered by 20
252(TickObject) init by 251
get foo:undefined from 251
undefined

20 is destroyed before my callbacks are initiated by it, so the context set to 9 is lost at the point when the callback passed to request is called. Do you have any tip on how to resolve this problem?
We would like to use async_hooks and your lib to track request ids during our logging without the need to pass a context object everywhere.

Thanks in advance,
Daniel

Edit: using node v10.15.3 (npm v6.9.0) the lib works just as expected

Unexpected crash with "maximum call stack size exceeded" error

The problem occurs locally, but I'm worried about possibility of reproducing this on production servers
version 1.2.0

(node:65032) UnhandledPromiseRejectionWarning: RangeError: Maximum call stack size exceeded
    at get (/Users/mac/project/node_modules/async-local-storage/index.js:21:13)
    at get (/Users/mac/project/node_modules/async-local-storage/index.js:28:12)
    at get (/Users/mac/project/node_modules/async-local-storage/index.js:28:12)
    at get (/Users/mac/project/node_modules/async-local-storage/index.js:28:12)
    at get (/Users/mac/project/node_modules/async-local-storage/index.js:28:12)
    at get (/Users/mac/project/node_modules/async-local-storage/index.js:28:12)
    at get (/Users/mac/project/node_modules/async-local-storage/index.js:28:12)
    at get (/Users/mac/project/node_modules/async-local-storage/index.js:28:12)
    at get (/Users/mac/project/node_modules/async-local-storage/index.js:28:12)
    at get (/Users/mac/project/node_modules/async-local-storage/index.js:28:12)
    at get (/Users/mac/project/node_modules/async-local-storage/index.js:28:12)
    at get (/Users/mac/project/node_modules/async-local-storage/index.js:28:12)
    at get (/Users/mac/project/node_modules/async-local-storage/index.js:28:12)
    at get (/Users/mac/project/node_modules/async-local-storage/index.js:28:12)
    at get (/Users/mac/project/node_modules/async-local-storage/index.js:28:12)
    at get (/Users/mac/project/node_modules/async-local-storage/index.js:28:12)
    at get (/Users/mac/project/node_modules/async-local-storage/index.js:28:12)
    at get (/Users/mac/project/node_modules/async-local-storage/index.js:28:12)
    at get (/Users/mac/project/node_modules/async-local-storage/index.js:28:12)
    at get (/Users/mac/project/node_modules/async-local-storage/index.js:28:12)
    at get (/Users/mac/project/node_modules/async-local-storage/index.js:28:12)
    at get (/Users/mac/project/node_modules/async-local-storage/index.js:28:12)
(node:65032) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)

Should it be destoyed when Promise.resolve is invoked?

when the resolve function is invoked, the promiseResolve callback function will be called.
I used your library in my project, and got memory leak in the end.
I think the reason is promiseResolve isn't handled. Am I right?

Garbage Collection

Inspecting the data map over multiple requests with the koa example reveals that the map is always growing in size, not just in keys, but the data is left lying around too. This will lead to larger and larger memory use over time, despite those continuations being finished. Is there anything that can be done about this? Using remove doesn't make the problem go away.

I'm guessing that since it's not possible to know if async functions have finished with the storage yet that we're forced to leave it lying around?

TickObject

What is the reason behind:

if (type === 'TickObject') {
  return;
}

during asyncHooks.createHook - init

Support to setting all the data in top parent

app.use((ctx, next) => {
  setImmediate(() => {
    als.set('immediate', true);
    console.info(`set immediate function: ${als.get('name')}`);
  });
  return next();
});

app.use((ctx, next) => {
  return request.get('https://www.baidu.com/').then(() => {
    console.info(als.get('immediate'));
    console.info(`promise function: ${als.get('name')}`);
    return next();
  });
});

I set the data in setImmediate function callback, then the function at the same level can't get the data, because the design is just for parent-child.

thread local test

RT:

const als = require('async-local-storage');
als.enable();
let sleep = n => {
    return new Promise(ok => {
        setTimeout(() => ok(), n);
    });
};
let doit = async (n) => {
    await sleep(3000);
    console.log(`doit ${n}: ${als.get('id')}`);
};

global.__a = 0;
setInterval(async () => {
    let value = Math.random();
    als.set('id', value);
    console.log(`a: ${value}`);
    await doit('a');
}, 1000);
setInterval(async() => {
    let value = Math.random();
    als.set('id', value);
    console.log(`b: ${value}`);
    await doit('b');
}, 1000);

// print
a: 0.37037671460949206
b: 0.9374930852905572
a: 0.4802650440623768
b: 0.8712013565643415
a: 0.31939900533078425
b: 0.2948936063257914
doit a: 0.31939900533078425
doit b: 0.2948936063257914
a: 0.0540918765065832
b: 0.6340173327025751
doit a: 0.0540918765065832
doit b: 0.6340173327025751
a: 0.6644920305032487
b: 0.15058185161722482
doit a: 0.6644920305032487
doit b: 0.15058185161722482
a: 0.954869207760314
b: 0.24920446605439084

其中 :
a: 0.37037671460949206
b: 0.9374930852905572
a: 0.4802650440623768
b: 0.8712013565643415
这几个消失了。。。,请问方便讨论下么?

state stored in "global variable"

Dear @vicanso,

I am really interested in your library as it would help us track a request flow in our logs in an elegant way. I was checking the implementation, and I noticed currentId is stored in a simple let. https://github.com/vicanso/async-local-storage/blob/master/als.js#L55
My question is, what happens if multiple requests come in and calls the before async hook while another request is "in the system". The old currentId is always overwritten when a new request comes in, right?
My understanding of async hooks is limited, so there is probably something I don't see, so I would be really happy if you could explain this part, and how currentId stays valid, during parallel requests.
Thanks in advance,
Daniel

Build is failing

I have seen the error in Travis build:

1) async-local-storage get id(http) success:
    Error: Timeout of 2000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.

Does it mean the package is malfunctioning?

Can this be used in api calls?

Please carify if this can be used in node APIs where the methods from different files are being called in a single api and should carry some dynamic value throughout the files to be used in all methods.

Is there a reason this won't work on AWS Lambda?

I'm attempting to run this in a Lambda. All works fine locally and on build server from the perspective of unit tests and some functional tests. However when running in Lambda the simplest set/get won't work:

// index.ts

const als = require('async-local-storage');
als.enable();
// ... some code
exports.handler = async (event: APIGatewayEvent, context: APIGatewayEventRequestContext) => {
  // some code
  als.set('blah', '123');
  const blah = als.get('blah');
  console.log(`blah ${blah}`);
}

The console above shows blah null instead of blah 123.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.