valyouw / njstrace Goto Github PK

This library was failing to parse files that contain the spread operator. I fixed it by updating the ecmaVersion passed to falafel to 9. I can submit a PR if you want.

Hey,
thank you for offering such a nice project! I appreciate the work you do with this module.

Currently, I'm trying to integrate njstrace in OWASP's juice-shop for a student project (Novice here – sorry ;)). I wrote a custom logger which works fine when I don't trace the node_modules. But when I do, it get following error:

(node:65302) UnhandledPromiseRejectionWarning: /Users/de1630871/Spielwiese/juice-shop/node_modules/graceful-fs/graceful-fs.js:595

var njsTmp9244 = fs$ReadStream.apply(this, arguments), this;
^^^^

SyntaxError: Unexpected token this
at new Script (vm.js:80:7)
at createScript (vm.js:274:10)
at Object.runInThisContext (vm.js:326:10)
at Module._compile (internal/modules/cjs/loader.js:664:28)
at Module._compile (/Users/de1630871/Spielwiese/juice-shop/node_modules/njstrace/njsTrace.js:158:15)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:712:10)
at Module.load (internal/modules/cjs/loader.js:600:32)
at tryModuleLoad (internal/modules/cjs/loader.js:539:12)
at Function.Module._load (internal/modules/cjs/loader.js:531:3)
at Module.require (internal/modules/cjs/loader.js:637:17)
(node:65302) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:65302) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

I forked the juice-shop-repo and made a minimal working version of the project:
https://github.com/christophschw/juice-shop

The formatter / chainloader:
https://github.com/christophschw/juice-shop/blob/master/njstrace.js
Comment out line 50 and it works ...

And the npm script is npm run start:njstrace

Thank you for your help!

If I understand correctly, the process of tracing Angular/TS/JS code executed in browser, while running app, isn't possible for njsTrace, because the code generated by Angular Dev Server (based on Webpack) is ESM, not CommonJS?
It's UI app not backend, native Node.js scripts, so it may not work, I guess?

Hey, me again ;)

I wonder if it would be possible to trace Node's core modules like fs or util. I know that this is a very unusual demand but it would me help to detect anomalies during malicious behavior in Node apps.
Since the source modules are implemented in Javascript, I thought it should be possible to extend the code with njsTrace, no?
https://github.com/nodejs/node/tree/master/lib
But I suppose that this not that trivial since core modules won't processed by Module._compiled, right?
Maybe, we could use the (deprecated) natives module in order to fetch and manipulate the source code of the core modules. What do you think?

Do you have any hints in order to meet this requirement?
Have a nice weekend & thanks again!

Hello, I'm trying to trace vm.Script which is executed in a custom vm context.

Do you know any workaround to trace it?

import jsdom from "jsdom";
import vm from "vm";
import fs from "fs";

const dom = new jsdom.JSDOM(``)

const test = new vm.Script(fs.readFileSync('./mymod.js', 'utf-8'));
test.runInContext(dom.getInternalVMContext())

How does NJSTrace handle async call?

I had to change } catch { to } catch(error) { for this to start working again.

This is valid javascript, I'm not sure which specific spec adds support but I've been using this with node 12 for quite some time.

try {
    // Handler non fatal errors
    $injector.resolve('globalErrorHandler')(coreError);
} catch {
    throw coreError;
}

> require('./app')
njsTrace: Instrumenting: /usr/local/node/graphql-api/app/index.js
njsTrace: ERROR: Error instrumenting file: /usr/local/node/graphql-api/app/index.js , Exception: { SyntaxError: Unexpected token (75:10)
    at Parser.pp$4.raise (/usr/local/node/graphql-api/node_modules/falafel/node_modules/acorn/dist/acorn.js:2757:13)
    at Parser.pp.unexpected (/usr/local/node/graphql-api/node_modules/falafel/node_modules/acorn/dist/acorn.js:647:8)
    at Parser.pp$1.parseTryStatement (/usr/local/node/graphql-api/node_modules/falafel/node_modules/acorn/dist/acorn.js:1018:49)
    at Parser.pp$1.parseStatement (/usr/local/node/graphql-api/node_modules/falafel/node_modules/acorn/dist/acorn.js:784:32)
    at Parser.pp$1.parseBlock (/usr/local/node/graphql-api/node_modules/falafel/node_modules/acorn/dist/acorn.js:1112:23)
    at Parser.pp$3.parseFunctionBody (/usr/local/node/graphql-api/node_modules/falafel/node_modules/acorn/dist/acorn.js:2600:22)
    at Parser.pp$3.parseArrowExpression (/usr/local/node/graphql-api/node_modules/falafel/node_modules/acorn/dist/acorn.js:2561:8)
    at Parser.pp$3.parseExprAtom (/usr/local/node/graphql-api/node_modules/falafel/node_modules/acorn/dist/acorn.js:2136:23)
    at Parser.pp$3.parseExprSubscripts (/usr/local/node/graphql-api/node_modules/falafel/node_modules/acorn/dist/acorn.js:2047:19)
    at Parser.pp$3.parseMaybeUnary (/usr/local/node/graphql-api/node_modules/falafel/node_modules/acorn/dist/acorn.js:2024:17)
  pos: 1935,
  loc: Position { line: 75, column: 10 },
  raisedAt: 1936 }
{}

Hello. I've fell into troubles using njsTrace with generators. If I remove the asterisk symbol from mymod.js or launch it with node directly, everything compiles fine.

main.js:

var njstrace = require('njstrace').inject(),
    mymod = require('./mymod.js');

// Use only 4 digits so the output would be easier to read
mymod.run();

mymod.js

exports.run = function() {
    var test = function*(){}
    console.log(123)
}

Output:

$ node main.js 
/home/me/tst/insp/main.js:5
mymod.run();
      ^

TypeError: mymod.run is not a function
    at Object.<anonymous> (/home/me/tst/insp/main.js:5:7)
    at Module._compile (module.js:541:32)
    at Object.Module._extensions..js (module.js:550:10)
    at Module.load (module.js:456:32)
    at tryModuleLoad (module.js:415:12)
    at Function.Module._load (module.js:407:3)
    at Function.Module.runMain (module.js:575:10)
    at startup (node.js:160:18)
    at node.js:445:3

I'm trying to add njsTrace to Renovate. It works fine when running with nodejs, but when using babel-node, which we use for debugging, I get the following stacktrace:

TypeError: this.log is not a function
    at NJSTrace.inject (/home/jamie/code/renovate/node_modules/njstrace/njsTrace.js:73:7)
    at levels (/home/jamie/code/renovate/lib/logger/index.ts:129:5)
    at Object.parseConfigs (/home/jamie/code/renovate/lib/config/index.ts:70:3)
    at process._tickCallback (internal/process/next_tick.js:68:7)
    at Function.Module.runMain (internal/modules/cjs/loader.js:834:11)
    at Object.<anonymous> (/home/jamie/code/renovate/node_modules/@babel/node/lib/_babel-node.js:180:21)
    at Module._compile (internal/modules/cjs/loader.js:775:14)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)
    at Function.Module.runMain (internal/modules/cjs/loader.js:831:12)
    at startup (internal/bootstrap/node.js:283:19)
    at bootstrapNodeJSCore (internal/bootstrap/node.js:623:3

Here's my PR so far renovatebot/renovate#5632.

Any suggestions or help would be greatly appreciated.

Hi,

It seems that it can track functions like:

() => { console.log('abc') }

but not like:

() => console.log('abc')

Regards,

this.config.files = config.files || this.config.files; // In arrays we want to replace, not extend

If this.config is not set, this throws an error.
https://github.com/ValYouW/njsTrace/blob/master/njsTrace.js#L68

It might be because you're extending config which can be undefined if you do

var njstrace = require('njstrace').inject()

A simple fix is to start with {}, like

extend(true, this.config, DEFAULT_CONFIG, config, {});

hey, this is an awesome project.

I want to extend this project to trace javascript code on each step of the execution of code. here's an example to understand my requirements better,

I've a simple js code that looks something like this,

const add = (a, b) => {
    return a + b;
}

console.log(add(5, 4))

I want to store the traced output in a JSON file, in structured data that should look something similar to this,

[
    {
        "line_number": 1,
        "code_on_line": "const add = (a, b) => {",
        "variables": [
            {
                "variable_name": "add",
                "variable_value": "undefined"
            },
            {
                "variable_name": "a",
                "variable_value": "undefined"
            },
            {
                "variable_name": "b",
                "variable_value": "undefined"
            }
        ]
    },
    {
        "line_number": 5,
        "code_on_line": "console.log(add(5, 4))",
        "variables": [
            {
                "variable_name": "add",
                "variable_value": "(a, b) => { return a + b; }"
            }
        ]
    },
    {
        "line_number": 2,
        "code_on_line": "    return a + b;",
        "variables": [
            {
                "variable_name": "a",
                "variable_value": 5
            },
            {
                "variable_name": "b",
                "variable_value": 4
            }
        ]
    },
    {
        "line_number": 2,
        "code_on_line": "    return a + b;",
        "variables": [
            {
                "variable_name": "a",
                "variable_value": 5
            },
            {
                "variable_name": "b",
                "variable_value": 4
            },
            {
                "variable_name": "Return value",
                "variable_value": 9
            }
        ]
    },
    {
        "line_number": 5,
        "code_on_line": "console.log(add(5, 4))",
        "variables": [
            {
                "variable_name": "add",
                "variable_value": "(a, b) => { return a + b; }"
            }
        ]
    }
]

ik this library works great for functions call and keeps track of the args and return values,
how can I go about tracing the code execution step-by-step while keeping track of variables' values?

thank you.

Hi,
I have a problem trying to make njsTrace include the files inside of the node_modules folder. I have tried giving inject the following parameters:
var njstrace = require('njstrace').inject({ files: ["**/*.js"], formatter: new TSVFormatter() });
Which should include all .js files; I also tried to include '**/node_modules/** in the files array. This should override the default negation and even when removing the default negation - it still does not work with the files inside of node_modules; only my own files are being traced.

I would love to continue with njsTrace as part of bachelor project, so do you have any idea of why this does not work?

I just got an error trying to use njstrace with bluebird. I set up a small project to reproduce the error in https://github.com/sanji-programmer/njstrace-bluebird-issue

The error is:

ReferenceError: njsEntryData is not defined
at Object. (/home/sanji/git/njstrace-bluebird-issue/node_modules/bluebird/js/release/util.js:999:34)
at Module._compile (internal/modules/cjs/loader.js:776:30)
at Module._compile (/home/sanji/git/njstrace-bluebird-issue/node_modules/njstrace/njsTrace.js:158:15)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:787:10)
at Module.load (internal/modules/cjs/loader.js:653:32)
at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
at Function.Module._load (internal/modules/cjs/loader.js:585:3)
at Module.require (internal/modules/cjs/loader.js:690:17)
at require (internal/modules/cjs/helpers.js:25:18)
at module.exports (/home/sanji/git/njstrace-bluebird-issue/node_modules/bluebird/js/release/promise.js:66:12)
npm ERR! Test failed. See above for more details.

Need to limit the size of the arguments printed.
Using JSON.stringify might throw errors (circular ref) and is not a good method.

Steps to reproduce:

1. Navigate to a subdirectory of your application.
2. Run your application with using its relative location: node ../app.js

Currently the "arguments" object is passed to onTraceEntry
Per this article: https://github.com/petkaantonov/bluebird/wiki/Optimization-killers#3-managing-arguments
a leaking of the arguments object like so will prevent v8 from optimizing the function, not good.

First of all. Awesome library!

I have been going through njsTrace and I'd like to use in in the context of running tests with libraries such as jest, mocha, vitest, jasmine, playwright, cypress, and etc.

My initial strategy would be run a script before running the tests.
In jest I would utilize setupFilesAfterEnv to setup the tests before running them.

The setup process would involve injecting the njsTrace injector to each file I matching a certain pattern in the project
Something inline with:

find . -name '*test.js' -exec sed -i '1s;^;/* Your tracing code here\n;' {} \;

That way, when running npm run test, each test will have the inject which will capture all the necessary tracing.

Then afterwards, the clean up script will run and remove the injected code into the test files using globalteardown.

One issue that I cannot figure out how to fix is how do I keep track on each test file and each test case being logged?

One approach would be to run all tests, check the reports for which tests that fail/pass and then inject njsTrace to those tests separately and execute them in isolation and thus getting proper tracing.
The main problem with this approach is that all testing frameworks require a customized testing reporter, however, there are open source projects helping out with common interface for test reporting.

Do you as, the author, have any other ideas or approaches that could be better, or anything in general really.

Update

One more question:
divergent-tracer uses ava and it runs onEntry and onExit when using a formatter. But those functions are not called when I use jest testing framework for some reason.

I am interested in seeing how the callstack of my express app looks like, but when I plugged njstrace there, I am not getting any trace, but for my local modules, it is working.

How to use for a full-fledged application.