Memory leak about dmn

[Trott]

Updated to a version of globby that's only two years out of date rather than four, but it doesn't fix the problem.

[silverwind]

Confirmed. The module also got like 10x faster with this change, great work!

[Trott]

By the way, I'm kind of tempted to rewrite the module, probably under a new name and with the only functionality of cleaning node_modules 😉

If it matters, there's already other modules that do that like https://www.npmjs.com/package/modclean.

[Trott]

Looks like the problem was introduced some time relatively recently. I don't have the problem with npx [email protected] clean -f but I see it in npx [email protected].

[Trott]

Much to my surprise, git bisect reports this as the culprit: 782348b

Unfortunately, that's almost certainly not the code that caused the problem, but the code that exposes the problem in this situation.

[Trott]

Moved all .ts and .map files in the created react app to .pyc so that the problem would manifest in older versions.

Now the problem is being reported in 857d6b5 which seems also like it almost certainly is not the cause, but simply exposing the issue in this situation....

This might go way back, and seems like it might be in globby (which we're using a 4-year-old version of) or in the way globby is used here.

[silverwind]

FWIW, it's possible to workaround at least for the react case by increasing V8 heap size:

NODE_OPTIONS=--max_old_space_size=4096 npx dmn clean -f

I see the process peaking up to around 3GB, but then some GC takes place and it falls to 1.5GB for a while.

Probably need to attach a debugger and inspect the heap using a breakpoint.

[Trott]

globby uses fast-glob under the hood so I switched to fast-glob since we're not using any of the features globby adds. Then I updated to the latest fast-glob. I had to add an option to the calls to glob(). (That probably would have fixed it for globby too.) Leak appears to be gone now. Will publish 2.2.1 with the fix soon. If you want to test against current master, please do!

[Trott]

Fixed in be25a9b and released in [email protected].