Nx CLI is very slow when local plugins implement `createNodes` about nx

[kylebjordahl]

After more tinkering with my perf-test plugin, I have discovered that if I build the plugin and place it within the node_modules dir to approximate installing a 3rd party plugin, it loads remarkably fast. (Logs below)

It seems there is something wrong with loading local TS plugins

workspace git:(kb/RN-769-upgrade-to-nx-19) time NX_PERF_LOGGING=true NX_DAEMON=false NX_CACHE_PROJECT_GRAPH=false NX_ISOLATE_PLUGINS=true pnpm nx graph --file=tmp_file3.json --verbose

Time for 'loading dotenv files' 1.1174579999999992
Time for 'workspace context init' 3.954625000000007
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected]/node_modules/nx/src/plugins/project-json/build-nodes/package-json-next-to-project-json' 0.44529099999999744
Time for 'Load Nx Plugin: @nx/eslint/plugin' 2.4542920000000095
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected]/node_modules/nx/src/plugins/target-defaults/target-defaults-plugin' 2.9343749999999886
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected]/node_modules/nx/src/plugins/js' 7.256458000000009
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected]/node_modules/nx/src/plugins/package-json-workspaces' 0.47204099999999016
Time for 'Load Nx Plugin: @nx/vite/plugin' 254.830208
Time for 'Load Nx Plugin: @rh-plugins/perf-test' 51.146124999999984
Time for 'Load Nx Plugin: /home/vscode/workspace/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected]/node_modules/nx/src/plugins/project-json/build-nodes/project-json' 0.37720899999999347
Time for 'Load Nx Plugin: @nx/storybook/plugin' 213.89875
Time for 'Load Nx Plugin: @nx/remix/plugin' 76.697
Time for 'Load Nx Plugin: @nxlv/python' 194.76016700000002
Time for '@nx/remix/plugin:createNodes' 0.046625000000062755
Time for 'nx/js/dependencies-and-lockfile:createNodes' 40.426874999999995
Time for '@rh-plugins/perf-test:createNodes' 139.61029199999996
Time for 'nx/core/package-json:createNodes' 223.16429199999993
Time for 'nx/core/package-json-workspaces:createNodes' 177.91375000000005
Time for 'nx/core/project-json:createNodes' 176.405125
Time for '@nx/storybook/plugin:createNodes' 321.95854199999997
Time for 'workspace context init' 5.202041000000008
Time for '@nx/vite/plugin:createNodes' 328.05991700000004
Time for 'workspace context init' 10.170041999999967
Time for 'nx/core/target-defaults:createNodes' 341.43791699999997
Time for '@nx/eslint/plugin:createNodes' 1881.7331680000002
Time for 'workspace context init' 5.841625000000022
Time for 'build-project-configs' 1942.886418
Time for 'createNodes:merge' 7.167707999999948
Time for 'native-file-deps' 0.1515409999997246
Time for 'get-workspace-files' 2.2273339999997006
Time for 'get-all-workspace-files' 10.753375000000233
Time for '@nx/storybook/plugin:createDependencies' 98.18079200000011
Time for '@nx/remix/plugin:createDependencies' 84.69016600000032
Time for '@nx/vite/plugin:createDependencies' 62.47445800000014
Time for '@nxlv/python:createDependencies' 239.52779200000032
Time for 'build typescript dependencies' 549.6482080000001

 NX   JSON output created in /home/vscode/workspace

/home/vscode/workspace/tmp_file3.json

NX_PERF_LOGGING=true NX_DAEMON=false NX_CACHE_PROJECT_GRAPH=false =true pnpm   0.66s user 0.31s system 26% cpu 3.720 total

[kylebjordahl]

Narrowing it down even further, I added some performance marks in my local nx install, specifically in node_modules/nx/src/plugins/js/utils/typescript.js:readTsConfig and it looks as though this is where all the time is getting spent on plugin loading, specifically in tsModule.parseJsonConfigFileContent

Time for 'Load Nx Plugin: @rh-plugins/ruff' 62171.26652999999
Time for 'readTsConfig - readConfigFile total' 5.7970000000000255
Time for 'readTsConfig - Total' 61845.963154

So, going out on a wild limb, I tried making a few under-the-hood tweaks and managed to get some pretty significant speed improvements by avoiding the call to tsModule.parseJsonConfigFileContent altogether; from what I can tell, registerPluginTSTranspiler doesn't actually need most of the output generated by parseJsonConfigFileContent, it really just wants the contents of the tsconfig.json file.

I'll provide a PR that represents what I've found, and hopefully this can get sorted out quickly

[kylebjordahl]

After investigating the performance of parseJsonConfigFileContent (mostly by running tsc --showConfig) I finally diagnosed this as an issue in our tsconfig.base.json excludes.

I ended up adding some extra excludes, it might be ideal to include some of these in the default which is generated for new workspaces:

"exclude": ["**/node_modules", "**/.*/", "tmp", "**/cdk.out", "dist"]