NX unable to rebuild project graph with more than ~ 250 libraries in the repo.

[frobean]

Current Behavior

After creating a repo with 250+ libraries, NX will exhaust available memory and crash when trying to build the project graph from scratch (such as after running nx reset).
When I say it exhausts all memory, on my workstation, the node process will show (in activity monitor) that it's consuming 30+ GB of memory.

With 250+ libraries in the NX repo, I can't even run npm install as it invokes an nx postinstall script that also crashes
Note that the point of failure may depend on the available memory on your computer. On my dev machine (16GB or RAM) that number is around the 250 library mark.

The attached gzipped node script can be used to generate a repo. Note that NX commands will work in the newly created repo until nx reset is invoked
makeTestRepo.js.gz

Expected Behavior

The process graph is rebuilt without consuming 10s of gigs of memory and without crashing

GitHub Repo

https://github.com/frobean/nx-repro-crash

Steps to Reproduce

From the github repo linked above

1. Clone the repo
2. run npm install
3. observe the crash output from an nx postinstall script

Using the attached script

1. create a target directory (eg, test)
2. unzip the makeTestRepo.js.gz file in the test directory
3. read through the makeTestRepo.js to verify I'm not doing anything evil to you ;)
4. execute "node makeTestRepo.js". It will create a test repo and create 300 empty libraries in it.
5. cd test-repo
6. verify the existence of the libraries in test-repo/libs
7. verify that you can run nx commands such as "npx nx report"
8. run npx nx reset
9. run npx nx report
10. wait and observe. a node process will consume multiple gigs of memory, eventually crashing. nx cli will show an error. If using the daemon, there will be some (not very useful) logs in the daemon.log file

Nx Report

(nx report run prior to running nx reset)

 NX   Report complete - copy this into the issue template

Node   : 20.14.0
OS     : darwin-arm64
npm    : 10.7.0

nx                 : 19.4.0
@nx/js             : 19.4.0
@nx/jest           : 19.4.0
@nx/linter         : 19.4.0
@nx/eslint         : 19.4.0
@nx/workspace      : 19.4.0
@nx/cypress        : 19.4.0
@nx/devkit         : 19.4.0
@nx/eslint-plugin  : 19.4.0
@nx/react          : 19.4.0
@nrwl/tao          : 19.4.0
@nx/vite           : 19.4.0
@nx/web            : 19.4.0
typescript         : 5.4.5
---------------------------------------
Registered Plugins:
@nx/vite/plugin
@nx/eslint/plugin
@nx/cypress/plugin
@nx/jest/plugin

Failure Logs

This is the output when running nx commands without the daemon.  Daemon logs have this same error in them when running with the daemon enabled.

test-repo % npx nx report
thread '<unnamed>' panicked at /Users/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/swc_common-0.33.26/src/input.rs:31:9:
assertion failed: start <= end
stack backtrace:
   0:        0x11fa674ac - _napi_register_module_v1
   1:        0x11f75a9b8 - _blake3_compress_in_place_portable
   2:        0x11fa3f1a8 - _napi_register_module_v1
   3:        0x11fa68ab0 - _napi_register_module_v1
   4:        0x11fa68248 - _napi_register_module_v1
   5:        0x11fa69eb8 - _napi_register_module_v1
   6:        0x11fa68d80 - _napi_register_module_v1
   7:        0x11fa68d10 - _napi_register_module_v1
   8:        0x11fa68d04 - _napi_register_module_v1
   9:        0x120bfa494 - _wasmer_vm_imported_memory32_atomic_notify
  10:        0x120bfa534 - _wasmer_vm_imported_memory32_atomic_notify
  11:        0x12032bf4c - _napi_register_module_v1
  12:        0x11fdcbd98 - _napi_register_module_v1
  13:        0x11fc6d458 - _napi_register_module_v1
  14:        0x11fc7f8b8 - _napi_register_module_v1
  15:        0x11fc85a1c - _napi_register_module_v1
  16:        0x11fc9b6c8 - _napi_register_module_v1
  17:        0x11fc9bb94 - _napi_register_module_v1
  18:        0x11f5310a4 - <unknown>
  19:        0x11f56c03c - <unknown>
thread '<unnamed>' panicked at /Users/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/swc_common-0.33.26/src/input.rs:31:9:
assertion failed: start <= end
stack backtrace:
   0:        0x11fa674ac - _napi_register_module_v1
   1:        0x11f75a9b8 - _blake3_compress_in_place_portable
   2:        0x11fa3f1a8 - _napi_register_module_v1
   3:        0x11fa68ab0 - _napi_register_module_v1
   4:        0x11fa68248 - _napi_register_module_v1
   5:        0x11fa69eb8 - _napi_register_module_v1
   6:        0x11fa68d80 - _napi_register_module_v1
   7:        0x11fa68d10 - _napi_register_module_v1
   8:        0x11fa68d04 - _napi_register_module_v1
   9:        0x120bfa494 - _wasmer_vm_imported_memory32_atomic_notify
  10:        0x120bfa534 - _wasmer_vm_imported_memory32_atomic_notify
  11:        0x12032bf4c - _napi_register_module_v1
  12:        0x11fdcbd98 - _napi_register_module_v1
  13:        0x11fc6d458 - _napi_register_module_v1
  14:        0x11fc7f8b8 - _napi_register_module_v1
  15:        0x11fc85a1c - _napi_register_module_v1
  16:        0x11fc9b6c8 - _napi_register_module_v1
  17:        0x11fc9bb94 - _napi_register_module_v1
  18:        0x11f5310a4 - <unknown>
  19:        0x11f56c03c - <unknown>

Package Manager Version

No response

Operating System

• macOS
• Linux
• Windows
• Other (Please specify)

Additional Information

The threshold of approximately 250 libraries seems dependent on the amount of RAM available to the test machine. 250 is the approximate number on my 16GB dev workstation. It's much lower on the 4GB linux VM I was playing around with when making the repro case

[frobean]

Wow, that was a quick response! Thanks!