Open a album in externalLibrary within over 5000 pieces images, will surely trigger OOM of javascript heap allocator.even memory is not occupied by immich server process.

[afrojewelz]

The bug

Open a album in externalLibrary within over 5000 pieces images, will surely trigger OOM of javascript heap allocator.even memory is not occupied by immich server process.

the reproduce step is easy. :

The OS that Immich Server is running on

Debian GNU/Linux" VERSION_ID="12" VERSION="12 (bookworm)

Version of Immich Server

v1.122.1

Version of Immich Mobile App

v1.122.1

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

networks:
  immichsvc_my_bridge_network:
    driver: bridge
    
services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:v1.122.1
    # extends:
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
    # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
    - /rpool/immichupload:/usr/src/app/upload
    - /satar5/photoz:/satar5/photoz:ro
    - /etc/localtime:/etc/localtime:ro
    environment:
      TZ: Asia/Singapore
      REDIS_URL: >-
        ioredis://ewogICAibmFtZSI6ICJkcmFnb25mbHkiLAogICAiaG9zdCI6ICIxOTIuMTY4LjUwLjEwNCIsCiAgICJwb3J0IjogNjM3OQp9
      DB_URL: postgresql://postgres:[email protected]:5432/immich
      IMMICH_IGNORE_MOUNT_CHECK_ERRORS: 'true'
      UPLOAD_LOCATION: /usr/src/app/upload
      NODE_OPTIONS: '--max-old-space-size=32768 --max-semi-space-size=8192 --inspect ./dist/workers/api.js'
    dns:
      - 192.168.50.1
      - 8.8.4.4
    ports:
      - '192.168.50.23:83:2283'
    restart: unless-stopped
    tty: true
    stdin_open: true
    healthcheck:
      disable: false
    deploy:
      resources:
        reservations:
          devices:
          - driver: nvidia
            count: 1
            capabilities:
            - video

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: v1.121.0-cuda
    image: ghcr.io/immich-app/immich-machine-learning:v1.122.1-cuda
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
    - /rpool/k3sdb/model-cache:/cache
    ports:
    - '3003:3003'
    environment:
      TZ: Asia/Singapore
      REDIS_URL: >-
        ioredis://ewogICAibmFtZSI6ICJkcmFnb25mbHkiLAogICAiaG9zdCI6ICIxOTIuMTY4LjUwLjEwNCIsCiAgICJwb3J0IjogNjM3OQp9
      DB_URL: postgresql://postgres:[email protected]:5432/immich
    restart: unless-stopped
    tty: true
    stdin_open: true
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all  # Use all available GPUs
              capabilities: [gpu]
    healthcheck:
      disable: false
  # database:
  #   container_name: immich_postgres
  #   image: tensorchord/pgvecto-rs:pg16-v0.2.1
  #   ports:
  #   - '5432:5432'
  #   environment:
  #     TZ: "Asia/Singapore"
  #     POSTGRES_PASSWORD: tensorchord
  #     POSTGRES_USER: postgres
  #     POSTGRES_DB: immich
  #     POSTGRES_INITDB_ARGS: '--data-checksums'
  #   volumes:
  #   # Do not edit the next line. If you want to change the database storage location on your system, edit the value of DB_DATA_LOCATION in the .env file
  #   - /rpool/pgrsdvol:/var/lib/postgresql/data
  #   healthcheck:
  #     test: pg_isready --dbname='immich' --username='postgres' || exit 1; Chksum="$$(psql --dbname='immich' --username='postgres' --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1
  #     interval: 5m
  #     start_interval: 30s
  #     start_period: 5m
  #   command: [ 'postgres', '-c', 'shared_preload_libraries=vectors.so', '-c', 'search_path="$$user", public, vectors', '-c', 'logging_collector=on', '-c', 'max_wal_size=2GB', '-c', 'shared_buffers=768MB', '-c', 'wal_compression=on' ]
  #   restart: unless-stopped
  #   tty: true
  #   stdin_open: true

Your .env content

DB_URL=postgresql://postgres:[email protected]:5432/immich
IMMICH_IGNORE_MOUNT_CHECK_ERRORS=false
NODE_OPTIONS=--max-old-space-size=32768 --initial-old-space-size=12800 --max-semi-space-size=8192 --gc-interval=1500 --inspect  --turbo-optimization ./dist/workers/api.js
REDIS_URL=ioredis://ewogICAibmFtZSI6ICJkcmFnb25mbHkiLAogICAiaG9zdCI6ICIxOTIuMTY4LjUwLjEwNCIsCiAgICJwb3J0IjogNjM3OQp9
TZ=Asia/Singapore
UPLOAD_LOCATION=/usr/src/app/upload

Reproduction steps

step1:
fill with 4000+ pictures and dozens of videos in one folder ,import it as external library path into immich.
step2:
complete scan \thumb gen and facial recognize ,metadata extraction etc, ok ,you have 5K+ images at photo stream wall
step3:
select all 5K photo, then goto menu click stack to stack them all, you will see all of them stack into a one stack.ok then add all those images into a album and type in any name you wanna name the album.
step4:
return to photo wall at 1st screen, 1st row. then click album you just created. the more you stack in one stack,the harder the album will load, or even load nothing after stucked,few minutes later ,your v8 will tell you the immich workers/api.js is crashed due V8 heap memory allocate OOM.

Relevant log output

internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [immich-api]

 6: 0x145c0e8 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [immich-api]

 7: 0x145d015 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [immich-api]

 8: 0x1435cee v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin

Additional information

once you think of the reason of stuck, it's not issue of db,db is running fine,just bit high CPU usage concentrated in one core at 100%,other core is just lazy, try un-stack the huge stack with 5K+ images ,if un-stack success, the album may load smoother and without huge memory heap allocation leads to OOM.
SO ,is it stack feature that causes HEAP MEM overflow?
No response

[etnoy]

Does this happen if you don't stack the photo first? I don't think we ever intended stacking to be used on 5k images

[afrojewelz]

you don't stack the photo first? I don't think we

no , it seems not, or better if already crashed due stack ,just un-stack and wait the queue process unstack finished then heap won't stuck loading the album till the OOM death