feat/restore-snapshot #30

kklemon · 2024-04-01T13:02:27Z

Motivation

Many users and workflows rely on custom nodes and extensions which correspondingly should also be available in a RunPod worker.

ComfyUI Manager, perhaps the most popular ComfyUI management exension, provides a convenient feature to export a snapshot of the ComfyUI instance with all installed extensions.

This PR implements optional restoring of a snapshot if provided as snapshot.json in the root directory. If such a file is found, ComfyUI Manager will be installed and restoring of the snapshot will be triggered by starting ComfyUI with the --cpu --quick-test-for-ci flags within the Docker image build process.

Issues closed

#6 #22 #18 #9 (partially)

Todo

Implement tests after this initial proposal is approved.

… a missing "package.json"

* Support network volumes * README tweaks * docs: added comment on what is happening * feat: don't overwrite the default paths, but add "runpod_worker_comfy" to have additional paths * docs: updated "bring your own models" --------- Co-authored-by: Tim Pietrusky <[email protected]>

…RETRIES

triplecookedchips · 2024-05-20T18:18:38Z

@kklemon Hi - is it possible to provide a little more guidance here? I added a snapshot.json in the root directory, but nothing changed when I built the image in Docker. Thank you

kklemon · 2024-05-21T07:08:42Z

Adding a snapshot.json in the root directory should be sufficient. Also, make sure that you are building the image from scratch. Apart from these suggestions, I would need to take a look at the build logs to provide more help.

Edit: Since the Dockerfile uses glob-based file copying, you will need a relatively new Docker installation. I couldn't figure out the exact version, but it should work from Docker version 20 upwards.

triplecookedchips · 2024-05-21T07:44:29Z

Thanks @kklemon - I didn't build the image from scratch, so will try again with your advice

TimPietrusky · 2024-05-29T19:38:48Z

@kklemon thanks for adding this, it sounds really nice. We will have to test this, as this is a feature we haven't used yet. But it sounds like this would solve a lot of problems regarding custom nodes and backing them into the actual Docker image.

TimPietrusky

@kklemon I have two questions, please take a look!

TimPietrusky · 2024-05-29T19:40:00Z

Dockerfile

@@ -39,15 +41,16 @@ RUN if [ -z "$SKIP_DEFAULT_MODELS" ]; then wget -O models/vae/sdxl_vae.safetenso
 RUN if [ -z "$SKIP_DEFAULT_MODELS" ]; then wget -O models/vae/sdxl-vae-fp16-fix.safetensors https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/resolve/main/sdxl_vae.safetensors; fi
 RUN if [ -z "$SKIP_DEFAULT_MODELS" ]; then wget -O models/loras/xl_more_art-full_v1.safetensors https://civitai.com/api/download/models/152309; fi

-# Support for the network volume
-ADD src/extra_model_paths.yaml ./


Why is this removed? As this is also usable for adding models into ComfyUI via the network volume.

TimPietrusky · 2024-05-29T19:42:10Z

Dockerfile

+RUN chmod +x /start.sh /restore_snapshot.sh
+
+# Optionally copy snapshot file
+ADD snapshot.jso[n] /


Why would we have jso[n] here when we always expect a json-file?

TimPietrusky · 2024-06-04T09:24:21Z

@kklemon I updated the base branch as we want to move away from having the dev branch.

TimPietrusky · 2024-08-06T14:54:55Z

@kklemon ping 🙏

martintomov · 2024-10-25T18:27:45Z

@TimPietrusky @kklemon I have around 30 custom nodes - what's the best way to install nodes right now? Using the Network Volume where we store models? Have you considered comfy-cli and the node registry?

kklemon · 2024-10-27T19:25:19Z

@TimPietrusky I've completely forgotten about this PR. I will check whether it's still relevant, update it if that's the case or close it otherwise.

kklemon · 2024-10-27T19:32:18Z

@martintomov I haven't been active in the ComfyUI community for a while and don't know what the current de facto standard for managing extensions is, but when I created this PR, I assumed that it's ComfyUI Manager.

If you want to incorporate custom nodes into a RunPod deployment, my recommendation is to install your nodes or extensions via ComfyUI Manager and export a snapshot of your setup as JSON file. You can then just put the snapshot.json in the root directory of this repository, which will be recognized by the build script when building the Docker image. The extensions will be installed during the build process.

Sumukh · 2024-11-03T03:37:05Z

src/restore_snapshot.sh

+
+# Trigger restoring of the snapshot by performing a quick test run
+# Note: We need to use `yes` as some custom nodes may try to install dependencies with pip
+/usr/bin/yes | python main.py --cpu --quick-test-for-ci


Does this need to be python3?

I think it's a good idea to add a symlink for python to point to python3.10.

Here is the relevant section of the Dockerfile with the symlink added:

RUN apt-get update && apt-get install -y \ python3.10 \ python3-pip \ git \ wget \ && ln -sf /usr/bin/python3.10 /usr/bin/python

TimPietrusky · 2024-11-10T20:57:27Z

@kklemon thank you very much, then I will go down this road and make sure it is working.

TimPietrusky · 2024-11-17T22:51:12Z

Some changes are coming:

added comfy-cli
install ComfyUI with comfy-cli
restore a snapshot using comfy-cli
update docs
added test for "restore-snapshot"

In my tests this made sure, that we can now install custom nodes using the snapshot created using the ComfyUI Manager, which is way more nice than adding the nodes directly into the Dockerfile.

TimPietrusky · 2024-11-18T17:47:23Z

🎉 This PR is included in version 3.2.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

A-BMT02 · 2024-11-19T11:32:46Z

I can't seem to get this working . When i start a runpod pod and open my comfy ui on port 8188 it works fine , i can generate my images fine

But when i use the serverless endpoint with the same workflow json , i get an error
2024-11-19 14:00:45.074 [c7msa02ut39p9w] [info] invalid prompt: {'type': 'invalid_prompt', 'message': 'Cannot execute because node FaceDetailer does not exist.',

It seems like the custom nodes are not being recognized in the serverless endpoint even though i have the snapshot.json in the /runpod-volume (/workspace) directory

If i try a request with no custom nodes on the runpod serverless endpoint , it works fine.

TimPietrusky · 2024-11-19T14:30:13Z

@A-BMT02 I moved this conversation over into an issue: #79

* feat: provide option to run the handler locally as API * ci: run the workflow on our extended instance * feat: the local API should run on 0.0.0.0 * feat: make the image smaller * ci: use semantic-version to create releases automatically * chore: we don't want to break anyone with a minor release * docs: added section for local API testing * ci: use custom runner * fix: added .releaserc, otherwise semantic-release will complain about a missing "package.json" * feat: support network volumes, skip default models (blib-la#16) * Support network volumes * README tweaks * docs: added comment on what is happening * feat: don't overwrite the default paths, but add "runpod_worker_comfy" to have additional paths * docs: updated "bring your own models" --------- Co-authored-by: Tim Pietrusky <[email protected]> * feat: provide access to ComfyUI via web * fix: use the full path to the output image * feat: added env vars COMFY_POLLING_INTERVAL_MS and COMFY_POLLING_MAX_RETRIES * test: added "subfolder" * Implement optional restoring of ComfyUI snapshots * feat: use comfy-cli to install ComfyUI & restore snapshot * fix: install all dependencies from the snapshot * chore: moved example snapshot to test_resources * feat: allow any kind of snapshot file * feat: allow any file that has "snapshot" in its name * ci: added test for "restore-snapshot" * ci: run restore snapshat test automatically * docs: use "snapshots" to bake custom nodes into the docker image * docs: add link to ComfyUI Manager docs on how to export snapshots * ci: use correct path to example_snapshot.json * ci: fix the path * ci: fix the file path * ci: use a mock instead of the actual file * ci: fix the path * chore: don't ignore snapshot.json as people might use this --------- Co-authored-by: Tim Pietrusky <[email protected]> Co-authored-by: Meptl <[email protected]>

TimPietrusky and others added 16 commits February 11, 2024 17:21

feat: provide option to run the handler locally as API

69848e5

ci: run the workflow on our extended instance

e3de2fa

feat: the local API should run on 0.0.0.0

fb17089

feat: make the image smaller

e748931

ci: use semantic-version to create releases automatically

8f5a379

chore: we don't want to break anyone with a minor release

f66b2ca

docs: added section for local API testing

b6972f3

ci: use custom runner

fea12e8

fix: added .releaserc, otherwise semantic-release will complain about…

d603c91

… a missing "package.json"

Merge branch 'main' into dev

f11dbef

feat: provide access to ComfyUI via web

201f64e

fix: use the full path to the output image

4b48ef0

feat: added env vars COMFY_POLLING_INTERVAL_MS and COMFY_POLLING_MAX_…

18c510c

…RETRIES

test: added "subfolder"

5435d6c

Implement optional restoring of ComfyUI snapshots

c7747ce

TimPietrusky requested changes May 29, 2024

View reviewed changes

TimPietrusky changed the title ~~Implement optional restoring of ComfyUI snapshots~~ feat: optional restoring of ComfyUI snapshots to bake custom nodes into the Docker Image May 29, 2024

TimPietrusky deleted the branch blib-la:main June 4, 2024 09:21

TimPietrusky closed this Jun 4, 2024

TimPietrusky reopened this Jun 4, 2024

TimPietrusky changed the base branch from dev to main June 4, 2024 09:23

TimPietrusky mentioned this pull request Jun 4, 2024

Support for 3rd party save nodes and alternate formats? #22

Open

TimPietrusky mentioned this pull request Jun 16, 2024

Custom nodes from the network storage pod aren't found #37

Open

Sumukh reviewed Nov 3, 2024

View reviewed changes

TimPietrusky changed the title ~~feat: optional restoring of ComfyUI snapshots to bake custom nodes into the Docker Image~~ feat/restore-snapshot Nov 16, 2024

TimPietrusky added 9 commits November 16, 2024 15:53

feat: use comfy-cli to install ComfyUI & restore snapshot

6d67d7e

fix: install all dependencies from the snapshot

926a6a1

chore: moved example snapshot to test_resources

c949e94

feat: allow any kind of snapshot file

6a83143

feat: allow any file that has "snapshot" in its name

9926f75

ci: added test for "restore-snapshot"

7607a43

Merge branch 'main' into feat/snapshot-restoring

a6b0c6b

ci: run restore snapshat test automatically

07c5a8c

docs: use "snapshots" to bake custom nodes into the docker image

4e3e76d

TimPietrusky added 7 commits November 17, 2024 23:54

docs: add link to ComfyUI Manager docs on how to export snapshots

3f2daef

ci: use correct path to example_snapshot.json

7ed6c19

ci: fix the path

0b2bcd1

ci: fix the file path

cf24d23

ci: use a mock instead of the actual file

0393d84

ci: fix the path

473f4f3

chore: don't ignore snapshot.json as people might use this

f3929cb

TimPietrusky merged commit 7e4b7d0 into blib-la:main Nov 18, 2024
1 check passed

TimPietrusky added the released label Nov 18, 2024

TimPietrusky mentioned this pull request Nov 19, 2024

custom nodes from snapshot.json in network-volume are not restored #79

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat/restore-snapshot #30

feat/restore-snapshot #30

kklemon commented Apr 1, 2024

triplecookedchips commented May 20, 2024

kklemon commented May 21, 2024 •

edited

Loading

triplecookedchips commented May 21, 2024

TimPietrusky commented May 29, 2024

TimPietrusky left a comment •

edited

Loading

TimPietrusky May 29, 2024

TimPietrusky May 29, 2024

TimPietrusky commented Jun 4, 2024

TimPietrusky commented Aug 6, 2024

martintomov commented Oct 25, 2024 •

edited

Loading

kklemon commented Oct 27, 2024

kklemon commented Oct 27, 2024

Sumukh Nov 3, 2024

martintomov Nov 3, 2024

TimPietrusky commented Nov 10, 2024

TimPietrusky commented Nov 17, 2024 •

edited

Loading

TimPietrusky commented Nov 18, 2024

A-BMT02 commented Nov 19, 2024

TimPietrusky commented Nov 19, 2024

feat/restore-snapshot #30

feat/restore-snapshot #30

Conversation

kklemon commented Apr 1, 2024

Motivation

Issues closed

Todo

triplecookedchips commented May 20, 2024

kklemon commented May 21, 2024 • edited Loading

triplecookedchips commented May 21, 2024

TimPietrusky commented May 29, 2024

TimPietrusky left a comment • edited Loading

Choose a reason for hiding this comment

TimPietrusky May 29, 2024

Choose a reason for hiding this comment

TimPietrusky May 29, 2024

Choose a reason for hiding this comment

TimPietrusky commented Jun 4, 2024

TimPietrusky commented Aug 6, 2024

martintomov commented Oct 25, 2024 • edited Loading

kklemon commented Oct 27, 2024

kklemon commented Oct 27, 2024

Sumukh Nov 3, 2024

Choose a reason for hiding this comment

martintomov Nov 3, 2024

Choose a reason for hiding this comment

TimPietrusky commented Nov 10, 2024

TimPietrusky commented Nov 17, 2024 • edited Loading

TimPietrusky commented Nov 18, 2024

A-BMT02 commented Nov 19, 2024

TimPietrusky commented Nov 19, 2024

kklemon commented May 21, 2024 •

edited

Loading

TimPietrusky left a comment •

edited

Loading

martintomov commented Oct 25, 2024 •

edited

Loading

TimPietrusky commented Nov 17, 2024 •

edited

Loading