[FEATURE REQUEST] replace masking with inpainting (model weights download provided) #213
Replies: 11 comments 9 replies
-
|
You can load a different checkpoint now with the config option, does this solve your issue? |
Beta Was this translation helpful? Give feedback.
-
|
Also masking itself is unrelated to inpainting. I believe you meant to say img2img, which is also not quite the same as inpainting. |
Beta Was this translation helpful? Give feedback.
-
right, I'm requesting adding inpainting as a feature. I've provided a download to the inpainting model weights according to the compvis repo (so it will be possible to test the feature) |
Beta Was this translation helpful? Give feedback.
-
|
Inpainting is an advanced feature and afaik it can't be implemented yet, I believe the model you are referring to is not a stable diffusion model, you may have found it in the code, but it's not release by stable diffusion. If you have more information about implementation you should provide it. |
Beta Was this translation helpful? Give feedback.
-
These are the old inpainting weights from latent diffusion, doubt they're going to be of much use in SD not without the serious work to be done. |
Beta Was this translation helpful? Give feedback.
-
|
Isn't inpainting possible using stable diffusion via diffusers (https://github.com/huggingface/diffusers/tree/main/examples/inference) and thus also possible here? Or am I missing something? |
Beta Was this translation helpful? Give feedback.
-
yeah, it would appear @codedealer is correct (I was misinformed I apologize), but also what @guwa1 said. I'm not sure how useful the inpainting.py is since it relies on the diffusers library which seems to have the huggingface token requirement, which I'm not a big fan of. |
Beta Was this translation helpful? Give feedback.
-
|
You can try Lama Cleaner, it integrates multiple inpainting models, including inpainting model from latent-diffusion.
|
Beta Was this translation helpful? Give feedback.
-
|
Oh nice, it's like what my Google Pixel 6 Pro can do with the Magic Eraser feature! I actually really like that. On this point though, I know I heard that in-painting was coming to DreamStudio, and maybe Stable Diffusion as a whole, at some point. Here's just some sources I found referencing this as being a thing, Emmad in particular is heavily suggesting it will be:
These were just a couple I saw, but I've seen references to this elsewhere too. It'll be interesting to see it applied natively to Stable Diffusion once it's released. |
Beta Was this translation helpful? Give feedback.
-
|
https://github.com/basujindal/stable-diffusion has found a way to create an inpainting script could we ask them? |
Beta Was this translation helpful? Give feedback.
-
|
I've been using the mask feature here as crude as it might be, then going to photoshop with the original image and putting the newly created on a layer and painting out the unformed parts - decent workaround - would be great if I didn't have to though ;) |
Beta Was this translation helpful? Give feedback.
-
|
diffusers has an example script for inpainting https://github.com/huggingface/diffusers/tree/main/examples/inference#in-painting-using-stable-diffusion |
Beta Was this translation helpful? Give feedback.
-
|
https://github.com/basujindal/stable-diffusion/blob/main/scripts/inpaint.py |
Beta Was this translation helpful? Give feedback.
-
|
needs trained model |
Beta Was this translation helpful? Give feedback.
-
|
so which model was the script noted above using? Apologies I'm new but trying to learn fast. Is the script noted above not based on the same stable diffusion model we are using here? thanks for the help |
Beta Was this translation helpful? Give feedback.
-
|
it's a placeholder by stability devs, the model is private for now |
Beta Was this translation helpful? Give feedback.
-
|
thank you! So there is a separate model called in the inpainting script that is not part of the stablediffusion 1.4 release that this UI is based on? I think I'm tracking now ;) WILL they be releasing it? |
Beta Was this translation helpful? Give feedback.
-
|
I don't know but I don't see why won't they at some point in the future |
Beta Was this translation helpful? Give feedback.
-
|
Finally make SD inpainting works in lama-cleaner.
To get a good results, you need right prompt and a lot of luck(maybe SD1.5 will do better job). Here are some of my test results Original Image
Inpaint Result
|
Beta Was this translation helpful? Give feedback.
-
download here: https://heibox.uni-heidelberg.de/f/4d9ac7ea40c64582b7c9/?dl=1
Is your feature request related to a problem? Please describe.
current masking feature doesn't use visual context so it looks out of place
Describe the solution you'd like
use inpainting with the official inpainting weights
Describe alternatives you've considered
not having inpainting
Additional context
taken from compvis repo for inpainting line:
wget -O models/ldm/inpainting_big/last.ckpt https://heibox.uni-heidelberg.de/f/4d9ac7ea40c64582b7c9/?dl=1Beta Was this translation helpful? Give feedback.
All reactions