Skip to content

Commit

Permalink
[Enhance] Improve powerpaint (open-mmlab#2080)
Browse files Browse the repository at this point in the history
* add readme

* add readme

* add require

* fix readme

* fix according to readme

* fix lint
  • Loading branch information
liuwenran authored Dec 7, 2023
1 parent 1864048 commit 374ee50
Show file tree
Hide file tree
Showing 4 changed files with 86 additions and 3 deletions.
60 changes: 60 additions & 0 deletions projects/powerpaint/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# A Task is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpainting

### [Project Page](https://powerpaint.github.io/) | [Paper](https://arxiv.org/abs/2312.03594)

This README provides a step-by-step guide to download the repository, set up the required virtual environment named "PowerPaint" using conda, and run PowerPaint with or without ControlNet.

**Stronger Model Weights and Online Demo Coming Soon!**

<img src='https://github.com/open-mmlab/mmagic/assets/12782558/acd01391-c73f-4997-aafd-0869aebcc915'/>

## Getting Started

```bash
# Clone the Repository
git clone https://github.com/open-mmlab/mmagic.git

# Navigate to the Repository
cd projects/powerpaint

# Create Virtual Environment with Conda
conda create --name PowerPaint python=3.9
conda activate PowerPaint

# Install Dependencies
pip install -r requirements.txt

# Create Models Folder
mkdir models

# Set up Git LFS
git lfs install

# Clone PowerPaint Model
git lfs clone https://huggingface.co/JunhaoZhuang/PowerPaint-v1/ ./models
```

## Run PowerPaint

To run PowerPaint, execute the following command:

```bash
python gradio_PowerPaint.py
```

This command will launch the Gradio interface for PowerPaint.

Feel free to explore and create stunning images with PowerPaint!

## BibTeX

```
@misc{zhuang2023task,
title={A Task is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpainting},
author={Junhao Zhuang and Yanhong Zeng and Wenran Liu and Chun Yuan and Kai Chen},
year={2023},
eprint={2312.03594},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
```
4 changes: 2 additions & 2 deletions projects/powerpaint/gradio_PowerPaint.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,9 @@
initialize_tokens=['a', 'a', 'a'],
num_vectors_per_token=10)
pipe.unet.load_state_dict(
torch.load('./models/unet/diffusion_pytorch_model.bin'), strict=False)
torch.load('./models/diffusion_pytorch_model.bin'), strict=False)
pipe.text_encoder.load_state_dict(
torch.load('./models/text_encoder/pytorch_model.bin'), strict=False)
torch.load('./models/pytorch_model.bin'), strict=False)
pipe = pipe.to('cuda')

depth_estimator = DPTForDepthEstimation.from_pretrained(
Expand Down
8 changes: 8 additions & 0 deletions projects/powerpaint/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
controlnet-aux==0.0.3
diffusers==0.23.1
gradio==3.23.0
mmengine
opencv-python
torch
torchvision
transformers
17 changes: 16 additions & 1 deletion projects/powerpaint/utils/utils.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import copy
import importlib
import os
import random
from logging import WARNING
Expand All @@ -8,7 +9,21 @@
import torch.nn as nn
from mmengine import print_log

from mmagic.utils import try_import

def try_import(name: str):
"""Try to import a module.
Args:
name (str): Specifies what module to import in absolute or relative
terms (e.g. either pkg.mod or ..mod).
Returns:
ModuleType or None: If importing successfully, returns the imported
module, otherwise returns None.
"""
try:
return importlib.import_module(name)
except ImportError:
return None


class TokenizerWrapper:
Expand Down

0 comments on commit 374ee50

Please sign in to comment.