Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Please update script, sd-webui-controlnet updated new version #65

Open
onlinerender opened this issue May 4, 2023 · 3 comments
Open

Comments

@onlinerender
Copy link

Please update script, sd-webui-controlnet updated new version

@afterglowstudios
Copy link

afterglowstudios commented May 9, 2023

yes, when I run the examples with controlnet 1.1 installed automatic1111 v3.0 it behaves as though no control net unit is passed. i updated to automatic1111 5.0 (commit: 22bcc7be) and it worked.

@AllanChen
Copy link

I have a same problem. PLS upgrade your sd version, and use controlnet 1.1.112 ~ 120 version.

@zacharyfmarion
Copy link

FYI I was getting some strange results with the tile model - it looks like the ControlNetUnit needs to be update - this worked for me:

from enum import Enum
from typing import List, Any, Optional, Union, Tuple, Dict
import base64
import io
from PIL import Image

class ControlMode(str, Enum):
    """
    The improved guess mode.
    """

    BALANCED = "Balanced"
    PROMPT = "My prompt is more important"
    CONTROL = "ControlNet is more important"


class ResizeMode(Enum):
    """
    Resize modes for ControlNet input images.
    """

    RESIZE = "Just Resize"
    INNER_FIT = "Crop and Resize"
    OUTER_FIT = "Resize and Fill"

    def int_value(self):
        if self == ResizeMode.RESIZE:
            return 0
        elif self == ResizeMode.INNER_FIT:
            return 1
        elif self == ResizeMode.OUTER_FIT:
            return 2
        assert False, "NOTREACHED"

def raw_b64_img(image: Image) -> str:
    # XXX controlnet only accepts RAW base64 without headers
    with io.BytesIO() as output_bytes:
        metadata = None
        for key, value in image.info.items():
            if isinstance(key, str) and isinstance(value, str):
                if metadata is None:
                    metadata = PngImagePlugin.PngInfo()
                metadata.add_text(key, value)
        image.save(output_bytes, format="PNG", pnginfo=metadata)

        bytes_data = output_bytes.getvalue()

    return str(base64.b64encode(bytes_data), "utf-8")

class ControlNetUnit:
    '''
    This class was taken from the controlnet automatic1111 plugin since the unit that
    ships with webuiapi is not up to date and gives weird results for some models
    '''
    def __init__(
        self,
        input_image: Image = None,
        mask: Image = None,
        module: str = "none",
        model: str = "None",
        weight: float = 1.0,
        resize_mode: Union[ResizeMode, int, str] = ResizeMode.INNER_FIT,
        lowvram: bool = False,
        processor_res: int = -1,
        threshold_a: float = -1,
        threshold_b: float = -1,
        guidance: float = 1.0,
        guidance_start: float = 0.0,
        guidance_end: float = 1.0,
        control_mode: Union[ControlMode, int, str] = ControlMode.BALANCED,
        pixel_perfect: bool = False
    ):
        self.input_image = input_image
        self.mask = mask
        self.module = module
        self.model = model
        self.weight = weight
        self.resize_mode = resize_mode
        self.lowvram = lowvram
        self.processor_res = processor_res
        self.threshold_a = threshold_a
        self.threshold_b = threshold_b
        self.guidance = guidance
        self.guidance_start = guidance_start
        self.guidance_end = guidance_end
        self.control_mode = control_mode
        self.pixel_perfect = pixel_perfect

    def to_dict(self):
        return {
            "input_image": raw_b64_img(self.input_image) if self.input_image else "",
            "mask": raw_b64_img(self.mask) if self.mask is not None else None,
            "module": self.module,
            "model": self.model,
            "weight": self.weight,
            "resize_mode": self.resize_mode.int_value(),
            "lowvram": self.lowvram,
            "processor_res": self.processor_res,
            "threshold_a": self.threshold_a,
            "threshold_b": self.threshold_b,
            "guidance": self.guidance,
            "guidance_start": self.guidance_start,
            "guidance_end": self.guidance_end,
            "control_mode": self.control_mode,
            "pixel_perfect": self.pixel_perfect,
        }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants