Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: Can't save bento with custom transformers pipeline #5132

Open
beatanyari opened this issue Dec 16, 2024 · 1 comment
Open

bug: Can't save bento with custom transformers pipeline #5132

beatanyari opened this issue Dec 16, 2024 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@beatanyari
Copy link

beatanyari commented Dec 16, 2024

Describe the bug

The support for custom transformers pipeline seems to be broken, I believe introduced by this commit When trying to save a custom pipeline, I'm getting this error:

BentoMLException: Argument 'pipeline' is not an instance of <class '__main__.MyPipeline'>. It is an instance of <class '__main__.MyPipeline'>.

To reproduce

from transformers import pipeline
from transformers import Pipeline
from transformers import AutoTokenizer
from transformers import AutoModelForSequenceClassification
from transformers.pipelines import SUPPORTED_TASKS

import typing as t

import bentoml


class MyPipeline(Pipeline):
    def _sanitize_parameters(self, **kwargs):
        preprocess_kwargs = {}
        if "maybe_arg" in kwargs:
            preprocess_kwargs["maybe_arg"] = kwargs["maybe_arg"]
        return preprocess_kwargs, {}, {}

    def preprocess(self, text, maybe_arg=2):
        input_ids = self.tokenizer(text, return_tensors="pt")
        return input_ids

    def _forward(self, model_inputs):
        outputs = self.model(**model_inputs)
        return outputs

    def postprocess(self, model_outputs):
        return model_outputs["logits"].softmax(-1).numpy()


TASK_NAME: str = "my-classification-task"
TASK_DEFINITION: t.Dict[str, t.Any] = {
    "impl": MyPipeline,
    "tf": (),
    "pt": (AutoModelForSequenceClassification,),
    "default": {},
    "type": "text",
}
SUPPORTED_TASKS[TASK_NAME] = TASK_DEFINITION

pipe = pipeline(
    task=TASK_NAME,
    model=AutoModelForSequenceClassification.from_pretrained(
        "distilbert-base-uncased-finetuned-sst-2-english"
    ),
    tokenizer=AutoTokenizer.from_pretrained(
        "distilbert-base-uncased-finetuned-sst-2-english"
    ),
)

saved_pipe = bentoml.transformers.save_model(
    "my_classification_model",
    pipeline=pipe,
    task_name=TASK_NAME,
    task_definition=TASK_DEFINITION,
)

Expected behavior

No response

Environment

bentoml: 1.3.16

@beatanyari beatanyari added the bug Something isn't working label Dec 16, 2024
@aarnphm
Copy link
Contributor

aarnphm commented Dec 17, 2024

Did you register your pipeline class with huggingface?

Can you send a small reproducer here?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants