Generating FastAPI endpoints from spacy models with vetiver

[isabelizimm]

Hi all--

I am working on vetiver, an MLOps Python library that helps data scientists version, deploy, and monitor models in a lightweight way (docs, GitHub). I'm interested in supporting spacy, and am looking for user feedback to make this as ergonomic as possible for spacy users. 😄

I would love your ideas! Feel free to install the PR:

pip install git+https://github.com/rstudio/vetiver-python@spacy

The core functionality that I am interested in feedback on is vetiver's generation of FastAPI endpoints. Right now I have made a few assumptions: that people are interested in deploying the nlp model (rather than the Pipeline object or something else), and that input data will be in some sort of one-column data frame. Does this seem correct for most people?

Example usage

import spacy
from vetiver import VetiverModel, VetiverAPI

@spacy.language.Language.component("animals")
def animal_component_function(doc):
    matches = matcher(doc)
    spans = [spacy.tokens.Span(doc, start, end, label="ANIMAL") for match_id, start, end in matches]
    doc.ents = spans
    return doc


nlp = spacy.load("en_core_web_sm")
animals = list(nlp.pipe(["dog", "cat", "lizard"]))
matcher = spacy.matcher.PhraseMatcher(nlp.vocab)
matcher.add("ANIMAL", animals)

nlp.add_pipe("animals", after="ner")

v = VetiverModel(nlp, "animals")
vetiver_api = VetiverAPI(v)
vetiver_api.run()

You can interact with the API that will start running at http://127.0.0.1:8000, or in a new notebook, make predictions from the endpoint using a script like below ⬇️

from vetiver import vetiver_endpoint, predict
import pandas as pd

data = pd.DataFrame({'text': ["wow! a lizard!", "cats > dogs", "i <3 octocat"]})
endpoint = vetiver_endpoint()
predict(endpoint = endpoint, data = data)

---

Any and all feedback is appreciated. I'm keen to learn how spacy is currently living in production, and how to make that a more pleasant experience ✨

Cheers,
Isabel

[shadeMe]

Thanks for adding support for spaCy! Deploying the nlp, i.e., Language object does seem like the right approach to exposing a spaCy pipeline to the end-user.

The Language object consumes text (and sequences thereof) and outputs the corresponding Doc object with the predicted annotations. In our quick testing of vetiver, it already appears to be set-up to correctly process multiple text inputs and return the Doc objects in JSON format.

[isabelizimm]

I appreciate the feedback, I'll move forward with supporting Language objects for returning annotations. Thank you for trying out vetiver!