Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Workers AI types #2181

Open
Cherry opened this issue May 29, 2024 · 0 comments
Open

Improve Workers AI types #2181

Cherry opened this issue May 29, 2024 · 0 comments
Assignees
Labels
types Related to @cloudflare/workers-types

Comments

@Cherry
Copy link
Contributor

Cherry commented May 29, 2024

Today, the Workers AI types rely heavily on function overloads to specify arguments for different models. This unfortunately results in very difficult to debug types, and poor DX.

As an example with a more simplified non-AI class: https://tsplay.dev/Nan9ym

Throughout this during development, you get very little assistance with auto-complete, and the errors you get back, are extremely complex and hard to parse:

No overload matches this call.
  Overload 1 of 3, '(fruit: "apple" | "orange", color: "apple" | "orange", options: { foo: boolean; bar: boolean; }): Promise<void>', gave the following error.
    Argument of type '"lemon"' is not assignable to parameter of type '"apple" | "orange"'.
  Overload 2 of 3, '(fruit: "tomato" | "pear", color: "tomato" | "pear", options: { foo: string; bar?: boolean | undefined; }): Promise<void>', gave the following error.
    Argument of type '"lemon"' is not assignable to parameter of type '"tomato" | "pear"'.
  Overload 3 of 3, '(fruit: "lemon" | "lime", color: "lemon" | "lime", options: { foo?: string | undefined; bar: number; somethingElse: boolean; }): Promise<void>', gave the following error.
    Type 'null' is not assignable to type 'number'.

Let's use some real AI examples. These can all be found in https://tsplay.dev/wQB41N - scroll down to line 255 for examples.

Invalid model names

AI.run('@hf/thebloke/neural-chat-7b-v3-awq', {
    messages: [
        {
            role: "system",
            content: `[very large system prompt]`
        },
        {
            role: "user",
            content: `How many pounds of food does a person eat in a year?`
        }
    ],
    stream: false
});

This errors with:

No overload matches this call.
  The last overload gave the following error.
    Argument of type '"@hf/thebloke/neural-chat-7b-v3-awq"' is not assignable to parameter of type '"@cf/unum/uform-gen2-qwen-500m" | "@cf/llava-hf/llava-1.5-7b-hf"'.

This is because I typo'd the model name and it should be @hf/thebloke/neural-chat-7b-v3-1-awq.

Bad inputs

AI.run('@hf/thebloke/neural-chat-7b-v3-1-awq', {
    message: [
        {
            role: "system",
            content: `[very large system prompt]`
        },
        {
            role: "user",
            content: `How many pounds of food does a person eat in a year?`
        }
    ],
    stream: false
});

Here I've typo'd messages to message, but the error is still just:

No overload matches this call.
  The last overload gave the following error.
    Argument of type '"@hf/thebloke/neural-chat-7b-v3-1-awq"' is not assignable to parameter of type '"@cf/unum/uform-gen2-qwen-500m" | "@cf/llava-hf/llava-1.5-7b-hf"'.

Bad options

AI.run('@hf/thebloke/neural-chat-7b-v3-1-awq', {
    messages: [
        {
            role: "system",
            content: `[very large system prompt]`
        },
        {
            role: "user",
            content: `How many pounds of food does a person eat in a year?`
        }
    ],
    stream: false
}, {
    extraHeaders: true
});

Here I mis-typed extraHeaders to be a boolean instead of an object, and my error is still:

No overload matches this call.
  The last overload gave the following error.
    Argument of type '"@hf/thebloke/neural-chat-7b-v3-1-awq"' is not assignable to parameter of type '"@cf/unum/uform-gen2-qwen-500m" | "@cf/llava-hf/llava-1.5-7b-hf"'.

Essentially, due to the function overload complexity here, it seems that the errors surfaced to users from these types are really not that helpful. People are confused about this regularly in the Discord.

@Cherry Cherry added the types Related to @cloudflare/workers-types label May 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
types Related to @cloudflare/workers-types
Projects
Status: Untriaged
Development

No branches or pull requests

2 participants