Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Local apps #659

Merged
merged 11 commits into from
May 10, 2024
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ We use `pnpm` as our package manager. You need to use it, eg `pnpm install` inst

If you want to format the whole codebase, you can do `pnpm -r format` at the root.

Otherwise, we avoid runtime dependencies unless they're strictly needed. For example, our only dependency is `hash-wasm`, and it's only in the browser context and when uploaded files are > 10MB.
Other than that, we avoid runtime dependencies unless they're strictly needed. For example, our only dependency is `hash-wasm`, and it's only in the browser context and when uploaded files are > 10MB.

## Pull requests

Expand Down
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,9 @@ This is a collection of JS libraries to interact with the Hugging Face API, with
- [@huggingface/inference](packages/inference/README.md): Use Inference Endpoints (dedicated) and Inference API (serverless) to make calls to 100,000+ Machine Learning models
- [@huggingface/hub](packages/hub/README.md): Interact with huggingface.co to create or delete repos and commit / download files
- [@huggingface/agents](packages/agents/README.md): Interact with HF models through a natural language interface
- [@huggingface/gguf](packages/gguf/README.md): A GGUF parser that works on remotely hosted files.
- [@huggingface/tasks](packages/tasks/README.md): The definition files and source-of-truth for the Hub's main primitives like pipeline tasks, model libraries, etc.



We use modern features to avoid polyfills and dependencies, so the libraries will only work on modern browsers / Node.js >= 18 / Bun / Deno.
Expand Down
26 changes: 12 additions & 14 deletions packages/tasks/README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,18 @@
# Tasks

This package contains data used for https://huggingface.co/tasks.
This package contains the definition files (written in Typescript) for the huggingface.co hub's:

- **pipeline types** (a.k.a. **task types**) - used to determine which widget to display on the model page, and which inference API to run.
- **default widget inputs** - when they aren't provided in the model card.
- definitions and UI elements for **model libraries** (and soon for **dataset libraries**).

Please add any missing ones to these definitions by opening a PR. Thanks 🔥

## Philosophy behind Tasks
⚠️ The hub's definitive doc is at https://huggingface.co/docs/hub.

## Definition of Tasks

This package also contains data used to define https://huggingface.co/tasks.

The Task pages are made to lower the barrier of entry to understand a task that can be solved with machine learning and use or train a model to accomplish it. It's a collaborative documentation effort made to help out software developers, social scientists, or anyone with no background in machine learning that is interested in understanding how machine learning models can be used to solve a problem.

Expand All @@ -19,16 +29,4 @@ We have a [`dataset`](https://huggingface.co/datasets/huggingfacejs/tasks) that

This might seem overwhelming, but you don't necessarily need to add all of these in one pull request or on your own, you can simply contribute one section. Feel free to ask for help whenever you need.

## Other data

This package contains the definition files (written in Typescript) for the huggingface.co hub's:

- **pipeline types** a.k.a. **task types** (used to determine which widget to display on the model page, and which inference API to run)
- **default widget inputs** (when they aren't provided in the model card)
- definitions and UI elements for **model libraries** (and soon for **dataset libraries**).

Please add to any of those definitions by opening a PR. Thanks 🔥

⚠️ The hub's definitive doc is at https://huggingface.co/docs/hub.

## Feedback (feature requests, bugs, etc.) is super welcome 💙💚💛💜♥️🧡
3 changes: 3 additions & 0 deletions packages/tasks/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,3 +45,6 @@ export { SPECIAL_TOKENS_ATTRIBUTES } from "./tokenizer-data";

import * as snippets from "./snippets";
export { snippets };

export { LOCAL_APPS } from "./local-apps";
export type { LocalApp, LocalAppKey } from "./local-apps";
119 changes: 119 additions & 0 deletions packages/tasks/src/local-apps.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
import type { ModelData } from "./model-data";
import type { PipelineType } from "./pipelines";

/**
* Elements configurable by a local app.
*/
export type LocalApp = {
/**
* Name that appears in buttons
*/
prettyLabel: string;
julien-c marked this conversation as resolved.
Show resolved Hide resolved
/**
* Link to get more info about a local app (website etc)
*/
docsUrl: string;
/**
* main category of app
*/
mainTask: PipelineType;
julien-c marked this conversation as resolved.
Show resolved Hide resolved
/**
* Whether to display a pill "macOS-only"
*/
macOSOnly?: boolean;

comingSoon?: boolean;
/**
* IMPORTANT: function to figure out whether to display the button on a model page's main "Use this model" dropdown.
*/
displayOnModelPage: (model: ModelData) => boolean;
} & (
| {
/**
* If the app supports deeplink, URL to open.
*/
deeplink: (model: ModelData) => URL;
}
| {
/**
* And if not (mostly llama.cpp), snippet to copy/paste in your terminal
*/
snippet: (model: ModelData) => string;
}
);

function isGgufModel(model: ModelData) {
return model.tags.includes("gguf");
}

const snippetLlamacpp = (model: ModelData): string => {
return `./main \
--hf-repo ${model.id} \
-m file.gguf \
julien-c marked this conversation as resolved.
Show resolved Hide resolved
-p "I believe the meaning of life is " -n 128`;
};

/**
* Add your new local app here.
*
* This is open to new suggestions and awesome upcoming apps.
*
* /!\ IMPORTANT
*
* If possible, you need to support deeplinks and be as cross-platform as possible.
*
* Ping the HF team if we can help with anything!
*/
export const LOCAL_APPS = {
"llama.cpp": {
prettyLabel: "llama.cpp",
docsUrl: "https://github.com/ggerganov/llama.cpp",
mainTask: "text-generation",
displayOnModelPage: isGgufModel,
snippet: snippetLlamacpp,
},
lmstudio: {
prettyLabel: "LM Studio",
docsUrl: "https://lmstudio.ai",
mainTask: "text-generation",
displayOnModelPage: isGgufModel,
deeplink: (model) => new URL(`lmstudio://open_from_hf?model=${model.id}`),
},
jan: {
prettyLabel: "Jan",
docsUrl: "https://jan.ai",
mainTask: "text-generation",
displayOnModelPage: isGgufModel,
julien-c marked this conversation as resolved.
Show resolved Hide resolved
deeplink: (model) => new URL(`jan://open_from_hf?model=${model.id}`),
},
faraday: {
prettyLabel: "Faraday",
docsUrl: "https://faraday.dev",
mainTask: "text-generation",
macOSOnly: true,
displayOnModelPage: isGgufModel,
deeplink: (model) => new URL(`faraday://open_from_hf?model=${model.id}`),
},
drawthings: {
prettyLabel: "Draw Things",
docsUrl: "https://drawthings.ai",
mainTask: "text-to-image",
macOSOnly: true,
/**
* random function, will need to refine the actual conditions:
*/
displayOnModelPage: (model) => model.tags.includes("textual_inversion"),
deeplink: (model) => new URL(`drawthings://open_from_hf?model=${model.id}`),
},
diffusionbee: {
prettyLabel: "DiffusionBee",
docsUrl: "https://diffusionbee.com",
mainTask: "text-to-image",
macOSOnly: true,
comingSoon: true,
displayOnModelPage: (model) => model.library_name === "diffusers" && model.pipeline_tag === "text-to-image",
julien-c marked this conversation as resolved.
Show resolved Hide resolved
deeplink: (model) => new URL(`diffusionbee://open_from_hf?model=${model.id}`),
},
} satisfies Record<string, LocalApp>;

export type LocalAppKey = keyof typeof LOCAL_APPS;
6 changes: 1 addition & 5 deletions packages/tasks/src/model-data.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,6 @@ export interface ModelData {
* id of model (e.g. 'user/repo_name')
*/
id: string;
/**
* Kept for backward compatibility
*/
modelId?: string;
/**
* Whether or not to enable inference widget for this model
*/
Expand Down Expand Up @@ -84,7 +80,7 @@ export interface ModelData {
/**
* all the model tags
*/
tags?: string[];
tags: string[];
/**
* transformers-specific info to display in the code sample.
*/
Expand Down
30 changes: 15 additions & 15 deletions packages/tasks/src/model-libraries-snippets.ts
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ predictions = predictor.predict_json(predictor_input)`,
];

export const allennlp = (model: ModelData): string[] => {
if (model.tags?.includes("question-answering")) {
if (model.tags.includes("question-answering")) {
return allennlpQuestionAnswering(model);
}
return allennlpUnknown(model);
Expand Down Expand Up @@ -85,11 +85,11 @@ pipeline.load_textual_inversion("${model.id}")`,
];

export const diffusers = (model: ModelData): string[] => {
if (model.tags?.includes("controlnet")) {
if (model.tags.includes("controlnet")) {
return diffusers_controlnet(model);
} else if (model.tags?.includes("lora")) {
} else if (model.tags.includes("lora")) {
return diffusers_lora(model);
} else if (model.tags?.includes("textual_inversion")) {
} else if (model.tags.includes("textual_inversion")) {
return diffusers_textual_inversion(model);
} else {
return diffusers_default(model);
Expand Down Expand Up @@ -118,9 +118,9 @@ text, *_ = model(speech)[0]`,
const espnetUnknown = () => [`unknown model type (must be text-to-speech or automatic-speech-recognition)`];

export const espnet = (model: ModelData): string[] => {
if (model.tags?.includes("text-to-speech")) {
if (model.tags.includes("text-to-speech")) {
return espnetTTS(model);
} else if (model.tags?.includes("automatic-speech-recognition")) {
} else if (model.tags.includes("automatic-speech-recognition")) {
return espnetASR(model);
}
return espnetUnknown();
Expand Down Expand Up @@ -228,7 +228,7 @@ inference.crop("file.wav", excerpt)`,
];

export const pyannote_audio = (model: ModelData): string[] => {
if (model.tags?.includes("pyannote-audio-pipeline")) {
if (model.tags.includes("pyannote-audio-pipeline")) {
return pyannote_audio_pipeline(model);
}
return pyannote_audio_model(model);
Expand Down Expand Up @@ -258,9 +258,9 @@ model = TFAutoModel.from_pretrained("${model.id}")
];

export const tensorflowtts = (model: ModelData): string[] => {
if (model.tags?.includes("text-to-mel")) {
if (model.tags.includes("text-to-mel")) {
return tensorflowttsTextToMel(model);
} else if (model.tags?.includes("mel-to-wav")) {
} else if (model.tags.includes("mel-to-wav")) {
return tensorflowttsMelToWav(model);
}
return tensorflowttsUnknown(model);
Expand Down Expand Up @@ -309,7 +309,7 @@ model = joblib.load(
};

export const sklearn = (model: ModelData): string[] => {
if (model.tags?.includes("skops")) {
if (model.tags.includes("skops")) {
const skopsmodelFile = model.config?.sklearn?.model?.file;
const skopssaveFormat = model.config?.sklearn?.model_format;
if (!skopsmodelFile) {
Expand Down Expand Up @@ -413,7 +413,7 @@ export const transformers = (model: ModelData): string[] => {
if (!info) {
return [`# ⚠️ Type of model unknown`];
}
const remote_code_snippet = model.tags?.includes(TAG_CUSTOM_CODE) ? ", trust_remote_code=True" : "";
const remote_code_snippet = model.tags.includes(TAG_CUSTOM_CODE) ? ", trust_remote_code=True" : "";

let autoSnippet: string;
if (info.processor) {
Expand Down Expand Up @@ -564,7 +564,7 @@ model = create_model(${model.id})`,
export const nemo = (model: ModelData): string[] => {
let command: string[] | undefined = undefined;
// Resolve the tag to a nemo domain/sub-domain
if (model.tags?.includes("automatic-speech-recognition")) {
if (model.tags.includes("automatic-speech-recognition")) {
command = nemoDomainResolver("ASR", model);
}

Expand Down Expand Up @@ -605,11 +605,11 @@ wav = model.generate(descriptions) # generates 3 samples.`,
];

export const audiocraft = (model: ModelData): string[] => {
if (model.tags?.includes("musicgen")) {
if (model.tags.includes("musicgen")) {
return musicgen(model);
} else if (model.tags?.includes("audiogen")) {
} else if (model.tags.includes("audiogen")) {
return audiogen(model);
} else if (model.tags?.includes("magnet")) {
} else if (model.tags.includes("magnet")) {
return magnet(model);
} else {
return [`# Type of model unknown.`];
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@
};

$: widgetComponent =
model.pipeline_tag === "text-generation" && model.tags?.includes("conversational")
model.pipeline_tag === "text-generation" && model.tags.includes("conversational")
? (ConversationalWidget as typeof SvelteComponent)
: model.pipeline_tag && model.pipeline_tag in WIDGET_COMPONENTS
? WIDGET_COMPONENTS[model.pipeline_tag as keyof typeof WIDGET_COMPONENTS]
Expand Down
Loading
Loading