Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

⚗️ Use tshy instead of tsup for releasing tasks #1029

Merged
merged 10 commits into from
Nov 15, 2024
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ jobs:
working-directory: e2e
run: |
sleep 3
pnpm i --filter root --filter inference... --filter hub... --frozen-lockfile
pnpm i --filter root --filter inference... --filter hub... --filter tasks-gen --frozen-lockfile
pnpm --filter inference --filter hub --filter tasks publish --force --no-git-checks --registry http://localhost:4874/

- name: E2E test - test yarn install
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
"prettier": "^3.1.0",
"prettier-plugin-svelte": "^3.1.2",
"semver": "^7.5.0",
"tshy": "^3.0.2",
"tsup": "^6.7.0",
"tsx": "^4.7.0",
"typescript": "^5.4.2",
Expand Down
26 changes: 26 additions & 0 deletions packages/tasks-gen/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
{
"name": "@huggingface/tasks-gen",
"packageManager": "[email protected]",
"version": "0.13.1",
"description": "Utilities to generate files for @huggingface/tasks, and avoid unnecessary deps in the published package",
"repository": "https://github.com/huggingface/huggingface.js.git",
"private": true,
"scripts": {
"lint": "eslint --quiet --fix --ext .cjs,.ts .",
"lint:check": "eslint --ext .cjs,.ts .",
"format": "prettier --write .",
"format:check": "prettier --check .",
"check": "tsc",
"inference-codegen": "tsx scripts/inference-codegen.ts && prettier --write ../tasks/src/tasks/*/inference.ts",
"inference-tgi-import": "tsx scripts/inference-tgi-import.ts && prettier --write ../tasks/src/tasks/text-generation/spec/*.json && prettier --write ../tasks/src/tasks/chat-completion/spec/*.json",
"inference-tei-import": "tsx scripts/inference-tei-import.ts && prettier --write ../tasks/src/tasks/feature-extraction/spec/*.json"
},
"type": "module",
"author": "Hugging Face",
"license": "MIT",
"devDependencies": {
"@types/node": "^20.11.5",
"quicktype-core": "https://github.com/huggingface/quicktype/raw/pack-18.0.17/packages/quicktype-core/quicktype-core-18.0.17.tgz",
"type-fest": "^3.13.1"
}
}
223 changes: 223 additions & 0 deletions packages/tasks-gen/pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,6 @@ const PYTHON_HEADER_FILE = `
# - specs: https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
`;

const PYTHON_DIR = "./.python_generated";

const rootDirFinder = function (): string {
let currentPath = path.normalize(import.meta.url);

Expand Down Expand Up @@ -172,8 +170,10 @@ async function postProcessOutput(path2generated: string, outputSpec: Record<stri
return;
}

const rootDir = rootDirFinder();
const rootDir = path.join(rootDirFinder(), "..", "tasks");
Comment on lines -175 to +173
Copy link
Member Author

@coyotte508 coyotte508 Nov 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could just do const rootDir = "../tasks" since the script is always executed in the /packages/tasks-gen context

const tasksDir = path.join(rootDir, "src", "tasks");
const PYTHON_DIR = path.join(rootDir, ".python_generated");

const allTasks = await Promise.all(
(await fs.readdir(tasksDir, { withFileTypes: true }))
.filter((entry) => entry.isDirectory())
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ const rootDirFinder = function (): string {
return "/";
};

const rootDir = rootDirFinder();
const rootDir = path.join(rootDirFinder(), "..", "tasks");
const tasksDir = path.join(rootDir, "src", "tasks");

function toCamelCase(str: string, joiner = "") {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ const rootDirFinder = function (): string {
return "/";
};

const rootDir = rootDirFinder();
const rootDir = path.join(rootDirFinder(), "..", "tasks");
const tasksDir = path.join(rootDir, "src", "tasks");

function toCamelCase(str: string, joiner = "") {
Expand Down
8 changes: 8 additions & 0 deletions packages/tasks/.tshy/build.json
Pierrci marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"extends": "../tsconfig.json",
"compilerOptions": {
"rootDir": "../src",
"module": "nodenext",
"moduleResolution": "nodenext"
}
}
16 changes: 16 additions & 0 deletions packages/tasks/.tshy/commonjs.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"extends": "./build.json",
"include": [
"../src/**/*.ts",
"../src/**/*.cts",
"../src/**/*.tsx",
"../src/**/*.json"
],
"exclude": [
"../src/**/*.mts",
"../src/package.json"
],
"compilerOptions": {
"outDir": "../.tshy-build/commonjs"
}
}
15 changes: 15 additions & 0 deletions packages/tasks/.tshy/esm.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{
"extends": "./build.json",
"include": [
"../src/**/*.ts",
"../src/**/*.mts",
"../src/**/*.tsx",
"../src/**/*.json"
],
"exclude": [
"../src/package.json"
],
"compilerOptions": {
"outDir": "../.tshy-build/esm"
}
}
Loading
Loading