Clean up the filesystem stuff and add support for HF ckpt downloading #2127
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Apr 23, 2024
Closed
monorimet
reviewed
Apr 25, 2024
monorimet
reviewed
Apr 25, 2024
|
Thanks @gpetters-amd. I left a few comments, but looks better for the most part. |
gpetters-amd
force-pushed
the
2.0-stable
branch
from
c264c60
to
d4fb664
Compare
monorimet
approved these changes
Apr 30, 2024
monorimet
pushed a commit
that referenced
this pull request
May 23, 2024
* Initial filesystem cleanup * More filesystem cleanup * Fix some formatting issues * Address comments
monorimet
added a commit
that referenced
this pull request
May 28, 2024
…ixes to LLM, gitignore (#2129) * Shark Studio SDXL support, HIP driver support, simpler device info, small fixes * Fixups to llm API/UI and ignore user config files. * Small fixes for unifying pipelines. * Update requirements.txt for iree-turbine (#2130) * Fix Llama2 on CPU (#2133) * Filesystem cleanup and custom model fixes (#2127) * Fix some formatting issues * Remove IREE pin (fixes exe issue) (#2126) * Update find links for IREE packages (#2136) * Shark Studio SDXL support, HIP driver support, simpler device info, small fixes * Abstract out SD pipelines from Studio Webui (WIP) * Switch from pin to minimum torch version and fix index url * Fix device parsing. * Fix linux setup * Fix custom weights. --------- Co-authored-by: saienduri <[email protected]> Co-authored-by: gpetters-amd <[email protected]> Co-authored-by: gpetters94 <[email protected]>
monorimet
added a commit
that referenced
this pull request
May 28, 2024
* Update requirements.txt for iree-turbine (#2130) * Fix Llama2 on CPU (#2133) * Filesystem cleanup and custom model fixes (#2127) * Initial filesystem cleanup * More filesystem cleanup * Fix some formatting issues * Address comments * Remove IREE pin (fixes exe issue) (#2126) * Diagnose a build issue * Remove IREE pin * Revert the build on pull request change * Update find links for IREE packages (#2136) * (Studio2) Refactors SD pipeline to rely on turbine-models pipeline, fixes to LLM, gitignore (#2129) * Shark Studio SDXL support, HIP driver support, simpler device info, small fixes * Fixups to llm API/UI and ignore user config files. * Small fixes for unifying pipelines. * Update requirements.txt for iree-turbine (#2130) * Fix Llama2 on CPU (#2133) * Filesystem cleanup and custom model fixes (#2127) * Fix some formatting issues * Remove IREE pin (fixes exe issue) (#2126) * Update find links for IREE packages (#2136) * Shark Studio SDXL support, HIP driver support, simpler device info, small fixes * Abstract out SD pipelines from Studio Webui (WIP) * Switch from pin to minimum torch version and fix index url * Fix device parsing. * Fix linux setup * Fix custom weights. --------- Co-authored-by: saienduri <[email protected]> Co-authored-by: gpetters-amd <[email protected]> Co-authored-by: gpetters94 <[email protected]> * Remove leftover merge conflict line from setup script. (#2141) * Add a few requirements for ensured parity with turbine-models requirements. (#2142) * Add scipy to requirements. Adds diffusers req and a note for torchsde. * Update linux setup script. * Move brevitas install --------- Co-authored-by: saienduri <[email protected]> Co-authored-by: gpetters-amd <[email protected]> Co-authored-by: gpetters94 <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This works fine for SD2.1 but I've noticed an issue downloading the safetensors file for clip for SD1.5. There's still some more cleanup necessary here, so I'd appreciate a second pair of eyes on it. I'm also not sure if the way I'm centralizing all the filesystem logic (just using the args and calling
os.path.joinat each usage) is the right thing to do or if we want to have everything go through, say, functions infile_utils.py.