Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Partial Loading PR1: Tidy ModelCache #7492

Merged
merged 9 commits into from
Dec 24, 2024
Merged

Conversation

RyanJDick
Copy link
Collaborator

@RyanJDick RyanJDick commented Dec 23, 2024

Summary

This PR tidies up the model cache code in preparation for further refactoring to support partial loading of models onto the GPU. These code changes should not change the functional behavior in any way.

Changes:

  • Remove the ModelCacheBase class. ModelCache is the only implementation, so there is no benefit to the separate abstract class.
  • Split CacheRecord and CacheStats out into their own files.
  • Remove the ModelLocker class. This extra layer of indirection was not providing any benefit. Locking is now done directly with the ModelCache.
  • Tidy up relative imports that were contributing to circular import issues.
  • Pull the 'submodel' concern out of the ModelCache. The ModelCache should not need to be aware of the model manager submodel system.
  • Delete unused properties from the ModelCache (e.g. .lazy_offloading, .storage_device, etc.)

QA Instructions

I ran smoke tests with a variety of SD1, SDXL and FLUX models. No change to behavior is expected.

Merge Plan

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

@github-actions github-actions bot added api python PRs that change python files backend PRs that change backend files services PRs that change app services python-tests PRs that change python tests docs PRs that change docs labels Dec 23, 2024
@RyanJDick RyanJDick marked this pull request as ready for review December 23, 2024 23:32
@RyanJDick RyanJDick enabled auto-merge (rebase) December 24, 2024 14:21
@RyanJDick RyanJDick disabled auto-merge December 24, 2024 14:22
@RyanJDick RyanJDick force-pushed the ryan/model-offload-1-tidy branch from 6b2a3b2 to 55b13c1 Compare December 24, 2024 14:23
@RyanJDick RyanJDick merged commit d3916db into main Dec 24, 2024
15 checks passed
@RyanJDick RyanJDick deleted the ryan/model-offload-1-tidy branch December 24, 2024 14:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api backend PRs that change backend files docs PRs that change docs python PRs that change python files python-tests PRs that change python tests services PRs that change app services
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants