Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhancement: support larger file download #2

Open
anotherjesse opened this issue Jun 27, 2023 · 3 comments
Open

Enhancement: support larger file download #2

anotherjesse opened this issue Jun 27, 2023 · 3 comments
Labels
enhancement New feature or request
Milestone

Comments

@anotherjesse
Copy link
Contributor

@daanelson has shared that https://storage.googleapis.com/replicate-weights/llama-13b-fp16.tensors which is ~24GB.

To compare, gcloud can download this in parallel between 1-2 GBps

@joehoover
Copy link

I just have an anecdotal sample size, but I've found that pget works as-is for that model when run on a A100 instance.

Download time with pget was between 21-24 seconds. I tried tweaking -c and found that -c 10 to -c 12 seemed to slightly improve on speed obtained with the default.

Tests with gcloud yielded downloads between 16-24 seconds (with download speeds ranging from 1.1-1.7 GBS).

@tempusfrangit
Copy link
Contributor

Potentially it makes sense to also compare to available ram, if one cannot buffer the whole file into memory use scratchspace and bind files together after.

@tempusfrangit tempusfrangit added the enhancement New feature or request label Nov 30, 2023
@tempusfrangit tempusfrangit changed the title support larger file download Enhancement: support larger file download Nov 30, 2023
@tempusfrangit tempusfrangit added this to the 1.0.0 milestone Dec 3, 2023
@tempusfrangit
Copy link
Contributor

Partially covered by #177

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants