Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing full.pt when running latency benchmark #15

Open
PaulWang0513 opened this issue Dec 23, 2023 · 0 comments
Open

Missing full.pt when running latency benchmark #15

PaulWang0513 opened this issue Dec 23, 2023 · 0 comments

Comments

@PaulWang0513
Copy link

Hi authors,

Thank you for publishing the impressive work, along with the source code. I am currently attempting to run the latency benchmark by following the instructions in the README, but end up with some errors.

I found that in the DejaVu/Dejavu/benchmarks/benchmark_generation_opt.py file, the model parameter is loaded from a single file named "full.pt," which differs from the approach used in the accuracy benchmark (using files like pytorch_0.pt, pytorch_1.pt, and so on).

After completing the accuracy benchmark (perplexity on c4), I couldn't locate the "full.pt" file. Is there perhaps an instruction for generating the "full.pt" file that I may have missed? Your guidance on this matter would be greatly appreciated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant