Question about quantize time for custom flux transformer

[chuck-ma]

I'm currently using H800 to do Smooth Quantization for my custom flux transformer. I'm wondering how long it would take to finish quantization. I have been quantizing for 20 minutes, but the progress bar is still empty.

python -m deepcompressor.app.diffusion.ptq configs/model/flux.1-custom.yaml configs/svdquant/int4.yaml --save-model /root/autodl-tmp/flux.1-custom-svdquant-int4

[adhikjoshi]

Do share update if it did work,

[chuck-ma]

Well, it will take 70 hours to quantize. But I have no money. Any idea about how to speed up ? @synxlin @bobboli

[dome272]

Also encountering this taking 12 hours on an H100

[synxlin]

Hi, @chuck-ma @dome272 ,

We are working on improving our codebase to support fast calibration without online activation generation. We'll keep this issue updated.

[chuck-ma]

Understood. I have read your paper and studied the general framework of your "nunchaku" project. If there is anything I can do to help you, I will be very happy to contribute. @synxlin