Can you share your create svdq-int4-flux.1-dev.safetensors script?

[chuck-ma]

Hi, thanks for your work. But I'm wondering how to create an int4 model like svdq-int4-flux.1-dev.safetensors, because I have a model fine-tuned from the dev model.

[chuck-ma]

Gotcha, it seems that i should use deepcompressor to create w4a4 quantized model

[ZukaChilachava]

@chuck-ma was wondering if you managed to load the quantization results from deepcompressor to nunchaku, because I got scaler.pt, smooth.pt, ... as a result and not sure how to convert it to nunchaku format.

[zzz11223345]

@chuck-ma was wondering if you managed to load the quantization results from deepcompressor to nunchaku, because I got scaler.pt, smooth.pt, ... as a result and not sure how to convert it to nunchaku format.
@ZukaChilachava Hello, I have the same question, have you solved it?

[ZukaChilachava]

@zzz11223345 Here is the response from the devs. mit-han-lab/deepcompressor#27 (comment)

[batuhan-ince]

@chuck-ma was wondering if you managed to load the quantization results from deepcompressor to nunchaku, because I got scaler.pt, smooth.pt, ... as a result and not sure how to convert it to nunchaku format.

Hey @ZukaChilachava, I quantized my fine-tuned flux model, but my model.pt (22GB) is bigger than my fine-tuned model (12GB). Is your files just like that or am I missing something. I did the exactly same things in the README.md. Also @lmxyy can you give me your thougths about this?