qwen_2.5_vl_7b_fp4.safetensors
#21
by Landsharkbaby - opened
Could you please make NVFP4 version of qwen_2.5_vl_7b_fp4.safetensors? Thank you
Yes, it will be wonderful. Or may be are there any instructions how to create qwen_2.5_vl_7b_ "awq" "nf4" "4bit" "fp4" versions from files for diffusers? "Turboderp" has EXL2 models, "TheBloke" has(AWQ/GPTQ models and "Bartowski" has EXL2 models but they are not for ComfyUI. May be there is some method to transform them? On my GeForce RTX5060Ti with 16 GB qwen_2.5_vl_7b_fp8_scaled.safetensors works too slow.