smk2295/SparseGPTQ-Qwen3-8B-Base-uns90-int4

SparseGPT+GPTQ — Qwen3-8B-Base (90% unstructured sparsity + INT4, symmetric)

Baseline compressed checkpoint for compression research.

• Method: SparseGPT (pruning) + GPTQ (quantization)
• Sparsity: 90% unstructured
• Quantization: INT4, per-group 128, symmetric
• Base: Qwen/Qwen3-8B-Base

Files

• model-*.safetensors (sharded): sparse + fake-quantized weights in fp16, loadable via AutoModelForCausalLM.
• compression/scales.safetensors: per-group-128 quantization scales, shape [out_features, in_features/128] per layer.
• compression/zeros.safetensors: zero points (constant for symmetric).
• compression_config.json: method / sparsity / bits / granularity / symmetric.

Analysis notes

• Sparsity mask is recoverable as weight == 0 (no separate mask stored).
• Symmetric quantization: weight = q * scale (per group of 128 input channels).

Load

from transformers import AutoModelForCausalLM
m = AutoModelForCausalLM.from_pretrained("smk2295/SparseGPTQ-Qwen3-8B-Base-uns90-int4", torch_dtype="float16")