shengliangx commited on
Commit
2bd7b1c
·
verified ·
1 Parent(s): 52f93b7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -2
README.md CHANGED
@@ -61,7 +61,6 @@ Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated sys
61
  ## Software Integration:
62
  **Supported Runtime Engine(s):** <br>
63
  * vLLM <br>
64
- * SGLang <br>
65
 
66
  **Supported Hardware Microarchitecture Compatibility:** <br>
67
  * NVIDIA Blackwell <br>
@@ -100,7 +99,9 @@ The model version is NVFP4 1.0 version and is quantized with nvidia-modelopt **v
100
  **Test Hardware:** NVIDIA Blackwell GB200 <br>
101
 
102
  ## Post Training Quantization
103
- This model was obtained by quantizing the weights and activations of GLM-5.3-Flash to NVFP4 data type, ready for inference with vLLM. Only the weights and activations of the linear operators within transformer blocks in MoE are quantized. This optimization reduces the number of bits per parameter from 8 to 4, reducing the disk size and GPU memory requirements by approximately 1.66x.
 
 
104
 
105
  ## Usage
106
 
 
61
  ## Software Integration:
62
  **Supported Runtime Engine(s):** <br>
63
  * vLLM <br>
 
64
 
65
  **Supported Hardware Microarchitecture Compatibility:** <br>
66
  * NVIDIA Blackwell <br>
 
99
  **Test Hardware:** NVIDIA Blackwell GB200 <br>
100
 
101
  ## Post Training Quantization
102
+ This model was obtained by quantizing the weights and activations of GLM-5.3-Flash to NVFP4 data type, ready for inference with vLLM. Only the weights and activations of the linear operators within transformer blocks in sparse MoE shared experts and dense MLP are quantized. This optimization reduces the number of bits per parameter from 16 to 4, reducing the disk size and GPU memory requirements by approximately 3.33x.
103
+
104
+ ** modelopt PTQ recipe: [nvfp4_experts_dense_mlp-kv_fp8_cast](https://github.com/NVIDIA/Model-Optimizer/blob/4956213d670c382385d9bc43e17379b9fc064e50/modelopt_recipes/models/zai-org/GLM-5.3-Flash/ptq/nvfp4_experts_dense_mlp-kv_fp8_cast.yaml)
105
 
106
  ## Usage
107