Update README.md
Browse files
README.md
CHANGED
|
@@ -61,7 +61,6 @@ Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated sys
|
|
| 61 |
## Software Integration:
|
| 62 |
**Supported Runtime Engine(s):** <br>
|
| 63 |
* vLLM <br>
|
| 64 |
-
* SGLang <br>
|
| 65 |
|
| 66 |
**Supported Hardware Microarchitecture Compatibility:** <br>
|
| 67 |
* NVIDIA Blackwell <br>
|
|
@@ -100,7 +99,9 @@ The model version is NVFP4 1.0 version and is quantized with nvidia-modelopt **v
|
|
| 100 |
**Test Hardware:** NVIDIA Blackwell GB200 <br>
|
| 101 |
|
| 102 |
## Post Training Quantization
|
| 103 |
-
This model was obtained by quantizing the weights and activations of GLM-5.3-Flash to NVFP4 data type, ready for inference with vLLM. Only the weights and activations of the linear operators within transformer blocks in MoE are quantized. This optimization reduces the number of bits per parameter from
|
|
|
|
|
|
|
| 104 |
|
| 105 |
## Usage
|
| 106 |
|
|
|
|
| 61 |
## Software Integration:
|
| 62 |
**Supported Runtime Engine(s):** <br>
|
| 63 |
* vLLM <br>
|
|
|
|
| 64 |
|
| 65 |
**Supported Hardware Microarchitecture Compatibility:** <br>
|
| 66 |
* NVIDIA Blackwell <br>
|
|
|
|
| 99 |
**Test Hardware:** NVIDIA Blackwell GB200 <br>
|
| 100 |
|
| 101 |
## Post Training Quantization
|
| 102 |
+
This model was obtained by quantizing the weights and activations of GLM-5.3-Flash to NVFP4 data type, ready for inference with vLLM. Only the weights and activations of the linear operators within transformer blocks in sparse MoE shared experts and dense MLP are quantized. This optimization reduces the number of bits per parameter from 16 to 4, reducing the disk size and GPU memory requirements by approximately 3.33x.
|
| 103 |
+
|
| 104 |
+
** modelopt PTQ recipe: [nvfp4_experts_dense_mlp-kv_fp8_cast](https://github.com/NVIDIA/Model-Optimizer/blob/4956213d670c382385d9bc43e17379b9fc064e50/modelopt_recipes/models/zai-org/GLM-5.3-Flash/ptq/nvfp4_experts_dense_mlp-kv_fp8_cast.yaml)
|
| 105 |
|
| 106 |
## Usage
|
| 107 |
|