Qwen3.6-35B-A3B ยท AWQ INT8 (W8A16)

8-bit quantization of Qwen/Qwen3.6-35B-A3B, produced for efficient vLLM serving. Quantized by axteam.

Revision 2026-11-01 (rebuilt): The MoE router (mlp.gate) and the shared-expert gate are now compensated during AWQ smoothing (checked on every layer against the original: the router rescale matches the norm rescale, median per-channel difference below 2%). The first upload of this repo left the router and the shared-expert gate uncompensated, so tokens were routed to different experts than in the original. If you downloaded this repo before 2026-11-01 06:13:53 UTC (revision 05a077a), download it again. The benchmark and fidelity figures on this card were measured on the rebuilt files.

Base model Qwen/Qwen3.6-35B-A3B
Method AWQ INT8 (W8A16)
Format compressed-tensors pack-quantized
Weights / activations W8A16 โ€” INT8 symmetric weights, group size 32, BF16 activations
Architecture Hybrid (linear-attention + full-attention) ยท MoE (256 experts + shared, ~3B / 35B total active) ยท multimodal (vision) ยท MTP head
Size on disk ~40.4 GB
Calibration 256 samples (Korean-heavy mixed corpus)
License Apache-2.0

Quantization details

  • Scheme โ€” W8A16 โ€” INT8 symmetric weights, group size 32, BF16 activations, stored as compressed-tensors pack-quantized.
  • Kept in BF16 โ€” the vision tower & projector (model.visual.*), the multi-token-prediction head (mtp.*), all RMSNorms, token embeddings, the LM head, the MoE router (mlp.gate) and shared-expert gate, the SSM non-linear parts (linear_attn.conv1d/A_log/dt_bias). Left unquantized to preserve quality (about 6% of all parameters in this build).
  • Tokenizer / chat_template / generation_config are identical to the original. config.json is the original plus a quantization_config block.

This is post-training quantization (PTQ) โ€” no retraining. AWQ uses a small calibration set (no gradient) to protect salient weight channels before rounding.

Benchmarks (measured)

Every column was measured with the same harness and settings โ€” lm-evaluation-harness on vLLM, greedy, completion-style prompts without a chat template (0- or 5-shot, as listed in each row). MMLU here is lm-eval's generative variant (mmlu_generative: the model writes the answer letter and only an exact match on the first line counts), so it is not directly comparable with MMLU scores measured another way (e.g. log-likelihood scoring). Percentages are recovery vs. the BF16 original.

Benchmark BF16 (original) FP8-dynamic AWQ-8bit (this repo) NVFP4A16 NVFP4 AWQ-4bit
MMLU (5-shot, generative) 64.44 67.45 (104.7%) 64.02 (99.3%) 69.31 (107.6%) 68.39 (106.1%) 74.12 (115.0%)
GSM8K (5-shot, strict) 30.40 21.53 (70.8%) 28.81 (94.8%) 24.49 (80.5%) 46.55 (153.1%) 71.11 (233.9%)
ARC-Challenge (0-shot) 55.38 54.61 (98.6%) 55.46 (100.2%) 55.03 (99.4%) 56.23 (101.5%) 54.78 (98.9%)
Winogrande (5-shot) 76.32 75.37 (98.8%) 76.24 (99.9%) 74.43 (97.5%) 74.27 (97.3%) 75.85 (99.4%)
KMMLU (5-shot, direct) 65.14 64.86 (99.6%) 65.18 (100.1%) 64.50 (99.0%) 64.09 (98.4%) 64.61 (99.2%)
HAE-RAE (0-shot) 74.89 74.89 (100.0%) 75.07 (100.2%) 75.34 (100.6%) 74.06 (98.9%) 73.69 (98.4%)
Average 61.09 59.78 (97.9%) 60.80 (99.5%) 60.52 (99.1%) 63.93 (104.6%) 69.03 (113.0%)
Average without MMLU ยท GSM8K 67.93 67.43 (99.3%) 67.99 (100.1%) 67.33 (99.1%) 67.16 (98.9%) 67.23 (99.0%)

Note on MMLU and GSM8K: these tasks are scored on generated text (mmlu_generative, gsm8k strict-match), so the score depends on whether the model answers in exactly the format the scorer expects (for MMLU only a bare answer letter on the first line counts). For this model they swing widely between builds of the same model (MMLU 64.0โ€“74.1; GSM8K 21.5โ€“71.1), and two quantized builds score well above the BF16 original (MMLU: AWQ-4bit 115.0%; GSM8K: AWQ-4bit 233.9%, NVFP4 153.1%). This most likely reflects how each build formats its answers rather than quantization quality, but per-question outputs were not saved, so it was not checked answer by answer. The second average leaves them out.

Evaluation (measured)

Fidelity to the BF16 original on an internal mixed English/Korean set (100 prompts: the BF16 original's greedy answers, scored under each build with the same prompts and scoring code). This build was measured on the rebuilt files with vLLM 0.32 on 2026-11-01; the BF16 reference column was measured on 2026-10-27.

Metric This model BF16 original
Perplexity (lower = closer) 1.2948 1.2934 (+0.1%)
Top-1 next-token agreement 97.6% 98.0%

These measure fidelity to the base model, not absolute task scores.

Requirements

Runtime vLLM โ€” the architecture is in vLLM โ‰ฅ 0.17.0; tested here with vLLM 0.32.0.
vLLM install vLLM 0.32 itself needs Python 3.11โ€“3.14, torch 2.13 (CUDA 13.0 build, so an NVIDIA driver with CUDA 13 support), transformers โ‰ฅ5.16.1,<5.20 and compressed-tensors 0.17.0 (both pinned by vLLM; 0.17.0 reads this checkpoint, which was saved with 0.19.0 โ€” do not upgrade it separately).
GPU NVIDIA Turing (sm75) or newer โ€” weight-only INT, no special tensor cores needed. V100 (sm70) is not supported. (Kernel choice per GPU from vLLM 0.32's selection code; only B200 was run here.)
Memory Weights: 40.4 GB. Add KV cache (grows with context length ร— concurrent requests) and a few GB for activations / CUDA graphs; use --tensor-parallel-size N when the weights do not fit one GPU.
Transformers (without vLLM) transformers>=5.13.0.
trust_remote_code not needed

Usage (vLLM)

Start an OpenAI-compatible server:

vllm serve axteam/Qwen3.6-35B-A3B-AWQ-8bit --max-model-len 32768

Query it:

curl http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "axteam/Qwen3.6-35B-A3B-AWQ-8bit",
  "messages": [{"role": "user", "content": "๋Œ€ํ•œ๋ฏผ๊ตญ์˜ ์ˆ˜๋„๋Š” ์–ด๋””์ธ๊ฐ€์š”?"}],
  "temperature": 1.0, "top_p": 0.95, "top_k": 20, "max_tokens": 512
}'

Offline inference in Python:

from vllm import LLM, SamplingParams

llm = LLM(model="axteam/Qwen3.6-35B-A3B-AWQ-8bit", max_model_len=32768)
params = SamplingParams(temperature=1.0, top_p=0.95, top_k=20, max_tokens=1024)
out = llm.chat([{"role": "user", "content": "Explain model quantization in two sentences."}], params)
print(out[0].outputs[0].text)

# image input (URL or data: URI)
messages = [{"role": "user", "content": [
    {"type": "image_url", "image_url": {"url": "https://example.com/your-image.png"}},
    {"type": "text", "text": "Describe this image."},
]}]
out = llm.chat(messages, params)
print(out[0].outputs[0].text)

Recommended sampling (from the original generation_config.json): temperature=1.0, top_p=0.95, top_k=20. Thinking mode is on by default in the chat template; to turn it off pass chat_template_kwargs={"enable_thinking": False} (Python llm.chat(..., chat_template_kwargs=...), or "chat_template_kwargs": {"enable_thinking": false} in the request body). Text-only serving: add --limit-mm-per-prompt '{"image": 0, "video": 0}' to skip loading the vision encoder's memory budget.

Reducing KV-cache memory (optional, serve-time)

The KV cache is kept in BF16 by default. KV-cache precision is a runtime choice, independent of the weight quantization above. This is a hybrid model: only its 10 full-attention layers keep a KV cache, while its 30 linear-attention (Gated DeltaNet) layers keep a fixed-size state per request, which --kv-cache-dtype does not change. FP8 halves the full-attention KV cache, so the saving grows with context length and is small for many short requests:

vllm serve axteam/Qwen3.6-35B-A3B-AWQ-8bit --kv-cache-dtype fp8

License

Quantized derivative of Qwen/Qwen3.6-35B-A3B (Apache-2.0). All original weights, tokenizer and chat template belong to the Qwen team.


ํ•œ๊ตญ์–ด ์š”์•ฝ

Qwen/Qwen3.6-35B-A3B์˜ AWQ INT8 (W8A16) ์–‘์žํ™” ๋นŒ๋“œ ยท ์ œ์ž‘ axteam

๊ฐœ์š”

ํ•ญ๋ชฉ ๋‚ด์šฉ
์›๋ณธ ๋ชจ๋ธ Qwen/Qwen3.6-35B-A3B
์–‘์žํ™” ๋ฐฉ์‹ AWQ INT8 (W8A16)
์ €์žฅ ํ˜•์‹ compressed-tensors pack-quantized
๋ชจ๋ธ ๊ตฌ์กฐ ํ•˜์ด๋ธŒ๋ฆฌ๋“œ ์–ดํ…์…˜(์„ ํ˜• ์–ดํ…์…˜ + ํ’€ ์–ดํ…์…˜) ยท MoE (์ „๋ฌธ๊ฐ€ 256๊ฐœ + ๊ณต์œ  ์ „๋ฌธ๊ฐ€, ์ด 35B ์ค‘ ์•ฝ 3B ํ™œ์„ฑ) ยท ๋ฉ€ํ‹ฐ๋ชจ๋‹ฌ(๋น„์ „) ยท MTP ํ—ค๋“œ
์šฉ๋Ÿ‰ ์•ฝ 40.4 GB
๋ณด์ • ๋ฐ์ดํ„ฐ ์ƒ˜ํ”Œ 256๊ฐœ (ํ•œ๊ตญ์–ด ์œ„์ฃผ ํ˜ผํ•ฉ ์ฝ”ํผ์Šค)

์–‘์žํ™” ์ƒ์„ธ

  • ๊ฐ€์ค‘์น˜: INT8 ๋Œ€์นญ, ๊ทธ๋ฃน ํฌ๊ธฐ 32
  • ํ™œ์„ฑํ™”: BF16 (์–‘์žํ™”ํ•˜์ง€ ์•Š์Œ)
  • ์–‘์žํ™” ์ ˆ์ฐจ: PTQ(ํ•™์Šต ํ›„ ์–‘์žํ™”, ์žฌํ•™์Šต ์—†์Œ), AWQ ๋ฐฉ์‹ โ€” ๋ณด์ • ๋ฐ์ดํ„ฐ๋กœ ํ™œ์„ฑํ™”๊ฐ€ ํฐ ๊ฐ€์ค‘์น˜ ์ฑ„๋„์„ ์ฐพ์•„ ์ฑ„๋„๋ณ„ ์Šค์ผ€์ผ๋กœ ๋ณดํ˜ธํ•œ ๋’ค INT8๋กœ ๋ฐ˜์˜ฌ๋ฆผ
  • BF16 ์œ ์ง€: ๋น„์ „ ํƒ€์›Œยทํˆฌ์˜์ธต(model.visual.*), MTP ํ—ค๋“œ(mtp.*), ๋ชจ๋“  RMSNorm, ํ† ํฐ ์ž„๋ฒ ๋”ฉ, LM ํ—ค๋“œ, MoE ๋ผ์šฐํ„ฐ(mlp.gate)์™€ ๊ณต์œ  ์ „๋ฌธ๊ฐ€ ๊ฒŒ์ดํŠธ, SSM์˜ ์„ ํ˜•์ธต ์™ธ ํŒŒ๋ผ๋ฏธํ„ฐ(linear_attn.conv1dยทA_logยทdt_bias) โ€” ์ „์ฒด ํŒŒ๋ผ๋ฏธํ„ฐ์˜ ์•ฝ 6%
  • ์›๋ณธ ํŒŒ์ผ: ํ† ํฌ๋‚˜์ด์ €ยทchat templateยทgeneration config๋Š” ์›๋ณธ๊ณผ ๋™์ผ, config.json์€ ์›๋ณธ์— quantization_config ๋ธ”๋ก๋งŒ ์ถ”๊ฐ€

ํ’ˆ์งˆ (์‹ค์ธก)

  • ์›๋ณธ ๋Œ€๋น„ ์ถฉ์‹ค๋„: PPL 1.2948 (์›๋ณธ 1.2934, +0.1%, ๋‚ฎ์„์ˆ˜๋ก ์›๋ณธ์— ๊ฐ€๊นŒ์›€), ๋‹ค์Œ ํ† ํฐ top-1 ์ผ์น˜์œจ 97.6% (์›๋ณธ ์ž์ฒด ๊ธฐ์ค€๊ฐ’ 98.0%) โ€” ๋‚ด๋ถ€ ์˜์–ดยทํ•œ๊ตญ์–ด ํ˜ผํ•ฉ ํ”„๋กฌํ”„ํŠธ 100๊ฐœ, ์›๋ณธ(BF16)์˜ greedy ๋‹ต๋ณ€์„ ๊ธฐ์ค€ ํ…์ŠคํŠธ๋กœ ์ธก์ •
  • ๋ฒค์น˜๋งˆํฌ ํ‰๊ท : 60.80 (์›๋ณธ 61.09, 99.5%) โ€” MMLU(์ƒ์„ฑํ˜•)ยทGSM8KยทARC-ChallengeยทWinograndeยทKMMLUยทHAE-RAE 6์ข… ํ‰๊ท , lm-evaluation-harness + vLLM, chat template ์—†๋Š” ์™„์„ฑํ˜• ํ”„๋กฌํ”„ํŠธ(๊ณผ์ œ๋ณ„ 0ยท5-shot, ํ‘œ ์ฐธ๊ณ )ยทgreedy, ๋ชจ๋“  ๋นŒ๋“œ๋ฅผ ๊ฐ™์€ ์„ค์ •์œผ๋กœ ์ธก์ •
  • MMLU(์ƒ์„ฑํ˜•)ยทGSM8K ์ œ์™ธ ํ‰๊ท : 67.99 (์›๋ณธ 67.93, 100.1%) โ€” MMLU(์ƒ์„ฑํ˜•)ยทGSM8K๋Š” ์ƒ์„ฑํ•œ ๋‹ต์ด ์ฑ„์ ๊ธฐ๊ฐ€ ๊ธฐ๋Œ€ํ•˜๋Š” ํ˜•์‹ ๊ทธ๋Œ€๋กœ์—ฌ์•ผ ์ ์ˆ˜๊ฐ€ ๋‚˜๋Š” ๊ณผ์ œ; ์ด ๋ชจ๋ธ์—์„œ๋Š” ๊ฐ™์€ ๋ชจ๋ธ์˜ ๋นŒ๋“œ๋ผ๋ฆฌ MMLU(์ƒ์„ฑํ˜•) 64.074.1ยทGSM8K 21.571.1๋กœ ํฌ๊ฒŒ ํ”๋“ค๋ฆฌ๊ณ  ์–‘์žํ™” ๋นŒ๋“œ 2๊ฐœ๊ฐ€ ์›๋ณธ๋ณด๋‹ค ํฌ๊ฒŒ ๋†’์Œ(MMLU(์ƒ์„ฑํ˜•): AWQ-4bit 115.0%; GSM8K: AWQ-4bit 233.9%ยทNVFP4 153.1%), ๋‹ต ํ˜•์‹ ์ฐจ์ด๋กœ ๋ณด์ด๋‚˜ ๋ฌธํ•ญ๋ณ„ ์ถœ๋ ฅ์„ ์ €์žฅํ•˜์ง€ ์•Š์•„ ๋ฌธํ•ญ ๋‹จ์œ„๋กœ๋Š” ํ™•์ธํ•˜์ง€ ์•Š์Œ
  • ํ•œ๊ตญ์–ด ๋ฒค์น˜๋งˆํฌ: KMMLU 65.18 (์›๋ณธ 65.14, 100.1%) ยท HAE-RAE 75.07 (์›๋ณธ 74.89, 100.2%)
  • MMLU ๋ฐฉ์‹: lm-eval ์ƒ์„ฑํ˜• ๋ณ€ํ˜•(mmlu_generative) โ€” ๋ชจ๋ธ์ด ๋‹ต ๊ธ€์ž๋ฅผ ์ƒ์„ฑํ•˜๊ณ  ์ฒซ ์ค„์ด ์ •ํ™•ํžˆ ์ผ์น˜ํ•ด์•ผ ์ •๋‹ต์œผ๋กœ ์ฑ„์ ํ•˜๋ฏ€๋กœ, ๋‹ค๋ฅธ ๋ฐฉ์‹(์˜ˆ: log-likelihood ์ฑ„์ )์œผ๋กœ ์žฐ MMLU ์ ์ˆ˜์™€ ์ง์ ‘ ๋น„๊ต ๋ถˆ๊ฐ€

์š”๊ตฌ์‚ฌํ•ญ

  • ์‹คํ–‰ ์—”์ง„: vLLM 0.17.0 ์ด์ƒ(์ด ๊ตฌ์กฐ๋ฅผ ์ง€์›ํ•˜๋Š” ์ฒซ ๋ฒ„์ „), ์ด ๋นŒ๋“œ๋Š” vLLM 0.32.0์—์„œ ์‹คํ–‰ ํ™•์ธ
  • vLLM 0.32 ์„ค์น˜ ์กฐ๊ฑด: Python 3.11~3.14, torch 2.13(CUDA 13.0 ๋นŒ๋“œ, CUDA 13 ์ง€์› ๋“œ๋ผ์ด๋ฒ„ ํ•„์š”), transformers 5.16.1 ์ด์ƒ 5.20 ๋ฏธ๋งŒ, compressed-tensors 0.17.0 โ€” transformersยทcompressed-tensors๋Š” vLLM์ด ๋ฒ„์ „์„ ๊ณ ์ •ํ•˜๋ฏ€๋กœ compressed-tensors๋ฅผ ๋”ฐ๋กœ ์—…๊ทธ๋ ˆ์ด๋“œํ•˜์ง€ ๋ง ๊ฒƒ (์ด ์ฒดํฌํฌ์ธํŠธ๋Š” 0.19.0์œผ๋กœ ์ €์žฅ๋์ง€๋งŒ 0.17.0์—์„œ๋„ ๋กœ๋“œ๋จ)
  • GPU: NVIDIA Turing(sm75) ์ด์ƒ, V100(sm70) ๋ฏธ์ง€์› โ€” ๊ฐ€์ค‘์น˜ ์ „์šฉ ์ •์ˆ˜ ํ˜•์‹์ด๋ผ FP8ยทFP4 ์ „์šฉ ํ…์„œ์ฝ”์–ด ๋ถˆํ•„์š” (GPU๋ณ„ ์ง€์› ์—ฌ๋ถ€๋Š” vLLM 0.32 ์ปค๋„ ์„ ํƒ ์ฝ”๋“œ ๊ธฐ์ค€, ์‹ค์ œ ์‹คํ–‰์€ B200์—์„œ๋งŒ ํ™•์ธ)
  • ๋ฉ”๋ชจ๋ฆฌ: ๊ฐ€์ค‘์น˜ 40.4 GB + KV ์บ์‹œ(์ปจํ…์ŠคํŠธ ๊ธธ์ด ร— ๋™์‹œ ์š”์ฒญ ์ˆ˜์— ๋น„๋ก€) + ํ™œ์„ฑํ™”ยทCUDA graph์šฉ ์ˆ˜ GB โ€” GPU ํ•œ ์žฅ์— ๋‹ค ๋“ค์–ด๊ฐ€์ง€ ์•Š์œผ๋ฉด --tensor-parallel-size N์œผ๋กœ ๋ถ„์‚ฐ
  • Transformers๋กœ ์ง์ ‘ ์‚ฌ์šฉ: transformers 5.13.0 ์ด์ƒ
  • trust_remote_code: ๋ถˆํ•„์š”

์‚ฌ์šฉ๋ฒ•

  • ์„œ๋ฒ„ ์‹คํ–‰: vllm serve axteam/Qwen3.6-35B-A3B-AWQ-8bit --max-model-len 32768 โ€” OpenAI ํ˜ธํ™˜ API
  • Python: LLM(model="axteam/Qwen3.6-35B-A3B-AWQ-8bit", max_model_len=32768).chat(...) โ€” ์ „์ฒด ์˜ˆ์‹œ๋Š” ์œ„ ์˜์–ด ๋ณธ๋ฌธ 'Usage (vLLM)' ์ ˆ ์ฐธ๊ณ 
  • ๊ถŒ์žฅ ์ƒ˜ํ”Œ๋ง: temperature=1.0, top_p=0.95, top_k=20 (์›๋ณธ generation_config.json ๊ธฐ์ค€)
  • thinking ๋ชจ๋“œ: ๊ธฐ๋ณธ ์ผœ์ง โ€” ๋„๋ ค๋ฉด chat_template_kwargs={"enable_thinking": False} ์ „๋‹ฌ(Python์€ llm.chat() ์ธ์ž, HTTP๋Š” ์š”์ฒญ ๋ณธ๋ฌธ์˜ "chat_template_kwargs": {"enable_thinking": false})
  • ์ด๋ฏธ์ง€ ์ž…๋ ฅ: ๋ฉ”์‹œ์ง€์— image_url ํ•ญ๋ชฉ ์‚ฌ์šฉ
  • ํ…์ŠคํŠธ๋งŒ ์‚ฌ์šฉํ•  ๋•Œ: --limit-mm-per-prompt '{"image": 0, "video": 0}' โ€” ๋น„์ „ ํƒ€์›Œ์šฉ ๋ฉ”๋ชจ๋ฆฌ ์˜ˆ์•ฝ์„ ์ƒ๋žตํ•ด ๋ฉ”๋ชจ๋ฆฌ ์ ˆ์•ฝ
  • KV ์บ์‹œ ์ ˆ๊ฐ(์„ ํƒ): vllm serve์— --kv-cache-dtype fp8 ์ถ”๊ฐ€ โ€” ํ•˜์ด๋ธŒ๋ฆฌ๋“œ ๊ตฌ์กฐ๋ผ KV ์บ์‹œ๋Š” ํ’€ ์–ดํ…์…˜ 10๊ฐœ ์ธต์—๋งŒ ์žˆ๊ณ , ์„ ํ˜• ์–ดํ…์…˜(Gated DeltaNet) 30๊ฐœ ์ธต์˜ ์š”์ฒญ๋ณ„ ๊ณ ์ • ํฌ๊ธฐ ์ƒํƒœ๋Š” ์ด ์˜ต์…˜์œผ๋กœ ๋ฐ”๋€Œ์ง€ ์•Š์Œ โ†’ ํ’€ ์–ดํ…์…˜ KV ์บ์‹œ๋งŒ ์ ˆ๋ฐ˜์œผ๋กœ ์ค„์–ด ๊ธด ์ปจํ…์ŠคํŠธ์—์„œ ํšจ๊ณผ๊ฐ€ ํฌ๊ณ  ์งง์€ ์š”์ฒญ์ด ๋งŽ์„ ๋•Œ๋Š” ์ž‘์Œ (๊ฐ€์ค‘์น˜ ์–‘์žํ™”์™€ ๋ณ„๊ฐœ์ธ ์‹คํ–‰ ์˜ต์…˜)

์ฐธ๊ณ 

  • ์ˆ˜์ • ์ด๋ ฅ (2026-11-01, ์žฌ์–‘์žํ™”): MoE ๋ผ์šฐํ„ฐ(mlp.gate)ยท๊ณต์œ  ์ „๋ฌธ๊ฐ€ ๊ฒŒ์ดํŠธ์— AWQ ์ƒ์‡„ ์Šค์ผ€์ผ ์ ์šฉ (๋ชจ๋“  ์ธต์—์„œ ์›๋ณธ ๋Œ€๋น„ ๋ผ์šฐํ„ฐ ๋ฐฐ์œจ์ด norm ๋ฐฐ์œจ๊ณผ ์ผ์น˜ํ•จ์„ ํ™•์ธ, ์ฑ„๋„๋ณ„ ์ฐจ์ด ์ค‘์•™๊ฐ’ 2% ๋ฏธ๋งŒ) โ€” ์ด์ „ ์—…๋กœ๋“œ์˜ ๊ฒฐํ•จ: ์ฒซ ๋ฒˆ์งธ ์—…๋กœ๋“œ๋Š” ๋ผ์šฐํ„ฐยท๊ณต์œ  ์ „๋ฌธ๊ฐ€ ๊ฒŒ์ดํŠธ ์ƒ์‡„ ๋ˆ„๋ฝ์œผ๋กœ ์›๋ณธ๊ณผ ๋‹ค๋ฅธ ์ „๋ฌธ๊ฐ€ ์„ ํƒ โ€” 2026-11-01 06:13:53 UTC(๋ฆฌ๋น„์ „ 05a077a) ์ด์ „์— ๋ฐ›์•˜๋‹ค๋ฉด ๋‹ค์‹œ ๋ฐ›์„ ๊ฒƒ, ํ’ˆ์งˆ ํ•ญ๋ชฉ์˜ ๋ฒค์น˜๋งˆํฌยท์ถฉ์‹ค๋„๋Š” ์žฌ์–‘์žํ™” ๋นŒ๋“œ๋กœ ์žฐ ๊ฐ’

๋ผ์ด์„ ์Šค

  • ๋ผ์ด์„ ์Šค: Apache-2.0 โ€” ์›๋ณธ ๋ชจ๋ธ์˜ ๋ผ์ด์„ ์Šค๋ฅผ ๊ทธ๋Œ€๋กœ ๋”ฐ๋ฆ„
  • ๊ถŒ๋ฆฌ: ์›๋ณธ ๊ฐ€์ค‘์น˜ยทํ† ํฌ๋‚˜์ด์ €ยทchat template์€ Qwen ํŒ€(Alibaba) ์†Œ์œ , ์ด ๋ฆฌํฌ๋Š” ์–‘์žํ™” ํŒŒ์ƒ๋ฌผ
Downloads last month
346
Safetensors
Model size
36B params
Tensor type
I32
ยท
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for axteam/Qwen3.6-35B-A3B-AWQ-8bit

Quantized
(1,003)
this model

Collection including axteam/Qwen3.6-35B-A3B-AWQ-8bit