FlameF0X/LFM2.5-1.2B-Distilled-Claude | INT4 (W4A16)
#22
by INC4AI - opened
Pipeline Failure Report
Model: FlameF0X/LFM2.5-1.2B-Distilled-Claude
Quantization Scheme: INT4 (W4A16)
Failed Phase: quantize
Run ID: LFM2.5-1.2B-Distilled-Claude-AutoRound-W4A16-Tuning
Error Category: auto_round_compatibility
Full Error Log
Map: 0%| | 0/10000 [00:00<?, ? examples/s]
Map: 100%|██████████| 10000/10000 [00:09<00:00, 1100.45 examples/s]
Filter: 0%| | 0/10000 [00:00<?, ? examples/s]
Filter: 100%|██████████| 10000/10000 [00:04<00:00, 2130.27 examples/s]
Casting the dataset: 0%| | 0/1245 [00:00<?, ? examples/s]
Casting the dataset: 100%|██████████| 1245/1245 [00:03<00:00, 344.79 examples/s]
12:19:55 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/NeelNanda/pile-10k/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
12:19:55 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/NeelNanda/pile-10k/127bfedcd5047750df5ccf3a12979a47bfa0bafa/README.md "HTTP/1.1 200 OK"
12:19:55 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/NeelNanda/pile-10k/resolve/127bfedcd5047750df5ccf3a12979a47bfa0bafa/pile-10k.py "HTTP/1.1 404 Not Found"
12:19:55 [INFO] HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/NeelNanda/pile-10k/NeelNanda/pile-10k.py "HTTP/1.1 404 Not Found"
12:19:56 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/NeelNanda/pile-10k/revision/127bfedcd5047750df5ccf3a12979a47bfa0bafa "HTTP/1.1 200 OK"
12:19:56 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/NeelNanda/pile-10k/resolve/127bfedcd5047750df5ccf3a12979a47bfa0bafa/.huggingface.yaml "HTTP/1.1 404 Not Found"
12:19:56 [INFO] HTTP Request: GET /static-proxy?url=https%3A%2F%2Fdatasets-server.huggingface.co%2Finfo%3Fdataset%3DNeelNanda%2Fpile-10k "HTTP/1.1 200 OK"
12:19:56 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/NeelNanda/pile-10k/tree/127bfedcd5047750df5ccf3a12979a47bfa0bafa/data?recursive=true&expand=false "HTTP/1.1 200 OK"
12:19:56 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/NeelNanda/pile-10k/tree/127bfedcd5047750df5ccf3a12979a47bfa0bafa?recursive=false&expand=false "HTTP/1.1 200 OK"
[38;20m2026-06-24 12:20:14 INFO data_driven.py L685: caching done[0m
0%| | 0/16 [00:00<?, ?it/s]
Quantizing model.layers.0: 0%| | 0/16 [00:00<?, ?it/s][38;20mquantized 5/5 layers in the block, loss iter 0: 0.000003 -> iter 175: 0.000001[0m
[38;20m2026-06-24 12:26:17 INFO device.py L1840: 'peak_ram': 6.9GB, 'peak_vram': 4.25GB[0m
Quantizing model.layers.0: 6%|▋ | 1/16 [06:02<1:30:40, 362.73s/it]
Quantizing model.layers.1: 6%|▋ | 1/16 [06:02<1:30:40, 362.73s/it][38;20mquantized 5/5 layers in the block, loss iter 0: 0.000002 -> iter 72: 0.000001[0m
[38;20m2026-06-24 12:32:15 INFO device.py L1840: 'peak_ram': 6.9GB, 'peak_vram': 4.3GB[0m
Quantizing model.layers.1: 12%|█▎ | 2/16 [12:01<1:24:02, 360.18s/it]
Quantizing model.layers.2: 12%|█▎ | 2/16 [12:01<1:24:02, 360.18s/it]12:32:16 [ERROR] Quantization failed: Expected attn_mask dtype to be bool or float or to match query dtype, but got attn_mask.dtype: long int and query.dtype: c10::Half instead.
Traceback (most recent call last):
File "/root/_work/1/s/auto_quant/phases/quantize.py", line 282, in <module>
quantize(args)
File "/root/_work/1/s/auto_quant/phases/quantize.py", line 183, in quantize
autoround.quantize()
File "/root/.venv/lib/python3.12/site-packages/auto_round/compressors/data_driven.py", line 722, in quantize
self._quantize_blocks(
File "/root/.venv/lib/python3.12/site-packages/auto_round/compressors/data_driven.py", line 529, in _quantize_blocks
reference_output = self.quantizer._get_block_outputs(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/root/.venv/lib/python3.12/site-packages/auto_round/algorithms/quantization/base.py", line 452, in _get_block_outputs
tmp_output = _bf(
^^^^
File "/root/.venv/lib/python3.12/site-packages/auto_round/compressors/utils.py", line 208, in block_forward
output = block(**input_others)
^^^^^^^^^^^^^^^^^^^^^
File "/root/.venv/lib/python3.12/site-packages/transformers/modeling_layers.py", line 93, in __call__
return super().__call__(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1750, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.venv/lib/python3.12/site-packages/transformers/models/lfm2/modeling_lfm2.py", line 427, in forward
hidden_states, _ = self.self_attn(
^^^^^^^^^^^^^^^
File "/root/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1750, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.venv/lib/python3.12/site-packages/transformers/models/lfm2/modeling_lfm2.py", line 269, in forward
attn_output, attn_weights = attention_interface(
^^^^^^^^^^^^^^^^^^^^
File "/root/.venv/lib/python3.12/site-packages/transformers/integrations/sdpa_attention.py", line 92, in sdpa_attention_forward
attn_output = torch.nn.functional.scaled_dot_product_attention(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Expected attn_mask dtype to be bool or float or to match query dtype, but got attn_mask.dtype: long int and query.dtype: c10::Half instead.
Quantizing model.layers.2: 12%|█▎ | 2/16 [12:02<1:24:15, 361.11s/it]
Auto-generated by error_analysis pipeline. cc @lvkaokao
fixed
lvkaokao changed discussion status to closed