berkay123001/turnike-jev-4b-fp8

Turnike-JEV-4B — Open-Weight Decision Model (FP8, torchao)

The merged Turnike-JEV-4B weights (berkay123001/turnike-jev-4b-merged) with the language-model linear layers stored in FP8 (torchao Float8DynamicActivationFloat8WeightConfig, per-row weight scales, dynamic activations; the vision tower, lm_head and the small in_proj_a / in_proj_b projections stay bf16). About half the size of the bf16 weights (loads on an 8 GB GPU; the Turnike-JEV-4B card lists the measured memory). Needs torchao and a GPU with FP8 matmul (sm_89+: RTX 40xx / 50xx, L4, H100, Blackwell).

# pip install torch "transformers>=5.17" torchao accelerate ; karar.py: hf download berkay123001/turnike-jev-4b karar.py --local-dir .
from karar import Karar
k = Karar(None, taban="berkay123001/turnike-jev-4b-fp8")          # already quantized: do not pass fp8=True
k.sor("Kulaklık bozuk geldi, paramı geri istiyorum.", {"Yazar iade istiyor mu?": {"type": "noul"}})

Same decisions as Karar(None, taban="berkay123001/turnike-jev-4b-merged", fp8=True) (load-time FP8 from the bf16 weights). FP8 keeps accuracy within noise (EN-300 0.769 → 0.767, tr_insan 0.748 → 0.740) and the argmax on our test prompts; individual probabilities move by up to a few hundredths, so use the bf16 weights if you threshold near 0.5. Model card, evaluation, training data, limitations and declarations: see berkay123001/turnike-jev-4b. License: Apache-2.0 (base model Qwen/Qwen3.5-4B: Apache-2.0, Copyright 2026 Alibaba Cloud).