Post
211
🏙️ Hugging Face Community Post
Title: 🧬 Experimenting with "Dynamic Chaos" in Tamil SLMs
Hi everyone! I just published a new experimental study on Small Language Model (SLM) resilience.
I took the Qwen2.5-0.5B model and put it through a "Chaos Phase" to see how much weight data a tiny model can lose before its understanding of classical Tamil grammar breaks.
Key highlights of the study:
Target Data: Fine-tuned on the Thirukkural (1,330 couplets + modern explanations).
The Chaos Step: Applied 20% random weight pruning but implemented "Layer Protection" for the Token Embeddings and LM Head to keep the characters readable.
Compression: 4-bit (Q4_K_M) quantization for extreme efficiency.
Result: A surrealist classical Tamil model that is ultra-light (~300MB) and ultra-fast!
Check out the model and the experiment logic here: Sri-Vigneshwar-DJ/qwen-tamil-chaos-v1
Title: 🧬 Experimenting with "Dynamic Chaos" in Tamil SLMs
Hi everyone! I just published a new experimental study on Small Language Model (SLM) resilience.
I took the Qwen2.5-0.5B model and put it through a "Chaos Phase" to see how much weight data a tiny model can lose before its understanding of classical Tamil grammar breaks.
Key highlights of the study:
Target Data: Fine-tuned on the Thirukkural (1,330 couplets + modern explanations).
The Chaos Step: Applied 20% random weight pruning but implemented "Layer Protection" for the Token Embeddings and LM Head to keep the characters readable.
Compression: 4-bit (Q4_K_M) quantization for extreme efficiency.
Result: A surrealist classical Tamil model that is ultra-light (~300MB) and ultra-fast!
Check out the model and the experiment logic here: Sri-Vigneshwar-DJ/qwen-tamil-chaos-v1