balochiml (Balochi Machine Learning)

posted an update about 19 hours ago

Post

64

I made fastText 4× faster on CPU! 🚀

Introducing fastText-turbo:
⚡ 4.1× faster on x86, 1.7× on Apple Silicon
✅ Same models, same predictions (labels)
✅ Drop-in replacement, no retraining needed
How it works:
https://commonglot.github.io/blog/fasttext-turbo.html
Code:
https://github.com/CommonGlot/fasttext-turbo

kargaranamir

authored a paper 4 months ago

Who Flips? Self- and Cross-Model Counterarguments Reveal Answer Instability in LLMs

Paper • 2606.16011 • Published Jun 14 • 4

kargaranamir

submitted a paper to Daily Papers 4 months ago

Who Flips? Self- and Cross-Model Counterarguments Reveal Answer Instability in LLMs

Paper • 2606.16011 • Published Jun 14 • 4

kargaranamir

authored a paper 6 months ago

GlotOCR Bench: OCR Models Still Struggle Beyond a Handful of Unicode Scripts

Paper • 2604.12978 • Published Apr 14 • 6

kargaranamir

submitted a paper to Daily Papers 6 months ago

GlotOCR Bench: OCR Models Still Struggle Beyond a Handful of Unicode Scripts

Paper • 2604.12978 • Published Apr 14 • 6

kargaranamir

authored a paper 11 months ago

Insights from the ICLR Peer Review and Rebuttal Process

Paper • 2511.15462 • Published Nov 19, 2025 • 7

kargaranamir

authored a paper about 1 year ago

CoBia: Constructed Conversations Can Trigger Otherwise Concealed Societal Biases in LLMs

Paper • 2510.09871 • Published Oct 10, 2025 • 3

kargaranamir

authored 4 papers over 1 year ago

FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language

Paper • 2506.20920 • Published Jun 26, 2025 • 81

authored 3 papers almost 2 years ago

How Transliterations Improve Crosslingual Alignment

Paper • 2409.17326 • Published Sep 25, 2024 • 1

GlotCC: An Open Broad-Coverage CommonCrawl Corpus and Pipeline for Minority Languages

Paper • 2410.23825 • Published Oct 31, 2024 • 4

MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment

Paper • 2410.05873 • Published Oct 8, 2024 • 3

kargaranamir

posted an update over 2 years ago

Post

1,909

Introducing GlotCC: a new 2TB corpus based on an early 2024 CommonCrawl snapshot with data for 1000+ languages.

🤗 corpus v1: cis-lmu/GlotCC-V1
🐱 pipeline v3: https://github.com/cisnlp/GlotCC

More details? Stay tuned for our upcoming paper.
More data? In the next version, we plan to include additional snapshots of CommonCrawl.

Limitation: Due to the lower frequency of low-resource languages compared to others, there are sometimes only a few sentences available for very low-resource languages. However, the data volume for English in this version stands at 750GB, and the top 200 languages still have a strong presence in our data (see plot attached; we write the index for every 20 languages, meaning the 10th index is the 200th language).