Every Time I Hire a Linguist, Inference Costs Go Down: On Linguistic Rules as Effective Prompt Compressors
Paper • 2607.25335 • Published • 1
Stefan Schweter PRO
stefan-it
AI & ML interests
Flair Library 💕, NER & PoS Tagging, LM Pretraining (mostly encoder-only & encoder-decoder), Historical Language Models, German Language Models, Bavarian NLP 🥨, xLSTM
Recent Activity
new activity about 11 hours ago
lang-uk/electra-base-ukrainian-v2-cased-discriminator:Adding `safetensors` variant of this model liked a dataset 2 days ago
Qyrou/reasoning-corpus-4K-5M-v1Organizations
Adding `safetensors` variant of this model
#1 opened 11 days ago
by 
SFconvertbot
upvoted a paper 1 day ago
liked a
dataset 2 days ago
reacted to FredyRivera-dev's post with 🚀 2 days ago
Post
6193
We wrote a full technical guide on how to train a bilingual (ES/EN) LLM from scratch: TinyQwen.
Covers:
- Hybrid architecture based on Qwen3.5
- Pre-training with 15B tokens
- Cost benchmark between H200 and B200
- Post-training with SFT + LoRA
- Full code and data, open source
With ~$11 of compute on an H200 we ran an initial training run, enough to validate the full architecture and pipeline.
Blog post: https://aquiles-ai.vercel.app/blog/tinyqwen-from-scratch
Technical feedback welcome, especially from anyone looking to replicate the pipeline with more compute.
Covers:
- Hybrid architecture based on Qwen3.5
- Pre-training with 15B tokens
- Cost benchmark between H200 and B200
- Post-training with SFT + LoRA
- Full code and data, open source
With ~$11 of compute on an H200 we ran an initial training run, enough to validate the full architecture and pipeline.
Blog post: https://aquiles-ai.vercel.app/blog/tinyqwen-from-scratch
Technical feedback welcome, especially from anyone looking to replicate the pipeline with more compute.
- 3 replies
commented a
paper 2 days ago
upvoted a paper 2 days ago
upvoted a collection 2 days ago
upvoted a paper 3 days ago
liked a
model 3 days ago
commented a
paper 3 days ago
upvoted a paper 3 days ago
upvoted a paper 6 days ago
liked 2
datasets 6 days ago
liked 2
models 6 days ago
upvoted a collection 6 days ago
upvoted a paper 6 days ago
liked a
dataset 7 days ago
upvoted an article 8 days ago
Article
Be Ready Before the Attack: A Practical Guide to Self-Hosting an Open Model for Cyber Defense
jeffboudier
• • 20