Hugging Face's logo
ManniX's picture

ManniX PRO

ManniX-ITA
30 6 28
Aptronym's profile picture DarioTD's profile picture bujchen's profile picture
Β·
https://github.com/mann1x

AI & ML interests

None yet

Recent Activity

repliedto their post about 3 hours ago
opencoti-llamafile β€” single-file inference engine for long-context agentic serving This is the bundled runtime of opencoti, a soft fork of opencode adding a multi-tier local inference engine. The main opencoti project is not yet publicly released on GitHub β€” its engine ships here first as a standalone, zero-dependency single file usable with any OpenAI-compatible client. One executable (Cosmopolitan APE) runs on Linux, Windows, macOS & BSD. Three artifacts: x86_64 with embedded CUDA (sm_75β†’sm_120, Turingβ†’Blackwell), a bare Windows .exe, and (untested!) aarch64 with embedded sbsa CUDA for DGX Spark (GB10). Side-load DSOs under dso/. Base: llamafile 0.10.3 / llama.cpp + 80 additive patches (full series published in this repo). Runs every llama.cpp-supported GGUF; tuning targets Gemma-4 (26B-A4B MoE, dense 12B/31B, E-series) and Qwen 2.5/3/3.5/3.6 (dense, MoE, recurrent-hybrid). Highlights: - DCA (dual-chunk attention): context extension validated to 768k–1M - MTP speculative decoding: Qwen NextN self-spec (fused N-step) + Gemma-4 assistant drafter (dual-context) β€” up to ~1.9Γ— decode - KV-quant ladder: TurboQuant 2/3-bit, TCQ trellis-coded 2/3-bit, asymmetric hi-K/cheap-V mixes, q6_0 - Rolling-KV window: streaming host-RAM spill of KV overflow β€” contexts beyond VRAM, auto T*-aware policy - Mixed position-axis KV: f16 recent window βŠ• quantized tail, auto-tiered at boot - Agentic multi-session serving: PolyKV shared-prefix KV pool, session-keyed KV reuse, lazy slot grow/shrink - Sparse-V + sparse attention (vertical-slash) long-ctx decode wins - Runtime introspection (/props, /slots), layer-repeat (RYS) self-stacking, and more Every feature is gated by KLD / logit-equivalence and RULER retrieval evals β€” quality numbers live in the docs alongside the binaries. Grab a binary + MANIFEST + USAGE.md and point your agent at it. Feedback very welcome! https://huggingface.co/ManniX-ITA/opencoti-llamafile
updated a model about 5 hours ago
ManniX-ITA/opencoti-llamafile
repliedto their post about 11 hours ago
opencoti-llamafile β€” single-file inference engine for long-context agentic serving This is the bundled runtime of opencoti, a soft fork of opencode adding a multi-tier local inference engine. The main opencoti project is not yet publicly released on GitHub β€” its engine ships here first as a standalone, zero-dependency single file usable with any OpenAI-compatible client. One executable (Cosmopolitan APE) runs on Linux, Windows, macOS & BSD. Three artifacts: x86_64 with embedded CUDA (sm_75β†’sm_120, Turingβ†’Blackwell), a bare Windows .exe, and (untested!) aarch64 with embedded sbsa CUDA for DGX Spark (GB10). Side-load DSOs under dso/. Base: llamafile 0.10.3 / llama.cpp + 80 additive patches (full series published in this repo). Runs every llama.cpp-supported GGUF; tuning targets Gemma-4 (26B-A4B MoE, dense 12B/31B, E-series) and Qwen 2.5/3/3.5/3.6 (dense, MoE, recurrent-hybrid). Highlights: - DCA (dual-chunk attention): context extension validated to 768k–1M - MTP speculative decoding: Qwen NextN self-spec (fused N-step) + Gemma-4 assistant drafter (dual-context) β€” up to ~1.9Γ— decode - KV-quant ladder: TurboQuant 2/3-bit, TCQ trellis-coded 2/3-bit, asymmetric hi-K/cheap-V mixes, q6_0 - Rolling-KV window: streaming host-RAM spill of KV overflow β€” contexts beyond VRAM, auto T*-aware policy - Mixed position-axis KV: f16 recent window βŠ• quantized tail, auto-tiered at boot - Agentic multi-session serving: PolyKV shared-prefix KV pool, session-keyed KV reuse, lazy slot grow/shrink - Sparse-V + sparse attention (vertical-slash) long-ctx decode wins - Runtime introspection (/props, /slots), layer-repeat (RYS) self-stacking, and more Every feature is gated by KLD / logit-equivalence and RULER retrieval evals β€” quality numbers live in the docs alongside the binaries. Grab a binary + MANIFEST + USAGE.md and point your agent at it. Feedback very welcome! https://huggingface.co/ManniX-ITA/opencoti-llamafile
View all activity

Organizations

None yet