Sorika Labs Releases Swen 1.1 Model Family & Foundational Technical Report
- Three models released: Swen-1.1-Instruct (1.2B), Swen-1-Math (350M), and Swen-1.1-Thinking (1.2B)
- HumanEval Pass@1 of 66.0% — crushing SmolLM2-1.7B (22.6%) and Llama-3.2-1B (25.0%)
- Swen-1-Math scores 85.0% on MultiArith and 35.0% on Olympiad-grade MATH-500 in only 350M parameters
- Hybrid 16-layer interleaved topology: 10 double-gated causal convolutions and 6 sparse GQA attention layers
- Full technical report published by Founder Darsh Yadav with 128,000 token context window
Today, Sorika Labs is announcing the official release of the Swen 1.1 Model Family alongside our foundational Technical Report. Engineered under the leadership of founder Darsh Yadav, Swen 1.1 solves the quadratic memory wall of edge LLMs, delivering frontier-level reasoning, code synthesis, and mathematical deduction within a compact sub-2B parameter budget.
Breaking the Edge Intelligence Memory Wall
Why traditional Transformers fail on consumer edge devices.
Small language models (≤ 2B parameters) are pivotal for privacy-preserving, low-latency, on-device artificial intelligence. However, standard causal Transformers suffer from quadratic sequence prefill complexity and linearly expanding key-value (KV) memory footprints during autoregressive decoding.
For a 128,000 token context window, a traditional Transformer requires over 16.8 GB of RAM just to maintain the KV cache. Swen 1.1 introduces an interleaved hybrid topology—combining 10 double-gated causal convolutions with 6 sparse grouped-query attention layers—slashing the KV cache memory to just 1.57 GB (a 10.6× reduction).
“Our core thesis with Swen 1.1 is that heterogeneous token mixing—local linear-time convolutions combined with sparse global attention—unlocks true frontier-grade reasoning on standard edge hardware with zero compromises.”
— Darsh Yadav — Founder & Chief AI Architect, Sorika LabsThe Three Swen 1.1 Architectures
Targeted specialization across instruction, mathematics, and metacognitive reflection.
1. Swen-1.1-Instruct (1.2B): General-purpose compact assistant scoring 66.0% on HumanEval, 54.0% on GSM8K, and 38.0% on GPQA Diamond, with native tool-calling and 8-language fluency.
2. Swen-1-Math (350M): Pure mathematical specialist delivering 85.0% MultiArith and 35.0% Olympiad MATH-500 via autonomous Chain-of-Thought scratchpads, running entirely in CPU cache.
3. Swen-1.1-Thinking (1.2B): Metacognitive reasoning engine generating at 95 tokens/sec with constrained-entropy decoding (T=0.08) for zero-drift backward induction in game theory and logic puzzles.
Open Research & Next Steps
Full technical report and open intelligence access.
The complete 48-page technical report by Darsh Yadav is available today in the Sorika Research Hub. Model weights, inference runtimes, and evaluation harnesses are being made accessible under the Sorika Open Intelligence initiative.