Swen 1.1 Technical Report: Efficient Edge Intelligence via Hybrid Double-Gated Convolutions and Test-Time Metacognitive Reasoning
darsh@sorika.aiAbstract
Small language models (SLMs, ≤ 2B parameters) are pivotal for privacy-preserving, low-latency, on-device artificial intelligence. However, standard causal Transformers suffer from two acute structural handicaps: quadratic sequence prefill complexity and linearly expanding key-value (KV) memory footprints during autoregressive decoding. In this paper, we introduce the Swen 1.1 Model Family, a suite of three ultra-efficient foundation and specialized models developed by Sorika Labs:
Swen-1.1-Instruct (1.2B)
Compact assistant optimized for instruction following, multilingual dialogue across 8 languages, agentic tool execution, and code synthesis.
Swen-1-Math (350M)
Ultra-compact mathematical specialist leveraging an autonomous Chain-of-Thought scratchpad to deliver 1B-class arithmetic & algebraic deduction.
Swen-1.1-Thinking (1.2B)
Self-reflective, backtrack-capable reasoning engine designed for combinatorial puzzles and game theory with zero-drift backward induction.
1. Introduction & The Sub-2B Dilemma
The exponential scaling of frontier autoregressive large language models (LLMs) has yielded extraordinary reasoning capabilities, yet these gains have come at the cost of immense computational, energetic, and financial budgets. In resource-constrained operating domains—such as client laptops, edge devices, smart terminals, and local embedded systems—pure quadratic-attention Transformer architectures become prohibitive:
- The KV Cache Memory Wall: Generating text over 128k context consumes over 16.8 GB of RAM in standard architectures, crashing consumer hardware.
- Quadratic Prefill Latency: Standard attention requires O(L^2 · d) FLOPs, resulting in unbearable time-to-first-token delay on long document inputs.
- The Sub-2B Reasoning Deficit: Traditional compact models (≤ 1.5B) suffer catastrophic capacity loss on multi-step reasoning, deteriorating into factual hallucinations.
BibTeX Citation
@article{yadav2026swen11,
title={Swen 1.1 Technical Report: Efficient Edge Intelligence via Hybrid Double-Gated Convolutions and Test-Time Metacognitive Reasoning},
author={Yadav, Darsh and {The Sorika Labs Research Team}},
journal={Sorika Labs Technical Report Series 1.1},
year={2026},
month={September},
url={https://sorikalabs.com/research/swen}
}