AitopkitDeepSeek announced on October 20, 2026 that it has open-sourced DeepInfer, its in-house inference framework, releasing the code under the fully permissive Apache 2.0 license. The company says DeepInfer powers its flagship DeepSeek-V4.1-Flash at scale "with high throughput and low latency," and it is sharing the production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.
The repository ships the scheduler, optimized kernels, and token pipeline behind DeepSeek's serving infrastructure: distributed inference with Attention DP, context parallelism and MoE expert parallelism, a zero-overhead batch scheduler, structured outputs, and confidence-scheduled verification. It currently supports a single model — DeepSeek-V4.1-Flash — and runs on NVIDIA SM100 (Blackwell) GPUs and Huawei Ascend 950 NPUs. The README credits vLLM as the main base, with inspiration from SGLang and TensorRT-LLM, and DeepSeek also released deepseek-recipe, its production token encoding and decoding component.
Worth noting what is not included: the KV-cache system, prefill/decode disaggregation, and load balancing across replicas are left out, which DeepSeek says are tightly coupled to its internal infrastructure. This is offline inference software, not a finished managed serving product — and all performance figures so far are DeepSeek's own benchmarks. Still, the symbolism is hard to miss: a Hangzhou-based lab is giving away the exact kind of production-serving code that OpenAI and Anthropic keep inside their own data centers. The release landed one day after the US Department of Homeland Security published an order barring DeepSeek technology from federal civilian agency systems, and it quickly became one of the day's biggest developer threads on Hacker News.