DeepSeek has released DeepInfer, an open source inference engine designed to lower the computing costs of running artificial intelligence models.
The announcement, made on October 20, 2026, intensifies competitive pressure on OpenAI and Anthropic.
The software supports both NVIDIA and Huawei hardware, giving developers access to technologies DeepSeek uses in its own production environment.
Inference is the computationally intensive process through which trained AI models generate responses to user prompts.
DeepSeek says the engine is intended to reduce computing expenses and energy consumption, though the repository does not provide independently verified performance improvements.
The company described its objective in an announcement published on X.
"We're sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable," DeepSeek said.
DeepInfer currently supports DeepSeek-V4.1-Flash-1017, with additional models expected as DeepSeek releases them publicly.
The engine uses an Apache 2.0 license, while the model weights are distributed under the MIT license, according to Forkast News.
The release offers a technical alternative to commercial inference systems.
Its code supports NVIDIA SM100 graphics processing units and Huawei Ascend 950 neural processing units through the same repository, with hardware selection controlled by the DEEPINFER_ACCELERATOR environment variable.
DeepSeek's repository includes FlashMLA, DeepGEMM, DeepEP, TileKernels, DeepSelect and custom TileLang kernels.
Other features include confidence-scheduled decoding, bounded replay, batch scheduling and structured outputs intended to support more controlled generation.
DeepSeek describes the engine's intended performance characteristics in its documentation.
"High throughput and low latency" is the performance description published by DeepSeek, according to Forkast News.
However, the repository does not publish numerical measurements demonstrating the engine's performance gains.
Forkast News reported that its benchmark charts use four SM100 GPUs for prefill and 64 GPUs for decoding within a single scale-up domain, and that independent benchmarks were not yet available.
DeepInfer also does not provide every component needed to operate a complete inference service.
Its documentation identifies key-value caching, separation of prefill and decoding workloads, and load balancing across data-parallel replicas as additional requirements for production deployment.
Developers may need to integrate these components from existing open source projects.
DeepSeek's acknowledgments describe the origins of the engine and identify related infrastructure.
"Originally built primarily on vLLM, with additional inspiration from SGLang and TensorRT-LLM," DeepSeek's acknowledgments state.
Those projects provide infrastructure for serving models at scale, including attention management, workload distribution and orchestration.
DeepSeek has also released a companion project, deepseek-recipe, containing Rust and Python components for translating API requests into DeepSeek prompts and processing model outputs.
The companion project's documentation specifies the remaining responsibilities for developers integrating the components.
"The caller must provide model inference, tool execution, and HTTP transport," the deepseek-recipe documentation states.
DeepInfer's release comes amid mounting pressure on AI developers to control the costs of running increasingly capable models.
A report from McKinsey & Company cited by Gizmodo projected that most data center computing capacity would be devoted to inference rather than training by the end of the decade.
Pricing differences illustrate the competitive challenge.
According to OpenRouter, DeepSeek-V4.1-Flash costs $0.035 per million input tokens and $0.12 per million output tokens, compared with $10 and $50, respectively, for OpenAI's GPT-6 Astra.
Meanwhile, the Trump administration announced a prohibition on federal civilian agencies using DeepSeek's AI tools on Monday, October 19, according to Gizmodo.
The restriction does not prohibit private companies or other private organizations from using the technology.
For US-based developers, the regulatory change introduces an additional consideration when deploying DeepSeek-based products.
Forkast News reported that the exclusion order was beginning to affect third-party products built on DeepSeek implementations, while the newly released engine remains available under its stated open source license.
