Deploying Kimi K3 on AWS

Moonshot AI released Kimi K3 on July 27, 2026 — a 2.8-trillion-parameter Mixture of Experts model with open weights, using 896 experts (16 active per token) and a 1-million-token context window. It is the first open-weight system to approach the 3-trillion-parameter class, offering frontier-level reasoning, coding, and agentic capabilities that organizations can now self-host. AWS published a deployment guide covering two paths: (1) Amazon SageMaker HyperPod with its Inference Operator for a streamlined, managed setup, and (2) a self-managed Amazon EKS cluster using EC2 Capacity Blocks. Both require p6-b300 instances (8× NVIDIA B300 GPUs) and a dedicated vLLM serving container with tensor parallelism=8, MXFP4 weight loading, and MoE optimizations. The endpoint exposes an OpenAI-compatible chat completions API. The combination enables users to run one of the largest publicly available models on AWS infrastructure.