[{"data":1,"prerenderedAt":269},["ShallowReactive",2],{"blog-en-kubernetes-ai-inference-reversal-conformance-design":3,"blog-related-en-kubernetes-ai-inference-reversal-conformance-design":220,"blog-en-kubernetes-ai-inference-reversal-conformance-design-alt":209},{"id":4,"title":5,"author":6,"body":7,"date":203,"description":204,"extension":205,"image":206,"locale":207,"meta":208,"navigation":209,"path":210,"seo":211,"stem":212,"tags":213,"__hash__":219},"blog\u002Fblog\u002Fen\u002Fkubernetes-ai-inference-reversal-conformance-design.md","Inference Has Overtaken Training: What KubeCon Japan Revealed About Kubernetes Cluster Design in the AI Era","Kubo Team",{"type":8,"value":9,"toc":194},"minimark",[10,15,19,26,37,50,59,62,66,69,75,84,93,96,100,108,114,123,138,141,145,148,154,161,174,178,181],[11,12,14],"h2",{"id":13},"ai-compute-demand-was-built-around-training-thats-quietly-reversing","AI Compute Demand Was Built Around Training. That's Quietly Reversing.",[16,17,18],"p",{},"Chances are you've noticed the phrase \"Kubernetes AI inference\" showing up more often over the past year. At the keynote of KubeCon + CloudNativeCon Japan 2026, CNCF leadership spoke about a shift in how AI computing capacity is being allocated. Until a few years ago, the vast majority of AI-related compute went toward model \"training.\" Today, the center of gravity is moving to \"inference.\"",[16,20,21],{},[22,23],"img",{"alt":24,"src":25},"section01","https:\u002F\u002Fcdn.kubo.hexabase.io\u002Fimages\u002Fblog\u002Fkubernetes-ai-inference-reversal-conformance-design\u002Fsection01.webp",[16,27,28,29,36],{},"According to ",[30,31,35],"a",{"href":32,"rel":33},"https:\u002F\u002Fbuild.inc\u002Finsights\u002Ftraining-vs-inference-data-center-design-differences",[34],"nofollow","an analysis of data center demand",", AI inference compute capacity is projected to exceed 90 gigawatts by 2030, growing at a 35% CAGR. McKinsey's own estimates are even more specific: data center demand for AI training is expected to grow from 23.1 gigawatts in 2025 to 62.2 gigawatts by 2030 (22% CAGR), while demand for inference is expected to surge from 20.9 gigawatts to 93.3 gigawatts over the same period (35% CAGR). By 2030, inference is projected to account for more than half of all AI compute, overtaking training as the single largest workload category.",[16,38,39,40,45,46],{},"Other research points in the same direction. ",[30,41,44],{"href":42,"rel":43},"https:\u002F\u002Favidsolutionsinc.com\u002F13-data-center-growth-projections-that-will-shape-2026-2030\u002F",[34],"An article on data center growth trends"," cites Deloitte's estimate that inference already accounted for half of all AI compute in 2025 and will reach two-thirds by 2026, while Brookfield projects inference will make up 75% of AI compute demand by 2030. The exact numbers vary, but the direction is consistent: ",[47,48,49],"strong",{},"infrastructure designed around training no longer matches reality.",[16,51,52,53,58],{},"Given the scale of this shift, cost efficiency and freedom from vendor lock-in are becoming critical decision criteria for any organization choosing or re-evaluating its infrastructure. It's part of why managed K3s-based Kubernetes platforms like ",[30,54,57],{"href":55,"rel":56},"https:\u002F\u002Fkubo.hexabase.io\u002F",[34],"Kubo"," are increasingly being considered.",[16,60,61],{},"What makes this shift tricky is that training and inference demand fundamentally different things from infrastructure. The next section looks at how that difference is reshaping Kubernetes operations.",[11,63,65],{"id":64},"why-inference-workloads-are-forcing-kubernetes-to-rebuild-itself","Why Inference Workloads Are Forcing Kubernetes to Rebuild Itself",[16,67,68],{},"Training jobs are essentially batch processes: they consume a large pool of GPUs for hours or days at a stretch, then release them. Inference workloads, on the other hand, run continuously, scale up and down within seconds or minutes based on request volume, and translate any latency degradation directly into a worse user experience. Scheduling and resource-management approaches optimized for training tend to waste resources when applied directly to inference.",[16,70,71],{},[22,72],{"alt":73,"src":74},"section02","https:\u002F\u002Fcdn.kubo.hexabase.io\u002Fimages\u002Fblog\u002Fkubernetes-ai-inference-reversal-conformance-design\u002Fsection02.webp",[16,76,77,78,83],{},"Kubernetes itself is adding inference-oriented capabilities to address this. A prime example is \"In-Place Pod Resize.\" According to the ",[30,79,82],{"href":80,"rel":81},"https:\u002F\u002Fkubernetes.io\u002Fblog\u002F2025\u002F05\u002F16\u002Fkubernetes-v1-33-in-place-pod-resize-beta\u002F",[34],"official Kubernetes blog",", this feature lets you change a container's CPU and memory requests\u002Flimits without restarting the Pod, and it graduated to beta with default-enabled status in v1.33. Inference serving often needs a burst of CPU right after startup and then settles into a lower steady state — being able to adjust resources without a restart has a direct impact on availability.",[16,85,86,87,92],{},"The ",[30,88,91],{"href":89,"rel":90},"https:\u002F\u002Fopensource.googleblog.com\u002F2026\u002F04\u002Fkubernetes-goes-ai-first-unpacking-the-new-ai-conformance-program.html",[34],"Google Cloud blog"," further highlights four technical building blocks for supporting both inference and training: \"Dynamic Resource Allocation\" (DRA) for fine-grained control over GPUs\u002FTPUs, \"All-or-Nothing Scheduling\" so distributed training jobs wait until all required resources are available, custom-metrics-based autoscaling driven by signals like GPU utilization, and standardized observability for accelerators. Together, these show Kubernetes expanding its role from \"container orchestrator\" to \"AI workload platform.\"",[16,94,95],{},"If every cloud and distribution implements these extensions independently, the result is a patchwork of environments where things only \"just work\" in specific places. That's exactly the fragmentation problem CNCF's standardization effort is trying to solve.",[11,97,99],{"id":98},"what-is-the-cncfs-ai-conformance-program-actually-guaranteeing","What Is the CNCF's \"AI Conformance Program\" Actually Guaranteeing?",[16,101,28,102,107],{},[30,103,106],{"href":104,"rel":105},"https:\u002F\u002Fwww.cncf.io\u002Fannouncements\u002F2025\u002F11\u002F11\u002Fcncf-launches-certified-kubernetes-ai-conformance-program-to-standardize-ai-workloads-on-kubernetes\u002F",[34],"CNCF's official announcement",", the Certified Kubernetes AI Conformance Program was first announced in beta at KubeCon Japan in June 2025 and officially launched at KubeCon North America in Atlanta on November 11 of the same year. The goal is straightforward: ensure interoperability and portability for AI workloads and reduce fragmentation across vendors. Major cloud providers — AWS, Google Cloud, Microsoft Azure, Oracle, Red Hat, and others — have already joined as certified platforms.",[16,109,110],{},[22,111],{"alt":112,"src":113},"section03","https:\u002F\u002Fcdn.kubo.hexabase.io\u002Fimages\u002Fblog\u002Fkubernetes-ai-inference-reversal-conformance-design\u002Fsection03.webp",[16,115,116,117,122],{},"A ",[30,118,121],{"href":119,"rel":120},"https:\u002F\u002Fwww.forbes.com\u002Fsites\u002Fjanakirammsv\u002F2025\u002F11\u002F18\u002Fcncf-establishes-standards-for-running-ai-workloads-on-kubernetes\u002F",[34],"Forbes explainer"," notes that the program emerged in response to research showing that 82% of organizations are building custom AI solutions, with 58% of them relying on Kubernetes — and that incompatibility between platforms was raising the risk of vendor lock-in. With certification in place, an AI application validated on one Kubernetes environment can be expected to behave the same way on any other certified environment.",[16,124,125,126,131,132,137],{},"The program's requirements are also becoming stricter over time. According to ",[30,127,130],{"href":128,"rel":129},"https:\u002F\u002Fcloudnativenow.com\u002Ffeatures\u002Fcncf-expands-efforts-to-run-ai-inference-workloads-on-kubernetes-clusters\u002F",[34],"reporting from Cloud Native Now",", the 2026 KubeCon Europe event introduced the \"Kubernetes AI Requirements (KAR) v1.35\" certification standard, adding stable support for In-Place Pod Resize and Workload-Aware Scheduling as mandatory requirements. At the same time, new platforms — including OVHcloud, SpectroCloud, JD Cloud, and China Unicom Cloud — achieved certification, bringing the total number of certified platforms to 31. CNCF's Jonathan Bryce has said that \"AI inference workloads will, in the near future, come to overwhelm every other class of workload running on Kubernetes clusters,\" and the detailed requirements are being developed in the open on ",[30,133,136],{"href":134,"rel":135},"https:\u002F\u002Fgithub.com\u002Fcncf\u002Fk8s-ai-conformance",[34],"GitHub",".",[16,139,140],{},"In short, what CNCF is aiming for is a state where \"even as inference workloads surge, you get consistent behavior regardless of environment.\" That same principle turns out to be a useful lens for choosing a managed Kubernetes platform.",[11,142,144],{"id":143},"what-should-managed-k3s-environments-do-to-meet-inference-era-infrastructure-requirements","What Should Managed K3s Environments Do to Meet Inference-Era Infrastructure Requirements?",[16,146,147],{},"Pulling together everything covered so far, the requirements for inference-era infrastructure boil down to three things. First, autoscaling based on AI-specific signals like GPU utilization and inference latency. Second, observability that surfaces accelerator metrics in a standardized way. Third, portability that isn't tied to a specific cloud or distribution — in other words, avoiding vendor lock-in.",[16,149,150],{},[22,151],{"alt":152,"src":153},"section04","https:\u002F\u002Fcdn.kubo.hexabase.io\u002Fimages\u002Fblog\u002Fkubernetes-ai-inference-reversal-conformance-design\u002Fsection04.webp",[16,155,156,157,160],{},"These three conditions are exactly where using a fully managed service like EKS or AKS on its own tends to force a trade-off between cost and lock-in. ",[30,158,57],{"href":55,"rel":159},[34],", a K3s-based managed Kubernetes platform, is built on standards-compliant Pure Kubernetes, which aligns naturally with the environment-level consistency CNCF's certification is aiming for. Its built-in Prometheus + Grafana stack also gives you a ready-made foundation for visualizing accelerator metrics.",[16,162,163,164,167,168,173],{},"Much of today's inference workload runs as part of AI agents or MLOps pipelines. For teams thinking ahead to operating these autonomous workloads, combining ",[30,165,57],{"href":55,"rel":166},[34]," with an AI agent execution platform like ",[30,169,172],{"href":170,"rel":171},"https:\u002F\u002Fwww.hexabase.com\u002Fproduct\u002Fcaptain-ai\u002F",[34],"Captain.AI"," is one option worth considering. Built-in GitOps and Helm chart support also means you don't need to rebuild your inference deployment pipeline from scratch.",[11,175,177],{"id":176},"summary","Summary",[16,179,180],{},"AI compute demand is flipping from \"training-centric\" to \"inference-centric,\" and multiple research organizations now project that inference compute capacity will substantially outpace training by 2030. Kubernetes is responding by adopting inference-specific capabilities like In-Place Pod Resize and Dynamic Resource Allocation, while CNCF's AI Conformance Program pushes toward a standard where workloads \"behave the same regardless of environment.\"",[16,182,183,184,187,188,193],{},"Given this trajectory, cluster design going forward needs to prioritize three things: autoscaling based on inference-specific metrics, observability, and portability that doesn't lock you into a single vendor. For organizations without the resources to build robust infrastructure from the ground up, ",[30,185,57],{"href":55,"rel":186},[34]," — a standards-compliant, cost-efficient, K3s-based platform — is a realistic option for supporting inference-era workloads. Start by ",[30,189,192],{"href":190,"rel":191},"https:\u002F\u002Fwww.hexabase.com\u002Fcontact-us\u002F",[34],"getting in touch"," to map out where your inference infrastructure stands today.",{"title":195,"searchDepth":196,"depth":196,"links":197},"",2,[198,199,200,201,202],{"id":13,"depth":196,"text":14},{"id":64,"depth":196,"text":65},{"id":98,"depth":196,"text":99},{"id":143,"depth":196,"text":144},{"id":176,"depth":196,"text":177},"2026-08-01","AI compute demand has flipped from training to inference, with inference compute projected to reach 1.5x training capacity by 2030. Drawing on KubeCon Japan discussions and the CNCF AI Conformance Program, this article outlines what Kubernetes\u002FK3s clusters need to look like in the inference era.","md","https:\u002F\u002Fcdn.kubo.hexabase.io\u002Fimages\u002Fblog\u002Fkubernetes-ai-inference-reversal-conformance-design\u002Feyecatch.webp","en",{},true,"\u002Fblog\u002Fen\u002Fkubernetes-ai-inference-reversal-conformance-design",{"title":5,"description":204},"blog\u002Fen\u002Fkubernetes-ai-inference-reversal-conformance-design",[214,215,216,217,218],"kubernetes","k3s","ai-inference","cncf","managed-kubernetes","tZ5CR5UTL375EEleusMI-aUDIbYt92BEtR9YU8Ec6p0",[221,229,238,245,253,261],{"path":222,"title":223,"description":224,"date":225,"tags":226},"\u002Fblog\u002Fen\u002Fcncf-graduated-project-oss-selection-criteria","Should You Trust the CNCF 'Graduated' Badge? What TOC Public Meetings Reveal About the Unglamorous Reality of the Review Process","The CNCF 'Graduated' badge is not a safety certificate. Drawing on TOC public meeting notes and official documentation, this article explains the real review process behind Sandbox, Incubating, and Graduated status, and how to judge OSS maturity before adopting it in production.","2026-07-23",[217,214,215,227,218,228],"oss","governance",{"path":230,"title":231,"description":232,"date":233,"tags":234},"\u002Fblog\u002Fen\u002Faks-cost-optimization-fixed-cost-structure-k3s-comparison","You Optimized AKS Costs to the Letter. So Why Won't the Bill Move?","Right-sizing nodes and adopting Spot VMs on AKS often plateaus because LoadBalancer-per-service billing and monitoring add-ons are structural fixed costs that operational tuning can't touch. We break down the real bottleneck and compare it to K3s's lightweight cost structure.","2026-09-07",[215,214,235,236,237,218],"aks","azure","cost-optimization",{"path":239,"title":240,"description":241,"date":242,"tags":243},"\u002Fblog\u002Fen\u002Fkubernetes-cost-management-eks-aks-billing-visibility","Your EKS Bill Only Makes Sense at Month-End: Why Kubernetes Costs Are Structurally 'Discovered Too Late'","Why do Kubernetes costs on EKS\u002FAKS balloon unexpectedly? We break down how autoscaling and cross-AZ billing hide costs, and explore how K3s-based managed infrastructure turns them into a fixed cost.","2026-08-22",[215,214,237,218,235,244],"finops",{"path":246,"title":247,"description":248,"date":249,"tags":250},"\u002Fblog\u002Fen\u002Fkubernetes-operations-specialization-managed-k3s-hiring","eBPF, cert-manager, Service Mesh: Why Kubernetes Operations Outgrew What One Engineer Can Handle","Why do Kubernetes operations roles stay unfilled for months? It isn't a lack of tool knowledge — it's that the discipline has splintered into too many specialties. Instead of hiring more heads, absorb the specialization into the platform with managed K3s.","2026-08-21",[215,214,218,251,252],"devops","platform-engineering",{"path":254,"title":255,"description":256,"date":257,"tags":258},"\u002Fblog\u002Fen\u002Fcncf-project-maturity-graduation-criteria-production","10,000 GitHub Stars Isn't a Diploma. Why the Real Criterion for Choosing a CNCF Project Isn't Commit Count but Maturity Stage","When adopting CNCF projects for your Kubernetes stack, are you judging them by GitHub stars and name recognition alone? Learn the Sandbox\u002FIncubating\u002FGraduated maturity framework and the Harbor case study to know what to check before production.","2026-08-19",[215,214,217,259,260],"oss-governance","container-registry",{"path":262,"title":263,"description":264,"date":265,"tags":266},"\u002Fblog\u002Fen\u002Fkubernetes-ebpf-inspektor-gadget-observability","No strace. No Sidecars. Still Debuggable: Inspektor Gadget's Answer for Seeing Inside Kubernetes with eBPF","When you can't add a privileged container or inject a sidecar into a production Kubernetes cluster, how do you diagnose Pod traffic and syscalls? We explain how the eBPF tool Inspektor Gadget works, and what happened to it in 2026.","2026-08-15",[215,214,267,268,217],"ebpf","observability",1792488906064]