[{"data":1,"prerenderedAt":578},["ShallowReactive",2],{"blog-en-k3s-baremetal-loadbalancer-bgp-metallb-cilium":3,"blog-related-en-k3s-baremetal-loadbalancer-bgp-metallb-cilium":525,"blog-en-k3s-baremetal-loadbalancer-bgp-metallb-cilium-alt":514},{"id":4,"title":5,"author":6,"body":7,"date":508,"description":509,"extension":510,"image":511,"locale":512,"meta":513,"navigation":514,"path":515,"seo":516,"stem":517,"tags":518,"__hash__":524},"blog\u002Fblog\u002Fen\u002Fk3s-baremetal-loadbalancer-bgp-metallb-cilium.md","Bare-Metal K3s LoadBalancer: When ARP Hits Its Limit, Design for BGP","Kubo Team",{"type":8,"value":9,"toc":497},"minimark",[10,24,31,36,50,63,80,86,95,99,108,111,127,136,142,146,155,168,187,384,391,400,406,415,419,422,427,442,446,449,453,460,466,470,473,476,493],[11,12,13,14,18,19,23],"p",{},"Factory floors, data center racks, the back room of a retail store — as edge Kubernetes deployments spread into these environments, one problem always surfaces sooner or later: ",[15,16,17],"strong",{},"designing a K3s load balancer around BGP",", a concern you never had to think about in the cloud. On EKS or AKS, writing ",[20,21,22],"code",{},"type: LoadBalancer"," is enough to get an external IP automatically. But on bare metal, where no cloud API exists, you have to build that mechanism yourself.",[11,25,26],{},[27,28],"img",{"alt":29,"src":30},"Architecture comparison: three patterns for how traffic enters a cluster — cloud LB, L2 mode, and BGP mode","https:\u002F\u002Fcdn.kubo.hexabase.io\u002Fimages\u002Fblog\u002Fk3s-baremetal-loadbalancer-bgp-metallb-cilium\u002Fsection01.webp",[32,33,35],"h2",{"id":34},"how-does-service-type-loadbalancer-work-without-a-cloud-lb","How Does \"Service type: LoadBalancer\" Work Without a Cloud LB?",[11,37,38,39,46,47,49],{},"According to the ",[40,41,45],"a",{"href":42,"rel":43},"https:\u002F\u002Fkubernetes.io\u002Fdocs\u002Fconcepts\u002Fservices-networking\u002Fservice\u002F#loadbalancer",[44],"nofollow","Kubernetes documentation",", a ",[20,48,22],{}," Service normally goes through the Cloud Controller Manager, which calls the cloud provider's load balancer API to automatically assign an external IP. This mechanism is tightly coupled to public clouds like AWS, Azure, and GCP.",[11,51,52,53,56,57,62],{},"In bare-metal environments — factories, branch sites, edge locations — ",[15,54,55],{},"there is no cloud API",", so this automatic assignment simply doesn't happen. This is where software load balancers like MetalLB come in: they use standard routing protocols to \"announce\" an external IP onto the network. The ",[40,58,61],{"href":59,"rel":60},"https:\u002F\u002Fmetallb.universe.tf\u002Fconcepts\u002F",[44],"MetalLB documentation"," describes two operating modes: Layer2 (ARP\u002FNDP) and BGP.",[11,64,65,66,69,70,75,76,79],{},"If you're running K3s, there's one more thing to watch for. K3s ships with a built-in load balancer called ",[20,67,68],{},"ServiceLB"," (formerly klipper-lb), enabled by default. If you want to install MetalLB or another implementation instead, the ",[40,71,74],{"href":72,"rel":73},"https:\u002F\u002Fdocs.k3s.io\u002Fnetworking\u002Fnetworking-services",[44],"K3s documentation"," requires you to pass ",[20,77,78],{},"--disable servicelb"," on every server node — otherwise the two implementations will fight over which one owns the external IP assignment.",[11,81,82],{},[27,83],{"alt":84,"src":85},"Flowchart: three steps to disable ServiceLB before deploying MetalLB in a K3s cluster","https:\u002F\u002Fcdn.kubo.hexabase.io\u002Fimages\u002Fblog\u002Fk3s-baremetal-loadbalancer-bgp-metallb-cilium\u002Fsection02.webp",[11,87,88,89,94],{},"If you'd rather not own this design decision from scratch, a K3s-based managed service like ",[40,90,93],{"href":91,"rel":92},"https:\u002F\u002Fkubo.hexabase.io\u002F",[44],"Kubo"," is another option — it removes the need to build the network design yourself.",[32,96,98],{"id":97},"where-layer2-mode-hits-a-wall-a-single-node-arp-caches-and-bandwidth","Where Layer2 Mode Hits a Wall: A Single Node, ARP Caches, and Bandwidth",[11,100,101,102,107],{},"MetalLB's Layer2 mode, according to its ",[40,103,106],{"href":104,"rel":105},"https:\u002F\u002Fmetallb.universe.tf\u002Fconcepts\u002Flayer2\u002F",[44],"official documentation",", works through leader election: \"one node takes responsibility for announcing a service IP to the local network.\" ARP (IPv4) or NDP (IPv6) responses then make it look as if a single machine holds multiple external IP addresses.",[11,109,110],{},"That's fine in a test environment. But in production on-prem or edge deployments, two structural limits appear:",[112,113,114,121],"ul",{},[115,116,117,120],"li",{},[15,118,119],{},"Single-node bottleneck",": All traffic to a given external IP funnels into whichever node was elected leader. No matter how many nodes you add to the cluster, that one service's inbound bandwidth can never exceed the physical bandwidth of a single node.",[115,122,123,126],{},[15,124,125],{},"Failover delay",": When the leader node goes down, a new node has to resend ARP responses to tell clients the target has switched. In most environments this converges within seconds, but clients holding stale ARP cache entries can experience delays measured in minutes.",[11,128,129,130,135],{},"The ",[40,131,134],{"href":132,"rel":133},"https:\u002F\u002Fwww.redhat.com\u002Fen\u002Fblog\u002Fmetallb-in-bgp-mode",[44],"Red Hat engineering blog"," makes the same point explicitly: Layer2 mode turns a node into \"a single point of failure that limits performance.\" As a K3s cluster grows from three nodes to ten in an edge rollout, this constraint becomes a hard ceiling on that service's scalability.",[11,137,138],{},[27,139],{"alt":140,"src":141},"Architecture diagram: in L2 mode, traffic to one external IP concentrates on a single leader node, which caps overall service bandwidth","https:\u002F\u002Fcdn.kubo.hexabase.io\u002Fimages\u002Fblog\u002Fk3s-baremetal-loadbalancer-bgp-metallb-cilium\u002Fsection03.webp",[32,143,145],{"id":144},"moving-to-bgp-metallbs-bgp-mode-vs-ciliums-bgp-control-plane","Moving to BGP: MetalLB's BGP Mode vs. Cilium's BGP Control Plane",[11,147,148,149,154],{},"The standard way past Layer2's limits is BGP mode. According to ",[40,150,153],{"href":151,"rel":152},"https:\u002F\u002Fmetallb.universe.tf\u002Fconcepts\u002Fbgp\u002F",[44],"MetalLB's BGP concept guide",", in BGP mode every node in the cluster establishes a BGP peering session with the upstream router, and the same external IP is advertised simultaneously from multiple nodes. If the router supports ECMP (Equal-Cost Multi-Path), traffic is actually load-balanced across those nodes based on packet hashing.",[11,156,129,157,161,162,167],{},[40,158,160],{"href":132,"rel":159},[44],"Red Hat blog"," sums up BGP mode's advantages as stateless load balancing, simultaneous advertisement from multiple nodes, and fast failure detection via the BFD protocol. MetalLB supports delegating the actual BGP session handling to an FRR-based backend called ",[40,163,166],{"href":164,"rel":165},"https:\u002F\u002Fwww.redhat.com\u002Fen\u002Fblog\u002Ffrr-k8s-bgp-backend-metallb",[44],"FRR-K8s",", which unlocks capabilities Layer2 mode doesn't have, such as combining BFD sessions with IPv6 advertisement.",[11,169,170,171,174,175,180,181,186],{},"There's a second option if your CNI is Cilium: ",[15,172,173],{},"Cilium's BGP Control Plane",". According to the ",[40,176,179],{"href":177,"rel":178},"https:\u002F\u002Fdocs.cilium.io\u002Fen\u002Fstable\u002Fnetwork\u002Fbgp-control-plane\u002Fbgp-control-plane\u002F",[44],"Cilium documentation",", the BGP Control Plane advertises Pod network and Service external IPs over BGP, while ",[40,182,185],{"href":183,"rel":184},"https:\u002F\u002Fdocs.cilium.io\u002Fen\u002Fstable\u002Fnetwork\u002Flb-ipam\u002F",[44],"LB-IPAM"," handles IP address allocation. If your cluster already runs Cilium as its CNI, you can consolidate BGP load balancing entirely into the CNI layer without adding MetalLB as a separate component.",[188,189,194],"pre",{"className":190,"code":191,"language":192,"meta":193,"style":193},"language-yaml shiki shiki-themes tokyo-night","# Example: simplified Cilium BGP Control Plane configuration\napiVersion: cilium.io\u002Fv2alpha1\nkind: CiliumBGPPeeringPolicy\nmetadata:\n  name: rack01-bgp-policy\nspec:\n  nodeSelector:\n    matchLabels:\n      rack: \"01\"\n  virtualRouters:\n    - localASN: 65001\n      neighbors:\n        - peerAddress: \"10.0.0.1\u002F32\"\n          peerASN: 65000\n      serviceSelector:\n        matchLabels:\n          advertise: \"bgp\"\n","yaml","",[20,195,196,205,220,231,240,251,259,267,275,292,300,316,324,342,353,361,369],{"__ignoreMap":193},[197,198,201],"span",{"class":199,"line":200},"line",1,[197,202,204],{"class":203},"sbD-w","# Example: simplified Cilium BGP Control Plane configuration\n",[197,206,208,212,216],{"class":199,"line":207},2,[197,209,211],{"class":210},"s0U2E","apiVersion",[197,213,215],{"class":214},"sAklC",":",[197,217,219],{"class":218},"sPY7s"," cilium.io\u002Fv2alpha1\n",[197,221,223,226,228],{"class":199,"line":222},3,[197,224,225],{"class":210},"kind",[197,227,215],{"class":214},[197,229,230],{"class":218}," CiliumBGPPeeringPolicy\n",[197,232,234,237],{"class":199,"line":233},4,[197,235,236],{"class":210},"metadata",[197,238,239],{"class":214},":\n",[197,241,243,246,248],{"class":199,"line":242},5,[197,244,245],{"class":210},"  name",[197,247,215],{"class":214},[197,249,250],{"class":218}," rack01-bgp-policy\n",[197,252,254,257],{"class":199,"line":253},6,[197,255,256],{"class":210},"spec",[197,258,239],{"class":214},[197,260,262,265],{"class":199,"line":261},7,[197,263,264],{"class":210},"  nodeSelector",[197,266,239],{"class":214},[197,268,270,273],{"class":199,"line":269},8,[197,271,272],{"class":210},"    matchLabels",[197,274,239],{"class":214},[197,276,278,281,283,286,289],{"class":199,"line":277},9,[197,279,280],{"class":210},"      rack",[197,282,215],{"class":214},[197,284,285],{"class":214}," \"",[197,287,288],{"class":218},"01",[197,290,291],{"class":214},"\"\n",[197,293,295,298],{"class":199,"line":294},10,[197,296,297],{"class":210},"  virtualRouters",[197,299,239],{"class":214},[197,301,303,307,310,312],{"class":199,"line":302},11,[197,304,306],{"class":305},"sgJMe","    -",[197,308,309],{"class":210}," localASN",[197,311,215],{"class":214},[197,313,315],{"class":314},"sOJ5S"," 65001\n",[197,317,319,322],{"class":199,"line":318},12,[197,320,321],{"class":210},"      neighbors",[197,323,239],{"class":214},[197,325,327,330,333,335,337,340],{"class":199,"line":326},13,[197,328,329],{"class":305},"        -",[197,331,332],{"class":210}," peerAddress",[197,334,215],{"class":214},[197,336,285],{"class":214},[197,338,339],{"class":218},"10.0.0.1\u002F32",[197,341,291],{"class":214},[197,343,345,348,350],{"class":199,"line":344},14,[197,346,347],{"class":210},"          peerASN",[197,349,215],{"class":214},[197,351,352],{"class":314}," 65000\n",[197,354,356,359],{"class":199,"line":355},15,[197,357,358],{"class":210},"      serviceSelector",[197,360,239],{"class":214},[197,362,364,367],{"class":199,"line":363},16,[197,365,366],{"class":210},"        matchLabels",[197,368,239],{"class":214},[197,370,372,375,377,379,382],{"class":199,"line":371},17,[197,373,374],{"class":210},"          advertise",[197,376,215],{"class":214},[197,378,285],{"class":214},[197,380,381],{"class":218},"bgp",[197,383,291],{"class":214},[11,385,386,387,390],{},"Note that MetalLB itself used to offer an integrated mode called ",[20,388,389],{},"MetalLB BGP ControlPlane"," that paired with Cilium's data plane, but that mode is now in a maintenance-only phase receiving security fixes, with migration toward Cilium's newer BGP Control Plane recommended instead. Which path makes sense depends heavily on your existing CNI choice — that's the real starting point for a migration decision.",[11,392,393,394,399],{},"If you want BGP handled entirely by dedicated switches, ",[40,395,398],{"href":396,"rel":397},"https:\u002F\u002Fwww.infoq.com\u002Fjp\u002Fnews\u002F2020\u002F08\u002Fporter-kubernetes-bare-metal\u002F",[44],"alternative OSS implementations like Porter"," exist too, but in terms of track record and documentation depth within K3s environments, MetalLB and Cilium remain the two realistic starting points.",[11,401,402],{},[27,403],{"alt":404,"src":405},"Comparison diagram: MetalLB BGP mode vs. Cilium BGP Control Plane, showing ECMP-distributed traffic","https:\u002F\u002Fcdn.kubo.hexabase.io\u002Fimages\u002Fblog\u002Fk3s-baremetal-loadbalancer-bgp-metallb-cilium\u002Fsection04.webp",[11,407,408,409,414],{},"For teams that would rather not own this migration end-to-end, ",[40,410,413],{"href":411,"rel":412},"https:\u002F\u002Fwww.hexabase.com\u002Fproduct\u002Fkubo\u002Fon-premise",[44],"Kubo On-Premise"," is an option. It's K3s-based, supports air-gapped environments, and lets an operations team hand off network design — including BGP peering — without vendor lock-in. The tighter your external network connectivity requirements (think financial institutions or manufacturing edge sites), the bigger the payoff.",[32,416,418],{"id":417},"three-things-to-check-before-you-migrate","Three Things to Check Before You Migrate",[11,420,421],{},"Moving from Layer2 mode to BGP mode isn't something you can finish purely by configuring the Kubernetes cluster. In practice, there are three things you always need to confirm.",[423,424,426],"h3",{"id":425},"_1-have-you-disabled-k3ss-built-in-servicelb-klipper","1. Have you disabled K3s's built-in ServiceLB (klipper)?",[11,428,429,430,432,433,438,439,441],{},"As mentioned above, K3s enables ",[20,431,68],{}," by default. Per ",[40,434,437],{"href":435,"rel":436},"https:\u002F\u002Fmetallb.universe.tf\u002Fconfiguration\u002Fk3s\u002F",[44],"MetalLB's K3s configuration guide",", installing MetalLB without disabling it first causes the two implementations to compete over external IP assignment, leading to unpredictable load balancer behavior. Set ",[20,440,78],{}," on every server node first, then install MetalLB or enable Cilium's BGP features — the order matters.",[423,443,445],{"id":444},"_2-do-your-upstream-routersswitches-support-bgp-and-have-you-aligned-with-the-network-team","2. Do your upstream routers\u002Fswitches support BGP, and have you aligned with the network team?",[11,447,448],{},"BGP configuration doesn't stop at the cluster boundary. You also need BGP peer settings (ASN, peer IP addresses, route filters) on the upstream router or switch side, and in most organizations that falls under the network team's authority. This alignment process — unnecessary under Layer2 mode — tends to be the biggest practical hurdle in a BGP migration.",[423,450,452],{"id":451},"_3-have-you-verified-failover-behavior","3. Have you verified failover behavior?",[11,454,455,456,459],{},"Whether ECMP routes get withdrawn correctly on node failure, and whether BFD-based fast detection actually works, are things you must verify before going to production. As the ",[40,457,160],{"href":132,"rel":458},[44]," notes, combining BFD can shorten failure detection to well under a second — but that depends entirely on router-side support, which is why pre-verification is non-negotiable.",[11,461,462],{},[27,463],{"alt":464,"src":465},"Process diagram: disable ServiceLB, align with the network team, then verify failover — three steps","https:\u002F\u002Fcdn.kubo.hexabase.io\u002Fimages\u002Fblog\u002Fk3s-baremetal-loadbalancer-bgp-metallb-cilium\u002Fsection05.webp",[32,467,469],{"id":468},"summary","Summary",[11,471,472],{},"For bare-metal and edge K3s clusters that can't rely on a cloud load balancer, Layer2 (ARP) mode load balancing is really only a compromise fit for test environments. It inevitably runs into structural limits — traffic concentrating on a single node, delayed failover — which is why migrating to BGP mode becomes practically inevitable for production use.",[11,474,475],{},"Which path to take depends on your CNI setup. If you're already running Cilium, consolidating into its BGP Control Plane and LB-IPAM is the sensible choice; for any other CNI, MetalLB's BGP mode is the realistic option. Either way, the deciding factor in a successful migration is doing the unglamorous groundwork: disabling K3s's ServiceLB and reaching agreement with the network team.",[11,477,478,479,482,483,486,487,492],{},"As Kubernetes spreads beyond the cloud into factories, retail stores, and even vehicles, this kind of network design decision becomes unavoidable. K3s-based managed services like ",[40,480,93],{"href":91,"rel":481},[44]," and ",[40,484,413],{"href":411,"rel":485},[44]," can take on this design and operational burden for you. If you'd rather not shoulder the verification cost of a BGP migration yourselves, ",[40,488,491],{"href":489,"rel":490},"https:\u002F\u002Fwww.hexabase.com\u002Fcontact-us\u002F",[44],"reaching out"," is worth considering.",[494,495,496],"style",{},"html pre.shiki code .sbD-w, html code.shiki .sbD-w{--shiki-default:#51597D;--shiki-default-font-style:italic}html pre.shiki code .s0U2E, html code.shiki .s0U2E{--shiki-default:#F7768E}html pre.shiki code .sAklC, html code.shiki .sAklC{--shiki-default:#89DDFF}html pre.shiki code .sPY7s, html code.shiki .sPY7s{--shiki-default:#9ECE6A}html pre.shiki code .sgJMe, html code.shiki .sgJMe{--shiki-default:#9ABDF5}html pre.shiki code .sOJ5S, html code.shiki .sOJ5S{--shiki-default:#FF9E64}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}",{"title":193,"searchDepth":207,"depth":207,"links":498},[499,500,501,502,507],{"id":34,"depth":207,"text":35},{"id":97,"depth":207,"text":98},{"id":144,"depth":207,"text":145},{"id":417,"depth":207,"text":418,"children":503},[504,505,506],{"id":425,"depth":222,"text":426},{"id":444,"depth":222,"text":445},{"id":451,"depth":222,"text":452},{"id":468,"depth":207,"text":469},"2026-08-26","When you can't rely on a cloud load balancer for on-prem or edge K3s clusters, why does Layer2 (ARP) load balancing eventually hit a wall? A practical look at MetalLB's BGP mode vs. Cilium's BGP Control Plane, and how to decide when to migrate.","md","https:\u002F\u002Fcdn.kubo.hexabase.io\u002Fimages\u002Fblog\u002Fk3s-baremetal-loadbalancer-bgp-metallb-cilium\u002Feyecatch.webp","en",{},true,"\u002Fblog\u002Fen\u002Fk3s-baremetal-loadbalancer-bgp-metallb-cilium",{"title":5,"description":509},"blog\u002Fen\u002Fk3s-baremetal-loadbalancer-bgp-metallb-cilium",[519,520,521,381,522,523],"k3s","kubernetes","metallb","load-balancer","edge-computing","7Xw4LsLqCtyn2mEe3cchJTxjuIaEgI-dsZOiU9_ypUc",[526,534,542,550,558,569],{"path":527,"title":528,"description":529,"date":530,"tags":531},"\u002Fblog\u002Fen\u002Fedge-k3s-hardware-selection-beyond-appliance","Homelabbers Are Ditching ISP Routers for Self-Built PCs. Edge Kubernetes Should Rethink Its Appliance Habit Too","A growing number of individuals are replacing ISP-supplied routers with self-built mini PCs running open-source firmware. The same logic applies to edge Kubernetes: a hardware selection checklist and a GitOps operating model for standardizing dedicated appliances out of the fleet.","2026-08-29",[519,520,523,532,533],"hardware-selection","network-appliance",{"path":535,"title":536,"description":537,"date":538,"tags":539},"\u002Fblog\u002Fen\u002Fk3s-edge-fleet-declarative-management","One Device's Troubleshooting Is a Funny Story. A Thousand Devices Is a Business Risk: How Rancher Fleet Rescues K3s Edge Operations from Tribal Knowledge","Fleet management for K3s edge operations breaks down once you're troubleshooting devices one at a time by hand. Here's how declarative management and Rancher Fleet let you design edge operations that don't depend on any single person.","2026-08-03",[519,520,523,540,541],"gitops","fleet-management",{"path":543,"title":544,"description":545,"date":546,"tags":547},"\u002Fblog\u002Fen\u002Fhybrid-k3s-edge-metrics-network-overhead","2,000 IoT Devices Were Clogging the Network. The Day Push Metrics Bit Back in a Hybrid K3s Deployment","A field report from a large-scale K3s edge deployment covering 2,000+ devices: why lightweight Kubernetes gets chosen, and the hidden network cost of push-based metrics collection in a hybrid cluster architecture, backed by concrete numbers. For engineers and platform operators.","2026-07-31",[519,520,523,548,549],"hybrid-cluster","managed-kubernetes",{"path":551,"title":552,"description":553,"date":554,"tags":555},"\u002Fblog\u002Fen\u002Fedge-k3s-observability-homelab-dashboard","A Wall-Mounted Dashboard Taught Me What 'Peace of Mind' Really Means: Observability Design Lessons for Edge K3s Clusters","The trial-and-error troubleshooting behind a home-lab wall-mounted dashboard is the same trap that hits edge Kubernetes clusters scattered across factories and stores. Drawing on official K3s, Prometheus, and Grafana docs, this piece lays out design principles for not deferring observability.","2026-07-26",[519,520,523,556,557,549],"observability","monitoring",{"path":559,"title":560,"description":561,"date":562,"tags":563},"\u002Fblog\u002Fen\u002Fk3s-local-path-provisioner-capacity-node-affinity-pvc-pending","Your K3s Pod Won't Leave Pending. The 'Ignored Capacity' and Node Affinity Trap in Local Storage","A K3s PVC stuck in Pending often traces back to one fact: local-path-provisioner never enforces capacity limits. This post digs into hypervisor thin provisioning, Volume Node Affinity Conflicts, and the backup misconceptions hiding in K3s local storage design.","2026-09-12",[519,520,564,565,566,567,568],"storage","local-path-provisioner","persistent-volume","node-affinity","zfs",{"path":570,"title":571,"description":572,"date":573,"tags":574},"\u002Fblog\u002Fen\u002Fk3s-keda-vs-hpa-event-driven-autoscaling-decision","KEDA vs HPA on K3s: A Decision Framework for Event-Driven Autoscaling","When CPU\u002Fmemory-based HPA on K3s can't keep up with event-driven workloads, should you extend HPA or adopt KEDA, the CNCF-graduated project? Here's a 3-point decision framework, plus what to watch for on K3s.","2026-09-10",[519,520,575,576,577],"hpa","keda","autoscaling",1792488901619]