oneke_lb

Load Balancing

OneKE includes the following load balancing components:

• Cilium
• MetalLB
• OpenNebula Cloud Provider

Cilium

When ONEAPP_K8S_CNI_PLUGIN is set to "cilium" and we enable the cilium BGP control plane setting ONEAPP_K8S_CILIUM_BGP_ENABLED to YES or we undeclare it, specifying a ONEAPP_K8S_CILIUM_RANGE (for example 172.20.0.240/28), then OneKE should be capable of deploying LoadBalancer services (without MetalLB). For more information on using BGP load balancers in Cilium please refer to the official documentation.

Important

We must take into account that this feature could conflict with other Load Balancer Service Controllers, (e.g. MetalLB or ONE Cloud Provider) as it will assign the external-IP to the service even if the LoadBalancer Service doesn't have defined a .spec.loadBalancerClass parameter. This will be improved in cilium 1.17 where we will be allowed to define a special defaultLBServiceIPAM parameter in the configuration, so we expect to make it available in next OneKE versions.

MetalLB

---
apiVersion: v1
kind: Namespace
metadata:
  name: metallb-system
---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: one-metallb
  namespace: kube-system
spec:
  targetNamespace: metallb-system
  chartContent: <BASE64 OF A METALLB HELM CHART TGZ FILE>
  valuesContent: |
    controller:
      image:
        pullPolicy: IfNotPresent
    speaker:
      image:
        pullPolicy: IfNotPresent
    loadBalancerClass: <ONEAPP_K8S_METALLB_CLASS, defaults to 'metallb'>

• A dedicated namespace metallb-system is provided.
• Image Pull Policy is optimized for airgapped deployments.
• Precreated CRD configuration objects are provided (managed by RKE2 with Helm Integration installed in /var/lib/rancher/rke2/server/manifests/one-metallb-config.yaml). For information on MetalLB use cases please refer to the official documentation on MetalLB's configuration.

Important

To deploy MetalLB:

• Set ONEAPP_K8S_METALLB_ENABLED to "YES".

Important

For advertising routes to your Kubernetes Services through MetalLB using ARP, you should set the .spec.loadBalancerClass of your service to metallb (the LoadBalancer class name could be parameterized through ONEAPP_K8S_METALLB_CLASS). More info in the official documentation.

Warning

MetalLB is not suitable for use in AWS Edge Clusters, since AWS VPC is API-oriented and doesn't fully support networking protocols such as ARP or BGP in a standard way. For more information please refer to the MetalLB's Cloud Compatibility.

OpenNebula Cloud Provider (beta)

The provider has been initially developed with Cluster-API in mind and tested with Kubeadm-provisioned clusters, but it is perfectly possible to use it in RKE2 (as there is nothing in OneKE that would break that basic compatibility).

The installation procedure has been sketched here. Please note, that it's not fully automated via CONTEXT=[] attributes yet and it's excluded from the airgapped image. External intervention is required (for example manual).

Warning

If you'd like to use both Cloud-Provider and MetalLB load-balancers, then please make sure you set the ONEAPP_K8S_METALLB_CLASS context attribute, otherwise there is nothing that will prevent race condition between both solutions.

Warning

Please avoid using Cilium BGP load-balancers together with Cloud-Provider ones, as it may result in a race condition.