High-Performance Kubernetes Networking with Calico eBPF

Kubernetes has revolutionized cloud-native applications, but networking remains a crucial aspect of ensuring scalability, security, and performance. Default networking approaches, such as iptables-based packet filtering, often introduce performance bottlenecks due to inefficient packet processing and complex rule evaluations. This is where Calico eBPF comes into play, offering a powerful alternative that enhances networking efficiency and security at scale.

Understanding Kubernetes Networking

Kubernetes networking consists of two primary components:

Physical Network Infrastructure – Connects cloud resources to external networks, ensuring communication between nodes and the broader internet.
Cluster Network Infrastructure – Manages internal workload communication within the Kubernetes cluster, including service-to-service traffic and pod-to-pod interactions.

Choosing the right data plane is critical for optimal performance. Factors such as cluster size, throughput, and security requirements should guide this choice. Poor networking choices can lead to congestion, excessive latency, and resource starvation.

Data Plane Options in Kubernetes Networking

Networking in Kubernetes is an abstract idea. While Kubernetes lays the foundation, your Container Networking Interface (CNI) is in charge of the actual networking. To better understand networking, we usually divide it into two sections: a control plane and a data plane.

Control Plane – Control plane is the part that manages how traffic should flow, what routes it should take, and how it should behave in certain scenarios.
Data Plane – Forwards network packets as directed by the control plane. The data plane is responsible for the actual packet forwarding, ensuring that network traffic is delivered as efficiently as possible. Depending on which data plane you choose, Calico will use certain technology (eBPF, nftables, iptables, VPP, or HNS) to achieve this.

Calico-Supported Data Planes

Calico supports multiple data plane implementations, each suited for different use cases:

Standard (iptables-based) – Uses default Linux iptables for network policy enforcement and routing, which can become inefficient as rule complexity grows. There is also a native nftable data plane to support RHEL 9 after iptable support was dropped.
eBPF (Extended Berkeley Packet Filter) – Implements packet processing directly in the Linux kernel, eliminating iptables overhead and providing enhanced observability and performance.
Windows Host Networking – Provides networking support for Windows-based Kubernetes nodes.
VPP (Vector Packet Processing) – High-performance user-space packet processing.

What is eBPF?

eBPF (Extended Berkeley Packet Filter) is a revolutionary Linux kernel technology that enables efficient and flexible packet processing. By running sandboxed programs inside the kernel, eBPF eliminates the performance overhead associated with default iptables-based packet filtering.

Advantages of eBPF over Standard Iptables Data Plane

Traffic Control: eBPF enables direct packet manipulation, reducing routing complexity and improving efficiency.
Network Policies: In eBPF mode, related policies are injected into each interface as native code. Policy decisions become easier to benchmark and show, allowing for faster policy implementation compared to other data planes. eBPF network policies support XDP (eXpress Data Path) and other advanced security enforcement mechanisms.
Connect-Time Load Balancing: Load balancing happens at the source, avoiding unnecessary NAT overhead, improving connection speeds.
Performance Gains: Eliminates kernel bottlenecks by optimizing packet flow and allowing high-throughput data processing.

Keep in mind that in recent years, nftables has become streamlined in Linux, allowing the Linux networking stack to compete with eBPF in many of these areas.

Implementing Calico eBPF

Installation and Migration

Switching between data planes and selecting eBPF in Calico is simple and straightforward:

1. Install any flavor of Calico (Calico Open Source, Calico Enterprise or Calico Cloud).

2. Configure the data plane mode to eBPF, using just one command.

Note: This snippet is taken from one of our self-paced workshops, Turbocharging host workloads with eBPF and XDP. I recommend taking a look at our documentation for enabling the eBPF data plane for more information.

3. Validate the eBPF setup by inspecting calico-node logs.

kubectl logs --namespace calico-system ds/calico-node --container calico-node | egrep -i "BPF enabled"

If you are using the tigera-operator to install Calico, you can also run this command to check which data plane is in use:

kubectl get installation default -o=jsonpath="{.spec.calicoNetwork.linuxDataplane}"

Leveraging Calico eBPF for Network Policies

Calico eBPF offers enhanced network policy capabilities:

More flexible match rules and selectors, allowing fine-grained security enforcement.
Global and namespace-scoped policies to control access dynamically.
eBPF security policies are applied directly to the interfaces, which provides an efficient way to enforce control without the overhead of iptables.

Source IP Preservation

Unlike the standard Linux data plane, which uses kube-proxy and obscures the source IP address, Calico eBPF preserves the source IP of each connection that comes into your workloads. This is a huge advantage when you want to troubleshoot a problem, use geo-loadbalancing, or create access control for sensitive or confidential workloads.

Excellent for troubleshooting
Good for geo-based loadbalancing
Better access control for sensitive workloads and services

Direct Server Return (DSR)

When traffic enters your cluster through a node that doesn’t host the target workload, it gets forwarded to the correct node. Without DSR, it then has to return back to the original receiver node before returning to its source. DSR eliminates this extra step by allowing the intended node to send the response directly to the source, reducing latency and CPU overhead.

Direct Server Return:

Shortens the path that requests must travel inside the cluster
Offers faster response times, improving performance

XDP to prevent DOS attacks and optimize performance for high-demand workloads

XDP (eXpress Data Path) bypasses the Linux networking stack, and significantly improves packet-processing speed by intercepting a packet at its earliest point in the Network Interface Card (NIC) or the kernel. To put this into perspective: If you are using the standard Linux data plane with iptables, your first point of packet manipulation is the “raw” chain, which happens well after XDP.

The following illustration shows the position of XDP and raw interaction in Linux networking:

Source: Wikipedia

Enabling XDP with Calico provides:

Lower latency by intercepting packets at their earliest point of discovery/initiation.
Higher throughput for high-traffic environments, ensuring data flows efficiently between pods.
Bypassing the Linux conntrack table when doNotTrack policies are in use (requires explicit return policies). This uses eBPF programs and maps to deal with connections and it’s great for high-connection workloads such as in memory databases and streaming applications.

Conclusion

The Calico eBPF data plane brings significant performance, scalability, and security improvements to Kubernetes networking. By leveraging eBPF for packet processing, network policies, and XDP-based optimizations, organizations can achieve high-throughput, low-latency networking at scale. Unlike iptables, which uses the Linux networking stack to interact with packets, the eBPF data plane injects programs directly into certain parts of the kernel to intercept packets in an optimized manner, reducing CPU usage and improving network efficiency.

Adopting Calico eBPF is a game-changer for Kubernetes networking, offering a high-performance alternative to traditional approaches.

Ready to make the switch? Schedule a demo to explore the possibilities with Calico eBPF.

Thanks to Jen Luther Thomas for her help writing this article.