Cilium Service Mesh: Key Features, Examples, and Limitations

What Is Cilium Service Mesh?

Cilium Service Mesh, part of the open source Cilium project, is a networking, security, and observability framework for cloud-native application environments. It adds a management layer for service-to-service communication, ensuring those interactions are secure, fast, and reliable without requiring changes to individual applications.

Cilium Service Mesh uses eBPF (extended Berkeley Packet Filter) technology to provide high-performance, low-overhead kernel-level network functions. It supports fine-grained policy enforcement, security, and visibility for application traffic.

This is part of a series of articles about Cilium vs. Calico.

In this article:

Key Features of Cilium Service Mesh
Understanding How Cilium Service Mesh Works with Examples
Cilium Service Mesh Limitations

Key Features of Cilium Service Mesh

Kernel-Level Integration Using eBPF

Cilium Service Mesh stands out due to its kernel-level integration using eBPF, setting a new standard for efficiency in service mesh technology. By leveraging eBPF, Cilium directly interacts with the Linux kernel, bypassing the traditional overhead associated with proxy-based service meshes. This integration facilitates rapid and secure communication between services, optimizing performance and reducing latency.

The use of eBPF allows Cilium to offer superior networking capabilities by efficiently managing both networking and application protocol layers. It supports a wide range of protocols such as IP, TCP, UDP, HTTP, Kafka, gRPC, and DNS. This broad protocol support ensures that applications can communicate more effectively, with enhanced security and monitoring capabilities directly baked into the infrastructure layer.

Sidecar-Free Architecture

Cilium Service Mesh introduces a sidecar-free architecture, eliminating the need for deploying sidecar proxies alongside application containers. By integrating directly with the Linux kernel using eBPF, Cilium facilitates efficient communication between services without the added latency of passing traffic through sidecars.

This architecture not only simplifies deployment but also improves overall system performance. Organizations can benefit from a streamlined operational model that lowers the barrier to entry for adopting service mesh technology. With Cilium, users gain flexibility in how they implement service meshes, choosing between sidecar and sidecar-free configurations based on their specific needs and constraints.

Read our related guide: Service Mesh Architecture

Advanced Protocol Handling

Cilium Service Mesh excels in advanced protocol handling, managing a wide array of protocols including IP, TCP, UDP, HTTP, Kafka, gRPC, and DNS. This capability enables Cilium to efficiently facilitate communication between services across different layers of the network stack. By doing so, it ensures that applications can leverage complex protocols without compromising on performance or security.

The handling of these protocols directly within the kernel via eBPF technology allows for greater efficiency and scalability. This approach minimizes latency and maximizes throughput by eliminating unnecessary layers of abstraction. Consequently, Cilium Service Mesh provides organizations with the flexibility to support diverse application requirements while maintaining high levels of security and observability.

Flexible Control Plane Options

Cilium Service Mesh offers a variety of control plane options to cater to different organizational needs and preferences. Users can select from simpler mechanisms like Ingress and Gateway API for straightforward scenarios or opt for more comprehensive solutions such as Istio for complex environments. Additionally, the integration with Envoy through the Envoy CRD is available for those seeking advanced routing capabilities and granular policy control.

Related content: Learn more about Cilium Cluster Mesh

Understanding How Cilium Service Mesh Works with Examples

Cilium Service Mesh and Network Layer 7

Cilium is able to handle network layer 7 (the application layer), offering detailed visibility and control over application traffic within the service mesh. By supporting a broad spectrum of application layer protocols and services, including HTTP, FTP, and SMTP, Cilium enables the enforcement of network policies specifically tailored to application-level interactions.

The integration of Cilium with container runtime environments is facilitated through the Cilium CNI (Container Network Interface).

To enforce network policies at the application layer, Cilium allows the creation of L7 policies. These policies can be applied to a CiliumNetworkPolicy resource, either through the kubectl command-line tool or by crafting a YAML file. Specifically, to implement an L7 policy, one must:

Create a CiliumNetworkPolicy resource, outlining the desired network policies.
Incorporate an L7 policy by adding either a spec.egress or spec.ingress field to the CiliumNetworkPolicy resource and specifying the L7 rules.
Detail the HTTP rules within the L7 policy using the http field in the spec.egress or spec.ingress section. These rules can dictate allowed HTTP methods or specify paths to match.

After configuring the CiliumNetworkPolicy resource, applying it to the cluster is accomplished with the kubectl apply or kubectl create command, thus enforcing the specified HTTP rules.

An example of a YAML configuration for an L7 policy that permits HTTP traffic only on the path /mypath is as follows:

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: service-account
spec:
  endpointSelector:
    matchLabels: null
    io.cilium.k8s.policy.serviceaccount: server
  ingress:
    - fromEndpoints:
        - matchLabels: null
          io.cilium.k8s.policy.serviceaccount: client
      toPorts:
        - ports: null
          port: "3555"
          protocol: TCP
          rules: null
          http:
            - method: GET
              path: /mypath

Cilium Service Mesh and Identity Security

Cilium improves security through identity management features. A central aspect of this is identity-based authorization, which regulates access to resources according to the entity’s identity making the request. Identities within Cilium can be derived from various sources, including pod or service labels, Kubernetes namespaces, or IP addresses.

Additionally, Cilium supports identity-based encryption to secure communications between components within the cluster. This encryption is contingent on the identities of the communicating entities, bolstering data security and protecting against unauthorized data access.

Here is an example of a CiliumNetworkPolicy that implements identity-based access restrictions:

apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
  name: "restrict-access"
spec:
  ingress:
  - fromEndpoints:
    - matchLabels:
        app: "example-app-1"
  egress:
  - toEndpoints:
    - matchLabels:
        app: "example-app-2"

This policy restricts ingress traffic to pods labeled with app: example-app-1 and egress traffic to pods labeled with app: example-app-2, thus enforcing traffic flow based on pod identities. Applying such policies necessitates labeling pods according to their roles within the application architecture.

CiliumNetworkPolicy resources leverage Kubernetes label selectors to define the scope of the policy, enabling the creation of nuanced and granular access controls based on labels, namespaces, and IP addresses.

Cilium Service Mesh Limitations

Cilium Service Mesh is a relatively new offering and is considered less mature than other service mesh solutions on the market. Here are a few limitations highlighted by the Cilium community.

Mutual Authentication Can Compromise Security

According to The New Stack, Cilium’s mutual authentication mechanism uses an eventual consistency model that introduces significant security risks. The foundational concept behind mutual authentication is that both parties in a communication verify each other’s identities before proceeding with data exchanges. However, Cilium’s approach does not immediately establish this verification. Instead, it relies on an initially unauthenticated connection which, if not authenticated, drops the packets until the authentication can be validated asynchronously.

This design introduces a window where unauthenticated packets are dropped but subsequent packets might be allowed before the authentication verification is solidified. This lag can expose the system to periods where invalid or malicious traffic could be accepted, especially in highly dynamic environments like Kubernetes where service instances can frequently change.

Another concern is that this authentication process depends on the reliability and timeliness of updates to a node-local “auth cache”; there’s a risk that inconsistencies in this cache could allow unauthorized access. If the cache has not yet been updated to reflect the latest authentications, it may either deny legitimate traffic or accept illegitimate traffic. This undermines the security guarantees that mutual authentication seeks to provide, leaving significant security gaps.

Overhead and Complexity in Implementation

Implementing Cilium’s service mesh and its security features, particularly the custom “mTLess” mutual authentication, adds a significant level of complexity to Kubernetes deployments. While Cilium uses eBPF to enhance performance and reduce overhead by processing policies at the kernel level, the introduction of features like SPIFFE (Secure Production Identity Framework For Everyone) for identity verification in mutual authentication scenarios can complicate the setup.

Configuring and maintaining these authentication mechanisms requires a deep understanding of both the underlying technology and the specific security configurations. This can be a barrier for teams without extensive networking or security expertise, potentially leading to misconfigurations or suboptimal deployments.

Furthermore, the reliance on an authentication mechanism based on eventual consistency means that administrators must carefully manage the synchronization and updating of security policies and identity verifications across a distributed environment. This requirement for careful management adds to the operational overhead.

Limited Encryption Capabilities

Despite its mutual authentication features, Cilium does not inherently encrypt traffic between services. The “mTLess” authentication method focuses solely on verifying the identity of communicating parties without ensuring that the data exchanged between them remains confidential and without verifying its integrity.

This approach means that while services may authenticate each other’s identities, the data they exchange can still be transmitted in plaintext unless additional encryption measures, such as WireGuard or IPSec, are explicitly configured. This setup requires additional steps and resources, potentially complicating deployments and scaling operations.

The distinction between authentication and encryption in Cilium’s design could lead to scenarios where data is authenticated but not encrypted, posing a risk of data breaches if malicious entities access the communication channels. Organizations must therefore evaluate additional encryption solutions, increasing both the complexity and the cost of their infrastructure security.

Do you really need a service mesh? Calico offers an operationally simpler approach

A service mesh adds operational complexity and introduces an additional control plane for teams to manage. Platform owners, DevOps teams, and SREs have limited resources, so adopting a service mesh is a significant undertaking due to the resources required for configuration and operation.

Calico enables a single-pane-of-glass unified control to address the three most popular service mesh use cases—security, observability, and control—with an operationally simpler approach, while avoiding the complexities associated with deploying a separate, standalone service mesh. With Calico, you can easily achieve full-stack observability and security, deploy highly performant encryption, and tightly integrate with existing security infrastructure like firewalls.

Encryption for data in transit – Calico leverages the latest in crypto technology, using open-source WireGuard. As a result, Calico’s encryption is highly performant while still allowing visibility into all traffic flows.
Dynamic Service and Threat Graph – Kubernetes-native visualization of all collected data that allows the user to visualize communication flows across services and team spaces, to facilitate troubleshooting.
Operational simplicity with Envoy integrated into the data plane – Calico provides observability, traffic flow management, and control by deploying a single instance of Envoy as a daemon set on each node of your cluster, instead of a sidecar approach, thus making it more resource efficient and cost effective.
Zero-trust workload access controls – Integrate with firewalls or other kinds of controls where you might want to understand the origin of egress traffic. Identify the origin of egress traffic, to the point where you have visibility into the specific application or namespace from which egress traffic seen outside the cluster came.

Next steps: