TL;DR: A Service Mesh provides a dedicated infrastructure layer for managing communication between services. It can standardize service-to-service security, traffic management, and observability across distributed applications, with minimal changes to application code.
Modern business applications are often divided into smaller, interconnected services. For example, an Inventory service may need to communicate with Procure and Distribute services to fulfill a single request.
A typical workflow might involve a Web UI, an application service, a caching layer, a database, inventory verification, purchase confirmation, and a payment gateway. As the number of services grows, managing their communication becomes increasingly complex.
How do we secure these connections, control traffic, and troubleshoot failures when services run across multiple Kubernetes clusters, regions, or cloud providers?
This is where a Service Mesh becomes useful.
Pods, Pause Containers, and Proxy Sidecars
To understand how a service mesh works, we first need to understand the Kubernetes Pod and how a proxy can participate in its network communication.
Part 1: The Container Foundation — Pause Container
The Pod is the smallest deployable unit in Kubernetes. It can contain one or more containers that run together and share selected resources.
In a typical Linux-based Kubernetes environment, an infrastructure container called the pause container helps maintain the Pod’s shared namespaces. It starts early in the Pod’s lifecycle and provides a process that keeps those namespaces alive, even if an application container crashes or restarts.
Containers in the same Pod share a network namespace. They therefore use the same Pod IP address and can communicate through localhost, provided they use different listening ports.
The pause container does not perform application networking itself. The container runtime and Kubernetes networking components establish the Pod’s network configuration.
+----------------------------------------+
| KUBERNETES POD |
| |
| +---------------+ +----------------+ |
| | PAUSE | | APPLICATION | |
| | CONTAINER | | CONTAINER | |
| +---------------+ +----------------+ |
| |
| Shared network namespace |
| Shared Pod IP address |
| localhost communication |
+----------------------------------------+
This shared network environment is important because a proxy sidecar can communicate with the application locally while also handling network traffic to and from other services.
Part 2: Introducing the Sidecar Proxy
The sidecar pattern places an additional container alongside the main application container in the same Pod.
In a sidecar-based service mesh, this additional container runs a network proxy. For example, Istio uses Envoy proxies to mediate communication between services.
The proxy belongs to the service mesh’s data plane. Its responsibilities can include:
- Mutual TLS (mTLS): Encrypting connections and authenticating both communicating workloads when the mesh is configured to enforce mTLS.
- Traffic management: Applying routing rules, timeouts, retries, and circuit-breaking policies where supported.
- Security policies: Enforcing service-to-service access controls.
- Observability: Collecting traffic metrics and supporting access logs and distributed tracing.
These capabilities allow infrastructure teams to manage many networking concerns centrally, reducing the amount of networking-specific code that application developers need to implement.
However, simply adding a proxy container does not guarantee that all traffic passes through it. Traffic interception must be configured through mechanisms such as iptables rules or other implementation-specific networking components.
+-------------------------------------------+
| KUBERNETES POD |
| |
| Shared network namespace |
| |
| +------------------+ |
| | APPLICATION | |
| | CONTAINER | |
| +------------------+ |
| | |
| | Local network communication |
| v |
| +------------------+ |
| | SIDECAR PROXY | |
| | e.g., Envoy | |
| +------------------+ |
| ^ |
+-----------|-------------------------------+
|
| Configured traffic interception
|
v
Other services or
external destinations
The diagram is a simplified logical view. Actual traffic paths depend on the mesh implementation and its interception configuration.
Part 3: Understanding the Control Plane
A service mesh requires a way to configure and manage its proxies. This is the responsibility of the control plane.
The control plane distributes configuration describing how services should communicate. Depending on the implementation, it can also manage service discovery, workload identities, certificates, and policy configuration.
For example, in Istio, Istiod manages service discovery, configuration, and certificate management, while Envoy proxies handle the corresponding data-plane traffic.
The relationship can be summarized as follows:
+-------------------------------------+
| CONTROL PLANE |
| |
| Service discovery |
| Traffic configuration |
| Certificate management |
| Policy distribution |
+------------------+------------------+
|
| Configuration
v
+-------------------------------------+
| DATA PLANE |
| |
| Application <-> Proxy |
| | |
| v |
| Network traffic |
| Security policies |
| Telemetry |
+-------------------------------------+
The control plane defines and distributes the desired behavior. The data plane processes the actual service-to-service communication.
This separation makes it possible to change routing rules or security policies without rebuilding every application, subject to the mesh’s capabilities and configuration.
Part 4: Building a Multi-Cluster Service Mesh
The service mesh model can extend beyond a single Kubernetes cluster.
Consider a business operating four clusters:
- Cluster A: Web applications and customer-facing APIs.
- Cluster B: Inventory and procurement services.
- Cluster C: Payment and transaction processing.
- Cluster D: Reporting and analytics.
These services may need to communicate across cluster boundaries. A service mesh can help standardize their security, routing, and observability.
MULTI-CLUSTER SERVICE MESH
+------------------+ +------------------+
| CLUSTER A | | CLUSTER B |
| | | |
| [Application] | | [Application] |
| | | | | |
| [Proxy] | | [Proxy] |
+--------|---------+ +--------|---------+
| |
+------ Mesh Link -----+
|
+----------+-----------+
| |
+--------|---------+ +--------|---------+
| [Proxy] | | [Proxy] |
| | | | | |
| [Application] | | [Application] |
| | | |
| CLUSTER C | | CLUSTER D |
+------------------+ +------------------+
This diagram illustrates the logical relationship between clusters. It does not represent a mandatory network topology or a direct connection between every proxy.
In a real multi-cluster deployment, the mesh must be configured to support cross-cluster service discovery, network connectivity, workload identity, certificate trust, and traffic routing. The exact requirements depend on the chosen service mesh and whether the clusters share a network or use separate networks.
When correctly configured, the mesh can provide consistent service-to-service security and traffic management across these environments. For example, an inventory service in Cluster B could communicate securely with a payment service in Cluster C without requiring each application team to implement its own networking and certificate-management logic.
Part 5: Sidecar Is One Implementation Model
The sidecar pattern is a common service mesh architecture, but it is not the only option.
For example, Istio supports ambient mode, which uses node-level proxies to handle Layer 4 traffic and can use waypoint proxies for Layer 7 capabilities.
This alternative can reduce the need to deploy a dedicated sidecar in every application Pod. The appropriate model depends on the required features, operational overhead, security requirements, and deployment environment.
Part 6: When Should We Use a Service Mesh?
A service mesh can be useful when an organization operates many services and needs consistent communication policies across them.
Common reasons include:
- Standardizing workload-to-workload authentication and encryption.
- Applying consistent traffic-routing and resilience policies.
- Improving visibility into communication failures and latency.
- Managing service communication across multiple clusters.
- Reducing duplicated networking logic across application teams.
However, a service mesh also introduces operational complexity, resource consumption, configuration requirements, and additional troubleshooting considerations. A small application with only a few services may not need one.
The decision should be based on actual networking, security, and operational requirements rather than adopting a service mesh simply because the application uses Kubernetes.
Back to Certified Cloud Native Platform Engineering Associate Preparation