Understanding Pod Connectivity

Introduction

This tutorial will guide you through understanding and verifying pod-to-pod connectivity within a Kubernetes cluster. You’ll learn how to deploy test pods, use kubectl commands to inspect network policies and services, and utilize nslookup and curl to test connectivity. This tutorial assumes you have a running Kubernetes cluster and kubectl configured to connect to it.

Prerequisites

A running Kubernetes cluster (Minikube, Kind, or a cloud provider).
kubectl installed and configured.
Basic understanding of Kubernetes concepts like Pods, Services, and Namespaces.

Task 1: Creating Test Namespaces

To isolate our test deployments, we’ll create two namespaces: test-ns-a and test-ns-b.

Create the test-ns-a namespace:
NODE_TYPE // yaml
```
apiVersion: v1
kind: Namespace
metadata:
  name: test-ns-a
```
Apply the manifest:
NODE_TYPE // bash
```
kubectl apply -f - <<EOF
apiVersion: v1
kind: Namespace
metadata:
  name: test-ns-a
EOF
```
NODE_TYPE // output
```
namespace/test-ns-a created
```
Create the test-ns-b namespace:
NODE_TYPE // yaml
```
apiVersion: v1
kind: Namespace
metadata:
  name: test-ns-b
```
Apply the manifest:
NODE_TYPE // bash
```
kubectl apply -f - <<EOF
apiVersion: v1
kind: Namespace
metadata:
  name: test-ns-b
EOF
```
NODE_TYPE // output
```
namespace/test-ns-b created
```

Task 2: Deploying Test Pods

We’ll deploy simple nginx pods in each namespace. These pods will act as our connectivity endpoints.

Create a deployment file named nginx-deployment-a.yaml for test-ns-a:
NODE_TYPE // yaml
```
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-a
  namespace: test-ns-a
  labels:
    app: nginx-a
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx-a
  template:
    metadata:
      labels:
        app: nginx-a
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 80
```
Apply the deployment:
NODE_TYPE // bash
```
kubectl apply -f nginx-deployment-a.yaml
```
NODE_TYPE // output
```
deployment.apps/nginx-a created
```
Create a deployment file named nginx-deployment-b.yaml for test-ns-b:
NODE_TYPE // yaml
```
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-b
  namespace: test-ns-b
  labels:
    app: nginx-b
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx-b
  template:
    metadata:
      labels:
        app: nginx-b
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 80
```
Apply the deployment:
NODE_TYPE // bash
```
kubectl apply -f nginx-deployment-b.yaml
```
NODE_TYPE // output
```
deployment.apps/nginx-b created
```
Verify the pods are running:
NODE_TYPE // bash
```
kubectl get pods -n test-ns-a
kubectl get pods -n test-ns-b
```
NODE_TYPE // output
```
NAME                       READY   STATUS    RESTARTS   AGE
nginx-a-xxxxxxxxxx-xxxxx   1/1     Running   0          XXs

NAME                       READY   STATUS    RESTARTS   AGE
nginx-b-yyyyyyyyyy-yyyyy   1/1     Running   0          YYs
```
Replace xxxxxxxxxx-xxxxx and yyyyyyyyyy-yyyyy with the actual pod names generated by your cluster.

Task 3: Testing Connectivity Without a Service

By default, pods within the same cluster can communicate with each other directly using their IP addresses. Let’s test this.

Get the IP address of nginx-a pod:
NODE_TYPE // bash
```
kubectl get pod -n test-ns-a -o wide
```
NODE_TYPE // output
```
NAME                       READY   STATUS    RESTARTS   AGE   IP            NODE       NOMINATED NODE   READINESS GATES
nginx-a-xxxxxxxxxx-xxxxx   1/1     Running   0          XXs   10.244.0.10   node-1   <none>           <none>
```
Replace 10.244.0.10 with the actual IP address of your nginx-a pod.
Exec into the nginx-b pod and use curl to access the nginx-a pod’s IP address. If curl is not present, install it with apt update && apt install -y curl:
NODE_TYPE // bash
```
kubectl exec -it -n test-ns-b nginx-b-yyyyyyyyyy-yyyyy -- bash
```
NODE_TYPE // bash
```
apt update && apt install -y curl
curl 10.244.0.10
```
NODE_TYPE // output
```
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
...
</html>
```
A successful response indicates that the nginx-b pod can reach the nginx-a pod directly via its IP address.

Task 4: Creating Services for Stable Endpoints

Direct pod IP addresses are ephemeral. Services provide a stable endpoint for accessing pods. Let’s create services for our nginx deployments.

Create a service file named nginx-service-a.yaml for nginx-a:
NODE_TYPE // yaml
```
apiVersion: v1
kind: Service
metadata:
  name: nginx-service-a
  namespace: test-ns-a
spec:
  selector:
    app: nginx-a
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
```
Apply the service:
NODE_TYPE // bash
```
kubectl apply -f nginx-service-a.yaml
```
NODE_TYPE // output
```
service/nginx-service-a created
```
Create a service file named nginx-service-b.yaml for nginx-b:
NODE_TYPE // yaml
```
apiVersion: v1
kind: Service
metadata:
  name: nginx-service-b
  namespace: test-ns-b
spec:
  selector:
    app: nginx-b
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
```
Apply the service:
NODE_TYPE // bash
```
kubectl apply -f nginx-service-b.yaml
```
NODE_TYPE // output
```
service/nginx-service-b created
```
Get the service details:
NODE_TYPE // bash
```
kubectl get svc -n test-ns-a nginx-service-a
kubectl get svc -n test-ns-b nginx-service-b
```
NODE_TYPE // output
```
NAME              TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
nginx-service-a   ClusterIP   10.96.147.126   <none>        80/TCP    XXs

NAME              TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
nginx-service-b   ClusterIP   10.96.228.110   <none>        80/TCP    YYs
```
Note the CLUSTER-IP for each service. These are the stable endpoints we’ll use for communication.

Task 5: Testing Connectivity Using Service Names

Kubernetes provides DNS resolution for services within the cluster. Pods can use service names to access other services.

Exec into the nginx-b pod:
NODE_TYPE // bash
```
kubectl exec -it -n test-ns-b nginx-b-yyyyyyyyyy-yyyyy -- bash
```
Use curl to access the nginx-service-a using its service name within the test-ns-a namespace:
NODE_TYPE // bash
```
curl nginx-service-a.test-ns-a.svc.cluster.local
```
NODE_TYPE // output
```
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
...
</html>
```
This command uses the fully qualified domain name (FQDN) of the service. nginx-service-a is the service name, test-ns-a is the namespace, svc indicates it’s a service, and cluster.local is the default cluster domain.
You can also use nslookup to resolve the service name to its Cluster IP:
NODE_TYPE // bash
```
nslookup nginx-service-a.test-ns-a.svc.cluster.local
```
NODE_TYPE // output
```
Server:         10.96.0.10
Address:        10.96.0.10#53

Name:   nginx-service-a.test-ns-a.svc.cluster.local
Address: 10.96.147.126
```
This confirms that DNS resolution is working correctly within the cluster.

Task 6: (Optional) Introducing Network Policies

Network policies control traffic flow between pods. Let’s demonstrate how to restrict connectivity.

Create a network policy file named deny-all-egress.yaml in test-ns-b to block all outgoing traffic:
NODE_TYPE // yaml
```
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-all-egress
  namespace: test-ns-b
spec:
  podSelector: {}
  egress: []
  policyTypes:
  - Egress
```
Apply the network policy:
NODE_TYPE // bash
```
kubectl apply -f deny-all-egress.yaml
```
NODE_TYPE // output
```
networkpolicy.networking.k8s.io/deny-all-egress created
```
Repeat the curl command from Task 5:
NODE_TYPE // bash
```
kubectl exec -it -n test-ns-b nginx-b-yyyyyyyyyy-yyyyy -- bash
curl nginx-service-a.test-ns-a.svc.cluster.local
```
This time, the curl command will likely timeout. The deny-all-egress policy prevents the nginx-b pod from initiating connections to any other pod, including the nginx-a pod.

Network Policy behavior depends on your CNI (Container Network Interface) provider. Some CNIs may not fully support Network Policies.
To remove the network policy:
NODE_TYPE // bash
```
kubectl delete networkpolicy deny-all-egress -n test-ns-b
```

Conclusion

In this tutorial, you learned how to verify connectivity between pods in Kubernetes. You created test deployments, used kubectl to inspect pod IP addresses and create services, and tested connectivity using curl and service names. You also explored the impact of network policies on pod-to-pod communication. Understanding these concepts is crucial for troubleshooting network issues and building secure, well-connected applications in Kubernetes.