Skip to main content
Cloud & AI Hub
Browse
Glossary AI Directory Playgrounds Models Prompts Explainers Strategy Matrix Benchmark Decoder
intermediate 9 mins Read

How Blue-Green Deployments Eliminate Downtime

A technical walkthrough of blue-green deployment strategy using Kubernetes Service selectors, database migration challenges, smoke testing, and instant rollback.

Introduction

Blue-green deployment is a release strategy that maintains two identical production environments — “blue” (currently live) and “green” (staging the new release) — and switches traffic between them atomically. The key insight is that the cutover is a traffic routing change, not a deployment operation: the new version is fully running, health-checked, and smoke-tested on green before any user traffic reaches it. When the switch happens (flipping a Service selector in Kubernetes, changing a load balancer target group, or swapping a DNS weighted record), it is nearly instantaneous, and rollback is equally fast — just reverse the traffic pointer.

The pattern trades infrastructure cost (running two full environments simultaneously) for deployment confidence and rollback speed. In a rolling deployment, you’re upgrading individual pods one by one, and if a critical bug surfaces mid-rollout, you’ve already replaced 50% of your capacity with the broken version — rollback means another slow rolling replacement. With blue-green, the blue environment is untouched until you’ve verified green is healthy, making rollback a sub-second operation. This is why blue-green is the preferred pattern for teams with strict SLA requirements and complex, difficult-to-test changes.

Step-by-Step: Blue-Green Deployment on Kubernetes

flowchart TD
    A["1. Define Two Identical Deployments with Version Labels"]
    B["2. Deploy Green and Wait for Readiness"]
    C["3. The Kubernetes Service"]
    D["4. Database Migration"]
    E["5. Smoke Test Automation Before Cutover"]
    F["6. Atomic Traffic Cutover"]
    G["7. Rollback Procedure"]
    A --> B
    B --> C
    C --> D
    D --> E
    E --> F
    F --> G

Step 1: Define Two Identical Deployments with Version Labels

Both blue and green deployments are defined in advance, differentiated only by their version label:

# blue-deployment.yaml (currently live)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-blue
  namespace: production
spec:
  replicas: 6
  selector:
    matchLabels:
      app: api
      version: blue
  template:
    metadata:
      labels:
        app: api
        version: blue
    spec:
      containers:
      - name: api
        image: registry.example.com/api:v1.4.2
        ports:
        - containerPort: 8080
        readinessProbe:
          httpGet:
            path: /healthz
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 10
---
# green-deployment.yaml (new version, not yet live)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-green
  namespace: production
spec:
  replicas: 6
  selector:
    matchLabels:
      app: api
      version: green
  template:
    metadata:
      labels:
        app: api
        version: green
    spec:
      containers:
      - name: api
        image: registry.example.com/api:v1.5.0

Step 2: Deploy Green and Wait for Readiness

Apply the green deployment and wait for all pods to pass their readiness probe before touching the Service:

kubectl apply -f green-deployment.yaml

# Wait until all 6 replicas are ready
kubectl rollout status deployment/api-green -n production

# Verify all pods are running the correct image
kubectl get pods -n production -l version=green \
  -o jsonpath='{.items[*].spec.containers[0].image}'

At this point, green is running and serving its own health checks, but no user traffic is reaching it — the Kubernetes Service selector still points to version: blue.

Step 3: The Kubernetes Service — Traffic’s Control Point

The Service definition is the routing mechanism. Its selector field determines which pods receive traffic:

# service.yaml (currently pointing to blue)
apiVersion: v1
kind: Service
metadata:
  name: api-service
  namespace: production
spec:
  selector:
    app: api
    version: blue  # <-- This is the traffic switch
  ports:
  - port: 80
    targetPort: 8080
  type: ClusterIP

The Kubernetes kube-proxy and kube-endpoints controller maintain iptables/IPVS rules mapping the Service ClusterIP to the pod IPs matching the selector. Changing the selector immediately updates the endpoints, rerouting all new connections to the new pod set. Existing long-lived connections (WebSockets, HTTP keep-alive) continue to the old pods until closed — plan for connection draining.

Step 4: Database Migration — The Hard Part

The trickiest aspect of blue-green deployments is database schema changes. If green requires a new column (user_preferences JSONB NOT NULL DEFAULT '{}'), you cannot just run the migration before the cutover — blue is still writing to the database with the old schema and may break if it encounters the new column with a NOT NULL constraint.

The expand-contract pattern (also called parallel change) solves this:

Phase 1 (Expand) — before the cutover, run a migration that is backward-compatible with both blue and green:

-- Safe: adds nullable column, blue ignores it, green can use it
ALTER TABLE users ADD COLUMN IF NOT EXISTS
  user_preferences JSONB DEFAULT '{}';

Phase 2 (Contract) — after blue is decommissioned, in a separate deployment cycle, apply the breaking constraint:

-- Only safe after blue is fully retired
ALTER TABLE users ALTER COLUMN user_preferences SET NOT NULL;

This means you may need to keep backward-compatible schemas for 2 deployment cycles, not just one.

Step 5: Smoke Test Automation Before Cutover

Before flipping the Service selector, run automated smoke tests against green using a test Service or direct pod port-forwarding that bypasses production routing:

# Port-forward directly to a green pod for smoke testing
kubectl port-forward deployment/api-green 8080:8080 -n production &

# Run smoke test suite against green
curl -sf http://localhost:8080/healthz | jq '.status == "ok"'
curl -sf http://localhost:8080/api/v1/ping
curl -sf -H "Authorization: Bearer $TEST_TOKEN" \
  http://localhost:8080/api/v1/users/me | jq '.id'

# Run integration test suite
./scripts/smoke-test.sh --target http://localhost:8080 --suite critical

kill %1  # Stop port-forward

Only proceed to the cutover if all smoke tests pass. Many teams gate this with a CI/CD pipeline step that fails the deployment if any test returns non-zero.

Step 6: Atomic Traffic Cutover

Once smoke tests pass, patch the Service selector to point at green:

# Atomic patch — changes the selector in a single API call
kubectl patch service api-service -n production \
  --type=json \
  -p='[{"op": "replace", "path": "/spec/selector/version", "value": "green"}]'

# Verify the endpoints now point to green pods
kubectl get endpoints api-service -n production

# Watch traffic in real-time via access logs
kubectl logs -l version=green -n production --follow --prefix

The switch takes effect in milliseconds (the time for kube-proxy to update iptables rules across all nodes).

Step 7: Rollback Procedure

If monitoring reveals issues — error rate spike, p99 latency regression, a critical bug — rollback is a single command:

# Instant rollback: switch selector back to blue
kubectl patch service api-service -n production \
  --type=json \
  -p='[{"op": "replace", "path": "/spec/selector/version", "value": "blue"}]'

# Confirm rollback
kubectl describe service api-service -n production | grep Selector

The blue deployment was never scaled down, so all 6 blue pods are immediately available to serve traffic. Total rollback time: sub-second for the API call, plus iptables propagation across nodes (usually < 5 seconds).

Key Takeaways

  • The Kubernetes Service selector is the atomic traffic switch: Patching spec.selector.version from blue to green reroutes all new connections instantly, with no rolling update delay.
  • Database migrations must use the expand-contract pattern: Applying backward-compatible schema changes before cutover and breaking changes only after the old version is fully retired prevents blue-green from introducing data layer failures.
  • Smoke test against green pods before touching the Service: Use direct pod port-forwarding or a dedicated test Service to validate the new version in production infrastructure without exposing it to real traffic.
  • Rollback is equally fast as cutover: Since blue remains fully scaled and running until you explicitly decommission it, reverting is a single kubectl patch command with sub-second propagation.
  • Long-lived connections require graceful connection draining: WebSockets and HTTP keep-alive connections remain pinned to old pods after the cutover; configure preStop lifecycle hooks with a sleep to allow existing connections to close before pod termination.

Historical figures, architectures, and capabilities are for informational purposes only. Not technical, professional, legal, or financial advice. Sources: Research papers, developer documentation.