Understanding TCP Window Scaling and Congestion Control

Introduction

TCP is a reliable, ordered byte stream protocol — it guarantees delivery, ordering, and error detection. But “reliable” doesn’t mean “fast.” TCP’s flow control and congestion control mechanisms exist to prevent overwhelming the receiver’s buffer and collapsing the network under load. Understanding these mechanisms is essential for diagnosing TCP performance issues — why a 1 Gbps link might only achieve 100 Mbps for a single connection, or why high-latency satellite links have poor throughput despite high bandwidth.

flowchart TD
    A["Receiver advertises rwnd (buffer capacity)"]
    B["Sender starts cwnd small, grows via slow start"]
    B --> C["Steady state: CUBIC or BBR tunes cwnd to path capacity"]
    A --> D["Effective send window = min(rwnd, cwnd)"]
    C --> D

Flow Control: Receive Window

The TCP receive window (rwnd) is the amount of unacknowledged data the receiver can accept. Advertisied in each ACK:

TCP Header: Window Size (16-bit) = 65,535 bytes max
Throughput = window_size / RTT

Example: 65,535 bytes / 0.2s RTT = 327 KB/s  ← Terrible for modern networks!

Window Scaling (RFC 7323)

The 16-bit window field limits the window to 65,535 bytes — inadequate for high-bandwidth, high-latency links. Window scaling extends this with a multiplier:

Window Scale Option: scale_factor = 7 (means multiply by 2^7 = 128)
Effective window = 65,535 × 128 = 8,388,480 bytes (~8 MB)

With 200ms RTT: 8 MB / 0.2s = 40 MB/s  ← Now reasonable for a 1 Gbps link

Check window scaling:
ss -ti 'dst 203.0.113.42' | grep rcvbuf

Congestion Control

While rwnd prevents overwhelming the receiver, congestion control prevents overwhelming the network. TCP maintains a Congestion Window (cwnd) limiting unacknowledged bytes in transit:

Effective transmit rate = min(rwnd, cwnd) / RTT

Slow Start

Initial cwnd = 10 segments (~14,600 bytes, per RFC 6928)
Each ACK → cwnd += 1 segment (exponential growth)

Round 1: cwnd = 10 → 20 (if all 10 ACKed)
Round 2: cwnd = 20 → 40
Round 3: cwnd = 40 → 80  ← Reaches ssthresh, switches to congestion avoidance

CUBIC (Linux default before kernel 5.8)

CUBIC uses a cubic growth function after packet loss, recovering faster than older algorithms:

After packet loss detected (triple duplicate ACK):
  ssthresh = cwnd × β (β = 0.7 for CUBIC)
  cwnd = ssthresh (multiplicative decrease)
  
Recovery follows cubic function of time since last loss:
  cwnd(t) = C × (t - K)³ + W_max
  Where K = time to reach W_max, C = scaling factor

BBR (Bottleneck Bandwidth and RTT — Linux default since kernel 5.8)

BBR is fundamentally different — it models the network path rather than reacting to packet loss:

BBR continuously estimates:
  BtlBw (bottleneck bandwidth) = max delivery rate observed over recent RTTs
  RTprop (minimum RTT observed) = estimated propagation delay

Sends at rate: cwnd = BtlBw × RTprop (fill the pipe, not more)
Does not rely on packet loss as congestion signal (loss = filled buffer, not optimal)

BBR advantage on lossy networks (WiFi, satellite):
  CUBIC: Packet loss → halve cwnd (even if loss is random, not congestion)
  BBR: Packet loss → maintain rate if RTT and delivery rate are stable

# Check current congestion control
sysctl net.ipv4.tcp_congestion_control
# Enable BBR
sysctl -w net.ipv4.tcp_congestion_control=bbr

# Monitor TCP connection stats
ss -ti src :443 | head -20
# Output includes: cwnd, rtt, retrans, bytes_sent, bytes_acked

TCP Throughput Formula

Ideal throughput = (MSS / RTT) × (1 / √loss_rate)

Example:
  MSS = 1460 bytes
  RTT = 100ms = 0.1s
  Loss rate = 0.001 (0.1%)

Throughput = (1460 / 0.1) × (1 / √0.001)
           = 14,600 × 31.6
           = ~461 KB/s

↑ This is why high-latency satellite links have terrible TCP throughput even with high bandwidth!

Key Takeaways

• Receive Window (rwnd) limits throughput to window_size / RTT — without window scaling, this caps single-connection throughput at ~327 KB/s on a 200ms RTT link.
• Window Scaling (RFC 7323) extends the 16-bit window to effective sizes up to 1 GB — critical for high-bandwidth, high-latency paths (satellite, cross-continental).
• Slow start grows cwnd exponentially until packet loss, then switches to congestion avoidance (linear growth) — the initial burst is why new connections are slow.
• BBR models the network pipe (bandwidth-delay product) rather than reacting to loss — significantly better throughput on lossy or high-BDP links.
• Use ss -ti to inspect live TCP connection parameters including cwnd, retransmit counts, and RTT estimates.