How TCP/IP Networking Actually Works

Every time you open a browser tab, a quiet storm of packet surgery happens between your NIC and the remote server. Understanding it end-to-end — not just isolated trivia — changes how you debug, tune, and secure Linux networks. This guide traces a single TCP connection from process to wire and back, covering routing decisions, the three-way handshake, MTU and fragmentation, and NAT translation.

The Network Stack in Brief

Linux processes network traffic through a layered stack. A user-space application calls connect() or send(); the kernel's socket layer hands the data to TCP; TCP hands segments to IP; IP decides the next hop and hands datagrams to the link layer (Ethernet, Wi-Fi, etc.), which finally puts frames on the wire. On receipt, the process reverses. Every layer adds or strips a header — that overhead is not waste, it is the mechanism.

Routing: How a Packet Finds Its Next Hop

IP is hop-by-hop. Your host does not know the full path to 93.184.216.34 (example.com); it only knows where to send the packet next. That decision comes from the kernel routing table.

ip route show

Typical output on a home machine (will vary):

# Example output — yours will differ
default via 192.168.1.1 dev eth0 proto dhcp src 192.168.1.105 metric 100
192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.105

The kernel matches the destination address against every prefix in the table using longest-prefix match: the most specific matching route wins. 192.168.1.0/24 is more specific than 0.0.0.0/0 (the default route), so traffic to your local subnet goes directly to the link; everything else goes to the gateway at 192.168.1.1.

ARP: Turning an IP into a MAC Address

Once the kernel knows the next-hop IP, it needs the corresponding MAC address to build an Ethernet frame. It checks the ARP cache first:

ip neigh show

A miss triggers an ARP broadcast: "Who has 192.168.1.1? Tell 192.168.1.105." The gateway replies with its MAC, the kernel caches it, and the frame is sent. ARP only operates within a single L2 segment; beyond the router, the MAC changes at every hop while the IP destination stays constant.

The Three-Way Handshake

TCP is connection-oriented. Before a single byte of application data moves, the two endpoints agree on initial sequence numbers through a three-step exchange.

1. SYN — Your host sends a segment with the SYN flag set and a random Initial Sequence Number (ISN), e.g. SEQ=1000.
2. SYN-ACK — The server acknowledges your ISN (ACK=1001) and sends its own ISN (SEQ=5000).
3. ACK — Your host acknowledges the server's ISN (ACK=5001). The connection is ESTABLISHED on both ends.

Watch it live with ss or capture it with tcpdump:

ss -tn state established

sudo tcpdump -i eth0 -nn 'tcp[tcpflags] & (tcp-syn|tcp-ack) != 0' and host 93.184.216.34

The sequence numbers are the backbone of TCP's reliability. Every byte is numbered; the receiver ACKs what it got; the sender retransmits anything not acknowledged within a timeout. The ISN is randomised to harden against sequence-prediction attacks (RFC 6528).

Connection Teardown

Closing a connection is a four-way exchange: each side sends FIN and receives ACK independently, because TCP is full-duplex and either side can finish sending before the other. The TIME_WAIT state (default 60 s on Linux) keeps the socket open after the final ACK to absorb delayed duplicates — this is intentional, not a bug.

MTU and Fragmentation

Every link has a Maximum Transmission Unit: the largest payload an Ethernet frame can carry is 1500 bytes by default. The IP header (typically 20 bytes) and TCP header (typically 20 bytes) eat into that, leaving 1460 bytes for TCP payload — the Maximum Segment Size (MSS).

Check the MTU on your interfaces:

ip link show

If an IP datagram is larger than the link MTU and the DF (Don't Fragment) bit is clear, a router will fragment it into multiple datagrams. The destination reassembles them. Fragmentation is expensive and broken in many real networks — routers that silently drop oversized packets cause mysterious stalls.

Path MTU Discovery

Modern TCP avoids fragmentation entirely via Path MTU Discovery (PMTUD). All outgoing TCP segments set the DF bit. If an intermediate router cannot forward the datagram, it drops it and sends an ICMP Type 3 Code 4 (Fragmentation Needed) message back, including the MTU of the bottleneck link. The sender reduces its segment size accordingly.

PMTUD breaks when firewalls block ICMP. The symptom: large transfers stall or fail while small ones (ping, DNS) work fine — PMTUD black hole. Linux's TCP MSS clamping can work around this:

# On a router/gateway with nftables — clamp MSS on forwarded traffic
nft add table ip filter
nft add chain ip filter FORWARD { type filter hook forward priority 0 \; }
nft add rule ip filter FORWARD tcp flags syn tcp option maxseg size set rt mtu

VPN tunnels add encapsulation overhead and frequently require a reduced MTU — 1420 bytes is common for WireGuard. Always verify end-to-end with a large ping:

# Test for PMTUD issues: send a 1472-byte payload (1472 + 28 byte IP/ICMP header = 1500)
ping -M do -s 1472 93.184.216.34

NAT: Network Address Translation

Most home and office networks use private RFC 1918 addresses (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16) that are not routable on the public internet. NAT — specifically Source NAT (SNAT) or IP Masquerade — lets many private hosts share a single public IP.

What Actually Happens

1. Your host (192.168.1.105:54321) sends a SYN to 93.184.216.34:443.
2. The gateway rewrites the source to its public IP and a new source port, e.g. 203.0.113.1:61000, and records this mapping in a conntrack table.
3. The server sees the connection coming from 203.0.113.1:61000 and replies to that address.
4. The gateway looks up the conntrack entry, rewrites the destination back to 192.168.1.105:54321, and forwards the packet inbound.

Inspect the conntrack table on a Linux gateway:

sudo conntrack -L -p tcp --dport 443

Conntrack entries time out. Long-lived idle connections (SSH over NAT, database pools) can be silently dropped when the NAT mapping expires — typically 120–300 s for established TCP. Counter this with TCP keepalives (net.ipv4.tcp_keepalive_time) or application-level heartbeats.

DNAT: Port Forwarding

Destination NAT reverses the translation: incoming traffic on a public port is rewritten to reach a private host. With nftables on a gateway:

# Forward public port 8080 to internal host 192.168.1.20:80
nft add table ip nat
nft add chain ip nat PREROUTING { type nat hook prerouting priority -100 \; }
nft add rule ip nat PREROUTING tcp dport 8080 dnat to 192.168.1.20:80

Verification: Tracing a Packet End to End

Combine these tools to confirm the full path:

# Trace the routing path (uses UDP probes by default)
traceroute 93.184.216.34

# Or with ICMP (often less filtered)
sudo traceroute -I 93.184.216.34

# Full packet capture: handshake + data on port 443
sudo tcpdump -i eth0 -nn -w /tmp/capture.pcap host 93.184.216.34 and port 443
# Open with: wireshark /tmp/capture.pcap

# Check socket state and retransmit counters
ss -tipm

Troubleshooting Quick Reference

• Connection refused: TCP RST from the destination — the port is closed or the service is not listening. Check ss -tlnp on the server.
• Connection timeout: No reply at all — packet is being dropped by a firewall, routing is broken, or the host is unreachable. Use traceroute to find where packets stop.
• Large transfers stall, small ones work: Classic PMTUD black hole. Test with ping -M do -s 1472; reduce MTU or enable MSS clamping.
• SSH drops after idle periods: NAT conntrack timeout. Enable TCP keepalives: ssh -o ServerAliveInterval=60 ... or set ClientAliveInterval 60 in /etc/ssh/sshd_config.
• Wrong interface used: Check ip route get <destination> to see exactly which interface and gateway the kernel selects for a specific destination.

# See exactly which route the kernel picks for a destination
ip route get 93.184.216.34