I have a question I like to ask when I start working with a new client's infrastructure: "How do your services talk to each other?" The answer is usually something like "over HTTPS to our domain name." That sounds reasonable until you realize the services are running on the same host. Or the same network. And the traffic is leaving through the public internet only to come right back in.
This pattern has a name. It is called hairpin NAT, and it is more common than most people think. It quietly adds latency, wastes bandwidth, and creates failure modes that are genuinely confusing to debug.
What hairpin NAT actually is
Imagine two services running on the same local network. Service A needs to make an HTTP request to Service B. Instead of connecting to Service B directly using its local IP address, Service A resolves the public domain name, gets the public IP, and sends the request outward toward the internet.
The router sees the incoming packet destined for its own public IP, which matches a port-forwarding rule. It performs destination NAT (DNAT) first, rewriting the destination from the public IP to Service B's internal address. But now the packet needs to go back into the same network it came from, and if Service B sees Service A's internal IP as the source, it will reply directly — bypassing the router, so Service A will drop the reply (it expects a response from the public IP). To prevent this, the router also performs source NAT (SNAT/masquerade), rewriting the source to its own internal IP. This forces Service B's reply back through the router, where the reverse translations are applied.
The traffic goes out through the NAT gateway and comes right back in, like a hairpin turn. Hence the name.
flowchart TB
GW("Gateway / Load Balancer
203.0.113.1"):::warn
subgraph net["same network — 192.168.1.0/24"]
direction LR
A("Service A
192.168.1.10")
B("Service B
192.168.1.20")
end
A -->|"request to public IP"| GW
GW -->|"DNAT + SNAT"| B
A -.->|"direct path"| B
classDef warn fill:#fef2f2,stroke:#ef4444,color:#991b1b
style net fill:none,stroke:none
The solid lines are the hairpin: traffic goes up from Service A to the gateway and comes right back down to Service B on the same network. The return trip follows the same path in reverse — the gateway's SNAT forces Service B to reply through the gateway instead of directly. The dashed line is what should happen: a direct local connection that never touches the gateway.
Why this is a problem
It might seem harmless. The traffic still gets there, right? But the costs add up in ways that are not obvious until something breaks.
Added latency. Every request takes a detour through the NAT gateway instead of going directly between two hosts that might be on the same switch or even the same machine. For high-frequency service-to-service calls, this adds measurable delay. I have seen internal API calls go from sub-millisecond to 2-5ms purely because of the hairpin path. That does not sound like much until you have a request that fans out to twenty internal services.
NAT table exhaustion. Every connection through the NAT gateway creates an entry in the connection tracking table. These tables have a finite size. On Linux, the default nf_conntrack_max is often 65536 or similar. When services are chatty and every internal call goes through NAT, you can exhaust this table surprisingly fast. When the table fills up, new connections get dropped. Not slowed down, dropped. The symptoms look like random network failures, and they are miserable to diagnose because the issue is in the NAT gateway, not in any of the services.
Wasted bandwidth on metered links. If your NAT gateway is a cloud provider's NAT gateway, you are often paying per gigabyte of data processed. Traffic between two services in the same VPC that should cost nothing is suddenly incurring data processing charges because it is flowing through the NAT device.
Confusing network debugging. When you look at Service B's access logs, the source IP is the NAT gateway's address, not Service A's address. Every internal client looks like the same IP. You lose visibility into which service is calling which. Tracing a misbehaving client becomes much harder when all clients appear identical.
Higher resource consumption on the gateway. The NAT gateway is doing double work: translating outgoing packets and then translating them again on the way back in. On a busy network with many internal services, this can become a bottleneck that affects all traffic, not just the hairpinned calls.
How to fix it
The fix depends on your setup, but the principle is always the same: keep local traffic local.
Split-horizon DNS
The most common cause of hairpin NAT is DNS resolution. Services resolve a public domain name and get a public IP, so they connect via the public path. Split-horizon DNS (also called split DNS) solves this by returning different answers depending on where the query comes from.
When an external client resolves api.example.com, they get the public IP. When an internal service resolves the same name, they get the private IP. The service code does not change. It still connects to api.example.com. But the traffic stays local.
You can implement this with a local DNS resolver like CoreDNS or dnsmasq, or through your cloud provider's private DNS zones. Most cloud platforms support this natively: AWS has Route 53 private hosted zones, GCP has Cloud DNS private zones, and so on.
Direct local connections
In some cases, split DNS is more infrastructure than you need. If Service A always runs alongside Service B on the same network, just configure Service A to connect to Service B's local address directly. Use an environment variable or a config file for the address so it is easy to change per environment.
This is less elegant than split DNS but perfectly valid for simpler setups where the topology is well-known and stable.
Unix domain sockets for same-host services
When two services run on the same host, the best option is often to skip the network stack entirely. Unix domain sockets provide inter-process communication through the filesystem instead of through TCP/IP. There is no IP address, no port, no TCP handshake, no NAT, and no network overhead at all.
The performance difference is real. Unix sockets avoid the entire TCP/IP stack: no three-way handshake, no Nagle algorithm, no congestion control, no sequence numbering or retransmission logic. For high-throughput same-host communication, I have measured 20-30% lower latency and noticeably higher throughput compared to localhost TCP connections.
Beyond performance, Unix sockets eliminate an entire class of problems. Port exhaustion is off the table since there are no ports to exhaust. There are no conntrack entries because there is no network connection to track. And hairpin NAT? Not possible when there is no network involved in the first place.
Say you run nginx as a reverse proxy in front of an application server on the same host — you can connect them via a Unix socket instead of a TCP port:
# /etc/nginx/conf.d/app.conf
upstream app_backend {
# connect via unix socket instead of tcp
server unix:/run/app/gunicorn.sock;
}
server {
listen 80;
server_name app.example.com;
location / {
proxy_pass http://app_backend;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
}
The application server (Gunicorn, uWSGI, Puma, whatever you use) binds to the socket file instead of a TCP port:
# gunicorn binding to a unix socket
gunicorn myapp:app --bind unix:/run/app/gunicorn.sock
This is the setup I use in my own infrastructure. HAProxy terminates TLS, Varnish handles caching, and nginx connects to backend services through Unix sockets. No hairpins. No TCP overhead for local communication. The traffic that needs to traverse the network does, and the traffic that does not, stays on the filesystem.
When to care about this
If you are running a small application with a single service, hairpin NAT is not going to be your bottleneck. Do not over-optimize.
But if you are running multiple services on the same network, especially if they communicate frequently, and especially if they are on the same host, take a few minutes to check whether your traffic is hairpinning. A quick way to check: run tcpdump on the NAT gateway and look for traffic where both the source and destination resolve to internal hosts. If you see it, you have hairpin traffic.
None of the fixes are complicated. Split DNS takes an afternoon to set up and eliminates the problem for all services at once. Unix sockets take a config change per service but give you the best possible performance for same-host communication.
In my experience, fixing hairpin NAT is one of those changes that looks minor on paper but has an outsized impact — latency drops, NAT tables stop filling up, debugging actually makes sense because source IPs are real again, and you might even shave a bit off your cloud bill. It is the kind of fix where you wonder why you did not do it sooner.