Reading a TCP handshake to tell a firewall drop from a dead service

When a client cannot reach a service, the cause is rarely obvious from the error message alone. A connection timeout, refused connection, and a firewall drop all feel identical to the end user. However, a packet capture reveals the truth in the TCP handshake itself. By reading what the server sends back--or does not send back--you can pinpoint whether a firewall is blocking traffic, the service is down, or the port is simply closed.

The normal TCP handshake

A successful TCP connection follows three steps: the client sends a SYN packet, the server responds with SYN-ACK, and the client sends ACK. This three-way handshake establishes the connection. When troubleshooting, you are looking for where this sequence breaks or deviates.

Client                          Server
  |                              |
  |------------ SYN ------------->|
  |                              |
  |<------- SYN-ACK -------------|  
  |                              |
  |------------ ACK ------------->|
  |                              |
  |     Connection established    |

Firewall drop: the silent treatment

When a firewall silently drops traffic, it discards the packet without sending any response. From the client's perspective, nothing comes back. In a packet capture, you see the client send a SYN, then wait. After a timeout (typically 30 to 60 seconds, depending on the OS), the client gives up and reports a timeout error.

• →Client sends SYN to destination IP and port
• →No response arrives (firewall silently drops it)
• →Client retransmits SYN multiple times
• →After timeout period, connection times out
• →Packet capture shows outgoing SYN but no return traffic

This pattern is the hallmark of a firewall block or an unreachable network. The server never sees the traffic, so it never responds.

Dead service: the RST response

When a service is down or the port is closed, the server (or the OS on the server) sends back a TCP RST (reset) packet. This is an explicit rejection. In a packet capture, you see the client send SYN, and the server immediately respond with RST or RST-ACK. The client receives this rejection almost instantly and reports 'connection refused' rather than 'timeout.'

• →Client sends SYN to destination IP and port
• →Server responds with RST or RST-ACK (port closed or service not listening)
• →Connection refused error appears immediately
• →Packet capture shows SYN followed by RST within milliseconds

The RST response is fast and definitive. It tells you the server is reachable but the service is not listening on that port.

Reading the capture: what to look for

When you open a packet capture in Wireshark or tcpdump, filter for TCP traffic to the suspect port and look at the flags:

tcpdump -i eth0 -n 'tcp port 443'

# Firewall drop pattern:
# 10.0.0.5.54321 > 10.0.0.10.443: Flags [S], seq 0
# (no response, retransmit after 1 second, then 2 seconds, etc.)

# Dead service pattern:
# 10.0.0.5.54321 > 10.0.0.10.443: Flags [S], seq 0
# 10.0.0.10.443 > 10.0.0.5.54321: Flags [R.], seq 0, ack 1

• →Flags [S] = SYN (connection initiation)
• →Flags [S.] = SYN-ACK (server accepting)
• →Flags [.] = ACK (acknowledgment)
• →Flags [R] or [R.] = RST (reset, connection refused)
• →Flags [F] = FIN (graceful close)

If you see SYN retransmissions with no response, suspect a firewall or network unreachability. If you see an immediate RST, the service is down or the port is closed. If you see SYN-ACK, the connection is working and the issue lies in the application layer.

Next steps

• →Capture traffic from both client and server sides if possible to rule out asymmetric routing
• →Verify firewall rules and ACLs permit the traffic
• →Confirm the service is running and bound to the expected port
• →Check for intermediate proxies or load balancers that might alter the handshake
• →Use a port reference to confirm the expected protocol and default port for the service