Troubleshooting Packet Drops in Cisco ASA for Non-VPN Traffic

This packet flow is something I created based on the labs I did to understand how ASA works. This will help you to troubleshoot packet drops in ASA for non VPN traffic.

RX Ring Buffer Overrun

RX Ring is the software input queue which holds the packets for processing. If the RX Ring is filled, the traffic will be dropped before even processing. This can be seen by show interface <interface-name> command like the following.

Overrun counter will indicate how many packets were dropped due to RX Ring exhaustion. Here the "hardware (0/0)" is just because I use a ASAv for lab, but for real hardware it will show some value.

Existing Connection Check

Connection table can be viewed from show conn command.

Above snap shows an existing telnet connection originated from INSIDE interface source 192.168.10.10. If a packet which belongs to same traffic comes to the INSIDE interface, ASA will forward it to directly to NAT Translations.

If the packet is a TCP SYN or a UDP, it will be forwarded to NAT Untranslations and move on.

If there is no existing connection and if the packet is a non-TCP SYN, it will be dropped with a syslog similar to the following:

ASA-6-106015: Deny TCP (no connection) from 192.168.10.10 /32256 to 203.115.50.10 /23 flags PSH ACK on interface INSIDE

NAT Untranslation 

Packet it untranslated before ACL check; this is the reason we put ACLs to your private addresses instead of your public addresses.

The packet drop can occur due to the destination address translations and diverting to the wrong interface.

ACL Check

First packet in flow is checked against ACLs. This is why you can see fewer hits in show access-list command.

Denied packets are dropped and logged like the following:

%ASA-4-106023: Deny tcp src OUTSIDE:203.115.50.10/14379 dst INSIDE:192.168.10.10/24 by access-group "OUTLIST"

Allowed packets will be logged like the following:

%ASA-7-609001: Built local-host OUTSIDE:203.115.50.10

%ASA-7-609001: Built local-host INSIDE:192.168.10.10

%ASA-6-302013: Built inbound TCP connection 87 for OUTSIDE:203.115.50.10/18653 (203.115.50.10/18653) to INSIDE:192.168.10.10/23 (192.168.10.10/23)

NAT Translation

This is the place to run the NAT translation rules. This is done before routing so that the route lookups can be performed for translated addresses.

I don't see any reason to the traffic to be dropped here. Cisco ASA software older than version 8.2 needed the NAT to be performed mandatorily but now it is not the case. So for older ASAs, there could have been traffic drops here, but not now. 

L3 Route Lookup

Egress interface is determined 1st by NAT rules or existing connection entries. If couldn't find a macth, global routing table will be consulted.

If the L3 route lookup failed, you will get a syslog similar to following.

%ASA-6-110003: Routing failed to locate next hop for TCP from INSIDE 192.168.10.10/18653 to OUTSIDE:203.115.50.20/23

L2 Address Lookup

If layer 2 lookup fails, you will not see a syslog. show arp will not show required entry and debug arp will indicate if we are not receiving ARP reply. 

TX Ring Buffer Underrun

Same as the Buffer Overrun, if the output queue if full, the buffer underrun counter will increase and packets will be dropped.