Cisco ASA hairpinning

by Jimmy Larsson on 23 September, 2010 · 5 comments

Cisco Pix/ASA hairpinning

The term hairpinning comes from the fact that the traffic comes from one source into a router or similar devices, makes a U-turn and goes back the same way it came.
Visualize this and you see something that looks like a hairpin.
pinwheel hairpin Cisco ASA hairpinning
Hairpinning is only relevant when the firewall is in routed mode since the “turnaround” of traffic is a routing decision. Also there needs to be another router
involved. If the firewall is setup to only pass traffic between interfaces no hairpinning will be taking place. Typical hairpinning is done when there is a router
inside of the firewall beyond which there is another network that needs to be reached to/from the inside network. See picture:
hairpin1 300x287 Cisco ASA hairpinning
Another fundament for hairpinning taking place is that not all network equipment has full knowledge of the network topology. Typically these are computers with only
a default route (“default gateway”) to something but not aware of the fact that the remote network is reachable directly via the other router without taking the path
via the firewall (see picture above). If the computer had that knowledge it would never involve the firewall.
A workaround like this could be in a windows-host to add “route -p 192.168.2.0 mask 255.255.255.0 192.168.1.2″ from the command prompt. However this is in most cases
not very flexible since it is a manual work at each host.

ver 6.x and prior

No hairpinning was possible. The historic reason for this was that the fact that interfaces with the same security-level was not allowed to exchange traffic was a
security fature. If you needed to isolate two interfaces from each other (but allow each of them to talk to other interfaces) you could give them the same security-
level and by design there was no traffic allowed between these interfaces no matter of access-list, nat or conduit-configuration. Unfortunately this also meant that
since traffic in an out thru the same interface was per definition “same security-level”, no hair-pinning was possible.

ver 7.0

The command “same-security” was introuduced with this version. The purpose with this command was to override the isolation between interfaces with the same
security-level. The command has 2 parameters: “permit-inter-interface” that allows traffic between different interfaces with same security-level and “permit-intra-
interface” that allows traffic thru the same interface, aka hairpinning. However, with this version the intra-interface-parameter was only functional for vpn-
traffic, for example traffic from an outside vpn-client destined to internet (full tunneling).

ver 7.2

Beginning with v7.2 the “same-security permit-intra-interface”-command becomes useful and can be used for other traffic than vpn-initiated. Now we can do hair-
pinning. So, what is needed? First of all, the “same-security permit -intra-interface”. Also we need to allow inbound traffic if we have an access-list applied
inbound on the interface. Let´s have another look at our example:
hairpin2 300x276 Cisco ASA hairpinning
In this example we have one host at 192.168.1.100 using the firewall .1 as default gateway. The firewall has a route for the 192.168.2.0/24-network via 192.168.1.2
and the route is directly connected to both networks. Because of this configuration the traffic from 192.168.1.100 is hair-pinned back, to the router.


interface Vlan1
nameif inside
security-level 100
ip address 192.168.1.1 255.255.255.0
!
route inside 192.168.2.0 255.255.255.0 192.168.1.2 1
!
same-security-traffic permit intra-interface
!
no nat-control

All magic lies in the “same-security-traffic”-command. In the example above we have no access-list applied. If we have we must also open for that traffic:


access-list acl_inside extended permit ip 192.168.2.0 255.255.255.0 192.168.1.0 255.255.255.0
access-list acl_inside extended permit ip 192.168.1.0 255.255.255.0 192.168.2.0 255.255.255.0
access-list acl_inside extended deny ip any any
!
access-group acl_inside in interface inside

We most likely has a NAT/global configured for the inside network to be able to reach internet. If we add this to our example we kill our hair-pinning:


nat (inside) 1 0.0.0.0 0.0.0.0
global (outside) 1 interface

Now, when we try to ping from 192.168.1.100 to 192.168.2.200 we get this log output of the firewall:


%ASA-3-305006: portmap translation creation failed for icmp src inside:192.168.1.100 dst inside:192.168.2.200 (type 8, code 0)
%ASA-3-305006: portmap translation creation failed for icmp src inside:192.168.1.100 dst inside:192.168.2.200 (type 8, code 0)
%ASA-3-305006: portmap translation creation failed for icmp src inside:192.168.1.100 dst inside:192.168.2.200 (type 8, code 0)
%ASA-3-305006: portmap translation creation failed for icmp src inside:192.168.1.100 dst inside:192.168.2.200 (type 8, code 0)
%ASA-3-305006: portmap translation creation failed for icmp src inside:192.168.1.100 dst inside:192.168.2.200 (type 8, code 0)

As soon as we add ANY nat-configuration for an interface we must configure nat for all traffic from that interface, even hairpinned traffic. We do this with the static-command below. The purpose of this is to “static” translate traffic from interface “inside” to interface “inside” where the source is “192.168.1.0″ (netmask 255.255.255.0 and translate the source to “192.168.1.0″ (the same address). We also do the same for the 192.168.2.0-network to ensure that traffic can flow initiated in both directions.


static (inside,inside) 192.168.1.0 192.168.1.0 netmask 255.255.255.0
static (inside,inside) 192.168.2.0 192.168.2.0 netmask 255.255.255.0

The compilation of relevant configuration lines in the firewall to ackomplish this is shown here:


hostname fw
!
interface Vlan1
nameif inside
security-level 100
ip address 192.168.1.1 255.255.255.0
!
interface Vlan222
nameif outside
security-level 0
ip address dhcp setroute
!
boot system disk0:/asa725-k8.bin
!
same-security-traffic permit intra-interface
!
access-list acl_inside extended permit ip 192.168.2.0 255.255.255.0 192.168.1.0 255.255.255.0
access-list acl_inside extended permit ip 192.168.1.0 255.255.255.0 192.168.2.0 255.255.255.0
access-list acl_inside extended deny ip any any
!
nat-control
!
global (outside) 1 interface
nat (inside) 1 0.0.0.0 0.0.0.0
!
static (inside,inside) 192.168.1.0 192.168.1.0 netmask 255.255.255.0
static (inside,inside) 192.168.2.0 192.168.2.0 netmask 255.255.255.0
!
access-group acl_inside in interface inside
!
route inside 192.168.2.0 255.255.255.0 192.168.1.2 1

Ver 8.3

In this version the nat-concept is totally rewritten with a new command syntax. More about this can be read about elsewhere, but the technique is the same. This is the converted version of the configuration snippet above:


hostname fw
!
interface Vlan1
nameif inside
security-level 100
ip address 192.168.1.1 255.255.255.0
!
interface Vlan222
nameif outside
security-level 0
ip address dhcp setroute
boot system disk0:/asa831-k8.bin
!
same-security-traffic permit intra-interface
!
object network obj-192.168.1.0
subnet 192.168.1.0 255.255.255.0
object network obj-192.168.2.0
subnet 192.168.2.0 255.255.255.0
object network obj_any
subnet 0.0.0.0 0.0.0.0
object network obj_any-01
subnet 0.0.0.0 0.0.0.0
object network obj-0.0.0.0
host 0.0.0.0
!
access-list acl_inside extended permit ip 192.168.2.0 255.255.255.0 192.168.1.0 255.255.255.0
access-list acl_inside extended permit ip 192.168.1.0 255.255.255.0 192.168.2.0 255.255.255.0
access-list acl_inside extended deny ip any any
!
object network obj-192.168.1.0
nat (inside,inside) static 192.168.1.0
object network obj-192.168.2.0
nat (inside,inside) static 192.168.2.0
object network obj_any
nat (inside,outside) dynamic interface
object network obj_any-01
nat (inside,outside) dynamic obj-0.0.0.0
!
access-group acl_inside in interface inside
!
route inside 192.168.2.0 255.255.255.0 192.168.1.2 1

Assymetric routing

An issue with the configuration above is that since the firewall is stateful (which means that it keeps track of TCP-states) and the fact that traffic in one direction goes via the firewall (from 192.168.1.100 to 192.168.2.200) but the traffic in the other direction goes direct makes the firewall going bananas. If the traffic initiates from the 1-network all is fine since the firewall allowes the initial packet and never sees the return traffic. It might build a quite large list of incomplete sessions that eventually will time out, but it will work:
hairpin3 300x293 Cisco ASA hairpinning
However, when the traffic is initiated from the 2-network the first packet that will be seen by the firewall is the second (SYN-ACK) which is not very appreciated.
hairpin4 300x284 Cisco ASA hairpinning
Beginning with version 8.2 there is a solution for this in the “tcp state bypass”-functionality. By using this you can with MPF (modular policy framework) specify traffic that should not be handled stateful. Here is an example of doing that.
Genereally my solution to hairpinning- and/or assymetric routing-issues is to avoid it. If you have one or more routers in your topology it is probably a good idea to use that router as the default gateway instead and only direct traffic to the firewall that should pass it. The drawback from this is that internet-traffic needs to go via the router instead. However this seldom causes any trouble since the router is not stateful and has no issues with doing hairpinning.
hairpin5 300x281 Cisco ASA hairpinning
Another solution if you need to pass WAN-traffic thru the firewall (for security reasons) can be to put the WAN-router(s) on a dedicated firewall interface.
hairpin6 300x274 Cisco ASA hairpinning

Conclusion

Hairpinning of traffic in Cisco Pix/ASA and problems with assymetric routing can be a pain. There are workarounds but mostly assymetric issues are symptoms of bad network design rather than configuration issues.
There are more dimensions of hairpinning than just internal traffic turning around to a WAN-router. Such a common example is U-turning of VPN-traffic, for example traffic from an VPN-client going via the firewall out to internet or into another vpn-tunnel. Or spoke-hub-spoke VPN-traffic. This type of traffic seldom gives routing or assymetric issues but is more a matter of defining proxy ACL:s for vpn-traffic and not doing NAT on that traffic.

  • http://blog.fuelip.com Philip

    Fabulous!

    Very good explanation. I was lost with the use of the command:
    same-security-traffic permit intra-interface

    My issue was exactly as described with ASA 5505 running 7.2(3).

    Articles I stumbled across prior did not make it clear that I had to add static inside lines:
    static (inside,inside) 192.168.100.0 192.168.100.0 netmask 255.255.255.0
    static (inside,inside) 192.168.60.0 192.168.60.0 netmask 255.255.255.0

    I also forgot to ensure the ACL allowed both directions:
    access-list acl_inside extended permit ip 192.168.2.0 255.255.255.0 192.168.1.0 255.255.255.0
    access-list acl_inside extended permit ip 192.168.1.0 255.255.255.0 192.168.2.0 255.255.255.0

    Thanks again!!!

  • http://www.peapod.net David Sundstrom

    Excellent article. Clearly explained the exact problem I was having with a OpenVPN server inside my network. A static route on the ASA was set to forward VPN subnet traffic back out of the inside interface to the VPN server. Thank you so much for also covering the NAT and asymmetric routing issues. Unfortunately I can’t avoid it as I have no other “real” router to use as the inside default gateway. I’ll have to figure out how to make the hairpinning stateless in the firewall.

  • Igorh

    Holy S…T! You ARE AWESOME! I had the same issue as Philip. Thank you! Igor

  • Fran

    I have an ASA with version 8.3. This worked perfect for my TCP traffic. However, I can’t pass other kinds of traffic as there is only ’TCP state-bypass’ functionality for the ASA. 

    When I ping, the replies are rejected because the ASA doesn’t see the incoming requests. It only sees the reply and drops the packet because there is no initial session.

    Any clue on how to bypass other non-TCP traffic?

  • Jgilmour

    Wow. that explains why the NATs are weird as hell to me. Thanks for clearing this up. Coming from a 7.x environment to a 9.x environment had me scratching my head! 

Previous post:

Next post: