I have recently implemented a few Cisco Ironport WSA-solutions. When doing a follow-up after the implementation, the customer usually reacts with “Oh… WSA? We forgot about that. It probably works…”

But what difference does it make? If the customer forgets about their web proxy, what good does it do? Lets have a look at an implementation…

I asked one of our customers for permission to peek into their WSA for the purpose of this blog post. This customer has a few hundred users and is a fairly traditional type of user with mostly office users, each with a personal computer. This customer doesnt limit web browsing, except for filtering out access to known obviously bad web categories like child porn. Except for that, free access to the Web.

Fig1: General Statistics

The first thing to look at is an overview of web activity above. The average web traffic an business day is roughly one million is a working day consists of one million web requests. A web page contains several objects (images, scripts) where each object needs to be requested individually. In this implementation the clients generates 1 million transactions (requests) per day, or 20 million transactions per month.

But what is the content of the requested material? If we look at But WHAT users to surf? If you then look at the purity of operations as it starts to get interesting for real!

Fig2: Purity

Here you can see that just over 10,000 (10.6K) transactions have been stopped this month because of URL category! That is, such as child porn! There are objects (pages, images, etc.) that the user consciously or unconsciously sought but that the system has already been blocked at the access-trial because the source is known and undesirable.

One can also see that almost 3,000 (2.797) object has been blocked due to malware detection. Remember that the WSA scans all through traffic for known viruses, scripts, or other type of malware. The source category has been approved or unknown the WSA have downloaded content. But when checking the contents, they have discovered something unwanted. This little fella has thus stopped nearly 3,000 viruses in the past month!

Overall, 99.8% of web traffic this month has been “clean”. 0.2% may seem to be disappearing bit, but it is still almost 34 000 (33.8K) potentionella threat that was blocked already at the front door!

If you want more detailed information about the type of threat blocked, you can obviously get it also:

Fig3: Malware

With the help of the dynamic Sender Base system scored all websites on the internet. Based on a number of factors such as known virus outbreak or the credibility of a domain, each site a web reputation score from -10 to +10. WSA is configured to always block the sources with the lowest score and always allow the web site with the highest score. But how does this when in reality?

Fig4: Web Reputation

Here we can see that nearly 10,000 transactions in the last month blocked because of Web reputation.

The conclusion I draw every time I look at this type of reporting is that the WSA is blocking lots of web traffic in the covert, and it’s surprisingly rare that users react to the IT department because they can not browse to a specific site. It may be that the user deliberately tries to make stupid mistakes on the internet, but my experience and absolute conviction is that it almost always is something that happens unconsciously. A link to an email or on facebook that look “nice”, but takes the user to a  malware site in some obscure corner of the Internet.

Key figures for this particular device, a typical month “at work”:

• The number of transactions: 20.4 million pieces.
• The number of blocked transactions: 33 800 pcs.
• The number of blocked Malwares / viruses: 2797 pcs, or one every 3 minutes during business hours!
• Dare you not to check the content of your web traffic?

The problem

have you ever had an open TAC case with Cisco, just waiting for them to provide either a solution or some other kind of feedback, and all that happens is that the TAC engineer sends you an email telling you that they “have work in progress” or something else not-making-the-case-evolve?

If so, I guess you have seen that the moment the engineer sends you an email, you also get a case update email telling you that the case has changed status to “customer pending”.

And that is a bit evil. I am pretty sure that more often than not, the reason for the engineer to send that email to you is not to tell you something, but to to actually change the case status. I have a feeling that the engineers effeciency is measured in how long the case is “Cisco pending” and as soon as the case is put over to the customer side, it is “all cool”. just like throwing a burning ball between two perssons. Or like a chess-clock that measure the time spent on each side.

The solution

The best way to handle this is to get even with their own weapons. Last week I had a mail dialogue with TAC that looked like this:

TAC: we are working on the information You sent. we will get back to you tomorrow.
[case status: Customer pending]

Me: thank you very much, I appretiate it.
[case status: customer updated -> Cisco pending]

TAC: you are welcome. have a nice day.
[case status: Customer pending]

Me: you too…
[case status: customer updated -> Cisco pending]

TAC: thank you very much!
[case status: Customer pending]

Me: please do not answer this email, since it changes the status of the case to “Customer pending”, which does NOT reflect the current situation.
[case status: customer updated -> Cisco pending]

I won!!!

When purging and cleaning ancient posts I found this post where I wished everyone a Happy New 2011. And I felt that it was time for an update.

So, what happened during 2011 – did I become a Cisco CCIE Security? The short answer is: No.

In february 2011 my written CCIE Security exam expired. Shortly after that my CCNA/CCNP/CCSP/whatever certifications also was about to expire, and to prevent that from happen I passed the CCIE Security Written once more. So, that means that I have another 18 (like 12 from now) months to do another Lab attempt.

During 2011 there was no way that I could find enough time to study for the lab. Primary of course because of the general work load, but also was my schedule filled with cool projects. Not only have I continued my journey to teach (I have made  my own study material on which 2 different Cisco ASA-workshops were based), I have also done a lot of implementations of Cisco ACS5 and 802.1x, and lately a few ISE-implementations as well.

So, will I ever get that CCIE number? I dont know, but I will continue to try. I have recently purchased the “Ultimate CCIE Security Self Paced bundle” from INE as a complement to the material I already have from IPExpert. I find a few hours every now and then and try to focus to gain the speed/accuracy needed for the dreaded exam.

Stay tuned, I´ll be back.

/Jimmy

Every single decent Cisco-device on earth has the ability to make an CLI-user jump to another device with telnet or ssh. Except the ASA. I really wish that this feature could be added. Right now I am troubleshooting a firewall and from where I am right now the only way in is to SSH to the ASA. I can do whatever I want inside the firewall from my SSH-window, but I need to access a router inside of that firewall, and if this feature wasn´t missing i could simply run “ssh ip-address” to jump to the switch´s CLI.

Am I the last CLI-.guy on this planet? Please, Cisco?

Update: Greg Ferro wrote an reply on this and here are my comments:

This could be divided into several different questions.

1) Should we use SSH to manage the firewall? In my opinion CLI is superior to GUI for most tasks. There are exceptions, but for daily maintenance I prefer CLI for several reasons.  The alternative ASDM-GUI is equally safe/secure because both SSH and ASDM uses encrypted transports and the authentication-part can be configured equally for both entrance-types.

2) From where should we allow maintance of the firewall? Of course the most obvious answer to this is “from somewhere inside, but not from internet”. Sure, I agree. And you SHOULD lock down from which networks/hosts/directions management of the firewall should be enabled, and you SHOULD lock it down as tight as possible. 

But what if you NEED to manage your firewall “from internet”? In most implementations there is some kind of fallback needed so that the administrator can reach the network from abroad and do changes. This can be done in a ton of ways: VPN-client, SSL-portal, Citrix, you name it. The common thing with all these access ways is that they must be enabled “from anywhere”. What´s the point of allowing vpn-client in if you must be at a specific location (from a specific IP) to connect your vpn-client? Or Citrix-session? So this must be enabled from anywhere.

So there are 2 ways to make this “from anywere”-connection secure:

1) It is encrypted. VPN-client-traffic is encrypted. The Citrix access-gateway traffic is encrypted, the VPN-portal is encrypted. And you know what? SSH is encrypted. 

2) Authentication is safe enough. Validation of user rights can be done in a number of ways. Most common is of course username/password, but you can any other method available, from soft tokens and hard tokens to biometry or certificates. And you know what? All these authentication methods can be done for both VPN-clients, all other access method mentioned above, as well as for SSH-traffic.

So, what is it that makes people (not only you Greg ) so stubornly convinced that SSH-access to the firewall should be avoided? I can see no differences in security between SSH and other access methods.

And a final note: the original post was about SSH:ing FROM the device, not to. Following my dialogue with myself above I come to the conclusion that you CAN allow ssh into the device. Given that, what is so unsafe about giving someone that you trust, using a secure connection, the ability to reach the network behind the device? After all, this user has already God access to the firewall and could alter any configuration in the firewall.

In my previous post I successfully made ASA-generated traffic go into an VPN-tunnel. The catch with that was that the traffic (in my case: radius) was sources from the interface closest to the destination (outside) and I had to add that traffic to my crypto access-list to make it into the tunnel.

This case inducted an discussion on my favorite ASA mailing-list OSL and with good help from Tyson and the rest of the guys there I understood what I describes below.

Basic setup:


interface Vlan1
nameif inside
security-level 100
ip address 10.10.10.1 255.255.255.0
!
interface Vlan2
nameif outside
security-level 0
ip address 1.2.3.4 255.255.255.0
!
interface Ethernet0/0
switchport access vlan 2
!
interface Ethernet0/1
!
route outside 0.0.0.0 0.0.0.0 1.2.3.1 1
!
aaa-server RAD protocol radius
aaa-server RAD (inside) host 5.6.7.8
key cisco
!


If I wanna talk to the outside radius-server using my outside ip-address I would simply change the “aaa-server RAD (inside) host 5.6.7.8″ above to “aaa-server RAD (outside) host 5.6.7.8″. That is what I did in the previous post and it works. In that post I also prooved that the above config doesn´t work. If the radius-server is on one interface (in my case outside) and the radius-definition points to another interface (inside) there will be no outbound radius traffic generated. Let´s see it again:
ciscoasa(config)#capture inside type raw-data interface inside
ciscoasa(config)#capture outside type raw-data interface outside
ciscoasa(config)#
ciscoasa(config)#test aaa-server authen RAD host 5.6.7.8 user user pass pass
INFO: Attempting Authentication test to IP address <5.6.7.8> (timeout: 12 seconds)
ERROR: Authentication Server not responding: No error
ciscoasa(config)#
ciscoasa(config)# sh capture inside

2 packets captured

1: 23:02:38.662838 802.1Q vlan#2 P0 192.168.2.10.138 > 192.168.2.255.138: udp 201
2: 23:11:22.075618 802.1Q vlan#2 P0 192.168.2.10.138 > 192.168.2.255.138: udp 216
2 packets shown
ciscoasa(config)#

But there is a solution! (Thanks OSL!) And the solution is within the “management-access” command. This is what is written in the configuration guide about the command:

Managing the Security Appliance on a Different Interface from the VPN Tunnel Termination Interface

If your IPSec VPN tunnel terminates on one interface, but you want to manage the adaptive security appliance by accessing a different interface, then enter the following command:

hostname(config)# management access management_interface

where management_interface specifies the name of the management interface you want to access when entering the security appliance from another interface. For example, if you enter the adaptive security appliance from the outside interface, this command lets you connect to the inside interface using Telnet; or you can ping the inside interface when entering from the outside interface.

You can define only one management-access interface.

So, what has this to do with radius-packets? The undocumented secret here is that this command is also used to define a source-interface for outbound packets, for example radius-dito. Look. We add this command:

ciscoasa(config)# management-access inside
ciscoasa(config)#

Next we reset our capture buffers:

ciscoasa(config)# clear capture inside
ciscoasa(config)# clear capture outside
ciscoasa(config)#


…and generates radius-packets…


ciscoasa(config)# test aaa-server authen RAD host 5.6.7.8 user user pass pass
INFO: Attempting Authentication test to IP address <5.6.7.8> (timeout: 12 seconds)
ERROR: Authentication Server not responding: No error
ciscoasa(config)#

Please ignore the fact that there is no answer. There is simply no radius-server in this lab…But, what happened in our captures.

ciscoasa(config)# sh capture inside

1: 23:49:06.205433 802.1Q vlan#2 P0 10.10.10.1.1025 > 5.6.7.8.1645: udp 62
2: 23:50:39.478994 802.1Q vlan#2 P0 192.168.2.10.138 > 192.168.2.255.138: udp 201
2 packets shown
ciscoasa(config)#


Hey! Look at that packet, #1 on outside! It is sources from out inside ip, destined to our radius-server on outside, and sent out on our outside interface. And it is a radius-packet (udp 1645). Cool!

Conclusion: With the management-access interface you can select the source ip for packets generated from the ASA, for example radius.

So we have 3 different parameters for this traffic that controls the source address and/or destination interface:

1. Routing-entry. In our example 5.6.7.8 is beyond another router and we have an outbound default route. Without that the device would never know in which direction to send the traffic.
2. The interface-relation in the aaa-server-command. See below.
3. The “management-interface”-command that can be used to configure the source ip.

But how about #2. That interface-definition bothered me already in my last post. Why does it exist?

It surely isn´t used to define the source interface/address because above I proove that it is the addition of the “management-access”-command that makes all the differ. Before adding that there was no packets sent out on outside when the radius-server was defined as “(inside)”.

And at the same time, it is not being used to define the outbound interface. This is being done with the routing-table. And as we see above stating (“inside”) doesn´t make the packet go out on interface inside.

So, my officially question to Cisco is: Why is there an mandatory parameter to the aaa-server command that makes me define “the name of the network interface where the designated AAA server is accessed“?

Recently I got an request from a Cisco ASA customer who wanted to authenticate VPN-users with a remote Radius-server. Using Radius is piece of cake, but those of us that have been working with Cisco Pix/ASA for a while know that traffic to/from the box is no nearly treated the same way as traffic going thru the box. And this customer wanted to use a Radius-server via an Lan2Lan-tunnel that terminates in the same ASA-box.

So. Does it work? First of all I built a small little lab with 2 ASA:s connected back to back and an Lan2Lan-tunnel connecting the both inside networks. Plain vanilla.

Topology:

Relevant parts of ASA1 config:

ASA Version 8.2(1)
!
hostname ASA1
!
interface Vlan1
nameif inside
security-level 100
ip address 192.168.1.234 255.255.255.0
!
interface Vlan2
nameif outside
security-level 0
ip address 192.168.169.1 255.255.255.0
!
interface Ethernet0/0
switchport access vlan 2
!
interface Ethernet0/1
!
access-list crypto-acl extended permit ip 192.168.1.0 255.255.255.0 192.168.2.0 255.255.255.0
access-list OUTSIDE extended permit ip any any
no nat-traversal
route outside 0.0.0.0 0.0.0.0 192.168.169.2 1
!
crypto ipsec transform-set tset esp-aes esp-sha-hmac
!
crypto map cmap-outside 10 match address crypto-acl
crypto map cmap-outside 10 set peer 192.168.169.2
crypto map cmap-outside 10 set transform-set tset
crypto map cmap-outside interface outside
crypto isakmp enable outside
crypto isakmp policy 10
authentication pre-share
encryption aes
hash sha
group 5
lifetime 86400
tunnel-group 192.168.169.2 type ipsec-l2l
tunnel-group 192.168.169.2 ipsec-attributes
pre-shared-key cisco
!


And ASA2:
ASA Version 8.2(1)
!
hostname ASA2
!
interface Vlan1
nameif inside
security-level 100
ip address 192.168.2.1 255.255.255.0
!
interface Vlan2
nameif outside
security-level 0
ip address 192.168.169.2 255.255.255.0
!
interface Ethernet0/0
switchport access vlan 2
!
interface Ethernet0/1
!
access-list crypto-acl extended permit ip 192.168.2.0 255.255.255.0 192.168.1.0 255.255.255.0
!
crypto ipsec transform-set tset esp-aes esp-sha-hmac
!
crypto map cmap-outside 10 match address crypto-acl
crypto map cmap-outside 10 set peer 192.168.169.1
crypto map cmap-outside 10 set transform-set tset
crypto map cmap-outside interface outside
crypto isakmp enable outside
!
crypto isakmp policy 10
authentication pre-share
encryption aes
hash sha
group 5
lifetime 86400
!
tunnel-group 192.168.169.1 type ipsec-l2l
tunnel-group 192.168.169.1 ipsec-attributes
pre-shared-key cisco


So, this configuration connects the inside networks over the unsecure network 192.168.169.0/24. But what if ASA1 wanna talk radius to the ACS-server 192.168.2.10 beyond ASA2?

First of all, the way of configuring an aaa-server in ASA is in my opinion a bit confusing. It´s all about pointing out the server ip-address together with a radius key. But if we look at the syntax for defining a radius-server we see that we also need to define an interface. Whatfor?


ASA1(config)# aaa-server GROUPTAG protocol radius
ASA1(config-aaa-server-group)# aaa-server GROUPTAG ?
configure mode commands/options:
( Open parenthesis for the name of the network interface
where the designated AAA server is accessed
deadtime Specify the amount of time that will elapse between the
disabling of the last server in the group and the
subsequent re-enabling of all servers
host Enter this keyword to specify the IP address for the
server
max-failed-attempts Specify the maximum number of failures that will be
allowed for any server in the group before that server
is deactivated
protocol Enter the protocol for a AAA server group
ASA1(config-aaa-server-group)# aaa-server GROUPTAG


So, we need to specify an interface. The reason that I think this is a bit weird is that there should already be a route in the routing-table for our radius-server 192.168.2.10. If nothing else, there is probably an default route, and in our case there is definately one. So why stating that “in order to reach 192.168.2.10 go via interface outside” in the radius-definition? I have no idea. A few moment I thought of this not as a way to specify outbound interface but source interface. What if I wanna send the radius packets to outside (according to routing table, with or without an vpn-tunnel) but use the inside interface ip as source? That would be cool, because then I didn´t have to add anything in the crypto acl (see below). This is still untested, but when we look at the syntax help above it certanly states “for the name of the network interface where the designated AAA server is accessed”, which of course is outside in my example.

So, lets add the radius definition. And what else? We need to add traffic to the crypto acl for making it go into the vpn tunnel. And since it is traffic from the ASA1 outside interface to the host 192.18.2.10 behind ASA2, that is what we add:

ASA1 – addition:

access-list crypto-acl extended permit ip host 192.168.169.1 host 192.168.2.10
aaa-server RAD protocol radius
aaa-server RAD (outside) host 192.168.2.10
key cisco


and ASA2 – addition:

access-list crypto-acl extended permit ip host 192.168.2.10 host 192.168.169.1


And, as a proof that this works we use the “test aaa”-command to generate an radius authentication request from ASA1 to the Radius-server.


ASA1(config)# test aaa authentication RAD host 192.168.2.10 username user pass$
INFO: Attempting Authentication test to IP address <192.168.2.10> (timeout: 12 seconds)
INFO: Authentication Successful
ASA1(config)#
ASA1(config)#
ASA1(config)# sh crypto ipsec sa
interface: outside
Crypto map tag: cmap-outside, seq num: 10, local addr: 192.168.169.1

access-list crypto-acl permit ip host 192.168.169.1 host 192.168.2.10
local ident (addr/mask/prot/port): (192.168.169.1/255.255.255.255/0/0)
remote ident (addr/mask/prot/port): (192.168.2.10/255.255.255.255/0/0)
current_peer: 192.168.169.2

#pkts encaps: 1, #pkts encrypt: 1, #pkts digest: 1
#pkts decaps: 1, #pkts decrypt: 1, #pkts verify: 1
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 1, #pkts comp failed: 0, #pkts decomp failed: 0
#pre-frag successes: 0, #pre-frag failures: 0, #fragments created: 0
#PMTUs sent: 0, #PMTUs rcvd: 0, #decapsulated frgs needing reassembly: 0
#send errors: 0, #recv errors: 0

local crypto endpt.: 192.168.169.1, remote crypto endpt.: 192.168.169.2

inbound esp sas:
spi: 0x53870178 (1401356664)
transform: esp-aes esp-sha-hmac no compression
in use settings ={L2L, Tunnel, }
slot: 0, conn_id: 24576, crypto-map: cmap-outside
sa timing: remaining key lifetime (kB/sec): (3914999/28682)
IV size: 16 bytes
replay detection support: Y
Anti replay bitmap:
0x00000000 0x00000003
outbound esp sas:
spi: 0x28DAEB5B (685435739)
transform: esp-aes esp-sha-hmac no compression
in use settings ={L2L, Tunnel, }
slot: 0, conn_id: 24576, crypto-map: cmap-outside
sa timing: remaining key lifetime (kB/sec): (3914999/28682)
IV size: 16 bytes
replay detection support: Y
Anti replay bitmap:
0x00000000 0x00000001


So, what happens if we follow my idea that the interface-relation within the radius-configuration was not a way to define outbound interface but instead a way to define source address for our radius packets. Well, let´s try. We have already a working tunnel between our LAN:s, so if we reconfigure ASA1 to use inside instead, that traffic (from 192.168.1.234 to 192.168.2.10) should already be included in our proxy acl so nothing else should be needed. Lets try:


ASA1(config)# sh run aaa-server
aaa-server RAD protocol radius
aaa-server RAD (outside) host 192.168.2.10
key cisco
aaa-server GROUPTAG protocol radius
ASA1(config)#
ASA1(config)# clear configure aaa-server RAD
ASA1(config)#
ASA1(config)# aaa-server RAD proto radius
ASA1(config-aaa-server-group)# aaa-server RAD (inside) host 192.168.2.10
ASA1(config-aaa-server-host)# key cisco
ASA1(config-aaa-server-host)#
ASA1(config-aaa-server-host)# end
ASA1#


ok, let´s give it a shot!
ASA1# test aaa authentication RAD host 192.168.2.10 username user password cis$
INFO: Attempting Authentication test to IP address <192.168.2.10> (timeout: 12 seconds)
ERROR: Authentication Server not responding: No error
ASA1#
ASA1#
ASA1# test aaa authentication RAD host 192.168.2.10 username user password cis$
INFO: Attempting Authentication test to IP address <192.168.2.10> (timeout: 12 seconds)
ERROR: Authentication Server not responding: No error
ASA1#
ASA1# test aaa authentication RAD host 192.168.2.10 username user password cis$
INFO: Attempting Authentication test to IP address <192.168.2.10> (timeout: 12 seconds)
ERROR: Authentication Server not responding: No error
ASA1#


It obviously doesnt work. But what happened?
ASA1# sh crypto ipsec sa
There are no ipsec sas
ASA1#

Ok, so no tunnels triggered. But this really mean that the radius packets were sent to inside instead of outside? Lets capture packets!
ASA1# capture OUTSIDE interface outside
ASA1# capture INSIDE interface inside
ASA1#
ASA1# test aaa authentication RAD host 192.168.2.10 username user password cis$
INFO: Attempting Authentication test to IP address <192.168.2.10> (timeout: 12 seconds)
ERROR: Authentication Server not responding: No error
ASA1#
ASA1#
ASA1# sh capture OUTSIDE

0 packet captured

0 packet shown
ASA1# sh capture INSIDE

17 packets captured

1: 03:45:53.705194 802.1Q vlan#1 P0 arp who-has 192.168.1.213 tell 192.168.1.1
2: 03:45:58.704019 802.1Q vlan#1 P0 arp who-has 192.168.1.213 tell 192.168.1.1
3: 03:46:03.702844 802.1Q vlan#1 P0 arp who-has 192.168.1.213 tell 192.168.1.1
4: 03:46:03.968089 802.1Q vlan#1 P0 arp who-has 192.168.1.213 tell 192.168.1.1
5: 03:46:05.702340 802.1Q vlan#1 P0 arp who-has 192.168.1.213 tell 192.168.1.1
6: 03:46:06.702112 802.1Q vlan#1 P0 arp who-has 192.168.1.213 tell 192.168.1.1
7: 03:46:08.335904 802.1Q vlan#1 P0 192.168.1.203.57621 > 255.255.255.255.57621: udp 44
8: 03:46:10.277665 802.1Q vlan#1 P0 192.168.1.72.17500 > 255.255.255.255.17500: udp 176
9: 03:46:10.278244 802.1Q vlan#1 P0 192.168.1.72.17500 > 192.168.1.255.17500: udp 176
10: 03:46:10.701150 802.1Q vlan#1 P0 arp who-has 192.168.1.213 tell 192.168.1.1
11: 03:46:14.292892 802.1Q vlan#1 P0 192.168.1.107.138 > 192.168.1.255.138: udp 201
12: 03:46:15.699976 802.1Q vlan#1 P0 arp who-has 192.168.1.213 tell 192.168.1.1
13: 03:46:15.806858 802.1Q vlan#1 P0 192.168.1.73.138 > 192.168.1.255.138: udp 201
14: 03:46:17.743522 802.1Q vlan#1 P0 192.168.1.203.17500 > 255.255.255.255.17500: udp 172
15: 03:46:17.745795 802.1Q vlan#1 P0 192.168.1.203.17500 > 255.255.255.255.17500: udp 172
16: 03:46:17.746146 802.1Q vlan#1 P0 192.168.1.203.17500 > 192.168.1.255.17500: udp 172
17: 03:46:17.746512 802.1Q vlan#1 P0 192.168.1.203.17500 > 255.255.255.255.17500: udp 172
17 packets shown
ASA1#
ASA1#


What we see above is that there is absolutely no packets at all seen on ASA1 outside interface. On inside interface we see various packets (because that is my home network), but no radius packets.

So, what are our conclusions?

• Traffic generated from the ASA can very well be included in our Lan2Lan-tunnel so that for example the ASA can have a secure connection to an remote authentication server.
• The interface-definition in the aaa-server command has nothing to do with source addresses. As a matter of fact, you cannot configure a source interface/address for radius-traffic the way you can do in an IOS-router.
• The interface-definition shouldnt really be needed. The way to the remote server is pointed out by the routing table. The interface-definition must point the same direction as the routing-table, otherwise the ASA won´t know where to send the packets.
• I was wrong
• I can tell the customer that we can do radius over vpn.

am I missing something here? Please don´t hesitate to comment!

Update: I was missing something. Look at my update post…

/Jimmy

 A while ago I did a migration of Cisco ACS from v4.x to v5.x. As you probably know there is a migration tool that can be used to migrate some (but not all!) of the configuration-details from 4.x into 5.x. What happened to me was very strange and I would like to share it with you…

What I did, and what is also the recommended way of doing this was to:
1) Make a complete database backup of the existing 4.x-installation.
2) Install a new ACS 4.x for windows, running on a virtual (temporary) Win2003 server.
3) Restore the backup to this temporary migration isntance.
4) Run the script locally on the migration instance.

The script will connect to the old (local) database and the new 5.x-appliance and migrate the configuration database.

However, what happened to me was that when the script started I got this error message:

Fatal Error !! - cannot connect to ACS 4.x DB !!

I struggled with this for a while, trying different credentials and other settings without success. When opening a TAC case I got an instant solution to the problem:

This happens if you RDP or VNC to the migration server without connecting to the console. Since this server was running under Vmware ESX i could do either of:
1) use the console from vmware vsphere client
2) Use the /admin parameter of mstsc when RDP:ing to the migration server.

Who could have guessed? I hope that this blog post will be indexed and found when someone else google this error message, like I tried to do. . Also, this is filed as bugId CSCsr62965.

Below is the full content of the Cisco TAC case, for reference.

Problem Details:
From the existing ACS 4.x appliance (10.0.0.16) I have done a complete backup. This
has been restored to a ACS 4.1 (trial) for Windows (10.0.0.19). On this machine I run
the migration script. The goal is to migrate the configuration to the new ACS 5.1
appliance (10.0.0.18).

When running the script it saids ” Fatal Error !! – cannot connect to ACS 4.x DB !!”

This is the content of migration.log after a unsuccessfull migration attempt:

06-28-2010 09:08:21 JavaUtils.isAttachmentSupported(JavaUtils.java:1308) WARN – Unable to
find required classes (javax.activation.DataHandler and
javax.mail.internet.MimeMultipart). Attachment support is disabled.
06-28-2010 09:08:34 ACS4Connector.checkDBConnectivity(ACS4Connector.java:137)FATAL -
Fatal Error !! – cannot connect to ACS 4.x DB !!
java.sql.SQLException: [Sybase][ODBC Driver][Adaptive Server Anywhere]Database server not
found
at ianywhere.ml.jdbcodbc.IDriver.makeODBCConnection(Native Method)
at ianywhere.ml.jdbcodbc.IDriver.connect(IDriver.java:354)
at java.sql.DriverManager.getConnection(Unknown Source)
at java.sql.DriverManager.getConnection(Unknown Source)
at com.cisco.nm.acs.mgmt.migration.ACS4Connector.getConnecter(ACS4Connector.java:66)
at
com.cisco.nm.acs.mgmt.migration.ACS4Connector.checkDBConnectivity(ACS4Connector.java:133)
at
com.cisco.nm.acs.mgmt.migration.MigrationApplicationCLI.runExport(MigrationApplicationCLI.
java:605)
at
com.cisco.nm.acs.mgmt.migration.MigrationApplicationCLI.main(MigrationApplicationCLI.java:
266)

—————————
Microsoft Windows [Version 5.2.3790]
(C) Copyright 1985-2003 Microsoft Corp.

C:Documents and SettingsAdministrator>cd migrationmigration

C:migrationmigration>cd bin

C:migrationmigrationbin>
C:migrationmigrationbin>
C:migrationmigrationbin>ipconfig

Windows IP Configuration

Ethernet adapter Local Area Connection:

Connection-specific DNS Suffix . :
IP Address. . . . . . . . . . . . : 10.0.0.19
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . : 10.0.0.1

C:migrationmigrationbin>ping 10.0.0.18

Pinging 10.0.0.18 with 32 bytes of data:

Reply from 10.0.0.18: bytes=32 time<1ms TTL=64

Ping statistics for 10.0.0.18:
Packets: Sent = 1, Received = 1, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms
Control-C
^C
C:migrationmigrationbin>del migration.log

C:migrationmigrationbin>migration.bat
**************************************************
Copyright (c) 2008, 2009 Cisco Systems, Inc.
All rights reserved.
**************************************************

This utility migrates data from ACS 4.x to ACS 5. You can migrate directly from
the following ACS versions:

1. ACS 4.1.1.24
2. ACS 4.1.4
3. ACS 4.2.0.124
4. ACS 4.2.1

The migration utility analyzes the ACS 4.x data, exports the data from ACS 4.x t
hat can be migrated automatically, and imports the data into ACS 5.
You can manually consolidate and resolve data according to the analysis report,
before the import stage, to maximize the amount of data that the utility can mig
rate.
After migration, use the imported data to recreate your policies in ACS 5.
——————————————————————————–
————————————–

Use saved user defaults?[yes]
no
Make sure that the database is running.
Enter ACS 5 IP address or hostname:[10.0.0.18]

Enter ACS 5 Administrator username:[ACSAdmin]

Enter ACS 5 password:[******]

Change user preferences?[no]
yes

User Groups
——————————————————————————–
————————————–
Existing user groups will be migrated to the Identity Group.
Enter the name of new Root:[Migrated_Group]

Network Device Groups
——————————————————————————–
————————————–
Existing network device groups will be migrated to the Network Device Group.
Enter the name of new Root:[Migrated_NDGs]

Consolidation Prefix
——————————————————————————–
————————————–
Identical objects found are consolidated into one.
Enter the prefix to be added to such a consolidated object:[]

Users
——————————————————————————–
————————————–
ACS 5 supports authentication for internal users against the internal database o
nly.
ACS 4.x users that were configured to use an external database for authenticatio
n are migrated with a default authentication password.
Specify the default password.[******]

Disabled Group Users
——————————————————————————–
————————————–
ACS 4.x users and hosts that are associated with disabled group will be migrated
as disabled:[yes]

Configure these users as disabled in ACS 5, or ask for a change of password on f
irst access by the user to ACS 5.
Select the option:
1 – DisableExternalUser
2 – SetPasswordChange
Selected option:[2]

Network Device
——————————————————————————–
————————————–

TACACS+ and RADIUS network devices with same IP will be unified.
Select the name to be used for unified devices.
1 – RADIUSName
2 – TACACSName
3 – CombinedName
Selected option:[3]

DACL name construction
——————————————————————————–
————————————–

Existing downloadable ACL will be migrated.
Select the name to be used for the migrated DACL
1 – DaclName_AclName
2 – AclName
Selected option:[1]

Save user defaults?[yes]

Connecting to ACS5, please wait…

Enter ACS 4.x Server ID:
acs4
Warning: This server id was entered before.

Add server specific migration prefixes?[no]

Show full report also on screen?[yes]

——————————————————————————–
————————————–

Select the ACS 4.x Configuration groups to be migrated:[1]
1 – ALLObjects
2 – AllUsersObjects
3 – AllDevicesObjects
4 – SharedCommandSet
5 – SharedDACLObject
6 – MasterKeys
7 – SharedRACObjectWithVSA
——————————————————————————–
————————————–

——————————————————————————–
————————————–

The following object types will be extracted:
——————————————————————————–
————————————–

User Attributes
User Attribute Values
Network Device Groups
User Groups
Groups Shell Exec
Users Shell Exec
Users
Shared Command Sets
Groups Command Set
Users Command Set
Network Device
Shared Downloadable ACL
EAP FAST – Master Keys
MAB
VSA Vendor
VSA
RAC
——————————————————————————–
————————————–

Choose one of the following:
1 – AnalyzeAndExport
2 – Import
3 – CreateReportFiles
4 – Exit
——————————————————————————–
————————————–

1
Fatal Error !! – cannot connect to ACS 4.x DB !!
——————————————————————————–
————————————–

Choose one of the following:
1 – AnalyzeAndExport
2 – Import
3 – CreateReportFiles
4 – Exit
——————————————————————————–
————————————–

4
Would you like to migrate another ACS4.x server?[no]
no
C:migrationmigrationbin>

—————–

*** Service Request LOG 2010-06-28 08:40:25.0 GMT, FZILIOTT, Action Type: Email In ***

From: fziliott@cisco.com
Subject: SR 614750319 – * C3A – Problem running ACS4 -> ACS5 migration script

See detail note for email textFrom: xxxx@cisco.com
To: xxxx@xx.com
Cc: attach@cisco.com
Subject: SR 123456789 – * C3A – Problem running ACS4 -> ACS5 migration script

Hello Jimmy,

Thank you for contacting the Cisco TAC.
My name is Federico Ziliotto and I am the engineer assisting you for the
Service Request 614750319.

Looking at the notes, the current problem description regards the fact
that the migration utility for ACS 5.1 fails with the following
exception when trying to migrate a database from ACS 4.1:

JavaUtils.isAttachmentSupported(JavaUtils.java:1308) WARN – Unable to
find required classes (javax.activation.DataHandler and
javax.mail.internet.MimeMultipart). Attachment support is disabled.

ACS4Connector.checkDBConnectivity(ACS4Connector.java:137)FATAL – Fatal
Error !! – cannot connect to ACS 4.x DB !!

If needed, do not hesitate to update or modify my understanding at any time.

As a first action plan, I would like to collect some more details from
the following points:

1. Could you please confirm the full version of ACS for Windows, on
whose server you are running the migration tool?
This should be visible in the ACS web interface, on the home page, right
after logging in: 4.1.x.y

2. I would also like to please verify whether you are accessing the
Windows server via RDP or VNC, for example.
If so, I would suggest to please test running the migration tool while
being physically on the Windows server.
If this is a VMware machine, please use the console.

3. Should the issue still persists after having checked point #3, at the
end of the process, please also select the option “CreateReportFiles”
and then forward me the logs on the Windows server under

…migrationbinmigration.log
and
…migrationconfig

Please also take some time to document the business impact related to
the case, if relevant, so that I can focus on the issue accordingly.

For your convenience I have included my contact details below; feel free
to use them for any further questions.

Please make sure to always keep attach@cisco.com in the CC list so that
the SR is updated automatically, and delete the lines from the previous
emails when replying, in order to keep the case notes clear.

Best regards,

xxxxx

–
Cisco Systems – EU TAC hotline: +32 2704 5555
Technical Support worldwide Contacts and Home Page:

http://www.cisco.com/en/US/support/tsd_cisco_worldwide_contacts.html
http://www.cisco.com/go/tsdelivery
***
For corporate legal information go to:

http://www.cisco.com/web/about/doing_business/legal/cri/index.html
***

*** Service Request LOG 2010-06-28 09:47:19.0 GMT, JILAHBG, Action Type: Email In ***

From: xxxx@xx.com
Subject: RE: SR 614750319 – * C3A – Problem running ACS4 -> ACS5 migration script

See detail note for email textFrom: xxxx@xx.com
To: xxxx@cisco.com
Cc: attach@cisco.com
Subject: RE: SR 123456789 – * C3A – Problem running ACS4 -> ACS5 migration script

Hello

Running rdp to the console (start->Run->mstsc /admin) was successful. I am all done
with my migration now. Thanks a lot! The case can be closed.

Br Jimmy

My goal for 2010 was to nail that CCIE Security exam. I didn´t. During the first half of 2010 I spent almost all spare time studying and making practice labs. I did an attempt in Brussels in July but didnt make it. The goal then was to continue my studies asap after summer vacation, while it was still calm and quiet at work. But there was no calm at all. I have been busy doing consulting from august to the day before christmas. No studies whatsoever. So the next plan was to continue my studies in january when it is normally extremely calm at work. But now when I look in my schedule for 2011 I am already fully booked for january and first half of february.

So right now I have no idea when I will be able to get back to my ccie studies. I need to pass the lab before february 2012, otherwise I need to retake my written exam. On the flip side I must say that I have never had more challenging and fun missions at work as I will have the next months. Implementations of lager 802.1x-solutions as well as teaching for the first time ever. It will be a blast!

Merry christmas and a Happy New year to you all! /Jimmy

Most often we Cisco-guys uses radius or tacacs when we are about to do authentication of users. But did you know that doing authentication from VPN to a user-database in an Active Directory doesn´t require IAS, ACS or any third party software at all. In fact there are multiple ways in ASA to talk to AD built-in.

I have tried them in my home lab by using an ASA firewall and a Windows 2003 Server with Active Directory installed.

LDAP


aaa-server LDAP protocol ldap
aaa-server LDAP (outside) host 192.168.1.51
ldap-base-dn CN=Users,DC=kvistofta,DC=local
ldap-scope subtree
ldap-naming-attribute sAMAccountName
ldap-login-password *
ldap-login-dn CN=admin,CN=Users,DC=kvistofta,DC=local
server-type microsoft


Verification:

FW(config)# test aaa authen KERBEROS host 192.168.1.51 username vpnuser passwo$
INFO: Attempting Authentication test to IP address <192.168.1.51> (timeout: 12 seconds)
INFO: Authentication Successful
FW(config)# test aaa authen LDAP host 192.168.1.51 username vpnuser password 1$
INFO: Attempting Authenticatio
[75] Session Start
n test to IP address <192.168.1.51> (timeout: 12 seconds)
[75] New request Session, context 0xd5954260, reqType = 1
[75] Fiber started
[75] Creating LDAP context with uri=ldap://192.168.1.51:389
[75] Connect to LDAP server: ldap://192.168.1.51:389, status = Successful
[75] supportedLDAPVersion: value = 3
[75] supportedLDAPVersion: value = 2
[75] Binding as administrator
[75] Performing Simple authentication for admin to 192.168.1.51
[75] LDAP Search:
Base DN = [CN=Users,DC=kvistofta,DC=local]
Filter = [sAMAccountName=vpnuser]
Scope = [SUBTREE]
[75] User DN = [CN=vpnuser,CN=Users,DC=kvistofta,DC=local]
[75] Talking to Active Directory server 192.168.1.51
[75] Reading password policy for vpnuser, dn:CN=vpnuser,CN=Users,DC=kvistofta,DC=local
[75] Read bad password count 0
[75] Binding as user
[75] Performing Simple authentication for vpnuser to 192.168.1.51
[75] Processing LDAP response for user vpnuser
[75] Authentication successful for vpnuser to 192.168.1.51
[75] Retrieved User Attributes:
[75] objectClass: value = top
[75] objectClass: value = person
[75] objectClass: value = organizationalPerson
[75] objectClass: value = user
[75] cn: value = vpnuser
[75] givenName: value = vpnuser
[75] distinguishedName: value = CN=vpnuser,CN=Users,DC=kvistofta,DC=local
[75] instanceType: value = 4
[75] whenCreated: value = 20100706114926.0Z
[75] whenChanged: value = 20100706114926.0Z
[75] displayName: value = vpnuser
[75] uSNCreated: value = 13726
[75] uSNChanged: value = 13731
[75] name: value = vpnuser
[75] objectGUID: value = ..1....O.c.v....
[75] userAccountControl: value = 66048
[75] badPwdCount: value = 0
[75] codePage: value = 0
[75] countryCode: value = 0
[75] badPasswordTime: value = 0
[75] lastLogoff: value = 0
[75] lastLogon: value = 129228917453688826
[75] pwdLastSet: value = 129228905663476095
[75] primaryGroupID: value = 513
[75] objectSid: value = .............LP...r{..."S...
[75] accountExpires: value = 9223372036854775807
[75] logonCount: value = 5
[75] sAMAccountName: value = vpnuser
[75] sAMAccountType: value = 805306368
[75] userPrincipalName: value = vpnuser@kvistofta.local
[75] objectCategory: value = CN=Person,CN=Schema,CN=Configuration,DC=kvistofta,DC=local
[75] Fiber exit Tx=547 bytes Rx=2053 bytes, status=1
[75] Session End
INFO: Authentication Successful
FW(config)#


KERBEROS


aaa-server KERBEROS protocol kerberos
aaa-server KERBEROS (outside) host 192.168.1.51
kerberos-realm KVISTOFTA.LOCAL

Verification:

FW(config)# test aaa authen KERBEROS host 192.168.1.51 username vpnuser passwo$
INFO: Attempting Authentication test to IP address <192.168.1.51> (timeout: 12 seconds)
kip_lookup_by_sessID: kip with id 76l not found
Kerberos library reports: "Additional preauthentication required"
INFO: Authentication Successful
FW(config)#


NT Domain


aaa-server NT (outside) host 192.168.1.51
nt-auth-domain-controller kvistofta


Verification:

FW(config)# test aaa auth NT host 192.168.1.51 username vpnuser password 1qaz!$
INFO: Attempting Authentication test to IP address <192.168.1.51> (timeout: 12 seconds)
smb_iod_request :
smb_iod_process_message :
smb_iod_negotiate : iod_state = No connect
smb_iod_negotiate : tcreate
smb_iod_negotiate : bind
smb_iod_negotiate : tconnect
smb_iod_addrq :
smb_iod_sendrq : iod_state = transport active
smb_iod_waitrq :
smb_iod_removerq :
smb_iod_negotiate : completed
smb_iod_process_message :
smb_iod_thread : going to sleep for 2 secs 0 nsecs
smb_iod_process_message :
smb_iod_thread : going to sleep for 2 secs 0 nsecs
smb_iod_request :
smb_iod_process_message :
smb_iod_ssnsetup : iod_state = unknown stat(3)
smb_iod_addrq :
smb_iod_sendrq : iod_state = unknown stat(4)
smb_iod_waitrq :
smb_iod_removerq :
smb_iod_ssnsetup : completed
smb_iod_process_message :
smb_iod_thread : going to sleep for 2 secs 0 nsecs
smb_iod_process_message :
smb_iod_thread : going to sleep for 2 secs 0 nsecs
Connected to VPNUSER
smb_iod_request :
smb_iod_process_message :
smb_iod_addrq :
smb_iod_sendrq : iod_state = session established
smb_iod_waitrq :
smb_iod_removerq :
smb_iod_process_message :
smb_iod_thread : going to sleep for 2 secs 0 nsecs
smb_iod_request :
smb_iod_process_message :
INFO: Authentication Successful
FW(config)#


I recently got a question from a collegue regarding address translations in Cisco ASA. He wrote:

Got a question from a customer if you can do the following:

ok. Lets try. I put together a quick lab-setup. A 3-legged ASA with a one-legged router on each firewall-interface:


interface Vlan10
nameif outside
security-level 0
ip address 200.1.1.1 255.255.255.0
!
interface Vlan20
nameif inside
security-level 100
ip address 10.1.1.1 255.255.255.0
!
interface Vlan30
no forward interface Vlan20
nameif dmz
security-level 50
ip address 10.2.2.1 255.255.255.0
!


And the routers.

R1:

interface FastEthernet0
ip address 200.1.1.99 255.255.255.0
duplex auto
speed auto
end


R2:

interface FastEthernet0
ip address 10.1.1.99 255.255.255.0
duplex auto
speed auto
end


and R3:

interface FastEthernet0
ip address 10.2.2.2 255.255.255.0
duplex auto
speed auto
end


Ok. R1 is the outside host on internet. R2 is an inside host on our corporate network. R3 is this magical server on DMZ. In this example it is a high performance telnet server!


line vty 0 4
no login


First, make sure that this is reachable from internet. We do a static and allow the traffic on outside acl:


static (dmz,outside) 200.1.1.2 10.2.2.2 netmask 255.255.255.255
access-list OUTSIDE extended permit tcp any host 200.1.1.2 eq telnet


Verification. Telnet from R1 to public IP:

R1#telnet 200.1.1.2
Trying 200.1.1.2 ... Open
R3>


Great. Now we want to reach the DMZ server from inside. Since this is higher security level to lower and we dont have any acl on inside we dont have to care about open the traffic. But we want to use an OUTSIDE address as destination ip to reach a DMZ host. Lets try a static:


FW(config)# static (dmz,inside) 200.1.1.2 10.2.2.2


The command above seems weird, right? I agree. Someday when I have a lot of time I will explain the theory but for now, just trust me!

Verification:

R2#telnet 200.1.1.2
Trying 200.1.1.2 ... Open
R3>


Next step is to hide the source address of that telnet client on inside. Right now it is using its own source ip:

R3>en
Password:
R3#
R3#sh users
Line User Host(s) Idle Location
0 con 0 idle 00:08:09
* 6 vty 0 idle 00:00:00 10.1.1.99

R3#


So, how do we accomplish that? The easiest way is to use a policy nat-statement. We create an access-list which defines which traffic to translate. We then create a nat-statement with a nat-id of your choice and call the access-list. Finaly we define which global ip to use. (outside. Remember dmz is outside relative to inside since dmz has lower security-level)

access-list Inside2DMZ extended permit tcp 10.1.1.0 255.255.255.0 host 200.1.1.2 eq telnet
nat (inside) 1 access-list Inside2DMZ
global (dmz) 1 200.1.1.10


Verification:

R2#telnet 200.1.1.2
Trying 200.1.1.2 ... Open

R3>

Voila! So, the client is on a private ip network 10.1.1.0 and establish a connection to what he think is on outside, because it is an public/outside ip. The traffic passes the magic ASA and the server on DMZ believes that the client is on internet since it has a public/outside source ip.

Mission accomplished.