My goal for 2010 was to nail that CCIE Security exam. I didn´t. During the first half of 2010 I spent almost all spare time studying and making practice labs. I did an attempt in Brussels in July but didnt make it. The goal then was to continue my studies asap after summer vacation, while it was still calm and quiet at work. But there was no calm at all. I have been busy doing consulting from august to the day before christmas. No studies whatsoever. So the next plan was to continue my studies in january when it is normally extremely calm at work. But now when I look in my schedule for 2011 I am already fully booked for january and first half of february.

So right now I have no idea when I will be able to get back to my ccie studies. I need to pass the lab before february 2012, otherwise I need to retake my written exam. On the flip side I must say that I have never had more challenging and fun missions at work as I will have the next months. Implementations of lager 802.1x-solutions as well as teaching for the first time ever. It will be a blast!

Merry christmas and a Happy New year to you all! /Jimmy

I did not pass the test today.

I just left Brussels after my first take on the CCIE Security lab. So, what happened? I Showed up early, 7:40. The test was about to start 8.15 and I waited in the reception with the other candidates untill we were escorted to the lab room. The proctor was We had a lunch break that I can barely remember anything from and all of a sudden it was 5pm and time to leave.

I made a list of tasks and marked each item as it was configured. My sum total was 78p. (100 is max, 80 is passing score). I am not 100% sure that all checked tasks was correct so I expect a somewhat lower score. Score report will probably be available later tonight.

The first hour I did all preparations. Diagram, task list, lab reading and all that. After 1:30 I had the basic l2-l3 setup with ASAs and IPS. I got kinda stuck at the same point as so many times before: understanding the topology, get a feeling about what part of the network should work like what and where the boundaries are. Which addresses should be hidden and which should be universal routable. Doing a few mistakes with that costed me at least an hour to troubleshoot and fix what I killed by mistake.

The self confidence was way low when it was time for lunch. After having something to eat (a hamburger, but I honestly don’t remember what else was on that plate!) I decided to do some cherry-picking. Selecting and gaining the easies points in the work book always do miracles to your mind! I summarized the task list and found that I had cleared about 70p!

The last hour or 2 I just tried my best to get as many of the remaining points as possible.

When I left I had the following list of uncleared tasks:

* 2 individual tasks within the same technology. I definitely know what to study for the next attempt! I tried them both bud left them unfinished after I spent way too much time on them. These could have pushed me over the line!

* one task worth 5 (or so) points. I left it untouched because I realized how much work it would have taken. If I had more Time at the end I’d probably fixed it.

* one 3p task that I immediately saw that I had no idea how to solve. I could have done this as well if I had more time.

Conclusion: I wasn’t prepared enough. I need to speed up my workflow even more and focus on configuring a few specific technologies over and over with different tweaks.

Oh. And the OEQ;s. My worst nightmare came thru: I got 2 hard questions. Now when I think about them I am quite sure that I nailed them. But they are EVIL!

I am SO focused on getting this done. I can hardly get on the plane back to Sweden, I just wanna have one more attempt on it right now!

Out of those other candidates I met today there was none that was confident with their result. At least 3 blew it for sure. One candidate lost ALL configs when doing a reload the last 30 minutes.

While of course feeling a bit sad and worthless today I keep telling myself that there would probably be noone except for me at my company that would pass this test.

I keep repeating Markos words: there are no failures when it comes to the CCIE lab exam. There are only “pass” and “no pass”.

Wait and see, I’ll be back!

The entrance of the Cisco Campus in Brussels

An update: The support from OSL is overwhelming. Only an hour after posting a note in the mailing list I´ve plenty of supporting feedbacks from friends all over the world. Thanks guys, you are all the best!

snippets:

I know the feeling Jimmy. All the memories of my failed attempt came back as I read your e-mail.

and:

Jimmy,

Cheers,
TacACK

and:

Jimmy,

cheers,
Piotr

Hi

I haven´t been very active on my blog lately. Guess why? This Lab preparation is killing me…

But today I dived into Yusufs Practice Lab 1 and I did a few notes. Please comment.

/Jimmy

First of all. If you use proctorlabs gear to do Yusuf Labs you see that the naming of the device doesnt match. Here is the correct matching:
ProctorLabs Cat 3 – Yusuf Sw1
ProctorLabs Cat 2 – Yusuf Sw2
ProctorLabs R7 = Yusuf R1
ProctorLabs R8 = Yusuf R2
ProctorLabs R9 = Yusuf R3
ProctorLabs R4 = Yusuf R4
ProctorLabs R6 = Yusuf R5
ProctorLabs R5 = Yusuf R6

Also note that the interface names doesnt always match!

Q2.1 – configure NAT on ASA:s. Do not enable NAT Control. Configure static identity nat on context abc1 for web server.

Why configure identity nat? There is no NAT configured on the device, whats the purpose of adding a “static (i,o) 10.7.7.7 10.7.7.7.7″ statement? It works both with and without it.

Q2.1 – “Configure static NAT on ASA2 such that Sw2 can reach dest R6 Lo0 interface using local address 192.168.10.6″

this is an ugly one! I did source translation (Telnet from Sw2:s real address TO 192.168.10.6) but I was supposed to do destination translation (telnet FROM Sw2:s natted source address 192.168.10.6). It´s SO easy to misinterprete the questions!

Q3.2 – “Configure IPSEC on ASA2 and R5. Configure high-availability IPsec peering in such wah tyat it should continue to work if euther WAN link on R5 goes down. You are not allowed to configure multiple crypto maps of mutiple peer statements. Only one crypto map with one peer statement is allowed on bith sides”.
In my opinion “high availability IPsec” is plain IPsec on router spiced up with HSRP redundancy and RRI. But here is no HSRP involved since the the requirement is to esablish ipsec between one ASA and one router.

My solution to this was to create a new loopback on R5, route the remote network (Sw2 lo0) to that loopback and apply the crypto map on this loopback. I guess the drawback with this is routing ALL traffic destined for Sw2 Lo0 to the loopback interface, not only traffic hitting the crypto map (sourced R5 lo0). I doubt that my solution would get any points on the real lab… But either way have the desired results, imho.

Q4.2 – “configure NTP on IPS Sensor”

I was unable to configure NTP. Got the same error message both in IDM and CLI:
“Error: Authenticaion failed – invalid NTP key value or ID”

This happened in CLI:


IPS(config)# service host
IPS(config-hos)# ntp-option enabled
IPS(config-hos-ena)# ntp-keys 1 md5-key cisco
IPS(config-hos-ena)# ntp-servers 10.1.1.1 key-id 1
IPS(config-hos-ena)# exit
IPS(config-hos)# exit
Apply Changes?[yes]: yes
Error: Authentication failed - invalid NTP key value or ID


There is obviously communications because these ntp debugs shows up on the NTP server R1:


R1#
Jun 27 12:54:52.811: NTP message received from 192.168.2.12 on interface 'Loopback0' (10.1.1.1).
Jun 27 12:54:52.811: NTP Core(DEBUG): ntp_receive: message received
Jun 27 12:54:52.811: NTP Core(DEBUG): ntp_receive: peer is 0x00000000, next action is 3.
Jun 27 12:54:52.811: NTP Core(DEBUG): ntp_receive: doing fast answer to client.
Jun 27 12:54:52.811: NTP message sent to 192.168.2.12, from interface 'Loopback0' (10.1.1.1).
Jun 27 12:54:52.811: NTP message received from 192.168.2.12 on interface 'Loopback0' (10.1.1.1).
Jun 27 12:54:52.815: NTP Core(DEBUG): ntp_receive: message received
Jun 27 12:54:52.815: NTP Core(DEBUG): ntp_receive: peer is 0x00000000, next action is 3.
Jun 27 12:54:52.815: NTP Core(DEBUG): ntp_receive: doing fast answer to client.
Jun 27 12:54:52.815: NTP message sent to 192.168.2.12, from interface 'Loopback0' (10.1.1.1).
Jun 27 12:54:52.815: NTP message received from 192.168.2.12 on interface 'Loopback0' (10.1.1.1).
Jun 27 12:54:52.815: NTP Core(DEBUG): ntp_receive: message received
Jun 27 12:54:52.815: NTP Core(DEBUG): ntp_receive: peer is 0x00000000, next action is 3.
Jun 27 12:54:52.815: NTP Core(DEBUG): ntp_receive: doing fast answer to client.
Jun 27 12:54:52.815: NTP message sent to 192.168.2.12, from interface 'Loopback0' (10.1.1.1).
Jun 27 12:54:52.819: NTP message received from 192.168.2.12 on interface 'Loopback0' (10.1.1.1).
Jun 27 12:54:52.819: NTP Core(DEBUG): ntp_receive: message received
Jun 27 12:54:52.819: NTP Core(DEBUG): ntp_receive: peer is 0x00000000, next action is 3.
Jun 27 12:54:52.819: NTP Core(DEBUG): ntp_receive: doing fast answer to client.
Jun 27 12:54:52.819: NTP message sent to 192.168.2.12, from interface 'Loopback0' (10.1.1.1).
Jun 27 12:54:52.919: NTP message received from 192.168.2.12 on interface 'Loopback0' (10.1.1.1).
Jun 27 12:54:52.919: NTP Core(DEBUG): ntp_receive: message received
Jun 27 12:54:52.919: NTP Core(DEBUG): ntp_receive: peer is 0x00000000, next action is 3.
Jun 27 12:54:52.923: NTP Core(DEBUG): ntp_receive: doing fast answer to client.
Jun 27 12:54:52.923: NTP message sent to 192.168.2.12, from interface 'Loopback0' (10.1.1.1).
Jun 27 12:54:52.923: NTP message received from 192.168.2.12 on interface 'Loopback0' (10.1.1.1).
Jun 27 12:54:52.923: NTP Core(DEBUG): ntp_receive: message received
Jun 27 12:54:52.923: NTP Core(DEBUG): ntp_receive: peer is 0x00000000, next action is 3.
Jun 27 12:54:52.923: NTP Core(DEBUG): ntp_receive: doing fast answer to client.
Jun 27 12:54:52.923: NTP message sent to 192.168.2.12, from interface 'Loopback0' (10.1.1.1).
Jun 27 12:54:52.927: NTP message received from 192.168.2.12 on interface 'Loopback0' (10.1.1.1).
Jun 27 12:54:52.927: NTP Core(DEBUG): ntp_receive: message received
Jun 27 12:54:52.927: NTP Core(DEBUG): ntp_receive: peer is 0x00000000, next action is 3.
Jun 27 12:54:52.927: NTP Core(DEBUG): ntp_receive: doing fast answer to client.
Jun 27 12:54:52.927: NTP message sent to 192.168.2.12, from interface 'Loopback0' (10.1.1.1).
Jun 27 12:54:52.927: NTP message received from 192.168.2.12 on interface 'Loopback0' (10.1.1.1).
Jun 27 12:54:52.927: NTP Core(DEBUG): ntp_receive: message received
Jun 27 12:54:52.927: NTP Core(DEBUG): ntp_receive: peer is 0x00000000, next action is 3.
Jun 27 12:54:52.927: NTP Core(DEBUG): ntp_receive: doing fast answer to client.
Jun 27 12:54:52.927: NTP message sent to 192.168.2.12, from interface 'Loopback0' (10.1.1.1).


Q5.1 Typo. “Configure AAA auth on Sw1″ and “Add Sw2 ip address 192.168.8.11″. It should be Sw1 everywhere in this task.

Q5.2 CLI views assigned from ACS.
It feels abit weird that there is no pound-sign in the prompt when getting into a custom view:


R6#telnet 192.168.4.11
Trying 192.168.4.11 ... Open

R2>sh pars view
Current view is 'netop'
R2>configure
Configuring from terminal, memory, or network [terminal]? t
Enter configuration commands, one per line. End with CNTL/Z.
R2(config)>


Q5.3 Configure Sw2 Fa0/7 for 802.1x
Really? I was expecting the port to configure to be unused/down. Sw2 Fa0/7 is the trunk to R1. Enabling port-control here would kill alotá traffic in my network, right?

Q6.0 configure CoPP on R2 allowing ping source from RFC1918-addresses only.
I created an acl, class-map and policy-map but I applied on “control-plane host” instead of “control-plane”. For verification Yusuf runs “show policy-map control-plane” which in my solution would give an empty output. But is there any difference in my solution and Yusufs? We are talking about icmp pings TO the router, why not apply int to the CoP host?

Q7.1 Web server protection.
The task was to limit the number of incoming embryonics to an internal web server, on ASA. Of course with limitations on how to ackomplish it. I missed the “Do not use ACL” which made me fail. Yusufs solution was to do “match port” in the class-map but instead I matched an access-group. To my defense I must say that “match port” would put the same limits on ALL incoming tpc/80-traffic not only the one destined for our web server.

Until now I had my Cisco-devices console connected to a windows-pc. It was easy but not as flexible as I wanted since I had to rdp to it when I wasn´t at home and use a putty to serial port inside that rdp-session.

So I found an old laptop, installed linux on it (actually Backtrack 3) and connected my Usb2Serial-connectors to the USB-port via an USB-hub. They popped up as tty-ports within seconds:


Apr 19 22:21:11 (none) kernel: usb 1-4.1: Moschip 7840/7820 USB Serial Driver converter now attached to ttyUSB0
Apr 19 22:21:11 (none) kernel: usb 1-4.1: Moschip 7840/7820 USB Serial Driver converter now attached to ttyUSB1
Apr 19 22:21:11 (none) kernel: usb 1-4.1: Moschip 7840/7820 USB Serial Driver converter now attached to ttyUSB2
Apr 19 22:21:11 (none) kernel: usb 1-4.1: Moschip 7840/7820 USB Serial Driver converter now attached to ttyUSB3
Apr 19 22:21:11 (none) kernel: usb 1-4.2: Moschip 7840/7820 USB Serial Driver converter now attached to ttyUSB4
Apr 19 22:21:11 (none) kernel: usb 1-4.2: Moschip 7840/7820 USB Serial Driver converter now attached to ttyUSB5
Apr 19 22:21:11 (none) kernel: usb 1-4.2: Moschip 7840/7820 USB Serial Driver converter now attached to ttyUSB6
Apr 19 22:21:11 (none) kernel: usb 1-4.2: Moschip 7840/7820 USB Serial Driver converter now attached to ttyUSB7
Apr 19 22:21:11 (none) kernel: usb 1-4.3: Moschip 7840/7820 USB Serial Driver converter now attached to ttyUSB8
Apr 19 22:21:11 (none) kernel: usb 1-4.3: Moschip 7840/7820 USB Serial Driver converter now attached to ttyUSB9
Apr 19 22:21:11 (none) kernel: usb 1-4.3: Moschip 7840/7820 USB Serial Driver converter now attached to ttyUSB10
Apr 19 22:21:11 (none) kernel: usb 1-4.3: Moschip 7840/7820 USB Serial Driver converter now attached to ttyUSB11
Apr 19 22:21:11 (none) kernel: usb 1-4.4.3: pl2303 converter now attached to ttyUSB12
Apr 19 22:30:36 (none) kernel: usb 1-4.4.2: pl2303 converter now attached to ttyUSB13


The easiest way (that I´ve found out. I am not a Linux-exert) to connect to the serial-port is by using screen. Like this:


bt ~ # screen /dev/ttyUSB8


I created a few scripts/aliases to simplify this:


bt ~ # ls -l
total 732968
-rwxr-xr-x 1 root root 22 Apr 19 23:08 fw*
-rwxr-xr-x 1 root root 21 Apr 19 23:08 r1*
-rwxr-xr-x 1 root root 21 Apr 19 23:08 r2*
-rwxr-xr-x 1 root root 21 Apr 19 23:08 r3*
-rwxr-xr-x 1 root root 21 Apr 19 23:08 sw*
..
..
..
bt ~ #
bt ~ # cat r1
screen /dev/ttyUSB6
bt ~ #
bt ~ # cat r2
screen /dev/ttyUSB0
bt ~ #
bt ~ # cat r3
screen /dev/ttyUSB9
bt ~ #


After opening ssh-access thru my internet-firewall I can now access my home lab from anywhere by just creating one or multiple ssh-sessions and connect to each serial port by using the aliases. Or even create multiple connection entries in my terminal software and configure each one with a script that executes “r1″ or “r2″ and so on after login.I exit each session with CTRL-A + K.

Cisco recently released a Exam Preparation Checklist which is kinda like a extended blueprint. It´s an extensive and detailed list of topics that you should know before taking the CCIE lab exam.

I made a copy of that Checklist and graded my current knowledge of each topic on a scale from 1 to 5 where 1 is “I´ve no idea what this is” and 5 is “I know it completely!”.

My idea is to do a new grading of my knowledges again every now and then to get a feeling on my progress.

At the bottom I´ve summarized the grades and displays it as a percentage. Simply “how close am I to having a 5 on all tasks?”.

Also, if my boss some day ask me what the heck I am doing all these work-hours, I will gladly give him a link to this blog.

Task 4.1 – IOS CA

Task 4.2 – IOS L2L

This is all about enrollment of certificates from the CA in previous task to two IOS-routers and setup an ipsec-tunnel.

• What confuses me is that there is nothing in the configuation telling it to authenticate with certificates. All there is compared to “normal” preshared-key-auth is a missing “authen pre-share”. Which ofcours means that authentication is done with the certificates by default. I understand, I just have to get used to the fact that there is no command visible in the crypto isakmp policy saying “authentication MY-CA-TRUSTPOINT”.
• When entering a wrong peer in the crypto map, it´s not just enough to re-enter a new ip. Since a crypto map sequence can have multiple peers for redundancy the old one doesnt go away. The effect is that the tunnel goes up, after a while, since it first tries with the bad peer ip before trying the second one. Remove the first.
• Me being more used to vpns in asa than in ios usually tear down vpn-tunnels with the commands “clear crypto isakmp sa” and “clear crypto ipsec sa”. In IOS the corresponding command is “clear crypto session”. Cool.

Task 4.3 – VPN IOS-ASA

The task was to setup a tunnel between IOS and ASA. Preshared-key, all straight-forward. However, I was asked to prioritize to certan traffic going into the tunnel from the IOS-router. This was done by creating a service-policy on outside-interface like this:


class-map match-all VPN-CLASS

match access-group 150  ! The ACL that defines the traffic to prioritize

policy-map VPN-POLICY

class VPNCLASS

service-policy output VPN-POLICY


• And, dont forget to do “qos pre-classify” on the crypto map! Otherwise your class-map has to look for ESP-traffic and that is not very granular, is it?
• “create lo3 on r2, assign it ip 192.168.3.2/24″ and “create a vpn tunnel between Vlan100 and the newly created loopback network”. I used “host 192.168.3.2″ in acl, but it clearly states “the loopback _network_”. Darn!

Task 4.4 L2L Aggressive mode with PSK

Hi

Not sure if this is it or not but you have crypto isakmp key ipexpert
hostname r5.ipexpert.com and the debug shows    FQDN name    : R5.ipexpert.com

Task 4.5 L2L Overlapping subnets.

Task 4.6 – Easy VPN Server on IOS

Todays lab-preparations was dealing with IPS. But it could be OSPF or english grammar or anything. What I am learning nowadays when working with IPExpert Workbooks has not much todo with technology. I pretty much know how to configure stuff. The big challenges for me are to understand the scope of the task and not to ruin what´s already been built.

Many tasks in the worksbooks are like:

• Configure X between router R1 and router R2.

Sounds easy as long as you know how to configure X, right? But should X be used for all traffic that crosses the path between R1 and R2, including traffic generated far behind R1 and destined to somewhere hops behind R2? Or should X just be configured to handle traffic sourced from R1 itself and destined to the actual R2-router (and vice versa)? And if, should traffic generated from ANY R1/R2-interfaces be included in the scope of the task, including loopbacks?

I have no answer yet. If I look into the solution guides it can be all of the above…

CCIE Security Lab blueprint specifies ACS v4.1 for windows. It seems that Cisco has removed links to the previous Evaluation version download.

However, it still exists there. Here is the link: Cisco ACS 4.1 for windows eval. It requires CCO-login.

The main challenge and discoveries during the last days of my “labbing” had nothing to do with technologies, TLA:s or ETLA:s. It has all been about finding out how to attack the lab. How to work focused and be well prepared before beginning to configuring boxes.

I have read on several different places that everyone recommends to read thru the entire lab before configuring, and also to make your own network diagrams. But I didn´t fully understand it until now.

When You look at one singe task within the lab it most of the times look like this:

• Configure device X to do yada yada yada
• You are not allowed to use technology Y
• Make sure that Device Z can connect to device Q with protocol W

The problem is that when You configure device X (which is probably the only device you need to touch in this task) the configuration steps needed depends upon other task telling you to configure a totally different device. And is that other task completed yet?

Let´s say that in order to verify that “Device Z can connect to device Q with protocol W” you rely on two other routers in transit between Z and Q. And these routers in the end will be fully stuffed with address translations and filtering. So, if you configure this device X prior to the other tasks you need to either not be able to verify functionality until the other tasks are also completed (and you need to remember to do it!) or you need to configure parts of other tasks first to make sure that all devices “on the way” between Z and Q are properly configured so that you can do your verification of the task. Or should you configure the technology Y now and modify it later when doing these address translations?

You see the challenge? And believe me, the lab is stuffed with dependencies like this!

My conclusion is that everyone else was right when they said:

• Read the entire lab first!
• Make your own network diagram while reading the lab!

What I promise to myself to do from now on is to add the following to my diagram while reading the lab:

• The topology on L3 (but don´t forget to add L2 filtering devices!)
• All IP addressing. Networks and devices.
• Placement of access lists
• address translations (including the global or translated addresses between brackets)
• Special functions to watch out for, like firewalls

This will prevent me from destroying now what I built an hour ago!

I have three main enemies on this journey: Me, Myself and I.

Here is my lab diagram for Ipexpert Workbook 1, Lab 2a. The lab is still unfinished, I am eager to finish it as soon as possible!

This blog is transforming from a swedish all-purpose blog into a english-speaking tech-blog. The reason for that is my preparations for the CCIE Security certification lab. All old swedish posts are still here, just click on the swedish/English categories-link above to filter.