Tag Archives: Data Center

My CCIE Journey – Act II

CCIE_DC_Logo
In fact the title should be “My CCIE Journey – Act III” but I don’t want to use that one because I had a bad experience with the CCIE Voice lab exam 🙂

There are many (very good) links about that specific subject but I wanted to give my own opinion as well :). Here is a list (incomplete for sure) of the people that have blogged about their CCIE DC lab experience :

I have shared my journey towards the CCIE RS in 2011 and I wanted to share it again with you. I passed the CCIE DC lab exam one month ago and it was tough, long, hard,arduous, baffling, difficult, exacting, exhausting, hard (yeah I already used it on purpose 🙂 ), intractable,perplexing, puzzling, strenuous, thorny, troublesome, uphill.

As soon as I failed my CCIE Voice exam, my frustration went so high and I needed a break from the Voice exam a little bit. The Data Center exams were released by Cisco and I always wanted to be involved in a Data Center infrastructure project. I immediately decided to jump into the DC field and start to climb the (infinite) ladder.

At this time my DC infrastructure background wasn’t enough to pass the CCIE DC Written, I decided to spend a year reading books and solidify my knowledge.

First and foremost the CCIE DC blueprint is like any CCIE DC, it is VERY large. As an expert that will face customers and other experts, you definitely have to dig very deep to understand what’s going on in every section of your infrastructure (Compute / Storage / Infrastructure).

In my previous CCIE Journey post I used this expression from Brian McGahan: “a CCIE journey is not a short race, it is a marathon”. 4 years after, this applies even greater today. If you have a family, you better have to have a very supportive wife/husband. My wife is the most supportive person I’ve ever met.

We had our 3rd baby 10 months ago and my daughter couldn’t sleep at night. My wife was taking care of all 3 children 24/7 while I was studying. She even stayed at my parents home for several weeks to make my study time more efficient. After all, I can say that we are both CCIE RS-DC right now :).She deserves the title as much as I do … I am pretty sure that the CCIE exam is easier than taking care of the children. What I am trying to say here, is that you have to be dedicated to this exam.

CCIE Written Preparation

I already mentioned before but I read LOTS and LOTS of books. I will give you my list very soon but first I would like to start with one of the best technical book I have read in my entire career.

Data Center Virtualization Fundamentals  written by Gustavo Santana is definitely the best Data Center book out there. If you have some Routing and Switching Skills, you probably read the very famous Routing TCP/IP Books (Volume 1 covers IGP and Volume 2 covers BGP,Multicast and IPv6). All I can say is that Santana is as awesome as Doyle. I don’t want to overemphasize but I really enjoyed every words of the book.

HTML5 Icon
 

The others books are the following:

  • Cisco UCS (a bit outdated but still nice to understand)

HTML5 Icon

HTML5 Icon
 

HTML5 Icon

HTML5 Icon
 

HTML5 Icon
 

I also read some free ebooks written by EMC and IBM. To me these 2 books regarding Storage Area Networks are great free resources:

I was almost ready to sit the CCIE DC Written exam but I decided to solidify all the theory I have gained throughout the year. In order to do that I gave a look at CCIE Training vendors.

I have a very good experience with all the main vendors and this is probably the most frequently asked question so far : “Which vendor did you use for your preparation”

First I never really picked up a vendor. I tend to prefer to choose an instructor. I went with INE and Micronics Training for my CCIE RS because I heard from close friends that Brian McGahan and Narbik were top notch instructors (and they are). For my voice studies, I went with IPX because Vik Malhi is the best Voice Trainer I’ve ever met (Since that time, Vik has its own training company CollabCert, you should definitely give it a try if you are interested in collaboration). So in my opinion, students should not pick a vendor, they should pick an instructor and an instructor that meets your personal requirements. Maybe McGahan, Kocharian and Malhi are not the best for you but I can tell you from my personal experience that they are the best for me.

Choose wisely ! A training vendor business is to make your studies time efficient.

I bought an All Access Path from INE and decided to enroll myself into the CCIE Data Center Written Bootcamp. If you want to have a look of the teaching style:

 The INE videos are matching all the blueprint : Nexus / Storage / UCS.

There is another useful (free) resource available for you guys: Cisco Live Portal. This place is the place to watch deep dive videos regarding every Cisco topic!  For the DC stuff there are many listed by Brian McGahan on its “how to pass the CCIE DC” blogpost.

I passed my CCIE DC written exam on my second try. It was a really tough exam …

In order to track my studies during the journey, I have used trello and I love this app. Here is an example of how I managed my tasks

Trello_DC

CCIE LAB Preparation

The lab is a complete different story and I didn’t really relied on any vendors regarding the workbooks. I used INE and IPX for my online bootcamp but I will cover that later.

So regarding the workbooks, I didn’t really use any of them … I just did a few lab here and here from both vendor but I didn’t really like it. I just wanted to read the config guide, build the infrastructure and then run every show command I could.

For CCIE RS and Collaboration, it is very easy to host a rack in your home or at work. For the DC track, things can get more tricky since you will need a N7K (with VDCs you slice your switch into multiple virtual switches, don’t worry it is part of the blueprint 🙂 ), 2x N5K ,2x Nexus 2232 PP (in order to run FCoE), 2x MDS (9222 is my choice) and a small JBOD (I will make a separate post to show you how to build the cheapest JBOD ever 🙂 ).

INE and IPX racks can be very busy if you want to book the racks with UCS … I also recommend to use the Cisco UCS Platform Emulator on your own laptop (run on ESXi as well if you have a virtualization lab). You can do almost everything with it (except booting your favorite Operation System / Hypervisor).

My local Cisco SE (Vincent, thank you so much !) was kind enough to let me borrow 2x N5K with some FEX and  2x MDS 9222i. I have built a cheap jbod and I could test 100% of the storage feature for the lab exam.

CCIE DC Lab
I think the most valuable resources to practice is the Cisco Partner Education Collection .

There are so many labs and hardware there (sometimes fully booked of course) than you can spend countless hours of labs … Joel Sprague (which is an MVE [Most Valuable Engineer] I met during my studies) did a very good job by posting all the valuables labs that you can do with the Cisco PEC. I didn’t do ALL of them but the vPC / Fabricpath / UCS / N1000v are definitely mandatory … The UCS is one of the best because you can boot from SAN and the UCS is yours for 8 hours and for free.. Nothing can beat that !

CCIE DC UCS
 

Even if you are studying for the CCIE LAB exam and that you know that you are going to spend 8 tough hours configuring weird things, you still need to read a lot in order to configure your infrastructure.

I would recommend to read almost all the configuration guides related to the blueprint for the Nexus. For UCS and MDS, You can periodically check but there is no need to read everything like you should do for the nexus part.

I have watched both INE and IPX videos regarding the CCIE lab exam, McGahan and Rick Mur videos are perfect ! McGahan for INE was in charge of storage and Nexus while Snow was in charge for UCS.

I also attended 2 CCIE online bootcamp from INE (McGahan Again) and IPX with Jason Lunde. Both did a great job.

McGahan is definitely the big player here, his complete set of videos (Nexus – Storage – Lab Cram Session) are simply awesome. It covers way more than you need for the CCIE DC exam

Here is a preview of its DC lab cram session:

BM DC Lab
There are plenty of nice other resources that other CCIE DC have published on their own blog. Here is the 3 I used during my studies:

CCIE LAB Exam

I decided to book the CCIE the day before my vacations started because I didn’t want to go in vacations with the CCIE still in mind 🙂

So I went to Brussels on July 10th and I was very pleased by the proctor (if you read me, I would like to thank you. The experience was great). The exam is fair, it is hard but fair. There are no second guess like I had in voice. Questions were very precises and if I didn’t understand everything in the question, the task title made me clicked in my head : “Gotcha”.

You have to CAREFULLY read the tasks. If Cisco is asking for an ACL named MYCCIEDCLAB, you will not get the point if you configure it MYCCIEDCLAb. Even if your configuration is correct, they will look for the right naming convention. If you want to prevent all sorts of easy mistakes, your best weapon is the CTRL+C , CTRL+V. I can tell you this is the best thing you will ever need in the lab. Notepad is so useful as well !

During your daily job you would still do it right ? What if you want to configure vlan 100,200,300,400,500,600 in all your devices (let’s assume VTP is bad … wait a minute … it is bad .. in my opinion 🙂 ) You would open a notepad, type your commands , and paste into all devices right ?

My advice is to do the same for your CCIE Labs.

As Brian McGahan said, I did my happy dance when you see the UCS-B series booting ESXi 🙂

HTML5 Icon
I finished the lab with 1 hour left. Now the critical thing to do was to stay there and look for small mistakes I could have make during these very long 8 hours. I found some and for every tasks I checked that what I did was still working and that 100% of the requirements were met.

Finally I left the building and asked the proctor when can I expect the results to be delivered. He told me : “within few hours” . I thought he was making fun of me but he was right.

I went to the airport to meet a friend from Belgium and I received the score report notification.

Was thrilled to see the results : “PASS”

The exam can be tought but again it is doable. During my studies I have met a much better DC engineer than me, he failed the exam twice 🙁 . So please be sure to read slowly and try to understand what they really want…

So what’s up to me now that I am a double CCIE. In the beginning of the post I said that I started to climb the infinite ladder, what does that really mean ? It doesn’t mean that now that I am a CCIE, I can rest and that I can live like that and that my knowledge will stay at the same level through my career. People who think they are done with learning  are wrong.

Knowledge has to be sustained ! I still have to work on every protocol if I want my knowledge to be intact. I also have to learn new emerging technologies like Dev-Ops (not new but still new to me) / ACI / NSX etc etc in order to become a better engineer !

I hope you enjoyed the blogpost and in the meantime, if you have some questions, you can leave a comment below.

 

Nicolas

vPC order of operations

Cisco Nexus can be very temperamental or capricious (pick the one you prefer 🙂 ) and the vPC technology is not an isolated case.

There is a certain way to configure vPC and we will see that in that blogpost.

The following topology will be used:

 


vPC diagram
 

Enabling the feature

Obviously we need to activate the vPC and LACP feature in order to build a proper vPC configuration.

 

Peer Keepalive connectivity

The Peer Keepalive link is used to detect failure between the peers. It is definitely not used in the data plane. On N5K, the management interface is usually used for the Peer Keepalive Link and management purpose. On N7K, Cisco does not recommend to use the supervisor interface as a vPC peer-keepalive link since it can introduce failure if that supervisor crashes…..

In our example we will use the management interface in order to have IP connectivity between our vPC peers.

Now let’s check if N5K-1 can reach N5K-2

The behaviour above is normal since we are trying to ping 10.2.8.84 using the global RIB and the mgmt 0 interface does not belong to that RIB. Instead it does belong to the management vrf . So if we add the “vrf management”  keywords, everything should be fine !

and it is 🙂

 

vPC domain and vPC peer link configuration

Now let’s create the vPC domain and the vPC peer link.

Please be aware that it is a best practice to configure an unique vpc domain ID for every pair of Nexus. The vPC domain ID will be used to build a system mac-address. So if 4 nexus are connected together created 2 vPC domains and if both vPC domains have the same system mac-address, you will experience something funny 🙂

I decided that N5K-1 will be elected as the Primary vPC peer and N5K-2 will remain Secondary vPC peer.

Please note that if you do not specify a keyword for the peer-keepalive destination command, the switch automatically use the management vrf.

As we can see above, the vPC keep-alive status is OK since we have reachability between our mgmt 0 interface.

We will now create our vPC peer-link that will be mainly used for CSF and many other things (will be detailed in a future blogpost)

Enabling the vPC peer-link on the port-channel automatically set the type to network for that interface. This means that we just enabled bridge assurance on that link. In a vPC topology, Cisco recommends to enable Bridge Assurance only on the vPC peer-link.

We can now enable the interface to check the status of our port-channel.

Our Port-channel is functionning properly so we can check the status of our vPC peer link now

Everything looks so far so good !

Do you remember when I explain about the dependencies between vPC domain and vPC system mac-address ? 🙂 here is the result



vPC Configuration

We will now create a vPC towards an LACP neighbor which in this case will be a server.

Here is the configuration used on both Nexus

Let’s assume that our server’s NIC team is already configured for LACP. Let’s now check the vPC status

Consistency parameters are matching on both vPC peers and our vPC towards that server is functional 🙂

 

LACP verification

The following output will confirm that we are indeed having an LACP port-channel between the server and the vPC peers

This post was a bit basic but it can get tricky to troubleshoot a vPC so be sure to follow the steps above in order to save time in your deployment/labs.

We will dig deeper on vPC tricks soon 🙂

Nicolas