Here’s my typical disclaimer: I’m not a developer. I have the ability to make code give me an expected output, but I do not do anything “the right way.”

All the code I write for these blog posts is in my Github repo that you can and should freely copy and modify. Here’s the environment I’m running this stuff in. Python. Pynetbox. You know the drill by now.

Python         :  3.9.10
Pynetbox       :  7.0.0
Netbox version :  3.5.8

We’ve been working through some stuff, and, at this point, we have a lot of stuff in our Netbox instance. Let’s step up the game a little, though, and see if we can’t generate a network diagram based on that data. Let’s set some expectations, though. This is not going to be comparable to that Visio diagram you’ve managed by hand for the last 8 years. This is going to be a very simple diagram with subnet, nodes, and IP addresses – enough for an auditor or for some architect who doesn’t know what’s in their own data centers.

I just upgraded my Netbox server from v2.7.6 to v3.4.8. This is just a record of what I did in case anyone want to know how I did it.

Environment

• The source v2.7.6 server is an Ubuntu 18.04 VM. Yes, both are very old.

• The destination v3.4.8 server is an Ubuntu 20.04 VM.

• We have no media, scripts, or reports in Netbox.

• I’m running Virtualbox on my laptop to do the data migrations.

Here’s a quick summary of what we’ve talked about in the last few posts – all with Python.

• We’ve asked Netbox to provide some info using pynetbox.

• We’ve added stuff to Netbox using pynetbox.

• We’ve updated and deleted stuff in Netbox using pynetbox.

• We’ve logged our messages with Python logging.

This is all fine and dandy, but I would guess that you’re not the only engineer in the company and production maintenance scripts don’t run off of your laptop. We need a way to let a group of people know what’s happening when one of your scripts is run. And please don’t say email. Email has been worthless for alerting for over a decade, and there are better ways to do it. Search your feelings…you know it to be true!

As a warning to everyone, I am not a developer. I am a network engineer who is trying to do some automation stuff. Some of what I’m doing sounds logical to me, but I would not trust my own opinions for production work. I’m sure you can find a Slack channel or Mastodon instance with people who can tell you how to do things properly.

I use too many print statements to figure out what’s going on. Get an object and print it to screen to make sure it’s right. Do a calculation and print the result. There are so many print statements in my code that I had to start using a debug variable to tell it when to print stuff. I even use that technique in my functions.

As a warning to everyone, I am not a developer. I am a network engineer who is trying to do some automation stuff. Some of what I’m doing sounds logical to me, but I would not trust my own opinions for production work. I’m sure you can find a Slack channel or Mastodon instance with people who can tell you how to do things properly.

We’ve added stuff and updated stuff, so let’s delete some stuff. “Hey, man…you already did that,” you say? You’re right! When we started creating API tokens based on user/pass, we made sure to delete the token at the end. That means we should all be professional pynetbox deleters, then, right? :)

Let’s see. We’ve queried stuff on Netbox and added stuff to Netbox. Now let’s update stuff.

Netbox, like all sources of truth, needs to be kept up-to-date if it’s going to be useful. Without doing some maintenance on the data, it will wind up being like that one Visio diagram that you give the auditors – it might have been accurate at one point but gets further and further from the truth every day. We’ll need to keep our stuff updated today in order to use it more effectively tomorrow.

As a warning to everyone, I am not a developer. I am a network engineer who is trying to do some automation stuff. Some of what I’m doing sounds logical to me, but I would not trust my own opinions for production work. I’m sure you can find a Slack channel or Mastodon instance with people who can tell you how to do things properly.

I think there’s a theme in the last few posts. I can’t quite put my finger on it, though. :) We’ve talked about querying Netbox, but it’s pretty useless without data actually in it. Let’s look at how to get stuff in there using pynetbox.

As a warning to everyone, I am not a developer. I am a network engineer who is trying to do some automation stuff. Some of what I’m doing sounds logical to me, but I would not trust my own opinions for production work. I’m sure you can find a Slack channel or Mastodon instance with people who can tell you how to do things properly.

A bit ago, we talked about getting information out of Netbox with Pynetbox. The example was very simple, but I’m afraid the real world dictates that querying every device every time is not very efficient or manageable. At some point, we’ll need to ask for a subset of everything, so let’s look at filtering.

As a warning to everyone, I am not a developer. I am a network engineer who is trying to do some automation stuff. Some of what I’m doing sounds logical to me, but I would not trust my own opinions for production work. I’m sure you can find a Slack channel or Mastodon instance with people who can tell you how to do things properly.

The last time, I talked about using pynetbox to make queries to Netbox. This was a very simple example, and one of the things that bugged me the most about it was the API token. In that post, we used a statically-assigned API token where I went into the Netbox GUI and generated one for myself. I think I may have even noted that this was definitely not the best way to handle those things. A possibly-better way to do it is to use your username and password on Netbox to generate a token for yourself. This would a token that you then delete when you’re done.

I feel like a teenage girl with a fashion blog who hasn’t posted in 6 months and comes back with “I know I haven’t posted in a while…”  Sigh.  It’s been right at a year since I actually published a post, so I figured I would give everyone an update.

I’ve had some personal things going on lately, and those have taken all of my energy.  We’ve made it through those rough times, so my energy is coming back.  I’m feeling better every day, and I hope I can get back to producing some content.  And, let me tell you…I’ve got some stuff to talk about.

Put it on your calendar.  Cisco Live US is June 25 - 29, 2017, in Las Vegas.  This is the largest conference I go to every year, and it’s the highlight of my professional year.  I’ve been going for a few years now and enjoy it for the content and camaraderie.  What are we doing this year?

We’ll fly in on Friday again and do something.  No idea what, but I imagine we’ll throw out an invitation for dinner to the public and meet somewhere.  If you’re going to be in town, let me know, and we’ll meet up.

I wrote this post during Cisco Live and said “I’ll just give it a once-over tonight and publish it.”  That was something like 6 weeks ago now. What a loser I am.

Yes, really. QoS has actually gotten some attention this year. After how many years of living in the dark and being feared by junior and senior engineers alike, we’re seeing some really cool technologies coming out for it.

It seems that Cisco Live is about the only thing I blog about in the last…well, few years.  At least I’m still writing, even if it is twice a year.  :)

Here’s a summary about Cisco Live for those who live in a dark hole.  It’s July 10 - 14, 2016, in Las Vegas.  If you do anything with Cisco, you should go.  If you do anything with technology that isn’t Cisco, you should go.  Bring your significant other.  There’s plenty to do for everyone.  Anyway, on to the details for this year’s show.

You should know by now that I always find something to complain about.  Is that a bad thing?  Probably.  Does it help improve things?  Absolutely!

Again, I love going to Cisco Live every year.  Without question, it’s my favorite event of the year.  It’s a great event with great people and great things to do.  With that said, let’s look at what could have been a bit better this year.

Another year, another Cisco Live.  Boy, was it a good one.  San Diego is a great city, and convention center there is plenty big to take care of all 25k attendees.  On top of that, the city itself is equipped to handle groups of 40 roaming the streets looking for food and entertainment.

This year’s event had the usual stuff that everyone talks about - breakout session, keynotes, exams, etc. - but Cisco stepped outside of technology this year by helping others.

There are three ways to manipulate the interface cost in OSPF.  One is very direct, one changes the presentation of the interface, and the other changes the calculations for every interface.

Set the cost of the interface directly - Just give it the number you want.  Easy.  This is the number OSPF will use in the SPF calculations without doing any math on the interface.

R1(config-if)#ip ospf cost 8482

Set the bandwidth of the interface - The formula that OSPF uses to calculate interface cost is pretty easy to remember - (reference bandwidth) / (interface bandwidth).  Changing the interface bandwidth will obviously change the result of the calculation.  The same caveat for EIGRP route manipulation holds true here; if you change the bandwidth of the interface, you may affect other things like QoS…or EIGRP, now that I mention it.

When you configure OSPF network statements, IOS orders them most-specific to least-specific then does a top-to-bottom match of the interfaces. It doesn’t matter which order you put them in, the configuration will always be ordered with the longest prefix matches first.  Lab time!

I have router R1 with these interfaces.

R1#sh ip int brief
Interface                  IP-Address      OK? Method Status                Protocol
FastEthernet0/0            10.0.0.1        YES manual up                    up
FastEthernet0/1            unassigned      YES unset  administratively down down
Loopback100                10.0.101.1      YES manual up                    up
Loopback200                10.2.101.1      YES manual up                    up

Let’s add the OSPF configuration where 10.0.0.0/8 is in area 2 then check what OSPF thinks is happening.

Let’s get an IPv4 default route into EIGRP.  There are a few methods to do it.  I hate most of them, though.  I think it will be obvious which one I like.

Here’s the lab I have set up to test everything.  I want R4 to generate the default in each case.

Default Network - Candidate default.  I don’t think I’ve ever used that all my years in networking, but here’s how to use it in EIGRP for a default route.  You basically say “If you don’t know where to send a packet, send it to where network X lives.”  We’re going to set the 192.168.1.0/24 as the default network, so, in our case X = 192.168.1.0. R4 will tag that route as a default candidate when it advertises it to the rest of the network.  The config is easy but requires a classful (yes, classful) network to be configured as the default.

For both OSPF and EIGRP routers to become neighbors, their interface’s primary IP address must be on the same subnet. That statement is true. There is a difference in the definition of “same subnet”, though.

In OSPF, both routers have to be configured to be on the same subnet with the same mask or else they won’t neighbor up.  When an hello packet is sent, the subnet mask is sent embedded in there.  The router does a quick look to be sure the subnets are defined the same way on both ends.  If everything doesn’t match, they don’t neighbor. Here’s a Wireshark screenshot to show you the OSPF hello.  Note: See edit below.

I wanted to do some analysis of the EIGRP topology table last night, so I fired up a small lab. I was especially interested in how external routes appear there and compare to internal entries. Like all good scientific endeavors, the whole thing got derailed when I made a realization.

Here’s the lab I set up. You can ignore the IPv6 info for this exercise.

It’s a simple little thing.  All the networks you see are included in EIGRP 100 for simplicity.  I limited the network statements to 192.0.2.0/24 to keep my options open. I went ahead and added Loopback100 on R3 with an address of 3.3.3.3/32 and added a redistribute connected with a route-map to get the route out in the wild.  Here’s what I had.

Everything is in order for my trip to Cisco Live 2014 in San Francisco.  Conference passes are purchased.  Hotels are reserved.  Flights are booked.  It’s going to be a great event, and I can’t wait!

Note:  My wife will be with me again this year, and she is trying to get a tour group going to look around the city while others are in sessions.  If you want to be in on the tourist action, contact her via Twitter.

On Friday, and for the fourth time, I took the CCIE R&S Written exam (350-001).  For the third time, though, I failed.  Let me tell you, I am absolutely devastated.  I worked my buns off for the past few weeks, but I’m obviously missing some important piece to put me over the top.

Not only was I disappointed with my overall score, I was disappointed by my score in some of the focus areas.  For God’s sake, I made a 50% on each of the routing and switching sections, which is just absolutely embarrassing.  I mean, this is my bread and butter here.  This is what I do all day every day, and I could only muster a 50%?

This is a study note post, so please don’t take this as written.  I’m not the authority on the subject, so please correct me if needed.

Back in the day, somebody decided that we all needed to have a Type of Service (ToS) field in the header of IP packets.  Only God knows what this spawn of Satan wanted to do with it, but we’re stuck with it on the CCIE R&S exams.

While walking through the World of Solutions, we ran across a big black truck with lots of antennas all over it.  It was obviously an emergency communications vehicle of some kind, but I was really surprised to see it was a Cisco truck.  It turns out that Cisco has a Tactical Operations group (Twitter) that was formed to provide disaster responders with much-needed communications for EMAs, fire, police, medical, etc.

The big truck was the NERV - the Network Emergency Response Vehicle (PDF link).  It’s full of traditional HF, VHF, and UHF radios that the ham radio operators usually bring to these disasters.  This is a necessity when all phones, cell, and Internet are down.  It could be the only way fire fighters are able to call for reinforcements or the only way a hospital can call for more supplies.  The NERV, though, takes it to the next level.  On top of the radio gear, it is equipped with satellite uplinks for Internet access, wifi, and digital voice and video through UCS Express, IP phones, and Telepresence.  Analog voice is always the first method of communications restored via battery- or generator-powered gear, but an area will eventually need a network with voice and video.  That’s where the NERV comes in.

Yes, I went to Cisco Live and survived.  It was the social event of the year, but the main focus is learning about the cool, new stuff.  One of the booths I visited was a demonstration of Cisco Active Advisor.

This is a cloud-based (BINGO!) application that keeps an eye on the lifecycles of your IOS devices.  Using the web interface, you can scan a range of IP addresses from your machine and have your gear automatically added to the service.  Once in there, you can see, among other things, the warranty and support contract information for your device.  If your contracts is about to expire, it’ll let you know via email.   It also tracks any vulnerabilities that may apply and emails you if any are detected.  This beats trusting your reseller to send you renewals or watching an RSS feed for PSIRTs and field notices.

I just got through a big weekend.  We upgraded our main production firewall, but the process had a few twists.

The old firewalls, a pair of ASA 5520s, were running at about 80% CPU during the day.  That’s high enough that even I cringe when I saw the utilization in ASDM.  It was obviously time to upgrade to something with more beef, but we also wanted something that will last for years.  After looking around and getting some quotes (that made me jump back in my seat), we finally decided to go with a pair of 5555Xs.  These guys give about 10 times the throughput of the 5520 with about 8 times the memory.  Seems to match the requirements.  Now for the complications we had to work through.

I’m all set up to go to Cisco Live in Orlando this year.  Good thing, too, since I couldn’t make it to San Diego last time. It’ll be a great and fun time as usual, and I’m quite excited.

As it turns out, ARRL Field Day happens to be the weekend leading up to the festivities.  I’ve been in contact with the local Orlando club, and they say the attendees are more than welcome to join them.  They are meeting at the City of Orlando Emergency Operations Center, which is about 20 minutes away from the Convention Center.

I spent the last of my Juniper exam vouchers on the JNCIS-SEC exam and passed by the skin of my teeth today.  Since I took a new job last month that’s 100% Cisco, this is the last Juniper exam I’ll take for the foreseeable future.  Too bad, too.  I really like the Juniper exams.

At my previous job, we were 90% Juniper with a whole mess of SRX firewalls around the world.  Since this exam is really about that platform, it was pretty logical that I should do alright on it.  Of course, a large part of the blueprint was on IDS and UTM, and I have no experience there.  For my entire career, those type of devices have been handled by other groups, so I had some studying to do.  That’s where I ran into problems.  I have absolutely no interest in IDS.  I have no interest in UTM.  There’s nothing about content scanning and analysis that interests me at all.  I promise you all that I tried my best to read up on these topics, but I was asleep after 10 words every time I tried.  After rescheduling the exam twice to try and study a bit more, I finally decided it wasn’t worth the trouble and just took the exam…and passed.

There are a few ways to leak routes in and out of virtual routers in Junos. On the list is a cool feature called the logical tunnel interface.

So, what am I talking about?  One way to separate traffic on a router is to use virtual routers (VRs) so that you wind up with multiple routing tables on the same router.  This separate traffic, but you will usually (read: always) have a demand to get traffic from one VR to another.  There are a few different way to do that (see rib-group, instance-import, next-table, et al.), but one really cool way to do it is through logical tunnel interfaces.

I quit my job…by design.  I start a new gig on Tuesday and am getting back to the world of Cisco.  As a last nod to Juniper, I decided to use an exam voucher I had and take the JNCIS-ENT exam.  Easy pass.

The content was right along with the exam objectives, so there were no surprises.  Most of the topics are things I’ve done a thousand times on the job.  There were some things, though, that were beyond my experience.  IS-IS was the big one.  The very first question I got was about IS-IS metrics, and I had absolutely no clue what the answer was.  Nor did I have any clue about the other IS-IS questions.  I went 0-for-3 on those guys.  The only other problematic topic was HA, which didn’t really surprised me.  I was able to answer the VRRP questions, but  I’ve never done any GRES, ISSUe, RTG, etc., at any point in my career.  It wasn’t surprising that I didn’t do too well on those.  Everything else was cake, and I only missed 6 questions in my comfort zone.

Yes, I know I’m late.  Just remember I’m lazy, and it all makes sense.

This year I’ve decided to go a little more practical with my goals.  Instead of “get this cert” or “learn about that”, I’ve decided to take some steps to help myself.  That is, in order to learn and advance, I need make sure I give myself the opportunities to do so.  Damn, that sounded like some crap from a marketing department, so let me use my own words.

The year is finally coming to an end, so it’s time yet again to look at goals and embarrass myself by publicly admitting that I didn’t meet them.  Oh, well.  Let’s get this done so I can go back to sleep.

I changed the layout of the blog, so the page with my goals isn’t really visible.  Here’s what I claimed I would do this past year.

• Select a CCIE training vendor - Yeah…this didn’t happen.  This is a very high-priced item, and I simply couldn’t afford the packages I wanted.  We’re talking $8k - $10k for everything.  Yikes!  I asked management at work to pay for it.  They said they would but that I would have to agree not to leave the company for some long length of time.  I didn’t want to put myself in a situation where finding a new job meant writing a check for $10k, so I decided to pass on it.  Without the financial backing, this ended with me just sighing pitifully on my couch.
• Take the CCIE R&S lab - Of course this didn’t happen without the first one.  I guess I could have bought the materials that I could and just got on a bus to Raleigh to see what happens.  This whole thing was complicated by the fact that the new job is 95% Juniper.  My waking hours at work and my study time at home were spent trying to figure out how Junos works; I tried my best, but it was just too difficult for me to study both at the same time.  For the trifecta of excuses, I also had an issue with my study area. I went from a 4-bedroom house to a 1-bedroom apartment when we moved for the new job.  There’s no quiet space at all to study at all - a huge problem I need to fix.
• Pass JNCIA-Junos exam - Wo!  I actually did this one.  I took this exam a few months back and passed it without any problems.  Good for me!  One out of three!

As for my goals, it really wasn’t a very good year.  Even for me, it was bad.  I’ll tell you, though, it’s very hard to study when you don’t have one subject or a place to do so.  Definitely things I need to work on in 2013.

Here’s one that I use every day at work. We have multiple customers coming into the same router, and, as luck would have it, they all use 192.168.1.0/24 (OK…not really but it might happen). That means we have to separate them into their own routing instance, or virtual router, so pass traffic to their firewall.  Think VRF lite on a Cisco router.  Let’s conflagrate.

First, we configure the instance as a virtual-router.

We’ve been looking for a new Network Engineer for quite a while but are having no luck at all.  There is plenty of talent out there, but finding a high-end Juniper guy is almost impossible around here.  We’ve loosened up our requirement for Juniper experience just to get someone in for interviews.  This led us to one prospect and an interesting story.

This guy’s resume was very impressive.  For the last 5 years, he’s been the Network Architect at a very large company.  His experiences were off the chart.  Large-scale Enterprise deployments.  Monster PCI environments.  Years of Juniper experience.  Years of Cisco experience.  I had to talk to this guy, so I got a phone interview with him.

I’m stuck deep in Junos these days.  I mean deep.  I have an F5 load balancer and an ASA 5520; the rest of my stuff is Juniper.  That means I have some learning to do.

Here’s one of the basics in Junos - configuring BGP.  I guess I’ve always said that BGP is BGP.  How much different can it  be from IOS?  Well, the end result is the same, but it’s different enough to have to look up how to do it.  :)  The first difference is the fact that all BGP configuration is done with groups just like peer groups in IOS.  You can act like you’re configuring neighbors, but there’s no way around using groups.  After going back and forth, I just settled with an group for eBGP neighbors and another for iBGP neighbors.  If settings are different, I just set them in the neighbor.  Here’s an example of that.

Here’s another story from the late night.  I’ve changed the details to protect the innocent, but you’ll get the idea.

I think most of you know that I started a new job late last year, and I’ve spent my waking hours getting caught up on how the new company works, how everything fits together, and all that jazz.  One of the big reasons that I (and a number of others) were brought in was to fix the biggest problem; the company doesn’t have a real central control over customer-facing technologies.  There’s a group that does central IT for the company (Exchange, SharePoint, Oracle apps, etc.), but there are dozens and dozens of applications out there.  That means there are dozens of “network teams” around the world doing their own thing.

Maybe not epic, but a win nonetheless.

My boss is over all the network guys in the company, and that includes guys that support different divisions and departments.  He told me he was tired of waking up at 2am every morning to fix a problem the other groups can’t handle, so he’s working to get the junior guys motivated to learn for themselves.  One technique he’s implemented is to force them to get their CCNAs and JNCIAs by June.  Since he made it part of the job description, that means that everyone above the Analysts has to meet those requirements, too.  I made the deadline with plenty of time to spare.

Oh, my.  Another Junos post.  Somebody stop me before I get my JNCIA!

This isn’t hard stuff at all.  I’m sure there are a couple of cool tricks I don’t know yet, but let’s try anyway.  I"m working on an SRX240 here running 11.1 and some change.

Let’s put interfaces ge-0/0/0.0 and lo0.0 in OSPF area 0. If you know the Junos configuration hierarchy, this will be very easy to you. Even if you don’t, you can stare at the config for a little bit and see what we’re doing.

The year is finally over.  Actually, it sort of snuck up on me.  I must be getting really old or something to let that happen.

At the beginning of the year, I posted my goals for 2011.  How did I do?  Not too well.  I batted .500, so feel free to boo me.

• Hurry up and finish CCNA Voice : I finished that on 7 February.  Was it worth it?  Not really.  I haven’t used the knowledge, and voice isn’t my thing.  I got it to spice up the resume, but it didn’t really come into play at all.  Oh, well.  It’ll expire in about 2 years.
• Pass CCIE R&S written exam : I got this one finally.  I flunked out at Cisco Live this year, but I redeemed myself on 23 August with a online pokie games pass.  Jody still owes me a drink since I hold the record for lowest passing score.
• Select a CCIE training vendor : Yeah…I never got to that one.  When I finally got through the written, my job had completely drained my motivation.  I fixed that problem by getting a new job, but that didn’t help free up any time to figure out which vendor I wanted to use.  #fail
• Schedule CCIE R&S lab : That obviously didn’t work out, either, since it’s dependent on selecting a training vendor.  #fail

What does this hold for this year?  Getting some training and scheduling an exam is obviously priority.  Since my new job is going all Juniper, going through those certifications would be next.  Another super-busy year, I’m sure.

Wow! A Junos post! Amazing.

We all know that the configuration on a Junos box is very hierarchical. Sometimes it doesn’t make a lot of sense, but it’s all a pretty cascade of code. One of the big messes that I’ve found is the VPN configuration hierarchy; there are way more items to configure than on an IOS device.  To reinforce the stpes in my head, I thought I’d get some of the pieces into a post. These aren’t all the options, but it’s all you need to get a static IPSec tunnel up and running.

Man, time is hard to come by of late.  I’ve had so little time to rest that’s it’s hard to get my thoughts together.  It’s a good thing in this case, though, since it’s my fantastic job that’s taking all my time.  It’s great to see new network and learn their internals…especially when they were designed by some long-time CCIEs who actually knew what they were doing.

One of the big things that I’m dealing with lately is VRFs.  I’ve implemented some VRF-lite stuff, but I’ve never had any practical experience with the full force of them.  I’m definitely learning here.  Since the blog here is really about my sharing what I’ve learned, let’s go through something that came up recently - terminating VPNs on one VRF while passing traffic to another.

I like to take a month or so off from blogging during the summer, but my CCIE R&S written studies pushed that back a bit.  I’ve finally got my lazy self back on track, but it may just be for a few days since I’ve accepted a new job in another city and am in the process of moving.

I am really excited about the new position.  Since I haven’t started yet, I’m not going to reveal who the company is, but you’ve all seen the name.  They’re forming a new group to handle specialty services for customers, and I’ll be working for the manager of that team as the Senior Network Engineer.  My future boss is a CCIE, so that’s a great start; we didn’t even have a CCNA at my current company until about 2007.  The job is going to be great, and the wife and I are both up for new adventures.

Here’s a story from last week with little of no teaching value.

I got a call from one of our business units looking for some routing help.  We don’t usually care about their production networks, but they were seeing some funky traceroutes, so I agreed to try and help them out.

They sent over two fresh traceroutes from a host on a 7600.  In one of them, the trace went to the 7600 and then on down the line as expected.  In the other, the trace showed the 7600, another router’s far interface IP (that is, an interface not facing the 7600), then the 7600’s interface facing that router.  Every few minutes, the path was switch between the two.  The dude told me that they were an OSPF shop, so I asked him to send me the standard show ip route and show ip ospf database commands so I could see what’s going on.  The word “unexpected” comes to mind when trying to describe what I found.  So do other words that aren’t very appropriate.

The wife and I had a romantic day driving several hours to a small town to take Cisco exams.  If this doesn’t get me some action, I don’t know what else to try.

I’ve already used the phrases “skin of my teeth” and “a pass is a pass” on Twitter today for good reason.  Passing is a score of 790, and I blew that away with a 790.  One more lapse in concentration and I would have been making up more excuses instead of smiling.  I think I’ve mentioned this before, but I have this weird reaction to taking exams where I don’t get nervous at all until after I’m finished.  Walking into the testing center, I was fine.  Walking out, I was shaking like Northern Virginia.  It was so bad that I could barely hold on to the door knob when trying to leave, so I guess that I’m really prouder than I thought I was.

This week we have a guest post from CJ Infantino. He is currently writes on convergingontheedge.com. You can find him hanging out on Google Plus as CJ Infantino or follow him @cjinfantino on twitter.

The other day I was adding VLANs to the the allowed list on the core routers at work. It was then a question came to mind, “Does the VLAN allowed list filter ingress or egress traffic?”.

Now, because all good engineers would configure the allowed list on both ends – as Aaron would say – in the grand scheme of things this really doesn’t matter, but being the inquisitive guy that I am, I wanted to know.

So I searched, and searched and google’d and could not find the answer. At that point there was only one thing left to do – lab it up!

I was studying via Google+ Hangout the other day with CJ and Rob, and one of the topics that came up was that OSPFv2 advertises all loopbacks as 32-bit no matter what the configured mask is.  I rarely use loopbacks outside of a lab and had no idea it did that, so I set up a quick lab to see for myself.  Sure enough!  That’s exactly what I saw.

Of course, being the inquisitive network guys that we are, we went on to discuss methods for making OSPF advertise the configured network instead of the single IP.  The guys mentioned two methods - to redistribute the connected interfaces and to manually set the OSPF network type on the loopback.  We were using IPv4 during the session, but I went back and added some IPv6 addresses and processes to compare.

It’s been a long time, eh? I’ve spent the last month or so with my nose down in a book and my mouse in a Google+ Hangout window studying my rear off for the CCIE R&S Written. Too bad I didn’t pass it.

The exam consisted of 77 questions over a 2 hour window. That’s plenty of time to finish; I think I had 48 minutes left when I was through, so time wasn’t a problem. There were only 2 or 3 questions where I was totally lost, so the technology wasn’t a problem. The big problem, like always, was the usual crap questions that are in these exams. Some didn’t provide all the required information. Some were impractical examples of deployments you would never use in the field. Some were on deprecated technologies. Hell, I had one that involved CatOS. Really? CatOS? Since I only failed by about 2 questions (like I always do), these shenanigans are magnified in my mind. It really irks me how these exams are being done; foggy questions don’t really measure ability.

• Tagging provides a way to mark common or similar routes to manipulate later.
• In redistribution scenarios with mutual redistribution on two different routers, any routes that gets redistributed from one route process to another are tagged.
  • When the other router sees those tags on the route, that route to keep from adding non-optimal routes to its routing table.
• Tags can also be used to do other manipulation such as setting higher metrics or changing ADs.

OSPF

This is required blogging…and reading for that matter.  A good chunk of this is taken from my CCNP posts from last year.  Corrections, please.

How does a BGP router decide which BGP route is the best?

Next-hop : Does the router have a route to the next-hop?

Weight : This is a numeric value where bigger is better.  Weight is not passed onto other peers and is a Cisco proprietary feature.

Make my corrections!  Please!

Well-known mandatory : These PAs must be recognized by all BGP routers and passed along to other peers.

Well-known discretionary : These PAs do not need to be in every update, but they must be recognized by all BGP routers.

Optional transitive : These PAs don’t have to be recognized but they must be passed along to other BGP peers if they are present in an update.

Corrigeme, por favor.

Open : When a neighbor is configured, the router sends an open to that neighbor to get the ball rolling.

Destination:  The neighbor's configured IP
Important fields:
  My AS

Update : The routing  information

Destination:  The neighbor's configured IP
Important fields:
  Advertised network Klonopin Online
  Path attributes

Keepalive : Sent every 60 seconds by default

Destination:  The neighbor's configured IP
Important fields:
  Nothing, really

Notification : When something is amiss, the router sends a notification message.  The receiver then closes the connection.

Corrections appreciated.

Idle : There is no relationship, but the router sends out a TCP SYN to the neighbor to get the ball rolling.

Idle (admin) : The neighbor is admined down.

Connect : The router is waiting for the TCP connection to finish.  If the TCP connection finishes, the router sends an open and transitions to OpenSent.  If it times out, it transitions to Active.

Active : The router tries Cialis to initiate a TCP connection.  If the TCP connection finishes, the router sends an open and transitions to OpenSent.

Per the standard rules, please correct anything that’s wrong.

One of EIGRP’s big features is the ability to use unequal cost paths for load balancing.  This is done with the variance command.

variance : A multiplier used to calculate which feasible successors can be used as active routes.  The router takes integer and multiplies it by the successor’s feasible distance, and any FS with a an FD less than this new number gets submitted to the routing table manager.

Please correct if I’m being stupid…which is a lot of the time.

Hello : Discovers and maintains neighbors

Destination:  224.0.0.10
Important fields:
  K values

Update : An update to the topology such as a route withdrawal or a metric change

Destination:  224.0.0.10 -or- unicast during neighbor discovery
Important fields:
  Message sequence number
  Route being updated including k values to compute metric

Query : Used to ask a neighbor if it has a route to a certain network; see casino online for free stuck-in-active

Corrections are always welcome.

Broadcast : Think an Ethernet segement

DR/BDR? : Yes Default hello interval : 10 sec Neighbor config required? : No

Point-to-point : Physical point-to-point links, frame-relay point-to-point subifs

DR/BDR? : No Default hello interval : 10 sec Neighbor config required? : No

Nonbroadcast Multiaccess : Frame-relay multipoint or physical

DR/BDR? : Yes Default hello interval : 30 sec Neighbor config required? : Yes

Point-to-multipoint : Partial mesh networks like a frame-relay hub-and-spoke configuration

Yes, it is inevitable that I cover these.  I’m sure network types will be next.  Per my usual request, please correct my stupidity.

Type 1 - Router : This LSA type lists all the routers by RID as well as the networks to which that router connects.

Type 2 - Network : These LSAs represent broadcast network where more than one OSPF router may live.  Think Ethernet or multipoint segment.  These LSAs are flooded by the DR for that segment.

My prediction about covering network types was wrong.  I’m going to puke out some information about neighbor states for now.  As is always the case, corrections are welcome.

Down : No hellos have been received from this router.

Attempt : This state only applies to manually-configured neighbors on an NBMA network.  In this state, a router has sent unicast hellos to the neighbor but has not received any back from it.

I have had my nose deep in several books in preparation for my CCIE R&S written exam, so I haven’t been blogging much at all.  Now that I’ve made it to the more familiar topics, I’m hoping to get some notes posted.  I’ll start with OSPF message types.

As always, please feel free to correct me here.  I’m learning just like the rest of us.

Hello : These messages are used to establish neighbors and serve as keepalives among other things.

For the first time ever, I’m headed to Cisco Live - the big Cisco users conference in Las Vegas! I usually don’t go to these things since I wind up just hanging out by myself, but I’m meeting all sorts of people there - from bloggers to Tweeps to personal friends. It should be a huge blast, and I can’t wait to get there.

For those interested, here’s my schedule.

If you’ve worked in any particular area for some significant amount of time, you have probably noticed that how much you think you know about a subject has changed over time.  This is nothing earth-shattering, and we’ve all had this realization over the course of our lives; it’s come up a lot lately in the course of my career, so I thought I’d share.

When you learn a new topic, your actual knowledge levels starts at a low point and gradually works its way up until you’re an expert (if you’ve lasted that long).  You start with nothing and learn more and more until you get bored and stop.  If you evaluate how much you know at intervals during the process, you’ll see that your self-assessment is more of wave than the straight(er) line of actual knowledge.

ACLs in IPv6 aren’t that different from what you’re used to dealing with in the IPv4 world.  You create a list of denies and permits for use with some other structure like filtering, PBR, and all sorts of other stuff.  Let’s take a look at building an ACL and filtering traffic with it.

For those playing at home, here’s the setup I used to generate the configs and get the output.  Execute some click action for the whole thing.

This is a surprise, but Cisco has announced the end of life of the 6500 switches that we all know and love.  Usually Cisco gives a platform a few more years after they decide to retire it, but the schedule only gives the 6500s one more year of service.  I’m sure this goes back the success and recent expansion of the Nexus line of switches.

Here’s the lowdown from Cisco.

Today was my last day at the office, and I start a new job and adventure on Monday.  What does that mean for the blog?

The biggest impact is the fact that I won’t have access to a CSM or FSWM any more.  These are two pretty unique devices, and I get quite a few questions on these guys.  I’ll try my best to recount what I know, but I’ll have to get information from buddies at the old job if research is needed.  The CSM has been a backbone of this blog for a while, so I’ll need to find another area with a similar demand for examples and configuration help.  The FWSM doesn’t measure to the same scale, but the old company casino online australia is the only one I know that runs those things, so the world may be that much poorer in FWSM information.

It’s pretty widely known that I hate Cisco 3750 switches. We’ve had so many hardware and software failures with them that I’ve got a seriously bad taste in my mouth. Since I’m leaving for a new company, I thought I’d publish some statistics while I still have access to the numbers.

Total TAC cases online casino usa european roulette opened related to 3750s: 21 Number of 3750G-12S-S replaced: 21 Number of 3750G-24TS replaced: 7 Total number of RMAs issued: 28 Total number of 3750s in the company: ~120 Failure rate: 23.3%

Are you sensing a theme lately?  Since we covered the basics of the main IGPs (I’m an enterprise guy, so no IS-IS comments, please.), I thought I’d try to describe the basics of advertising IPv6 routes over BGP.  Yet again, we’re not going to do any route manipulation or change any of the 948284928 BGP attributes.  We’re just trying to get routes exchanged.

Configuration

There’s no new version of BGP for IPv6 here.  It’s the standard BGP version 4 that we’ve all been using for years, but we’re going to take advantage of the multiprotocol support (MPBGP, RFC 2858 RFC 4760).  We’ll get to the differences in a second, but the first thing to do is to set up the BGP process as normal.  

I passed the IIUC yesterday, so now I’m a CCNA Voice.  It’s kind of belittling to get a CCNA-level certification at this point in my career, but I didn’t want to be completely left behind, so I figured I should move into some voice stuff before I’m left in the dust.

The exam was probably the best Cisco exam I’ve ever taken.  Of all the exams I’ve taken in the last few years, this is the only one that didn’t have questions with huge misspellings or grammatical errors.  I was really taken aback at that since a good portion of the questions from some of my recent CCNP exams were plain unreadable.  I think I remember leaving a comment on one IIUC question about the word “an” being left out of a description, but that was no big deal.  I’m not that obesessive-compulsive.

I’m not going to go all out like Jeremy over at Packetlife.net has, but I’m going to start to discuss a few IPv6 topics.  In time (like in September when APNIC runs out of IPv4 addresses), I’m sure I’ll ramp up the IPv6 talk, but let’s start easy and get EIGRP for IPv6 up and running.  

Configuration

There are quite a few differences between EIGRP for IPv6 (yes, that’s an official name) and the IPv4 version.  First of all, all IPv6 routing is disabled by default on a Cisco router, so, if you’re doing any routing in IPv6, you’ll want to enable it or risk smashing your head into the desk trying to figure out what’s going on.

RichardF commented on an article I wrote last November and mentioned the prompt command in the ASA.  I never set aside any time to research it, but I finally took the time today while waiting for a maintenance window.

This is one of those little things in life that make me happy.  Since the active ASA always has the same hostname and IP address, I find it hard to keep track of to which firewall I’m actually connected.  That “configurtions are no long in sync” message you get when you conf t on the standby firewall really irks me.  With the prompt command, I can see which firewall I’m on and in what state it is.

More IIUC notes.  As always, feel free to correct as needed.

To match inbound calls to a dial peer, CME (and CUCM?) uses the following steps.

1. Match DNIS (the dialed number) with the incoming called-address config in the dial peer
2. Match the ANI (the calling number or caller ID) with the answer-address config in the dial peer
3. Match the ANI with the destination-pattern config in the dial peer
4. Match an incoming POTS call to the port config in the dial peer
5. Match dial peer 0

Matching dial peer 0 is bad, and it took me an inquiry on Twitter and a buddy to realize why.  Here are a few highlights as to why.  I believe the full scope of the badness of dial peer 0 is really beyond the IIUC exam.

As always, feel free to correct anything that needs correcting or add anything that needs adding.  There is a lot more to the full definition of wildcards, but these are the basics.  Note to *nix guys:  This isn’t regex as you understand it.  Yes, the use of curly braces would be nice, but we don’t get that here.

T:  Represents anywhere from 0 to 32 digits

destination-patter 9T  <- matches a 9 followed by 0 - 32 other digits

Here are some packet overhead numbers for a few popular protocols to help with doing bandwidth requirement calculations.  This may be another add-as-we-go post, so please comment with additions or corrections.

Ethernet : 20 bytes
Frame Relay : 4 - 6 bytes
PPP : 6 bytes
MLPPP: 10 bytes
MPLS : 4 bytes

IP : 20 bytes

TCP : 20+ bytes
UDP : 8 bytes
GRE:  4 - 20+ bytes

I don’t usually cover news from Cisco, but they’ve changed some certification stuff around again, and I thought I would bring it up.  This time they’ve changed the CCNA Voice, CCVP, and CCSP, so, if you’ve on those tracks, be careful what you’re studying!

CCNA Voice

Circle 28 February 2011 on your calendars.  That’s when the CCNA Voice track gets a shakeup.  The IIUC (640-460) exam will be no more, and passing CVOICE (642-436) will no longer be a valid way to get the cert.  After the big day, you’ll have to take ICOMM (640-461).  This seems to be a much broader exam instead of having the enterprise and commercial focuses in CVOICE and IIUC, respectively.  Look out for both CME- and CUCM-based topics including a troubleshooting section. 

Here’s my solution to the exercise I posed last week.  Let’s see if we can get this right.

I’m going to assume you know how to give a router an IP address.  If you don’t, let me know and I’ll help you out.  We’ll use 10.10.10.1/24 for our CME router.

Let’s work on the telephony section first.  We need to limit our CME to 10 ephone-dns and 5 ephones.  Easy enough.  While we’re at it, we’ll have to give the telephony service a source IP address.  We might as well do the music on hold, too.

I tried something like this earlier this year with STP.  It got rave reviews (from my mother), so I figured I try it again.  

Below is a list of requirements for configuring a router as a call processor.  In a lab or in your head, configure the router to support the features as listed.  This isn’t a contest or anything like that.  If you get it right, a virtual thumbs up is all I can afford to give you.  There are some licensing issues for running this stuff in GNS3/dynamips, so I can’t help you out on that.  I’ll just hint that GNS3 and dynamips will bind to real networks and that copies of a compatible IP softphone are available.

More of my IIUC study notes.  As always, feel free to correct.  I really need to have a real post, don’t I?

show voice port summary

• Shows the voice ports available for use

R1#show voice port summary
                                          IN       OUT
PORT           CH   SIG-TYPE   ADMIN OPER STATUS   STATUS   EC
============== == ============ ===== ==== ======== ======== ==
50/0/1         1      efxs     up    up   on-hook  idle     y
50/0/1         2      efxs     up    up   on-hook  idle     y
50/0/2         1      efxs     up    up   on-hook  idle     y
50/0/2         2      efxs     up    up   on-hook  idle     y
50/0/3         1      efxs     up    up   on-hook  idle     y
50/0/4         1      efxs     up    up   on-hook  idle     y
50/0/5         1      efxs     up    up   on-hook  idle     y

• An ephone-dn shows up as efxs, so all these are ephone-dns.
• Channels are numbered 0-23; timeslots are numbered 1-24

FXS Ports

Here are some more notes from my IIUC studies.  As always, corrections requested.

Paging

• Broadcasts messages to a group for a one-way communication
• Paging groups are used to limit which phones get the broadcast
• Paging can be unicast or multicast
  • Unicast groups limited to 10 members
  • Multicast requires mcast support on the network
• Paging configurations can be unicast, multicast, or multiple-group

!  Unicast Paging
!  When 1044 is dialed, ephone 1 is paged
R1(config)#ephone-dn 44
R1(config-ephone-dn)#number 1044
R1(config-ephone-dn)#paging
R1(config-ephone-dn)#exit
R1(config)#ephone 1
R1(config-ephone)#paging-dn 44

Here are some more notes from my IIUC studies.  As always, corrections requested.

Local Directory

• Allows users to look up names
• Allows names to show up when dialing or receiving a call
• Most phones have a directory button; some have a menu options for the directory

R1(config)#ephone-dn 1
R1(config-ephone-dn)#name Roger Smith

• Directory entries can be added manually

R1(config-telephony)#directory entry 1 1700 Corporate Fax
R1(config-telephony)#directory entry 2 1701 HR Fax

• By default, sorting is done alphabetically by first name.
• Sorting can be changed

R1(config-telephony)#directory last-name-first

More study notes.  Correct if wrong, though I hope I get some of it right since I already since I’m an R&S guy.  :$

**Switchport Configuration
**

• switchport mode access:  This config makes the port an access port that carries the primary and voice VLAN traffic
• switchport mode trunk:  This config akes the port a trunk unconditionally, but it will still send DTP messages
• switchport nonegotiate:  This config keeps the port from sending DTP messages.
• switchport mode dynamic auto:  If the port receives DTP messages, it will become a trunk.  If not, it will be an access port.
• switchport mode dynamic desirable:  The port actively sends DTP messages trying to become a trunk.  This is the default configuration on a Cisco switch.

Cisco IP Phone Boot Process

These are some of my notes on my IIUC studies.  Since I am a novice as voice stuff, please let me know what I get wrong.

An ephone is a representation of a phone.  It’s basically a structure of features that a phone will have. 

Configuration in CME:

R1(config)#ephone 34  <– This is just a tag and has nothing to do with an extension or phone
R1(config-ephone)#mac-address 1111.2222.3333    <– Assigns this ephone to the phone with that MAC address

Feel free to correct anything that is wrong or incomplete.

• Power over Ethernet (PoE)

  • Can provide power to a Cisco phone, access point, security camera, etc., through the network cabling, eliminating the need to plug the phone into the wall for power.
  • Generic term for providing power on the Ethernet cable
  • Provides centralized power that can be put on a UPS
  • Allows devices to be located away from power outlets
  • Removes cabling clutter at the user’s desk
  • Can be provided through PoE-enabled switches, power panels or inline couplers (power injectors)
  • Oversubscription is common
    • If every device on a switch asks for full power, the switch may not be able to handle the load.
  • Of course, devices can be powered with a power brick at the desk

• 802.3af

Feel free to correct.  No need to sugar-coat it; I’m pretty new at this stuff.  :)

• Advantages of VoIP

  • Reduces costs of communications:  Eliminates/reduces long distance and international call tolls
  • Reduces costs of cabling:  No need for second network of phone lines
  • Integrates all voice into one large network:  All your remote offices can be implemented/maintained/controlled centrally
  • Provides mobility:  Moves, adds, and changes (MACs) are (nearly) eliminated since your phone is just a network node
  • Allows use of IP Softphones
  • Unifies emails, voice mails, and faxes:  All these can be treated as a single box for user messages
  • Increases productivity:  Ringing multiple devices at the same time eliminates phone tag.   <— pushing it, eh?
  • Enhances communications:  Applications can be launched/updated from a voice call through application servers
  • Provides open, compatible standards:  You can connect different vendor devices into the same VoIP network.   <— I’ve never seen that happen

• Cisco VoIP Structure

I’m way behind in talking about this, but Jeremy Stretch over at Packetlife.net has a community lab that is free to use.  This is a great resource for those of us who are too poor to have their own physical devices for Cisco studies.  All you need is an account on the site and a sense of community.

There are two labs to reserve, and each contains a firewall, routers, and switches.  This is plenty of stuff to get your feet wet with the gear, let you research some functionality that Cisco promised is great, and to lab out something you’re looking to implement.  The lab is offered for free, but Jeremy is giving his time and money for this lab.  I think it would be a great idea to drop a few dollars to him via his donate link if you use his stuff.   If you’re a regular user and don’t donate, I ask that you do a moral inventory on yourself so you might see just how bad you are being.

I’ve done it.  You’ve done it.  We’ve all done it.  You turn up another EtherChannel bundle and realize the hard way that your interface descriptions aren’t accurate.  Or you’ve swapped out a piece-of-crap 3750 and didn’t notice that the labels on the cables were wrong.  In either case, we all know that EtherChannel bundles don’t really work if the links aren’t plugged into the right switches.

So, what do you to make sure that your links are cabled the way you think they are?  Personally, I don’t trust any label at all - no matter if I did it or not.  At some point, someone has changed something on a switch, and that just might have been a change to where the port is question is cabled.  If I was onsite, I would hand-trace the cabling from one end to the other then do it again to make sure I didn’t hose it up the first time.  The big problem with this technique is that I’m not everywhere at the same time, and the travel budget isn’t very big these days.  If I can’t get my hands on the cables, I relegate myself to using CDP to see what’s on the other end of links when putting ports into EtherChannel bundles.

These are the notes I’ve taken as I read through the study materials.  Feel free to correct anything you see.

• Analog phone signaling

  • Misc
    • Ground = positive = tip
    • Battery = negative = ring
    • Signaling uses specific frequencies for specific events
  • Loop start signaling
    • When a circuit in the phone is completed (i.e., you take it off-hook), the CO detects it and provides services.
    • Susceptible to glare, where the phone requests dialtone at the same time that the CO sends a call.
      • Can connect two different calls if in a business with multiple lines
  • Ground start signaling
    • The circuit is temporarily completed to signal the CO for services
    • Doesn’t connect any call to any phone directly
    • Used in PBXes.
  • Supervisory signaling
    • On-hook:  Circuit is open
    • Off-hook:  Circuit is completed
    • Ringing:  AC current generated by CO to tell the phone to ring
  • Informational signaling
    • Gives information for the caller to use
    • Dial tone
    • Busy
    • Ringback: the ring you hear when you call
    • Confirmation:  the call is being attempted
    • Congestion:  no lines available to make the call
    • Receiver off-hook
    • Reorder:  can’t make the call
    • No such number:  can’t find the endpoint
  • Address signaling
    • Used to send digits
    • Dual-tone multifrequency (DTMF):  uses two electrical signals to indicate a digit; touch tone
    • Pulse:  flashes the circuit to indicate a digit; rotary dial
  • Disadvantages of analog signaling
    • Attenuation
    • Repeaters can’t differentiate between call and noise
    • One cable pair for each call; think about a pair for each call taking place in Manhattan right now

• Digitizing voice

I posed the philosophical question on Twitter the other day asking if single trunk links should be in an EtherChannel bundle just in case you need to expand later.  I didn’t really expect an answer, but the ever-verbose @WannabeCCIE pointed out (in not so many words) that you should watch your IDBs.  What is that?

That’s an interface descriptor block.  I admit that I’m not intimately familiar with them, bu they’re data structs in IOS used to keep track of the interfaces on that device.  They come in two flavors - hardware and software.  HWIDBs usually represent a physical interface but they also represent tunnels, SVIs, PortChannels, subinterfaces, and any other virtual interface that you can configure.  The SWIDBs represent the layer-2 encapsulation of each HWIDB, so you’ll see entries talking about Ethernet, HDLC, PPP, etc.  That means that every interface you have on a router consumes two IDBs (there are always exceptions).  That’s important because each platform and IOS version combination has a limit to the number IDBs that device supports.

Last week, @fletcherjoyce posted an article on his blog about his positive experiences with Cisco’s 3750 switches.  If you follow my complaints tweets, you know that I’ve had quite the opposite experience with them.  I would never pick on anyone, but I had to throw in my 2 cents.

I’m guessing here, but we have about 50 3750 stacks in the enterprise.  Most of them are pairs, you wind up with roughly 120 switches.  Since we’ve done about 20 replacements over the last 5 years, that means we have a 17% failure rate.  That’s pretty horrible, isn’t it?

Another year of Aaron’s Worthless Words has come and gone.  This month marks the third full year of blog posts for me, and things sure have changed since the beginning.

At first, this blog was just for my personal rants, but no one cares about that stuff (thus the title), so I looked to move on to something else.  I decided that I would go into the non-technical side of the network field, so I started talking about the Principle of Least Privilege and about cabling standards.  That got a bit boring, so I started puking out information on the Content Switching Module from Cisco since I couldn’t find anything worth a cracker outside of the documentation.  That was a hit, and the topics started expanding and expanding until we got to where we are now.  Today, the articles are published in online magazine and are being translated into other languages around the world.  Quite a change from complaining about drivers stopping in the crosswalk.  :)

We ran into an issue the other day where an application was setting the DF bit in IP packets to 1.  We thought it may be causing problems, so we looked at setting up a route-map to set the DF bit to 0.  It turned out to be a different application problem, but it was a good exercise in looking at what you can do with route-maps and policies.

I set up a lab in GNS3 to replicate and do some captures.  It’s a really simple setup.  R1 connected to R2 connected to R3.

For those that don’t know, when I say “stack”, I mean a group of 3750s connected together using the StackWise technology.  When you use a very expensive and very proprietary cable, your individual switches are combined into a single logical device.  This means you configure one device to control potentially many switches.

To the point.  I’ve spent the last few weeks replacing a mess of 3750s in stacks.  These guys are very easy to replace, but the big problem I find is getting the IOS version in sync.  When the RMA comes, it’s inevitably got a different version on it, and you’ll see something like this.

Woohoo!  I passed the ROUTE test this morning.  That means I’m done with the CCNP track!  :)

If you remember, I took it over a week ago and had some bad luck on it.  Alright, bad luck is the wrong phrase.  I didn’t study enough and failed it.  This time, though, I had a special weapon on my side - the ROUTE Foundations book.  I haven’t used the Foundations books before, but, I saw some tweets about this one, so I picked it up off of Safari.  In just a couple pages, I realized that I was reading the answers to several questions directly out of the book.  It was amazing.  I only studied my weak points and wound up with 144 more points than I did last time.  I can’t say that was entirely because of the book, but I must say it was a big reason.

As always, corrections are requested.

Study Questions

• I’ve got IGRP and EIGRP both configured with the same AS number.  What’s special about this configuration?

If both use the same AS number, then they automatically redistribute their routes into each other without using the redistribute command.

• When redistributing one IGP into another, where’s a good place to filter routes?

There’s no one good place, but at the router(s) that’s doing the redistribution is a good start.  There’s no need to send an IGP a bunch of routes it doesn’t need.

I didn’t do so well on IGP redistribution the last time out, so here’s some more stuff to study.  As always, feel free to correct.

Study Questions

• What three things are needed to be able to redistribute one routing protocol into another?

1. One or more links into each routing protocol 2. A proper, working config for each protocol 3. The addition of the redistribute command to one or more of the protocols

I took the ROUTE test today and failed like I usually do.  That makes me 3-4 on these P-level tests if you’re scoring at home.  Don’t worry, though.  I’m not giving up.  :)

In atypical fashion, I must say that the ROUTE test was a good test.  Let me say that again.  The ROUTE test was a good test.  I said good, though…not great.  There were a few problems with it that I’ll get to, but, overall, this is the best test I’ve ever taken for a Cisco cert.  The questions were very well-written and there were no obvious omissions or wrong details.  I failed this test because I simply didn’t put in enough work.

Corrections, please.  I skipped a bunch of BGP intro stuff to get to the juicy center.  I’ll see if I can come back later and finish the other parts for posterity.

Study Notes

• Is BGP route selection a controversial subject?

Yes.  If you ask 1000 network guys the best way to influence BGP, you’ll probably get 1000 different answers.

• At what position in the PA list of a BGP update do you find the weight attribute?

You don’t.  Weight is a Cisco-proprietary thing.

Corrigeme, por favor.

Study Notes

• What do IPSec tunnels give you when a branch office is on a broadband connection?

Privacy through encryption Authentication of the remote peer through ISAKMP Delivery of private data over the public Internet

• What do you need to configure to get your branch router talking to the Internet?

ISP connection configuration such as PPPoE or PPPoA DHCP server configuration for internal users NAT Firewall services like inspection and filtering

Study Questions

• Your boss says that ever host in the network needs to be converted over to IPv6 by the end of the day.  Which of multipoint tunnels, point-to-point tunnels, or native IPv6 would be the most appropriate to use to help with that conversion?

Native IPv6

• The engineering department wants to permanently use IPv6 on their test boxes in two offices.  Which of multipoint tunnels, point-to-point tunnels, or native IPv6 would be the most appropriate to use?

Point-to-point tunnels

Study Questions

• Why would anyone develop a version of RIP that supports IPv6?

I have no idea.  Boredom, maybe.  Whatever the case, it works just like RIPv2, which is pretty scary.

• In EIGRP for IPv4, there are several requirements for two routers to neighbor up.  Which of those is not true for EIGRP for IPv6?

The two routers don’t need to be in the same subnet.  The concept of the link local address takes care of that need since neighbors always share a common medium like an Ethernet segment or a serial link.

Study Notes

• Exactly how big is an IPv6 address?

It’s 128 bits long.

• This shouldn’t be on the test, but how many unique addresses is that?

That’s 2^128 or a “3” with 38 zeros after it.  That’s also 2^95 addresses for each person on earth.

• Surely we’re not writing in binary, are we?

No way.  IPv6 uses 32 hex characters.  Each character is 4 bits, so we wind up with 128 bits of data.

Feel free to correct.

Study Questions

• What’s the most primitive way to get traffic destined to a single host to use a different path than your dynamic IGP dictates?

Use a static route.

• What’s the most primitive way to get traffic sourced from a single host to use a different path than your dynamic IGP dictates?

Use policy-based routing (PBR).

• What’s the most primitive way to get traffic sourced from a single host and destined for another host to use a different path than your dynamic IGP dictates?

Use PBR.

As always, feel free to correct.

Study Notes

• When a router redistributes from one routing protocol to another, where does the router get the list of routes to redistribute?

From the routing table.  Only IGP A’s routes (not topology or successors) are redistributed into IGP B’s domain.

• What are two methods of filtering redistributed routes?

Use a route-map in the redistribute line or a distribute-list.

• Of the two methods for filtering, which one has more options?

The route-map method has more options.  You can match on all sorts of stuff, including an ACL or interface, and filter based on that.

As always, feel free to correct.

Study Questions

• When you redistribute OSPF into EIGRP, what are you really redistributing?

Routes knows via OSPF Networks of OSPF-enabled interfaces

• What’s the default cost of an EIGRP route redistributed into OSPF?

20

• What’s the default metric of an OSPF route redistributed into EIGRP?

There is none since EIGRP has all those nifty k-values that have to be processed.  Routes actually won’t redistribute without them.

Feel free to correct.  I feel like I’m missing a big piece here, so please fill in a gap if you see one.  Thanks.  :)

Study Questions

• How many area 0s (zero) can you have in an OSPF implementation

Just one.

• If my company merges with another company, and we’re both running OSPF, how can we get our networks routing together properly?

The easiest thing to do is to connect your two area 0s together through some physical link.  If you can, you can use virtual links to connect an ABR to another ABR to extend the zones together.

Feel free to correct all this stuff.  Additions are also welcome.

Study Questions

• How do I keep an area route from reaching a router in that area?

You don’t.  That defeats the whole purpose of having the topology database on every router.  If you filtered one route from a router, there’s no way that SPF could calculate routes correctly.

• Fine, then.  Where do I filter routes?

You filter routes on an ABR or ASBR.  Since routers only have the whole topology for their area, it’s safe to filter routes from another area or from a redistributed routing protocol.  On a more technical note, you’re filtering type-3 LSAs on an ABR and type-5 LSAs on an ASBR.

Feel free to correct.

Study Questions

• The obvious first question involves the common LSA types and their function.  Can you list them?

Type-1 - Router - Lists each router their connected IP addresses Type-2 - Network - Lists all the transit, or multiaccess, networks Type-3 - Net Summary - Defines a  host route for interarea routes; this is from the ABR Type-4 - ASBR Summary - Defines a host route for an external (to OSPF) route; this is from an ASBR Type-5 - AS External - Lists the networks advertised into OSPF from external sources (redistribution) Type-7 - NSSA External - External routes injected into a not-so-stubby area

Feel free to correct.

Study Questions

• What are the definitions of the hello and dead intervals?

The hello intervals is how often a router sends hello messages.  The dead interval is how long to wait before considering a neighbor dead from lack of hello messages; this is 4x the hello interval by default.

• How do you keep OSPF from trying to detect neighbors on an interface?

Don’t configure a network statement for that interface Make that interface passive

Corrections welcome.

Study Questions

• Why would you ever want to summarize routes?

Summarizing routes minimizes the routes advertised to the network.  For example, instead of advertising 192.168.0.0/24, 192.168.1.0/24…192.168.n.0/24, a router can advertise a single route to 192.168.0.0/16.  Keeping routing tables small saves hardware resources, minimizes convergence times, helps avoid route flapping, and makes the routing table easier to read for humans.

• When will an EIGRP router auto-summarize a route?

If a router has interfaces that that are in different classes of network (Class A, B, C), then that router will auto-summarize those routes up to the classful boundary.  For example, if you have a 10.0.0.1/24 and a 192.168.100.1/30, the router will advertise 10.0.0.0/8 and 192.168.100.0/24.

Or neighborships, as they call it in the book.  What a terrible word.

Study Questions

• What settings must match between two routers in order to become EIGRP neighbors?

Both routers must be in the same primary subnet Both routers must be configured to use the same k-values Both routers must in the same AS Both routers must have the same authentication configuration (within reason) The interfaces facing each other must not be passive

Study Questions

• How do you keep EIGRP from killing your WAN?

You can use the ip bandwidth-percent eigrp AS X command to limit the amount of bandwidth that EIGRP uses to update neighbors.

• How does EIGRP calculate how much bandwidth it can use for each frame relay PVC?

By default, EIGRP takes 50% of the (sub)interface’s configured bandwidth (with the bandwidth command) to use for updates on NBMA (non-broadcast mutliaccess) networks like frame relay.  This value is divided equally among all the PVC configured on that interface.

I’ve seen and used the command before, but I’ve never really seen any use of the show ip protocols command until tonight while reading up for my ROUTE test.  There’s a lot of good information in the output, and, from the way the book is reading, this is a great candidate for use in a lab question.

To check it out a bit, I set up a small network with four routers connected only to a single Ethernet segment.  I set up one router to run EIGRP, OSPF, and BGP to each one of the other routers just so I could see the output for the different routing protocols.  Here’s what puked out after struggling with GNS for a few minutes.

Last time, we talked about a nifty little lab I set up for redistribution and how the OSPF ASBRs acted a little differently than I expected.  This time, let’s look at how changing external OSPF routes to a metric-type of 1 (E1) affects the routing tables.

Here’s the network again.

The static routes are being redistributed into their respective IGPs, and EIGRP is being redistributed into OSPF.  Let’s look at the routing table on R1.

I just got back from Global Knowledge’s ROUTE class, and I must say that it was a great class.  John Barnes puts on quite the show and is the best instructor I’ve ever had.  I digress, though.

One of the topics we covered was route redistribution, so I went back to the hotel one night and fired off this network in GNS3 to study a bit.

The object was to see how redistributing statics into OSPF and into EIGRP differ.  It was also an opportunity to see how EIGRP redistributes into OSPF (and OSPF into EIGRP, but I didn’t make it that far).  To do that, I redistributed 10.10.10.0/24 from R1 into OSPF and 10.10.20.0/24 from R4 into EIGRP.  I then had R2 and R5 redistribute all EIGRP routes into OSPF.  It’s a nice mix, but I saw some weirdness in the paths to 10.10.20.0/24.

Just because I like giving more money to Pearson Vue, I took the BCMSN test today to see how I would do.  I passed with no problem.

In my mind, the CCNP is a technical certification, so I expect to be tested on technical topics.  Are there topics beyond technology that P-levels should know?  Of course there are, but I really don’t think whole chunks of the test should be about a preparation plan and rollback procedures.  The BCMSN had a lot more technical questions at a much higher level of expertise; it seems much better suited to the CCNP track than the SWITCH test did.

I did my standard 2ish-hour drive to the closest testing center today to take the SWTCH test (642-813).  Utter failure.  That’s 3 for those scoring at home.

The test was the absolute worst I’ve ever taken.  I know that I complain a lot, but this is totally justified in my eyes.  My 4th grade spelling tests were better than this.  I’ve seen kindergarten plays with better production value.

First of all, it was poorly written.  Whoever wrote those questions has a few pieces of information about English sentence structure missing from their skill set.  A sentence needs a verb, right?  Well, a lot of the sentences were missing those.  It’s kind of important to know what the whole point of the sentence is, or is that too much to ask?  The “drag this over here” exercise questions all started with the same 13-word phrase that left the question so long that it was unreadable.  A couple of commas would have been nice in some.  Others I just had to infer from the answers what they were trying to ask.

Certain divisions of the company tend to shoot themselves in the foot by kicking off large file transfers during business hours, so I had a thought that maybe we could use time-based ACLs to do some QoSing for those guys. I fired up GNS3 with a 3600 running 12.4(25b) with some virtual PCs on it’s Ethernet interfaces.

time-range BUSINESSHOURS
 periodic daily 8:00 to 17:00
!
ip access-list extended PINGS
 permit icmp any any time-range BUSINESSHOURS
!
class-map match-all PINGS
 match access-group name PINGS
!
policy-map PM-F0/0-OUT
 class PINGS

First, I set the router’s time to outside of the time range and sent some pings over.

I’ve got a few switches daisy chained together with single links and have enabled UplinkFast on them.  This switch is not the root bridge; F0/24 is the root port and F0/23 is a blocked alternate port. I’ve got debug spanning-tree uplinkfast on to help out.

SW3#sh span | incl 0/2[34]
Fa0/23           Altn BLK 3019      128.23   P2p
Fa0/24           Root FWD 3019      128.24   P2p

Now let’s unplug F0/24 and see what happens.

Did you guys have any trouble with the solution to the STP exercise?  Let’s work through it and see what happens.  I got a few responses to the solution, and everyone seems to get the same answer, so I assume we’re all right.

Before we get started, I wanted to mention the tie breakers since there can be ties in STP.  If there is a tie in any calculation, the same tie breakers are used, so I’ll list them here to use as we move through the calculations.

Here’s an STP exercise for you.  Given the bridge priorities, MAC addresses, and interface types in the diagram, calculate the root bridge, root ports, designated ports, and blocked ports.  You can click on the image to enlarge it.  I’ll post a solution in the next few days.  As always, feel free to comment and ridicule my utter idiocy.  Be gentle, though; I don’t usually post exercises like this.

Send any configuration BPDUs questions my way.

Yeled left a comment earlier this week asking if I’d seen Google’s Capirca.  I’d heard of it and checked out some presentation slides on it, but I’d never actually tried it out, so, in keeping with the script, I downloaded it to see what it could do.  Remember, now, that I’ve been playing with it for about 2 hours now, so I’m no expert on its use.

Capirca is a Python-based solution that Google came up with to automate ACL creation on their many thousands of routers around the world.  You can’t blame them for wanting to automate it, either.  How many times do you think they ran into problems with typos or keying errors from their network guys across those devices?

Two down, two to go.  After much groaning and moaning, I’ve finally passed my ONT test.  The path to this point has been full of road blocks and covered in potholes, but I finally managed to power through it.  Thank $deity.

If you remember, I’ve had quite a time with finding a testing center that’s convenient (or open for that matter), so I took the test at yet another center to see what they offer.  The facility was great; it was very quiet and clean, and the people were wonderfully friendly, which is a new concept to me.  Usually, the people don’t care about testers, but, being a center for inmates at state prisons (yes, prisoners), they do nothing but vocational and professional testing there.  That’s a lot better than the facilities who give their own students priority or who make money on training instead of testing.  The center is just over 2 hours away, but I think this place may be the best so far.  I’ll have to see what the future holds, though.

It’s not what you think.

I was talking with a buddy online last night, and he made a good point.  If you keep putting off taking a test, you’ll never make any progress.  I took that to heart, went online, and scheduled another sitting of ONT for today at 3pm at the closest center.  I took the day off, too, so I could get some tax stuff done and get over to the center and back before dinner.  I got some really good rest last night for sure, too, and had some very productive study time before heading off for my day’s adventures.

*  For definitions of “changed” and “Internet”

Today Cisco announced their new CRS-3 that replaces the CRS-1.  The CRS-3 has some damn impressive numbers for sure with 322Tbps, or about 1 LOC/sec (that’s a Library of Congress per second).  In three to five years, it might enable some technologies that we can’t use today, but I think “chang[ing] the Internet” is a bit of a stretch.  I’m sure it’s ultra-cheap, too.

I’m still working on the ONT test and doing labs, so I marked up a lab for me to work.  I’m using the same setup as I did last time.  The two routers are 3640s running 12.4(25b).

Part of the lab was to identify HTTP traffic coming into F0/0 and mark it as CS3.  That’s pretty easy, right?  Of course, the lab I made up was a little more complicated, but the point comes clear with a simpler example.

I failed the ONT test today.  It was an utter lack of subject matter knowledge that did me in from the beginning.  When the first three questions mention things that I’ve never even heard, it’s going to be a long test.  I’ll take blame on it for sure, but the test was a lot darker than I imagined it would be.

I heard from a couple people that the ONT test was the easiest of the 4 CCNP test.  I must say today’s test was a LOT harder than the ISCW test I took back in December.  Most of the questions were fair, but there were a few that were down-right evil or unanswerable.  Without giving too much away, there were some matching questions that had multiple items with multiple answers, rendering the answer to a guess.  I even ran into a CLI question about the WLC, which surely wasn’t mentioned anywhere I studied, and I don’t have a spare sitting around on which to test.  The icing, though, was the number of questions about FRTS; I know I need to understand it, but the magical question dice landed on that topic way too many times in my opinion.

Elements of Cisco Unified Wireless Network

• Client devices - Cisco compatible extensions on WLAN clients
• Mobility platform - allows configuration of LWAPs through WLCs
• Network unification - integration into the rest of the network with WLCs doing RF management, IPS, etc.
• World-class network management - centralized management through WCS
• Unified advanced services - supports advanced technologies and threat detection

WLAN Implementation

Autonomous and LWAP

Wireless LAN Services Engine (WLSE)

• Traditional WLAN weaknesses
  • SSID for security
  • Vulnerable to rogue APs
  • MAC filtering for security
  • WEP
• WEP weaknesses
  • Disribution of static keys is not scalable
  • WEP keys can be cracked easily
  • Vulnerable to dictionary attacks
  • No protection against rogue APs
• Benefits of 802.1x
  • Centralized authentication through Radius via AAA
  • Mutual authentication between client and auth server
  • Can use multiple encryption algorithms (AES, WPA, TKIP, WEP)
  • Automatic dynamic WEP keys
  • Roaming
• Requirements of 802.1x
  • EAP-capable client (supplicant)
  • 802.1x-capable AP (authenticator)
  • EAP-capable auth server

Table 1. Characteristics of the EAP variants

A coworker brought an interesting problem to me the other day.  He wanted to move a serverfarm from one server VLAN to another without taking an outage.  Since I didn’t want to have to come into the office late at night to do work, I decided to see what we could do.

It turned out to be pretty easy.  We tend to think of CSM VLANs as pairs – you have the client VLAN for the web servers where the vserver sits and the server VLAN where the serverfarm sits.  The CSM doesn’t know about these relationships; all it cares about is whether the servers are in a server VLAN, and we can use that to our advantage here.

Here are some more notes from my studies.  Of course, no one cares about them but me, but it’s my blog.  I’m sure someone will find it useful.  Please help to correct dumbass mistakes.

• Congestion

  • Speed mismatch - traffic leaves a lower-bandwidth interface than the one it came in on
  • Aggregation problem - lots of links with one egress of equal bandwidth
  • Confluence problem - a bunch of traffic needs to egress out of the same interface

• Queuing

Here’s another set of notes from my ONT studies.  I’m sure someone will find it useful.  Please help to correct dumbass mistakes.

• Classification is done with traffic desriptors

  • Ingress interface
  • CoS value on ISL or 802.1P frames
  • Source/destination IP address
  • IP Precedence or DSCP value
  • MPLS EXP
  • Application type

• Layer 3 QoS

  • Type of Service (ToS) is 8-bit field.
  • First 3 bits of ToS are the IP precedence.
  • First 6 bits of ToS are the DSCP value.
  • Last 2 bits of ToS are explicit congestion notification (ECN).

• Layer 2 QoS

I’ll try to keep it a little shorter this time.

Major issues for converged enterprise networks

• Available bandwidth: competition among applications
  • Fixes
    • Increase bandwidth: More power!
    • Properly queue based on classification and marking: QoS
    • Compress: cRTP, TCP header compression, etc.
• Delay: Lead time to get a packet to the destination
  • Types of delay
    • Processing delay: routing, switch delay
    • Queuing delay: how long a frame stays in an output queue
    • Serialization delay:  how long to put the frame on the wire
    • Propagation delay: the time to cross the physical medium
• Jitter (delay variation): Variation is the delay
  • Different delays mean different arrival times
  • De-jitter buffers save up packets to reduce jitter (like the old CD writers)
  • Fixes
    • More bandwidth
    • Prioritize sensitive data and forward first
    • Remark (reclassify) packets based on sensitivity
    • Enable L2 payload compression: make sure compression delay isn’t worse than the jitter
    • Use header compression
• Packet loss: Packets are lost in the network somewhere
  • Fixes
    • More bandwidth
    • Increase buffers space: more room for the queue on the interface
    • Provide guaranteed bandwidth: Queuing and QoS
    • Congestion avoidance
      • Random Early Detection (RED) and weighted RED (WRED) drop packets before the queue is full
      • Selective dropping is better than FIFO or LIFO dropping

QoS History

Here are some of the notes I’ve been taking while reading over the ONT book. I hope it benefits somebody.  Feel free to correct any stupid mistakes as a paraphrase to avoid a lawsuit.

There’s way too much info here.  I’ll refine the process a little better for the next topics.

Benefits of Packet Telephony Networks

• More efficient use of bandwidth and equipment - Packet telephony networks don’t dedicate channels or a static bandwidth to a call; it’s just another network application.
• Consolidate network expense - The common infrastructure (IP-based networks) keeps you from having to support another distinct network for voice like in traditional PBX implementations.
• Improved employee productivity - The phone can be used for more than just phone calls by utilizing the XML interface to run applications or provide content from the network.
• Access to new communications devices - IP phones can communicate with computers, network gear, PDAs, etc., and not just the PBX.

Packet Telephony Components

Here we are at the end of another year.  It’s been a pretty good one for me - the wife got a new job, we have a new house, we’re in good health.  I hope that you, too, have had a great year.

Here are some highlights from 2009 to read over while your boss is out on vacation.  I know that I’m a little wiser thanks to the many, many network-related blogs out there, and I hope that I was able to give someone an answer or just some new knowledge in 2009.

I took and passed the ISCW test today.  I was super-nervous going into it, which is weird for me, but I finally calmed down after the first few questions.  Here’s my take.  I don’t want to get into any trouble so I’m not going to include very much detail.

The testing center wasn’t very good at all.  It’s in an old building on the busiest road in town, and the noise from the street was barely dampened by the 1960s building materials.  I can tell you that there are three different pipes in the walls since their vibrations resonated through the room every time somebody flushed or brewed some coffee.  There was also a little foot traffic, which can be expected anywhere; they were working through some software problems on another testing station and were very respectful, so it wasn’t too bad.  The worst part of the whole ordeal, though, was the Microsoft class I sat through while taking the test.  They were across the hall, but it sounded like they were in the room with me.  Usually, you hear the instructor yelling at the top of his lungs so the whole class can hear, but I could hear questions being asked and papers being moved.  I think I can go pass a test of AD replication, though.  I certainly won’t be using that facility for any more tests.

I coworker sent over a link today that got me thinking about an old adage that I’ve been sharing for years.  The link actually has nothing to do with the philosophy but did trigger a random spewing of words from my brain.

Here’s what I tell everyone.  When I deliver these lines, I usually picture myself as Socrates talking to a bunch of Greeks in togas.

I don’t know if this really pertains to the ISCW test per se, but this is something I learned in my class last week.  I’m sure I should have learned this years ago, but, alas, I didn’t.

Access lists get messy.   You build one, apply it to an interface, and think all is well.  Then, ask for more access, so you may have to insert new entries between existing lines.  Your security team may ask you to deny access from a host while allowing it from others.  The next thing you know, you ACL looks something like this.

I’m at training for the ISCW test this week, and this topic came up yesterday.  Since it came up last week at the office, I figure it was a sign from $deity that it was time for a blog entry.

An admin in another business unit was trying to set up command access for some of his techs.  He was going through a couple of routers and assigning commands to privilege levels so that his techs could access them.  He was having a boat load of problems, though, and couldn’t get it to work

It looks like one of those Russian b*%*#rds got me some time last week.  I don’t know how long the site was down for sure, but I would guess that he first got access on Thursday, 22 October.  Since we’re talking about WordPress here, I just restored back to 15 October to be safe, and it looks like we’re back in business.

As a precaution, I’ve reset some passwords and deleted a whole mess of accounts.  I tried to leave the ones that look familiar to me like Blindhog and LBSources, but, if I killed your account, I apologize.  I’m afraid you’ll have to sign up again for the sake of security.

My biggest complain about modern firewalls is their lack of the ability to create rules based on URLs or HTTP streams; you have to open access between IP addresses.  Yes, I know there are other means to do that, but I want my ASA/PIX/FWSM to do it without making me do so much work.

Anyway, the fact that you have to use IPs brings up some interesting problems.  Let’s say you have a server in a DMZ that needs to query Google for some content.  Since you’re a hard-ass network guy like I am, you tell the admin that they have provide the data flow they want to use – source IP, destination IP, protocol, port.  They come back and tell you that they need their server to connect via HTTP to 74.125.45.100.  You put in the rules as given, but the IP has suddenly changed on you.

I can’t believe I haven’t talked about object-groups yet.  I had a whole other blog entry written up, and, when I went to link things over, I realized I couldn’t find an intro to it.  Here it goes.

Welcome to the modern world.  A world of wonder.  A world of quickly-advancing technology.  A world where clusters of machines sit behind load balancers for scalability and availability.  A world where those clusters need access to other clusters.  A world where your firewall rulebase gets so big that it’s unreadable without some help.

Wow.  A new entry.  Everyone sit down before you pass out.

I’ve got a real-world example for you today.  We have an ASA 5540 installed at a business unit with interfaces in multiple networks, including one containing the production servers and another containing the accounting servers.  The production network sits on a 7600 that’s not ours, so, to avoid IP conflicts, we are statically NATting connections into that network.  The 7600 has with many, many VLANs, and, since the firewall production servers are on different VLANs, there’s an interface VLAN between us.  Sounds pretty straightforward, but it just wasn’t working when we try to connect between the interfaces.

I apologize to my adoring fans (both of you) for the lack of posting.  I’m in the middle of moving, buying a new house, selling my current house, getting a mortgage, etc.  I’ve up until 11:30 nearly every night filling out forms and going through red tape.  Don’t get me started on getting money from a 401k!  Anyway…

I got in this morning, and a coworker was telling me that the data center’s HVAC was crippled due to an oil leak, and it was 90F in there.  D’oh!  It wasn’t quite that high, but it was warm.  Luckily, all of our network gear is on the end of the rows with AC, so we’re safe, but it got me thinking about monitoring temperature of our 6500s via SNMP.  I’ve done it via Cacti, but I never really looked how to do it manually.

I can’t believe I stuck with something for 2 years.  I usually get a month in before I get bored and give up.

Anyway, 2 years ago this month, I wrote a terribly-written article on The Principle of Least Privilege.  Since then, though, my writing has improved (at least, I think it has), the topics have broadened, and I’ve picked up some faithful readers (thanks, Mom!).

EtherChannel lets you aggregate links into one logical connection, but the distribution of traffic is not uniform.  It does not use per-packet load-balancing or the like to determine what interface in the bundle to use.  Instead, it uses a XOR function on packet information to generate a hash that is used to determine what interface to use.

By default, the switch will use both the source and destination IP addresses to generate the hash, but there are lots of others.

I’ve decided to take on the CCNP certification, so I’m going to wind up with a few posts will be more my own notes than anything.  :)

A switch port on a 2960 comes up with a default configuration on VLAN 1.  What happens from the perspective of spanning-tree?

• First, the port comes up on blocking mode.  This is to make sure that loops aren’t created without first listening to the network to see what’s going on.
• Next, if the port may be a root or designated port, the port is moved to the listening state.  In this state, the port can send and receives BPDUs only.  It can’t send traffic, but it can discover the other switches participating in STP.
• After the forwarding delay, the port goes into the learning state.   In this state, the port can send and receive BPDUs as in listening, but it can now receive traffic.  It can’t yet send any.
• After the forwarding delay again, the port goes into the forwarding state.  The port can now send and receive data.

If the port is configured with spanning-tree portfast, the mode goes from blocking directly to forwarding without going through these steps.  Obviously you don’t want a switch plugged into a port configured for portfast since you may wind up with a loop.

I’ve got a non-technical one for you today.  If you’re paying attention to stuff around you, you’ll probably end up with a little paranoia after reading this.

We’re having another circuit installed, and the LEC came out to do their end-to-end testing.  The tech, Dan, calls me up on the phone and tells me who he was and what he needed to do; I agree to meet him in the lobby to escort him on his way.  Now, I’ve never met Dan and can’t really vouch for him.  He had the polo shirt and khakis that we all come to expect.  He had a pile of generic-looking badges on his belt with his picture and name on them.  He had a satchel full of fulls and equipment.  He looked the part, but how hard is it to get a shirt, print up & laminate a few badges, and put some tools in a bag?  Was Dan really who he said he was?  Should I really have let Dan in the telco room?

SSH is more than just a shell.  You can copy files from and to a server or piece of network gear with it.  You can use it to tunnel traffic.  Possibly my favorite, though, is to use SSH to run a command on a remote box without interacting with a shell.

One of my biggest pet peeves with IOS (or pretty much any Cisco OS) is the lack of complex filtering.  Let’s say I want to look at all the downed ports and interfaces on modules 3 and 6 of my 6509.  I can’t easily do that with command from the IOS, but, on my Linux box, I can use multiple grep commands to get exactly what I want really easily.  Let’s work through the example, shall we?

This is a great one that I have to share.

A couple of coworkers walk in today and ask for some help on an issue.  It seems that a business unit was having latency problems with a web app, and, after research by the product team and sysadmins, nothing wrong could be found.  Lots of sites use the product, and only this one was having issues.  Also, the site was having no problems getting to other web sites and apps like Yahoo! or Google.

We discussed SPANs earlier, but let’s talk about RSPANs for a bit.

Can anyone guess what the “R” means?  You guessed it – “Remote”.  An RSPAN is a way to get traffic from a SPAN source on one switch to a SPAN destination on another switch that’s connected via a trunk.

The basic premise is that a special VLAN is created on all the switches and allowed to traverse the trunks.  You then set up a SPAN session that copies your traffic to this special VLAN.  This VLAN then gets the traffic to the other switches through some voodoo magic to be used as source for a SPAN on another switch.

I can’t believe I haven’t blogged on this yet.  SPANs are one of my favorite things in the world.

The switched port analyzer (SPAN) is a mechanism on Cisco switches that allows you to take traffic on one port and copy it to another.  It’s generally used to get traffic to a sniffer or IDS for analysis, but it’s a great tool to use to sample traffic from a host for troubleshooting.

I must be bored since I’m posting again.

A colleague asked me to change the failed value of a TCP probe today.  It was no big deal, but, when I looked to see the status of the change, I noticed interesting stati of the RIPs.

switch#sh mod csm 7 probe name TCP80-PROBE detail
probe           type    port  interval retries failed  open   receive
---------------------------------------------------------------------
TCP80-PROBE  tcp     80    20       3       120     10
Description: Quick fail recovery
recover = 3
real                  vserver         serverfarm      policy          status
------------------------------------------------------------------------------
192.168.1.45:80       VS01            FARM01        (default)       ???
192.168.1.44:80       VS01            FARM01        (default)       ???
192.168.1.43:80       VS01            FARM01        (default)       ???
192.168.1.42:80       VS01            FARM01        (default)       ???

It seems that when a change is made to a probe, the CSM discards the state of the probe and starts over.  If you catch it before the first probe is finished, you’ll get a status of “???".  I’m just picturing the CSM saying “Uhh…I…don’t…know”.

I’ve talked about probes and stuff on the CSM, but I never mentioned what happens to the connections to a server that fails.  That is, if I’m connected to server A in a cluster and that server suddenly commits ritual seppuku, what happens to my connection through the CSM?

Remember how the CSM works?  You connect to the VIP, some state tables are updated, your packet’s destination IP is changed to a RIP, and the packet is forwarded.  The point I want to emphasize this time is the state table.  If you were to send another packet to the same VIP on the same port, the CSM would look in its state table and see that you’re already connected to a server and just forward you on over after a NAT.  What if that server has suddenly died?

Earl Zmijewski of Renesys has an analysis of the SuproNet incident that took down a good bit of the Internet on Monday.  From the blog:

This single Czech provider announcing a single prefix caused a huge increase in the global rate of updates, peaking at 107,780 updates per-second. This peak occurred at 16:30:54 UTC, less than 8 minutes after the first announcement.

There’s a lot of noise on the Internet.  I’m not talking about certain news sites, either; I’m talking about stuff like port scans or attempts on weak services from all sorts of bad people on the Internet.  A large chunk of that noise can be filtered by the edge routers, taking some of the load off of the network and firewalls.

Here are a few things that we filter inbound on our Internet links.  Your mileage will vary.

Happy new year, all.  I’m finally over my hangover from the party and ready to blog.

Everywhere I go, I always wind up in a debate about how to alert on log messages as they come in.  I was at the grocery store yesterday, and the cashier told me that she had a list of log messages that she watched for, and, if she saw one of them, she sent an email.  I asked her what she would do if she got a log message that she had never seen before, and she said that she would have to find it first, then research the message and put in an alert for the next time it showed up.

Stretch at Packetlife has a lively little write-up on the Australian government’s attempt to implement a nation-wide web filtering service.

From Packetlife.net:

Setting aside the myriad of technical barriers to implementing such a system, the most obvious question is, “who decides what gets blocked?” When a corporation implements a web filter, it does so in accordance with corporate policy – policy that is set by the owner of the network. But the Internet doesn’t belong to any one entity, be it governmental or commercial, so such an authority simply doesn’t exist at this scale. In a very Orwellian sense, this filtering initiative appears to want to create that authority out of thin air.

I thought I’d throw an easy one out before taking off for the holiday.  Merry Christmas, Hannukah, Kwanzaa, Saturnia, etc., to all.

A few years ago, I was looking through some Cacti graphs of gigabit trunks between 6500s and noticed an abrupt change in traffic.  The graphs were nice and smooth at around 135Mpbs until, seemingly randomly, they just started going wild.  It seriously looked like a lie detector from the movies; I saw spikes up to 140Mbps in one sample and 2Mpbs the next sample for days and days.  I looked around to see if anything weird was going on somewhere on the network, but I didn’t find anything.

Did you catch the article on setting up fault tolerance on the CSM?  In that article, I mentioned that Cisco recommends a dedicated trunk for the FT VLAN if you have two HA CSMs in two chassis.  Discuss amongst yourselves while I drone on.

Why should you set up a dedicated trunk for this stuff?  The most obvious reason is to be sure that normal traffic doesn’t step on the syncing traffic.  Since we’re syncing state information as well as configuration, the frames need to arrive in a timely manner.  Any errors could potentially disrupt the FT process, which is bad.  You surely don’t want the primary to fail only to find out that the standby doesn’t have the complete or current config.

There are three different ways that a CSM checks for the health of the servers – active probes, inband health checking, and inband HTTP monitoring.  Let’s talk about active probes.

Active probes (or just probes) typically send traffic to one of the RIPs of a serverfarm, do some stuff, and give a pass or fail grade.  If the probe fails a certain number of times in a row, that server is considered sick and taken out of the pool for use.  The CSM keeps checking the unhealthy until it passes a number of times in a row, at which point it is placed back in the pool for use.  Almost everything is configurable, of course, so let’s look at some of those settings.

We ran into this today, and, though I knew it existed, I never actually saw it in the wild.  I’m talking about MAC access-lists.

In the example setup, we have a DMZ off of a firewall that contains a whole mess of servers – email, web, ftp, etc.  These should all be in the DMZ for sure, but they shouldn’t talk to each other.  If a bad guy was able to own my FTP server, he would have a nice platform to use to attack my email server.  That’s not cool, so we’ve put in MAC access-lists to help out.

Alright, that’s an exaggeration, but screen is pretty freaking cool.  It’s an app that’s (usually) run under Linux that lets you run commands then detach from that session and reattach later.  It doesn’t seem like much, but a few examples can show what it does for me.

I have a backup script at home that takes a target file, tars up everything listed in there, zips up the new file, and puts it on an external drive.  It’s very simple but takes about 3 hours to run.  I run it manually, so, in normal circumstances, I have to SSH in to my box and keep that window open for 3 hours while the backup runs.  With screen, I can open a new shell, run the script, and detach from it while everything gets backed up.

The CSM is pretty bad little box.  It not only watches layer 4 items like TCP connections, but also talks HTTP, which you can use to do some custom, or policy-based, load balancing.

Policies are the objects that make custom balancing work.  Like everything else (it seems) on the CSM, a policy is an object made up of other objects – maps and serverfarms.  A map matches patterns based on a number of things including the URL and HTTP header values, while the serverfarm directive tells where to send traffic that matches the map.  If, for example, you want to send all requests with “/admin” in the URL to a management server instead of the regular web servers, you can do it with a policy.

Here’s a handy list of ACL entries to allow your devices to speak routing protocols, availability protocols, and some other stuff. We’ll assume you have ACL 101 applied to your Ethernet inbound; your Ethernet has an IP of 192.168.0.1.

• BGP : Runs on TCP/179 between the neighbors

access-list 101 permit tcp any host 192.168.0.1 eq 179

• EIGRP : Runs on its own protocol number from the source interface IP to the multicast address of 224.0.0.10

access-list 101 permit eigrp any host 224.0.0.10

If you’ve never used TCPDump before, you’re missing out on one of the best parts of being a network guy – pointing fingers at everyone else.

TCPDump is an open-source app that copies packets on a machine’s NIC to screen or to file. TCPDump is typically a Linux/Unix app; in the Windows world, TCPDump is replaced by WinDump or Ethereal, now known as Wireshark. It’s a must-know for network dude(tte)s since it lets you capture the packets that a machine is generating. An app may be documented to work one way, but I’ve seen many times where the documentation is out-of-date or just wrong, and I’ve had to look at captures to see what it was actualy doing. I used it one time way back when a developer told me the switch was changing his HTTP POST to an HTTP GET; I captured the packets he was sending, pointed to the GET, and never answered a phone call from him ever again.

Here’s a simple one since I haven’t updated in a while. I have my ASA 5505 at home and want to forward TCP/80 traffic to my public IP to my webserver at 10.10.10.10. There are two steps here – forward the port and open the ACL.

To forward the port, I would use the static directive, but there are two ways to do that. I can either set up a one-to-one NAT or a port redirection. In the one-to-one NAT, you have a outside address that’s mapped directly to an inside address, and any traffic to that IP is passed to the inside host (if it passes ACLS, of course). One of the limitation, though, of using this setup is that you can’t use that IP as your PAT address, and, since I only have one IP, no other inside hosts would have a outside address to which to be NATted. The other method – port redirection – is a much better solution. In this setup, I actually forward a protocol/port on a outside address to a protocol/port on an inside address. Since there are other ports available on that outside address, the address is still available for other hosts to use as a NAT address.

We run a large number of LANs all over the country that are “controlled” by the particular business unit. We manage the gear, but, since they have the money and have to pay for anything we do, they make the final decision on what gets put in. Sometimes that gets out of hand, as you can well imagine.

A good terrible example came up a few months ago. It seems that, at some time in the past, one site needed some more LAN ports, but, instead of calling us and having us send them another switch, one of the “technical people” there brought in a hub from home. It really irks me to see a hub on the switched LAN, but we really have no control over those decisions. They plugged the hub into one of the existing drops somewhere in the building and plugged everyone in. It worked…until somebody moved one of the machines. The machine was at a desk near the hub, and the network cable, still with one end plugged into the hub, was just left lying there. A good Samaritan came by, saw that the hub was not plugged into the network (though it was through another path), and plugged it back in for us – providing a nice second link from the hub to the switch stack in the closet. Take one switch stack, add a hub, insert a switching loop, bake at 350F for a few milliseconds, and you have a broadcast storm. If you don’t know already, broadcast storms are bad and eat switch CPU like the yummy cookies we baked. In this case, several 3750s were taken completely down.

Here’s a scenario I ran into long ago. We had several sites that had a frame relay link back to headquarters and a DSL line. Each link was terminated into a different router on a flat LAN with the users. The DSL was for Internet access, but also terminated a VPN as a backup to the frame circuit. The requirements were something like this.

• Corporate traffic had to go across the frame relay link during normal operations.
• Internet traffic had to go across the DSL line during normal operations.
• If the DSL circuit went down, Internet traffic should be moved over to the frame relay circuit to use the corporate Internet link.
• If the frame went down, traffic should be sent out the VPN tunnel for access to corporate stuff.

We set the default routes of the machines (via DHCP) to the frame relay router. That router’s default route sent traffic to the DSL router, which, of course, had a default route towards the provider. Both routers were participating in EIGRP with the rest of the corporate network, so they all knew where to route traffic destined for corporate traffic. If there was a frame outage, the default routes kicked in and sent traffic to the DSL router, which had the VPN tunnels. The problem came when there was a DSL outage.

In my professional life at some point, I came across someone who had a stack of Catalyst 2950 switches all trunked together with their Internet routers connected to the top of the stack. This was all well and good until they kept adding hosts to the “middle” of the stack, then they had all sorts of latency and packet loss.

The old adage of your chain only being as strong as your weakest length holds true in this case. Here, the weakest link is actually the most-congested trunk, though. Let’s step through to see. A 2950 is a 10/100 switch, so a single trunk can handle 100Mbps of traffic. We have 10 of these guys, Switch1 to Switch10, all trunked to the one above and below. If a server in the center of the stack on Switch5 is sending a lot of data to the Internet routers on Switch1, the trunks off of Switch5 will start to get saturated. Switch4 has a few hosts doing the same thing, so traffic from both Switch4 and Switch5 heads towards Switch1, further filling the trunks. Same for Switch3. Same for Switch2. Next thing you know, there’s 184Mbps or so trying to go across a 100Mbps link.

VLAN Trunk Protocol (VTP) is a little gem on Cisco switches that allows you configure VLANs in one place and have them appear on all of your switches. This is great for large enterprises with 8457839 switches all trunked together because who wants to configure the new VLAN for that one-off application on all 8457839 switches?

VTP works by having designated VTP servers (not real servers like your Linux box, but a switch) tell the rest of the switches in the network with what VLANs they should be configured. All the designated VTP clients say “OK” and configure themselves with those VLANs. When you take a VLAN out of the server, all the clients take it out; when you add a new VLAN, all the clients add it as well. The server and client designation is known as the VTP mode, and there’s one more to mention. When a switch is in VTP transparent mode, he will see VTP from the servers but will ignore them and pass them on to the next switch as if nothing ever happened.

A lot of IOS commands give you a lot of information. Most of the time, though, it’s way too much information, and it sure would be nice to do some grep-like stuff on the output, right? Well, just like on Linux, you can use the pipe (|) to do such. That’s not a butt cheek, by the way.

The most useful function is the include directive. This is the equivalent of just plain grep on Linux, and will show you only lines that match a string that you give it. Say that you want to find what ports on your switch are down, but don’t want to grind through all the lines of a show ip interface brief. If you just pipe it to the include command followed by the word “down”, you’ll see something like this.

I realized the other day that I haven’t mentioned EIGRP once. As a Cisco guy, I think I’m required to do at least one article on it, so here it goes.

Enhanced Interior Gateway Routing Protocol (EIGRP) is a Cisco-proprietary routing protocol. Routing protocols share routes, right, but “interior” is the keyword here; it’s used to distribute routes on your internal network (Contrast that with BGP, which is allows you to share your routes with others). In a nutshell, each router in the EIGRP cloud tells everyone what subnets it has connected to him.  A receiving router then combines that information with everything that it already knows and passes on any new information.  Do that recursively for a while, and, eventually, every routers knows all the subnets in the network.

We just talked about tagging traffic and policing traffic, but we haven’t talked about prioritizing traffic. Tagging just sets a value in the header. Policing sets a “bandwidth ceiling” that can’t be crossed. Prioritization guarantees a certain amount of bandwidth for a flow/app/etc. no matter what’s going on.

Prioritization offers you a certain amount of bandwidth; it doesn’t carve it out and hand it over. If you’re using less than the priority value, you only get as much as you need and the rest of the reserved bandwidth goes into the pot for everyone to use. As priority traffic grows, though, you’re given as much as you need up to the configured value. When you go over that, your extra traffic just goes into the best-effort queue with everything else (Note: Don’t go over the limit with VOIP traffic. Echoes and artifacts suck). For example, if you give your VOIP traffic 70% of the bandwidth of an interface but are only using 40%, the other 30% can be used by the other apps on the line. If you’re using 80%, that 10% over is competing with everything else for bandwidth.

I use Google Analytics to track the 2 or 3 hits I get a day, and sometimes I see some interesting search terms. Yesterday, some googled up the term “does the ASA 5505 run HSRP”; I think that deserves a short article.

The ASA and PIX firewalls don’t actually run any of the usual HA solutions you use on routers. They don’t do HSPR, VRRP, or GLBP at all. Since firewalls have all sorts of state tables, connection tables, translation tables, blah, blah, blah, they need to share more information than just if they’re alive or not, so they use different methods to provide HA.

Let’s keep going with our example setup on the ASA 5505 and set up DHCP on this guy. You can set it up to either forward (relay) DHCP requests to a DHCP server somewhere or have it be the DHCP server. Let’s do it.

To set up DHCP forwarding, you have to configure where the DHCP server is and then enable the relaying on the proper interfaces. Let’s say we have a DHCP server on the inside interface at 192.168.14.11 and we want it to serve IPs to the guests network. Setting up the DHCP server is beyond the scope here, so you’ll have to look elsewhere on how to set that up.

Believe it or not, I got a request for an article on how to configure GLBP. I’m as shocked as you are, so here it goes.

The Gateway Load Balancing Protocol (GLBP) is another Cisco-proprietary protocol for providing highly-available gateways on a network…but there’s a twist. GLBP, as you can figure out from the name, load-balances the traffic going through the participating routers. With HSRP and VRRP, one host is the active peer and handles all the traffic until it dies, then another peer takes over. With GLBP, all the routers accept traffic.

I finally got my ASA 5505 up and running at the house, but I ran into a little problem – the box wouldn’t add the DHCP-provided default route into its routing table.  That one threw me for a loop since the box is made for SOHOs, but it makes sense in some corporate, lazy way.

I got an IP from the DHCPD on the 5505, but I couldn’t get to the Internet.  I checked the console, and it had an IP from the provider, so I checked ACLs; those were fine.  I looked at the log and found this.

If you didn’t now already, trunks are connections between switches that carry traffic for all VLANs. It allows you to have, say, VLAN 10 and VLAN 20 on two switches appear as the same network. Unless you’re a really small shop, you’ve already dealt with trunks, so there’s no need for an introduction.

Let’s say we have a Catlyst 2950 switch with multiple VLANs connected to another 2950 configured with those same VLANs. We’ll say we have VLANs 10, 20, and 30 and that the switches are connected to port F0/24 of each switch. First, let’s turn on the trunk.

HSRP (Hot Standby Router Protocol) is a Cisco-proprietary method for supplying a highly-available gateway for hosts to use. GLBP (Gateway Load Balancing Protocol) does the same thing. So, what’s the difference?

HSRP works on layer 3 and provides a standby IP address for hosts on that network to use as their gateway (or other routers to use as a next-hop for a route). Two or more routers are configured with the standby IP on a broadcast interface (usually an Ethernet of some kind), and a passive election is held to determine the active router. This router answers ARP requests for the standby IP with a virtual MAC address, so every host that sends packets to the standby IP winds up sending it to the active router. If the active router dies, another election is held, and a new king is crowned who answers for the virtual MAC; the hosts never know anything happened.

I was configuring a switch the other day and realized I had configured a trunk on the wrong port. God, I hate that. Instead of dumping the configuration for the port and doing a “no” on each line, I used the default command.

Switch(config)#default interface G0/1

This resets the configuration on interface G0/1 to how it was when Cisco shipped it to you. Much better than killing all the lines of the configuration one-at-a-time, eh?

NATting sucks and can be confusing. I’m sure everyone agrees to that, but you have to use it at some times. In a PIX/ASA, it’s easy to configure a simple setup, but can be super-complicated in larger networks. In a simple lab, we have set up an ASA with inside and outside interfaces, with the inside as your internal and outside as the Internet.

The NAT setup here is easy.

There’s a very-overlooked feature of access-lists – the remark. Yes, this is very basic, but it’s worth mentioning, as it has saved me anguish time and time again.

I use remarks to document each line of an ACL (on IOS, PIX, FWSM, ASA, etc.) so that when I go back later, I actually know what I did. They’re simple to use, and, I promise you, you’ll thank yourself for using it when the CTO asks why access to TCP/80 is open from the Internet to the development server.

We all have these at our desks. Not the bluetooth guys for your [tag]phone[/tag] (we could talk about that for a while), but the 900MHz headsets that your company gave you for those long and annoying calls with the boss. These things rocks, but they are oh-so [tag]insecure[/tag].

A coworker who fields support calls has one, and we decided to see how far we could go with these. We were shocked to discover that he could field a call 2 full stories downstairs from his desk. I was able to take mine 1 story away without even a single bit of static in it. I’m sure I could have taken it farther, but construction kept me from going any farther.

I’ve been looking around at some lists and forums for technical help on Cisco gear, and one thing keeps coming up – people new to [tag]Cisco[/tag] devices don’t know how to look at logs. The [tag]logs[/tag] are your friends and a great tool. You can use them to see what your router is doing, what’s going wrong, and even when something happened.

Get on your router and do a “show logging”. You’re looking at the log buffer where the router logs [tag]events[/tag]. If you don’t have anything in there, you need to run a “logging buffered informational” and “logging on” from config mode. This will turn on some logging at a basic level, and you should see some stuff going on. Keep doing a “show logging” and you’ll see the buffer start to fill up.

BGP has issues; the main one being transitive [tag]trust[/tag]. [tag]BGP[/tag] works by having networks (companies, providers, etc.) advertise [tag]routes[/tag] that it owns to its peers. These peers pass those routes on to their peers, ad nauseum, until everyone knows what networks everyone has. The big assumption here is that you are advertising only networks for which you are responsible. The word “assumption” should be emphasized.

The Pakistani government decided that a video on [tag]YouTube[/tag] was bashing Islam, so they ordered the Pakistani Internet services to block it. Instead of filtering from their network out, they decided to advertise via BGP that they were YouTube. To make things worse, they used a more-specific, 24-bit route; since YouTube advertises a 22-bit route, the new Pakistani route is preferred since its more specific. The transitive trust of the BGP cloud allowed them to tell the world that YouTube was on their network, effectively taking YouTube completely off the Internet for few hours. YouTube finally changed their advertising to a bunch of 25-bit networks, which restored connectivity, and, eventually, the Pakistani ASN withdrew the route. Here’s a timetable from Martin Brown of [tag]Renesys[/tag].

I deployed a Cisco ASA at a location and couldn’t get logged in via SSH. I would get prompted, but, no matter what username/password I put in, it would just reject me. After some digging, it turns out that I forgot this command.

aaa authentication ssh console LOCAL

When I put this in, it let me right in as expected. I have no clue what the deal was. I guess I assumed that the ASA would use the local userbase if a AAA service wasn’t configured. I guessed wrong.

Back in the day, when I used to put a new piece of IOS-based gear on the network, I would have to go through the gear already in production to remember what all those “little configurations” were that kept the devices running. Guess how many times I remembered to set the NTP server or turn off the HTTP server? Never.

To fix that problem, I started to keep a list of IOS commands that every IOS device on the network was configured with. That way, if I had another device to configure and deploy, I could just paste the list in and then do the IP and hostname stuff. It makes me feel a little more confident that the gear I deploy is more standardized and maybe even a little more secure.

I’ve run across articles for these apps a thousand times, so I thought I’d get in on the action. Dynamips and dynagen are a pair of apps that make simulating Cisco routers very easy. I use them constantly at the office (and even at home on the couch) to try out new configs and even new IOS versions.

Dynamips is the brains behind the operation. It was written to simulate Cisco 7200s for testing, but, eventually, it came to support several platforms, including 3600s, 3700s, and 2600s. You can use it to simulate a whole series of routers that are directly connected together through their interfaces, through virtual switches, or even connected to real interfaces on your box to pass traffic out through the real network. It uses real ]IOS images, so you can run whatever you can download. The problem with it is that it’s very complicated to use; if you did a fully-populated 7206, your command line would be 5 lines long and not make a lot of sense.

I like [tag]layer-3[/tag] [tag]switch[/tag]es. They give some great flexibility and bang-for-the buck, but most people overlook one issue with these things that can cause security problems. Most people configure the [tag]VLAN[/tag]s, put an IP on the VLAN interfaces, and put it in production, but the packets don’t actually flow the way they think they do.

Let’s check an example. Here’s what the proverbial you had in mind when you plugged your web server, management server, and firewall into your 3750.

I’ve seen a thousand [tag]firewalls[/tag] in my time, and nearly all of them are poorly configured. The biggest culprit? No [tag]outbound[/tag] [tag]filtering[/tag]. I guess a lot of people think that firewalls are there to protect the network from the Internet, but that’s only part of it. The firewall is to protect every segment from every other segment – all segments both inbound and outbound.

I guess that way back in the day that was true. You had your well-behaved network behind a firewall, and the only threat was from the evil hackers of the Internet. That’s not true any more, though. What about viruses? Or spyware? You don’t want those things spreading out from your network, do you? Think about liability, too. If you run a corporate network and an employee starts illegally downloading stuff from Kazaa, the company is liable for that, and the first step is to block any unneeded traffic from getting out.

Remember the article on router-on-a-stick? And the one on HSRP? Let’s add to that example network, shall we? Let’s make those routers into edge routers so they connect your internal network to the Internet with some size circuit. Let’s just say they each terminate DS3s to different providers.

Here’s our network now (I’m experimenting with Visio alternatives, so excuse the diagram footer there). Let’s assume that we have [tag]HSRP[/tag] set up like the HSRP article and that we have many sub-interfaces on the Ethernet side of the routers like the ROAS article. Also, Router1 is the HSRP active peer and each router has a default route pointing to the upstream ISP through interface Serial 0/0.

Ever heard of Samba? You should.

Samba is an open-source project “that provides seamless file and print services to SMB/CIFS clients.” That’s from the project’s website, but what the hell does it mean? In a nutshell, it’s an open-source application that lets non-Windows machines share files and printers with Windows machines. In most cases, people use Samba to share files on a Linux box in a really simple setup. I’ve read about several cases, though, where superhuman admins have used Samba machines to set up a Windows domain. I’m talking full-scale domain login, domain machine registration, and everything. I tried that once and all my Windows machines stopped working. It sucked.

In the article describing a router-on-a-stick, I mentioned that I would use two routers that run HSRP for availability, so I figured that I would write up a short post on what it is and how it works.

HSRP (Hot Standby Router Protocol) is a Cisco-proprietary protocol for establishing two or more layer-3 devices as a fault-tolerant gateway. Please note that it is not a routing protocol like OSPF or BGP. HSRP provides availability and fault-tolerance…it does not advertise routes. I actually found several Google results that said it was a routing protocol. Those were on the first page of the results, so be careful when searching! Webopedia.com is terrible.

Ever heard of a router-on-a-stick? Go ahead and laugh…everyone does. It’s a funny name for a very serious topic, though. A router-on-a-stick is a network configuration that uses a single router interface as a gateway for more than one network segment. You literally take a single Ethernet interface, put it on multiple VLANs, and set up the IP address stuff.

Here’s how it works: The router is plugged into a port on a switch that is configured as a trunk that carries all the important VLANs. The router is configured with Ethernet sub-interfaces (just as you do frame-relay or ATM sub-interfaces) – one for each VLAN. Piece of cake.

The Principle of Least Privilege says that users or applications should only have access to the what it needs to access and that access should be as limited as possible.  This idea can be applied to any number of things, but it is a very important topic when talking about security.

The idea is that processes, users, modules, or whatever can only access what they need to in order to function.   This keeps users in check since they don’t have any access to anything outside their home directories (or whatever).  It keeps developers in check since their code can only access a small set of files or processes.  It keeps hackers in check since the Apache server they’re hacking can’t access the password file.  It even keeps administrators in check since it forces them to use sudo, which is logged to syslog.

You can use an old PC as a firewall at home (and at work, I guess). It’s not that hard to do if you have a basic knowledge of Linux, DHCP, and IPtables, but that may be saying a lot.

Why would anyone want to do this, though? If you’re like me, you like to know what’s going on in the network. One of the Linksys routers you buy at Best Buy or Circuit City just doesn’t let you monitor very well. You can’t get very good logs off of it, so you don’t really know what it’s doing or complaining about. It also doesn’t let you query the interfaces, so you really don’t know how much bandwidth you’re using.  If you have a Linux box as your router/firewall/gateway, you can get really good logs, monitor the interfaces with SNMP, and have some really great, granular control over your network.