BlueWolf's Howl

« What a test! | Bluewolf's Howl | Next Step »

April 04, 2009

Oh, that Spanning Tree

Okay, so I know quite a bit about spanning tree right now. It's still fresh in my memory from the test. I know the difference between 802.1d and 802.1w and 802.1s. No problem. I know what it does and how it does it to provide the wonder of a loop-free Layer 2 per vlan path between devices. Still, I ran into a thorny spanning tree problem (in real life) that none of the chapters of material on spanning tree ever covered.

I did searches on spanning tree and compatibility between Nortel and Cisco - but that didn't help my problem either. Not a lot was coming up on that. The reason being -- it was not a spanning tree problem. It was a problem that affected spanning tree, but was not a problem with spanning tree itself. Quite a lesson on - just because this is the symptom does not mean it has anything to do with the cause...

Below is a diagram of the environment I'm discussing:
View image

Say you have VLAN20 on the Nortel Switches. It's happy with that. Yes, it's a happy Layer 3 vlan on a Nortel switch. All is well. Until you need to extend that for whatever reason. Say you need more ports for that vlan than the Nortels have available. Or say you want to migrate these devices to a Cisco switch. Well... just create a little Layer 2 VLAN 20 on the Cisco switch and link it up and you have an extension of that VLAN. We've done this several times and it works. What you do is create an access port in the Cisco (port 2/12) and connect it to an access port in the Nortel (port 3/8) and you're in like Flynn. So then you put port 2/7 into that same VLAN on the Cisco and give it a proper IP address, mask, gateway, blood sacrifice and you should be on the network.

Not this time - it can't ping the gateway. Perhaps the blood sacrifice wasn't good enough because it was only a paper cut? I dunno. But I have to figure it out and fix it. I know the basic concept works because we've already done a few of these. But that was about a year ago and it's not so fresh in my memory. Okay, so let's take a look.

Look on Nortel 25 -- it believes that it is the ROOT of VLAN 20. That's what we want. We want all 4 switches to see Nortel 25 as the root and we want 2/12 on Cisco 18 to be blocking - with a higher cost (easily configured). Look on Nortel 26 and it sees Nortel 25 as the root. Then look on Cisco 17 -- and it sees Cisco 18 as the ROOT! It also has 2/12 as the root port and Po1 in the blocking state. I look at Cisco 18 and sure enough, it sees itself as ROOT for VLAN 20. Both ports 2/12 and Po1 are designated ports and forwarding.

Arrrrgh! Dang that spanning tree. So how do we fix this? We have a Nortel VLAN 20 and a Cisco VLAN 20 that recognize the coincidence of the same last name, but aren't exactly sure they're related in any way. So let's take a look and try to convince them of the relation.

I'm hesitant to do much about Po1. It's a trunk and this is something that has been in place for quite a while. If anything's wrong, it has to do with the new stuff that was put in - which is reasonable. So let's look at those two ports.

Port 2/12 on Cisco 17 --
description
switchport
switchport access vlan 20
speed 1000
duplex full

Everything looks fine. Port 2/12 on Cisco 18 is the same. Okay, so keep looking because -something- is wrong somewhere...

sh int status (my favorite command)
All of a sudden I noticed that port 2/12 is a trunk. A trunk! Why is that a trunk? I don't want a trunk there! So then I looked at Cisco 18 and found the same thing. Dang. I didn't want a trunk there either! I looked even deeper and found the REAL culprit -- default settings.

The default for a port is dynamic desirable.

Although the port was a switchport - this made sure it wasn't a routed port. The switchport access vlan 20 ONLY put it in the vlan. It didn't insure that the port would be an access port. And the default dynamic desirable did what it was designed to do. It used the Nortel's as a 'shortcut' through which it promptly negotiated a 'dynamic desirable' - 'dynamic desirable' trunk with port 2/12 on Cisco 18. So I ended up (inadvertently) with a trunk between the 2/12 ports on the Cisco switches that were completely ignoring the Nortel VLAN 20 that they were using to create the trunk. Bastards.

I configured "switchport mode access" on both 2/12 ports on the Cisco switches. Trumpets started blaring and great rays of sunshine burst through the drop ceiling tiles. I heard a chorus of angels singing "Hallelujah" over and over. [No one else heard that - but, they must be somewhat deaf.] I checked all four switches again. Nortel Switch 25 was indeed the ROOT of VLAN 20 -- in the hearts and minds of all FOUR switches. We connected a laptop with the proper IP, mask and gateway. It worked!

My first reaction -- I gotta blog about this...

So good luck if you ever run into this yourself. At least now you know - it's not a spanning tree compatibility problem.

Posted by BlueWolf on April 4, 2009 10:48 AM