April 24, 2008

Lovin' the Shark

So my most recent method of procrastination happens to be viewing the Wireshark University course videos. I got lucky enough (read:whined enough) that my supervisor kicked in for the 4 DVD set of Wireshark U course materials. Of course, he figured I'd never look at them, but having them would at least shut me up...

I just finished the first DVD. It took hours, but the material was good and was presented in an interesting manner. We all have to admit - this is dry stuff. You're taking captures of network traffic and then looking at them under a microscope to figure out which one of the million possible things could be pissing on your connectivity parade. Okay, so there's really no microscope. That's an analogy. But when you look that closely, it can be tedious and the material can get very detailed. These videos help to keep you interested and capture your attention. And it's pretty much like actually being in a classroom course.

The first video basically focuses on how to use Wireshark. I had a little head start in that area, having already read the Ethereal book. And the Wireshark interface/GUI is somewhat intuitive. Basically, your average network admin can stumble their way through it and find the obvious problems. But, naturally, that's not good enough for me. I want to be able to go a step beyond that. These videos definitely help in that area.

Just within the first DVD, I've already learned how to do some customizations. These are the little things that will make an average person look more polished. And it will definitely help the captures and analysis go much faster. On top of that, Laura Chappell also lends us some of her tricks and tips. She shows us not only how to use this, but how *she* uses it. The section on graphing the data and displaying it professionally will definitely 'up' your game. You will not only know what you're doing, but you'll also *look like* you know what you're doing. Professionalism usually lends points to credibility.

I'm up to the second chapter of the second DVD (went through that tonight). The second DVD is exploring the protocols in depth and relating it to what you see in a capture. Yeah, you probably already know how DNS works. But don't skip that part. You've probably learned enough to fill a book about DNS. This focuses that knowledge so that you look at the key areas and it relates specific parts of the packets to the tools that will help you look at the protocol from a trace analysis viewpoint (versus your install, configure, maintain viewpoint). It's a little less overwhelming that way. You don't have to look at every line of every packet in a 546,000 packet capture. She shows you how to build filters to make the capture 'speak' to you and tell you its story.

This is some good stuff. And it comes from a name that should already be familiar. If you've read the ICRC or ACRC or CIT books, you've seen the name before. She knows her stuff - and knows how to teach it too. That's something that isn't found very often in this field. The 'experts' put you in a coma with their stuff. The good teachers often don't give you enough 'meat' in their classes. This set of videos has both. Get them if you can.

Posted by BlueWolf on April 24, 2008

April 03, 2008

Word Shortage

Apparently there are not enough words in the English language to describe computers and networking. This became painfully obvious to me today -- since so many words are used over and over to describe very different things.

One of the server guys wanted me to change the port settings from 'hard-coded' at 100 Mbps / Full Duplex to Auto Detect (both speed and duplex). I asked him for the MAC or IP addresses of the connections so that I could find the ports to change. Since the device he was working with was a little different from your average server (they are on the SAN management connections), he was having quite the time coming up with that info. Finally, he came by my cubie. He told me that all my worries were over and that he knew for sure what the ports were. He was positive of it because he had asked ____ (insert smart guy's name here). Then he confidently told me that it was Port 80 and Port 443.

Yeeeeeeaaaaaaaah. I'm sure this stuff does run over ports 80 and 443. So we took a walk down the hall and I explained the difference between a logical port and a physical port. I explained to him what actually comes 'over the wire' to the switch... what I can see and not see. Yes, I can always see the MAC address. That's why we always ask you for it - that's the one way we can be sure that we're working on the same exact connection. If we only go by server name (and whatever resolves in DNS), we might be tweaking the wrong connection - since most of the servers have several connections to different networks. We're not being lazy and not looking it up for you. We're just trying to make sure that we can compare our information to yours and verify it.

So he asks me "Can't you just do a trace?" Well...that's another word that had to be reused. Did he mean a 'trace' as in pulling out WireShark and capturing several hundred MB of data to see something? Or did he mean the results of the 'tracert' or 'traceroute' command? He meant 'tracert' -- since he didn't even know about the other kind of trace. Again, I had to point out that a 'trace' will only show the path (of routers/layer 3) to the device ... as resolved by DNS. This will not show you the connections - just ones that resolve to that device name. In this case, it's not giving you any more information than a ping.

And this got me thinking of all the reused words in our field. When the Sun admins ask for a 'trunk' ... they're not passing several VLANs over it. They just want an EtherChannel bundle (using LACP). So why are both things called a trunk? Didn't they know this would be confusing? Why are both things called ports? Can't you call one of them something else? Did they isolate the creators of TCP/IP on a Survivor island? Did the Sun company not know about 802.1q when they created the ability to bundle their connections?

Of course they knew about the other definition of their word. But they used it anyway. So there you have it - proof that they didn't have enough words. And they used the verbal equivalent of PAT. One word - multiple translated meanings. Maybe someone will come up with Nomenclature V6 or something. That way we can have globally unique names for these technologies. Nah, on second thought... let's keep it confusing. I got a pretty good giggle out of the port thing today.

Posted by BlueWolf on April 03, 2008

March 02, 2008

Upgrades

Well, the worst of the upgrade is finally over. I was tasked to move servers off of the old Nortel switches and move them to Cisco switches. Sounds simple, eh? Yeah, until you involve the business aspect of it.

There's some oddness about legacy systems. They've been there for a long time. That means they probably grew up in a patchwork manner without a clear focus toward any particular design. Of course, since it just mutated from some little thing to the monster it has become... nobody really knows what's on the system. The switch people (well, me) don't know what servers are on the switch. The cable labels can't be trusted since some cables were 'reused', you can't tell which connection is an idle backup connection and which is not in use, and there are cables everywhere running like dreadlocks throughout the switch interface. Once you fight through all those obstacles -- you let the server people know which servers will be affected. Great - however, the server people don't really know what applications are on each server. Some of the apps are authorized and documented, but many are not. Especially the really old ones that were placed before a documentation system was put in place. You know - the apps that everyone uses *all the time* without thinking (or knowing where it came from).

So you prep and prep from your 'network' point of view. New switch is ready. Cables are in place. All you have to do is switch out the cables at the patch panel and the servers are moved. Well, that's only part of it. The actual swapping of the cables and moving the gateway for that VLAN will take only a short time. VERIFICATION of the servers and apps will take up the majority of your maintenance window.

The first 'lift and shift' move took over 4 hours for the server team to verify their servers. I had planned and put in for a 4 hour window. [The actual move of the cables took less than 20 minutes.] I can't prove it, but I think it took so long because they were using the server list that I provided to them to verify connectivity. Two people were manually pinging each server (and probably not starting at opposite ends). I got a bit more slick for the next move and was able to cut the verification time down considerably. Although I provided the server teams with server names and IP addresses, I let them know that I would personally ping all those servers. I wrote a small batch file which did a ping on each address and returned the results to a text file. It was not the fastest, best or most efficient script, but it got the job done. I ran the script before starting and then after the cables were moved. Once I resolved all those, I checked the speed and duplex on the switch connections (a simple sh int status) against what I knew them to be in the old switch. At that point, I knew I had done all that I could do on my own. Sure, it really was the server team's responsibility to ping the servers. Sure, they should have been able to come up with such a simple batch file. But they didn't - and wouldn't if just given a list of servers and IP addresses. Once I let them know that connectivity will already be tested and that they are responsible for 'everything else' - they magically found other ways to show they were doing something. Suddenly SMS reports sprouted up and services that were historically problematic were checked. I guess if you take away the simplest and easiest thing to do, they'll look for something to show they checked the servers...

So now I've moved 4 subnets and decommissioned 3 switches. There are two more to do with some kind of HA or load balancing on them. They might be tricky, but there are only a handful of connections. Then there are two other switches that are all 'monitoring' connections. Yeah, I just found out about those. I still don't have a switch to move those connections to yet, so I guess that's off in the distant future. [Read: ignored by management until someone decides it's an immediate emergency.]

Wonder what happens after you complete all the 12 tasks of Hercules that they put upon you when you first arrive? Can I outpace their buying so that I can get some easy days at work? I guess I'm 4 switches away from finding out.

Posted by BlueWolf on March 02, 2008

February 10, 2008

Strange Problem

For quite some time I've been having a problem with my computer. It would run for a while and then some app would error out. There were a lot of freeze ups and I was constantly having to reboot. The screen was acting funny and slow. It was a real pain. The computer was getting to the point of barely usable.

Since I use my computer quite a bit, I really didn't have the time to properly troubleshoot the issue. I worked around it and figured I would get to it when I got a chance. So one night last week, I took the time and thoroughly updated SpyBot and ran a check on my system. It turned up nothing. I took another night and thoroughly updated Symantec and ran a scan. It took hours - but turned up nothing. It was quite frustrating.

This weekend I got a bigger block of time to work on it. I had resigned myself to having to format the entire hard drive and reinstall everything. Luckily I developed the habit of keeping the OS on one drive and the data on another. It would be long and tedious, but I didn't have to worry about losing any data. Friday night I started to make my list. I was checking everything that was on the computer so that I could duplicate it and not miss any apps. One thing I find annoying is having to do a task and needing to re-install the app first. As I was gathering the information I ended up eventually on the screen saver settings. == I had selected the CCIE study screen saver == How nice, but I never remember seeing any test questions when the screen saver comes on. Oh yeah, I haven't seen the screen saver come on - ever. I changed it and no more screen freezes or app errors. Lesson learned.

The time that I saved helped to move a few more things on my 'to do' list along rather nicely. I finally yanked the SPARC10 out of the closet and tried to get that working. That's going to take a bit of troubleshooting, but having it out is a step in the right direction. I know it works - and it has a fresh install of Solaris (can't remember what version) that I put on it last time I played with it. I'll get that rolling eventually. So then I turned my attention to the Linux box. Since now I have a router connecting to the DSL, I don't have to fiddle with getting PPoE running on it. I had previously installed RH7 and was going to update it. Yeah...that's an old version. Either the pages that *used to* work moved or were removed. Then the lightbulb went on: I have a better computer - I can install Fedora. Usually I put Linux on my 'spare' computer. Usually it's a computer that has very little cpu or memory. Quite often I've seen some of the older comps choke when trying to load Fedora, so I use an older version. Then I run out of time and never get to update. This time I got lucky. Everyone is happily computing on the best and fastest systems possible. And that leaves me with a PIII with 256 RAM. Wow. That's the fastest system I've ever had as a 'spare.' And instead of trying to shove a system on a 2 gig hard drive, right now I have two 8 gig hard drives. I went for the whole enchillada - complete install with all packages. And it's on the Internet. And it has two nic cards in it. I can finally use that computer to *do* something... [I have a 'project' that I've been wanting to work on for a long time now. Perhaps I can finally do it.]

Then I updated/finished the Sniffer laptop for work. It has:
Basic, but updated install of Win2K Pro (unneeded services set to manual)
Word and Excel (basic install - most options installed on first use or not used)
Wireshark
PuTTy
Telnet (the old one)
A small text comparison proggie that I picked up in my travels
Edit Pad

Quite the productive weekend. I'm really enjoying the Fedora computer. I'll have to put more about that one soon.

Posted by BlueWolf on February 10, 2008

February 05, 2008

More Wishes Granted

I'm realizing that I *do* get everything I wish for - just sometimes not in the way I intend...

The latest realization of a wish granted came in the form of an email. It was a topic reply notification. I had always wanted to have a message board with a lot of users and a lot of chatter. I set one up about 3 to 4 years ago. It really didn't take off at all. So I forgot about it. Then I started another board. That one has a moderate amount of success and a smattering of chatter now and then. A handful of people visit regularly and play the games. Nice.

So this topic reply notification... Yeah, it was from the *old* board. The one I forgot about. WTF? So I went there today and found out my wish had been granted. Yes, it's a LIVELY board. Apparently a large number of people found it and saw it was abandoned. They gave themselves squatter's rights. The board had thousands and thousands of posts. There were tons of visitors. And it looked like some alt.* group from Usenet! Needless to say - I was mortified.

After I regained my senses and figured out the password to log in... I deleted the database for the board and removed the entire folder for the site. It's gone now. Wonder how many people were regulars and will be disappointed that it's gone? And I had no idea that this little smutty society sprung up all on its own - and thrived! Quite a lesson to be learned about monitoring the sites you run/own.

The other wish to come true came in a much more pleasant form. [They're not *all* tough lessons.] I had always wanted a laptop that was purposed for sniffing and other network troubleshooting. Luckily my supervisor gave me an old laptop to work with for just that purpose. Yipeeee! No more having to sacrifice my laptop while doing a trace. I can actually get some work done while the trace is running. I can actually pull all the 'other' stuff off and focus its every bit of resources towards the task at hand. Granted, it's an old stinkpad with only 512MB of RAM. But it will be streamlined and focused. That will make the difference - or so I hope.

I'm formatting the hard drive now. Once I get it set up, I'll probably post the specs here. Although the laptop came with XP Pro, I'm putting 2K Pro on it. I've always found it to be a tighter and cleaner OS than XP Pro. I'm going to strip it down and customize it. The challenge will be to only keep that which is necessary without throwing the baby out with the bathwater. Of course, I'll be putting Wireshark on it. I debated Word and Excel and figured I might really want those on it too. There might be a need for a csv file or two at some point and screen captures just don't work in Notepad. The OLD telnet is a must along with some type of term program - but I'm still debating which one (or two). TFTP is another 'must-have' along with WS_FTP or some ftp program. Perhaps I'll splurge and stick an IP subnet calculator on there. The more fluff I yank out, the more justified I'll feel in putting a small proggie of convenience on it.

I have to go that route because I'm going to have to share the laptop with others at work. But once that challenge is over, I would like to repeat the same thing on one of my old laptops so that I can keep it for myself. The greater challenge is going to be repeating it using Linux. I could probably eeek out a bit more power by using the command line interface and skipping X Windows. Now that would be a challenge. We shall see.

Posted by BlueWolf on February 05, 2008