Stacking Cisco 3750 switches

| Sunday, January 15th, 2012 | 15 Comments »

There are a lot of articles out there on the web for Cisco’s 3750 switches. However, there is no promise that those blogs will continue to stay up, so I’ve decided to create my own blog with my own experiences with Cisco’s 3750-E’s and 3750-X switches with many tips along the way!

I decided to break up the blog into two parts. Part 1 will cover stacking configuration, while part 2 covers stack power configuration.

I hope what you find below is comprehensive enough to get you on your way!

Cisco’s stackwise technology is pretty cool. Instead of your conventional daisy chaining of switches, you stack the switches from behind using their stackwise cables and software. The benefits are many. I’ll list what I’ve found the most important.

Stacking Benefits

  • 32G bandwidth – Cisco likes to advertise 64G of aggregate bandwidth. The keyword is “aggregate”. I find this very misleading because the stackwise cables function bi-directionally. This is where Cisco gets their 64G total speeds. However, when you purchase a 10/100 network card, it’s never advertised as 20/200 is it? The truth is, a 100Mbps card at full duplex technically gets you 200Mbps of available bandwidth. 100Mbps transmit and 100Mbps receive. In any event, all member switches have access to this available bandwidth.
  • Management – When stacked, all your member switches are seen as just one switch. From a management standpoint, this makes configuration easy. One management IP address and one configuration file. Instant access to all your member switches.
  • Cabling – Cleaner cabling since they stack from behind and would not interfere with any other cables you may have. So it is aesthetically pleasing.
  • 9 Member Switch capacity – You can stack up to nine switches in your stackwise switching fabric. This gives you much room for growth and 432 Ethernet 10/100/1000 ports with 18 10GbE ports.
  • Hot pluggable switches – You can remove and add switches while the stack is running. A working stack can accept new members or delete old ones without service interruption.
  • Stackwise Plus – Supports local switching with destination stripping. This allows traffic to stay off the stackwise fabric if the destination is already local to the switch.

When playing with equipment, I always try to understand their behavior. Go over different types of scenarios, ask myself questions. What happens if I do this? What happens if I do that? How will the switch behave? It’s not always possible to do this but I always try.

Stacking Rules

Below I list what you should know and understand before stacking.

  • There are three main software feature sets: LAN Base, IP Base, IP Services
  • You cannot mix software feature sets. You cannot have some switches with LAN Base and some with IP Services for example.
  • The Cisco StackWise technology requires that all units in the stack run the same release of Cisco IOS Software.
  • A standalone switch is a switch stack with one stack member that also operates as the stack master.
  • Make sure that you power off the switches that you add to or remove from the switch stack. I have not found a clear reason for this, perhaps it’s to prevent any corruption of the stack. Obviously you can add or remove a switch during stack production.
  • A new, out-of-the-box switch (one that has not joined a switch stack or has not been manually assigned a stack member number) ships with a default stack member number of 1. When it joins a switch stack, its default stack member number changes to the lowest available member number in the stack.
  • If you manually change the stack member number, it only takes effect when you reset that specific member switch.
  • A higher priority value for a stack member increases its likelihood of being elected stack master and retaining its stack member number. The priority value can be 1 to 15. The default priority value is 1.
  • You can manually define the priority value for a stack member. I always like to define which switch is the MASTER.
  • The configuration that you create on the switch stack is called the provisioned configuration.
  • The switch that is added to the switch stack and that receives this configuration is called the provisioned switch.
  • Each software image includes a stack protocol version. In order to remain compatible, protocol versions should be similar.
  • Connecting to individual console ports on a member switch still talks to MASTER switch.

Let the stacking begin!

Once your first switch has been turned on ( you don’t need to have the stackwise cables plugged in yet ), you can define this switch as the MASTER. To do that you need to go into configuration mode:

switch(config)# switch 1 priority 15
switch# copy run start
switch# reload

switch 1 indicates your current switch. All switches are switch 1 by default. Priority 15 is the highest you can set your switch which causes it to become the MASTER.
If you want your second switch to become the MASTER you can make it priority 14. If you only have two, then there’s no need to since the only remaining switch will become
MASTER by default if the original MASTER fails or goes offline.

Assuming you have at least two 3750′s, connect them together in a criss-cross fashion.

Now you can turn on switch#2. You can console into switch#2 as it boots up so you can get an idea of when the bootup process has finished. Once it is done, issue this command:

switch# show switch

Switch/Stack Mac Address : 001c.57bc.9c00
H/W   Current
Switch#  Role   Mac Address     Priority Version  State
———————————————————-
*1       Master 001c.57bc.9c00     15     0       Ready
2       Member 0012.00e9.4400     14     0       Ready

This command will indicate your MASTER switch and switch#2 as just a member switch.

Stack verification

How can we tell the stackwise cables were properly inserted?

switch# show switch stack-ring speed

Stack Ring Speed        : 32G
Stack Ring Configuration: Full
Stack Ring Protocol     : StackWise

Here you will see that the ring speed is 32G. If you read my “stacking benefits” further up, you will remember my gripe about this and why I think Cisco is misleading when they advertise 64G. Someone may be looking for 64G as the ring speed and wonder if their setup is incorrect.

switch# show ip int brief

Interface              IP-Address      OK? Method Status                Protocol
Vlan1                  192.168.50.2    YES manual up                    up
FastEthernet1/0/1      unassigned      YES unset  down                  down
FastEthernet1/0/2      unassigned      YES unset  down                  down
FastEthernet1/0/3      unassigned      YES unset  down                  down
FastEthernet1/0/4      unassigned      YES unset  down                  down
FastEthernet1/0/5      unassigned      YES unset  down                  down

……..

FastEthernet2/0/1      unassigned      YES unset  up                    up
FastEthernet2/0/2      unassigned      YES unset  down                  down
FastEthernet2/0/3      unassigned      YES unset  down                  down
FastEthernet2/0/4      unassigned      YES unset  down                  down
FastEthernet2/0/5      unassigned      YES unset  down                  down

LED lights – You can physically verify which switch is the MASTER by looking at the front panel LED lights. The MASTER LED light will be lit solid green.

Stack notes:

Below are commands and miscellaneous notes I’ve compiled along the way that I hope you will find useful.

Enable mode commands

show switch = show member switches that have successfully been added to the stack and their priorities. Find which switch is the MASTER switch.
show switch detail = Provide port status of stackwise ports.
show switch stack-ring speed = stack ring status, configuration and protocol. What really matters here are the ring status and configuration.
reload slot <member switch number> = For example: reload slot 4 will only reboot the member switch that is switch#4.
remote command <member switch number> show version = You can specify output specifically for a member switch. If you want the IOS version of member switch#2 you would type: remote command 2 show version. Not every command is supported under “remote command”.
no switch <member switch number> provision = If you’ve removed a member switch physically from your stack, you should run this command to permanently remove it from the stack status when you issue the “show switch” command.
archive copy-sw = copies IOS from one switch to another.

Configuration mode commands

switch <switch number> priority <priority number> = Configures the priority for a particular switch. Priorities range from 1 – 15.
switch 1 renumber 2 = Reconfigures switch 1 as switch 2 and only takes effect after this switch reloads. You can use the “reload slot” command do this so you don’t have to reload the entire stack. But you cannot renumber to an already used switch#.

Miscellaneous Notes

- Cisco recommends that you leave a blank module if your switch supports modules. If you leave it open, the switch will overheat due to airflow issues. Cisco sells blank modules if you don’t have one.

- Only power on/off switches after you already have the switch stacked. For example, if you’re adding a new switch to the stack, don’t have it powered on while you are connecting the stackwise cables. Connect the cables and then power on the new switch.

- archive copy-sw command – What does this command really do? It basically issues this command if you wanted to do it manually.
copy flash1: flashX (where X is your slot number from your member switch)

- Backup original IOS image – You can backup your original image before they become part of the stack. I ran into an issue with a corrupted image once and it was a bad experience. Issue a “show flash:” to find the location of your image. Then issue “copy flash: tftp:” to save it to your TFTP server. You can always use XMODEM to install the image if your switch can’t boot up.

- LED lights – Lights on the switch indicate who is MASTER.

- Once stacked, there are two types of levels: system-level and interface-level.

- Each stack has only one configuration file, which is distributed to each member in the stack. This allows each switch in the stack to share the same network topology, MAC address, and routing information. In addition, it allows for any member to become the master, if the master ever fails.

- A break in any one of the cables will result in the stack bandwidth being reduced to half of its full capacity. Subsecond timing mechanisms detect traffic problems and immediately institute failover.

Conclusion

I hope the information has been informative and that the article was just long enough but not too long! If I have left anything out that you find critically important, please let me know! Happy stacking!

Share
  • Anonymous

    Very good article! I came across the other day Stack incompatibility due to having different “non-standard” SDM templates defined. One was default access and the other default desktop. The switches can redefine an SDM template but only if the aren’t unique.

    Just my two cents
    Link to Troubleshooting SDM Templates in Stacks Fixing SDM Template Configuration Incompatibilities (Catalyst 3750G-12S Switches)

    Anthony
    @pandom_
    blog.ciscoinferno.net

  • http://twitter.com/brandontek Brandon Kim

    Glad you like it! You happen to touch on SDM which I completely left off. ha!
    Thanks for bringing it up as I’m sure it’s happened to enough people and me too one day.

    I like your website. I’ll add it to my blogroll!

  • Anonymous

    If I have a server with a teamed network card, and have one connection going to each of the stacked switches is there anything I have to configure on the switches to make it aggregate properly?

    Thanks!

  • http://twitter.com/brandontek Brandon Kim

    Yup that should work. You may need to play with the settings a bit depending on how your server will negotiate. I’m sure you will need to use LACP not PaGP protocol. You have to setup etherchannel. So create a port interface and then apply the two physical interfaces to that port channel.

    You can read my article below to see the commands you need to create a port channel.

    http://www.brandontek.com/cisco/hsrpetherchannel-and-trunks/

  • Pingback: Chatroulette Community Service | iFriends

  • Pingback: Switch stackelés « Cisco Blog

  • airborne1128

    So is there a benefit to stacking in a small network?  I have 2 networks seperated by a core 3750G.  On my “Land” based side I have 2x3750Gs and I was thinking about stacking for managment purposes.  They run 2 vlans, one for “internal” traffic and one for traffic bound for the other network.

  • http://www.learn2success.org/ Naveed

    we are about to add 3rd switch to our stack 3750 and you highlighted some of the key points which i were not aware before.

    Thanks for this post, it is really helpful.

  • Jason Rolla

    Great Article thanks for the information. I have something that may change things. I have a 3750E and a 3750X both in different closets and running independently both with IPBase and I will get them both on the same IOS version. I need to bring the 3750X over to the E closet and stack them. I will leave the 3750E powered on and plug the stack cable into it and then into the X and then turn it on (3750X). I want to make sure that the E config stays on there and does not disappear as I want the E to be the master.

    MAC for 3750E; Starts with 0c
    MAC for 3750X; Starts with 3c

    Which one will become master by default and will either of them lose their configs on the ports if one or the other becomes master. I will back both up but I really dont want the 3750E to lose its config.

    While they are seperated should I do the following;

    3750E; switch 1 priority 14
    3750X; switch 1 priority 15

  • http://twitter.com/brandontek Brandon Kim

    Hi Jason,

    Glad you found my article helpful. As long as you issue the priority commands then you should be ok. I would wipe out your 3750-X (of course save your config as you alluded to earlier), then issue the priority command on the 3750-X so when it comes back up, it should *not* become master.

    Let me know how it goes, good luck!

  • http://twitter.com/brandontek Brandon Kim

    Glad you liked it, let me know if you have any other questions or comments…

  • http://twitter.com/brandontek Brandon Kim

    It really depends on your specific needs, what applications you run on your network will really dictate that. A small company that transfers large amounts of high-res files may want to take advantage of the stacking features….

  • Keith

    Hey Brandon, thought you might want to learn about our new single fiber X2 transceivers. Check them out: http://www.championone.net/products/transceivers/x2/single-fiber/

  • Novan dwi cahyo

    hii..very nice article..
    i am really excited about cisco stuff.
    but..i wonder, what if the master switch experience a failure, or the other switch, if the second switch down, then the third switch also down?? or what happend??
    sory for my bad english.
    stop by my blog.. scubyx.blogspot.com
    my simple blog :D

  • emma

    Thanks, I have bought the C2960S-STACK. Maybe the method is the same.