NIC Bonding with Openfiler 2.3(Final Release)

| Saturday, November 21st, 2009 | 6 Comments »

Today I’ve decided to test Openfiler’s 2.3 NIC bonding feature. (Active Backup)

I won’t get into how to configure and setup Openfiler as there are many articles on how to do that, as well as it’s a piece of cake.  If you consider yourself a network admin of any kind, you’ll breeze through the 1-2-3 setup/configuration. So let’s just get to the NIC bonding test!

First our hardware:

Supermicro 1U P8SCI server
Pentium 4 CPU 3.00GHz
3 GB RAM
Seagate 80GB SATA harddrive
Dual Broadcom BCM5721 Gigabit LAN

We will be utilizing a Cisco Catalyst 2960G-24TC 24-Port Gigabit Ethernet Switch and a D-link DGS-3100-24 GigE switch.

Now the test!!!

Step 1: Head over to the system tab where you’ll find your network card configuration. You’ll see that mine only has one NIC (eth0) configured and plugged in.
Its IP Address assigned with 192.168.2.20/24

So what we want to do now is enable (eth1) by clicking on “configure”. You’ll then be redirected to this page below:

You want to choose STATIC, not DHCP. Then click continue.

I gave (eth1) an IP Address of 192.168.2.21/24then click confirm to continue.

You’ll get a simple confirmation page here.

Click on “Return to Network Page”.

Below you’ll see that the (eth1) NIC card is now configured, plugged in and the Link status says “Yes”, speed at 1000 Mb/s so we are good to go! Now click on “Create bonded interface“.

You’re now presented with the NIC bond page in which you will select both adapters and then click continue.
Notice the warning. It is highly recommended that you DO in fact have terminal access to your Openfiler server as this procedure can cause server downtime. And you’ll see why in a moment!

This page is where you can configure your BONDED IP Address. This is a VIRTUAL IP Address. I went with 192.168.2.22/24 just to be consistent with my other IP Addresses. For Bonding Mode, we have to select “Active Backup”, I also chose (eth0) as my Primary Interface. Everything else I went with default.

Now go ahead and click continue.

IMPORTANT: This is where I lost network connectivity to my Openfiler server. As soon as I clicked “continue” I lost all access to the server. Below you will see that I attempted to ping all interfaces just out of curiosity. None responded.

So I had to physically go to the server and do a hard reboot. After the reboot, it appears to have worked! Notice below that the IP Address that is now responding is the new bonded VIRTUAL IP Address of 192.168.2.22/24. Also notice that our (eth0 & eth1) NIC cards do not respond to pings.

After logging back in with the NEW bonded virtual IP address (https://192.168.2.22:446). You can now see the bonded interface.

Here I began to run into some intermittent ping issues. During the course of the pings, you can see that they randomly seem to time out. This began to get pretty frustrating. I am pinging the new bonded IP by the way. (192.168.2.22)

So what we want to do now is actually test the failover. So while pinging (.22) I will first unplug the primary interface (eth0) and let’s see what happens.

Below, you can clearly see that the new (eth0) link  status is set to “no”. Since it is now unplugged. Also notice the ping response! Not a single packet was lost! Perfect failover!

Now we plug back in (eth0) and unfortunately the random timeouts come back! Also notice the (eth0) link status is now back to “Yes”.

Next we will unplug (eth1) and here are the results. And again, you can clearly see a perfect failover with no packet loss. Also notice that eth1 link status is now set at “No”.

So what’s the reason behind the random timeouts when BOTH eth1 & eth0 interfaces are up? Well, like any admin, you need to do a little troubleshooting. I powered down the Openfiler server and started it back up again. Just to test for the second time if the random timeouts come back after a nice hard reboot.

10 minutes later…… The random ping timeouts indeed occur. So the next thing to test is to rule out any issues with the your switch. In this case, our Cisco 2960G.

We want to be able to compare apples to apples as much as possible. So we’ll be replacing the Cisco 2960G switch with another 24 port 1GigE switch. This time a D-link DGS-3100-24 switch. (Why not another Cisco 2960G? Because I don’t have another spare one laying around!!)

So what are the results??

The pings are much better. There are still some dropped packets. Actually, just one. It does seem to randomly occur like the Cisco but only ONE packet loss instead of a series of packet losses with the Cisco Switch.

With no bonding, or with one NIC card out of the loop (failover), the pings were perfect.

Here’s another issue to be concerned about. I wanted to remove the NIC bonding. So I went to click on “delete” and I assumed that openfiler deleted the config. I lost connectivity to the server once again. Couldn’t ping any IP address. So I had to do another hard reboot.

After the reboot and at the terminal, I issued the command: ifconfig

The results were interesting, I lost all my IP addresses! I had to manually edit the ifcfg-eth0 just to get it back up and running. After editing that file with this info:

IPADDR=192.168.2.20
NETMASK=255.255.255.0

BTW: the file is located here: /etc/sysconfig/network-scripts/ifcfg-eth0

save and then issue the command to restart the network services:

service network restart

This will get your NIC card back on the network so you can access openfiler via web browser.
I haven’t spent all that much time on Openfiler but from quickly setting it up and playing with it, I have to say it’s a little buggy. Perhaps my hardware is the cause of the problem? Perhaps it’s not supported. I’d say it’s a good application to play with, but I’m not sure you would really want to run your business off of it as your primary SAN or NAS.

Share
  • Sean Milheim

    Do you have multiple nics outside of eth0/eth1? If so udev may be assigning the incorrect interfaces.

    vi /etc/udev/rules.d/70-persistent-net.rules

    Verify that the nics are being assigned to the correct interfaces. I’ve had this issue a few times with Dell servers. Once I fixed this I haven’t had any dropped packets.

  • Sean Milheim

    Do you have multiple nics outside of eth0/eth1? If so udev may be assigning the incorrect interfaces.

    vi /etc/udev/rules.d/70-persistent-net.rules

    Verify that the nics are being assigned to the correct interfaces. I’ve had this issue a few times with Dell servers. Once I fixed this I haven’t had any dropped packets.

  • http://www.brandontek.com/ admin

    Hi Sean:

    The server currently only has 2 NIC interfaces. I was actually going to add a 3rd interface for admin purposes because the random flakyness of the 2 NIC’s interchanging the MAC addresses would cause me to not be able to access OF. I haven’t added a 3rd NIC just yet though.
    From the last time I’ve checked, they are being assigned correctly. The MAC addresses changing ports is most definitely confusing the Cisco switch, and any switch for that matter….however less noticeable with the D-link for whatever reasons…

  • http://www.brandontek.com admin

    Hi Sean:

    The server currently only has 2 NIC interfaces. I was actually going to add a 3rd interface for admin purposes because the random flakyness of the 2 NIC’s interchanging the MAC addresses would cause me to not be able to access OF. I haven’t added a 3rd NIC just yet though.
    From the last time I’ve checked, they are being assigned correctly. The MAC addresses changing ports is most definitely confusing the Cisco switch, and any switch for that matter….however less noticeable with the D-link for whatever reasons…

  • Lance C

    Same issue here…
    ECS 848P-
    Pentium 4 CPU 2.4 GHz2 GB RAMHitachi 3 TB SATA harddriveDual netgear (RTL-8169) Gigabit LAN

    Same exact situation. Glad I didn’t spend the money on buying a set of real NIC cards. Somewhat disappointing but you can’t complain about free.

  • http://usb3gvn.com USB 3G

    Thank you for the precious advice. It’s really useful for my job. If you have some related imformation about this problem, you can post here. One time, Thank for help!