Shared Ethernet Adapter (SEA) Failover with Load Balancing

Update: The developers and the manuals call this Load Sharing but most people think it is called Load Balancing. Perhaps, balancing gives the wrong impression of fine grain packet by packet balancing where we actually have higher level, cruder splitting of the work with Sharing. Below I use the word Balancing but mean Sharing.

I have got a few questions recently on how to set this up as there are announcement with near zero information on setup, the configuration needed and a worked example. So here goes.

For a long time now we have had SEA Failover where the VIOS pair work together to provide a simple to set up at the client VM (LPAR) redundant path to the network.  A single virtual Ethernet network is managed between two Virtual I/O Servers (VIOS). The one with the higher priority (lower number) is the primary and does all the network bridging I/O and the secondary does nothing unless the primary is taken down or fails. Then the secondary takes over and does all the network bridging until the primary returned when the primary takes over again.
Here is my diagram of a non-trivial network setup with one SEA connected to many internal virtual networks:
SEA Setup image
SEA Setup image

Note: All the network I/O is going via VIOS a.  This also assuming the other ent* adapters are present but not involved in the example.
 To create the SEA on VIOS a you rung the following command
mkvdev -sea ent0 -vadapter ent3,ent4,ent5, ent6 -default ent3 -defaultid=10 -attr ha_mode=auto ctl_chan=ent9
Note: I corrected a typo where it had "ctl_chan=42" was wrong, confusing the VLAN id with the network port.
In this symmetric VIOS example it is the same command for VIOS b.  The all important Priority which decides which is Primary and Secondary VIOS is set at the VIOS Virtual Ethernet Adapter Properties panels on the HMC as below:
Virtual Ethernet Properties image
Virtual Ethernet Adapter Properties image

VIOS b will of course have a higher Priority number meaning it is the Secondary VIOS.

Now let us move on to SEA Failover with Load Balancing
This is all wonderful and simple to setup up but it does mean only half of the physical network adapters are used at one time.  If you are running 10 Gbit network adapters, which are fairly expensive, that seems like wasting resources and hence the new Load Balancing feature to use of all the network adapters for higher bandwidth and better latency (when busy).  In the above, diagram when both VIO Servers are running ALL the packets will exit the machine via VIOS a and nothing via VIOS b.
Now some people may have the unrealistic idea that with SEA Failover with Load  Sharing will get you to network heaven!! Like every other packet from the client VM would go to different VIOS and its SEA. This is impossible as the client VM only "sees" the one virtual Ethernet and can't control the destination VIOS and it would be impossible for the VIO Server to some how co-operate to acheive a perfect spilt in traffic and organising that would slow the network down enormously. If that level of concurrency is required you need to set-up multiple virtual Ethernets on the client VM and use LInk-Aggregation (also called Ether-Channel or Teaming) at the client VM. The down side is a more complicated set-up on every single client VM. 

The SEA Failover with Load Balancing pair of VIOS have to vigorously agree which VIOS does what and this is done at the Trunk Adapter level. This is achieved with the Secondary VIOS communicating with the Primary VIOS to volunteer to take over a set of Trunk Adapters (now called Ethernet bridging on the HMC). Once acknowledged they split the network I/O work between them.  If later you take a VIOS down or there is a genuine failure the surviving VIOS takes on all the network I/O .... of course.
  • Both VIOS must be at Version
With both VIOS  running we have a configuration like this:
SEA setup image
SEA Setup 
 Note: The network I/O is going via both VIO Servers.
If you have a very simple network configuration like one internal virtual network and a SEA bridging to the outside physical Ethernet then the traffic can't be split between VIO Servers , obviously.   This is what we run a lot of the time in my Lab environment as our needs are simple.  But in more production like environments there is typically many virtual networks for user access, systems administration, multiple tier networks and a separate backup network.
If you already have a suitable SEA running you can change its mode with (assuming you SEA device is ent10) - this can be dynamically changed:
chdev -dev ent10  -attr ha_mode=sharing 
Here are some notes from the developers that might help out:
  • Load Sharing must be enabled first on the Primary SEA first - the Secondary SEA initiates the load sharing for you want the primary to be ready.
  • Obviously, both SEA's need to be in Load Sharing mode.
  • The VLAN id's of the Trunk adapters must match - If VIOS a has trunk adapter 1 with VLAN id's say 10 and 20 and a second trunk adapter with VLAN id's 40 and 50 then the other VIOS needs to have the same combination. This needs careful planned.
  • To disable Load Sharing shave the ha_mode to auto.  The other options are standby or disabled.Find the current state of the SEA and which is primary but getting to root on the VIOS (oem_setup_env) then
entstat -d ent10 | grep -i state
  • The entstat command on the VIOS SEA has other good information like the VLAN id's.
  • If you have a VIOS failure or take a SEA down then once back up, you can restart the Load Sharing with the chdev command above

I hope this helps you get Shared Ethernet Adapter Failover with Load Sharing working.


Popular posts from this blog

How to Install and Configure Kubernetes Cluster and Docker on CentOS8/RHEL8

How to install Ansible Tower on Red Hat / CentOS

How to install Ansible on RHEL8 / CentOS8