MICE physical server 2 (and thus by extension route server 2) is again
having errors on its peering interface. We will be performing
maintenance tomorrow to address this. Since this only affects one of the
two route servers, participants should not be impacted.
Details:
Sometime tomorrow (exact time depending on day job schedules), Jeremy
and I will:
1. Apply the usual security updates to VMs and the host, since the host
is going to be rebooted anyway.
2. Convert server 2's port to a LAG. This will make future changes
easier, especially if this error problem continues.
This will involve a reboot of the server. The reboot is "necessary"
because Linux's netplan does not remove VLAN configuration when changes
are made (only adds it). Without removing the old
direct-on-the-physical-interface VLAN config, we could (would?) create a
loop. You may recall that I created a loop that blew up the exchange in
more-or-less that exact way when setting up the servers originally.
While I could remove the VLAN configuration manually, in this case, it's
easier and safer to simply reboot. That also ensures our running
configuration is fully consistent with the configuration files.
3. Replace the optic, also moving to the other port on the NIC.
The underlying failure manifests as a decrease in optical transmit power
(i.e. power received at the switch) over time. We have replaced the
optic previously, which fixes it, but then it fails again after a while.
We are open to suggestions about why this might be occurring.
One theory is that this is due to heat. We are going to try the second
port in case that makes a difference. We are also considering switching
to a direct attach cable, as perhaps that would produce less heat.
--
Richard Laager
Chief Manager, Director
Midwest Internet Cooperative Exchange LLC
