- mice-discuss - lists.micemn.net

Re: Qwilt question

by Anthony Anderberg

The Qwilt MICE link has been down since Thursday, they’ve confirmed some kind of hardware problem on their end. The MICE graphs show an additional outage Wednesday into Thursday, but that’s just me taking a day to get caught up with all of Jeremy’s port moves. I can’t speak to your local appliance’s volume, mine here seems normal - perhaps an artifact of where they get their cache fill? Cheers, Anthony From: Frank Bulk <fbulk(a)mypremieronline.com> Date: Monday, February 10, 2025 at 11:05 PM To: "mice-discuss(a)lists.micemn.net" <mice-discuss(a)lists.micemn.net> Subject: Qwilt question FiberNet’s BGP peering with Qwilt at MICE has been down since Thursday, Feb 6 at 1:59 pm and our local Qwilt cache appliance’s volume dropped to zero early Sunday afternoon. MICE graph shows nothing since Wednesday, February 5: http://micelg.usinternet.com/cacti/graph.php?action=view&local_graph_id=120… Anyone know what’s up? Frank [cid:image001.png@01DB7C31.95E9C990] Frank Bulk Chief Technology Officer / Chief Operating Officer [PremierComm_logo_CMYK.png] 339 1st Ave NE | Sioux Center, IA 51250 | Find us on Facebook<https://www.facebook.com/PremierCommunicationsSiouxCenter> phone: 712.722.3451 web: mypremieronline.com<https://www.mypremieronline.com/> direct: (712) 722-7152 cell: (712) 441-6049

Feb. 11, 2025

Qwilt question

by Frank Bulk

FiberNet's BGP peering with Qwilt at MICE has been down since Thursday, Feb 6 at 1:59 pm and our local Qwilt cache appliance's volume dropped to zero early Sunday afternoon. MICE graph shows nothing since Wednesday, February 5: http://micelg.usinternet.com/cacti/graph.php?action=view&local_graph_id=120… Anyone know what's up? Frank [cid:a1ff5d57-4afc-4ec0-be61-51708bf94a4e.png] Frank Bulk Chief Technology Officer / Chief Operating Officer [cid:premiercomm_logo_cmyk_1388ffcb-8ec0-4842-9e40-a0e598482e7d.png] 339 1st Ave NE | Sioux Center, IA 51250 | Find us on Facebook<https://www.facebook.com/PremierCommunicationsSiouxCenter> phone: 712.722.3451 web: mypremieronline.com<https://www.mypremieronline.com/> direct: (712) 722-7152 cell: (712) 441-6049

Feb. 11, 2025

Round 2 - MICE Switch-Cabinet Migration

by Jeremy Lumby

On Wednesday February 12th at approximately 1PM CST I will be migrating the following networks off of the Arista 7504R (100Gig core switch) to a temporary switch in the new cabinet/space that the 100G chassis will eventually be moved to at 511. Prior to the migration BGP will be blocked on the specific participant's port to allow traffic to gracefully reroute. This is the second round of weekly moves as everything is migrated out of the existing cabinet. If you are interested in upgrading to 400Gig let us know ASAP so we can avoid double moves, and facilitate an easier transition for everyone. Netflix - 2906: Old Ports: Po3 Et6/20/1 Et5/11/1 Temporary New Ports: Po3 E1/1 E2/1 Meta - 32934: Old Ports: Po27 Et5/12/1 Temporary New Ports: Po27 E3/1 Amazon - 16509: Old Ports: Po24 Et3/21/1 Et5/5/1 Et5/6/1 Et6/1/1 Temporary New Ports: Po24 E4/1 E5/1 E6/1 E7/1 CloudFlare - 13335: Old Ports: Po13 Et6/17/1 Temporary New Ports: Po13 Et8/1 G-Core Labs - 199524: Old Ports: Et3/30/1 Temporary New Ports: E9/1 Apple - 714: Old Ports: Po31 Et4/49/1 Temporary New Ports: E10/1 Inteliquent - 40160: Old Port: Et4/32 Temporary New Port: E33 Inteliquent - 5715: Old Port: Et4/5 Temporary New Port: E34 Hurricane Electric - 6939: Old Ports: Po6 Et3/5/1 Et6/5/1 Et6/18/1 Temporary New Ports: Po6 E11/1 E12/1 E13/1 Metronet - 30600: Old Ports: Po40 Et6/7/1 Temporary New Ports: Po40 E14/1 Cedar Falls Utilities - 13855: Old Port: Et6/16/1 Temporary New Port: E15/1 Savage Communications - 23260: Old Port: Et6/19/1 Temporary New Port: E16/1 Nuvera - 23465: Old Ports: Po29 Et5/8/1 Temporary New Ports: Po29 E17/1 Long Lines - 32867: Old Port: Et6/21/1: Temporary New Port: E18/1 Nsight - 3663: Old Port: Et6/22/1 Temporary New Port: E19/1 Bluepeak - 20412: Old Port: Et6/23/1 Temporary New Port: E20/1 UnitedHealth Group - 10879: Old Port: Et6/24/1 Temporary New Port: E21/1 Hennepin County - 40382: Old Port: Et6/25/1 Temporary New Port: E22/1 TDS - 4181: Old Port: Et6/27/1 Temporary New Port: E23/1 Truestream - 398143: Old Port: Et6/28/1 Temporary New Port: E24/1 HBC - 14828: Old Port: Et6/30/1 Temporary New Port: E25/1 SRT Communications - 14543: Old Port: Et6/33/1 Temporary New Port: E26/1 Let me know if you have any questions Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington MN 55425 Main 612-355-7740 x211 Direct 612-392-6814 eFax 952-873-7425 jlumby(a)mnvoip.com

Feb. 7, 2025

RE: MICE Switch-Cabinet Migration Round 1

by Jeremy Lumby

The round 1 maintenance is complete. Let me know if you are seeing any issues. We are hands off at the moment, and will begin updating documentation to reflect the new port assignments. -----Original Message----- From: Jeremy Lumby <jlumby(a)mnvoip.com> Sent: Wednesday, February 5, 2025 1:05 PM To: 'MICE Discuss' <mice-discuss(a)lists.micemn.net> Subject: RE: MICE Switch-Cabinet Migration Round 1 This maintenance is beginning now. -----Original Message----- From: Jeremy Lumby <jlumby(a)mnvoip.com> Sent: Wednesday, January 29, 2025 9:03 PM To: 'MICE Discuss' <mice-discuss(a)lists.micemn.net> Subject: MICE Switch-Cabinet Migration Round 1 On Wednesday February 5th at approximately 1PM CST I will be migrating the following networks off of the Arista 7504R (100Gig core switch) to the new 7804R3 (400Gig core switch) at 511. Prior to the migration BGP will be blocked on the specific participant's port to allow traffic to gracefully reroute. This will be the first round of weekly moves as everything is migrated out of the existing cabinet. If you are interested in upgrading to 400Gig let us know ASAP so we can avoid double moves, and facilitate an easier transition for everyone. Ridgeview core switch uplink Old Ports: Port-channel10 Ethernet3/35/1 Ethernet3/36/1 Ethernet5/14/1 Ethernet5/15/1 Ethernet6/35/1 Ethernet6/36/1 New Ports: Port-channel3 Ethernet3/33/1 Ethernet3/34/1 Ethernet4/32/1 Ethernet4/33/1 Ethernet4/34/1 Ethernet4/36/1 MN VoIP extension switch uplink Old Ports: Port-channel15 Ethernet3/6/1 Ethernet6/6/1 Ethernet6/13/1 New Ports: Port-channel1 Ethernet3/1/1 (Native 400G port) Fiber Minnesota extension switch uplink Old Ports: Port-channel22 Ethernet3/7/1 Ethernet5/13/1 Ethernet5/16/1 Ethernet6/3/1 New Ports: Port-channel4 Ethernet3/2/1 (Native 400G port) Ethernet4/2/1 (Native 400G port) Akamai Old Ports: Port-channel19 Ethernet6/8/1 Ethernet6/10/1 Ethernet6/12/1 Ethernet6/31/1 Ethernet5/7/1 New Ports: Port-channel5 Ethernet3/3/1 Ethernet3/4/1 Ethernet3/5/1 Ethernet3/6/1 Ethernet3/7/1 Amazon Old Ports: Port-channel5 Ethernet3/12/1 Ethernet6/4/1 Ethernet5/3/1 Ethernet5/4/1 New Ports: Port-channel6 Ethernet3/8/1 Ethernet3/9/1 Ethernet3/10/1 Ethernet3/11/1 Apple Old Ports: Port-channel30 Ethernet 3/14/1 New Ports: Port-channel7 Ethernet 3/12/1 Arvig Old Ports: Port-channel42 Ethernet3/8/1 Ethernet6/11/1 Ethernet5/1/1 New Ports: Port-channel8 Ethernet3/13/1 Ethernet4/13/1 Ethernet4/14/1 Fastly Old Ports: Port-channel18 Ethernet4/50/1 New Ports: Port-channel9 Ethernet3/14/1 Meta Old Ports: Port-channel26 Ethernet3/10/1 New Ports: Port-channel10 Ethernet3/15/1 Microsoft Old Ports: Port-channel34 Ethernet3/18/1 New Ports: Port-channel11 Ethernet3/16/1 Netflix Old Ports: Port-channel2 Ethernet3/11/1 Ethernet3/27/1 New Ports: Port-channel12 Ethernet3/17/1 Ethernet3/18/1 Northern Lights GigaPOP Old Ports: Ethernet6/14/1 New Ports: Ethernet3/19/1 Qwilt Old Ports: Port-channel12 Ethernet6/29/1 Ethernet5/2/1 New Ports: Port-channel13 Ethernet3/20/1 Ethernet4/20/1 T-Mobile Old Ports: Port-channel41 Ethernet3/2/1 Ethernet6/9/1 Ethernet6/15/1 New Ports: Port-channel14 Ethernet3/21/1 Ethernet4/21/1 Ethernet4/22/1 Let me know if you have any questions Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington MN 55425 Main 612-355-7740 x211 Direct 612-392-6814 eFax 952-873-7425 jlumby(a)mnvoip.com

Feb. 5, 2025

RE: MICE Switch-Cabinet Migration Round 1

by Jeremy Lumby

This maintenance is beginning now. -----Original Message----- From: Jeremy Lumby <jlumby(a)mnvoip.com> Sent: Wednesday, January 29, 2025 9:03 PM To: 'MICE Discuss' <mice-discuss(a)lists.micemn.net> Subject: MICE Switch-Cabinet Migration Round 1 On Wednesday February 5th at approximately 1PM CST I will be migrating the following networks off of the Arista 7504R (100Gig core switch) to the new 7804R3 (400Gig core switch) at 511. Prior to the migration BGP will be blocked on the specific participant's port to allow traffic to gracefully reroute. This will be the first round of weekly moves as everything is migrated out of the existing cabinet. If you are interested in upgrading to 400Gig let us know ASAP so we can avoid double moves, and facilitate an easier transition for everyone. Ridgeview core switch uplink Old Ports: Port-channel10 Ethernet3/35/1 Ethernet3/36/1 Ethernet5/14/1 Ethernet5/15/1 Ethernet6/35/1 Ethernet6/36/1 New Ports: Port-channel3 Ethernet3/33/1 Ethernet3/34/1 Ethernet4/32/1 Ethernet4/33/1 Ethernet4/34/1 Ethernet4/36/1 MN VoIP extension switch uplink Old Ports: Port-channel15 Ethernet3/6/1 Ethernet6/6/1 Ethernet6/13/1 New Ports: Port-channel1 Ethernet3/1/1 (Native 400G port) Fiber Minnesota extension switch uplink Old Ports: Port-channel22 Ethernet3/7/1 Ethernet5/13/1 Ethernet5/16/1 Ethernet6/3/1 New Ports: Port-channel4 Ethernet3/2/1 (Native 400G port) Ethernet4/2/1 (Native 400G port) Akamai Old Ports: Port-channel19 Ethernet6/8/1 Ethernet6/10/1 Ethernet6/12/1 Ethernet6/31/1 Ethernet5/7/1 New Ports: Port-channel5 Ethernet3/3/1 Ethernet3/4/1 Ethernet3/5/1 Ethernet3/6/1 Ethernet3/7/1 Amazon Old Ports: Port-channel5 Ethernet3/12/1 Ethernet6/4/1 Ethernet5/3/1 Ethernet5/4/1 New Ports: Port-channel6 Ethernet3/8/1 Ethernet3/9/1 Ethernet3/10/1 Ethernet3/11/1 Apple Old Ports: Port-channel30 Ethernet 3/14/1 New Ports: Port-channel7 Ethernet 3/12/1 Arvig Old Ports: Port-channel42 Ethernet3/8/1 Ethernet6/11/1 Ethernet5/1/1 New Ports: Port-channel8 Ethernet3/13/1 Ethernet4/13/1 Ethernet4/14/1 Fastly Old Ports: Port-channel18 Ethernet4/50/1 New Ports: Port-channel9 Ethernet3/14/1 Meta Old Ports: Port-channel26 Ethernet3/10/1 New Ports: Port-channel10 Ethernet3/15/1 Microsoft Old Ports: Port-channel34 Ethernet3/18/1 New Ports: Port-channel11 Ethernet3/16/1 Netflix Old Ports: Port-channel2 Ethernet3/11/1 Ethernet3/27/1 New Ports: Port-channel12 Ethernet3/17/1 Ethernet3/18/1 Northern Lights GigaPOP Old Ports: Ethernet6/14/1 New Ports: Ethernet3/19/1 Qwilt Old Ports: Port-channel12 Ethernet6/29/1 Ethernet5/2/1 New Ports: Port-channel13 Ethernet3/20/1 Ethernet4/20/1 T-Mobile Old Ports: Port-channel41 Ethernet3/2/1 Ethernet6/9/1 Ethernet6/15/1 New Ports: Port-channel14 Ethernet3/21/1 Ethernet4/21/1 Ethernet4/22/1 Let me know if you have any questions Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington MN 55425 Main 612-355-7740 x211 Direct 612-392-6814 eFax 952-873-7425 jlumby(a)mnvoip.com

Feb. 5, 2025

Please welcome Fiber Minnesota to MICE!

by Jeremy Lumby

Name: Fiber Minnesota Switch: MICE 400G Core - 511 Ports: E 3/2 and E 4/2 in Po 4 (2x400G) ASN: This is a Remote Switch uplink upgrade from 400G to 800G IPv4: NA IPv6: NA Peering Contact: NA Route Servers: NA Welcome Aboard Again! Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 Main: 612-355-7740 Direct: 612-392-6814 Fax: 952-873-7425 jlumby(a)mnvoip.com

Jan. 31, 2025

Please welcome Minnesota VoIP to MICE!

by Jeremy Lumby

Name: Twin City VoIP d/b/a: Minnesota VoIP Switch: MICE 400G Core - 511 Ports: E 3/1 (1x400G) ASN: This is a Remote Switch uplink upgrade from 300G to 400G IPv4: NA IPv6: NA Peering Contact: peering(a)mnvoip.com Route Servers: NA Welcome Aboard Again! Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 Main: 612-355-7740 Direct: 612-392-6814 Fax: 952-873-7425 jlumby(a)mnvoip.com

Jan. 31, 2025

MICE Switch-Cabinet Migration Round 1

by Jeremy Lumby

On Wednesday February 5th at approximately 1PM CST I will be migrating the following networks off of the Arista 7504R (100Gig core switch) to the new 7804R3 (400Gig core switch) at 511. Prior to the migration BGP will be blocked on the specific participant's port to allow traffic to gracefully reroute. This will be the first round of weekly moves as everything is migrated out of the existing cabinet. If you are interested in upgrading to 400Gig let us know ASAP so we can avoid double moves, and facilitate an easier transition for everyone. Ridgeview core switch uplink Old Ports: Port-channel10 Ethernet3/35/1 Ethernet3/36/1 Ethernet5/14/1 Ethernet5/15/1 Ethernet6/35/1 Ethernet6/36/1 New Ports: Port-channel3 Ethernet3/33/1 Ethernet3/34/1 Ethernet4/32/1 Ethernet4/33/1 Ethernet4/34/1 Ethernet4/36/1 MN VoIP extension switch uplink Old Ports: Port-channel15 Ethernet3/6/1 Ethernet6/6/1 Ethernet6/13/1 New Ports: Port-channel1 Ethernet3/1/1 (Native 400G port) Fiber Minnesota extension switch uplink Old Ports: Port-channel22 Ethernet3/7/1 Ethernet5/13/1 Ethernet5/16/1 Ethernet6/3/1 New Ports: Port-channel4 Ethernet3/2/1 (Native 400G port) Ethernet4/2/1 (Native 400G port) Akamai Old Ports: Port-channel19 Ethernet6/8/1 Ethernet6/10/1 Ethernet6/12/1 Ethernet6/31/1 Ethernet5/7/1 New Ports: Port-channel5 Ethernet3/3/1 Ethernet3/4/1 Ethernet3/5/1 Ethernet3/6/1 Ethernet3/7/1 Amazon Old Ports: Port-channel5 Ethernet3/12/1 Ethernet6/4/1 Ethernet5/3/1 Ethernet5/4/1 New Ports: Port-channel6 Ethernet3/8/1 Ethernet3/9/1 Ethernet3/10/1 Ethernet3/11/1 Apple Old Ports: Port-channel30 Ethernet 3/14/1 New Ports: Port-channel7 Ethernet 3/12/1 Arvig Old Ports: Port-channel42 Ethernet3/8/1 Ethernet6/11/1 Ethernet5/1/1 New Ports: Port-channel8 Ethernet3/13/1 Ethernet4/13/1 Ethernet4/14/1 Fastly Old Ports: Port-channel18 Ethernet4/50/1 New Ports: Port-channel9 Ethernet3/14/1 Meta Old Ports: Port-channel26 Ethernet3/10/1 New Ports: Port-channel10 Ethernet3/15/1 Microsoft Old Ports: Port-channel34 Ethernet3/18/1 New Ports: Port-channel11 Ethernet3/16/1 Netflix Old Ports: Port-channel2 Ethernet3/11/1 Ethernet3/27/1 New Ports: Port-channel12 Ethernet3/17/1 Ethernet3/18/1 Northern Lights GigaPOP Old Ports: Ethernet6/14/1 New Ports: Ethernet3/19/1 Qwilt Old Ports: Port-channel12 Ethernet6/29/1 Ethernet5/2/1 New Ports: Port-channel13 Ethernet3/20/1 Ethernet4/20/1 T-Mobile Old Ports: Port-channel41 Ethernet3/2/1 Ethernet6/9/1 Ethernet6/15/1 New Ports: Port-channel14 Ethernet3/21/1 Ethernet4/21/1 Ethernet4/22/1 Let me know if you have any questions Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington MN 55425 Main 612-355-7740 x211 Direct 612-392-6814 eFax 952-873-7425 jlumby(a)mnvoip.com

Jan. 30, 2025

1G Disconnections

by Richard Laager

The following participants were disconnected yesterday, as they were still on 1G and the 1G switch is going away as part of the cabinet move and 400G switch upgrade. Coloplast [AS25111] Evolve IP (fka Advanced Integrated Tech / AI Tech) [AS13746] ipHouse [AS7753] ipHouse was going away in any event, due to acquisition activity as previously discussed. The other two did confirm they were aware of the disconnect. They elected not to connect at 10G or move to a remote. -- Richard Laager Chief Manager, Director Midwest Internet Cooperative Exchange LLC

Jan. 23, 2025

Re: 2025-01-20 MICE Maintenance - Route Server 2

by Corey Hines

Do note that Arista only supports passive CR DACs. Fixed length Active Optical Cables (AOC) are another option that can be leveraged for short distances but greater than a copper DAC. Here are a number of optical standards based options for 400G transceivers (attached screenshot or inline): [image: image.png] More detail: https://www.arista.com/en/products/transceivers-cables Thanks! Corey Hines Technical Lead, Systems Engineering ACE: L5 Certified ACE: CVP Specialist Arista Networks *612-209-6550chines(a)arista.com <chines(a)arista.com>* *TAC: support(a)arista.com <support(a)arista.com>* www.arista.com On Thu, Jan 23, 2025 at 9:58 AM Brady Kittel <bkittel(a)gmail.com> wrote: > As a server guy I'll mention two things. > > 1) If we're using SM fiber with LR optics I've seen issues with LR optics > burning out on the receive side when used with short distances to servers. > Depends on the optic it seems but it definitely happens. Fix for us was > just to either use SR optics with mm fiber or use a DAC. Definitely avoid > the use of active mode DACs too. I've seen all sorts of problems with > compatibility and even causing OS crashes. Never had problems with hardware > compatible passive mode DACs in my experience across our fleet of 350 > servers. > > On Sun, Jan 19, 2025, 7:50 PM Richard Laager <rlaager(a)micemn.net> wrote: > >> MICE physical server 2 (and thus by extension route server 2) is again >> having errors on its peering interface. We will be performing maintenance >> tomorrow to address this. Since this only affects one of the two route >> servers, participants should not be impacted. >> >> >> Details: >> >> Sometime tomorrow (exact time depending on day job schedules), Jeremy and >> I will: >> >> 1. Apply the usual security updates to VMs and the host, since the host >> is going to be rebooted anyway. >> >> 2. Convert server 2's port to a LAG. This will make future changes >> easier, especially if this error problem continues. >> >> This will involve a reboot of the server. The reboot is "necessary" >> because Linux's netplan does not remove VLAN configuration when changes are >> made (only adds it). Without removing the old >> direct-on-the-physical-interface VLAN config, we could (would?) create a >> loop. You may recall that I created a loop that blew up the exchange in >> more-or-less that exact way when setting up the servers originally. >> >> While I could remove the VLAN configuration manually, in this case, it's >> easier and safer to simply reboot. That also ensures our running >> configuration is fully consistent with the configuration files. >> >> 3. Replace the optic, also moving to the other port on the NIC. >> >> >> The underlying failure manifests as a decrease in optical transmit power >> (i.e. power received at the switch) over time. We have replaced the optic >> previously, which fixes it, but then it fails again after a while. We are >> open to suggestions about why this might be occurring. >> >> One theory is that this is due to heat. We are going to try the second >> port in case that makes a difference. We are also considering switching to >> a direct attach cable, as perhaps that would produce less heat. >> >> -- >> Richard Laager >> Chief Manager, Director >> Midwest Internet Cooperative Exchange LLC >> >> _______________________________________________ >> mice-discuss mailing list -- mice-discuss(a)lists.micemn.net >> To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net >> > _______________________________________________ > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net >

Jan. 23, 2025