- mice-discuss - lists.micemn.net

MICE Issues
by Jay Hanke 12 Apr '21

12 Apr '21

Cologix has done power maintenance in Suite 100 that has inadvertently affected the MN VoIP remote switch as well as most of the MICE monitoring. This has isolated the SFN Remote Switch in Albert Lea. Cologix is investigating. No ETTR yet. -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

5 7

MICE UG 39
by Richard Laager 10 Apr '21

10 Apr '21

With NANOG happening this fall, it may make sense to schedule MICE UG 39 at the same time (e.g. Wednesday November 3, time TBD). That's not official yet, but tentatively "save the date". Thanks to the person who suggested this (off-list)! -- Richard

1 0

SFN Remote Uplink Change
by Jay Hanke 01 Apr '21

01 Apr '21

SFN has upgraded the uplink LAG group on the SFN MICE remote switch to 12x10G from 8x10G. We were getting a small amount of tail drops where we convert from 100GE to the LAG group. Thanks! -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

1 0

belle plaine
by James Urwiller 01 Apr '21

01 Apr '21

anyone on the belle plain remote just loose connection? we are coming in from fremont, NE.

5 7

Please welcome Xiber LLC to MICE!
by Jay Hanke 30 Mar '21

30 Mar '21

Name: Xiber LLC Port: Main Ethernet 4/46 ASN: 393950 IPv4: 206.108.255.178/24 IPv6: 2001:504:27:0:6:2de:0:1/64 Peering Contact: peering(a)xiber.net Route Servers- Yes BFD: Yes This email has been filed with Cologix as your LOA/CFA. Members are no longer required to provide optics for installation in the main switch. Welcome aboard! -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

2 1

Nanog!
by Reid Fishler 30 Mar '21

30 Mar '21

It was just announced the first in person Nanog is in MSP Nov 1-3! Reid -- Reid Fishler Senior Director Hurricane Electric +1-510-580-4178

2 1

Please Welcome Range (AS30110) to MICE!
by Jay Hanke 24 Mar '21

24 Mar '21

Name: Range Port: Main Ethernet 5/38 ASN: 30110 IPv4: 206.108.255.176/24 IPv6: 2001:504:27::759e:0:1/64 Peering Contact: dave.berry(a)range.net Route Servers- Yes BFD: Yes This email has been filed with Cologix as your LOA/CFA. Welcome aboard! -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

2 1

Facebook Peers Drop
by Travis Fincham 21 Mar '21

21 Mar '21

Did anyone else have their peering with Facebook drop this morning about 7:00 AM? Travis Fincham, CCNA ACS Network Engineer Email: Travis.Fincham(a)keycoop.com<mailto:Travis.Fincham@keycoop.com> Cell: (515)-735-5011 ________________________________ NOTICE: This message (including attachments) is covered by the Electronic Communications Privacy Act, 18 U.S.C. § 2510-2521, is CONFIDENTIAL, and is intended only for the individual to whom it is addressed. If you are not the named addressee or an employee or agent responsible for delivering this message to the named addressee, you are hereby notified that you have received this message in error and that any review, retention, dissemination, distribution, copying or use of this communication is strictly prohibited. Please reply to the sender that you received this message in error and then delete it.

4 3

AS16842 Disconnect Notice
by Jay Hanke 16 Mar '21

16 Mar '21

AS16842 has disconnected from MICE. Please remove their route server and port configurations. 206.108.255.39 will be returned to the free pool as well as ge-1/1/0 on 1G main switch. -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

2 1

MICE Terabit Dreams?
by Jay Hanke 15 Mar '21

15 Mar '21

When do you think we'll cross over 1 terabit? [image: image.png] Blue line is more accurate as IXP Manager isn't able to grab stats from the Zayo remote. -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

3 2

Please welcome Les.net to MICE!
by Jay Hanke 15 Mar '21

15 Mar '21

Name: Les.net Port: MN VoIP 100G Ethernet4/1 ASN: 18451 IPv4: 206.108.255.175/24 IPv6: 2001:504:27::4813:0:1/64 Peering Contact: arin(a)les.net Route Servers- Yes BFD: NO Welcome aboard! -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

2 1

Supranet (AS4150) Port Move
by Jay Hanke 13 Mar '21

13 Mar '21

Supra net has moved from the 1G main Switch to port Ethernet7 on the MN VoIP remote 10G switch. -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

1 0

Oneneck Port Upgrade to 10G
by Jay Hanke 13 Mar '21

13 Mar '21

Oneneck has been allocated the following port on the MAIN Switch MICE MAIN :: Ethernet3/7/1 This will replace their existing 1G connection. All IP/Route Server information will remain the same. -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

1 1

Lost MICE VMWare server
by Jay Hanke 12 Mar '21

12 Mar '21

The VMWare server that hosts RS3 and IXP Manager for MICE died. We're onsite working on restoration now. -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

4 5

Please Welcome Metronet to MICE!
by Jay Hanke 09 Mar '21

09 Mar '21

Name: Metronet Port: Main Ethernet6/7/1 (100G New Single Member LAG) ASN: 30600 IPv4: 206.108.255.174/24 IPv6: 2001:504:27::7788:0:1/64 Peering Contact: peering(a)metronetinc.com Route Servers- NO BFD: NO Please note this is in addition to the existing Jaguar/Metronet ports not a replacement. Welcome aboard! -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

2 1

Re: Packet Loss between 00:00 and 05:00 this morning
by Jeremy Lumby 06 Mar '21

06 Mar '21

They do come in on one of the Zayo circuits that was impacted last night. Jeremy Lumby Minnesota VoIP 9217 17th Ave S Suite 216 Bloomington MN 55425 Main 612-355-7740 Direct 612-392-6814 EFax 952-873-7425 jlumby(a)mnvoip.com -------- Original message -------- From: Frank Bulk <fbulk(a)MYPREMIERONLINE.COM> Date: 3/6/21 2:45 PM (GMT-06:00) To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Packet Loss between 00:00 and 05:00 this morning _____ >From : Frank Bulk [fbulk(a)MYPREMIERONLINE.COM] To : MICE-DISCUSS(a)LISTS.IPHOUSE.NET [MICE-DISCUSS(a)LISTS.IPHOUSE.NET] Date : Saturday, March 6 2021 14:45:37 Our Qrator monitoring noticed that BANKONIT’s BGP session going in and out many times between 11:17 pm to 5:59 am Frank From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Keith Wirch Sent: Saturday, March 6, 2021 9:29 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] Packet Loss between 00:00 and 05:00 this morning Anyone else experience intermittent packet loss between 00:00 and 05:00 this morning? We noticed a bunch of our IPSEC tunnels were coming and going and website availability was hit and miss. We thought we saw loss in Level3 so we turned down Peering but we still were seeing loss on other providers. Didn’t seem to be MICE specific but were not able to pinpoint the issue exactly. Keith Wirch Infrastructure Engineer ---- Atomic Data® // Safe. Simple. Smart.® 250 Marquette Avenue Minneapolis, MN 55401 Main // 612.466.2000 Direct // 612.466.2106 _____ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

1 0

Packet Loss between 00:00 and 05:00 this morning
by Keith Wirch 06 Mar '21

06 Mar '21

2 1

Re: Packet Loss between 00:00 and 05:00 this morning
by Jeremy Lumby 06 Mar '21

06 Mar '21

I had some Zayo circuits leaving Minneapolis that were experiencing high packet loss during that exact same time frame Jeremy Lumby Minnesota VoIP 9217 17th Ave S Suite 216 Bloomington MN 55425 Main 612-355-7740 Direct 612-392-6814 EFax 952-873-7425 jlumby(a)mnvoip.com -------- Original message -------- From: Keith Wirch <kwirch(a)ATOMICDATA.COM> Date: 3/6/21 9:29 AM (GMT-06:00) To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] Packet Loss between 00:00 and 05:00 this morning _____ >From : Keith Wirch [kwirch(a)ATOMICDATA.COM] To : MICE-DISCUSS(a)LISTS.IPHOUSE.NET [MICE-DISCUSS(a)LISTS.IPHOUSE.NET] Date : Saturday, March 6 2021 09:29:00 Anyone else experience intermittent packet loss between 00:00 and 05:00 this morning? We noticed a bunch of our IPSEC tunnels were coming and going and website availability was hit and miss. We thought we saw loss in Level3 so we turned down Peering but we still were seeing loss on other providers. Didn’t seem to be MICE specific but were not able to pinpoint the issue exactly. Keith Wirch Infrastructure Engineer ---- Atomic Data® // Safe. Simple. Smart.® 250 Marquette Avenue Minneapolis, MN 55401 Main // 612.466.2000 Direct // 612.466.2106

1 0

Hulu Live/Fastly
by Frank Bulk 02 Mar '21

02 Mar '21

We've received 5 to 7 reports about Hulu Live issues ("freezing up") in the last week or so. They're not all on Roku as we've had at least one person with an Amazon Firestick. For those where we've identified the source IP it's been Fastly, which is present at MICE. Curious if anyone else has received an uptick in Hulu Live complaints? We have not (yet) been able to reproduce the issue internally. Other streaming services at these same customers appear fine. Raw throughput in speed tests to our speed test server are fine. The one customer that we know did call Hulu was told by Hulu support that their issue was with their ISP (us). Frank

2 2

MPW Port Move
by Jay Hanke 26 Feb '21

26 Feb '21

MPW has moved from Port Ethernet 7 on the MN VoIP switch to port Ethernet 12 on the SFN remote. -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

1 0

MICE UG 37
by Richard Laager 26 Feb '21

26 Feb '21

Reminder: UG 37 is on Wednesday, March 10, at 3:00pm This will be on Google Meet: https://meet.google.com/zgg-eeyu-dxz In the event of serious problems, we will fall back to: 952-562-5400 passcode MICE (6423) Agenda The agenda is attached. If you have something to add, let me know. The UG 36 draft minutes are also attached. -- Richard

1 0

Sling TV
by Jeremy Lumby 23 Feb '21

23 Feb '21

Has anyone else been experiencing issues with Sling TV's cloud DVR service, especially during peak hours within the past couple months? While they are not directly connected to MICE, some of their transit providers are. Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 M: 612-355-7740 D: 612-392-6814 F: 952-873-7425 jlumby(a)mnvoip.com

1 0

SFN MICE Remote Scheduled Maintenance
by Jay Hanke 23 Feb '21

23 Feb '21

The SFN Remote switch will be undergoing maintenance and will be power cycled during our transport provider doing software upgrades. The SFN remote switch will be hard down during the part of the maintenance and be isolated during part of the maintenance. Connectivity may be intermittent during the entire window as software updates are applied. This will also affect test filtering sessions to MICE-RS3. Primary Window: FROM: Tuesday, Feb 23, 2021 0030 hours (CST) TO: Tuesday, Feb 23, 2021 0230 hours (CST) Backup Window: FROM: Tuesday, Feb 24, 2021 0030 hours (CST) TO: Tuesday, Feb 24, 2021 0230 hours (CST) Thanks! -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

1 1

Apple links filling up?
by Frank Bulk 23 Feb '21

23 Feb '21

The last two nights our NAGIOS-based monitoring of HTTPv6 access to http://www.apple.com has been timing out from time to time. Am I interpreting MICE's graphs correctly to suggest that Apple's two (1x10G?) LAGs are filling up? Frank

2 2

HE.net Routes Lost - Around 15:05 CST ?
by Jesse Reed 19 Feb '21

19 Feb '21

Anyone else lose routes via HE shortly after 15:00 CST today? It was brief for us, but wondering if anyone else experienced this? Out BGP neighbor(s) remained stable but it appears like we dropped traffic during this time. Thanks ahead of time, Jesse Reed Network Operations Specialist [cid:image001.jpg@01D706D4.B26536D0] jreed(a)AcenTek.net<mailto:jreed@AcenTek.net> www.AcenTek.net

3 2