- mice-discuss - lists.micemn.net

Loss of connectivity to MICE
by Josh Hoppes Aug. 24, 2026

Aug. 24, 2026

We just saw connectivity to MICE go down as well as our Hurricane Electric transit circuit. Are there any known issues at Cologix Minneapolis? Josh Hoppes Network Engineer II Cedar Falls Utilities Office 319 268 5227 Cell 319 610 5332 www.cfu.net

15 27

RE: [EXTERNAL] Re: Loss of connectivity to MICE
by Jeremy Lumby Aug. 21, 2026

Aug. 21, 2026

I had to request it in the portal and then I was able to get one Jeremy Lumby Minnesota VoIP 9217 17th Ave S Suite 216 Bloomington MN 55425 Main 612-355-7740 Direct 612-392-6814 EFax 952-873-7425 jlumby(a)mnvoip.com -------- Original message -------- From: Justin Huffman via mice-discuss <mice-discuss(a)lists.micemn.net> Date: 8/21/26 4:08 PM (GMT-06:00) To: Brady Farver <bfarver(a)truestreamfiber.com>, mice-discuss(a)lists.micemn.net Subject: Re: [EXTERNAL] Re: Loss of connectivity to MICE _____ From : Justin Huffman via mice-discuss [mice-discuss(a)lists.micemn.net] To : Brady Farver [bfarver(a)truestreamfiber.com] mice-discuss(a)lists.micemn.net [mice-discuss(a)lists.micemn.net] Date : Friday, August 21 2026 16:06:03 Has anyone received a formal RFO yet from Cologix on the outage? Thanks, Justin Long Lines Broadband _____ From: Brady Farver via mice-discuss <mice-discuss(a)lists.micemn.net> Sent: Thursday, August 6, 2026 3:24:00 PM To: mice-discuss(a)lists.micemn.net <mice-discuss(a)lists.micemn.net> Subject: RE: [EXTERNAL] Re: Loss of connectivity to MICE WARNING!! This message originated from an External Source. Please use proper judgment and caution when opening attachments, clicking links, or responding to this email. Response from the Cologix support portal: --- Dear Valued Customer, We are aware of a communication issue affecting certain Facility monitoring systems and are actively working to resolve it. At this time, we are investigating an issue affecting one of the Facility's UPS systems. Our Operations Team is actively monitoring the electrical infrastructure and environmental conditions while the issue is being investigated. We will provide additional updates as more information becomes available. Our Operations Team is actively monitoring the environment while the communication issue is being investigated and resolved. Thank you for your patience and understanding. --- Brady Farver Truestream - Network Operations bfarver(a)truestreamfiber.com <mailto:bfarver@truestreamfiber.com> From: Ryan Gunther via mice-discuss <mice-discuss(a)lists.micemn.net> Sent: Thursday, August 6, 2026 15:21 To: mice-discuss(a)lists.micemn.net Subject: RE: [EXTERNAL] Re: Loss of connectivity to MICE Same with us. Loss connectivity to MICE and Cogent at 511. signature divider Ryan Gunther Network & Switching Engineer CCNA, MEF-CECP, A+, ABCP (406) 234-7458 105 N Montana Ave., Miles City, MT 59301 www.midrivers.com <https://www.midrivers.com/> logo <https://www.midrivers.com/> divider facebook <https://facebook.com/midrivers> instagram <https://www.instagram.com/midriverscommunications/> linkedin <https://www.linkedin.com/company/mid-rivers-communications/> Review Us on Google <http://www.google.com/search?q=Mid-Rivers+Communications,1101%20Main%20 St.,%20Miles%20City,%20MT%2059301> From: Abhi Devireddy <abhi(a)devireddy.com> Sent: Thursday, August 6, 2026 1:09 PM To: Josh Hoppes <Josh.Hoppes(a)cfunet.net>; mice-discuss(a)lists.micemn.net <mailto:mice-discuss@lists.micemn.net> Subject: [EXTERNAL] Re: Loss of connectivity to MICE CAUTION: This email originated from outside of Mid-Rivers or its subsidiaries. Do not click links or open attachments unless you recognize the sender and know the content is safe. We did as well. Lost all outbound connectivity w/ MICE and Cogent. Abhi From: Josh Hoppes <Josh.Hoppes(a)cfunet.net> Date: Thursday, August 6, 2026 at 2:06 PM To: mice-discuss(a)lists.micemn.net <mice-discuss(a)lists.micemn.net> Subject: Loss of connectivity to MICE We just saw connectivity to MICE go down as well as our Hurricane Electric transit circuit. Are there any known issues at Cologix Minneapolis? Josh Hoppes Network Engineer II Cedar Falls Utilities Office 319 268 5227 Cell 319 610 5332 www.cfu.net <http://www.cfu.net/> _______________________________________________ mice-discuss mailing list -- mice-discuss(a)lists.micemn.net <mailto:mice-discuss@lists.micemn.net> To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net <mailto:mice-discuss-leave@lists.micemn.net> _____ </

1 0

MICE Server Updates
by Cameron Skappel Aug. 20, 2026

Aug. 20, 2026

I plan to update the MICE servers on Wednesday, August 19th, at 13:00 Central and Thursday, August 20th, at 13:00 Central. I will update one route server per day. -- Cameron Skappel Wikstrom Telephone Company 218-436-5226

1 2

Please welcome Amazon to MICE!
by Jeremy Lumby Aug. 19, 2026

Aug. 19, 2026

Name: Amazon Switch: MICE Core 511-400G 7804 Ports: 3/14/1 & 3/26/1 in portchannel 16 and 4/14/1 & 4/26/1 to portchannel 17 for a total of 2x2x400G ASN: IPv4: All IP info remains the same, this is an upgrade to 4x400G IPv6: Peering Contact: peering-amer(a)amazon.com Route Servers: NO Welcome Aboard Again! Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 Main: 612-355-7740 Direct: 612-392-6814 Fax: 952-873-7425 jlumby(a)mnvoip.com

1 9

Discussion / Issues Distro List
by Jerry Spaight Aug. 7, 2026

Aug. 7, 2026

Hi - Could you please add Shaun Krug (SKrug(a)atccomm.net<mailto:SKrug@atccomm.net>) to the distribution list? Thanks in advance for your help! Jerry Spaight General Manager [ATC-Com[C]] 85 Main Ave Atkins, IA 52206 Now Offering 2.5 & 5.0 Gigabit Fiber WWW.ATCCOMM.NET<http://www.atccomm.net/> 319-446-0500 - Direct / Desk (Preferred) 319-521-4366 - Mobile 319-446-9100 - Fax

2 1

511 Power Issues
by Jay Hanke Aug. 6, 2026

Aug. 6, 2026

There are currently power issues in the 511 building affecting the MICE infrastructure there. The MICE Ridgeview switch and route server are still operational. -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 507-327-2398 https://southfront.io/ <https://southfront.io/path.html>

3 2

RE: [External] Re: Loss of connectivity to MICE
by Justin Huffman Aug. 6, 2026

Aug. 6, 2026

Yep, seeing 90G of traffic incoming from MICE. _________________________________ Justin Huffman Sr. Manager | Network Engineering Long Lines Broadband 712-271-5902 (W) jhuffman(a)longlines.biz<mailto:jhuffman@longlines.biz> [LL new] From: Tom Krenn <Tom.Krenn(a)hennepincounty.gov> Sent: Thursday, August 6, 2026 3:14 PM To: Jay Hanke <jayhanke(a)southfront.io>; Justin Huffman <jhuffman(a)longlines.biz> Cc: Josh Hoppes <Josh.Hoppes(a)cfunet.net>; mice-discuss(a)lists.micemn.net Subject: RE: [External] Re: Loss of connectivity to MICE You don't often get email from tom.krenn(a)hennepincounty.gov<mailto:tom.krenn@hennepincounty.gov>. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification> WARNING!! This message originated from an External Source. Please use proper judgment and caution when opening attachments, clicking links, or responding to this email. I see things starting to restore, but we are also seeing what looks like 80gbps+ of traffic from MICE toward us. That is extremely unusual. Any routing issues happening on the MICE core or route-reflectors? Tom Krenn Network Architect Enterprise Architecture - Information Technology [Hennepin County logo] From: Jay Hanke via mice-discuss <mice-discuss(a)lists.micemn.net<mailto:mice-discuss@lists.micemn.net>> Sent: Thursday, August 6, 2026 2:35 PM To: Justin Huffman <jhuffman(a)longlines.biz<mailto:jhuffman@longlines.biz>> Cc: Josh Hoppes <Josh.Hoppes(a)cfunet.net<mailto:Josh.Hoppes@cfunet.net>>; mice-discuss(a)lists.micemn.net<mailto:mice-discuss@lists.micemn.net> Subject: [External] Re: Loss of connectivity to MICE CAUTION: This email was sent from outside of Hennepin County. Unless you recognize the sender and know the content, do not click links or open attachments. Sounds like an issue in suite 100. No real details yet. On Thu, Aug 6, 2026 at 2:21 PM Justin Huffman via mice-discuss <mice-discuss(a)lists.micemn.net<mailto:mice-discuss@lists.micemn.net>> wrote: We did as well along with other connections. _________________________________ Justin Huffman Sr. Manager | Network Engineering Long Lines Broadband 712-271-5902 (W) jhuffman(a)longlines.biz<mailto:jhuffman@longlines.biz> -----Original Message----- From: Josh Hoppes <Josh.Hoppes(a)cfunet.net<mailto:Josh.Hoppes@cfunet.net>> Sent: Thursday, August 6, 2026 2:03 PM To: mice-discuss(a)lists.micemn.net<mailto:mice-discuss@lists.micemn.net> Subject: Loss of connectivity to MICE [You don't often get email from josh.hoppes(a)cfunet.net<mailto:josh.hoppes@cfunet.net>. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] WARNING!! This message originated from an External Source. Please use proper judgment and caution when opening attachments, clicking links, or responding to this email. We just saw connectivity to MICE go down as well as our Hurricane Electric transit circuit. Are there any known issues at Cologix Minneapolis? Josh Hoppes Network Engineer II Cedar Falls Utilities Office 319 268 5227 Cell 319 610 5332 http://www.cfu.net/ _______________________________________________ mice-discuss mailing list -- mice-discuss(a)lists.micemn.net<mailto:mice-discuss@lists.micemn.net> To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net<mailto:mice-discuss-leave@lists.micemn.net> _______________________________________________ mice-discuss mailing list -- mice-discuss(a)lists.micemn.net<mailto:mice-discuss@lists.micemn.net> To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net<mailto:mice-discuss-leave@lists.micemn.net> Disclaimer: If you are not the intended recipient of this message, please immediately notify the sender of the transmission error and then promptly permanently delete this message from your computer system.

1 0

RE: Graphing
by Jeremy Lumby April 29, 2026

April 29, 2026

Justin, That would be great if you could check he basics like you mentioned since tomorrow is the fastest I could get out there. Jeremy From: Justin Krejci <Justin.Krejci(a)metronet.com> Sent: Wednesday, April 29, 2026 1:20 PM To: Steve Savoy <stevesavoy(a)wctatel.com>; Jeremy Lumby <jlumby(a)mnvoip.com>; 'MICE Discuss' <mice-discuss(a)lists.micemn.net> Subject: Re: Graphing My router interface appears to have gone down at 2026-04-28T23:32:32.537203-05:00 The interface optic on my side is not receiving light currently from the MICE firewall. I can see about getting someone to go verify the cable is not disconnected and then power cycle the firewall. _____ From: Steve Savoy <stevesavoy(a)wctatel.com <mailto:stevesavoy@wctatel.com> > Sent: Wednesday, April 29, 2026 1:10 PM To: Jeremy Lumby <jlumby(a)mnvoip.com <mailto:jlumby@mnvoip.com> >; 'MICE Discuss' <mice-discuss(a)lists.micemn.net <mailto:mice-discuss@lists.micemn.net> > Subject: RE: Graphing WARNING: This mail is from an external source WARNING: This mail is from an external source Thanks Jeremy!! Steve From: Jeremy Lumby <jlumby(a)mnvoip.com <mailto:jlumby@mnvoip.com> > Sent: Wednesday, April 29, 2026 11:06 AM To: 'MICE Discuss' <mice-discuss(a)lists.micemn.net <mailto:mice-discuss@lists.micemn.net> > Subject: RE: Graphing The correlation is that they go through the firewall/internet connection that is at the Ridgeview datacenter. I am digging into it. From: Dean Bahls <dean.bahls(a)fiberminnesota.com <mailto:dean.bahls@fiberminnesota.com> > Sent: Wednesday, April 29, 2026 6:41 AM To: Steve Savoy <stevesavoy(a)wctatel.com <mailto:stevesavoy@wctatel.com> >; 'MICE Discuss' <mice-discuss(a)lists.micemn.net <mailto:mice-discuss@lists.micemn.net> > Subject: RE: Graphing Seems strange that some ports/devices are graphing fine but others are not. Looks like about 20% of ports are not working. And I'm too ignorant to know the correlation between those that are and those that aren't. Dean From: Steve Savoy <stevesavoy(a)wctatel.com <mailto:stevesavoy@wctatel.com> > Sent: Wednesday, April 29, 2026 6:18 AM To: 'MICE Discuss' <mice-discuss(a)lists.micemn.net <mailto:mice-discuss@lists.micemn.net> > Subject: Graphing Good morning, Anyone else seeing their graphing not working this am? Thank you, Steve Savoy WCTA Confidentiality Notice: This correspondence is the property of Winnebago Cooperative Telecom Association and is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. CAUTION: This email originated outside the organization. Do not click links or open attachments unless you recognize the sender.

2 2

Graphing
by Steve Savoy April 29, 2026

April 29, 2026

Good morning, Anyone else seeing their graphing not working this am? [cid:image001.png@01DCD79F.F39E1DF0] Thank you, Steve Savoy WCTA Confidentiality Notice: This correspondence is the property of Winnebago Cooperative Telecom Association and is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.

3 3

Please welcome T-Mobile USA, Inc to MICE!
by Jeremy Lumby April 19, 2026

April 19, 2026

Name: T-Mobile USA, Inc Switch: MICE Main Port: Ethernet 6/15/1 ASN: 21928 IPv4: This is a port addition to an existing LAG, all IP info remains the same. The new connection will be 3x100G IPv6: Peering Contact: peering(a)t-mobile.com Route Servers: Yes BFD: No Welcome Aboard Again! Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 Main: 612-355-7740 Direct: 612-392-6814 Fax: 952-873-7425 jlumby(a)mnvoip.com

2 9