- mice-discuss - lists.micemn.net

Re: MICE connectivity issues / elevated multicast traffic

by Anthony Anderberg

Andrew’s theory about uplinks is correct: Ridgeview 3/35 and Po1 are part of its link to 511-7804 And 511-7504 po11 and 6/36 are part of its link to 511-7804 I don’t have any theories but spent a little time looking around, my notes so far: The usual background of non-unicast packets jumped from ~100/sec to ~250/sec rom 14:10 to 15:01 They were egressing lots of ports but po4 (Charter) is the only ingress port I’ve found so far - but I don’t think it’s volume accounts for it all. A number of members had traffic volumes change but the only ones who stopped all together that I see are: Integris and Hoyos on your remote WiscNet on their remote Conversely Xcel Entergy had a spike in traffic at around that time. And then we had trouble graphing NOCIX remote ports during that time too, which is odd. From: Andrew Hoyos <hoyosa(a)gmail.com> Date: Wednesday, April 30, 2025 at 4:08 PM To: Jay Hanke <jayhanke(a)southfront.io> Cc: Sergii Polishchuk <sergi(a)infotec.ca>, Ryan Nelson <ryan.nelson(a)integrisit.com>, "mice-discuss(a)lists.micemn.net" <mice-discuss(a)lists.micemn.net> Subject: Re: MICE connectivity issues / elevated multicast traffic We are, but saw issues to all peers (non BFD peers as well) There was definitely some uptick in non-unicast traffic in a few spots. I’m sure some are switch to switch links... 7804 - 3/2/1 -> ridgeview http://micelg.usinternet.com/cacti/graph.php?action=view&local_graph_id=402… -> Ridgeview 3/35/1 + po1 7504 - po11 + 6/36/1 http://micelg.usinternet.com/cacti/graph.php?action=view&local_graph_id=185… Andrew Hoyos hoyosa(a)gmail.com On Apr 30, 2025, at 15:52, Jay Hanke via mice-discuss <mice-discuss(a)lists.micemn.net> wrote: Is everyone with issues running BFD? On Wed, Apr 30, 2025 at 3:48 PM Sergii Polishchuk <sergi(a)infotec.ca<mailto:sergi@infotec.ca>> wrote: We lost connection with both route-servers for a minute and direct peering with some members (but not all): Apr 30 14:02:18: %BGP-5-NBR_RESET: Neighbor 206.108.255.2 reset (BFD adjacency down) Apr 30 14:02:19: %BGP-5-ADJCHANGE: neighbor 206.108.255.2 Down BFD adjacency down Apr 30 14:02:19: %BGP_SESSION-5-ADJCHANGE: neighbor 206.108.255.2 IPv4 Unicast topology base removed from session BFD adjacency down Apr 30 14:02:35: %BGP-5-NBR_RESET: Neighbor 206.108.255.2 active reset (Peer closed the session) Apr 30 14:03:13: %BGP-5-ADJCHANGE: neighbor 206.108.255.2 Up Ping to AS42: <image009.png> Ping to AS6939: <image010.png> <image002.png><https://www.facebook.com/InfotecManitoba> <image003.png><https://twitter.com/InfotecManitoba> <image004.png><https://www.linkedin.com/in/sergip/> <image005.png> Sergi Infotec Manitoba <image006.png> (844) 777-3744 <tel:(844)%20777-3744> <image007.png> sergi(a)infotec.ca<mailto:sergi@infotec.ca> <image008.png> https://infotec.ca<https://infotec.ca/> From: Ryan Nelson via mice-discuss <mice-discuss(a)lists.micemn.net<mailto:mice-discuss@lists.micemn.net>> Sent: April 30, 2025 15:02 To: mice-discuss(a)lists.micemn.net<mailto:mice-discuss@lists.micemn.net> Subject: MICE connectivity issues / elevated multicast traffic Anyone else seeing some connectivity issues over MICE coinciding with an uptick in multicast traffic coming from the core starting at around 2:10pm CDT? Ryan Nelson Architect, Platform Services Office: (218) 336-2220<tel:(218)%20336-2220> ryan.nelson(a)integrisit.com<mailto:ryan.nelson@integrisit.com> <image001.png><https://integrisit.com/> Integris Engage<https://integrisit.com/resources/>: Check out upcoming Education, Training & Networking _______________________________________________ mice-discuss mailing list -- mice-discuss(a)lists.micemn.net<mailto:mice-discuss@lists.micemn.net> To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net<mailto:mice-discuss-leave@lists.micemn.net> _______________________________________________ mice-discuss mailing list -- mice-discuss(a)lists.micemn.net To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net

April 30, 2025

RE: MICE connectivity issues / elevated multicast traffic

by Jeremy Lumby

I have not been able to dig into a too deeply yet but there does appear to be a large amount of ARP and neighbor Discovery traffic coming from a handful of Networks Jeremy Lumby Minnesota VoIP 9217 17th Ave S Suite 216 Bloomington MN 55425 Main 612-355-7740 Direct 612-392-6814 EFax 952-873-7425 jlumby(a)mnvoip.com -------- Original message -------- From: Ryan Nelson via mice-discuss <mice-discuss(a)lists.micemn.net> Date: 4/30/25 3:02 PM (GMT-06:00) To: mice-discuss(a)lists.micemn.net Subject: MICE connectivity issues / elevated multicast traffic _____ From : Ryan Nelson via mice-discuss [mice-discuss(a)lists.micemn.net] To : mice-discuss(a)lists.micemn.net [mice-discuss(a)lists.micemn.net] Date : Wednesday, April 30 2025 15:01:33 Anyone else seeing some connectivity issues over MICE coinciding with an uptick in multicast traffic coming from the core starting at around 2:10pm CDT? Ryan Nelson Architect, Platform Services Office: (218) 336-2220 <tel:(218)%20336-2220> ryan.nelson(a)integrisit.com <https://integrisit.com/> Integris Engage <https://integrisit.com/resources/> : Check out upcoming Education, Training & Networking

April 30, 2025

Re: MICE connectivity issues / elevated multicast traffic

by Andrew Hoyos

We are, but saw issues to all peers (non BFD peers as well) There was definitely some uptick in non-unicast traffic in a few spots. I’m sure some are switch to switch links... 7804 - 3/2/1 -> ridgeview http://micelg.usinternet.com/cacti/graph.php?action=view&local_graph_id=402… -> Ridgeview 3/35/1 + po1 7504 - po11 + 6/36/1 http://micelg.usinternet.com/cacti/graph.php?action=view&local_graph_id=185… Andrew Hoyos hoyosa(a)gmail.com > On Apr 30, 2025, at 15:52, Jay Hanke via mice-discuss <mice-discuss(a)lists.micemn.net> wrote: > > Is everyone with issues running BFD? > > On Wed, Apr 30, 2025 at 3:48 PM Sergii Polishchuk <sergi(a)infotec.ca <mailto:sergi@infotec.ca>> wrote: >> We lost connection with both route-servers for a minute and direct peering with some members (but not all): >> >> >> >> Apr 30 14:02:18: %BGP-5-NBR_RESET: Neighbor 206.108.255.2 reset (BFD adjacency down) >> >> Apr 30 14:02:19: %BGP-5-ADJCHANGE: neighbor 206.108.255.2 Down BFD adjacency down >> >> Apr 30 14:02:19: %BGP_SESSION-5-ADJCHANGE: neighbor 206.108.255.2 IPv4 Unicast topology base removed from session BFD adjacency down >> >> Apr 30 14:02:35: %BGP-5-NBR_RESET: Neighbor 206.108.255.2 active reset (Peer closed the session) >> >> Apr 30 14:03:13: %BGP-5-ADJCHANGE: neighbor 206.108.255.2 Up >> >> >> >> Ping to AS42: >> >> >> >> <image009.png> >> >> >> >> Ping to AS6939: >> >> >> >> <image010.png> >> >> >> >> <image002.png> <https://www.facebook.com/InfotecManitoba> >> <image003.png> <https://twitter.com/InfotecManitoba> >> <image004.png> <https://www.linkedin.com/in/sergip/> >> >> >> <image005.png> >> >> Sergi >> >> Infotec Manitoba >> >> <image006.png> >> >> (844) 777-3744 <tel:(844)%20777-3744> >> <image007.png> >> >> sergi(a)infotec.ca <mailto:sergi@infotec.ca> >> <image008.png> >> >> https://infotec.ca <https://infotec.ca/> >> >> >> >> >> >> >> >> >> From: Ryan Nelson via mice-discuss <mice-discuss(a)lists.micemn.net <mailto:mice-discuss@lists.micemn.net>> >> Sent: April 30, 2025 15:02 >> To: mice-discuss(a)lists.micemn.net <mailto:mice-discuss@lists.micemn.net> >> Subject: MICE connectivity issues / elevated multicast traffic >> >> >> >> Anyone else seeing some connectivity issues over MICE coinciding with an uptick in multicast traffic coming from the core starting at around 2:10pm CDT? >> >> Ryan Nelson >> >> Architect, Platform Services >> >> Office: (218) 336-2220 <tel:(218)%20336-2220> >> ryan.nelson(a)integrisit.com <mailto:ryan.nelson@integrisit.com> >> <image001.png> <https://integrisit.com/> >> Integris Engage <https://integrisit.com/resources/>: Check out upcoming Education, Training & Networking >> >> >> >> >> _______________________________________________ >> mice-discuss mailing list -- mice-discuss(a)lists.micemn.net <mailto:mice-discuss@lists.micemn.net> >> To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net <mailto:mice-discuss-leave@lists.micemn.net> > _______________________________________________ > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net

April 30, 2025

Re: MICE connectivity issues / elevated multicast traffic

by Ryan Nelson

Jay, We are running BFD. Ryan Nelson Architect, Platform Services Office: (218) 336-2220<tel:(218)%20336-2220> ryan.nelson(a)integrisit.com<mailto:ryan.nelson@integrisit.com> [cid:image933818.png]<https://integrisit.com/> Integris Engage<https://integrisit.com/resources/>: Check out upcoming Education, Training & Networking From: Jay Hanke <jayhanke(a)southfront.io> Date: Wednesday, April 30, 2025 at 3:52 PM To: Sergii Polishchuk <sergi(a)infotec.ca> Cc: Ryan Nelson <ryan.nelson(a)integrisit.com>, mice-discuss(a)lists.micemn.net <mice-discuss(a)lists.micemn.net> Subject: Re: MICE connectivity issues / elevated multicast traffic EXTERNAL MESSAGE Is everyone with issues running BFD? On Wed, Apr 30, 2025 at 3:48 PM Sergii Polishchuk <sergi(a)infotec.ca<mailto:sergi@infotec.ca>> wrote: We lost connection with both route-servers for a minute and direct peering with some members (but not all): Apr 30 14:02:18: %BGP-5-NBR_RESET: Neighbor 206.108.255.2 reset (BFD adjacency down) Apr 30 14:02:19: %BGP-5-ADJCHANGE: neighbor 206.108.255.2 Down BFD adjacency down Apr 30 14:02:19: %BGP_SESSION-5-ADJCHANGE: neighbor 206.108.255.2 IPv4 Unicast topology base removed from session BFD adjacency down Apr 30 14:02:35: %BGP-5-NBR_RESET: Neighbor 206.108.255.2 active reset (Peer closed the session) Apr 30 14:03:13: %BGP-5-ADJCHANGE: neighbor 206.108.255.2 Up Ping to AS42: [cid:ii_1968878db2dbdbb399] Ping to AS6939: [cid:ii_1968878db2d1f3c04fa] [facebook]<https://www.facebook.com/InfotecManitoba> [twitter]<https://twitter.com/InfotecManitoba> [linkedin]<https://www.linkedin.com/in/sergip/> [cid:ii_1968878db2d855d355] Sergi Infotec Manitoba [cid:ii_1968878db2d9374b66] (844) 777-3744<tel:(844)%20777-3744> [cid:ii_1968878db2da18c377] sergi(a)infotec.ca<mailto:sergi@infotec.ca> [cid:ii_1968878db2dafa3b88] https://infotec.ca<https://infotec.ca/> From: Ryan Nelson via mice-discuss <mice-discuss(a)lists.micemn.net<mailto:mice-discuss@lists.micemn.net>> Sent: April 30, 2025 15:02 To: mice-discuss(a)lists.micemn.net<mailto:mice-discuss@lists.micemn.net> Subject: MICE connectivity issues / elevated multicast traffic Anyone else seeing some connectivity issues over MICE coinciding with an uptick in multicast traffic coming from the core starting at around 2:10pm CDT? Ryan Nelson Architect, Platform Services Office: (218) 336-2220<tel:(218)%20336-2220> ryan.nelson(a)integrisit.com<mailto:ryan.nelson@integrisit.com> [cid:ii_1968878db2c4cff311]<https://integrisit.com/> Integris Engage<https://integrisit.com/resources/>: Check out upcoming Education, Training & Networking _______________________________________________ mice-discuss mailing list -- mice-discuss(a)lists.micemn.net<mailto:mice-discuss@lists.micemn.net> To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net<mailto:mice-discuss-leave@lists.micemn.net>

April 30, 2025

Re: MICE connectivity issues / elevated multicast traffic

by Jay Hanke

Is everyone with issues running BFD? On Wed, Apr 30, 2025 at 3:48 PM Sergii Polishchuk <sergi(a)infotec.ca> wrote: > We lost connection with both route-servers for a minute and direct peering > with some members (but not all): > > > > Apr 30 14:02:18: %BGP-5-NBR_RESET: Neighbor 206.108.255.2 reset (BFD > adjacency down) > > Apr 30 14:02:19: %BGP-5-ADJCHANGE: neighbor 206.108.255.2 Down BFD > adjacency down > > Apr 30 14:02:19: %BGP_SESSION-5-ADJCHANGE: neighbor 206.108.255.2 IPv4 > Unicast topology base removed from session BFD adjacency down > > Apr 30 14:02:35: %BGP-5-NBR_RESET: Neighbor 206.108.255.2 active reset > (Peer closed the session) > > Apr 30 14:03:13: %BGP-5-ADJCHANGE: neighbor 206.108.255.2 Up > > > > Ping to AS42: > > > > > > Ping to AS6939: > > > > > > [image: facebook] <https://www.facebook.com/InfotecManitoba> > > [image: twitter] <https://twitter.com/InfotecManitoba> > > [image: linkedin] <https://www.linkedin.com/in/sergip/> > > > > *Sergi * > > Infotec Manitoba > > > > (844) 777-3744 > > sergi(a)infotec.ca > > https://infotec.ca > > > > > > > > > > > > *From:* Ryan Nelson via mice-discuss <mice-discuss(a)lists.micemn.net> > *Sent:* April 30, 2025 15:02 > *To:* mice-discuss(a)lists.micemn.net > *Subject:* MICE connectivity issues / elevated multicast traffic > > > > Anyone else seeing some connectivity issues over MICE coinciding with an > uptick in multicast traffic coming from the core starting at around 2:10pm > CDT? > > *Ryan Nelson* > > *Architect, Platform Services* > > *Office:* (218) 336-2220 <(218)%20336-2220> > > ryan.nelson(a)integrisit.com > > <https://integrisit.com/> > > Integris Engage <https://integrisit.com/resources/> > : Check out upcoming Education, Training & Networking > > > _______________________________________________ > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net >

April 30, 2025

RE: MICE connectivity issues / elevated multicast traffic

by Sergii Polishchuk

We lost connection with both route-servers for a minute and direct peering with some members (but not all): Apr 30 14:02:18: %BGP-5-NBR_RESET: Neighbor 206.108.255.2 reset (BFD adjacency down) Apr 30 14:02:19: %BGP-5-ADJCHANGE: neighbor 206.108.255.2 Down BFD adjacency down Apr 30 14:02:19: %BGP_SESSION-5-ADJCHANGE: neighbor 206.108.255.2 IPv4 Unicast topology base removed from session BFD adjacency down Apr 30 14:02:35: %BGP-5-NBR_RESET: Neighbor 206.108.255.2 active reset (Peer closed the session) Apr 30 14:03:13: %BGP-5-ADJCHANGE: neighbor 206.108.255.2 Up Ping to AS42: Ping to AS6939: <https://www.facebook.com/InfotecManitoba> <https://twitter.com/InfotecManitoba> <https://www.linkedin.com/in/sergip/> Sergi Infotec Manitoba <tel:(844)%20777-3744> (844) 777-3744 <mailto:sergi@infotec.ca> sergi(a)infotec.ca <https://infotec.ca/> https://infotec.ca From: Ryan Nelson via mice-discuss <mice-discuss(a)lists.micemn.net> Sent: April 30, 2025 15:02 To: mice-discuss(a)lists.micemn.net Subject: MICE connectivity issues / elevated multicast traffic Anyone else seeing some connectivity issues over MICE coinciding with an uptick in multicast traffic coming from the core starting at around 2:10pm CDT? Ryan Nelson Architect, Platform Services Office: <tel:(218)%20336-2220> (218) 336-2220 <mailto:ryan.nelson@integrisit.com> ryan.nelson(a)integrisit.com <https://integrisit.com/> <https://integrisit.com/resources/> Integris Engage: Check out upcoming Education, Training & Networking

April 30, 2025

MICE connectivity issues / elevated multicast traffic

by Ryan Nelson

Anyone else seeing some connectivity issues over MICE coinciding with an uptick in multicast traffic coming from the core starting at around 2:10pm CDT? Ryan Nelson Architect, Platform Services Office: (218) 336-2220<tel:(218)%20336-2220> ryan.nelson(a)integrisit.com<mailto:ryan.nelson@integrisit.com> [cid:image854150.png]<https://integrisit.com/> Integris Engage<https://integrisit.com/resources/>: Check out upcoming Education, Training & Networking

April 30, 2025

Re: MICE at 2T

by Anthony Anderberg

The good news is that I’ve finally (!) had a block of time to work on this - and am confident that the Aggregate graph now reflects reality, both today and historically. The bad news is that I was correct that I’d been double counting a few hundred gig by including ports that Jeremy added to our backbone links during this spring’s changes, so we’re back under 2T - for now at least. Cheers, Anthony From: David Farmer <farmer(a)umn.edu> Date: Saturday, March 29, 2025 at 7:33 PM To: Anthony Anderberg <AnthonyAnderberg(a)nuvera.net> Cc: "mice-discuss(a)lists.micemn.net" <mice-discuss(a)lists.micemn.net> Subject: Re: MICE at 2T Have you been able to validate the graphs? Are we regularly gating to 2 Tb now? On Sun, Mar 23, 2025 at 10:39 PM Anthony Anderberg <AnthonyAnderberg(a)nuvera.net<mailto:AnthonyAnderberg@nuvera.net>> wrote: I think I might be double-counting the packets that transverse the 511 building 7504-7804 link… I’ll look at that tomorrow and verify if I excluded the members of that LAG properly. Cheers, Anthony Sent from my iPhone On Mar 22, 2025, at 7:35 PM, David Farmer via mice-discuss <mice-discuss(a)lists.micemn.net<mailto:mice-discuss@lists.micemn.net>> wrote: I think Mice broke 2T in the last few days. But I'm having a hard time getting to the cacti graphs to confirm. Thanks. =============================================== David Farmer Email:farmer@umn.edu<mailto:Email%3Afarmer@umn.edu> Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 =============================================== _______________________________________________ mice-discuss mailing list -- mice-discuss(a)lists.micemn.net<mailto:mice-discuss@lists.micemn.net> To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net<mailto:mice-discuss-leave@lists.micemn.net> -- =============================================== David Farmer Email:farmer@umn.edu<mailto:Email%3Afarmer@umn.edu> Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 ===============================================

April 30, 2025

Re: Non-MICE/perhaps dumb question

by Mike Horwath

On Fri, Apr 25, 2025 at 08:53:28PM +0000, Frank Bulk wrote: > Is that just the reality of it, or is there a broader product (other > than direct peering with Amazon) that would improve AWS performance > for all our customers? Is your customer willing to cover costs for direct connect? In my experience, this is very performant (depending on link speed you choose). Are they pushing into S3, some API endpoint, custom app, over SSH or FTP or another protocol? From home (over the comcastic networkz) I can saturate my upload (~350Mbps) without issue, and lab environment (~500Mbps, Atomic Data) can also hit the peak. This is over SSH (into instances) or S3 endpoints. -- Mike Horwath, reachable via drechsau(a)Geeks.ORG

April 26, 2025

RE: Non-MICE/perhaps dumb question

by Jeremy Lumby

Correct, Waves are very transparent to errors and the carrier selling them cannot oversubscribe them, as well as the path is known and not easily changed. Most IXes are quite transparent, and require extension uplink upgrades at 66% as well as path diversity. You get a lot of stability/predictability since most of those things you do not get if you are just buying transit. From: Ben Wiechman via mice-discuss <mice-discuss(a)lists.micemn.net> Sent: Friday, April 25, 2025 4:51 PM To: Jeremy Lumby <jlumby(a)mnvoip.com> Cc: mice-discuss(a)lists.micemn.net Subject: Re: Non-MICE/perhaps dumb question So your solution is to provide transport to a location closer to the AWS region, and then the customer establishes a private connection to AWS from there? Megaport may provide a similar transport concept. They have transport to many different DC in the US, and also had a cloud router product designed to allow remote routing for public cloud traffic w/o installing hardware the remote DC. I haven't looked at their solution in a long time to give a better judgement call regarding its viability for this use case, or whether it would only be useful on a customer by customer basis where each customer would need to have their own instance of the cloud switching in relevant data centers. This may also only be useful if direct connectivity would provide a benefit or if the customer has a lot of E/W traffic that could route remotely between public cloud applications. Ben Wiechman Director of Network Strategy and Engineering 320.247.3224 | ben.wiechman(a)arvig.com <mailto:ben.wiechman@arvig.com> Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com <http://arvig.com> On Fri, Apr 25, 2025 at 4:23 PM Jeremy Lumby <jlumby(a)mnvoip.com <mailto:jlumby@mnvoip.com> > wrote: I do have customers that utilize my extension switches to other IXes so that they can connect to Amazon directly in other regions. Unfortunately I have a west coast option, and you are looking for an east coast option. From: Frank Bulk <fbulk(a)mypremieronline.com <mailto:fbulk@mypremieronline.com> > Sent: Friday, April 25, 2025 3:53 PM To: mice-discuss(a)lists.micemn.net <mailto:mice-discuss@lists.micemn.net> Subject: Non-MICE/perhaps dumb question I’ve been reviewing some of AWS’s connectivity products, but it seems like none of them are designed to improve the AWS experience for all of an ISP’s customers across AWS’s various regions. The background here is that we have a business customer using a key application hosted in AWS’s East Coast region. The applications’ document upload performance has been less than ideal, and their external consultant is blaming their ISP — us. As you know, once traffic leaves our network and hits upstream providers, our influence on the experience is very limited. My initial thought was to explore whether we could bring AWS closer to all of our customers through some kind of direct peering or other arrangement with AWS. However, from what I’m seeing, AWS’s connectivity options are mostly targeted at connecting a single enterprise to AWS, not enhancing general public access to AWS services. Is that just the reality of it, or is there a broader product (other than direct peering with Amazon) that would improve AWS performance for all our customers? Frank Frank Bulk Chief Technology Officer / Chief Operating Officer 339 1st Ave NE | Sioux Center, IA 51250 | <https://www.facebook.com/PremierCommunicationsSiouxCenter> Find us on Facebook phone: 712.722.3451 web: mypremieronline.com <https://www.mypremieronline.com/> direct: (712) 722-7152 cell: (712) 441-6049 _______________________________________________ mice-discuss mailing list -- mice-discuss(a)lists.micemn.net <mailto:mice-discuss@lists.micemn.net> To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net <mailto:mice-discuss-leave@lists.micemn.net>

April 25, 2025

Re: Non-MICE/perhaps dumb question

by Ben Wiechman

So your solution is to provide transport to a location closer to the AWS region, and then the customer establishes a private connection to AWS from there? Megaport may provide a similar transport concept. They have transport to many different DC in the US, and also had a cloud router product designed to allow remote routing for public cloud traffic w/o installing hardware the remote DC. I haven't looked at their solution in a long time to give a better judgement call regarding its viability for this use case, or whether it would only be useful on a customer by customer basis where each customer would need to have their own instance of the cloud switching in relevant data centers. This may also only be useful if direct connectivity would provide a benefit or if the customer has a lot of E/W traffic that could route remotely between public cloud applications. Ben Wiechman Director of Network Strategy and Engineering 320.247.3224 | ben.wiechman(a)arvig.com Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com On Fri, Apr 25, 2025 at 4:23 PM Jeremy Lumby <jlumby(a)mnvoip.com> wrote: > I do have customers that utilize my extension switches to other IXes so > that they can connect to Amazon directly in other regions. Unfortunately > I have a west coast option, and you are looking for an east coast option. > > > > *From:* Frank Bulk <fbulk(a)mypremieronline.com> > *Sent:* Friday, April 25, 2025 3:53 PM > *To:* mice-discuss(a)lists.micemn.net > *Subject:* Non-MICE/perhaps dumb question > > > > I’ve been reviewing some of AWS’s connectivity products, but it seems like > none of them are designed to improve the AWS experience for all of an ISP’s > customers across AWS’s various regions. > > > > The background here is that we have a business customer using a key > application hosted in AWS’s East Coast region. The applications’ document > upload performance has been less than ideal, and their external consultant > is blaming their ISP — us. As you know, once traffic leaves our network and > hits upstream providers, our influence on the experience is very limited. > My initial thought was to explore whether we could bring AWS closer to all > of our customers through some kind of direct peering or other arrangement > with AWS. > > > > However, from what I’m seeing, AWS’s connectivity options are mostly > targeted at connecting a single enterprise to AWS, not enhancing general > public access to AWS services. > > > > Is that just the reality of it, or is there a broader product (other than > direct peering with Amazon) that would improve AWS performance for all our > customers? > > > > Frank > > *Frank Bulk* > > *Chief Technology Officer / Chief Operating Officer* > > [image: PremierComm_logo_CMYK.png] > > 339 1st Ave NE *|* Sioux Center, IA 51250 *|* Find us on Facebook > <https://www.facebook.com/PremierCommunicationsSiouxCenter> > > *phone:* 712.722.3451 *web:* mypremieronline.com > <https://www.mypremieronline.com/> > *direct: *(712) 722-7152 *cell:* (712) 441-6049 > > > _______________________________________________ > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net >

April 25, 2025

RE: Non-MICE/perhaps dumb question

by Jeremy Lumby

I do have customers that utilize my extension switches to other IXes so that they can connect to Amazon directly in other regions. Unfortunately I have a west coast option, and you are looking for an east coast option. From: Frank Bulk <fbulk(a)mypremieronline.com> Sent: Friday, April 25, 2025 3:53 PM To: mice-discuss(a)lists.micemn.net Subject: Non-MICE/perhaps dumb question I've been reviewing some of AWS's connectivity products, but it seems like none of them are designed to improve the AWS experience for all of an ISP's customers across AWS's various regions. The background here is that we have a business customer using a key application hosted in AWS's East Coast region. The applications' document upload performance has been less than ideal, and their external consultant is blaming their ISP - us. As you know, once traffic leaves our network and hits upstream providers, our influence on the experience is very limited. My initial thought was to explore whether we could bring AWS closer to all of our customers through some kind of direct peering or other arrangement with AWS. However, from what I'm seeing, AWS's connectivity options are mostly targeted at connecting a single enterprise to AWS, not enhancing general public access to AWS services. Is that just the reality of it, or is there a broader product (other than direct peering with Amazon) that would improve AWS performance for all our customers? Frank Frank Bulk Chief Technology Officer / Chief Operating Officer 339 1st Ave NE | Sioux Center, IA 51250 | <https://www.facebook.com/PremierCommunicationsSiouxCenter> Find us on Facebook phone: 712.722.3451 web: mypremieronline.com <https://www.mypremieronline.com/> direct: (712) 722-7152 cell: (712) 441-6049

April 25, 2025

Non-MICE/perhaps dumb question

by Frank Bulk

I've been reviewing some of AWS's connectivity products, but it seems like none of them are designed to improve the AWS experience for all of an ISP's customers across AWS's various regions. The background here is that we have a business customer using a key application hosted in AWS's East Coast region. The applications' document upload performance has been less than ideal, and their external consultant is blaming their ISP - us. As you know, once traffic leaves our network and hits upstream providers, our influence on the experience is very limited. My initial thought was to explore whether we could bring AWS closer to all of our customers through some kind of direct peering or other arrangement with AWS. However, from what I'm seeing, AWS's connectivity options are mostly targeted at connecting a single enterprise to AWS, not enhancing general public access to AWS services. Is that just the reality of it, or is there a broader product (other than direct peering with Amazon) that would improve AWS performance for all our customers? Frank [cid:a1ff5d57-4afc-4ec0-be61-51708bf94a4e.png] Frank Bulk Chief Technology Officer / Chief Operating Officer [cid:premiercomm_logo_cmyk_1388ffcb-8ec0-4842-9e40-a0e598482e7d.png] 339 1st Ave NE | Sioux Center, IA 51250 | Find us on Facebook<https://www.facebook.com/PremierCommunicationsSiouxCenter> phone: 712.722.3451 web: mypremieronline.com<https://www.mypremieronline.com/> direct: (712) 722-7152 cell: (712) 441-6049

April 25, 2025

Re: Qwilt question

by David Farmer

According to the MICE port graphing, Qwilt's traffic recovered somewhat in late February. However, since February, we haven't seen any local traffic for Qwilt. [image: image.png] Thanks On Fri, Feb 14, 2025 at 1:04 PM Richard Laager <rlaager(a)micemn.net> wrote: > On 2025-02-13 08:20, Steve Savoy wrote: > > Has anyone heard anymore information on Qwilt and when the peering might > be back up? > > > I think it is very unlikely they will be reconnecting soon. > > They haven't paid in a year and are being disconnected for non-payment. > > -- > Richard Laager > Chief Manager, Director > Midwest Internet Cooperative Exchange LLC > > _______________________________________________ > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net > -- =============================================== David Farmer Email:farmer@umn.edu Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 ===============================================

April 23, 2025

Please welcome Cato Networks to MICE!

by Jeremy Lumby

Name: Cato Networks Inc Switch: MICE Core 511-100G 7504 Port: Ethernet 4/21 (10G) ASN: 13150 IPv4: 206.108.255.9/24 IPv6: 2001:504:27::335e:0:1/64 Peering Contact: peering(a)catonetworks.com Route Servers: YES Welcome Aboard! Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 Main: 612-355-7740 Direct: 612-392-6814 Fax: 952-873-7425 jlumby(a)mnvoip.com

March 30, 2025

Re: MICE at 2T

by David Farmer

Have you been able to validate the graphs? Are we regularly gating to 2 Tb now? On Sun, Mar 23, 2025 at 10:39 PM Anthony Anderberg < AnthonyAnderberg(a)nuvera.net> wrote: > I think I might be double-counting the packets that transverse the 511 > building 7504-7804 link… I’ll look at that tomorrow and verify if I > excluded the members of that LAG properly. > > Cheers, > Anthony > > Sent from my iPhone > > On Mar 22, 2025, at 7:35 PM, David Farmer via mice-discuss < > mice-discuss(a)lists.micemn.net> wrote: > > > I think Mice broke 2T in the last few days. > > But I'm having a hard time getting to the cacti graphs to confirm. > > Thanks. > > =============================================== > David Farmer Email:farmer@umn.edu > Networking & Telecommunication Services > Office of Information Technology > University of Minnesota > 2218 University Ave SE Phone: 612-626-0815 > Minneapolis, MN 55414-3029 Cell: 612-812-9952 > =============================================== > _______________________________________________ > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net > > -- =============================================== David Farmer Email:farmer@umn.edu Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 ===============================================

March 30, 2025

RE: Amazon bouncing

by Steve Savoy

I thought it might have been fixed as it stayed steady all night but I see it just flapped again 2 hrs ago. Steve Savoy WCTA From: Jeremy Lumby <jlumby(a)mnvoip.com> Sent: Thursday, March 27, 2025 3:01 PM To: mice-discuss(a)lists.micemn.net Subject: RE: Amazon bouncing As some background information Amazon has been migrating from 2x4x100G to 2x2x400G. When they migrated their first link to 400G it worked fine until they linked their old 100G ports up again (the ones that they had appeared to have migrated away from), and then the loss started. I let Amazon know, but I did not react since they were still mid migration, and it only impacted Amazon. Last night they finished the migration, and I received an official completion notice. It was running fine until about an hour ago. Coincidentally at that exact time one of their old 4x100G LAGs came back online. Since I had received a completion notice, this time I went ahead and forcibly shutdown their old 100G ports, and things have seemed stable since then. Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington MN 55425 Main 612-355-7740 x211 Direct 612-392-6814 Fax 952-873-7425 jlumby(a)mnvoip.com<mailto:jlumby@mnvoip.com> From: Shawn Rasmussen via mice-discuss <mice-discuss(a)lists.micemn.net<mailto:mice-discuss@lists.micemn.net>> Sent: Thursday, March 27, 2025 2:04 PM To: mice-discuss(a)lists.micemn.net<mailto:mice-discuss@lists.micemn.net> Subject: Amazon bouncing Has anyone else's Amazon peering been bouncing a lot lately? They are bouncing right now for us and have sporadic periods over the last week. Are any of you seeing the same? Anyone know anything about it? Thanks, SHAWN RASMUSSEN Network Manager D: (218) 454-1106 [cid:image001.gif@01DB9FB5.410EFD80]<https://youtu.be/A_tgcHZD6oA>[cid:image002.gif@01DB9FB5.410EFD80]<https://twitter.com/connectctc>[cid:image003.gif@01DB9FB5.410EFD80]<https://www.facebook.com/connectctc>goctc.com<http://www.goctc.com/> [cid:image004.gif@01DB9FB5.410EFD80]<https://www.goctc.com/> Confidentiality Notice: This correspondence is the property of Winnebago Cooperative Telecom Association and is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.

March 28, 2025

RE: Amazon bouncing

by Jeremy Lumby

The MICE order form does encourage every member to signup for the mailing lists. MICE posts as many new member connections, as well as upgrades to the list as possible. Beyond that it would be up to the member to share their specific maintenance details on the list. From: nanog(a)jima.us <nanog(a)jima.us> Sent: Thursday, March 27, 2025 2:51 PM To: mice-discuss(a)lists.micemn.net Subject: RE: [EXTERNAL] Amazon bouncing Someone should come up with a way to preemptively give all potential peers at a given IX a heads-up as to maintenance events like that, so that all of the other participants didn’t have to individually troubleshoot the failure condition, work out amongst themselves what was going on, and possibly multiple of them reach out to inquire. I think something like that would really be popular. 👀 (Thanks for passing that along, Ryan!) - Jima From: Ryan Gunther via mice-discuss <mice-discuss(a)lists.micemn.net <mailto:mice-discuss@lists.micemn.net> > Sent: Thursday, March 27, 2025 14:21 To: Shawn Rasmussen <shawnr(a)goctc.com <mailto:shawnr@goctc.com> >; mice-discuss(a)lists.micemn.net <mailto:mice-discuss@lists.micemn.net> Subject: RE: [EXTERNAL] Amazon bouncing Hi Shawn – We have been seeing lots of the same over the past week or two. Bounces every minute for hours until we shut down our sessions. It took a few emails to Amazon to get a response – but I did receive the following back from them just the other day – Dear partner, Thank you for your inquiry. We're currently in the process of a significant infrastructure upgrade, which involves relocating this circuit to our new fabric. Our team has initiated the turndown process, which will be followed by a cable migration. While we don't have a specific ETA for the completion of this work, please be assured that our team is working diligently to restore capacity as quickly as possible. No action is required from your end at this time. We will keep you informed of any significant developments or if any action becomes necessary. We understand the inconvenience this may cause and sincerely appreciate your patience during this important upgrade process. If you have any questions or concerns, please don't hesitate to reach out. Regards, Ryan Gunther Network & Switching Engineer CCNA, MEF-CECP, A+ (406) 234-7458 105 N Montana Ave., Miles City, MT 59301 <https://www.midrivers.com/> www.midrivers.com <https://www.midrivers.com/> <https://facebook.com/midrivers> <https://www.instagram.com/midriverscommunications/> <https://www.linkedin.com/company/mid-rivers-communications/> <http://www.google.com/search?q=Mid-Rivers+Communications,1101%20Main%20St.,…> Review Us on Google From: Shawn Rasmussen via mice-discuss <mice-discuss(a)lists.micemn.net <mailto:mice-discuss@lists.micemn.net> > Sent: Thursday, March 27, 2025 1:04 PM To: mice-discuss(a)lists.micemn.net <mailto:mice-discuss@lists.micemn.net> Subject: [EXTERNAL] Amazon bouncing Notice: External Source: Confirm authenticity, stay secure. Has anyone else’s Amazon peering been bouncing a lot lately? They are bouncing right now for us and have sporadic periods over the last week. Are any of you seeing the same? Anyone know anything about it? Thanks, SHAWN RASMUSSEN Network Manager D: (218) 454-1106 <http://www.goctc.com/> goctc.com _____

March 27, 2025

RE: Amazon bouncing

by Jeremy Lumby

As some background information Amazon has been migrating from 2x4x100G to 2x2x400G. When they migrated their first link to 400G it worked fine until they linked their old 100G ports up again (the ones that they had appeared to have migrated away from), and then the loss started. I let Amazon know, but I did not react since they were still mid migration, and it only impacted Amazon. Last night they finished the migration, and I received an official completion notice. It was running fine until about an hour ago. Coincidentally at that exact time one of their old 4x100G LAGs came back online. Since I had received a completion notice, this time I went ahead and forcibly shutdown their old 100G ports, and things have seemed stable since then. Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington MN 55425 Main 612-355-7740 x211 Direct 612-392-6814 Fax 952-873-7425 jlumby(a)mnvoip.com From: Shawn Rasmussen via mice-discuss <mice-discuss(a)lists.micemn.net> Sent: Thursday, March 27, 2025 2:04 PM To: mice-discuss(a)lists.micemn.net Subject: Amazon bouncing Has anyone else's Amazon peering been bouncing a lot lately? They are bouncing right now for us and have sporadic periods over the last week. Are any of you seeing the same? Anyone know anything about it? Thanks, SHAWN RASMUSSEN Network Manager D: (218) 454-1106 <https://youtu.be/A_tgcHZD6oA> <https://twitter.com/connectctc> <https://www.facebook.com/connectctc> <http://www.goctc.com/> goctc.com- <https://www.goctc.com/> -

March 27, 2025

Re: [EXTERNAL] Amazon bouncing

by Andrew Hoyos

RFC9003 is a thing - https://www.rfc-editor.org/rfc/rfc9003.html Juniper example: https://www.juniper.net/documentation/us/en/software/junos/cli-reference/to… I know a lot of folks using this, and it is indeed very helpful to not waste resources chasing down BGP peers. Andrew Hoyos hoyosa(a)gmail.com > On Mar 27, 2025, at 14:51, nanog(a)jima.us wrote: > > Someone should come up with a way to preemptively give all potential peers at a given IX a heads-up as to maintenance events like that, so that all of the other participants didn’t have to individually troubleshoot the failure condition, work out amongst themselves what was going on, and possibly multiple of them reach out to inquire. > > I think something like that would really be popular. > > 👀 > > (Thanks for passing that along, Ryan!) > > - Jima > > From: Ryan Gunther via mice-discuss <mice-discuss(a)lists.micemn.net <mailto:mice-discuss@lists.micemn.net>> > Sent: Thursday, March 27, 2025 14:21 > To: Shawn Rasmussen <shawnr(a)goctc.com <mailto:shawnr@goctc.com>>; mice-discuss(a)lists.micemn.net <mailto:mice-discuss@lists.micemn.net> > Subject: RE: [EXTERNAL] Amazon bouncing > > Hi Shawn – > We have been seeing lots of the same over the past week or two. Bounces every minute for hours until we shut down our sessions. It took a few emails to Amazon to get a response – but I did receive the following back from them just the other day – > > > Dear partner, > > Thank you for your inquiry. We're currently in the process of a significant infrastructure upgrade, which involves relocating this circuit to our new fabric. Our team has initiated the turndown process, which will be followed by a cable migration. > > While we don't have a specific ETA for the completion of this work, please be assured that our team is working diligently to restore capacity as quickly as possible. > > No action is required from your end at this time. We will keep you informed of any significant developments or if any action becomes necessary. > > We understand the inconvenience this may cause and sincerely appreciate your patience during this important upgrade process. If you have any questions or concerns, please don't hesitate to reach out. > > Regards, > > > <image001.png> > <image002.png> > Ryan Gunther > Network & Switching Engineer > CCNA, MEF-CECP, A+ > (406) 234-7458 > 105 N Montana Ave., Miles City, MT 59301 > www.midrivers.com <https://www.midrivers.com/> > <image003.png> <https://www.midrivers.com/> <image004.png> <image005.png> <https://facebook.com/midrivers> <image006.png> <https://www.instagram.com/midriverscommunications/> <image007.png> <https://www.linkedin.com/company/mid-rivers-communications/> > Review Us on Google <http://www.google.com/search?q=Mid-Rivers+Communications,1101%20Main%20St.,…> > > > From: Shawn Rasmussen via mice-discuss <mice-discuss(a)lists.micemn.net <mailto:mice-discuss@lists.micemn.net>> > Sent: Thursday, March 27, 2025 1:04 PM > To: mice-discuss(a)lists.micemn.net <mailto:mice-discuss@lists.micemn.net> > Subject: [EXTERNAL] Amazon bouncing > > Notice: External Source: Confirm authenticity, stay secure. > Has anyone else’s Amazon peering been bouncing a lot lately? They are bouncing right now for us and have sporadic periods over the last week. Are any of you seeing the same? Anyone know anything about it? > > Thanks, > > SHAWN > RASMUSSEN > Network Manager > D: (218) 454-1106 > <image008.gif> <https://youtu.be/A_tgcHZD6oA><image009.gif> <https://twitter.com/connectctc><image010.gif> <https://www.facebook.com/connectctc>goctc.com <http://www.goctc.com/> > <image011.gif> <https://www.goctc.com/> > > > > > > > > _______________________________________________ > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net <mailto:mice-discuss@lists.micemn.net> > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net <mailto:mice-discuss-leave@lists.micemn.net>

March 27, 2025

RE: [EXTERNAL] Amazon bouncing

by nanog＠jima.us

Someone should come up with a way to preemptively give all potential peers at a given IX a heads-up as to maintenance events like that, so that all of the other participants didn’t have to individually troubleshoot the failure condition, work out amongst themselves what was going on, and possibly multiple of them reach out to inquire. I think something like that would really be popular. 👀 (Thanks for passing that along, Ryan!) - Jima From: Ryan Gunther via mice-discuss <mice-discuss(a)lists.micemn.net> Sent: Thursday, March 27, 2025 14:21 To: Shawn Rasmussen <shawnr(a)goctc.com>; mice-discuss(a)lists.micemn.net Subject: RE: [EXTERNAL] Amazon bouncing Hi Shawn – We have been seeing lots of the same over the past week or two. Bounces every minute for hours until we shut down our sessions. It took a few emails to Amazon to get a response – but I did receive the following back from them just the other day – Dear partner, Thank you for your inquiry. We're currently in the process of a significant infrastructure upgrade, which involves relocating this circuit to our new fabric. Our team has initiated the turndown process, which will be followed by a cable migration. While we don't have a specific ETA for the completion of this work, please be assured that our team is working diligently to restore capacity as quickly as possible. No action is required from your end at this time. We will keep you informed of any significant developments or if any action becomes necessary. We understand the inconvenience this may cause and sincerely appreciate your patience during this important upgrade process. If you have any questions or concerns, please don't hesitate to reach out. Regards, [signature] Ryan Gunther Network & Switching Engineer CCNA, MEF-CECP, A+ (406) 234-7458 105 N Montana Ave., Miles City, MT 59301 www.midrivers.com<https://www.midrivers.com/> [logo]<https://www.midrivers.com/> [facebook] <https://facebook.com/midrivers> [instagram] <https://www.instagram.com/midriverscommunications/> [linkedin] <https://www.linkedin.com/company/mid-rivers-communications/> Review Us on Google<http://www.google.com/search?q=Mid-Rivers+Communications,1101%20Main%20St.,…> From: Shawn Rasmussen via mice-discuss <mice-discuss(a)lists.micemn.net<mailto:mice-discuss@lists.micemn.net>> Sent: Thursday, March 27, 2025 1:04 PM To: mice-discuss(a)lists.micemn.net<mailto:mice-discuss@lists.micemn.net> Subject: [EXTERNAL] Amazon bouncing Notice: External Source: Confirm authenticity, stay secure. Has anyone else’s Amazon peering been bouncing a lot lately? They are bouncing right now for us and have sporadic periods over the last week. Are any of you seeing the same? Anyone know anything about it? Thanks, SHAWN RASMUSSEN Network Manager D: (218) 454-1106 [cid:image008.gif@01DB9F27.AEA7B910]<https://youtu.be/A_tgcHZD6oA>[cid:image009.gif@01DB9F27.AEA7B910]<https://twitter.com/connectctc>[cid:image010.gif@01DB9F27.AEA7B910]<https://www.facebook.com/connectctc>goctc.com<http://www.goctc.com/> [cid:image011.gif@01DB9F27.AEA7B910]<https://www.goctc.com/> ________________________________

March 27, 2025

RE: [EXTERNAL] Amazon bouncing

by Steve Savoy

We have witnessed this as well. .111 has been up for 22 minutes .112 has been up for 11 hours [A logo with blue and green textDescription automatically generated]<https://www.wctatel.net/> Steve Savoy Network Administrator Winnebago Cooperative Telecom Association 704 E Main St | Lake Mills, IA 50450 Phone: 641.592.6105 | Toll Free: 800.592.6105 | Fax: 641.592.6102 Email: stevesavoy(a)wctatel.com<mailto:stevesavoy@wctatel.com> [cid:image015.png@01DB9F25.F4053570]<https://www.facebook.com/WCTAtel> [cid:image017.png@01DB9F25.F4053570]<https://www.instagram.com/wctatel/> [cid:image019.png@01DB9F25.F4053570]<https://www.linkedin.com/company/wctatel/> [cid:image021.png@01DB9F25.F4053570]<https://www.youtube.com/@wcta4892> [cid:image022.jpg@01DB9F25.F4053570] From: Nathan Beard <nbeard(a)7sigma.com> Sent: Thursday, March 27, 2025 2:30 PM To: Ryan Gunther <Ryan.Gunther(a)midrivers.coop> Cc: Shawn Rasmussen <shawnr(a)goctc.com>; mice-discuss(a)lists.micemn.net Subject: Re: [EXTERNAL] Amazon bouncing You don't often get email from nbeard(a)7sigma.com<mailto:nbeard@7sigma.com>. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification> Stellar Association has also seen this same behavior since the MICE port moves last week, and I've reached out to Amazon a couple times and have yet to get a response. Things seemed to stabilize around 3:10am this morning so I was hopeful it was fixed, but it looks like the .111 bi-lateral is flapping again as of 1:42pm. It's worth noting that the second bi-lateral sessions appears to be stable today. If anybody else gets any detailed updates from Amazon and could post those to the list it would be much appreciated. Nathan Beard 7Sigma Systems, Inc. Senior Network Engineer (218) 820-7636 www.7sigma.com<http://www.7sigma.com/> On Thu, Mar 27, 2025 at 2:23 PM Ryan Gunther via mice-discuss <mice-discuss(a)lists.micemn.net<mailto:mice-discuss@lists.micemn.net>> wrote: Hi Shawn – We have been seeing lots of the same over the past week or two. Bounces every minute for hours until we shut down our sessions. It took a few emails to Amazon to get a response – but I did receive the following back from them just the other day – Dear partner, Thank you for your inquiry. We're currently in the process of a significant infrastructure upgrade, which involves relocating this circuit to our new fabric. Our team has initiated the turndown process, which will be followed by a cable migration. While we don't have a specific ETA for the completion of this work, please be assured that our team is working diligently to restore capacity as quickly as possible. No action is required from your end at this time. We will keep you informed of any significant developments or if any action becomes necessary. We understand the inconvenience this may cause and sincerely appreciate your patience during this important upgrade process. If you have any questions or concerns, please don't hesitate to reach out. Regards, [signature] Ryan Gunther Network & Switching Engineer CCNA, MEF-CECP, A+ (406) 234-7458 105 N Montana Ave., Miles City, MT 59301 www.midrivers.com<https://www.midrivers.com/> [logo]<https://www.midrivers.com/> [facebook] <https://facebook.com/midrivers> [instagram] <https://www.instagram.com/midriverscommunications/> [linkedin] <https://www.linkedin.com/company/mid-rivers-communications/> Review Us on Google<http://www.google.com/search?q=Mid-Rivers+Communications,1101%20Main%20St.,…> From: Shawn Rasmussen via mice-discuss <mice-discuss(a)lists.micemn.net<mailto:mice-discuss@lists.micemn.net>> Sent: Thursday, March 27, 2025 1:04 PM To: mice-discuss(a)lists.micemn.net<mailto:mice-discuss@lists.micemn.net> Subject: [EXTERNAL] Amazon bouncing Notice: External Source: Confirm authenticity, stay secure. Has anyone else’s Amazon peering been bouncing a lot lately? They are bouncing right now for us and have sporadic periods over the last week. Are any of you seeing the same? Anyone know anything about it? Thanks, SHAWN RASMUSSEN Network Manager D: (218) 454-1106 [cid:image008.gif@01DB9F25.E3948790]<https://youtu.be/A_tgcHZD6oA>[cid:image009.gif@01DB9F25.E3948790]<https://twitter.com/connectctc>[cid:image010.gif@01DB9F25.E3948790]<https://www.facebook.com/connectctc>goctc.com<http://www.goctc.com/> [cid:image011.gif@01DB9F25.E3948790]<https://www.goctc.com/> ________________________________ _______________________________________________ mice-discuss mailing list -- mice-discuss(a)lists.micemn.net<mailto:mice-discuss@lists.micemn.net> To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net<mailto:mice-discuss-leave@lists.micemn.net> Confidentiality Notice: This correspondence is the property of Winnebago Cooperative Telecom Association and is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.

March 27, 2025

Re: [EXTERNAL] Amazon bouncing

by Nathan Beard

Stellar Association has also seen this same behavior since the MICE port moves last week, and I've reached out to Amazon a couple times and have yet to get a response. Things seemed to stabilize around 3:10am this morning so I was hopeful it was fixed, but it looks like the .111 bi-lateral is flapping again as of 1:42pm. It's worth noting that the second bi-lateral sessions appears to be stable today. If anybody else gets any detailed updates from Amazon and could post those to the list it would be much appreciated. *Nathan Beard* *7Sigma Systems, Inc.* Senior Network Engineer (218) 820-7636 www.7sigma.com On Thu, Mar 27, 2025 at 2:23 PM Ryan Gunther via mice-discuss < mice-discuss(a)lists.micemn.net> wrote: > Hi Shawn – > > We have been seeing lots of the same over the past week or > two. Bounces every minute for hours until we shut down our sessions. It > took a few emails to Amazon to get a response – but I did receive the > following back from them just the other day – > > > > > > Dear partner, > > > > Thank you for your inquiry. We're currently in the process of a > significant infrastructure upgrade, which involves relocating this circuit > to our new fabric. Our team has initiated the turndown process, which will > be followed by a cable migration. > > > > While we don't have a specific ETA for the completion of this work, please > be assured that our team is working diligently to restore capacity as > quickly as possible. > > > > No action is required from your end at this time. We will keep you > informed of any significant developments or if any action becomes necessary. > > > > We understand the inconvenience this may cause and sincerely appreciate > your patience during this important upgrade process. If you have any > questions or concerns, please don't hesitate to reach out. > > > > Regards, > > > > > > [image: signature] > > [image: divider] > > Ryan Gunther > > Network & Switching Engineer > > CCNA, MEF-CECP, A+ > > (406) 234-7458 > > 105 N Montana Ave., Miles City, MT 59301 > > www.midrivers.com > > [image: logo] <https://www.midrivers.com/> [image: divider] [image: > facebook] <https://facebook.com/midrivers> [image: instagram] > <https://www.instagram.com/midriverscommunications/> [image: linkedin] > <https://www.linkedin.com/company/mid-rivers-communications/> > > Review Us on Google > <http://www.google.com/search?q=Mid-Rivers+Communications,1101%20Main%20St.,…> > > > > > > *From:* Shawn Rasmussen via mice-discuss <mice-discuss(a)lists.micemn.net> > *Sent:* Thursday, March 27, 2025 1:04 PM > *To:* mice-discuss(a)lists.micemn.net > *Subject:* [EXTERNAL] Amazon bouncing > > > > *Notice:* External Source: Confirm authenticity, stay secure. > > Has anyone else’s Amazon peering been bouncing a lot lately? They are > bouncing right now for us and have sporadic periods over the last week. Are > any of you seeing the same? Anyone know anything about it? > > > > Thanks, > > > > *SHAWN* > > *RASMUSSEN* > > Network Manager > > D: (218) 454-1106 > > <https://youtu.be/A_tgcHZD6oA> <https://twitter.com/connectctc> > <https://www.facebook.com/connectctc>*goctc.com* <http://www.goctc.com/> > ** > > <https://www.goctc.com/> > > > > > > > > > > > > ------------------------------ > > > _______________________________________________ > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net >

March 27, 2025

RE: Amazon bouncing

by Randy Lehs - CarverLink

Yes, we have seen the same, with the bouncing starting late last week. Randy Lehs | CarverLink Fiber Manager CarverLink - A Department of Carver County Public Services 600 East Fourth Street Chaska, MN 55318-2102 www.carverlink.com<http://www.carverlink.com/> | rlehs(a)carverlink.com<mailto:rlehs@carverlink.com> Office: 952-361-1494 | Fax: 952-361-1582 | Cell: 612-719-1656 | Standard and Emergency Support 24/7/365: 952-361-1200 From: Shawn Rasmussen via mice-discuss <mice-discuss(a)lists.micemn.net> Sent: Thursday, March 27, 2025 2:04 PM To: mice-discuss(a)lists.micemn.net Subject: Amazon bouncing Has anyone else's Amazon peering been bouncing a lot lately? They are bouncing right now for us and have sporadic periods over the last week. Are any of you seeing the same? Anyone know anything about it? Thanks, SHAWN RASMUSSEN Network Manager D: (218) 454-1106 [cid:image005.gif@01DB9F24.442E84E0]<https://youtu.be/A_tgcHZD6oA>[cid:image006.gif@01DB9F24.442E84E0]<https://twitter.com/connectctc>[cid:image007.gif@01DB9F24.442E84E0]<https://www.facebook.com/connectctc>goctc.com<http://www.goctc.com/> [cid:image008.gif@01DB9F24.442E84E0]<https://www.goctc.com/>

March 27, 2025

RE: [EXTERNAL] Amazon bouncing

by Ryan Gunther

Hi Shawn - We have been seeing lots of the same over the past week or two. Bounces every minute for hours until we shut down our sessions. It took a few emails to Amazon to get a response - but I did receive the following back from them just the other day - Dear partner, Thank you for your inquiry. We're currently in the process of a significant infrastructure upgrade, which involves relocating this circuit to our new fabric. Our team has initiated the turndown process, which will be followed by a cable migration. While we don't have a specific ETA for the completion of this work, please be assured that our team is working diligently to restore capacity as quickly as possible. No action is required from your end at this time. We will keep you informed of any significant developments or if any action becomes necessary. We understand the inconvenience this may cause and sincerely appreciate your patience during this important upgrade process. If you have any questions or concerns, please don't hesitate to reach out. Regards, [signature] Ryan Gunther Network & Switching Engineer CCNA, MEF-CECP, A+ (406) 234-7458 105 N Montana Ave., Miles City, MT 59301 www.midrivers.com<https://www.midrivers.com/> [logo]<https://www.midrivers.com/> [facebook] <https://facebook.com/midrivers> [instagram] <https://www.instagram.com/midriverscommunications/> [linkedin] <https://www.linkedin.com/company/mid-rivers-communications/> Review Us on Google<http://www.google.com/search?q=Mid-Rivers+Communications,1101%20Main%20St.,…> From: Shawn Rasmussen via mice-discuss <mice-discuss(a)lists.micemn.net> Sent: Thursday, March 27, 2025 1:04 PM To: mice-discuss(a)lists.micemn.net Subject: [EXTERNAL] Amazon bouncing Notice: External Source: Confirm authenticity, stay secure. Has anyone else's Amazon peering been bouncing a lot lately? They are bouncing right now for us and have sporadic periods over the last week. Are any of you seeing the same? Anyone know anything about it? Thanks, SHAWN RASMUSSEN Network Manager D: (218) 454-1106 [cid:image012.gif@01DB9F1A.F436CDC0]<https://youtu.be/A_tgcHZD6oA>[cid:image013.gif@01DB9F1A.F436CDC0]<https://twitter.com/connectctc>[cid:image014.gif@01DB9F1A.F436CDC0]<https://www.facebook.com/connectctc>goctc.com<http://www.goctc.com/> [cid:image015.gif@01DB9F1A.F436CDC0]<https://www.goctc.com/> ________________________________ This email has been scanned for spam and viruses by Proofpoint Essentials. Click here<https://us3.proofpointessentials.com/app/report_spam.php?mod_id=11&mod_opti…> to report this email as spam.

March 27, 2025