- mice-discuss - lists.micemn.net

RE: Anyone else seeing instability to MICE route reflector #2

by Frank Bulk

We’re doing 10/30 against the two route servers. Frank [cid:premiercomm_logo_cmyk_thick_4b24da0e-a829-42f3-ba49-f303951df775.png] Frank Bulk Chief Technology Officer / Chief Operating Officer From: Richard Laager <rlaager(a)micemn.net> Sent: Wednesday, October 15, 2025 4:33 PM To: mice-discuss(a)lists.micemn.net Subject: Re: Anyone else seeing instability to MICE route reflector #2 What do you have for hold timers? And is it the same for both route servers? rs2 is proposing 240 seconds, if I'm reading this packet capture correctly. -- Richard Laager Chief Manager, Director Midwest Internet Cooperative Exchange LLC

Oct. 24, 2025

Re: v6 on rs1/rs2 fubar?

by Richard Laager

I've built 2.17.1 for 22.04. I'm thinking we'll upgrade BIRD on rs2 tomorrow noon-ish, and apply regular updates while we are at it. It might help with the timeouts we are seeing there. Regardless, as long as it's not worse, then next week (tentatively Monday), we can upgrade BIRD on rs1. -- Richard

Oct. 24, 2025

RE: v6 on rs1/rs2 fubar?

by Michael Hare

JTAC pointed me to TSB76599. It’s behind a login, so while I won’t copy/paste the entire PDF, in summary: “The following Junos and Junos Evolved software releases are now fully compliant with RFC7606” 22.4R3, 22.4R3-EVO, and subsequent releases 23.2R2, 23.2R2-EVO, and subsequent releases 23.4R1, 23.4R1-EVO, and subsequent releases 24.1R1, 24.1R1-EVO, and all subsequent releases -Michael From: David Farmer via mice-discuss <mice-discuss(a)lists.micemn.net> Sent: Thursday, October 23, 2025 6:31 PM To: Justin Krejci <JKrejci(a)usinternet.com> Cc: mice-discuss(a)lists.micemn.net; rlaager(a)wiktel.com Subject: Re: v6 on rs1/rs2 fubar? According to the NA-IX Slack BIRD 2.17.1. fixes the issue, or at least doesn't propagate it. Thanks On Thu, Oct 23, 2025 at 6:23 PM Justin Krejci <JKrejci(a)usinternet.com<mailto:JKrejci@usinternet.com>> wrote: As a point of comparison Ubuntu 24.04 LTS includes bird 2.14 in the default system packages, perhaps that will be sufficiently bug-fixed and is already very accessible to achieve with a simple upgrade to 24.04. FWIW, I have done the upgrade from 22.04 to 24.04 many times, all without issue <include typical ymmv disclaimer here> -----Original Message----- From: Richard Laager via mice-discuss <mice-discuss(a)lists.micemn.net<mailto:Richard%20Laager%20via%20mice-discuss%20%3cmice-discuss@lists.micemn.net%3e>> Reply-To: Richard Laager <rlaager(a)wiktel.com<mailto:Richard%20Laager%20%3crlaager@wiktel.com%3e>> To: mice-discuss(a)lists.micemn.net<mailto:mice-discuss@lists.micemn.net> Subject: Re: v6 on rs1/rs2 fubar? Date: Thu, 23 Oct 2025 17:50:22 -0500 IXP Manager does not yet support BIRD 3. They don't feel it is production ready. They suggested that maybe our issues would be fixed on newer BIRD 2. MICE is running Ubuntu 22.04, which has BIRD 2.0.8. In a few months, we can upgrade to 26.04 when it comes out, which will have at least BIRD 2.17.1 (which is the current release right now). I could attempt to rebuild the BIRD 2.17.1 package on Ubuntu 22.04. Then we could see if that helps. _______________________________________________ mice-discuss mailing list -- mice-discuss(a)lists.micemn.net<mailto:mice-discuss@lists.micemn.net> To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net<mailto:mice-discuss-leave@lists.micemn.net> -- =============================================== David Farmer Email:farmer@umn.edu<mailto:Email%3Afarmer@umn.edu> Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 ===============================================

Oct. 24, 2025

Re: v6 on rs1/rs2 fubar?

by David Farmer

According to the NA-IX Slack BIRD 2.17.1. fixes the issue, or at least doesn't propagate it. Thanks On Thu, Oct 23, 2025 at 6:23 PM Justin Krejci <JKrejci(a)usinternet.com> wrote: > As a point of comparison Ubuntu 24.04 LTS includes bird 2.14 in the > default system packages, perhaps that will be sufficiently bug-fixed and is > already very accessible to achieve with a simple upgrade to 24.04. > > FWIW, I have done the upgrade from 22.04 to 24.04 many times, all without > issue <include typical ymmv disclaimer here> > > > > -----Original Message----- > *From*: Richard Laager via mice-discuss <mice-discuss(a)lists.micemn.net > <Richard%20Laager%20via%20mice-discuss%20%3cmice-discuss(a)lists.micemn.net%3e> > > > *Reply-To*: Richard Laager <rlaager(a)wiktel.com > <Richard%20Laager%20%3crlaager(a)wiktel.com%3e>> > *To*: mice-discuss(a)lists.micemn.net > *Subject*: Re: v6 on rs1/rs2 fubar? > *Date*: Thu, 23 Oct 2025 17:50:22 -0500 > > IXP Manager does not yet support BIRD 3. They don't feel it is > production ready. > > They suggested that maybe our issues would be fixed on newer BIRD 2. > > MICE is running Ubuntu 22.04, which has BIRD 2.0.8. In a few months, we > can upgrade to 26.04 when it comes out, which will have at least BIRD > 2.17.1 (which is the current release right now). > > I could attempt to rebuild the BIRD 2.17.1 package on Ubuntu 22.04. Then > we could see if that helps. > > _______________________________________________ > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net > -- =============================================== David Farmer Email:farmer@umn.edu Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 ===============================================

Oct. 23, 2025

Re: v6 on rs1/rs2 fubar?

by Justin Krejci

As a point of comparison Ubuntu 24.04 LTS includes bird 2.14 in the default system packages, perhaps that will be sufficiently bug-fixed and is already very accessible to achieve with a simple upgrade to 24.04. FWIW, I have done the upgrade from 22.04 to 24.04 many times, all without issue <include typical ymmv disclaimer here> -----Original Message----- From: Richard Laager via mice-discuss <mice-discuss(a)lists.micemn.net<mailto:Richard%20Laager%20via%20mice-discuss%20%3cmice-discuss@lists.micemn.net%3e>> Reply-To: Richard Laager <rlaager(a)wiktel.com<mailto:Richard%20Laager%20%3crlaager@wiktel.com%3e>> To: mice-discuss(a)lists.micemn.net<mailto:mice-discuss@lists.micemn.net> Subject: Re: v6 on rs1/rs2 fubar? Date: Thu, 23 Oct 2025 17:50:22 -0500 IXP Manager does not yet support BIRD 3. They don't feel it is production ready. They suggested that maybe our issues would be fixed on newer BIRD 2. MICE is running Ubuntu 22.04, which has BIRD 2.0.8. In a few months, we can upgrade to 26.04 when it comes out, which will have at least BIRD 2.17.1 (which is the current release right now). I could attempt to rebuild the BIRD 2.17.1 package on Ubuntu 22.04. Then we could see if that helps.

Oct. 23, 2025

Re: v6 on rs1/rs2 fubar?

by Richard Laager

IXP Manager does not yet support BIRD 3. They don't feel it is production ready. They suggested that maybe our issues would be fixed on newer BIRD 2. MICE is running Ubuntu 22.04, which has BIRD 2.0.8. In a few months, we can upgrade to 26.04 when it comes out, which will have at least BIRD 2.17.1 (which is the current release right now). I could attempt to rebuild the BIRD 2.17.1 package on Ubuntu 22.04. Then we could see if that helps. -- Richard

Oct. 23, 2025

RE: v6 on rs1/rs2 fubar?

by Michael Hare

It's probably not a secret that this recurred last evening between roughly 20:34 and 23:02 GMT -5/Central. With the offending prefix [per my logs] being 2a0c:b641:309::/48. Looks like I'll finally be contacting my var today [Juniper] to see specifically which versions will address this issue, but I'd also like to ask, based on this discussion from the first go around [I'm not in the correct circles to see any recent discussion] https://social.bgp.tools/@bgptools/statuses/01K7072TB9HS4JYN8E5Z01GEPK "The main cause of disruption peoples IX route server sessions going down (see image of MegaIXs looking glass), if IX's were running a modern version of bird/OpenBGPd this would have been filtered out!" Is there something "more" that could be done with bird on the IX to prevent future unknown unknowns from causing similar issues? -Michael > -----Original Message----- > From: Michael Hare via mice-discuss <mice-discuss(a)lists.micemn.net> > Sent: Tuesday, October 7, 2025 4:28 PM > To: Chris Wopat <wopat(a)wiscnet.net>; mice-discuss(a)lists.micemn.net > Subject: RE: v6 on rs1/rs2 fubar? > > I am late to the game. > > Me too, starting 14:49 Central/GMT-5 for us. Yes I'm a Juniper. > > = Yes, we use BFD. > = Yes, we have bgp-error-tolerance enabled. > "set protocols bgp bgp-error-tolerance" > [yes, my session is in global] > = No, I haven't contacted Juniper, but I seem to have a misunderstanding about > "bgp-error-tolerance" at the very least. > > Oct 7 14:49:10 r-minneapolis-511.uwsys.net [LEVEL='daemon.warning'] > rpd[32044]: %DAEMON-4-RPD_BGP_NEIGHBOR_STATE_CHANGED: BGP peer > 2001:504:27::d1af:0:1 (External AS 53679) changed st > ate from Established to Idle (event RecvUpdate) (instance master) > Oct 7 14:49:10 r-minneapolis-511.uwsys.net [LEVEL='daemon.err'] rpd[32044]: > %DAEMON-3: BGP ERROR: Insufficient data for the packet > Oct 7 14:49:10 r-minneapolis-511.uwsys.net [LEVEL='daemon.warning'] > rpd[32044]: %DAEMON-4: bgp_read_v4_update:13800: NOTIFICATION sent to > 2001:504:27::d1af:0:1 (External AS 53679): co > de 3 (Update Message Error) subcode 1 (invalid attribute list) > Oct 7 14:49:10 r-minneapolis-511.uwsys.net [LEVEL='daemon.err'] rpd[32044]: > %DAEMON-3: Received malformed update from 2001:504:27::d1af:0:1 (External > AS 53679) > Oct 7 14:49:10 r-minneapolis-511.uwsys.net [LEVEL='daemon.err'] rpd[32044]: > %DAEMON-3: Family inet6-unicast, prefix 2402:2de0::/32 > Oct 7 14:49:10 r-minneapolis-511.uwsys.net [LEVEL='daemon.err'] rpd[32044]: > %DAEMON-3: Malformed Attribute Unknown(35) flag 0xe0 length 1024. > Oct 7 14:49:10 r-minneapolis-511.uwsys.net [LEVEL='daemon.err'] rpd[32044]: > %DAEMON-3: BGP ERROR: Insufficient data for the packet > > -Michael > > > -----Original Message----- > > From: Chris Wopat <wopat(a)wiscnet.net> > > Sent: Tuesday, October 7, 2025 4:10 PM > > To: mice-discuss(a)lists.micemn.net > > Subject: Re: v6 on rs1/rs2 fubar? > > > > A few years ago this came out: > > > > https://urldefense.com/v3/__https://blog.benjojo.co.uk/post/bgp-path- > > attributes-grave-error- > > > handling__;!!Mak6IKo!MP3CJWT3VXc9yyZOF25Zkl6ylazMccDKnoNw_w6IpTdMY > > ZVmAbEhYBGcObRfkZtQ-jEoiojfbC9uW1cwqw$ > > > > And the Juniper specific link is to enable bgp-error-tolerance: > > > > https://urldefense.com/v3/__https://supportportal.juniper.net/s/article/202… > > 08-29-Out-of-Cycle-Security-Bulletin-Junos-OS-and-Junos-OS-Evolved-A- > crafted- > > BGP-UPDATE-message-allows-a-remote-attacker-to-de-peer-reset-BGP- > > sessions-CVE-2023- > > > 4481__;!!Mak6IKo!MP3CJWT3VXc9yyZOF25Zkl6ylazMccDKnoNw_w6IpTdMYZV > > mAbEhYBGcObRfkZtQ-jEoiojfbC8RD26zjg$ > > > > > > > https://urldefense.com/v3/__https://www.juniper.net/documentation/us/en/soft > > ware/junos/bgp/topics/topic-map/bgp-error- > > > messages.html__;!!Mak6IKo!MP3CJWT3VXc9yyZOF25Zkl6ylazMccDKnoNw_w6I > > pTdMYZVmAbEhYBGcObRfkZtQ-jEoiojfbC9K24DUbQ$ > > > > We (as2381) are not having resets and do have this enabled, as a point > > of comparison. > > > > Cheers, > > -- > > Chris Wopat > > Network Engineer, WiscNet > > wopat(a)wiscnet.net 608-210-3965 > > _______________________________________________ > > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net > > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net > _______________________________________________ > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net

Oct. 23, 2025

Re: Anyone else seeing instability to MICE route reflector #2

by Richard Laager

BIRD 2.x is single-threaded. The config file is non-trivial. For IPv4, it is 359,018 lines and 38,367,665 bytes (37 MiB). This is due to the massive configuration generated by the IRR stuff. (This is especially true for HE, which is the majority of the config. HE's part of the config is 2,335 lines of ASNs and 217,914 lines of prefixes. Note that there is than one per line.) Since this problem occurs when BIRD is being reconfigured, my working theory is that BIRD is just getting overwhelmed for a bit. From the packet capture I have of CTC, it's definitely the CTC side tearing down the session due to the hold timer expiring. This will likely get better in BIRD 3.x. I think 3.x is multithreaded. On 2025-10-17 16:33, Jonathan Stewart wrote: > What is the problem with shorter hold timers? Shorter hold timers > allow BGP failures to be detected more quickly, reducing time to find > an alternate, functional route. > > 60 second hold timers with 20 second keepalives are fine in the real > world in my experience. -- Richard Laager Chief Manager, Director Midwest Internet Cooperative Exchange LLC

Oct. 17, 2025

Re: Anyone else seeing instability to MICE route reflector #2

by Jonathan Stewart

What is the problem with shorter hold timers? Shorter hold timers allow BGP failures to be detected more quickly, reducing time to find an alternate, functional route. 60 second hold timers with 20 second keepalives are fine in the real world in my experience. Jonathan Jonathan Stewart Network Engineer LES.NET - AS18451 Desk: 1-204-666-6191 Mobile: 1-204-990-2120 130 Portage Avenue E Winnipeg, MB R3C 0A1 CANADA On Fri, 2025-10-17 4:21 p.m., Richard Laager wrote: > On 2025-10-15 16:32, Richard Laager wrote: >> What do you have for hold timers? And is it the same for both route >> servers? >> >> rs2 is proposing 240 seconds, if I'm reading this packet capture >> correctly. > > I can set a "min hold time" value in BIRD. If we find that raising the > hold timer fixes the issues, I will probably set a minimum. If I set > that, it will break the sessions with lower values immediately, so > this would need some transition. That would probably look something > like: 1) See who it would affect and email them. 2) Enable it on one > route server. 3) After a bit, enable it on the other route server. > -- > Richard > > _______________________________________________ > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net

Oct. 17, 2025

Re: Anyone else seeing instability to MICE route reflector #2

by Richard Laager

On 2025-10-15 16:32, Richard Laager wrote: > What do you have for hold timers? And is it the same for both route > servers? > > rs2 is proposing 240 seconds, if I'm reading this packet capture > correctly. I can set a "min hold time" value in BIRD. If we find that raising the hold timer fixes the issues, I will probably set a minimum. If I set that, it will break the sessions with lower values immediately, so this would need some transition. That would probably look something like: 1) See who it would affect and email them. 2) Enable it on one route server. 3) After a bit, enable it on the other route server. -- Richard

Oct. 17, 2025

Re: Anyone else seeing instability to MICE route reflector #2

by Richard Laager

What do you have for hold timers? And is it the same for both route servers? rs2 is proposing 240 seconds, if I'm reading this packet capture correctly. -- Richard Laager Chief Manager, Director Midwest Internet Cooperative Exchange LLC

Oct. 15, 2025

RE: Anyone else seeing instability to MICE route reflector #2

by Frank Bulk

Here's the last three days ... and good catch, always around 40 minutes past the hour! Oct 11 01:40:04 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 11 01:40:04 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 11 01:40:13 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 11 02:38:57 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 11 02:38:57 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 11 02:39:20 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 11 03:40:29 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 11 03:40:29 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 11 03:40:35 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 11 04:39:38 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 11 04:39:38 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 11 04:39:46 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 11 05:42:26 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 11 05:42:26 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 11 05:42:42 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 11 10:39:33 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 11 10:39:33 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 11 10:39:43 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 11 11:40:23 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 11 11:40:23 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 11 11:40:33 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 11 13:41:53 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 11 13:41:53 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 11 13:41:59 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 11 14:39:12 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 11 14:39:12 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 11 14:39:23 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 11 16:39:28 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 11 16:39:28 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 11 16:39:36 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 11 17:40:01 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 11 17:40:01 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 11 17:40:16 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 11 19:41:57 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 11 19:41:57 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 11 19:43:28 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 11 21:39:15 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 11 21:39:15 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 11 21:39:21 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 11 23:40:59 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 11 23:40:59 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 11 23:41:07 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 12 01:39:50 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 12 01:39:50 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 12 01:40:12 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 12 08:39:51 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 12 08:39:51 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 12 08:41:22 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 12 09:39:44 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 12 09:39:44 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 12 09:39:53 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 12 10:40:35 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 12 10:40:35 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 12 10:41:03 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 12 11:41:45 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 12 11:41:45 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 12 11:43:12 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 13 05:41:18 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 13 05:41:18 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 13 05:41:26 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 13 06:41:05 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 13 06:41:05 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 13 06:41:15 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 13 11:39:38 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 13 11:39:38 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 13 11:39:50 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Frank Frank Bulk -----Original Message----- From: Shawn Rasmussen via mice-discuss <mice-discuss(a)lists.micemn.net> Sent: Monday, October 13, 2025 8:22 AM To: mice-discuss(a)lists.micemn.net Subject: RE: Anyone else seeing instability to MICE route reflector #2 Yes, we have been seeing drops to RS2 for months now. We can't really see a reason why. We have noticed that they always seem to happen near the 40th minute of the hour. I didn't fully break out my secret decoder ring on your output below, but your results seem like they might be similar. RS1 has not been affected for us. Shawn Rasmussen Network Manager CTC -----Original Message----- From: Frank Bulk <fbulk(a)mypremieronline.com> Sent: Friday, October 10, 2025 2:42 PM To: Subject: Anyone else seeing instability to MICE route reflector #2 CAUTION: This email originated from outside CTC. Do not open links or attachments unless you are expecting this message and know they are safe. Anyone else seeing instability to MICE route reflector #2? Oct 9 18:41:17 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 9 18:41:17 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 9 18:41:44 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 9 19:41:56 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 9 19:41:56 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 9 19:42:15 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Establ ished Oct 10 01:41:12 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 01:41:12 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 01:41:31 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 03:40:52 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 03:40:52 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 03:41:06 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 04:41:58 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 04:41:58 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 04:42:11 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 05:38:55 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 05:38:55 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 05:39:10 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established ne w state Established Oct 10 07:42:15 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 07:42:15 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 07:42:54 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 08:40:07 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 08:40:07 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 08:40:37 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Establi shed new state Established Oct 10 09:40:13 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 09:40:13 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 09:40:25 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 10:10:10 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 10:10:10 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 10:10:28 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 10:41:13 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 10:41:13 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 10:41:29 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 13:39:28 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 13:39:28 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 13:39:37 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 14:22:07 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 14:22:07 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 14:22:19 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Frank Frank Bulk Chief Technology Officer / Chief Operating Officer 339 1st Ave NE? |? Sioux Center, IA 51250?? phone:?712.722.3451?web:?mypremieronline.com direct:?(712) 722-7152?cell:?(712) 441-6049 _______________________________________________ mice-discuss mailing list -- mice-discuss(a)lists.micemn.net To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net _______________________________________________ mice-discuss mailing list -- mice-discuss(a)lists.micemn.net To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net

Oct. 13, 2025

RE: Anyone else seeing instability to MICE route reflector #2

by Shawn Rasmussen

Yes, we have been seeing drops to RS2 for months now. We can't really see a reason why. We have noticed that they always seem to happen near the 40th minute of the hour. I didn't fully break out my secret decoder ring on your output below, but your results seem like they might be similar. RS1 has not been affected for us. Shawn Rasmussen Network Manager CTC -----Original Message----- From: Frank Bulk <fbulk(a)mypremieronline.com> Sent: Friday, October 10, 2025 2:42 PM To: Subject: Anyone else seeing instability to MICE route reflector #2 CAUTION: This email originated from outside CTC. Do not open links or attachments unless you are expecting this message and know they are safe. Anyone else seeing instability to MICE route reflector #2? Oct 9 18:41:17 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 9 18:41:17 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 9 18:41:44 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 9 19:41:56 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 9 19:41:56 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 9 19:42:15 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 01:41:12 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 01:41:12 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 01:41:31 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 03:40:52 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 03:40:52 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 03:41:06 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 04:41:58 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 04:41:58 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 04:42:11 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 05:38:55 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 05:38:55 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 05:39:10 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 07:42:15 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 07:42:15 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 07:42:54 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 08:40:07 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 08:40:07 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 08:40:37 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 09:40:13 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 09:40:13 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 09:40:25 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 10:10:10 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 10:10:10 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 10:10:28 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 10:41:13 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 10:41:13 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 10:41:29 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 13:39:28 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 13:39:28 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 13:39:37 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 14:22:07 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 14:22:07 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 14:22:19 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Frank Frank Bulk Chief Technology Officer / Chief Operating Officer 339 1st Ave NE? |? Sioux Center, IA 51250?? phone:?712.722.3451?web:?mypremieronline.com direct:?(712) 722-7152?cell:?(712) 441-6049 _______________________________________________ mice-discuss mailing list -- mice-discuss(a)lists.micemn.net To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net

Oct. 13, 2025

Anyone else seeing instability to MICE route reflector #2

by Frank Bulk

Anyone else seeing instability to MICE route reflector #2? Oct 9 18:41:17 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 9 18:41:17 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 9 18:41:44 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 9 19:41:56 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 9 19:41:56 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 9 19:42:15 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 01:41:12 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 01:41:12 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 01:41:31 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 03:40:52 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 03:40:52 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 03:41:06 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 04:41:58 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 04:41:58 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 04:42:11 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 05:38:55 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 05:38:55 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 05:39:10 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 07:42:15 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 07:42:15 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 07:42:54 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 08:40:07 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 08:40:07 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 08:40:37 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 09:40:13 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 09:40:13 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 09:40:25 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 10:10:10 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 10:10:10 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 10:10:28 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 10:41:13 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 10:41:13 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 10:41:29 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 13:39:28 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 13:39:28 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 13:39:37 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Oct 10 14:22:07 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state Established event Stop new state Idle Oct 10 14:22:07 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.2 (VRF default AS 53679) 4/0 (Hold Timer Expired Error/None) 0 bytes Oct 10 14:22:19 CDT SiouxCity-Fibernet-Arista Bgp: %BGP-5-ADJCHANGE: peer 206.108.255.2 (VRF default AS 53679) old state OpenConfirm event Established new state Established Frank Frank Bulk Chief Technology Officer / Chief Operating Officer 339 1st Ave NE? |? Sioux Center, IA 51250?? phone:?712.722.3451?web:?mypremieronline.com direct:?(712) 722-7152?cell:?(712) 441-6049

Oct. 10, 2025

RE: [EXTERNAL] - UG 48 Meeting & Election

by Miles McCredie

FWIW, the link to the Zoom meeting seems to be invalid but the following information seems valid: ID: 98593962100, passcode: 9txWv0 [http://images.midcocomm.com/ES_MidcoLogo.png] Miles McCredie Principal Network Engineer II Core IP Office: 6052755192 Miles.McCredie(a)midco.com Midco.com Let's go beyond. From: Richard Laager <rlaager(a)micemn.net> Sent: Saturday, August 30, 2025 12:12 AM To: mice-announce(a)lists.micemn.net; MICE Discuss <mice-discuss(a)lists.micemn.net> Subject: [EXTERNAL] - UG 48 Meeting & Election CAUTION: This email originated from outside of MIDCO. Do not click links or open attachments unless you recognize the sender and know the content is safe. The MICE UG 48 meeting will be held Wednesday, October 8, 3:00 pm at UMN and via Zoom. Nominations We will be holding an election for a board seat. The board consists of three seats with staggered terms, such that one seat comes up for re-election each year (at the fall meeting). This year, I (Richard Laager) am up for election. As in the past, we will close nominations in advance of the meeting so people have a chance to instruct a proxy appropriately. Nominations will be open for two weeks, September 22 - October 5. I plan to send reminder emails approximately at the start of and just after the nomination period. In the event that the election is uncontested, our practice is to elect by acclimation at the meeting. Proxies are unnecessary in that situation, assuming that the meeting makes quorum. Proxies The bylaws state that you send proxies to the Chief Manager (me, Richard Laager), and electronic means are acceptable. So a simple email is sufficient: rlaager(a)micemn.net<mailto:rlaager@micemn.net> or chiefmanager(a)micemn.net<mailto:chiefmanager@micemn.net>. Since it is my seat up for election this time, if you wish, you may alternatively send proxies to Anthony Anderberg: ant(a)micemn.net<mailto:ant@micemn.net>. If you send a vote by email (e.g. "We vote for <X>."), we will treat it as an a proxy appointment, appointing me (or Anthony, if sent to him) to vote on your behalf, but restricted to only the election item and to only vote for the candidate you expressed. If you send a proxy with different instructions (e.g. "We appoint <Y> to vote for us."), obviously that will be honored as written. UMN Details (from David Farmer) [The meeting] will be held at the University's Information Technology Building, 2218 University Ave SE, Minneapolis, MN 55414, Room 185. It is across University Ave SE from Huntington Bank Stadium (Formerly TCF Bank Stadium). See the link to the Campus Map below. This room has a video room system, and remote access will be made available via Zoom video or phone audio, see Zoom details below. Please enter the building through the middle of three doors on the Ontario St SE side of the building. It will be open after 2:30 and will have a sign on it. If not, please knock hard, or call my cell phone (612-812-9952). This door is right outside the meeting room, this avoids everyone traipsing through the building. The Stadium Village Green Line Light Rail Station is less than a block east of the building, kitty-corner at the intersection of University Ave SE and Huron Blvd SE. See the link below for the station's info page. Limited Street Parking is available on Ontario St SE and Beacon St SE. Otherwise, public parking is available in the Oak St and University Ave Ramps, each a little over two blocks away. Also, the Maroon lot is a little further on the north side of the Stadium. See the link below for more parking details. DO NOT park in University Parking Lot 70 on the north of the building, by the front door, between University Ave SE and the building, or Parking Lot 71 near the loading dock to the east of the building, they are contract parking only, and you be ticketed. Also, DO NOT park in the private parking lot to the south of the building; you will likely be tagged, booted, and towed from that parking lot. https://campusmaps.umn.edu/information-technology https://www.metrotransit.org/stadium-village-station https://pts.umn.edu/Park/Basic-Parking-Information/Parking-Facilities Zoom Details https://umn.zoom.us/j/94150849014?pwd=KLBvs9gAlRyobV2T5oTKyHcWpnUpKq.1<https://umn.zoom.us/j/97601658469?pwd=C6JWkmoYfB9WjZsnRPr9S5u1l4Zkuz.1> ID: 98593962100, passcode: 9txWv0 301-715-8592, meeting ID 98593962100, passcode 820959 -- Richard Laager Chief Manager, Director Midwest Internet Cooperative Exchange LLC

Oct. 8, 2025

Re: UG 48 Meeting & Election

by Richard Laager

As per our standard practice, here is the post-nomination period email. I only saw one nomination, which was myself (Richard Laager). On 2025-09-23 10:59, Richard Laager wrote: > Here is your reminder email about the board election nomination period. > > On 2025-08-30 00:11, Richard Laager wrote: >> >> The MICE UG 48 meeting will be held Wednesday, October 8, 3:00 pm at >> UMN and via Zoom. >> >> >> *Nominations* >> >> We will be holding an election for a board seat. The board consists >> of three seats with staggered terms, such that one seat comes up for >> re-election each year (at the fall meeting). This year, I (Richard >> Laager) am up for election. >> >> As in the past, we will close nominations in advance of the meeting >> so people have a chance to instruct a proxy appropriately. >> *Nominations will be open for two weeks, September 22 - October 5. *I >> plan to send reminder emails approximately at the start of and just >> after the nomination period. >> -- Richard Laager Chief Manager, Director Midwest Internet Cooperative Exchange LLC

Oct. 7, 2025

Please welcome Cisco Umbrella to MICE!

by Jeremy Lumby

Name: Cisco Umbrella Switch: MICE Core - 511 7504 Ports: E 6/6/1 ASN: 36692 IPv4: All IP info stays the same, this is an upgrade to 100G IPv6: Peering Contact: peering-umbrella(a)external.cisco.com Route Servers: YES Welcome Aboard Again! Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 Main: 612-355-7740 Direct: 612-392-6814 Fax: 952-873-7425 jlumby(a)mnvoip.com

Oct. 7, 2025

RE: v6 on rs1/rs2 fubar?

by Michael Hare

I am late to the game. Me too, starting 14:49 Central/GMT-5 for us. Yes I'm a Juniper. = Yes, we use BFD. = Yes, we have bgp-error-tolerance enabled. "set protocols bgp bgp-error-tolerance" [yes, my session is in global] = No, I haven't contacted Juniper, but I seem to have a misunderstanding about "bgp-error-tolerance" at the very least. Oct 7 14:49:10 r-minneapolis-511.uwsys.net [LEVEL='daemon.warning'] rpd[32044]: %DAEMON-4-RPD_BGP_NEIGHBOR_STATE_CHANGED: BGP peer 2001:504:27::d1af:0:1 (External AS 53679) changed st ate from Established to Idle (event RecvUpdate) (instance master) Oct 7 14:49:10 r-minneapolis-511.uwsys.net [LEVEL='daemon.err'] rpd[32044]: %DAEMON-3: BGP ERROR: Insufficient data for the packet Oct 7 14:49:10 r-minneapolis-511.uwsys.net [LEVEL='daemon.warning'] rpd[32044]: %DAEMON-4: bgp_read_v4_update:13800: NOTIFICATION sent to 2001:504:27::d1af:0:1 (External AS 53679): co de 3 (Update Message Error) subcode 1 (invalid attribute list) Oct 7 14:49:10 r-minneapolis-511.uwsys.net [LEVEL='daemon.err'] rpd[32044]: %DAEMON-3: Received malformed update from 2001:504:27::d1af:0:1 (External AS 53679) Oct 7 14:49:10 r-minneapolis-511.uwsys.net [LEVEL='daemon.err'] rpd[32044]: %DAEMON-3: Family inet6-unicast, prefix 2402:2de0::/32 Oct 7 14:49:10 r-minneapolis-511.uwsys.net [LEVEL='daemon.err'] rpd[32044]: %DAEMON-3: Malformed Attribute Unknown(35) flag 0xe0 length 1024. Oct 7 14:49:10 r-minneapolis-511.uwsys.net [LEVEL='daemon.err'] rpd[32044]: %DAEMON-3: BGP ERROR: Insufficient data for the packet -Michael > -----Original Message----- > From: Chris Wopat <wopat(a)wiscnet.net> > Sent: Tuesday, October 7, 2025 4:10 PM > To: mice-discuss(a)lists.micemn.net > Subject: Re: v6 on rs1/rs2 fubar? > > A few years ago this came out: > > https://urldefense.com/v3/__https://blog.benjojo.co.uk/post/bgp-path- > attributes-grave-error- > handling__;!!Mak6IKo!MP3CJWT3VXc9yyZOF25Zkl6ylazMccDKnoNw_w6IpTdMY > ZVmAbEhYBGcObRfkZtQ-jEoiojfbC9uW1cwqw$ > > And the Juniper specific link is to enable bgp-error-tolerance: > > https://urldefense.com/v3/__https://supportportal.juniper.net/s/article/202… > 08-29-Out-of-Cycle-Security-Bulletin-Junos-OS-and-Junos-OS-Evolved-A-crafted- > BGP-UPDATE-message-allows-a-remote-attacker-to-de-peer-reset-BGP- > sessions-CVE-2023- > 4481__;!!Mak6IKo!MP3CJWT3VXc9yyZOF25Zkl6ylazMccDKnoNw_w6IpTdMYZV > mAbEhYBGcObRfkZtQ-jEoiojfbC8RD26zjg$ > > > https://urldefense.com/v3/__https://www.juniper.net/documentation/us/en/soft > ware/junos/bgp/topics/topic-map/bgp-error- > messages.html__;!!Mak6IKo!MP3CJWT3VXc9yyZOF25Zkl6ylazMccDKnoNw_w6I > pTdMYZVmAbEhYBGcObRfkZtQ-jEoiojfbC9K24DUbQ$ > > We (as2381) are not having resets and do have this enabled, as a point > of comparison. > > Cheers, > -- > Chris Wopat > Network Engineer, WiscNet > wopat(a)wiscnet.net 608-210-3965 > _______________________________________________ > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net

Oct. 7, 2025

Re: v6 on rs1/rs2 fubar?

by Jay Hanke

HE filtered, sessions are starting to come back. On Tue, Oct 7, 2025 at 4:05 PM Nathan Beard <nbeard(a)7sigma.com> wrote: > > This is what I'm seeing on my Cisco NCS edge routers in case it's helpful to anybody: > > RP/0/RP0/CPU0:Oct 7 14:49:10.478 CDT: bgp[1089]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:2 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32 > RP/0/RP0/CPU0:Oct 7 15:18:11.888 CDT: bgp[1089]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:1 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32 > > The router is discarding the malformed update so I'm not seeing any actual issues based on what appears to be a correct RFC7606 error handling implementation by IOS-XR. > > Nathan Beard > 7Sigma Systems, Inc. > Senior Network Engineer > (218) 820-7636 > www.7sigma.com > > > On Tue, Oct 7, 2025 at 4:03 PM Jay Hanke via mice-discuss <mice-discuss(a)lists.micemn.net> wrote: >> >> Looks like bird 1.x isn't affected 2.x is. >> >> Initial indications are that the route with the malformed attribute is >> coming from the philippines. >> >> >> On Tue, Oct 7, 2025 at 3:46 PM Andrew Hoyos <hoyosa(a)gmail.com> wrote: >> > >> > Similarly: >> > >> > Oct 7 20:43:21 rpd[32393]: bgp_read_v4_update:12939: NOTIFICATION sent to 2001:504:27::d1af:0:2 (External AS 53679): code 3 (Update Message Error) subcode 1 (invalid attribute list) >> > >> > >> > We do have bgp-error-tolerance enabled, however, so in theory, this should throw out update itself, not teardown session. >> > >> > We’re actually seeing BFD die to route servers as well. >> > >> > Any commonality w/ the IX’s below and route server software? >> > We’re only seeing this on MICE, and v6, FWIW. >> > >> > Andrew Hoyos >> > hoyosa(a)gmail.com >> > >> > >> > >> > On Oct 7, 2025, at 15:40, Mark Kane <markk(a)honeycomb.net> wrote: >> > >> > FWIW, we saw this from rs1/rs2 on a Cisco: >> > >> > Oct 7 14:49:10 CDT: bgp[1087]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:2 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32 >> > Oct 7 15:18:11 CDT: bgp[1087]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:1 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32 >> > >> > Both sessions have remained up and appear normal. >> > >> > -----Original Message----- >> > From: Jay Hanke via mice-discuss <mice-discuss(a)lists.micemn.net> >> > Sent: October 7, 2025 15:34 >> > To: Jonathan Stewart <jonathan(a)les.net> >> > Cc: mice-discuss(a)lists.micemn.net >> > Subject: Re: v6 on rs1/rs2 fubar? >> > >> > Confirmed at FLIX/CIX-ATL/PIT-IX so far. >> > >> > All juniper but not all junipers affected >> > >> > If someone affected can you send me a trace it would be helpful. >> > >> > On Tue, Oct 7, 2025 at 3:32 PM Jonathan Stewart <jonathan(a)les.net> wrote: >> > >> > >> > I've seen BGP sessions resetting this afternoon on Juniper. >> > >> > What more do we know from the global community? >> > >> > Jonathan Stewart >> > Network Engineer >> > LES.NET - AS18451 >> > Desk: 1-204-666-6191 >> > Mobile: 1-204-990-2120 >> > 130 Portage Avenue E >> > Winnipeg, MB R3C 0A1 >> > CANADA >> > >> > On Tue, 2025-10-07 3:11 p.m., Jay Hanke via mice-discuss wrote: >> > >> > Looks like it might be a worldwide issue. >> > >> > Running Juniper? >> > >> > On Tue, Oct 7, 2025 at 3:07 PM Andrew Hoyos <hoyosa(a)gmail.com> >> > >> > wrote: >> > >> > Hi folks, >> > >> > We dropped v6 to rs1/rs2. Spot checked a few other members we >> > >> > manage, and same deal. >> > >> > Any work taking place on route servers at the moment? >> > >> > >> > Andrew Hoyos >> > hoyosa(a)gmail.com >> > >> > >> > >> > _______________________________________________ >> > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net >> > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net >> > >> > _______________________________________________ >> > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net >> > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net >> > >> > _______________________________________________ >> > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net >> > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net >> > >> > _______________________________________________ >> > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net >> > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net >> > >> > _______________________________________________ >> > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net >> > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net >> > >> > >> _______________________________________________ >> mice-discuss mailing list -- mice-discuss(a)lists.micemn.net >> To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net

Oct. 7, 2025

Re: v6 on rs1/rs2 fubar?

by Chris Wopat

A few years ago this came out: https://blog.benjojo.co.uk/post/bgp-path-attributes-grave-error-handling And the Juniper specific link is to enable bgp-error-tolerance: https://supportportal.juniper.net/s/article/2023-08-29-Out-of-Cycle-Securit… https://www.juniper.net/documentation/us/en/software/junos/bgp/topics/topic… We (as2381) are not having resets and do have this enabled, as a point of comparison. Cheers, -- Chris Wopat Network Engineer, WiscNet wopat(a)wiscnet.net 608-210-3965

Oct. 7, 2025

Re: v6 on rs1/rs2 fubar?

by Nathan Beard

This is what I'm seeing on my Cisco NCS edge routers in case it's helpful to anybody: RP/0/RP0/CPU0:Oct 7 14:49:10.478 CDT: bgp[1089]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:2 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32 RP/0/RP0/CPU0:Oct 7 15:18:11.888 CDT: bgp[1089]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:1 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32 The router is discarding the malformed update so I'm not seeing any actual issues based on what appears to be a correct RFC7606 error handling implementation by IOS-XR. *Nathan Beard* *7Sigma Systems, Inc.* Senior Network Engineer (218) 820-7636 www.7sigma.com On Tue, Oct 7, 2025 at 4:03 PM Jay Hanke via mice-discuss < mice-discuss(a)lists.micemn.net> wrote: > Looks like bird 1.x isn't affected 2.x is. > > Initial indications are that the route with the malformed attribute is > coming from the philippines. > > > On Tue, Oct 7, 2025 at 3:46 PM Andrew Hoyos <hoyosa(a)gmail.com> wrote: > > > > Similarly: > > > > Oct 7 20:43:21 rpd[32393]: bgp_read_v4_update:12939: NOTIFICATION sent > to 2001:504:27::d1af:0:2 (External AS 53679): code 3 (Update Message Error) > subcode 1 (invalid attribute list) > > > > > > We do have bgp-error-tolerance enabled, however, so in theory, this > should throw out update itself, not teardown session. > > > > We’re actually seeing BFD die to route servers as well. > > > > Any commonality w/ the IX’s below and route server software? > > We’re only seeing this on MICE, and v6, FWIW. > > > > Andrew Hoyos > > hoyosa(a)gmail.com > > > > > > > > On Oct 7, 2025, at 15:40, Mark Kane <markk(a)honeycomb.net> wrote: > > > > FWIW, we saw this from rs1/rs2 on a Cisco: > > > > Oct 7 14:49:10 CDT: bgp[1087]: %ROUTING-BGP-3-MALFORM_UPDATE : > Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:2 (VRF: > default) - message length 171 bytes, error flags 0x00000100, action taken > "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", > Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 > Unicast] 2402:2de0::/32 > > Oct 7 15:18:11 CDT: bgp[1087]: %ROUTING-BGP-3-MALFORM_UPDATE : > Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:1 (VRF: > default) - message length 171 bytes, error flags 0x00000100, action taken > "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", > Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 > Unicast] 2402:2de0::/32 > > > > Both sessions have remained up and appear normal. > > > > -----Original Message----- > > From: Jay Hanke via mice-discuss <mice-discuss(a)lists.micemn.net> > > Sent: October 7, 2025 15:34 > > To: Jonathan Stewart <jonathan(a)les.net> > > Cc: mice-discuss(a)lists.micemn.net > > Subject: Re: v6 on rs1/rs2 fubar? > > > > Confirmed at FLIX/CIX-ATL/PIT-IX so far. > > > > All juniper but not all junipers affected > > > > If someone affected can you send me a trace it would be helpful. > > > > On Tue, Oct 7, 2025 at 3:32 PM Jonathan Stewart <jonathan(a)les.net> > wrote: > > > > > > I've seen BGP sessions resetting this afternoon on Juniper. > > > > What more do we know from the global community? > > > > Jonathan Stewart > > Network Engineer > > LES.NET - AS18451 > > Desk: 1-204-666-6191 > > Mobile: 1-204-990-2120 > > 130 Portage Avenue E > > Winnipeg, MB R3C 0A1 > > CANADA > > > > On Tue, 2025-10-07 3:11 p.m., Jay Hanke via mice-discuss wrote: > > > > Looks like it might be a worldwide issue. > > > > Running Juniper? > > > > On Tue, Oct 7, 2025 at 3:07 PM Andrew Hoyos <hoyosa(a)gmail.com> > > > > wrote: > > > > Hi folks, > > > > We dropped v6 to rs1/rs2. Spot checked a few other members we > > > > manage, and same deal. > > > > Any work taking place on route servers at the moment? > > > > > > Andrew Hoyos > > hoyosa(a)gmail.com > > > > > > > > _______________________________________________ > > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net > > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net > > > > _______________________________________________ > > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net > > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net > > > > _______________________________________________ > > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net > > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net > > > > _______________________________________________ > > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net > > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net > > > > _______________________________________________ > > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net > > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net > > > > > _______________________________________________ > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net >

Oct. 7, 2025

Re: v6 on rs1/rs2 fubar?

by Jay Hanke

https://social.bgp.tools/@bgptools/statuses/01K7072TB9HS4JYN8E5Z01GEPK On Tue, Oct 7, 2025 at 3:50 PM Jon Forbes <jforbes(a)americomnetworks.com> wrote: > > I am getting this same error on my Cisco 9K. > > > -----Original Message----- > From: Mark Kane <markk(a)honeycomb.net> > Sent: Tuesday, October 7, 2025 3:41 PM > To: Jay Hanke <jayhanke(a)southfront.io> > Cc: mice-discuss(a)lists.micemn.net > Subject: RE: v6 on rs1/rs2 fubar? > > FWIW, we saw this from rs1/rs2 on a Cisco: > > Oct 7 14:49:10 CDT: bgp[1087]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:2 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32 Oct 7 15:18:11 CDT: bgp[1087]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:1 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32 > > Both sessions have remained up and appear normal. > > > -----Original Message----- > > From: Jay Hanke via mice-discuss <mice-discuss(a)lists.micemn.net> > > Sent: October 7, 2025 15:34 > > To: Jonathan Stewart <jonathan(a)les.net> > > Cc: mice-discuss(a)lists.micemn.net > > Subject: Re: v6 on rs1/rs2 fubar? > > > > Confirmed at FLIX/CIX-ATL/PIT-IX so far. > > > > All juniper but not all junipers affected > > > > If someone affected can you send me a trace it would be helpful. > > > > On Tue, Oct 7, 2025 at 3:32 PM Jonathan Stewart <jonathan(a)les.net> wrote: > > > > > > I've seen BGP sessions resetting this afternoon on Juniper. > > > > > > What more do we know from the global community? > > > > > > Jonathan Stewart > > > Network Engineer > > > LES.NET - AS18451 > > > Desk: 1-204-666-6191 > > > Mobile: 1-204-990-2120 > > > 130 Portage Avenue E > > > Winnipeg, MB R3C 0A1 > > > CANADA > > > > > > On Tue, 2025-10-07 3:11 p.m., Jay Hanke via mice-discuss wrote: > > > > Looks like it might be a worldwide issue. > > > > > > > > Running Juniper? > > > > > > > > On Tue, Oct 7, 2025 at 3:07 PM Andrew Hoyos <hoyosa(a)gmail.com> > > wrote: > > > >> Hi folks, > > > >> > > > >> We dropped v6 to rs1/rs2. Spot checked a few other members we > > manage, and same deal. > > > >> Any work taking place on route servers at the moment? > > > >> > > > >> > > > >> Andrew Hoyos > > > >> hoyosa(a)gmail.com > > > >> > > > >> > > > >> > > > >> _______________________________________________ > > > >> mice-discuss mailing list -- mice-discuss(a)lists.micemn.net To > > > >> unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net > > > > _______________________________________________ > > > > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net To > > > > unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net > > > _______________________________________________ > > > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net To > > > unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net > > _______________________________________________ > > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net To > > unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net > _______________________________________________ > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net

Oct. 7, 2025

Re: v6 on rs1/rs2 fubar?

by Jay Hanke

Looks like bird 1.x isn't affected 2.x is. Initial indications are that the route with the malformed attribute is coming from the philippines. On Tue, Oct 7, 2025 at 3:46 PM Andrew Hoyos <hoyosa(a)gmail.com> wrote: > > Similarly: > > Oct 7 20:43:21 rpd[32393]: bgp_read_v4_update:12939: NOTIFICATION sent to 2001:504:27::d1af:0:2 (External AS 53679): code 3 (Update Message Error) subcode 1 (invalid attribute list) > > > We do have bgp-error-tolerance enabled, however, so in theory, this should throw out update itself, not teardown session. > > We’re actually seeing BFD die to route servers as well. > > Any commonality w/ the IX’s below and route server software? > We’re only seeing this on MICE, and v6, FWIW. > > Andrew Hoyos > hoyosa(a)gmail.com > > > > On Oct 7, 2025, at 15:40, Mark Kane <markk(a)honeycomb.net> wrote: > > FWIW, we saw this from rs1/rs2 on a Cisco: > > Oct 7 14:49:10 CDT: bgp[1087]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:2 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32 > Oct 7 15:18:11 CDT: bgp[1087]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:1 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32 > > Both sessions have remained up and appear normal. > > -----Original Message----- > From: Jay Hanke via mice-discuss <mice-discuss(a)lists.micemn.net> > Sent: October 7, 2025 15:34 > To: Jonathan Stewart <jonathan(a)les.net> > Cc: mice-discuss(a)lists.micemn.net > Subject: Re: v6 on rs1/rs2 fubar? > > Confirmed at FLIX/CIX-ATL/PIT-IX so far. > > All juniper but not all junipers affected > > If someone affected can you send me a trace it would be helpful. > > On Tue, Oct 7, 2025 at 3:32 PM Jonathan Stewart <jonathan(a)les.net> wrote: > > > I've seen BGP sessions resetting this afternoon on Juniper. > > What more do we know from the global community? > > Jonathan Stewart > Network Engineer > LES.NET - AS18451 > Desk: 1-204-666-6191 > Mobile: 1-204-990-2120 > 130 Portage Avenue E > Winnipeg, MB R3C 0A1 > CANADA > > On Tue, 2025-10-07 3:11 p.m., Jay Hanke via mice-discuss wrote: > > Looks like it might be a worldwide issue. > > Running Juniper? > > On Tue, Oct 7, 2025 at 3:07 PM Andrew Hoyos <hoyosa(a)gmail.com> > > wrote: > > Hi folks, > > We dropped v6 to rs1/rs2. Spot checked a few other members we > > manage, and same deal. > > Any work taking place on route servers at the moment? > > > Andrew Hoyos > hoyosa(a)gmail.com > > > > _______________________________________________ > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net > > _______________________________________________ > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net > > _______________________________________________ > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net > > _______________________________________________ > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net > > _______________________________________________ > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net > >

Oct. 7, 2025

RE: v6 on rs1/rs2 fubar?

by Jon Forbes

I am getting this same error on my Cisco 9K. -----Original Message----- From: Mark Kane <markk(a)honeycomb.net> Sent: Tuesday, October 7, 2025 3:41 PM To: Jay Hanke <jayhanke(a)southfront.io> Cc: mice-discuss(a)lists.micemn.net Subject: RE: v6 on rs1/rs2 fubar? FWIW, we saw this from rs1/rs2 on a Cisco: Oct 7 14:49:10 CDT: bgp[1087]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:2 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32 Oct 7 15:18:11 CDT: bgp[1087]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:1 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32 Both sessions have remained up and appear normal. > -----Original Message----- > From: Jay Hanke via mice-discuss <mice-discuss(a)lists.micemn.net> > Sent: October 7, 2025 15:34 > To: Jonathan Stewart <jonathan(a)les.net> > Cc: mice-discuss(a)lists.micemn.net > Subject: Re: v6 on rs1/rs2 fubar? > > Confirmed at FLIX/CIX-ATL/PIT-IX so far. > > All juniper but not all junipers affected > > If someone affected can you send me a trace it would be helpful. > > On Tue, Oct 7, 2025 at 3:32 PM Jonathan Stewart <jonathan(a)les.net> wrote: > > > > I've seen BGP sessions resetting this afternoon on Juniper. > > > > What more do we know from the global community? > > > > Jonathan Stewart > > Network Engineer > > LES.NET - AS18451 > > Desk: 1-204-666-6191 > > Mobile: 1-204-990-2120 > > 130 Portage Avenue E > > Winnipeg, MB R3C 0A1 > > CANADA > > > > On Tue, 2025-10-07 3:11 p.m., Jay Hanke via mice-discuss wrote: > > > Looks like it might be a worldwide issue. > > > > > > Running Juniper? > > > > > > On Tue, Oct 7, 2025 at 3:07 PM Andrew Hoyos <hoyosa(a)gmail.com> > wrote: > > >> Hi folks, > > >> > > >> We dropped v6 to rs1/rs2. Spot checked a few other members we > manage, and same deal. > > >> Any work taking place on route servers at the moment? > > >> > > >> > > >> Andrew Hoyos > > >> hoyosa(a)gmail.com > > >> > > >> > > >> > > >> _______________________________________________ > > >> mice-discuss mailing list -- mice-discuss(a)lists.micemn.net To > > >> unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net > > > _______________________________________________ > > > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net To > > > unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net > > _______________________________________________ > > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net To > > unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net > _______________________________________________ > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net To > unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net _______________________________________________ mice-discuss mailing list -- mice-discuss(a)lists.micemn.net To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net

Oct. 7, 2025

Re: v6 on rs1/rs2 fubar?

by Andrew Hoyos

Similarly: Oct 7 20:43:21 rpd[32393]: bgp_read_v4_update:12939: NOTIFICATION sent to 2001:504:27::d1af:0:2 (External AS 53679): code 3 (Update Message Error) subcode 1 (invalid attribute list) We do have bgp-error-tolerance enabled, however, so in theory, this should throw out update itself, not teardown session. We’re actually seeing BFD die to route servers as well. Any commonality w/ the IX’s below and route server software? We’re only seeing this on MICE, and v6, FWIW. Andrew Hoyos hoyosa(a)gmail.com > On Oct 7, 2025, at 15:40, Mark Kane <markk(a)honeycomb.net> wrote: > > FWIW, we saw this from rs1/rs2 on a Cisco: > > Oct 7 14:49:10 CDT: bgp[1087]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:2 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32 > Oct 7 15:18:11 CDT: bgp[1087]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:1 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32 > > Both sessions have remained up and appear normal. > >> -----Original Message----- >> From: Jay Hanke via mice-discuss <mice-discuss(a)lists.micemn.net> >> Sent: October 7, 2025 15:34 >> To: Jonathan Stewart <jonathan(a)les.net> >> Cc: mice-discuss(a)lists.micemn.net >> Subject: Re: v6 on rs1/rs2 fubar? >> >> Confirmed at FLIX/CIX-ATL/PIT-IX so far. >> >> All juniper but not all junipers affected >> >> If someone affected can you send me a trace it would be helpful. >> >> On Tue, Oct 7, 2025 at 3:32 PM Jonathan Stewart <jonathan(a)les.net> wrote: >>> >>> I've seen BGP sessions resetting this afternoon on Juniper. >>> >>> What more do we know from the global community? >>> >>> Jonathan Stewart >>> Network Engineer >>> LES.NET - AS18451 >>> Desk: 1-204-666-6191 >>> Mobile: 1-204-990-2120 >>> 130 Portage Avenue E >>> Winnipeg, MB R3C 0A1 >>> CANADA >>> >>> On Tue, 2025-10-07 3:11 p.m., Jay Hanke via mice-discuss wrote: >>>> Looks like it might be a worldwide issue. >>>> >>>> Running Juniper? >>>> >>>> On Tue, Oct 7, 2025 at 3:07 PM Andrew Hoyos <hoyosa(a)gmail.com> >> wrote: >>>>> Hi folks, >>>>> >>>>> We dropped v6 to rs1/rs2. Spot checked a few other members we >> manage, and same deal. >>>>> Any work taking place on route servers at the moment? >>>>> >>>>> >>>>> Andrew Hoyos >>>>> hoyosa(a)gmail.com >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> mice-discuss mailing list -- mice-discuss(a)lists.micemn.net >>>>> To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net >>>> _______________________________________________ >>>> mice-discuss mailing list -- mice-discuss(a)lists.micemn.net >>>> To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net >>> _______________________________________________ >>> mice-discuss mailing list -- mice-discuss(a)lists.micemn.net >>> To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net >> _______________________________________________ >> mice-discuss mailing list -- mice-discuss(a)lists.micemn.net >> To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net > _______________________________________________ > mice-discuss mailing list -- mice-discuss(a)lists.micemn.net > To unsubscribe send an email to mice-discuss-leave(a)lists.micemn.net

Oct. 7, 2025