v6 on rs1/rs2 fubar?
Hi folks, We dropped v6 to rs1/rs2. Spot checked a few other members we manage, and same deal. Any work taking place on route servers at the moment? Andrew Hoyos hoyosa@gmail.com
Looks like it might be a worldwide issue. Running Juniper? On Tue, Oct 7, 2025 at 3:07 PM Andrew Hoyos <hoyosa@gmail.com> wrote:
Hi folks,
We dropped v6 to rs1/rs2. Spot checked a few other members we manage, and same deal. Any work taking place on route servers at the moment?
Andrew Hoyos hoyosa@gmail.com
_______________________________________________ mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
I've seen BGP sessions resetting this afternoon on Juniper. What more do we know from the global community? Jonathan Stewart Network Engineer LES.NET - AS18451 Desk: 1-204-666-6191 Mobile: 1-204-990-2120 130 Portage Avenue E Winnipeg, MB R3C 0A1 CANADA On Tue, 2025-10-07 3:11 p.m., Jay Hanke via mice-discuss wrote:
Looks like it might be a worldwide issue.
Running Juniper?
On Tue, Oct 7, 2025 at 3:07 PM Andrew Hoyos <hoyosa@gmail.com> wrote:
Hi folks,
We dropped v6 to rs1/rs2. Spot checked a few other members we manage, and same deal. Any work taking place on route servers at the moment?
Andrew Hoyos hoyosa@gmail.com
_______________________________________________ mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
Confirmed at FLIX/CIX-ATL/PIT-IX so far. All juniper but not all junipers affected If someone affected can you send me a trace it would be helpful. On Tue, Oct 7, 2025 at 3:32 PM Jonathan Stewart <jonathan@les.net> wrote:
I've seen BGP sessions resetting this afternoon on Juniper.
What more do we know from the global community?
Jonathan Stewart Network Engineer LES.NET - AS18451 Desk: 1-204-666-6191 Mobile: 1-204-990-2120 130 Portage Avenue E Winnipeg, MB R3C 0A1 CANADA
On Tue, 2025-10-07 3:11 p.m., Jay Hanke via mice-discuss wrote:
Looks like it might be a worldwide issue.
Running Juniper?
On Tue, Oct 7, 2025 at 3:07 PM Andrew Hoyos <hoyosa@gmail.com> wrote:
Hi folks,
We dropped v6 to rs1/rs2. Spot checked a few other members we manage, and same deal. Any work taking place on route servers at the moment?
Andrew Hoyos hoyosa@gmail.com
_______________________________________________ mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
FWIW, we saw this from rs1/rs2 on a Cisco: Oct 7 14:49:10 CDT: bgp[1087]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:2 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32 Oct 7 15:18:11 CDT: bgp[1087]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:1 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32 Both sessions have remained up and appear normal.
-----Original Message----- From: Jay Hanke via mice-discuss <mice-discuss@lists.micemn.net> Sent: October 7, 2025 15:34 To: Jonathan Stewart <jonathan@les.net> Cc: mice-discuss@lists.micemn.net Subject: Re: v6 on rs1/rs2 fubar?
Confirmed at FLIX/CIX-ATL/PIT-IX so far.
All juniper but not all junipers affected
If someone affected can you send me a trace it would be helpful.
On Tue, Oct 7, 2025 at 3:32 PM Jonathan Stewart <jonathan@les.net> wrote:
I've seen BGP sessions resetting this afternoon on Juniper.
What more do we know from the global community?
Jonathan Stewart Network Engineer LES.NET - AS18451 Desk: 1-204-666-6191 Mobile: 1-204-990-2120 130 Portage Avenue E Winnipeg, MB R3C 0A1 CANADA
On Tue, 2025-10-07 3:11 p.m., Jay Hanke via mice-discuss wrote:
Looks like it might be a worldwide issue.
Running Juniper?
On Tue, Oct 7, 2025 at 3:07 PM Andrew Hoyos <hoyosa@gmail.com>
wrote:
Hi folks,
We dropped v6 to rs1/rs2. Spot checked a few other members we manage, and same deal. Any work taking place on route servers at the moment?
Andrew Hoyos hoyosa@gmail.com
_______________________________________________ mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
Similarly: Oct 7 20:43:21 rpd[32393]: bgp_read_v4_update:12939: NOTIFICATION sent to 2001:504:27::d1af:0:2 (External AS 53679): code 3 (Update Message Error) subcode 1 (invalid attribute list) We do have bgp-error-tolerance enabled, however, so in theory, this should throw out update itself, not teardown session. We’re actually seeing BFD die to route servers as well. Any commonality w/ the IX’s below and route server software? We’re only seeing this on MICE, and v6, FWIW. Andrew Hoyos hoyosa@gmail.com
On Oct 7, 2025, at 15:40, Mark Kane <markk@honeycomb.net> wrote:
FWIW, we saw this from rs1/rs2 on a Cisco:
Oct 7 14:49:10 CDT: bgp[1087]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:2 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32 Oct 7 15:18:11 CDT: bgp[1087]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:1 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32
Both sessions have remained up and appear normal.
-----Original Message----- From: Jay Hanke via mice-discuss <mice-discuss@lists.micemn.net> Sent: October 7, 2025 15:34 To: Jonathan Stewart <jonathan@les.net> Cc: mice-discuss@lists.micemn.net Subject: Re: v6 on rs1/rs2 fubar?
Confirmed at FLIX/CIX-ATL/PIT-IX so far.
All juniper but not all junipers affected
If someone affected can you send me a trace it would be helpful.
On Tue, Oct 7, 2025 at 3:32 PM Jonathan Stewart <jonathan@les.net> wrote:
I've seen BGP sessions resetting this afternoon on Juniper.
What more do we know from the global community?
Jonathan Stewart Network Engineer LES.NET - AS18451 Desk: 1-204-666-6191 Mobile: 1-204-990-2120 130 Portage Avenue E Winnipeg, MB R3C 0A1 CANADA
On Tue, 2025-10-07 3:11 p.m., Jay Hanke via mice-discuss wrote:
Looks like it might be a worldwide issue.
Running Juniper?
On Tue, Oct 7, 2025 at 3:07 PM Andrew Hoyos <hoyosa@gmail.com>
wrote:
Hi folks,
We dropped v6 to rs1/rs2. Spot checked a few other members we manage, and same deal. Any work taking place on route servers at the moment?
Andrew Hoyos hoyosa@gmail.com
_______________________________________________ mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
Looks like bird 1.x isn't affected 2.x is. Initial indications are that the route with the malformed attribute is coming from the philippines. On Tue, Oct 7, 2025 at 3:46 PM Andrew Hoyos <hoyosa@gmail.com> wrote:
Similarly:
Oct 7 20:43:21 rpd[32393]: bgp_read_v4_update:12939: NOTIFICATION sent to 2001:504:27::d1af:0:2 (External AS 53679): code 3 (Update Message Error) subcode 1 (invalid attribute list)
We do have bgp-error-tolerance enabled, however, so in theory, this should throw out update itself, not teardown session.
We’re actually seeing BFD die to route servers as well.
Any commonality w/ the IX’s below and route server software? We’re only seeing this on MICE, and v6, FWIW.
Andrew Hoyos hoyosa@gmail.com
On Oct 7, 2025, at 15:40, Mark Kane <markk@honeycomb.net> wrote:
FWIW, we saw this from rs1/rs2 on a Cisco:
Oct 7 14:49:10 CDT: bgp[1087]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:2 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32 Oct 7 15:18:11 CDT: bgp[1087]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:1 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32
Both sessions have remained up and appear normal.
-----Original Message----- From: Jay Hanke via mice-discuss <mice-discuss@lists.micemn.net> Sent: October 7, 2025 15:34 To: Jonathan Stewart <jonathan@les.net> Cc: mice-discuss@lists.micemn.net Subject: Re: v6 on rs1/rs2 fubar?
Confirmed at FLIX/CIX-ATL/PIT-IX so far.
All juniper but not all junipers affected
If someone affected can you send me a trace it would be helpful.
On Tue, Oct 7, 2025 at 3:32 PM Jonathan Stewart <jonathan@les.net> wrote:
I've seen BGP sessions resetting this afternoon on Juniper.
What more do we know from the global community?
Jonathan Stewart Network Engineer LES.NET - AS18451 Desk: 1-204-666-6191 Mobile: 1-204-990-2120 130 Portage Avenue E Winnipeg, MB R3C 0A1 CANADA
On Tue, 2025-10-07 3:11 p.m., Jay Hanke via mice-discuss wrote:
Looks like it might be a worldwide issue.
Running Juniper?
On Tue, Oct 7, 2025 at 3:07 PM Andrew Hoyos <hoyosa@gmail.com>
wrote:
Hi folks,
We dropped v6 to rs1/rs2. Spot checked a few other members we
manage, and same deal.
Any work taking place on route servers at the moment?
Andrew Hoyos hoyosa@gmail.com
_______________________________________________ mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
_______________________________________________ mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
_______________________________________________ mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
_______________________________________________ mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
_______________________________________________ mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
This is what I'm seeing on my Cisco NCS edge routers in case it's helpful to anybody: RP/0/RP0/CPU0:Oct 7 14:49:10.478 CDT: bgp[1089]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:2 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32 RP/0/RP0/CPU0:Oct 7 15:18:11.888 CDT: bgp[1089]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:1 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32 The router is discarding the malformed update so I'm not seeing any actual issues based on what appears to be a correct RFC7606 error handling implementation by IOS-XR. *Nathan Beard* *7Sigma Systems, Inc.* Senior Network Engineer (218) 820-7636 www.7sigma.com On Tue, Oct 7, 2025 at 4:03 PM Jay Hanke via mice-discuss < mice-discuss@lists.micemn.net> wrote:
Looks like bird 1.x isn't affected 2.x is.
Initial indications are that the route with the malformed attribute is coming from the philippines.
On Tue, Oct 7, 2025 at 3:46 PM Andrew Hoyos <hoyosa@gmail.com> wrote:
Similarly:
Oct 7 20:43:21 rpd[32393]: bgp_read_v4_update:12939: NOTIFICATION sent
to 2001:504:27::d1af:0:2 (External AS 53679): code 3 (Update Message Error) subcode 1 (invalid attribute list)
We do have bgp-error-tolerance enabled, however, so in theory, this
should throw out update itself, not teardown session.
We’re actually seeing BFD die to route servers as well.
Any commonality w/ the IX’s below and route server software? We’re only seeing this on MICE, and v6, FWIW.
Andrew Hoyos hoyosa@gmail.com
On Oct 7, 2025, at 15:40, Mark Kane <markk@honeycomb.net> wrote:
FWIW, we saw this from rs1/rs2 on a Cisco:
Oct 7 14:49:10 CDT: bgp[1087]: %ROUTING-BGP-3-MALFORM_UPDATE :
Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:2 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32
Oct 7 15:18:11 CDT: bgp[1087]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:1 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32
Both sessions have remained up and appear normal.
-----Original Message----- From: Jay Hanke via mice-discuss <mice-discuss@lists.micemn.net> Sent: October 7, 2025 15:34 To: Jonathan Stewart <jonathan@les.net> Cc: mice-discuss@lists.micemn.net Subject: Re: v6 on rs1/rs2 fubar?
Confirmed at FLIX/CIX-ATL/PIT-IX so far.
All juniper but not all junipers affected
If someone affected can you send me a trace it would be helpful.
On Tue, Oct 7, 2025 at 3:32 PM Jonathan Stewart <jonathan@les.net> wrote:
I've seen BGP sessions resetting this afternoon on Juniper.
What more do we know from the global community?
Jonathan Stewart Network Engineer LES.NET - AS18451 Desk: 1-204-666-6191 Mobile: 1-204-990-2120 130 Portage Avenue E Winnipeg, MB R3C 0A1 CANADA
On Tue, 2025-10-07 3:11 p.m., Jay Hanke via mice-discuss wrote:
Looks like it might be a worldwide issue.
Running Juniper?
On Tue, Oct 7, 2025 at 3:07 PM Andrew Hoyos <hoyosa@gmail.com>
wrote:
Hi folks,
We dropped v6 to rs1/rs2. Spot checked a few other members we
manage, and same deal.
Any work taking place on route servers at the moment?
Andrew Hoyos hoyosa@gmail.com
_______________________________________________ mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
_______________________________________________ mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
_______________________________________________ mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
_______________________________________________ mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
_______________________________________________ mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
_______________________________________________ mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
HE filtered, sessions are starting to come back. On Tue, Oct 7, 2025 at 4:05 PM Nathan Beard <nbeard@7sigma.com> wrote:
This is what I'm seeing on my Cisco NCS edge routers in case it's helpful to anybody:
RP/0/RP0/CPU0:Oct 7 14:49:10.478 CDT: bgp[1089]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:2 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32 RP/0/RP0/CPU0:Oct 7 15:18:11.888 CDT: bgp[1089]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:1 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32
The router is discarding the malformed update so I'm not seeing any actual issues based on what appears to be a correct RFC7606 error handling implementation by IOS-XR.
Nathan Beard 7Sigma Systems, Inc. Senior Network Engineer (218) 820-7636 www.7sigma.com
On Tue, Oct 7, 2025 at 4:03 PM Jay Hanke via mice-discuss <mice-discuss@lists.micemn.net> wrote:
Looks like bird 1.x isn't affected 2.x is.
Initial indications are that the route with the malformed attribute is coming from the philippines.
On Tue, Oct 7, 2025 at 3:46 PM Andrew Hoyos <hoyosa@gmail.com> wrote:
Similarly:
Oct 7 20:43:21 rpd[32393]: bgp_read_v4_update:12939: NOTIFICATION sent to 2001:504:27::d1af:0:2 (External AS 53679): code 3 (Update Message Error) subcode 1 (invalid attribute list)
We do have bgp-error-tolerance enabled, however, so in theory, this should throw out update itself, not teardown session.
We’re actually seeing BFD die to route servers as well.
Any commonality w/ the IX’s below and route server software? We’re only seeing this on MICE, and v6, FWIW.
Andrew Hoyos hoyosa@gmail.com
On Oct 7, 2025, at 15:40, Mark Kane <markk@honeycomb.net> wrote:
FWIW, we saw this from rs1/rs2 on a Cisco:
Oct 7 14:49:10 CDT: bgp[1087]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:2 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32 Oct 7 15:18:11 CDT: bgp[1087]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:1 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32
Both sessions have remained up and appear normal.
-----Original Message----- From: Jay Hanke via mice-discuss <mice-discuss@lists.micemn.net> Sent: October 7, 2025 15:34 To: Jonathan Stewart <jonathan@les.net> Cc: mice-discuss@lists.micemn.net Subject: Re: v6 on rs1/rs2 fubar?
Confirmed at FLIX/CIX-ATL/PIT-IX so far.
All juniper but not all junipers affected
If someone affected can you send me a trace it would be helpful.
On Tue, Oct 7, 2025 at 3:32 PM Jonathan Stewart <jonathan@les.net> wrote:
I've seen BGP sessions resetting this afternoon on Juniper.
What more do we know from the global community?
Jonathan Stewart Network Engineer LES.NET - AS18451 Desk: 1-204-666-6191 Mobile: 1-204-990-2120 130 Portage Avenue E Winnipeg, MB R3C 0A1 CANADA
On Tue, 2025-10-07 3:11 p.m., Jay Hanke via mice-discuss wrote:
Looks like it might be a worldwide issue.
Running Juniper?
On Tue, Oct 7, 2025 at 3:07 PM Andrew Hoyos <hoyosa@gmail.com>
wrote:
Hi folks,
We dropped v6 to rs1/rs2. Spot checked a few other members we
manage, and same deal.
Any work taking place on route servers at the moment?
Andrew Hoyos hoyosa@gmail.com
_______________________________________________ mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
_______________________________________________ mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
_______________________________________________ mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
_______________________________________________ mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
_______________________________________________ mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
_______________________________________________ mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
I am getting this same error on my Cisco 9K. -----Original Message----- From: Mark Kane <markk@honeycomb.net> Sent: Tuesday, October 7, 2025 3:41 PM To: Jay Hanke <jayhanke@southfront.io> Cc: mice-discuss@lists.micemn.net Subject: RE: v6 on rs1/rs2 fubar? FWIW, we saw this from rs1/rs2 on a Cisco: Oct 7 14:49:10 CDT: bgp[1087]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:2 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32 Oct 7 15:18:11 CDT: bgp[1087]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:1 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32 Both sessions have remained up and appear normal.
-----Original Message----- From: Jay Hanke via mice-discuss <mice-discuss@lists.micemn.net> Sent: October 7, 2025 15:34 To: Jonathan Stewart <jonathan@les.net> Cc: mice-discuss@lists.micemn.net Subject: Re: v6 on rs1/rs2 fubar?
Confirmed at FLIX/CIX-ATL/PIT-IX so far.
All juniper but not all junipers affected
If someone affected can you send me a trace it would be helpful.
On Tue, Oct 7, 2025 at 3:32 PM Jonathan Stewart <jonathan@les.net> wrote:
I've seen BGP sessions resetting this afternoon on Juniper.
What more do we know from the global community?
Jonathan Stewart Network Engineer LES.NET - AS18451 Desk: 1-204-666-6191 Mobile: 1-204-990-2120 130 Portage Avenue E Winnipeg, MB R3C 0A1 CANADA
On Tue, 2025-10-07 3:11 p.m., Jay Hanke via mice-discuss wrote:
Looks like it might be a worldwide issue.
Running Juniper?
On Tue, Oct 7, 2025 at 3:07 PM Andrew Hoyos <hoyosa@gmail.com>
wrote:
Hi folks,
We dropped v6 to rs1/rs2. Spot checked a few other members we manage, and same deal. Any work taking place on route servers at the moment?
Andrew Hoyos hoyosa@gmail.com
_______________________________________________ mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
https://social.bgp.tools/@bgptools/statuses/01K7072TB9HS4JYN8E5Z01GEPK On Tue, Oct 7, 2025 at 3:50 PM Jon Forbes <jforbes@americomnetworks.com> wrote:
I am getting this same error on my Cisco 9K.
-----Original Message----- From: Mark Kane <markk@honeycomb.net> Sent: Tuesday, October 7, 2025 3:41 PM To: Jay Hanke <jayhanke@southfront.io> Cc: mice-discuss@lists.micemn.net Subject: RE: v6 on rs1/rs2 fubar?
FWIW, we saw this from rs1/rs2 on a Cisco:
Oct 7 14:49:10 CDT: bgp[1087]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:2 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32 Oct 7 15:18:11 CDT: bgp[1087]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 2001:504:27::d1af:0:1 (VRF: default) - message length 171 bytes, error flags 0x00000100, action taken "TreatAsWdr". Error details: "Error 0x00000100, Field "Attr-length", Attribute 35 (Flags 0xe0, Length 1024), Data [f0230400]". NLRIs: [IPv6 Unicast] 2402:2de0::/32
Both sessions have remained up and appear normal.
-----Original Message----- From: Jay Hanke via mice-discuss <mice-discuss@lists.micemn.net> Sent: October 7, 2025 15:34 To: Jonathan Stewart <jonathan@les.net> Cc: mice-discuss@lists.micemn.net Subject: Re: v6 on rs1/rs2 fubar?
Confirmed at FLIX/CIX-ATL/PIT-IX so far.
All juniper but not all junipers affected
If someone affected can you send me a trace it would be helpful.
On Tue, Oct 7, 2025 at 3:32 PM Jonathan Stewart <jonathan@les.net> wrote:
I've seen BGP sessions resetting this afternoon on Juniper.
What more do we know from the global community?
Jonathan Stewart Network Engineer LES.NET - AS18451 Desk: 1-204-666-6191 Mobile: 1-204-990-2120 130 Portage Avenue E Winnipeg, MB R3C 0A1 CANADA
On Tue, 2025-10-07 3:11 p.m., Jay Hanke via mice-discuss wrote:
Looks like it might be a worldwide issue.
Running Juniper?
On Tue, Oct 7, 2025 at 3:07 PM Andrew Hoyos <hoyosa@gmail.com>
wrote:
Hi folks,
We dropped v6 to rs1/rs2. Spot checked a few other members we manage, and same deal. Any work taking place on route servers at the moment?
Andrew Hoyos hoyosa@gmail.com
_______________________________________________ mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
The errors in the logs are: Received: Malformed attribute list and Received: Malformed attribute list: f0230400025a94000000000000000000 Most do NOT have the ": ..." part. Only one particular AS has that additional information. It's not clear to me if that indicates that particular AS is originating the malformed data or not. That AS is not connected to all of the affected IXes, so that may be a red-herring. I'm intentionally not naming the particular AS at this time, to avoid publicly shaming them in case it's nothing they are doing. We are reaching out to them directly, though. -- Richard Laager Chief Manager, Director Midwest Internet Cooperative Exchange LLC
A few years ago this came out: https://blog.benjojo.co.uk/post/bgp-path-attributes-grave-error-handling And the Juniper specific link is to enable bgp-error-tolerance: https://supportportal.juniper.net/s/article/2023-08-29-Out-of-Cycle-Security... https://www.juniper.net/documentation/us/en/software/junos/bgp/topics/topic-... We (as2381) are not having resets and do have this enabled, as a point of comparison. Cheers, -- Chris Wopat Network Engineer, WiscNet wopat@wiscnet.net 608-210-3965
I am late to the game. Me too, starting 14:49 Central/GMT-5 for us. Yes I'm a Juniper. = Yes, we use BFD. = Yes, we have bgp-error-tolerance enabled. "set protocols bgp bgp-error-tolerance" [yes, my session is in global] = No, I haven't contacted Juniper, but I seem to have a misunderstanding about "bgp-error-tolerance" at the very least. Oct 7 14:49:10 r-minneapolis-511.uwsys.net [LEVEL='daemon.warning'] rpd[32044]: %DAEMON-4-RPD_BGP_NEIGHBOR_STATE_CHANGED: BGP peer 2001:504:27::d1af:0:1 (External AS 53679) changed st ate from Established to Idle (event RecvUpdate) (instance master) Oct 7 14:49:10 r-minneapolis-511.uwsys.net [LEVEL='daemon.err'] rpd[32044]: %DAEMON-3: BGP ERROR: Insufficient data for the packet Oct 7 14:49:10 r-minneapolis-511.uwsys.net [LEVEL='daemon.warning'] rpd[32044]: %DAEMON-4: bgp_read_v4_update:13800: NOTIFICATION sent to 2001:504:27::d1af:0:1 (External AS 53679): co de 3 (Update Message Error) subcode 1 (invalid attribute list) Oct 7 14:49:10 r-minneapolis-511.uwsys.net [LEVEL='daemon.err'] rpd[32044]: %DAEMON-3: Received malformed update from 2001:504:27::d1af:0:1 (External AS 53679) Oct 7 14:49:10 r-minneapolis-511.uwsys.net [LEVEL='daemon.err'] rpd[32044]: %DAEMON-3: Family inet6-unicast, prefix 2402:2de0::/32 Oct 7 14:49:10 r-minneapolis-511.uwsys.net [LEVEL='daemon.err'] rpd[32044]: %DAEMON-3: Malformed Attribute Unknown(35) flag 0xe0 length 1024. Oct 7 14:49:10 r-minneapolis-511.uwsys.net [LEVEL='daemon.err'] rpd[32044]: %DAEMON-3: BGP ERROR: Insufficient data for the packet -Michael
-----Original Message----- From: Chris Wopat <wopat@wiscnet.net> Sent: Tuesday, October 7, 2025 4:10 PM To: mice-discuss@lists.micemn.net Subject: Re: v6 on rs1/rs2 fubar?
A few years ago this came out:
https://urldefense.com/v3/__https://blog.benjojo.co.uk/post/bgp-path- attributes-grave-error- handling__;!!Mak6IKo!MP3CJWT3VXc9yyZOF25Zkl6ylazMccDKnoNw_w6IpTdMY ZVmAbEhYBGcObRfkZtQ-jEoiojfbC9uW1cwqw$
And the Juniper specific link is to enable bgp-error-tolerance:
https://urldefense.com/v3/__https://supportportal.juniper.net/s/article/2023... 08-29-Out-of-Cycle-Security-Bulletin-Junos-OS-and-Junos-OS-Evolved-A-crafted- BGP-UPDATE-message-allows-a-remote-attacker-to-de-peer-reset-BGP- sessions-CVE-2023- 4481__;!!Mak6IKo!MP3CJWT3VXc9yyZOF25Zkl6ylazMccDKnoNw_w6IpTdMYZV mAbEhYBGcObRfkZtQ-jEoiojfbC8RD26zjg$
https://urldefense.com/v3/__https://www.juniper.net/documentation/us/en/soft ware/junos/bgp/topics/topic-map/bgp-error- messages.html__;!!Mak6IKo!MP3CJWT3VXc9yyZOF25Zkl6ylazMccDKnoNw_w6I pTdMYZVmAbEhYBGcObRfkZtQ-jEoiojfbC9K24DUbQ$
We (as2381) are not having resets and do have this enabled, as a point of comparison.
Cheers, -- Chris Wopat Network Engineer, WiscNet wopat@wiscnet.net 608-210-3965 _______________________________________________ mice-discuss mailing list -- mice-discuss@lists.micemn.net To unsubscribe send an email to mice-discuss-leave@lists.micemn.net
participants (9)
-
Andrew Hoyos -
Chris Wopat -
Jay Hanke -
Jon Forbes -
Jonathan Stewart -
Mark Kane -
Michael Hare -
Nathan Beard -
Richard Laager