- mice-discuss - lists.micemn.net

Re: Could I have a MICE admin look at the route reflectors?

by Frank Bulk

Doug, Thanks. We have been transitioning from AS53347 to AS18883 .... so if you can remove the AS55347 configuration on MICE's RR's, that would be great. Frank -----Original Message----- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Doug McIntyre Sent: Saturday, September 7, 2019 4:15 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Could I have a MICE admin look at the route reflectors? On Fri, Sep 06, 2019 at 11:51:41PM +0000, Frank Bulk wrote: >For some reason at 5:38 pm we (AS18883, FiberNet) lost IPv4 and IPv6 peering with both of MICE RR's and Hurricane Electric and it's been unstable since then. > >All our other peers with MICE are fine. > >I didn't change any configuration, even though the "bad AS number" note in some of the error messages would suggest that. We've been getting a lot of these.. Sep 3 18:02:14 micemn-01 bird6: NASN_18883: Received: Connection collision resolution Sep 3 18:02:18 micemn-01 bird6: NASN_18883: Received: Connection collision resolution Sep 3 18:02:23 micemn-01 bird6: NASN_18883: Received: Connection collision resolution Sep 3 18:02:28 micemn-01 bird6: NASN_18883: Received: Connection collision resolution And these 2019-09-07 16:08:15 <RMT> NASN_53347: Error: Bad peer AS: 18883 As well as I suppose the expected 2019-09-07 16:08:16 <RMT> NASN_18883: Received: Administrative shutdown 2019-09-07 16:08:20 <RMT> NASN_18883: Received: Administrative shutdown 2019-09-07 16:08:24 <RMT> NASN_18883: Received: Administrative shutdown If you have 18883 link shutdown. But with the first and second errors, I see you have two AS's, and two IPs coming across Eth10 on the switch. The middle error message seems to imply that the configuration we have setup for AS 53347 on 206.108.255.67 is receiving a peer attempt for 18883 which is configured to talk on 206.108.255.133 on our side. Could these two be mixed up at all? -- Doug McIntyre <merlyn(a)iphouse.net> ~.~ ipHouse ~.~ Network Engineer/Provisioning/Jack of all Trades

Sept. 9, 2019

Re: Could I have a MICE admin look at the route reflectors?

by Doug McIntyre

On Fri, Sep 06, 2019 at 11:51:41PM +0000, Frank Bulk wrote: >For some reason at 5:38 pm we (AS18883, FiberNet) lost IPv4 and IPv6 peering with both of MICE RR's and Hurricane Electric and it's been unstable since then. > >All our other peers with MICE are fine. > >I didn't change any configuration, even though the "bad AS number" note in some of the error messages would suggest that. We've been getting a lot of these.. Sep 3 18:02:14 micemn-01 bird6: NASN_18883: Received: Connection collision resolution Sep 3 18:02:18 micemn-01 bird6: NASN_18883: Received: Connection collision resolution Sep 3 18:02:23 micemn-01 bird6: NASN_18883: Received: Connection collision resolution Sep 3 18:02:28 micemn-01 bird6: NASN_18883: Received: Connection collision resolution And these 2019-09-07 16:08:15 <RMT> NASN_53347: Error: Bad peer AS: 18883 As well as I suppose the expected 2019-09-07 16:08:16 <RMT> NASN_18883: Received: Administrative shutdown 2019-09-07 16:08:20 <RMT> NASN_18883: Received: Administrative shutdown 2019-09-07 16:08:24 <RMT> NASN_18883: Received: Administrative shutdown If you have 18883 link shutdown. But with the first and second errors, I see you have two AS's, and two IPs coming across Eth10 on the switch. The middle error message seems to imply that the configuration we have setup for AS 53347 on 206.108.255.67 is receiving a peer attempt for 18883 which is configured to talk on 206.108.255.133 on our side. Could these two be mixed up at all? -- Doug McIntyre <merlyn(a)iphouse.net> ~.~ ipHouse ~.~ Network Engineer/Provisioning/Jack of all Trades

Sept. 7, 2019

Re: Could I have a MICE admin look at the route reflectors?

by Frank Bulk

It did not -- still waiting to hear from MICE admins on what the RR saw on its side. Frank -----Original Message----- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Jay Hanke Sent: Saturday, September 7, 2019 10:53 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Could I have a MICE admin look at the route reflectors? Did this get resolved? On Fri, Sep 6, 2019 at 6:52 PM Frank Bulk <fbulk(a)mypremieronline.com> wrote: > > For some reason at 5:38 pm we (AS18883, FiberNet) lost IPv4 and IPv6 peering with both of MICE RR's and Hurricane Electric and it's been unstable since then. > > All our other peers with MICE are fine. > > I didn't change any configuration, even though the "bad AS number" note in some of the error messages would suggest that. > > Any hints would be welcomed. > > I've shutdown those six sessions for now. > > Frank > > Sep 6 17:38:49 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:38:49 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:39:21 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:2 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:39:21 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:2 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:39:28 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:39:28 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.1 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:39:28 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.1 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x000049c3 > Sep 6 17:39:33 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.2 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:39:33 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.2 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:39:36 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.2 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x000049c3 > Sep 6 17:39:39 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 17:40:02 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:40:02 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:40:06 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x0000d063 > Sep 6 17:40:22 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.1 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 17:40:28 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.2 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 17:40:48 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:40:48 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.1 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:41:46 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 17:41:51 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:2 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:41:51 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:2 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:42:00 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:2 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x0000d063 > Sep 6 17:42:33 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.1 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x000049c3 > Sep 6 17:42:34 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.1 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 17:42:44 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:42:44 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:42:44 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x0000d063 > Sep 6 17:43:46 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:2 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 17:44:36 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.2 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established > ... > > Sep 6 17:44:34 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 17:44:39 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 17:46:15 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 17:50:54 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 17:50:57 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:00:44 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 18:00:49 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:04:53 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:11:03 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 18:11:06 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:13:05 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 18:13:13 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:14:37 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 18:14:41 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:15:35 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:23:11 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 18:23:16 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:26:58 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:27:46 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 18:27:48 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:33:58 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 18:34:07 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

Sept. 7, 2019

Re: Could I have a MICE admin look at the route reflectors?

by Jay Hanke

Did this get resolved? On Fri, Sep 6, 2019 at 6:52 PM Frank Bulk <fbulk(a)mypremieronline.com> wrote: > > For some reason at 5:38 pm we (AS18883, FiberNet) lost IPv4 and IPv6 peering with both of MICE RR's and Hurricane Electric and it's been unstable since then. > > All our other peers with MICE are fine. > > I didn't change any configuration, even though the "bad AS number" note in some of the error messages would suggest that. > > Any hints would be welcomed. > > I've shutdown those six sessions for now. > > Frank > > Sep 6 17:38:49 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:38:49 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:39:21 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:2 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:39:21 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:2 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:39:28 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:39:28 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.1 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:39:28 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.1 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x000049c3 > Sep 6 17:39:33 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.2 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:39:33 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.2 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:39:36 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.2 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x000049c3 > Sep 6 17:39:39 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 17:40:02 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:40:02 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:40:06 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x0000d063 > Sep 6 17:40:22 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.1 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 17:40:28 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.2 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 17:40:48 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:40:48 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.1 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:41:46 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 17:41:51 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:2 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:41:51 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:2 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:42:00 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:2 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x0000d063 > Sep 6 17:42:33 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.1 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x000049c3 > Sep 6 17:42:34 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.1 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 17:42:44 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:42:44 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:42:44 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x0000d063 > Sep 6 17:43:46 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:2 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 17:44:36 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.2 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established > ... > > Sep 6 17:44:34 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 17:44:39 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 17:46:15 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 17:50:54 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 17:50:57 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:00:44 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 18:00:49 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:04:53 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:11:03 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 18:11:06 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:13:05 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 18:13:13 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:14:37 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 18:14:41 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:15:35 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:23:11 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 18:23:16 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:26:58 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:27:46 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 18:27:48 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:33:58 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 18:34:07 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

Sept. 7, 2019

Re: zayo again?

by Frank Bulk

Sorry for the delayed response to this thread. I did get the maintenance ... see attached. Frank -----Original Message----- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Mike Johnston Sent: Friday, August 30, 2019 10:03 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] zayo again? On 2019-08-30 05:00, James Urwiller wrote: > they also asked me how to gain access to the location and provide a > local contact. I tried explaining that it is their hut, not mine and I > don't have access...because its not my hut. Classic. That, unfortunately, sounds normal for Zayo.

Sept. 6, 2019

Could I have a MICE admin look at the route reflectors?

by Frank Bulk

For some reason at 5:38 pm we (AS18883, FiberNet) lost IPv4 and IPv6 peering with both of MICE RR's and Hurricane Electric and it's been unstable since then. All our other peers with MICE are fine. I didn't change any configuration, even though the "bad AS number" note in some of the error messages would suggest that. Any hints would be welcomed. I've shutdown those six sessions for now. Frank Sep 6 17:38:49 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes Sep 6 17:38:49 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state Established event RecvNotify new state Idle Sep 6 17:39:21 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:2 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes Sep 6 17:39:21 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:2 (AS 53679) old state Established event RecvNotify new state Idle Sep 6 17:39:28 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes Sep 6 17:39:28 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.1 (AS 53679) old state Established event RecvNotify new state Idle Sep 6 17:39:28 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.1 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x000049c3 Sep 6 17:39:33 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.2 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes Sep 6 17:39:33 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.2 (AS 53679) old state Established event RecvNotify new state Idle Sep 6 17:39:36 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.2 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x000049c3 Sep 6 17:39:39 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 17:40:02 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes Sep 6 17:40:02 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state Established event RecvNotify new state Idle Sep 6 17:40:06 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x0000d063 Sep 6 17:40:22 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.1 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 17:40:28 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.2 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 17:40:48 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes Sep 6 17:40:48 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.1 (AS 53679) old state Established event RecvNotify new state Idle Sep 6 17:41:46 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 17:41:51 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:2 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes Sep 6 17:41:51 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:2 (AS 53679) old state Established event RecvNotify new state Idle Sep 6 17:42:00 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:2 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x0000d063 Sep 6 17:42:33 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.1 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x000049c3 Sep 6 17:42:34 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.1 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 17:42:44 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes Sep 6 17:42:44 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state Established event RecvNotify new state Idle Sep 6 17:42:44 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x0000d063 Sep 6 17:43:46 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:2 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 17:44:36 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.2 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established ... Sep 6 17:44:34 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle Sep 6 17:44:39 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 17:46:15 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle Sep 6 17:50:54 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle Sep 6 17:50:57 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 18:00:44 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state Established event RecvNotify new state Idle Sep 6 18:00:49 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 18:04:53 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 18:11:03 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle Sep 6 18:11:06 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 18:13:05 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle Sep 6 18:13:13 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 18:14:37 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state Established event RecvNotify new state Idle Sep 6 18:14:41 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 18:15:35 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 18:23:11 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state Established event RecvNotify new state Idle Sep 6 18:23:16 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 18:26:58 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 18:27:46 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle Sep 6 18:27:48 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 18:33:58 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state Established event RecvNotify new state Idle Sep 6 18:34:07 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established

Sept. 6, 2019

US Internet - AS 10242 - Router Replacement

by Justin Krejci

MICE, US Internet is shutting down BGP sessions for a few minutes to migrate the MICE connection onto a new router. BGP sessions will be re-established within a few minutes. This will be happening in the next few minutes after this email. Justin Krejci 952-253-3200 ? 1-800-US-INTERNET 12450 Wayzata Blvd. Suite 315 ? Minnetonka, MN 55305 [http://www.usinternet.com/img/usi-signature.png]

Sept. 6, 2019

Re: zayo again?

by Mike Johnston

On 2019-08-30 05:00, James Urwiller wrote: > they also asked me how to gain access to the location and provide a > local contact. I tried explaining that it is their hut, not mine and I > don't have access...because its not my hut. Classic. That, unfortunately, sounds normal for Zayo.

Aug. 30, 2019

Re: zayo again?

by James Urwiller

Yes that is what I am learning - we were not notified of the maintenance and no one at zayo can find the maintenance ticket either.. we are back up and running for now. James Urwiller Network Architect Stealth Broadband james.urwiller(a)stealthbroadband.com -----Original Message----- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Lubitz, Jake (MNIT) Sent: Friday, August 30, 2019 08:46 To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] zayo again? For what's it worth, it sounds like Zayo had a maintenance event for a fiber relocate project at TH 13 and County 21 in Prior Lake last night starting at 12:01AM. We had some network appliances in Prior Lake go down at 12:01AM and come back up at 1:50AM. I'm not sure if it's related this issue but the timestamps match what you're seeing. Many thanks. Jake -----Original Message----- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of James Urwiller Sent: Friday, August 30, 2019 5:01 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] zayo again? This message may be from an external email source. Do not select links or open attachments unless verified. Report all suspicious emails to Minnesota IT Services Security Operations Center. I have been told they cannot reach a cyan shelf in belle plaine. they also asked me how to gain access to the location and provide a local contact. I tried explaining that it is their hut, not mine and I don't have access...because its not my hut. James Urwiller Network Architect Stealth Broadband james.urwiller(a)stealthbroadband.com -----Original Message----- From: James Urwiller <james.urwiller(a)turnkey-telecom.com> Sent: Friday, August 30, 2019 02:16 To: 'MICE-DISCUSS(a)LISTS.IPHOUSE.NET' <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> Subject: RE: zayo again? My initial down was at 00:14 CST, and came back 01:04 CST. Then back down again 01:20 CST, and then back around 01:43CST. Seems to be stable at the moment. Anyone else experience this? James Urwiller Network Architect Stealth Broadband james.urwiller(a)stealthbroadband.com -----Original Message----- From: James Urwiller <james.urwiller(a)turnkey-telecom.com> Sent: Friday, August 30, 2019 01:37 To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: zayo again? been seeing BGP seesion bounces sine around midnight. anyone else zayo2belleplaine??? Sent from my iPhone 7

Aug. 30, 2019

Re: zayo again?

by Lubitz, Jake (MNIT)

For what's it worth, it sounds like Zayo had a maintenance event for a fiber relocate project at TH 13 and County 21 in Prior Lake last night starting at 12:01AM. We had some network appliances in Prior Lake go down at 12:01AM and come back up at 1:50AM. I'm not sure if it's related this issue but the timestamps match what you're seeing. Many thanks. Jake -----Original Message----- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of James Urwiller Sent: Friday, August 30, 2019 5:01 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] zayo again? This message may be from an external email source. Do not select links or open attachments unless verified. Report all suspicious emails to Minnesota IT Services Security Operations Center. I have been told they cannot reach a cyan shelf in belle plaine. they also asked me how to gain access to the location and provide a local contact. I tried explaining that it is their hut, not mine and I don't have access...because its not my hut. James Urwiller Network Architect Stealth Broadband james.urwiller(a)stealthbroadband.com -----Original Message----- From: James Urwiller <james.urwiller(a)turnkey-telecom.com> Sent: Friday, August 30, 2019 02:16 To: 'MICE-DISCUSS(a)LISTS.IPHOUSE.NET' <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> Subject: RE: zayo again? My initial down was at 00:14 CST, and came back 01:04 CST. Then back down again 01:20 CST, and then back around 01:43CST. Seems to be stable at the moment. Anyone else experience this? James Urwiller Network Architect Stealth Broadband james.urwiller(a)stealthbroadband.com -----Original Message----- From: James Urwiller <james.urwiller(a)turnkey-telecom.com> Sent: Friday, August 30, 2019 01:37 To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: zayo again? been seeing BGP seesion bounces sine around midnight. anyone else zayo2belleplaine??? Sent from my iPhone 7

Aug. 30, 2019