- mice-discuss - lists.micemn.net

Re: Could I have a MICE admin look at the route reflectors?

by Doug McIntyre

On Fri, Sep 06, 2019 at 11:51:41PM +0000, Frank Bulk wrote: >For some reason at 5:38 pm we (AS18883, FiberNet) lost IPv4 and IPv6 peering with both of MICE RR's and Hurricane Electric and it's been unstable since then. > >All our other peers with MICE are fine. > >I didn't change any configuration, even though the "bad AS number" note in some of the error messages would suggest that. We've been getting a lot of these.. Sep 3 18:02:14 micemn-01 bird6: NASN_18883: Received: Connection collision resolution Sep 3 18:02:18 micemn-01 bird6: NASN_18883: Received: Connection collision resolution Sep 3 18:02:23 micemn-01 bird6: NASN_18883: Received: Connection collision resolution Sep 3 18:02:28 micemn-01 bird6: NASN_18883: Received: Connection collision resolution And these 2019-09-07 16:08:15 <RMT> NASN_53347: Error: Bad peer AS: 18883 As well as I suppose the expected 2019-09-07 16:08:16 <RMT> NASN_18883: Received: Administrative shutdown 2019-09-07 16:08:20 <RMT> NASN_18883: Received: Administrative shutdown 2019-09-07 16:08:24 <RMT> NASN_18883: Received: Administrative shutdown If you have 18883 link shutdown. But with the first and second errors, I see you have two AS's, and two IPs coming across Eth10 on the switch. The middle error message seems to imply that the configuration we have setup for AS 53347 on 206.108.255.67 is receiving a peer attempt for 18883 which is configured to talk on 206.108.255.133 on our side. Could these two be mixed up at all? -- Doug McIntyre <merlyn(a)iphouse.net> ~.~ ipHouse ~.~ Network Engineer/Provisioning/Jack of all Trades

Sept. 7, 2019

Re: Could I have a MICE admin look at the route reflectors?

by Frank Bulk

It did not -- still waiting to hear from MICE admins on what the RR saw on its side. Frank -----Original Message----- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Jay Hanke Sent: Saturday, September 7, 2019 10:53 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Could I have a MICE admin look at the route reflectors? Did this get resolved? On Fri, Sep 6, 2019 at 6:52 PM Frank Bulk <fbulk(a)mypremieronline.com> wrote: > > For some reason at 5:38 pm we (AS18883, FiberNet) lost IPv4 and IPv6 peering with both of MICE RR's and Hurricane Electric and it's been unstable since then. > > All our other peers with MICE are fine. > > I didn't change any configuration, even though the "bad AS number" note in some of the error messages would suggest that. > > Any hints would be welcomed. > > I've shutdown those six sessions for now. > > Frank > > Sep 6 17:38:49 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:38:49 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:39:21 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:2 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:39:21 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:2 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:39:28 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:39:28 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.1 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:39:28 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.1 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x000049c3 > Sep 6 17:39:33 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.2 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:39:33 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.2 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:39:36 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.2 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x000049c3 > Sep 6 17:39:39 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 17:40:02 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:40:02 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:40:06 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x0000d063 > Sep 6 17:40:22 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.1 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 17:40:28 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.2 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 17:40:48 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:40:48 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.1 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:41:46 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 17:41:51 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:2 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:41:51 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:2 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:42:00 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:2 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x0000d063 > Sep 6 17:42:33 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.1 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x000049c3 > Sep 6 17:42:34 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.1 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 17:42:44 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:42:44 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:42:44 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x0000d063 > Sep 6 17:43:46 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:2 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 17:44:36 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.2 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established > ... > > Sep 6 17:44:34 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 17:44:39 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 17:46:15 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 17:50:54 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 17:50:57 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:00:44 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 18:00:49 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:04:53 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:11:03 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 18:11:06 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:13:05 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 18:13:13 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:14:37 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 18:14:41 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:15:35 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:23:11 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 18:23:16 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:26:58 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:27:46 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 18:27:48 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:33:58 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 18:34:07 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

Sept. 7, 2019

Re: Could I have a MICE admin look at the route reflectors?

by Jay Hanke

Did this get resolved? On Fri, Sep 6, 2019 at 6:52 PM Frank Bulk <fbulk(a)mypremieronline.com> wrote: > > For some reason at 5:38 pm we (AS18883, FiberNet) lost IPv4 and IPv6 peering with both of MICE RR's and Hurricane Electric and it's been unstable since then. > > All our other peers with MICE are fine. > > I didn't change any configuration, even though the "bad AS number" note in some of the error messages would suggest that. > > Any hints would be welcomed. > > I've shutdown those six sessions for now. > > Frank > > Sep 6 17:38:49 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:38:49 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:39:21 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:2 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:39:21 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:2 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:39:28 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:39:28 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.1 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:39:28 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.1 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x000049c3 > Sep 6 17:39:33 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.2 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:39:33 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.2 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:39:36 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.2 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x000049c3 > Sep 6 17:39:39 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 17:40:02 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:40:02 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:40:06 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x0000d063 > Sep 6 17:40:22 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.1 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 17:40:28 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.2 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 17:40:48 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:40:48 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.1 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:41:46 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 17:41:51 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:2 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:41:51 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:2 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:42:00 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:2 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x0000d063 > Sep 6 17:42:33 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.1 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x000049c3 > Sep 6 17:42:34 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.1 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 17:42:44 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes > Sep 6 17:42:44 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state Established event RecvNotify new state Idle > Sep 6 17:42:44 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x0000d063 > Sep 6 17:43:46 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:2 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 17:44:36 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.2 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established > ... > > Sep 6 17:44:34 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 17:44:39 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 17:46:15 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 17:50:54 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 17:50:57 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:00:44 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 18:00:49 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:04:53 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:11:03 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 18:11:06 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:13:05 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 18:13:13 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:14:37 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 18:14:41 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:15:35 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:23:11 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 18:23:16 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:26:58 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:27:46 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 18:27:48 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established > Sep 6 18:33:58 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state Established event RecvNotify new state Idle > Sep 6 18:34:07 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

Sept. 7, 2019

Re: zayo again?

by Frank Bulk

Sorry for the delayed response to this thread. I did get the maintenance ... see attached. Frank -----Original Message----- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Mike Johnston Sent: Friday, August 30, 2019 10:03 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] zayo again? On 2019-08-30 05:00, James Urwiller wrote: > they also asked me how to gain access to the location and provide a > local contact. I tried explaining that it is their hut, not mine and I > don't have access...because its not my hut. Classic. That, unfortunately, sounds normal for Zayo.

Sept. 6, 2019

Could I have a MICE admin look at the route reflectors?

by Frank Bulk

For some reason at 5:38 pm we (AS18883, FiberNet) lost IPv4 and IPv6 peering with both of MICE RR's and Hurricane Electric and it's been unstable since then. All our other peers with MICE are fine. I didn't change any configuration, even though the "bad AS number" note in some of the error messages would suggest that. Any hints would be welcomed. I've shutdown those six sessions for now. Frank Sep 6 17:38:49 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes Sep 6 17:38:49 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state Established event RecvNotify new state Idle Sep 6 17:39:21 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:2 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes Sep 6 17:39:21 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:2 (AS 53679) old state Established event RecvNotify new state Idle Sep 6 17:39:28 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes Sep 6 17:39:28 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.1 (AS 53679) old state Established event RecvNotify new state Idle Sep 6 17:39:28 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.1 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x000049c3 Sep 6 17:39:33 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.2 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes Sep 6 17:39:33 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.2 (AS 53679) old state Established event RecvNotify new state Idle Sep 6 17:39:36 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.2 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x000049c3 Sep 6 17:39:39 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 17:40:02 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes Sep 6 17:40:02 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state Established event RecvNotify new state Idle Sep 6 17:40:06 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x0000d063 Sep 6 17:40:22 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.1 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 17:40:28 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.2 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 17:40:48 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes Sep 6 17:40:48 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.1 (AS 53679) old state Established event RecvNotify new state Idle Sep 6 17:41:46 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 17:41:51 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:2 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes Sep 6 17:41:51 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:2 (AS 53679) old state Established event RecvNotify new state Idle Sep 6 17:42:00 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:2 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x0000d063 Sep 6 17:42:33 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 206.108.255.1 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x000049c3 Sep 6 17:42:34 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.1 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 17:42:44 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes Sep 6 17:42:44 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:1 (AS 53679) old state Established event RecvNotify new state Idle Sep 6 17:42:44 207.32.15.28 Rib: %BGP-3-NOTIFICATION: received from neighbor 2001:504:27::d1af:0:1 (AS 53679) 2/2 (Open Message Error/bad AS number) 4 bytes 0x0000d063 Sep 6 17:43:46 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::d1af:0:2 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 17:44:36 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.2 (AS 53679) old state OpenConfirm event RecvKeepAlive new state Established ... Sep 6 17:44:34 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle Sep 6 17:44:39 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 17:46:15 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle Sep 6 17:50:54 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle Sep 6 17:50:57 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 18:00:44 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state Established event RecvNotify new state Idle Sep 6 18:00:49 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 18:04:53 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 18:11:03 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle Sep 6 18:11:06 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 18:13:05 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle Sep 6 18:13:13 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 18:14:37 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state Established event RecvNotify new state Idle Sep 6 18:14:41 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 18:15:35 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 18:23:11 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state Established event RecvNotify new state Idle Sep 6 18:23:16 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 18:26:58 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 18:27:46 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state Established event RecvNotify new state Idle Sep 6 18:27:48 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::1b1b:0:1 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established Sep 6 18:33:58 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state Established event RecvNotify new state Idle Sep 6 18:34:07 207.32.15.28 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.52 (AS 6939) old state OpenConfirm event RecvKeepAlive new state Established

Sept. 6, 2019

US Internet - AS 10242 - Router Replacement

by Justin Krejci

MICE, US Internet is shutting down BGP sessions for a few minutes to migrate the MICE connection onto a new router. BGP sessions will be re-established within a few minutes. This will be happening in the next few minutes after this email. Justin Krejci 952-253-3200 ? 1-800-US-INTERNET 12450 Wayzata Blvd. Suite 315 ? Minnetonka, MN 55305 [http://www.usinternet.com/img/usi-signature.png]

Sept. 6, 2019

Re: zayo again?

by Mike Johnston

On 2019-08-30 05:00, James Urwiller wrote: > they also asked me how to gain access to the location and provide a > local contact. I tried explaining that it is their hut, not mine and I > don't have access...because its not my hut. Classic. That, unfortunately, sounds normal for Zayo.

Aug. 30, 2019

Re: zayo again?

by James Urwiller

Yes that is what I am learning - we were not notified of the maintenance and no one at zayo can find the maintenance ticket either.. we are back up and running for now. James Urwiller Network Architect Stealth Broadband james.urwiller(a)stealthbroadband.com -----Original Message----- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Lubitz, Jake (MNIT) Sent: Friday, August 30, 2019 08:46 To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] zayo again? For what's it worth, it sounds like Zayo had a maintenance event for a fiber relocate project at TH 13 and County 21 in Prior Lake last night starting at 12:01AM. We had some network appliances in Prior Lake go down at 12:01AM and come back up at 1:50AM. I'm not sure if it's related this issue but the timestamps match what you're seeing. Many thanks. Jake -----Original Message----- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of James Urwiller Sent: Friday, August 30, 2019 5:01 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] zayo again? This message may be from an external email source. Do not select links or open attachments unless verified. Report all suspicious emails to Minnesota IT Services Security Operations Center. I have been told they cannot reach a cyan shelf in belle plaine. they also asked me how to gain access to the location and provide a local contact. I tried explaining that it is their hut, not mine and I don't have access...because its not my hut. James Urwiller Network Architect Stealth Broadband james.urwiller(a)stealthbroadband.com -----Original Message----- From: James Urwiller <james.urwiller(a)turnkey-telecom.com> Sent: Friday, August 30, 2019 02:16 To: 'MICE-DISCUSS(a)LISTS.IPHOUSE.NET' <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> Subject: RE: zayo again? My initial down was at 00:14 CST, and came back 01:04 CST. Then back down again 01:20 CST, and then back around 01:43CST. Seems to be stable at the moment. Anyone else experience this? James Urwiller Network Architect Stealth Broadband james.urwiller(a)stealthbroadband.com -----Original Message----- From: James Urwiller <james.urwiller(a)turnkey-telecom.com> Sent: Friday, August 30, 2019 01:37 To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: zayo again? been seeing BGP seesion bounces sine around midnight. anyone else zayo2belleplaine??? Sent from my iPhone 7

Aug. 30, 2019

Re: zayo again?

by Lubitz, Jake (MNIT)

For what's it worth, it sounds like Zayo had a maintenance event for a fiber relocate project at TH 13 and County 21 in Prior Lake last night starting at 12:01AM. We had some network appliances in Prior Lake go down at 12:01AM and come back up at 1:50AM. I'm not sure if it's related this issue but the timestamps match what you're seeing. Many thanks. Jake -----Original Message----- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of James Urwiller Sent: Friday, August 30, 2019 5:01 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] zayo again? This message may be from an external email source. Do not select links or open attachments unless verified. Report all suspicious emails to Minnesota IT Services Security Operations Center. I have been told they cannot reach a cyan shelf in belle plaine. they also asked me how to gain access to the location and provide a local contact. I tried explaining that it is their hut, not mine and I don't have access...because its not my hut. James Urwiller Network Architect Stealth Broadband james.urwiller(a)stealthbroadband.com -----Original Message----- From: James Urwiller <james.urwiller(a)turnkey-telecom.com> Sent: Friday, August 30, 2019 02:16 To: 'MICE-DISCUSS(a)LISTS.IPHOUSE.NET' <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> Subject: RE: zayo again? My initial down was at 00:14 CST, and came back 01:04 CST. Then back down again 01:20 CST, and then back around 01:43CST. Seems to be stable at the moment. Anyone else experience this? James Urwiller Network Architect Stealth Broadband james.urwiller(a)stealthbroadband.com -----Original Message----- From: James Urwiller <james.urwiller(a)turnkey-telecom.com> Sent: Friday, August 30, 2019 01:37 To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: zayo again? been seeing BGP seesion bounces sine around midnight. anyone else zayo2belleplaine??? Sent from my iPhone 7

Aug. 30, 2019

Re: zayo again?

by James Urwiller

I have been told they cannot reach a cyan shelf in belle plaine. they also asked me how to gain access to the location and provide a local contact. I tried explaining that it is their hut, not mine and I don't have access...because its not my hut. James Urwiller Network Architect Stealth Broadband james.urwiller(a)stealthbroadband.com -----Original Message----- From: James Urwiller <james.urwiller(a)turnkey-telecom.com> Sent: Friday, August 30, 2019 02:16 To: 'MICE-DISCUSS(a)LISTS.IPHOUSE.NET' <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> Subject: RE: zayo again? My initial down was at 00:14 CST, and came back 01:04 CST. Then back down again 01:20 CST, and then back around 01:43CST. Seems to be stable at the moment. Anyone else experience this? James Urwiller Network Architect Stealth Broadband james.urwiller(a)stealthbroadband.com -----Original Message----- From: James Urwiller <james.urwiller(a)turnkey-telecom.com> Sent: Friday, August 30, 2019 01:37 To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: zayo again? been seeing BGP seesion bounces sine around midnight. anyone else zayo2belleplaine??? Sent from my iPhone 7

Aug. 30, 2019

Re: zayo again?

by James Urwiller

My initial down was at 00:14 CST, and came back 01:04 CST. Then back down again 01:20 CST, and then back around 01:43CST. Seems to be stable at the moment. Anyone else experience this? James Urwiller Network Architect Stealth Broadband james.urwiller(a)stealthbroadband.com -----Original Message----- From: James Urwiller <james.urwiller(a)turnkey-telecom.com> Sent: Friday, August 30, 2019 01:37 To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: zayo again? been seeing BGP seesion bounces sine around midnight. anyone else zayo2belleplaine??? Sent from my iPhone 7

Aug. 30, 2019

zayo again?

by James Urwiller

been seeing BGP seesion bounces sine around midnight. anyone else zayo2belleplaine??? Sent from my iPhone 7

Aug. 30, 2019

Re: Of minor note

by Bahls, Dean

Yeah, I forgot to notify the group when we did a software update this morning. Sorry about that. Dean On Mon, Aug 26, 2019 at 8:33 PM Frank Bulk <fbulk(a)mypremieronline.com> wrote: > FYI, saw it again this morning: > > Aug 26 05:42:02 207.32.15.28 Rib: mpbgp_recv_mpreach: peer > 2001:504:27::d1af:0:1 (AS 53679) link-local next hop > fe80::e2ac:f1ef:fe4b:d93b%10000 improper, ignoring this next-hop > Aug 26 05:42:02 207.32.15.28 Rib: mpbgp_recv_mpreach: peer > 2001:504:27::d1af:0:2 (AS 53679) link-local next hop > fe80::e2ac:f1ef:fe4b:d93b%10000 improper, ignoring this next-hop > > Frank > > -----Original Message----- > From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Frank > Bulk > Sent: Wednesday, August 21, 2019 8:00 AM > To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET > Subject: [MICE-DISCUSS] Of minor note > > Our route logged these two items Monday morning with MICE's AS: > > Aug 19 10:35:41 207.32.15.28 Rib: mpbgp_recv_mpreach: peer > 2001:504:27::d1af:0:2 (AS 53679) link-local next hop > fe80::e2ac:f1ef:fe4b:d93b%10000 improper, ignoring this next-hop > Aug 19 11:08:50 207.32.15.28 Rib: mpbgp_recv_mpreach: peer > 2001:504:27::d1af:0:2 (AS 53679) link-local next hop > fe80::e2ac:f1ef:fe4b:d93b%10000 improper, ignoring this next-hop > > Not sure what that was about. > > Frank > -- *Dean Bahls* Network Services Manager | Cooperative Network Services, LLC 14 Main St SW | P.O. Box J | Menahga, MN 56464 218.346.3635 (Direct) | 218.564.3000 (Main Office) dean.bahls(a)cooperative-networks.com

Aug. 27, 2019

Re: Of minor note

by Frank Bulk

FYI, saw it again this morning: Aug 26 05:42:02 207.32.15.28 Rib: mpbgp_recv_mpreach: peer 2001:504:27::d1af:0:1 (AS 53679) link-local next hop fe80::e2ac:f1ef:fe4b:d93b%10000 improper, ignoring this next-hop Aug 26 05:42:02 207.32.15.28 Rib: mpbgp_recv_mpreach: peer 2001:504:27::d1af:0:2 (AS 53679) link-local next hop fe80::e2ac:f1ef:fe4b:d93b%10000 improper, ignoring this next-hop Frank -----Original Message----- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Frank Bulk Sent: Wednesday, August 21, 2019 8:00 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] Of minor note Our route logged these two items Monday morning with MICE's AS: Aug 19 10:35:41 207.32.15.28 Rib: mpbgp_recv_mpreach: peer 2001:504:27::d1af:0:2 (AS 53679) link-local next hop fe80::e2ac:f1ef:fe4b:d93b%10000 improper, ignoring this next-hop Aug 19 11:08:50 207.32.15.28 Rib: mpbgp_recv_mpreach: peer 2001:504:27::d1af:0:2 (AS 53679) link-local next hop fe80::e2ac:f1ef:fe4b:d93b%10000 improper, ignoring this next-hop Not sure what that was about. Frank

Aug. 27, 2019

Re: ADV: [MICE-DISCUSS] Can a PTR be created for Google IP?

by Doug McIntyre

On Fri, Aug 23, 2019 at 04:45:35AM +0000, Frank Bulk wrote: >Can a PTR be created for this IP: 206.108.255.141 (AS 15169: Google LLC, US) Sure, I've setup the newest batch of IPv4/IPv6 reverses for the MICE block. -- Doug McIntyre <merlyn(a)iphouse.net> ~.~ ipHouse ~.~ Network Engineer/Provisioning/Jack of all Trades

Aug. 23, 2019

Can a PTR be created for Google IP?

by Frank Bulk

Can a PTR be created for this IP: 206.108.255.141 (AS 15169: Google LLC, US) Frank

Aug. 23, 2019

Re: Saturated ports

by Jay Hanke

> If someone's port is full on a regular basis, having someone from the Board contact the member and ask if they are aware of the issue seems approprate and reasonable. Basically asking, "have you noticed your port is full quite a bit lately?" But its the member's responsibility to manage their network as they see fit, including their ports to the exchange. I'd tweak this a little to encourage the member reach out directly to the other peer whenever there might be a problem. If you need help getting attention of a non-responsive peer we can help with some "peer pressure". :) -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

Aug. 22, 2019

Re: Saturated ports

by David Farmer

On Wed, Aug 21, 2019 at 1:59 PM Jay Hanke <jayhanke(a)southfront.io> wrote: > > To be clear, I was not advocating for punitive action, and we can go and > pull the bandwidth reports periodically. I was just posing the question > regarding whether a more proactive stance regarding visibility would be > appropriate. > > What would you propose? > First, let's not make a federal case of this, links fill up it happens, get over it. In this case it occurred right after big changes by a big content provider. I'm actually surprised more participants' links didn't fill up. Second, you should send a polite note to any another member, if you notice anything that isn't quite right, their link is full, they are leaking routes, or something just looks weird. We are a community, talk to your neighbors, be their early warning system. Hopefully, they already know about what ever it is, but we are all busy and things slip through the cracks all the time. So if you notice something that doesn't seem right, let your neighbors know about it. But be polite about it, next time it might be you who has something weird happening. Third, the IX is an optimization, if you are having issues sending traffic to another member you have other routes, use them. We have had to do this, not because the other member ignored us or wouldn't upgrade their link, but sometimes upgrades can take a little time. A few weeks or even a couple months to complete an upgrade isn't that unrealistic and not uncommon depending on the circumstances. Not everyone has extra 10G or 100G ports just sitting around, and if you have to backhaul the traffic, arranging that upgrade can easily take several months. The policy we have for remote switches makes sense, but members need to manage their own capacity. Even if we were to forced them to add port capacity to the exchange the bottleneck could just move to the next hop. None of us has sufficient visibility into the other guys network or business to be making those kinds of judgements for them. If someone's port is full on a regular basis, having someone from the Board contact the member and ask if they are aware of the issue seems approprate and reasonable. Basically asking, "have you noticed your port is full quite a bit lately?" But its the member's responsibility to manage their network as they see fit, including their ports to the exchange. Thanks -- =============================================== David Farmer Email:farmer@umn.edu Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 ===============================================

Aug. 22, 2019

Re: Saturated ports

by Corey Hauer

Indeed I did! 😮 On Wed, Aug 21, 2019 at 2:00 PM Matthew Beckwell <mbeckwell(a)evolveip.net> wrote: > I feel like we’re mixing up “SDN” and “DCN” here? > > > > > > *From: *MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> on behalf of Corey > Hauer <coreyhauer(a)GMAIL.COM> > *Reply-To: *MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> > *Date: *Wednesday, August 21, 2019 at 3:10 PM > *To: *"MICE-DISCUSS(a)LISTS.IPHOUSE.NET" <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> > *Subject: *Re: [MICE-DISCUSS] Saturated ports > > > > Honestly, as an eyeball network, I would fix saturating that pipe that by > filtering prefixes or adding capacity. My customers would vote with their > feet if we started having packet loss or adverse latency because we filled > our IX facing pipes. I can't imagine SDN and their members choosing to > endure thousands of customer complaints. > > > > SDN is a well run, well capitalized network - they'll fix it one way or > another. I am sure once this lands on the right person's desk - it gets > fixed. > > > > > > On Wed, Aug 21, 2019 at 12:17 PM Ben Wiechman <ben.wiechman(a)arvig.com> > wrote: > > Is it possible to send a weekly/monthly reporting showing a high water > mark for utilization for each member? Is that appropriate? Or do we just > ask each member to monitor the utilization graphs if this is of interest to > them? > > > > For my part it is easier to ingest and handle a periodic review if we are > pushed the information versus having to have someone remember to go and > pull the information. I'm interested in what others would find useful. > > > > *Ben Wiechman* > > Director of IP Strategy and Engineering > > 320.247.3224 | ben.wiechman(a)arvig.com > > Arvig | 224 East Main Street | Melrose, MN 56352 > <https://www.google.com/maps/search/224+East+Main+Street+%7C+Melrose,+MN+563…> > | arvig.com > > > > > > > > On Wed, Aug 21, 2019 at 1:59 PM Jay Hanke <jayhanke(a)southfront.io> wrote: > > > To be clear, I was not advocating for punitive action, and we can go and > pull the bandwidth reports periodically. I was just posing the question > regarding whether a more proactive stance regarding visibility would be > appropriate. > > What would you propose? > > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

Aug. 21, 2019

Re: Saturated ports

by Darin Steffl

My original thread is about DCN, not SDN. On Wed, Aug 21, 2019, 4:00 PM Matthew Beckwell <mbeckwell(a)evolveip.net> wrote: > I feel like we’re mixing up “SDN” and “DCN” here? > > > > > > *From: *MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> on behalf of Corey > Hauer <coreyhauer(a)GMAIL.COM> > *Reply-To: *MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> > *Date: *Wednesday, August 21, 2019 at 3:10 PM > *To: *"MICE-DISCUSS(a)LISTS.IPHOUSE.NET" <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> > *Subject: *Re: [MICE-DISCUSS] Saturated ports > > > > Honestly, as an eyeball network, I would fix saturating that pipe that by > filtering prefixes or adding capacity. My customers would vote with their > feet if we started having packet loss or adverse latency because we filled > our IX facing pipes. I can't imagine SDN and their members choosing to > endure thousands of customer complaints. > > > > SDN is a well run, well capitalized network - they'll fix it one way or > another. I am sure once this lands on the right person's desk - it gets > fixed. > > > > > > On Wed, Aug 21, 2019 at 12:17 PM Ben Wiechman <ben.wiechman(a)arvig.com> > wrote: > > Is it possible to send a weekly/monthly reporting showing a high water > mark for utilization for each member? Is that appropriate? Or do we just > ask each member to monitor the utilization graphs if this is of interest to > them? > > > > For my part it is easier to ingest and handle a periodic review if we are > pushed the information versus having to have someone remember to go and > pull the information. I'm interested in what others would find useful. > > > > *Ben Wiechman* > > Director of IP Strategy and Engineering > > 320.247.3224 | ben.wiechman(a)arvig.com > > Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com > > > > > > > > On Wed, Aug 21, 2019 at 1:59 PM Jay Hanke <jayhanke(a)southfront.io> wrote: > > > To be clear, I was not advocating for punitive action, and we can go and > pull the bandwidth reports periodically. I was just posing the question > regarding whether a more proactive stance regarding visibility would be > appropriate. > > What would you propose? > > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

Aug. 21, 2019

Re: Saturated ports

by AnthonyAnderberg＠nuvera.net

I think there is some confusion – the original thread was talking about DCN, not SDN. SDN is a long-time MICE member and hasn’t ever had any issues that I recall. From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> on behalf of Gary Glissendorf <Gary.Glissendorf(a)SDNCOMMUNICATIONS.COM> Reply-To: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> Date: Wednesday, August 21, 2019 at 3:45 PM To: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Saturated ports I noticed this thread today but wasn’t sure what topic started the thread. SDN like Corey said our customers report latency, packet loss quickly if we have saturation. We monitor and alert on network saturation at 60% and if needed we adjust prefix advertisements until we can find the proper balance and then increase bandwidth when needed. We have not developed a cool automatic was to do this yet but we rarely have issues with Peaks or DDoS Attacks. It’s the occasional fiber cuts that can cause us issues and the need to adjust bandwidth. What was topic that started the Thread about BW and reports. Gary Glissendorf | Network Architect gary.glissendorf(a)sdncommunications.com 2900 W. 10th St. | Sioux Falls, SD 57104 (w) 605.978.3558 | (c) 605.359-3737 | (tf) 800.247.1442 SDN NOC 877.287.8023 NOC Support email: sdnsupport(a)sdncommunications.com “Ausgezeichnet Zueinander Sein” [cid:image001.jpg@01D55838.820116E0] <http://www.facebook.com/sdncommunications> [cid:image002.jpg@01D55838.820116E0] <http://twitter.com/sdncomm> [cid:image003.jpg@01D55838.820116E0] <http://plus.google.com/107192460129695945716/posts> [cid:image004.jpg@01D55838.820116E0] <http://www.linkedin.com/company/sdn-communications> [cid:image005.jpg@01D55838.820116E0] <http://www.youtube.com/user/sdncomm> From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Corey Hauer Sent: Wednesday, August 21, 2019 3:10 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Saturated ports Honestly, as an eyeball network, I would fix saturating that pipe that by filtering prefixes or adding capacity. My customers would vote with their feet if we started having packet loss or adverse latency because we filled our IX facing pipes. I can't imagine SDN and their members choosing to endure thousands of customer complaints. SDN is a well run, well capitalized network - they'll fix it one way or another. I am sure once this lands on the right person's desk - it gets fixed. On Wed, Aug 21, 2019 at 12:17 PM Ben Wiechman <ben.wiechman(a)arvig.com<mailto:ben.wiechman@arvig.com>> wrote: Is it possible to send a weekly/monthly reporting showing a high water mark for utilization for each member? Is that appropriate? Or do we just ask each member to monitor the utilization graphs if this is of interest to them? For my part it is easier to ingest and handle a periodic review if we are pushed the information versus having to have someone remember to go and pull the information. I'm interested in what others would find useful. Ben Wiechman Director of IP Strategy and Engineering 320.247.3224 | ben.wiechman(a)arvig.com<mailto:ben.wiechman@arvig.com> Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com<http://arvig.com> On Wed, Aug 21, 2019 at 1:59 PM Jay Hanke <jayhanke(a)southfront.io<mailto:jayhanke@southfront.io>> wrote: > To be clear, I was not advocating for punitive action, and we can go and pull the bandwidth reports periodically. I was just posing the question regarding whether a more proactive stance regarding visibility would be appropriate. What would you propose? ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ***This message and any attachments are solely for the intended recipient. If you are not the intended recipient, disclosure, copying, use or distribution of the information included in this message is prohibited -- Please immediately and permanently delete.*** ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Aug. 21, 2019

Re: Saturated ports

by Matthew Beckwell

I feel like we’re mixing up “SDN” and “DCN” here? From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> on behalf of Corey Hauer <coreyhauer(a)GMAIL.COM> Reply-To: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> Date: Wednesday, August 21, 2019 at 3:10 PM To: "MICE-DISCUSS(a)LISTS.IPHOUSE.NET" <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Saturated ports Honestly, as an eyeball network, I would fix saturating that pipe that by filtering prefixes or adding capacity. My customers would vote with their feet if we started having packet loss or adverse latency because we filled our IX facing pipes. I can't imagine SDN and their members choosing to endure thousands of customer complaints. SDN is a well run, well capitalized network - they'll fix it one way or another. I am sure once this lands on the right person's desk - it gets fixed. On Wed, Aug 21, 2019 at 12:17 PM Ben Wiechman <ben.wiechman(a)arvig.com<mailto:ben.wiechman@arvig.com>> wrote: Is it possible to send a weekly/monthly reporting showing a high water mark for utilization for each member? Is that appropriate? Or do we just ask each member to monitor the utilization graphs if this is of interest to them? For my part it is easier to ingest and handle a periodic review if we are pushed the information versus having to have someone remember to go and pull the information. I'm interested in what others would find useful. Ben Wiechman Director of IP Strategy and Engineering 320.247.3224 | ben.wiechman(a)arvig.com<mailto:ben.wiechman@arvig.com> Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com<http://arvig.com> On Wed, Aug 21, 2019 at 1:59 PM Jay Hanke <jayhanke(a)southfront.io<mailto:jayhanke@southfront.io>> wrote: > To be clear, I was not advocating for punitive action, and we can go and pull the bandwidth reports periodically. I was just posing the question regarding whether a more proactive stance regarding visibility would be appropriate. What would you propose? ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Aug. 21, 2019

Re: Saturated ports

by Gary Glissendorf

I noticed this thread today but wasn’t sure what topic started the thread. SDN like Corey said our customers report latency, packet loss quickly if we have saturation. We monitor and alert on network saturation at 60% and if needed we adjust prefix advertisements until we can find the proper balance and then increase bandwidth when needed. We have not developed a cool automatic was to do this yet but we rarely have issues with Peaks or DDoS Attacks. It’s the occasional fiber cuts that can cause us issues and the need to adjust bandwidth. What was topic that started the Thread about BW and reports. Gary Glissendorf | Network Architect gary.glissendorf(a)sdncommunications.com 2900 W. 10th St. | Sioux Falls, SD 57104 (w) 605.978.3558 | (c) 605.359-3737 | (tf) 800.247.1442 SDN NOC 877.287.8023 NOC Support email: sdnsupport(a)sdncommunications.com “Ausgezeichnet Zueinander Sein” [cid:image001.jpg@01D55837.7362FB90] <http://www.facebook.com/sdncommunications> [cid:image002.jpg@01D55837.7362FB90] <http://twitter.com/sdncomm> [cid:image003.jpg@01D55837.7362FB90] <http://plus.google.com/107192460129695945716/posts> [cid:image004.jpg@01D55837.7362FB90] <http://www.linkedin.com/company/sdn-communications> [cid:image005.jpg@01D55837.7362FB90] <http://www.youtube.com/user/sdncomm> From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Corey Hauer Sent: Wednesday, August 21, 2019 3:10 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Saturated ports Honestly, as an eyeball network, I would fix saturating that pipe that by filtering prefixes or adding capacity. My customers would vote with their feet if we started having packet loss or adverse latency because we filled our IX facing pipes. I can't imagine SDN and their members choosing to endure thousands of customer complaints. SDN is a well run, well capitalized network - they'll fix it one way or another. I am sure once this lands on the right person's desk - it gets fixed. On Wed, Aug 21, 2019 at 12:17 PM Ben Wiechman <ben.wiechman(a)arvig.com<mailto:ben.wiechman@arvig.com>> wrote: Is it possible to send a weekly/monthly reporting showing a high water mark for utilization for each member? Is that appropriate? Or do we just ask each member to monitor the utilization graphs if this is of interest to them? For my part it is easier to ingest and handle a periodic review if we are pushed the information versus having to have someone remember to go and pull the information. I'm interested in what others would find useful. Ben Wiechman Director of IP Strategy and Engineering 320.247.3224 | ben.wiechman(a)arvig.com<mailto:ben.wiechman@arvig.com> Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com<http://arvig.com> On Wed, Aug 21, 2019 at 1:59 PM Jay Hanke <jayhanke(a)southfront.io<mailto:jayhanke@southfront.io>> wrote: > To be clear, I was not advocating for punitive action, and we can go and pull the bandwidth reports periodically. I was just posing the question regarding whether a more proactive stance regarding visibility would be appropriate. What would you propose? ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ***This message and any attachments are solely for the intended recipient. If you are not the intended recipient, disclosure, copying, use or distribution of the information included in this message is prohibited -- Please immediately and permanently delete.***

Aug. 21, 2019

Re: Saturated ports

by Corey Hauer

Honestly, as an eyeball network, I would fix saturating that pipe that by filtering prefixes or adding capacity. My customers would vote with their feet if we started having packet loss or adverse latency because we filled our IX facing pipes. I can't imagine SDN and their members choosing to endure thousands of customer complaints. SDN is a well run, well capitalized network - they'll fix it one way or another. I am sure once this lands on the right person's desk - it gets fixed. On Wed, Aug 21, 2019 at 12:17 PM Ben Wiechman <ben.wiechman(a)arvig.com> wrote: > Is it possible to send a weekly/monthly reporting showing a high water > mark for utilization for each member? Is that appropriate? Or do we just > ask each member to monitor the utilization graphs if this is of interest to > them? > > For my part it is easier to ingest and handle a periodic review if we are > pushed the information versus having to have someone remember to go and > pull the information. I'm interested in what others would find useful. > > Ben Wiechman > > Director of IP Strategy and Engineering > > 320.247.3224 | ben.wiechman(a)arvig.com > > Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com > > > > > On Wed, Aug 21, 2019 at 1:59 PM Jay Hanke <jayhanke(a)southfront.io> wrote: > >> > To be clear, I was not advocating for punitive action, and we can go >> and pull the bandwidth reports periodically. I was just posing the question >> regarding whether a more proactive stance regarding visibility would be >> appropriate. >> >> What would you propose? >> > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

Aug. 21, 2019

Re: Saturated ports

by Andy Koch

On 8/21/19 2:17 PM, Ben Wiechman wrote: > Is it possible to send a weekly/monthly reporting showing a high water > mark for utilization for each member? Is that appropriate? Or do we > just ask each member to monitor the utilization graphs if this is of > interest to them? > > For my part it is easier to ingest and handle a periodic review if we > are pushed the information versus having to have someone remember to > go and pull the information. I'm interested in what others would find > useful. I don't mind if this is an opt-in item on a per-participant basis, but I am not in-favor of having yet more to my inbox. I feel that any good participant should have their own monitoring and be properly alerted based on that - possibly directly into a capacity planning workflow. If you want to have a periodic email, I am sure a curl request to the MICE monitoring server on a cron would be fairly simple to setup if you have cleared it with US Internet who graciously hosts that for us. If the recommendation is for a mass-email to the mailing list with all participant data, that I am even less in support of. We have several participants that do not publicly share their capacity data and we should respect that - not blast it out weekly/monthly, even if the recipient list is opt-in. Regards, Andy Andy Koch Hoyos Consulting LLC ofc: +1 608 616 9950 andyk(a)hoyosconsulting.com http://www.hoyosconsulting.com

Aug. 21, 2019