December 2019 - mice-discuss - lists.micemn.net

Re: Route reflector check

by Frank Bulk

Mike, It's possible that it's a single prefix that's setting QRator off. I just sent you a copy of our MICE-facing routing table for AS 2381 and we'll see I can jump on the next time I get a QRator alert to compare. I checked for 184.164.25[0-2].0/24 and I don't see it right now. Thanks! Frank From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Michael Hare Sent: Monday, December 30, 2019 3:59 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Route reflector check Hi everyone- Wanted to add a bit more detail to what we (AS 3128) checked on our side. Specifically, 1) our peering session(s) to the RS seems stable from an uptime perspective. 2) I don't see a large delta in outbound bgp messages from us to MICE. 3) we don't run BMP so I don't actually track outbound changes at a level that would conclusive prove what's going on. Frank, is it possible you are being notified about a single route flapping? I did check AS2381's looking glass, specifically a single route (143.235.0.0/16) and it looks stable. Admittedly I didn't look for LG across all MICE members but I'm not sure if any would allow me to run an as-path regexp to see if there is a specific flapping route that is flapping. If someone can help out on that front it could be useful. [output from AS2381 looking glass] [BGP/170] 1w6d 13:08:45, MED 10, localpref 200, from 206.108.255.1 AS path: 3128 3128 3128 3128 I, validation-state: unverified > to 206.108.255.66 via xe-0/1/5.300 [BGP/170] 3w3d 14:51:19, MED 10, localpref 200, from 206.108.255.2 AS path: 3128 3128 3128 3128 I, validation-state: unverified > to 206.108.255.66 via xe-0/1/5.300 Thanks- -Michael From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET>> On Behalf Of Frank Bulk Sent: Monday, December 30, 2019 3:37 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: [MICE-DISCUSS] Route reflector check QRator has been alerting me that AS 3128 is coming and going. It just happened again and wondering if anyone else is seeing this behavior and also how things look on MICE's route reflectors. I already contacted the good folks at the University of Wisconsin and they're not seeing anything on their side. AS Number: AS3128 AS Name: # First seen Last seen 1 2019-12-29 21:49:00 2019-12-29 21:50:00 2 2019-12-29 18:42:00 2019-12-29 18:43:00 3 2019-12-29 17:57:00 2019-12-29 17:58:00 4 2019-12-29 16:27:00 2019-12-29 16:28:00 5 2019-12-29 14:12:00 2019-12-29 14:13:00 6 2019-12-29 13:27:00 2019-12-29 13:28:00 7 2019-12-29 12:42:00 2019-12-29 12:43:00 8 2019-12-29 11:57:00 2019-12-29 11:58:00 9 2019-12-29 11:12:00 2019-12-29 11:13:00 10 2019-12-29 11:04:00 2019-12-29 11:05:00 11 2019-12-29 10:27:00 2019-12-29 10:28:00 12 2019-12-29 08:57:00 2019-12-29 08:58:00 13 2019-12-29 08:12:00 2019-12-29 08:13:00 14 2019-12-29 07:53:00 2019-12-29 07:54:00 15 2019-12-29 07:27:00 2019-12-29 07:28:00 16 2019-12-29 06:42:00 2019-12-29 06:43:00 17 2019-12-29 05:57:00 2019-12-29 05:58:00 18 2019-12-29 05:12:00 2019-12-29 05:13:00 19 2019-12-29 04:40:00 2019-12-29 04:41:00 20 2019-12-29 04:27:00 2019-12-29 04:28:00 21 2019-12-29 00:05:00 2019-12-29 00:06:00 22 2019-12-28 23:57:00 2019-12-28 23:58:00 23 2019-12-28 23:12:00 2019-12-28 23:13:00 24 2019-12-28 23:04:00 2019-12-28 23:05:00 25 2019-12-28 22:27:00 2019-12-28 22:28:00 26 2019-12-28 21:42:00 2019-12-28 21:43:00 27 2019-12-28 20:57:00 2019-12-28 20:58:00 28 2019-12-28 20:12:00 2019-12-28 20:13:00 29 2019-12-28 07:27:00 2019-12-28 07:28:00 30 2019-12-28 06:42:00 2019-12-28 06:43:00 31 2019-12-28 05:57:00 2019-12-28 05:58:00 32 2019-12-28 05:12:00 2019-12-28 05:13:00 33 2019-12-27 17:21:00 2019-12-27 17:22:00 34 2019-12-06 07:01:00 2019-12-27 06:10:00 Frank ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.iphouse.net_cgi-2…> ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.iphouse.net_cgi-2…>

Dec. 31, 2019

Re: Route reflector check

by Michael Hare

Frank- I may have found the answer to your question. Looking for tips on better ways to do this, but I did this. "show route community $ourCommunityForRoutesWeExport detail | match Age" and did a mix of lame exclusions and visual greps to find a few routes with an age of less than 60 minutes. specifically 184.164.250.0/24 *[BGP/170] 00:44:15 184.164.251.0/24 *[BGP/170] 00:44:15 184.164.252.0/24 *[BGP/170] 00:44:15 These are part of an anycast R&E project. We receive this over an eBGP peering with the project. It looks like they are setting community 47065:184 if you want to filter. Frank, how new is your detection software? This is an old project and I would call this normal behavior for them. -Michael From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Michael Hare Sent: Monday, December 30, 2019 3:59 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Route reflector check Hi everyone- Wanted to add a bit more detail to what we (AS 3128) checked on our side. Specifically, 1) our peering session(s) to the RS seems stable from an uptime perspective. 2) I don't see a large delta in outbound bgp messages from us to MICE. 3) we don't run BMP so I don't actually track outbound changes at a level that would conclusive prove what's going on. Frank, is it possible you are being notified about a single route flapping? I did check AS2381's looking glass, specifically a single route (143.235.0.0/16) and it looks stable. Admittedly I didn't look for LG across all MICE members but I'm not sure if any would allow me to run an as-path regexp to see if there is a specific flapping route that is flapping. If someone can help out on that front it could be useful. [output from AS2381 looking glass] [BGP/170] 1w6d 13:08:45, MED 10, localpref 200, from 206.108.255.1 AS path: 3128 3128 3128 3128 I, validation-state: unverified > to 206.108.255.66 via xe-0/1/5.300 [BGP/170] 3w3d 14:51:19, MED 10, localpref 200, from 206.108.255.2 AS path: 3128 3128 3128 3128 I, validation-state: unverified > to 206.108.255.66 via xe-0/1/5.300 Thanks- -Michael From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET>> On Behalf Of Frank Bulk Sent: Monday, December 30, 2019 3:37 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: [MICE-DISCUSS] Route reflector check QRator has been alerting me that AS 3128 is coming and going. It just happened again and wondering if anyone else is seeing this behavior and also how things look on MICE's route reflectors. I already contacted the good folks at the University of Wisconsin and they're not seeing anything on their side. AS Number: AS3128 AS Name: # First seen Last seen 1 2019-12-29 21:49:00 2019-12-29 21:50:00 2 2019-12-29 18:42:00 2019-12-29 18:43:00 3 2019-12-29 17:57:00 2019-12-29 17:58:00 4 2019-12-29 16:27:00 2019-12-29 16:28:00 5 2019-12-29 14:12:00 2019-12-29 14:13:00 6 2019-12-29 13:27:00 2019-12-29 13:28:00 7 2019-12-29 12:42:00 2019-12-29 12:43:00 8 2019-12-29 11:57:00 2019-12-29 11:58:00 9 2019-12-29 11:12:00 2019-12-29 11:13:00 10 2019-12-29 11:04:00 2019-12-29 11:05:00 11 2019-12-29 10:27:00 2019-12-29 10:28:00 12 2019-12-29 08:57:00 2019-12-29 08:58:00 13 2019-12-29 08:12:00 2019-12-29 08:13:00 14 2019-12-29 07:53:00 2019-12-29 07:54:00 15 2019-12-29 07:27:00 2019-12-29 07:28:00 16 2019-12-29 06:42:00 2019-12-29 06:43:00 17 2019-12-29 05:57:00 2019-12-29 05:58:00 18 2019-12-29 05:12:00 2019-12-29 05:13:00 19 2019-12-29 04:40:00 2019-12-29 04:41:00 20 2019-12-29 04:27:00 2019-12-29 04:28:00 21 2019-12-29 00:05:00 2019-12-29 00:06:00 22 2019-12-28 23:57:00 2019-12-28 23:58:00 23 2019-12-28 23:12:00 2019-12-28 23:13:00 24 2019-12-28 23:04:00 2019-12-28 23:05:00 25 2019-12-28 22:27:00 2019-12-28 22:28:00 26 2019-12-28 21:42:00 2019-12-28 21:43:00 27 2019-12-28 20:57:00 2019-12-28 20:58:00 28 2019-12-28 20:12:00 2019-12-28 20:13:00 29 2019-12-28 07:27:00 2019-12-28 07:28:00 30 2019-12-28 06:42:00 2019-12-28 06:43:00 31 2019-12-28 05:57:00 2019-12-28 05:58:00 32 2019-12-28 05:12:00 2019-12-28 05:13:00 33 2019-12-27 17:21:00 2019-12-27 17:22:00 34 2019-12-06 07:01:00 2019-12-27 06:10:00 Frank ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Dec. 30, 2019

Re: Route reflector check

by Michael Hare

Hi everyone- Wanted to add a bit more detail to what we (AS 3128) checked on our side. Specifically, 1) our peering session(s) to the RS seems stable from an uptime perspective. 2) I don't see a large delta in outbound bgp messages from us to MICE. 3) we don't run BMP so I don't actually track outbound changes at a level that would conclusive prove what's going on. Frank, is it possible you are being notified about a single route flapping? I did check AS2381's looking glass, specifically a single route (143.235.0.0/16) and it looks stable. Admittedly I didn't look for LG across all MICE members but I'm not sure if any would allow me to run an as-path regexp to see if there is a specific flapping route that is flapping. If someone can help out on that front it could be useful. [output from AS2381 looking glass] [BGP/170] 1w6d 13:08:45, MED 10, localpref 200, from 206.108.255.1 AS path: 3128 3128 3128 3128 I, validation-state: unverified > to 206.108.255.66 via xe-0/1/5.300 [BGP/170] 3w3d 14:51:19, MED 10, localpref 200, from 206.108.255.2 AS path: 3128 3128 3128 3128 I, validation-state: unverified > to 206.108.255.66 via xe-0/1/5.300 Thanks- -Michael From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Frank Bulk Sent: Monday, December 30, 2019 3:37 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] Route reflector check QRator has been alerting me that AS 3128 is coming and going. It just happened again and wondering if anyone else is seeing this behavior and also how things look on MICE's route reflectors. I already contacted the good folks at the University of Wisconsin and they're not seeing anything on their side. AS Number: AS3128 AS Name: # First seen Last seen 1 2019-12-29 21:49:00 2019-12-29 21:50:00 2 2019-12-29 18:42:00 2019-12-29 18:43:00 3 2019-12-29 17:57:00 2019-12-29 17:58:00 4 2019-12-29 16:27:00 2019-12-29 16:28:00 5 2019-12-29 14:12:00 2019-12-29 14:13:00 6 2019-12-29 13:27:00 2019-12-29 13:28:00 7 2019-12-29 12:42:00 2019-12-29 12:43:00 8 2019-12-29 11:57:00 2019-12-29 11:58:00 9 2019-12-29 11:12:00 2019-12-29 11:13:00 10 2019-12-29 11:04:00 2019-12-29 11:05:00 11 2019-12-29 10:27:00 2019-12-29 10:28:00 12 2019-12-29 08:57:00 2019-12-29 08:58:00 13 2019-12-29 08:12:00 2019-12-29 08:13:00 14 2019-12-29 07:53:00 2019-12-29 07:54:00 15 2019-12-29 07:27:00 2019-12-29 07:28:00 16 2019-12-29 06:42:00 2019-12-29 06:43:00 17 2019-12-29 05:57:00 2019-12-29 05:58:00 18 2019-12-29 05:12:00 2019-12-29 05:13:00 19 2019-12-29 04:40:00 2019-12-29 04:41:00 20 2019-12-29 04:27:00 2019-12-29 04:28:00 21 2019-12-29 00:05:00 2019-12-29 00:06:00 22 2019-12-28 23:57:00 2019-12-28 23:58:00 23 2019-12-28 23:12:00 2019-12-28 23:13:00 24 2019-12-28 23:04:00 2019-12-28 23:05:00 25 2019-12-28 22:27:00 2019-12-28 22:28:00 26 2019-12-28 21:42:00 2019-12-28 21:43:00 27 2019-12-28 20:57:00 2019-12-28 20:58:00 28 2019-12-28 20:12:00 2019-12-28 20:13:00 29 2019-12-28 07:27:00 2019-12-28 07:28:00 30 2019-12-28 06:42:00 2019-12-28 06:43:00 31 2019-12-28 05:57:00 2019-12-28 05:58:00 32 2019-12-28 05:12:00 2019-12-28 05:13:00 33 2019-12-27 17:21:00 2019-12-27 17:22:00 34 2019-12-06 07:01:00 2019-12-27 06:10:00 Frank ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Dec. 30, 2019

Re: [ADV] [MICE-DISCUSS] Route reflector check

by Doug McIntyre

On Mon, Dec 30, 2019 at 09:36:55PM +0000, Frank Bulk wrote: >QRator has been alerting me that AS 3128 is coming and going. It just happened again and wondering if anyone else is seeing this behavior and also how things look on MICE's route reflectors. The RR's show the connection to AS3128 has been up since 2019-12-17, no flapping. -- Doug McIntyre <merlyn(a)iphouse.net> ~.~ ipHouse ~.~ Network Engineer/Provisioning/Jack of all Trades

Dec. 30, 2019

Route reflector check

by Frank Bulk

QRator has been alerting me that AS 3128 is coming and going. It just happened again and wondering if anyone else is seeing this behavior and also how things look on MICE's route reflectors. I already contacted the good folks at the University of Wisconsin and they're not seeing anything on their side. AS Number: AS3128 AS Name: # First seen Last seen 1 2019-12-29 21:49:00 2019-12-29 21:50:00 2 2019-12-29 18:42:00 2019-12-29 18:43:00 3 2019-12-29 17:57:00 2019-12-29 17:58:00 4 2019-12-29 16:27:00 2019-12-29 16:28:00 5 2019-12-29 14:12:00 2019-12-29 14:13:00 6 2019-12-29 13:27:00 2019-12-29 13:28:00 7 2019-12-29 12:42:00 2019-12-29 12:43:00 8 2019-12-29 11:57:00 2019-12-29 11:58:00 9 2019-12-29 11:12:00 2019-12-29 11:13:00 10 2019-12-29 11:04:00 2019-12-29 11:05:00 11 2019-12-29 10:27:00 2019-12-29 10:28:00 12 2019-12-29 08:57:00 2019-12-29 08:58:00 13 2019-12-29 08:12:00 2019-12-29 08:13:00 14 2019-12-29 07:53:00 2019-12-29 07:54:00 15 2019-12-29 07:27:00 2019-12-29 07:28:00 16 2019-12-29 06:42:00 2019-12-29 06:43:00 17 2019-12-29 05:57:00 2019-12-29 05:58:00 18 2019-12-29 05:12:00 2019-12-29 05:13:00 19 2019-12-29 04:40:00 2019-12-29 04:41:00 20 2019-12-29 04:27:00 2019-12-29 04:28:00 21 2019-12-29 00:05:00 2019-12-29 00:06:00 22 2019-12-28 23:57:00 2019-12-28 23:58:00 23 2019-12-28 23:12:00 2019-12-28 23:13:00 24 2019-12-28 23:04:00 2019-12-28 23:05:00 25 2019-12-28 22:27:00 2019-12-28 22:28:00 26 2019-12-28 21:42:00 2019-12-28 21:43:00 27 2019-12-28 20:57:00 2019-12-28 20:58:00 28 2019-12-28 20:12:00 2019-12-28 20:13:00 29 2019-12-28 07:27:00 2019-12-28 07:28:00 30 2019-12-28 06:42:00 2019-12-28 06:43:00 31 2019-12-28 05:57:00 2019-12-28 05:58:00 32 2019-12-28 05:12:00 2019-12-28 05:13:00 33 2019-12-27 17:21:00 2019-12-27 17:22:00 34 2019-12-06 07:01:00 2019-12-27 06:10:00 Frank

Dec. 30, 2019

Re: Nuvera Port Upgrade

by Richard Laager

On 12/19/19 1:09 PM, Doug McIntyre wrote: > On Thu, Dec 19, 2019 at 11:40:29AM -0600, Jay Hanke wrote: >> Nuvera will be upgrading to 2x10G LAG on new ports and then removing >> their existing port. >> >> The new port assignments are (Main Switch) >> >> Ethernet 4/28 >> Ethernet 5/28 >> >> This port will be vacated Ethernet 4/15 > > Okay, I setup Po33 for them bound to Eth4/28 & Eth5/28. I've updated the website accordingly. Someone (Anthony, probably), please me know when the move is complete so I can mark 4/15 as unused again. -- Richard

Dec. 20, 2019

Re: Nuvera Port Upgrade

by Doug McIntyre

On Thu, Dec 19, 2019 at 11:40:29AM -0600, Jay Hanke wrote: >Nuvera will be upgrading to 2x10G LAG on new ports and then removing >their existing port. > >The new port assignments are (Main Switch) > >Ethernet 4/28 >Ethernet 5/28 > >This port will be vacated Ethernet 4/15 Okay, I setup Po33 for them bound to Eth4/28 & Eth5/28. -- Doug McIntyre <merlyn(a)iphouse.net> ~.~ ipHouse ~.~ Network Engineer/Provisioning/Jack of all Trades

Dec. 19, 2019

Nuvera Port Upgrade

by Jay Hanke

Nuvera will be upgrading to 2x10G LAG on new ports and then removing their existing port. The new port assignments are (Main Switch) Ethernet 4/28 Ethernet 5/28 This port will be vacated Ethernet 4/15 Thanks! -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

Dec. 19, 2019

Re: MICE MAC Address Limit

by Richard Laager

The board has approved the change in MAC address limit from 5 to 1. NOTE: This is orthogonal to the dedicated remote switch question, which is still pending. Non-dedicated remote switches can, for example, enforce this per-VLAN. We have already confirmed that most participants are using only 1 MAC address, and some that were using more than 1 have addressed that after being contacted. As discussed below, this change will be rolled out in a way to keep everyone working, by grandfathering if necessary. On 10/28/19 3:25 PM, Richard Laager wrote: > We have discussed this a bit in the past, and this came up at the last > UG. I'm looking for feedback from the group before formally proposing > this to the board for a vote. > > I first brought this up to the technical committee, CCing the board. I > have heard no objections. Jeremy is "in favor of it 100%". > > Based on our quick conversation after the UG, I _think_ Anthony is in > favor as well; we both made notes to follow up on this. > > ----- > > I am proposing that MICE change its MAC address limit from 5 MACs per > port to 1 MAC per port. 1 MAC address is enough for normal scenarios and > this would further limit the potential for problems on the fabric. This > restriction is one that SIX and AMS-IX both have, with the latter being > known for their excellent configuration guide for participants: > https://www.ams-ix.net/ams/documentation/config-guide > > To be clear, this is still only for end ports, not ports facing remote > switches, of course. The remote switches are responsible for enforcing > the current MAC limit, and would be responsible for changing those > settings as described here. > > If someone is swapping their MICE-facing router, the resulting port flap > on the MICE/remote switch will clear the limit anyway, if they are > directly connected. If they are unable to flap the port (e.g. because > they have someone else's layer 2 gear in the middle), they will have to > either work with that carrier or their switch operator (MICE/remote) to > flap the port. This is a non-zero burden, but I think it's pretty > minimal in practice. Additionally, if someone plans to make an equipment > swap, we would increase the limit in advance, temporarily, upon request. > > For reference, SIX goes further and locks you to a particular MAC > address in an ACL, so you have to contact them to change your MAC. I am > not proposing that. Still, I've been through that a couple of times, and > even that isn't really too big of a deal. So I don't think the change to > a limit of 1 MAC at MICE will be particularly onerous. > > There are a couple of participants who have two IP address sets (one > IPv4 and one IPv6) on the same port. For those ports, the limit > obviously must be at least 2 MACs, but I propose 3 MACs as the limit. If > they're using two IPs, they probably are doing so to get some redundancy > out of it. For this particular use case, needing to flap the port > defeats that redundancy goal, as it breaks the other router. Having a > MAC limit of 3 would allow them to swap a device without needing to flap > the port. We would grandfather these setups until they are looking for a > port-type upgrade; at that point they would need to get to one IP > address set per port (by splitting into two ports or reducing to one IP > address set) or request an exception as outlined below. The exception > process would allow us to better evaluate this use case at that time. > > There are some members who have multiple MACs showing up now, despite > using only one device. For those, I am proposing we set the MAC limit to > however many MACs are currently showing up on their port. That is, they > would be grandfathered at their current situation for now. At the time > of this change, we would encourage them to investigate and fix the > multiple MACs issue at their leisure. In the future, if they are doing a > equipment swap (especially like-for-like), we would again encourage them > to address it, but would otherwise leave their higher limit in place. If > they do a port-type upgrade, we would not bring the grandfathering over. > If they needed, they could ask for an official exception at that time. > > There may be cases where people cannot reasonably fix their routers to > use only 1 MAC, other scenarios I haven't considered, or things that may > come up in the future. For that reason, I think we should allow for the > possibility of exceptions upon request to the board. In practice, I > would expect the board would confer with the technical committee and/or > the technical committee would be bringing these to the board anyway. The > goal is to strike a reasonable balance between protecting the fabric > without preventing networks from connecting. > -- Richard

Dec. 14, 2019

NebraskaLink Port Move

by Jay Hanke

Nebraskalink is moving their connection from the Zayo remote switch to the main switch. All IP address/Contact information will remain the same. The new port assignment is Main Switch: Ethernet4/13 Welcome aboard again! -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

Dec. 12, 2019