April 2021 - mice-discuss - lists.micemn.net

Old/Stale Infrastructure information - IXP Dashboard / AcenTek

by Jesse Reed

To whom it may concern: Earlier this year, AcenTek worked w/someone to turn up a 2nd link inside a 20 Gig port-ch @ the 511 building facing our equipment. Logging into the IXP portal today, we see that our information is still only reflective of 1 single 10 Gig link. Can this information be updated accordingly inside the AcenTek portal? Thanks ahead of time, Jesse Reed

April 26, 2021

Re: Zayo Belle Plaine - April 15th

by Ben Wiechman

IOS-XR bottoms out at 15ms. IOS/XE has typically been 150ms. Agreed that in general unless you have <50ms failover requirements 150ms+ is probably a good compromise. Ben Wiechman Director of IP Strategy and Engineering Direct: 320.256.0184 Cell: 320.247.3224 ben.wiechman(a)arvig.com 150 Second Street SW | Perham, MN 56573 | arvig.com On Wed, Apr 21, 2021, 17:57 Andrew Hoyos <hoyosa(a)gmail.com> wrote: > On Apr 21, 2021, at 5:52 PM, David Farmer < > 0000000e7948cb21-dmarc-request(a)LISTS.IPHOUSE.NET> wrote: > > > On Wed, Apr 21, 2021 at 5:36 PM Richard Laager <rlaager(a)wiktel.com> wrote: > >> On 4/21/21 3:04 PM, Frank Bulk wrote: >> > And to follow up on my previous question, is Arista falling bit short in >> >> > our situation, by not supporting a receive interval of 10 msec? >> >> I've had a couple vendors suggest not to make it that short. Brocade, >> for example, suggested 150 ms as a minimum. Arista was more vague, but >> from your error message, apparently their implementation doesn't even >> try to do less than 50 ms. >> > > Maybe think about this from another perspective, 10 ms is 100 times a > second, 50 ms is 20 times a second, and 150 ms just over 6 times a second. > I think 10 ms is probably being a little impatient. > > > Not the mention, the added CPU load on both ends dealing with said BFD > packets 100x/sec. > > We’ve generally seen 50-250ms used in practice. 10ms does seem super > aggressive. We use 250ms x 3 here for backbone links and peers/transit that > support BFD, and 750ms x 3 facing internal gear. > > > — > Andrew Hoyos > hoyosa(a)gmail.com > > > > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

April 22, 2021

Re: Zayo Belle Plaine - April 15th

by Andrew Hoyos

On Apr 21, 2021, at 5:52 PM, David Farmer <0000000e7948cb21-dmarc-request(a)LISTS.IPHOUSE.NET> wrote: > > On Wed, Apr 21, 2021 at 5:36 PM Richard Laager <rlaager(a)wiktel.com <mailto:rlaager@wiktel.com>> wrote: > On 4/21/21 3:04 PM, Frank Bulk wrote: > > And to follow up on my previous question, is Arista falling bit short in > > our situation, by not supporting a receive interval of 10 msec? > > I've had a couple vendors suggest not to make it that short. Brocade, > for example, suggested 150 ms as a minimum. Arista was more vague, but > from your error message, apparently their implementation doesn't even > try to do less than 50 ms. > > Maybe think about this from another perspective, 10 ms is 100 times a second, 50 ms is 20 times a second, and 150 ms just over 6 times a second. I think 10 ms is probably being a little impatient. Not the mention, the added CPU load on both ends dealing with said BFD packets 100x/sec. We’ve generally seen 50-250ms used in practice. 10ms does seem super aggressive. We use 250ms x 3 here for backbone links and peers/transit that support BFD, and 750ms x 3 facing internal gear. — Andrew Hoyos hoyosa(a)gmail.com <mailto:hoyosa@gmail.com>

April 21, 2021

Re: Zayo Belle Plaine - April 15th

by David Farmer

On Wed, Apr 21, 2021 at 5:36 PM Richard Laager <rlaager(a)wiktel.com> wrote: > On 4/21/21 3:04 PM, Frank Bulk wrote: > > And to follow up on my previous question, is Arista falling bit short in > > our situation, by not supporting a receive interval of 10 msec? > > I've had a couple vendors suggest not to make it that short. Brocade, > for example, suggested 150 ms as a minimum. Arista was more vague, but > from your error message, apparently their implementation doesn't even > try to do less than 50 ms. > Maybe think about this from another perspective, 10 ms is 100 times a second, 50 ms is 20 times a second, and 150 ms just over 6 times a second. I think 10 ms is probably being a little impatient. Thanks. -- =============================================== David Farmer Email:farmer@umn.edu Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 ===============================================

April 21, 2021

Re: Zayo Belle Plaine - April 15th

by Richard Laager

On 4/21/21 3:04 PM, Frank Bulk wrote: > And to follow up on my previous question, is Arista falling bit short in > our situation, by not supporting a receive interval of 10 msec? I've had a couple vendors suggest not to make it that short. Brocade, for example, suggested 150 ms as a minimum. Arista was more vague, but from your error message, apparently their implementation doesn't even try to do less than 50 ms. -- Richard

April 21, 2021

Re: Zayo Belle Plaine - April 15th

by Jeremy Lumby

Frank, I have been gradually implementing the unknown unicast rate limiting (starting with my own extension) to make sure it does not cause any unexpected problems. I had not applied it to the route servers yet, so I can understand why you saw another blip when a member with a significant amount of traffic had their port go down unexpectedly today. Jeremy From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Frank Bulk Sent: Wednesday, April 21, 2021 3:05 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Zayo Belle Plaine - April 15th We saw another BFD event at 2:04 pm today – was there a disturbance in the force? 😉 And to follow up on my previous question, is Arista falling bit short in our situation, by not supporting a receive interval of 10 msec? Frank From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> Sent: Friday, April 16, 2021 4:43 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [External GPC] Re: [MICE-DISCUSS] Zayo Belle Plaine - April 15th In this case it was BFD that kicked in – I have BFD only set up for the two route reflectors. Other BGP sessions stayed up. I also got this alert from our Arista 7504: Apr 15 17:37:34 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net) Bfd: %BFD-5-INCOMPATIBLE_RX_INTERVAL: Received a BFD rx interval from peer (vrf:default, ip:206.108.255.2 (AS53679-MRS-2.micemn.net) intf:Port-Channel2, srcIp:0.0.0.0, type:normal) of 10 milliseconds, outside of the supported range of 50-60000 milliseconds. (message repeated 3 times in 1.18783e+06 secs) Is Arista falling bit short in this situation, by not supporting 10 msec? Frank From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Jeremy Lumby Sent: Thursday, April 15, 2021 10:52 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Zayo Belle Plaine - April 15th Frank, Ideally it should not have, however what tends to happen is that when large ports go down the MAC addresses leave the table, and therefore the traffic gets flooded to all ports until BGP times out on the networks sending the traffic. Since the route servers only have 1G ports, the 10G of traffic that typically is flowing to Belle Plaine probably saturated the route server ports for a short amount of time. If you have short timeouts, or use something more sensitive like BFD I could see how your sessions dropped from the saturation. Jeremy From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Frank Bulk Sent: Thursday, April 15, 2021 9:40 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Zayo Belle Plaine - April 15th We (AS18883) saw our BGP sessions with the MICE router reflectors bounce at that time – should that have happened? Frank From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of AnthonyAnderberg(a)nuvera.net Sent: Thursday, April 15, 2021 6:05 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] Zayo Belle Plaine - April 15th Looks like the Zayo Belle Plaine remote went down at 5:37pm, I believe Lee @ SW MN Broadband is opening a ticket now. >From the MICE side we have light but no frames, and looking at other unrelated systems it appears the cut is South of Hwy 212 someplace. Best of luck everyone, Anthony To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

April 21, 2021

Re: Zayo Belle Plaine - April 15th

by Frank Bulk

We saw another BFD event at 2:04 pm today – was there a disturbance in the force? 😉 And to follow up on my previous question, is Arista falling bit short in our situation, by not supporting a receive interval of 10 msec? Frank From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> Sent: Friday, April 16, 2021 4:43 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [External GPC] Re: [MICE-DISCUSS] Zayo Belle Plaine - April 15th In this case it was BFD that kicked in – I have BFD only set up for the two route reflectors. Other BGP sessions stayed up. I also got this alert from our Arista 7504: Apr 15 17:37:34 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net) Bfd: %BFD-5-INCOMPATIBLE_RX_INTERVAL: Received a BFD rx interval from peer (vrf:default, ip:206.108.255.2 (AS53679-MRS-2.micemn.net) intf:Port-Channel2, srcIp:0.0.0.0, type:normal) of 10 milliseconds, outside of the supported range of 50-60000 milliseconds. (message repeated 3 times in 1.18783e+06 secs) Is Arista falling bit short in this situation, by not supporting 10 msec? Frank From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET>> On Behalf Of Jeremy Lumby Sent: Thursday, April 15, 2021 10:52 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Zayo Belle Plaine - April 15th Frank, Ideally it should not have, however what tends to happen is that when large ports go down the MAC addresses leave the table, and therefore the traffic gets flooded to all ports until BGP times out on the networks sending the traffic. Since the route servers only have 1G ports, the 10G of traffic that typically is flowing to Belle Plaine probably saturated the route server ports for a short amount of time. If you have short timeouts, or use something more sensitive like BFD I could see how your sessions dropped from the saturation. Jeremy From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Frank Bulk Sent: Thursday, April 15, 2021 9:40 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Zayo Belle Plaine - April 15th We (AS18883) saw our BGP sessions with the MICE router reflectors bounce at that time – should that have happened? Frank From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET>> On Behalf Of AnthonyAnderberg(a)nuvera.net<mailto:AnthonyAnderberg@nuvera.net> Sent: Thursday, April 15, 2021 6:05 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: [MICE-DISCUSS] Zayo Belle Plaine - April 15th Looks like the Zayo Belle Plaine remote went down at 5:37pm, I believe Lee @ SW MN Broadband is opening a ticket now. From the MICE side we have light but no frames, and looking at other unrelated systems it appears the cut is South of Hwy 212 someplace. Best of luck everyone, Anthony ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

April 21, 2021

WiscNet MICE switch maintenance 2021-04-27 7am - Noon

by Chris Wopat

Hi Folks, Heads up that the WiscNet remote MICE switch will be going down for maintenance a week from today - Tuesday April 27th from 7am until approximately noon. We'll gracefully tear down BGP Sessions in advance and will chime in when complete. Cheers, -- Chris Wopat Network Engineer, WiscNet wopat(a)wiscnet.net 608-210-3965

April 20, 2021

Re: Zayo Belle Plaine - April 15th

by Jeremy Lumby

It is interesting that it is stated in the text, but not shown in the sample port config. I think it probably got missed when migrating from the Juniper core. I can correct it. From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Frank Bulk Sent: Friday, April 16, 2021 4:45 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Zayo Belle Plaine - April 15th If the MICE’s technical page documented limiting unknown unicast to 100 Mbps, shouldn’t its gear be set up that way? Frank From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Jeremy Lumby Sent: Friday, April 16, 2021 7:52 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Zayo Belle Plaine - April 15th Right now it is setup for broadcast, and multicast, however I am in favor of adding unknown unicast. From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Andrew Hoyos Sent: Friday, April 16, 2021 12:38 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Zayo Belle Plaine - April 15th Is there anything configured on the MICE core switches to police/drop unknown unicast? I saw mention on the MICE technical page of 100mbps limit of unknown unicast, seems like maybe that could be more aggressive (or at a minimum, on route server ports and/or interfaces facing the Juniper?) On Apr 15, 2021, at 10:52 PM, Jeremy Lumby <jlumby(a)MNVOIP.COM> wrote: Frank, Ideally it should not have, however what tends to happen is that when large ports go down the MAC addresses leave the table, and therefore the traffic gets flooded to all ports until BGP times out on the networks sending the traffic. Since the route servers only have 1G ports, the 10G of traffic that typically is flowing to Belle Plaine probably saturated the route server ports for a short amount of time. If you have short timeouts, or use something more sensitive like BFD I could see how your sessions dropped from the saturation. Jeremy From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Frank Bulk Sent: Thursday, April 15, 2021 9:40 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Zayo Belle Plaine - April 15th We (AS18883) saw our BGP sessions with the MICE router reflectors bounce at that time – should that have happened? Frank From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of AnthonyAnderberg(a)nuvera.net Sent: Thursday, April 15, 2021 6:05 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] Zayo Belle Plaine - April 15th Looks like the Zayo Belle Plaine remote went down at 5:37pm, I believe Lee @ SW MN Broadband is opening a ticket now. >From the MICE side we have light but no frames, and looking at other unrelated systems it appears the cut is South of Hwy 212 someplace. Best of luck everyone, Anthony To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

April 16, 2021

Re: Zayo Belle Plaine - April 15th

by Frank Bulk

If the MICE’s technical page documented limiting unknown unicast to 100 Mbps, shouldn’t its gear be set up that way? Frank From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Jeremy Lumby Sent: Friday, April 16, 2021 7:52 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Zayo Belle Plaine - April 15th Right now it is setup for broadcast, and multicast, however I am in favor of adding unknown unicast. From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Andrew Hoyos Sent: Friday, April 16, 2021 12:38 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Zayo Belle Plaine - April 15th Is there anything configured on the MICE core switches to police/drop unknown unicast? I saw mention on the MICE technical page of 100mbps limit of unknown unicast, seems like maybe that could be more aggressive (or at a minimum, on route server ports and/or interfaces facing the Juniper?) On Apr 15, 2021, at 10:52 PM, Jeremy Lumby <jlumby(a)MNVOIP.COM<mailto:jlumby@MNVOIP.COM>> wrote: Frank, Ideally it should not have, however what tends to happen is that when large ports go down the MAC addresses leave the table, and therefore the traffic gets flooded to all ports until BGP times out on the networks sending the traffic. Since the route servers only have 1G ports, the 10G of traffic that typically is flowing to Belle Plaine probably saturated the route server ports for a short amount of time. If you have short timeouts, or use something more sensitive like BFD I could see how your sessions dropped from the saturation. Jeremy From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Frank Bulk Sent: Thursday, April 15, 2021 9:40 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Zayo Belle Plaine - April 15th We (AS18883) saw our BGP sessions with the MICE router reflectors bounce at that time – should that have happened? Frank From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET>> On Behalf Of AnthonyAnderberg(a)nuvera.net<mailto:AnthonyAnderberg@nuvera.net> Sent: Thursday, April 15, 2021 6:05 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: [MICE-DISCUSS] Zayo Belle Plaine - April 15th Looks like the Zayo Belle Plaine remote went down at 5:37pm, I believe Lee @ SW MN Broadband is opening a ticket now. From the MICE side we have light but no frames, and looking at other unrelated systems it appears the cut is South of Hwy 212 someplace. Best of luck everyone, Anthony ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

April 16, 2021