April 2022 - mice-discuss - lists.micemn.net

Please welcome G-Core Labs S.A. to MICE!

by Jeremy Lumby

Name: G-Core Labs S.A. Switch: MICE Core Port: E 4/7 ASN: 199524 IPv4: 206.108.255.191/24 IPv6: 2001:504:27:0:3:0B64::1/64 Peering Contact: noc(a)gcore.lu Route Servers: YES BFD: No Welcome Aboard! Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 Main: 612-355-7740 Direct: 612-392-6814 Fax: 952-873-7425 jlumby(a)mnvoip.com

April 8, 2022

Re: IPv4 BGP session to Google down

by Jay Hanke

It's been oos for almost 2 weeks. On Thu, Apr 7, 2022, 4:28 PM Frank Bulk <fbulk(a)mypremieronline.com> wrote: > Ok, then maybe we're the only one down. > > Frank > > -----Original Message----- > From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Jeff > Wilde > Sent: Thursday, April 7, 2022 12:02 PM > To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET > Subject: Re: [MICE-DISCUSS] IPv4 BGP session to Google down > > I did not. > > > Jeff Wilde * Network Engineer * Park Region > Office 218.826.6161 * Direct 218.826.8330 * Fax 218.826.6298 > > > -----Original Message----- > From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Frank > Bulk > Sent: Thursday, April 7, 2022 11:57 AM > To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET > Subject: [MICE-DISCUSS] IPv4 BGP session to Google down > > Our BGP session bounce several times last might between 10:00 and 10:07 pm > Central, and then has been down from 10:21 am. > > Anyone else seeing this? I see a little drop in the graph ( > http://micelg.usinternet.com/cacti/graph_view.php?action=tree&tree_id=1&lea…) > this morning. > > Frank Bulk > FiberNet (AS18883) > > Apr 7 10:21:55 207.32.15.14 Rib: %BGP-3-NOTIFICATION: sent to neighbor > 206.108.255.167 (AS 15169) 4/0 (Hold Timer Expired Error/Unspecified) 0 > bytes Apr 7 10:21:55 207.32.15.14 Rib: %BGP-5-ADJCHANGE: peer > 206.108.255.167 (AS 15169) old state Established event HoldTime new state > Idle Apr 7 10:22:00 207.32.15.14 Rib: %BGP-5-ADJCHANGE: peer > 206.108.255.167 (AS 15169) old state OpenConfirm event RecvKeepAlive new > state Established Apr 7 10:22:30 207.32.15.14 Rib: %BGP-5-ADJCHANGE: peer > 206.108.255.167 (AS 15169) old state Established event HoldTime new state > Idle >

April 7, 2022

Re: IPv4 BGP session to Google down

by Frank Bulk

Ok, then maybe we're the only one down. Frank -----Original Message----- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Jeff Wilde Sent: Thursday, April 7, 2022 12:02 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] IPv4 BGP session to Google down I did not. Jeff Wilde * Network Engineer * Park Region Office 218.826.6161 * Direct 218.826.8330 * Fax 218.826.6298 -----Original Message----- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Frank Bulk Sent: Thursday, April 7, 2022 11:57 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] IPv4 BGP session to Google down Our BGP session bounce several times last might between 10:00 and 10:07 pm Central, and then has been down from 10:21 am. Anyone else seeing this? I see a little drop in the graph (http://micelg.usinternet.com/cacti/graph_view.php?action=tree&tree_id=1&lea…) this morning. Frank Bulk FiberNet (AS18883) Apr 7 10:21:55 207.32.15.14 Rib: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.167 (AS 15169) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes Apr 7 10:21:55 207.32.15.14 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.167 (AS 15169) old state Established event HoldTime new state Idle Apr 7 10:22:00 207.32.15.14 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.167 (AS 15169) old state OpenConfirm event RecvKeepAlive new state Established Apr 7 10:22:30 207.32.15.14 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.167 (AS 15169) old state Established event HoldTime new state Idle

April 7, 2022

Re: IPv4 BGP session to Google down

by Jeff Wilde

I did not. Jeff Wilde * Network Engineer * Park Region Office 218.826.6161 * Direct 218.826.8330 * Fax 218.826.6298 -----Original Message----- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Frank Bulk Sent: Thursday, April 7, 2022 11:57 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] IPv4 BGP session to Google down Our BGP session bounce several times last might between 10:00 and 10:07 pm Central, and then has been down from 10:21 am. Anyone else seeing this? I see a little drop in the graph (http://micelg.usinternet.com/cacti/graph_view.php?action=tree&tree_id=1&lea…) this morning. Frank Bulk FiberNet (AS18883) Apr 7 10:21:55 207.32.15.14 Rib: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.167 (AS 15169) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes Apr 7 10:21:55 207.32.15.14 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.167 (AS 15169) old state Established event HoldTime new state Idle Apr 7 10:22:00 207.32.15.14 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.167 (AS 15169) old state OpenConfirm event RecvKeepAlive new state Established Apr 7 10:22:30 207.32.15.14 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.167 (AS 15169) old state Established event HoldTime new state Idle

April 7, 2022

IPv4 BGP session to Google down

by Frank Bulk

Our BGP session bounce several times last might between 10:00 and 10:07 pm Central, and then has been down from 10:21 am. Anyone else seeing this? I see a little drop in the graph (http://micelg.usinternet.com/cacti/graph_view.php?action=tree&tree_id=1&lea…) this morning. Frank Bulk FiberNet (AS18883) Apr 7 10:21:55 207.32.15.14 Rib: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.167 (AS 15169) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes Apr 7 10:21:55 207.32.15.14 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.167 (AS 15169) old state Established event HoldTime new state Idle Apr 7 10:22:00 207.32.15.14 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.167 (AS 15169) old state OpenConfirm event RecvKeepAlive new state Established Apr 7 10:22:30 207.32.15.14 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.167 (AS 15169) old state Established event HoldTime new state Idle

April 7, 2022

Re: MICE Remote Switch Policy

by Justin Krejci

Jonathan, Very excellent points. Here are some reply thoughts to consider. 1. the member has their own private, long distance, layer 2 link to an IX. The member themself knows it's a long distance but most of the other members of the IX probably will not have any idea, unless they happen to infer it from latency to your router or you otherwise disclose it. 2. if the IX does a long distance layer 2 link to an official/remote switch, it would be a lot more clear to people at a glance that members on such a far away remote switch are indeed far away As suggested in an earlier post of this thread, BGP communities could be added on the route server sessions of members marking the member's routes as originating via switch X. Then you as a fellow member could choose to adjust your BGP knobs based on that data, maybe you want to not take their routes or lower your local pref for them specifically and even choose to AS prepend your routes to them or not advertise to them at all. With members individually connecting a long distance, only the members themselves are respectively aware of the fact that they are far away. For those not using route servers, they can still easily gauge the member distances if they are clearly documented as connected to switches classed as far away switches and therefore can make their own routing decisions based on that knowledge, which may include not peering with distant members at all at said IX. If geographic distance is a concern for connected/connecting members, maybe the conversation should be had on whether remote switches or individual members can be connecting from such a distance greater than XYZ distance or perhaps XYZ avg latency, as you mentioned. But now, what if I setup a router in a nearby geographic location with low latency but only have a single router and a single connection back to my main established network that is far away? I as the member look to be close to the fabric and indeed am close to the fabric, yet most of my eyeballs and/or servers are still far away, in this hypothetical situation. Though I do think, as you imply, there is a point of diminishing returns by adding more and more connectivity at increasing distances, not just in target market / brand but also in the very real operational issues that can arise from two networks taking a highly inefficient path via an IX because it is shorter AS hops than usual transit links but is much longer geographically. I have seen this happen first hand. ________________________________ From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> on behalf of Jonathan Stewart <jonathan(a)LES.NET> Sent: Tuesday, April 5, 2022 4:22 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] MICE Remote Switch Policy A bit late to the discussion, but here are my thoughts: On Thu., 2022-03-24 3:58 p.m., Richard Laager wrote: > At Wiktel, I peer with MN VoIP's far away extensions in Minneapolis. > For example, I peer at SeattleIX (SIX) in Minneapolis. This has caused > me some issues. For example, latency-sensitive gaming traffic was > tromboning > Wiktel-Minneapolis-Seattle-Chicago-Seattle-Minneapolis-Wiktel rather > than Wiktel-Chicago-Wiktel. This is the biggest issue I am concerned about. Layer-2 can't tell us how far away the peer is. BGP doesn't have knobs for varying latency across peers on the same Layer-2. The SIX example of really-remote peers, with resultant latency higher than Transit, is a performance degradation for SIX and their members, and reduces the quality of SIX, I'd argue. I would suggest that MICE have a limit, say 5 ms RTT, on latency to remote switches. Then we can say that MICE is in a geographic region, not potentially a pan-North-America layer-2. Part of MICE's identity should be its location, and the resultant low-latency across MICE itself. If MICE is in Kansas City, and KCIX is in Minneapolis, what's the difference between them? Why do they both exist if they serve the same datacentres in the same way: a Layer-2 ethernet fabric? I become confused at what problem is being solved when IXes start overlapping and becoming non-geographic, rather than limited to a metro area, or certain radius around a hub. I think there's a difference between an individual member (me) arriving over a long layer-2 link (from Winnipeg) to the IX, and the IX doing long layer-2 links themselves. The difference is that I have knowledge of and control over my long-haul links, but if the IX does it, unexpectedly long-haul paths happen without my decision to use them. Finally, I'm a big fan of what MICE has accomplished so far, I hope this message doesn't read as being overly critical. Cheers, Jonathan AS18451 - LES.NET Jonathan Stewart Network Engineer LES.NET - AS18451 Desk: 1-204-666-6191 Mobile: 1-204-990-2120 130 Portage Avenue E Winnipeg, MB R3C 0A1 CANADA

April 5, 2022

Re: MICE Remote Switch Policy

by Jonathan Stewart

A bit late to the discussion, but here are my thoughts: On Thu., 2022-03-24 3:58 p.m., Richard Laager wrote: > At Wiktel, I peer with MN VoIP's far away extensions in Minneapolis. > For example, I peer at SeattleIX (SIX) in Minneapolis. This has caused > me some issues. For example, latency-sensitive gaming traffic was > tromboning > Wiktel-Minneapolis-Seattle-Chicago-Seattle-Minneapolis-Wiktel rather > than Wiktel-Chicago-Wiktel. This is the biggest issue I am concerned about. Layer-2 can't tell us how far away the peer is. BGP doesn't have knobs for varying latency across peers on the same Layer-2. The SIX example of really-remote peers, with resultant latency higher than Transit, is a performance degradation for SIX and their members, and reduces the quality of SIX, I'd argue. I would suggest that MICE have a limit, say 5 ms RTT, on latency to remote switches. Then we can say that MICE is in a geographic region, not potentially a pan-North-America layer-2. Part of MICE's identity should be its location, and the resultant low-latency across MICE itself. If MICE is in Kansas City, and KCIX is in Minneapolis, what's the difference between them? Why do they both exist if they serve the same datacentres in the same way: a Layer-2 ethernet fabric? I become confused at what problem is being solved when IXes start overlapping and becoming non-geographic, rather than limited to a metro area, or certain radius around a hub. I think there's a difference between an individual member (me) arriving over a long layer-2 link (from Winnipeg) to the IX, and the IX doing long layer-2 links themselves. The difference is that I have knowledge of and control over my long-haul links, but if the IX does it, unexpectedly long-haul paths happen without my decision to use them. Finally, I'm a big fan of what MICE has accomplished so far, I hope this message doesn't read as being overly critical. Cheers, Jonathan AS18451 - LES.NET Jonathan Stewart Network Engineer LES.NET - AS18451 Desk: 1-204-666-6191 Mobile: 1-204-990-2120 130 Portage Avenue E Winnipeg, MB R3C 0A1 CANADA

April 5, 2022

OT: NANOG call for presentations

by steve ulrich

folks - long time lurker here. ( as well as a couple of times meatspace meeting attendee. ) apologies in advance for the off-topic email, but i'm on the NANOG program committee and we've had folks express interest in presentations regarding regional peering exchanges as well as talks regarding independent networks that have done interesting things from an operational perspective. e.g.: managing growth, novel deployment of access technologies. etc. it struck me that the folks peering at MICE or engaged in MICE operations might have great presentation ideas/material.. the next NANOG is in june in montreal, ca. but we're always looking for presentations for future NANOGs as well. NANOG has a variety of different presentation formats and we're open to suggestions. please feel free to submit an abstract online ( https://nanog.org/program/call-presentations/) and someone from the program committee thanks! -- steve ulrich (sulrich(a)botwerks.org)

April 5, 2022

Please welcome Long Lines Broadband to MICE!

by Jeremy Lumby

Name: Long Lines Broadband Switch: MICE Main Port: E6/21/1 ASN: 32867 IPv4: This is a port upgrade, all IP info remains the same. The new connection will be 100G IPv6: Peering Contact: data(a)longlines.biz Route Servers: Yes BFD: No Welcome Aboard Again! Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 Main: 612-355-7740 Direct: 612-392-6814 Fax: 952-873-7425 jlumby(a)mnvoip.com

April 4, 2022

ARIN NOAUTH

by Jay Hanke

ARIN NOAUTH (legacy IRR) is getting shut down today at 11 AM central time. There will likely be some filtering issues across the internet for networks that haven't updated their IRR. https://www.arin.net/blog/2022/02/16/Take-Action-Now-to-Prepare-for-the-ARI… -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

April 4, 2022