- mice-discuss - lists.micemn.net

Please welcome Flexnetworks to MICE!
by Jeremy Lumby May 24, 2022

May 24, 2022

Name: Flexnetworks Switch: MICE Core Port: E 4/23 ASN: 19523 IPv4: 206.108.255.194/24 IPv6: 2001:504:27:0:0:4C43::1/64 Peering Contact: support(a)flexnetworks.ca Route Servers: YES BFD: NO Welcome Aboard! Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 Main: 612-355-7740 Direct: 612-392-6814 Fax: 952-873-7425 jlumby(a)mnvoip.com

2 1

Re: Apple peering down
by AnthonyAnderberg＠nuvera.net May 23, 2022

May 23, 2022

Looks like MICE’s LAG Po30 to Apple went down at the same time - about 3:15am this morning. From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> on behalf of Frank Bulk <fbulk(a)MYPREMIERONLINE.COM> Reply-To: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> Date: Friday, May 20, 2022 at 8:48 AM To: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> Subject: [MICE-DISCUSS] Apple peering down Anyone else seeing Apple peer down over IPv4 and IPv6 since 3:18 am? May 20 03:18:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net) SiouxCity-Fibernet-Arista Rib: %BGP-3-NOTIFICATION: sent to neighbor 2001:504:27::2ca:0:1 (AS714-1.micemn.net) (AS 714 APPLE-ENGINEERING, US) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes May 20 03:18:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net) SiouxCity-Fibernet-Arista Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::2ca:0:1 (AS714-1.micemn.net) (AS 714 APPLE-ENGINEERING, US) old state Established event HoldTime new state Idle May 20 03:18:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net) SiouxCity-Fibernet-Arista Rib: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.146 (AS714-1.micemn.net) (AS 714 APPLE-ENGINEERING, US) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes May 20 03:18:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net) SiouxCity-Fibernet-Arista Rib: %BGP-5-ADJCHANGE: peer 206.108.255.146 (AS714-1.micemn.net) (AS 714 APPLE-ENGINEERING, US) old state Established event HoldTime new state Idle Fran k ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

4 7

Please welcome Communications 1 Network to MICE!
by Jeremy Lumby May 20, 2022

May 20, 2022

Name: Communications 1 Network Switch: SouthFront Albert Lea Port: E 7/4 ASN: 394911 IPv4: 206.108.255.195/24 IPv6: 2001:504:27:0:6:69F::1/64 Peering Contact: jkeiper(a)comm1net.net Route Servers: YES BFD: YES Welcome Aboard! Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 Main: 612-355-7740 Direct: 612-392-6814 Fax: 952-873-7425 jlumby(a)mnvoip.com

3 2

Apple peering down
by Frank Bulk May 20, 2022

May 20, 2022

Anyone else seeing Apple peer down over IPv4 and IPv6 since 3:18 am? May 20 03:18:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net) SiouxCity-Fibernet-Arista Rib: %BGP-3-NOTIFICATION: sent to neighbor 2001:504:27::2ca:0:1 (AS714-1.micemn.net) (AS 714 APPLE-ENGINEERING, US) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes May 20 03:18:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net) SiouxCity-Fibernet-Arista Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::2ca:0:1 (AS714-1.micemn.net) (AS 714 APPLE-ENGINEERING, US) old state Established event HoldTime new state Idle May 20 03:18:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net) SiouxCity-Fibernet-Arista Rib: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.146 (AS714-1.micemn.net) (AS 714 APPLE-ENGINEERING, US) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes May 20 03:18:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net) SiouxCity-Fibernet-Arista Rib: %BGP-5-ADJCHANGE: peer 206.108.255.146 (AS714-1.micemn.net) (AS 714 APPLE-ENGINEERING, US) old state Established event HoldTime new state Idle Fran k

2 1

Please welcome Bluepeak to MICE!
by Jeremy Lumby May 11, 2022

May 11, 2022

Name: Bluepeak Switch: MICE Core Port: E6/23/1 ASN: 20412 IPv4: This is a port upgrade and name change from Vast Broadband, all IP info remains the same. The new connection will be 100G IPv6: Peering Contact: neteng(a)mybluepeak.com Route Servers: Yes BFD: Yes Welcome Aboard Again! Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 Main: 612-355-7740 Direct: 612-392-6814 Fax: 952-873-7425 jlumby(a)mnvoip.com

1 0

Midwest Peering Summit 2022 Update
by Jay Hanke May 9, 2022

May 9, 2022

As of last night, we have 80 attendees registered for the summer event. The event is free of charge to peering networks (or potentially peering networks). Pre Registration is required. You can register here: https://www.eventcreate.com/e/mwps2022 This is our discounted room block: https://reservations.travelclick.com/109230?groupID=3466891 We ask if your company is bringing sales folks that you purchase a table ($200) to help with costs. MWPS 2022 Sponsors/Exhibitors ------------------------------------ 1623 Farnam Arista Aureon 616 Bluebird Networks NocIX South Front Networks MWPS 2022 Exhibitors ------------------------------------ Aurora Fiber Optic Networks Great Plains Communications Heartland Technology Data Center Hurricane Electric Precision Group US Internet Vantage Point Solutions IX Attendees ------------------------------------ ChIX (Chicago IL) DesMoinesIX (Des Moines IA) Fiber Data Internet Exchange (Altoona, Indy, STL) KCIX (Kansas City MO) MICE (Minneapolis MN) OhioIX (Columbus OH) OmahaIX (Omaha NE) QCIX (Davenport) SiouxIX (Sioux City IA) SpringIX (Springfield MO) STLIX (St Louis MO) Attending Companies ------------------------------------ Amazon Prime Video ConnectDSM Fiber Minnesota FiberComm Involta Iowa Communication Network Google Fiber Key Cooperative Liberty Communications Mankato Networks Midco MTC Technologies MTC Communications (not the same) Ninja-IX Nuvera Optk OzarksGo RLM Underground SDN Communications Smartsource Consulting South Slope Cooperative Communications Stream Data Centers SupraNet Communications Unite Private Networks Vaultas WIN Winnebago Cooperative Telecom Association -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

1 0

MICE Remote Switch Policy
by Richard Laager May 6, 2022

May 6, 2022

I've had some discussions with the board as well as with Jay and Jeremy on these topics. The board consensus was to bring this (in general) to the membership for more input. As to the specifics, while I know others agree with at least parts of this, I'm only speaking for myself here. I'll let everyone articulate their own positions. (This disclaimer should not be read as me signaling the existance of disagreement either. I just don't want to put words in other people's mouths.) Our current policy on remote switches is here: https://micemn.net/technical.html#remotes It has the proposal presented to the membership for discussion, then the board makes a final decision. Is this decision ministerial or discretionary? That is, if the remote switch proposal checks all the boxes in our policy, is MICE "required" (supposed to) always grant it, or is the board supposed to apply some discretion? If the decision is ministerial, then why bother bringing this to the board (or for that matter, the members) all? Couldn't we save a bunch of time and hassle and simply have management (in some form, whether that's me, Jay, and/or Jeremy) approve it? If the decision is discretionary, are there particular criteria that the board should consider (above and beyond the listed criteria)? One criteria used in a discussion I had (and I can't recall which of us said it first) is "MICE's strategic interests". What would that phrase mean to you; what are some strategic interests of MICE? For a bit of an absurd example for the thought experiment, imagine that someone was proposing a MICE remote switch, but we knew their goal was to attract a bunch of members and then convert that into a competing exchange. Is that something we would have to agree to simply because they met all the objective criteria? When we were new and little, MICE certainly had an interest in making every decision in a way that would maximize additional peering. However, at this point, the calculus may be (I'd argue is) different. We are moving a lot of traffic and are important to our members / in our region. We have to be careful that our decisions do not destabilize the exchange--in multiple ways: technical, financial, or political. Either way, should we expand the list of objective criteria in the policy? Some examples: * We have previously discussed dedicated vs non-dedicated switches. As time goes along, I am more convinced than ever that MICE remote switches should be required to be dedicated. Non-dedicated switches present extra complications for configuration and troubleshooting. (Jeremy has some additional insight on this that he will share.) I think we should make it a requirement that the switch be dedicated. (Perhaps the board could still grant an exception in exceptional cases.) * Should we require that a remote switch have X number of participants committed? And if so, what is X? In my view, it hardly makes sense to have a remote switch one or two participants. They could just as well backhaul to MICE directly. The criteria for allowing new remote switches vs disconnecting existing remotes need not be the same. If we set a minimum of e.g. 5 participants, we don't necessarily need to disconnect existing remotes that don't meet that. And I think the consensus is that we would not, barring them creating some significant problem. How do we feel about far-away remote switches? (This is a live issue in the context of the proposed Kansas City remote.) Some concerns: * At Wiktel, I peer with MN VoIP's far away extensions in Minneapolis. For example, I peer at SeattleIX (SIX) in Minneapolis. This has caused me some issues. For example, latency-sensitive gaming traffic was tromboning Wiktel-Minneapolis-Seattle-Chicago-Seattle-Minneapolis-Wiktel rather than Wiktel-Chicago-Wiktel. * Is it safe to have a broadcast domain that stretches across multiple states (or half a continent, in the SIX case)? * If we take this to its logical extreme... Imagine we had a MICE extension in every datacenter in the U.S. I think that is pretty obviously untenable for a bunch of reasons. Something close to that is actually within the realm of possibility, with some of these virtual extension things that people are doing. (Reid would be able to cite who.) Granted, nobody is proposing that today, but where should we draw the line? * Far-away extensions may reduce the incentive for CDNs to install locally. Some counterpoints: * Nobody is forcing networks to use the far-away remotes. * If people choose to use them, they take their routing into their own hands. They need to understand the tromboning risk and set their own routing policy. o Counter-counterpoint: Do they? Especially smaller / less experienced networks? Have we adequately warned them? o Counter-counterpoint: The existence of these far-away peers doesn't affect just them. It also affects the other networks with which they peer. Everyone on the exchange needs to be aware of the existence of far-away participants and handle their routing policy accordingly. If there are enough far-away peers, this might tip networks into an opt-in route server policy, or even to only do bilaterals. This will disadvantage small participants. * Networks can backhaul into far-away exchanges directly. o Counter-counterpoint: But a remote switch makes this cheaper / more feasible / more common, which is literally the point of creating such a remote switch. * For a local eyeball network in Des Moines, neither MICE nor Kansas City are far-away from me. Even MICE via Kansas City is not likely to be problematic. This might be the only economically feasible way they could peer with Minneapolis content. -- Richard

8 23

MICE Node for AS112
by David Farmer May 5, 2022

May 5, 2022

I'm seeing IPv6 routes from the MICE Node for AS112, but not IPv4 routes This is what the MICE Looking Glass sees; micemn-lg>sh bgp regexp _112_ BGP table version is 103138859, local router ID is 206.108.255.250 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, x best-external, a additional-path, c RIB-compressed, Origin codes: i - IGP, e - EGP, ? - incomplete RPKI validation codes: V valid, I invalid, N Not found Network Next Hop Metric LocPrf Weight Path * 192.31.196.0 206.108.255.52 255 0 6939 112 i *> 206.108.255.52 255 0 6939 112 i * 192.175.48.0 206.108.255.52 255 0 6939 112 i *> 206.108.255.52 255 0 6939 112 i micemn-lg>sh bgp ipv6 uni regexp _112_ BGP table version is 165890313, local router ID is 206.108.255.250 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, x best-external, a additional-path, c RIB-compressed, Origin codes: i - IGP, e - EGP, ? - incomplete RPKI validation codes: V valid, I invalid, N Not found Network Next Hop Metric LocPrf Weight Path *> 2001:4:112::/48 2001:504:27::70:0:1 0 112 i * 2001:504:27::70:0:1 0 112 i *> 2620:4F:8000::/48 2001:504:27::70:0:1 0 112 i * 2001:504:27::70:0:1 0 112 i micemn-lg> -- =============================================== David Farmer Email:farmer@umn.edu Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 ===============================================

2 2

Please welcome LTD Broadband. to MICE!
by Jeremy Lumby May 3, 2022

May 3, 2022

Name: LTD Broadband Switch: South Front Networks Albert Lea Port: E9/1 ASN: 394330 IPv4: This is a port upgrade/move, all IP info remains the same. The new connection will be 100G IPv6: Peering Contact: coreyhauer(a)ltdbroadband.com Route Servers: No BFD: No Welcome Aboard Again! Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 Main: 612-355-7740 Direct: 612-392-6814 Fax: 952-873-7425 jlumby(a)mnvoip.com

1 0

IPv4 BGP session to Google down
by Frank Bulk April 19, 2022

April 19, 2022

Our BGP session bounce several times last might between 10:00 and 10:07 pm Central, and then has been down from 10:21 am. Anyone else seeing this? I see a little drop in the graph (http://micelg.usinternet.com/cacti/graph_view.php?action=tree&tree_id=1&lea…) this morning. Frank Bulk FiberNet (AS18883) Apr 7 10:21:55 207.32.15.14 Rib: %BGP-3-NOTIFICATION: sent to neighbor 206.108.255.167 (AS 15169) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes Apr 7 10:21:55 207.32.15.14 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.167 (AS 15169) old state Established event HoldTime new state Idle Apr 7 10:22:00 207.32.15.14 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.167 (AS 15169) old state OpenConfirm event RecvKeepAlive new state Established Apr 7 10:22:30 207.32.15.14 Rib: %BGP-5-ADJCHANGE: peer 206.108.255.167 (AS 15169) old state Established event HoldTime new state Idle

5 8