- mice-discuss - lists.micemn.net

Re: MICE MAC Address Limit

by Jeffrey Anderson

Yup! Let's do it. Good Policy. -----Original Message----- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Richard Laager Sent: Monday, October 28, 2019 3:25 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] MICE MAC Address Limit We have discussed this a bit in the past, and this came up at the last UG. I'm looking for feedback from the group before formally proposing this to the board for a vote. I first brought this up to the technical committee, CCing the board. I have heard no objections. Jeremy is "in favor of it 100%". Based on our quick conversation after the UG, I _think_ Anthony is in favor as well; we both made notes to follow up on this. ----- I am proposing that MICE change its MAC address limit from 5 MACs per port to 1 MAC per port. 1 MAC address is enough for normal scenarios and this would further limit the potential for problems on the fabric. This restriction is one that SIX and AMS-IX both have, with the latter being known for their excellent configuration guide for participants: https://www.ams-ix.net/ams/documentation/config-guide To be clear, this is still only for end ports, not ports facing remote switches, of course. The remote switches are responsible for enforcing the current MAC limit, and would be responsible for changing those settings as described here. If someone is swapping their MICE-facing router, the resulting port flap on the MICE/remote switch will clear the limit anyway, if they are directly connected. If they are unable to flap the port (e.g. because they have someone else's layer 2 gear in the middle), they will have to either work with that carrier or their switch operator (MICE/remote) to flap the port. This is a non-zero burden, but I think it's pretty minimal in practice. Additionally, if someone plans to make an equipment swap, we would increase the limit in advance, temporarily, upon request. For reference, SIX goes further and locks you to a particular MAC address in an ACL, so you have to contact them to change your MAC. I am not proposing that. Still, I've been through that a couple of times, and even that isn't really too big of a deal. So I don't think the change to a limit of 1 MAC at MICE will be particularly onerous. There are a couple of participants who have two IP address sets (one IPv4 and one IPv6) on the same port. For those ports, the limit obviously must be at least 2 MACs, but I propose 3 MACs as the limit. If they're using two IPs, they probably are doing so to get some redundancy out of it. For this particular use case, needing to flap the port defeats that redundancy goal, as it breaks the other router. Having a MAC limit of 3 would allow them to swap a device without needing to flap the port. We would grandfather these setups until they are looking for a port-type upgrade; at that point they would need to get to one IP address set per port (by splitting into two ports or reducing to one IP address set) or request an exception as outlined below. The exception process would allow us to better evaluate this use case at that time. There are some members who have multiple MACs showing up now, despite using only one device. For those, I am proposing we set the MAC limit to however many MACs are currently showing up on their port. That is, they would be grandfathered at their current situation for now. At the time of this change, we would encourage them to investigate and fix the multiple MACs issue at their leisure. In the future, if they are doing a equipment swap (especially like-for-like), we would again encourage them to address it, but would otherwise leave their higher limit in place. If they do a port-type upgrade, we would not bring the grandfathering over. If they needed, they could ask for an official exception at that time. There may be cases where people cannot reasonably fix their routers to use only 1 MAC, other scenarios I haven't considered, or things that may come up in the future. For that reason, I think we should allow for the possibility of exceptions upon request to the board. In practice, I would expect the board would confer with the technical committee and/or the technical committee would be bringing these to the board anyway. The goal is to strike a reasonable balance between protecting the fabric without preventing networks from connecting. -- Richard

Oct. 29, 2019

Traffic Requirements

by Jeremy Lumby

I am wondering if many MICE members have been denied peering with other networks on MICE because you do not meet their traffic level requirements. I have, and I am brainstorming some options. Let me know off list if you fall into this category, and if you want to explore a potential solution with me. Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 M: 612-355-7740 D: 612-392-6814 F: 952-873-7425 jlumby(a)mnvoip.com

Oct. 29, 2019

MICE MAC Address Limit

by Richard Laager

We have discussed this a bit in the past, and this came up at the last UG. I'm looking for feedback from the group before formally proposing this to the board for a vote. I first brought this up to the technical committee, CCing the board. I have heard no objections. Jeremy is "in favor of it 100%". Based on our quick conversation after the UG, I _think_ Anthony is in favor as well; we both made notes to follow up on this. ----- I am proposing that MICE change its MAC address limit from 5 MACs per port to 1 MAC per port. 1 MAC address is enough for normal scenarios and this would further limit the potential for problems on the fabric. This restriction is one that SIX and AMS-IX both have, with the latter being known for their excellent configuration guide for participants: https://www.ams-ix.net/ams/documentation/config-guide To be clear, this is still only for end ports, not ports facing remote switches, of course. The remote switches are responsible for enforcing the current MAC limit, and would be responsible for changing those settings as described here. If someone is swapping their MICE-facing router, the resulting port flap on the MICE/remote switch will clear the limit anyway, if they are directly connected. If they are unable to flap the port (e.g. because they have someone else's layer 2 gear in the middle), they will have to either work with that carrier or their switch operator (MICE/remote) to flap the port. This is a non-zero burden, but I think it's pretty minimal in practice. Additionally, if someone plans to make an equipment swap, we would increase the limit in advance, temporarily, upon request. For reference, SIX goes further and locks you to a particular MAC address in an ACL, so you have to contact them to change your MAC. I am not proposing that. Still, I've been through that a couple of times, and even that isn't really too big of a deal. So I don't think the change to a limit of 1 MAC at MICE will be particularly onerous. There are a couple of participants who have two IP address sets (one IPv4 and one IPv6) on the same port. For those ports, the limit obviously must be at least 2 MACs, but I propose 3 MACs as the limit. If they're using two IPs, they probably are doing so to get some redundancy out of it. For this particular use case, needing to flap the port defeats that redundancy goal, as it breaks the other router. Having a MAC limit of 3 would allow them to swap a device without needing to flap the port. We would grandfather these setups until they are looking for a port-type upgrade; at that point they would need to get to one IP address set per port (by splitting into two ports or reducing to one IP address set) or request an exception as outlined below. The exception process would allow us to better evaluate this use case at that time. There are some members who have multiple MACs showing up now, despite using only one device. For those, I am proposing we set the MAC limit to however many MACs are currently showing up on their port. That is, they would be grandfathered at their current situation for now. At the time of this change, we would encourage them to investigate and fix the multiple MACs issue at their leisure. In the future, if they are doing a equipment swap (especially like-for-like), we would again encourage them to address it, but would otherwise leave their higher limit in place. If they do a port-type upgrade, we would not bring the grandfathering over. If they needed, they could ask for an official exception at that time. There may be cases where people cannot reasonably fix their routers to use only 1 MAC, other scenarios I haven't considered, or things that may come up in the future. For that reason, I think we should allow for the possibility of exceptions upon request to the board. In practice, I would expect the board would confer with the technical committee and/or the technical committee would be bringing these to the board anyway. The goal is to strike a reasonable balance between protecting the fabric without preventing networks from connecting. -- Richard

Oct. 28, 2019

Re: Please welcome Apple to MICE!

by Jeremy Lumby

Yes, I will do the optic swap. -----Original Message----- From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Doug McIntyre Sent: Friday, October 25, 2019 2:47 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Please welcome Apple to MICE! On Fri, Oct 25, 2019 at 12:28:23PM -0500, Jay Hanke wrote: >Name: Apple >Ports: Ethernet 3/14/1 (100G LAG #1), Ethernet 4/49/1 (100G LAG#2) >ASN: 714 >(LAG Group #1) >IPv4: 206.108.255.146/24 >IPv6: 2001:504:27:0:0:2ca::1/64 >(LAG Group #2) >IPv4: 206.108.255.147/24 >IPv6: 2001:504:27:0:0:2ca::2/64 >Peering Contact: peering(a)group.apple.com >Route Servers: NO >BFD: NO > >Please note: Ethernet 4/49/1 has been reclaimed from a member that >failed to migrate and did not respond. But Ether 4/49/1 is lit right now, albiet not passing data. I assume Jeremy will unhook them and their optic, and insert the new optic at some point? Otherwise, Port-Channel's 30 & 31 setup for these LAG groups. Port 3/14/1 configured, but I left 4/49/1 alone for now. -- Doug McIntyre <merlyn(a)iphouse.net> ~.~ ipHouse ~.~ Network Engineer/Provisioning/Jack of all Trades

Oct. 25, 2019

Re: Please welcome Apple to MICE!

by Doug McIntyre

On Fri, Oct 25, 2019 at 12:28:23PM -0500, Jay Hanke wrote: >Name: Apple >Ports: Ethernet 3/14/1 (100G LAG #1), Ethernet 4/49/1 (100G LAG#2) >ASN: 714 >(LAG Group #1) >IPv4: 206.108.255.146/24 >IPv6: 2001:504:27:0:0:2ca::1/64 >(LAG Group #2) >IPv4: 206.108.255.147/24 >IPv6: 2001:504:27:0:0:2ca::2/64 >Peering Contact: peering(a)group.apple.com >Route Servers: NO >BFD: NO > >Please note: Ethernet 4/49/1 has been reclaimed from a member that >failed to migrate and did not respond. But Ether 4/49/1 is lit right now, albiet not passing data. I assume Jeremy will unhook them and their optic, and insert the new optic at some point? Otherwise, Port-Channel's 30 & 31 setup for these LAG groups. Port 3/14/1 configured, but I left 4/49/1 alone for now. -- Doug McIntyre <merlyn(a)iphouse.net> ~.~ ipHouse ~.~ Network Engineer/Provisioning/Jack of all Trades

Oct. 25, 2019

Please welcome Apple to MICE!

by Jay Hanke

Name: Apple Ports: Ethernet 3/14/1 (100G LAG #1), Ethernet 4/49/1 (100G LAG#2) ASN: 714 (LAG Group #1) IPv4: 206.108.255.146/24 IPv6: 2001:504:27:0:0:2ca::1/64 (LAG Group #2) IPv4: 206.108.255.147/24 IPv6: 2001:504:27:0:0:2ca::2/64 Peering Contact: peering(a)group.apple.com Route Servers: NO BFD: NO Please note: Ethernet 4/49/1 has been reclaimed from a member that failed to migrate and did not respond. Welcome aboard! -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

Oct. 25, 2019

Re: Zayo to Omaha IX down

by Frank Bulk

Our 10G link was up the first time at 11:32 am, and then bounced at 12:43 pm, and has been stable since then. Frank -----Original Message----- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Jeff Dahna Sent: Thursday, October 24, 2019 8:12 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Zayo to Omaha IX down 288 ct fiber damaged near Worthington, MN. Zayo ticket 0646hrs - Current Update: Crews are finishing boring on one end and should have the fiber tails in the trailer in about an hour. On the other side they have fiber tails in the trailer beginning to prep.

Oct. 24, 2019

Re: Zayo to Omaha IX down

by Jeff Dahna

288 ct fiber damaged near Worthington, MN. Zayo ticket 0646hrs - Current Update: Crews are finishing boring on one end and should have the fiber tails in the trailer in about an hour. On the other side they have fiber tails in the trailer beginning to prep.

Oct. 24, 2019

Re: Zayo to Omaha IX down

by Frank Bulk

I think this one: TTN-0003647122 Frank -----Original Message----- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Jay Hanke Sent: Wednesday, October 23, 2019 6:50 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Zayo to Omaha IX down Is there a master Zayo ticket open for the MICE ports that are down? On Wed, Oct 23, 2019 at 2:50 PM Jay Hanke <jayhanke(a)southfront.io> wrote: > > Looks like several of the Nebraska folks lost their MICE connections as well. > > On Wed, Oct 23, 2019 at 2:41 PM Jeff Dahna <jeff.dahna(a)windommn.com> wrote: > > > > We lost our route to Omaha IX via Zayo around 2:04 PM CST. Got a ticket in with Zayo. Anyone else down to Omaha via Zayo? > > > > Jeff Dahna > > Windomnet > > > > -- > Jay Hanke, President > South Front Networks > jayhanke(a)southfront.io > Phone 612-204-0000 -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

Oct. 24, 2019

Re: Zayo to Omaha IX down

by Justin Huffman

Our link from Sioux City to Minneapolis went down around 2:10 today. This is getting out of hand. I think we've experienced outages every other week for almost 2 months now along with the outage lasting 16+ hours. Does anyone have recommendations for diverse routes up to 511? Justin Long Lines Broadband

Oct. 23, 2019