October 2022 - mice-discuss - lists.micemn.net

Re: Proposal: MICE Static MAC address Only

by Jeremy Lumby

I have mixed feelings about giving IXP manager access to the switch. The main reason is that if a member is proactive they can request the second MAC ahead of time, and it can be temporarily added for the transition. As it is most requests are processed in a couple hours. IXP manager is a program that none of the volunteers have a great grasp on, and lack of a timely security update on a portal that is made available to the entire internet in an invitation to get hacked. If it has direct access to the switch things can go wrong quickly. While I am concerned as it is about IXP manager having access to the route servers, they are an extra and not 100% necessary for the IX to function. -----Original Message----- From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Jay Hanke Sent: Monday, October 31, 2022 9:05 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Proposal: MICE Static MAC address Only The error disable takes 1-2 seconds to clamp down and disable the port. This results in a short, but severe, broadcast storm. The churn rate on members changing mac addresses is fairly low. I agree that we should automate the process so it can be done via self-service by the members. On Mon, Oct 31, 2022 at 8:59 AM Steve Howard <showard(a)paulbunyan.net> wrote: > > > Below is a message indicating the current policy as decided by the MICE Board in December 2019 is that the MAC address limit is 1. > > > The subject of Jay's message indicates "Static" MAC, but, the text seems to indicate a limit of 1. I like the 1 address limit, but, am not a fan of Static. Was the current issue caused by somebody not having a proper MAC address limit installed? > > > On 12/13/19 18:00, Richard Laager wrote: > > The board has approved the change in MAC address limit from 5 to 1. > > NOTE: This is orthogonal to the dedicated remote switch question, which > is still pending. Non-dedicated remote switches can, for example, > enforce this per-VLAN. > > We have already confirmed that most participants are using only 1 MAC > address, and some that were using more than 1 have addressed that after > being contacted. As discussed below, this change will be rolled out in a > way to keep everyone working, by grandfathering if necessary. > > On 10/28/19 3:25 PM, Richard Laager wrote: > > We have discussed this a bit in the past, and this came up at the last > UG. I'm looking for feedback from the group before formally proposing > this to the board for a vote. > > I first brought this up to the technical committee, CCing the board. I > have heard no objections. Jeremy is "in favor of it 100%". > > Based on our quick conversation after the UG, I _think_ Anthony is in > favor as well; we both made notes to follow up on this. > > ----- > > I am proposing that MICE change its MAC address limit from 5 MACs per > port to 1 MAC per port. 1 MAC address is enough for normal scenarios and > this would further limit the potential for problems on the fabric. This > restriction is one that SIX and AMS-IX both have, with the latter being > known for their excellent configuration guide for participants: > https://www.ams-ix.net/ams/documentation/config-guide > > To be clear, this is still only for end ports, not ports facing remote > switches, of course. The remote switches are responsible for enforcing > the current MAC limit, and would be responsible for changing those > settings as described here. > > If someone is swapping their MICE-facing router, the resulting port flap > on the MICE/remote switch will clear the limit anyway, if they are > directly connected. If they are unable to flap the port (e.g. because > they have someone else's layer 2 gear in the middle), they will have to > either work with that carrier or their switch operator (MICE/remote) to > flap the port. This is a non-zero burden, but I think it's pretty > minimal in practice. Additionally, if someone plans to make an equipment > swap, we would increase the limit in advance, temporarily, upon request. > > For reference, SIX goes further and locks you to a particular MAC > address in an ACL, so you have to contact them to change your MAC. I am > not proposing that. Still, I've been through that a couple of times, and > even that isn't really too big of a deal. So I don't think the change to > a limit of 1 MAC at MICE will be particularly onerous. > > There are a couple of participants who have two IP address sets (one > IPv4 and one IPv6) on the same port. For those ports, the limit > obviously must be at least 2 MACs, but I propose 3 MACs as the limit. If > they're using two IPs, they probably are doing so to get some redundancy > out of it. For this particular use case, needing to flap the port > defeats that redundancy goal, as it breaks the other router. Having a > MAC limit of 3 would allow them to swap a device without needing to flap > the port. We would grandfather these setups until they are looking for a > port-type upgrade; at that point they would need to get to one IP > address set per port (by splitting into two ports or reducing to one IP > address set) or request an exception as outlined below. The exception > process would allow us to better evaluate this use case at that time. > > There are some members who have multiple MACs showing up now, despite > using only one device. For those, I am proposing we set the MAC limit to > however many MACs are currently showing up on their port. That is, they > would be grandfathered at their current situation for now. At the time > of this change, we would encourage them to investigate and fix the > multiple MACs issue at their leisure. In the future, if they are doing a > equipment swap (especially like-for-like), we would again encourage them > to address it, but would otherwise leave their higher limit in place. If > they do a port-type upgrade, we would not bring the grandfathering over. > If they needed, they could ask for an official exception at that time. > > There may be cases where people cannot reasonably fix their routers to > use only 1 MAC, other scenarios I haven't considered, or things that may > come up in the future. For that reason, I think we should allow for the > possibility of exceptions upon request to the board. In practice, I > would expect the board would confer with the technical committee and/or > the technical committee would be bringing these to the board anyway. The > goal is to strike a reasonable balance between protecting the fabric > without preventing networks from connecting. > > > > > On 10/31/22 08:27, Jay Hanke wrote: > > I propose we change the MICE required port security settings to be > hard filtered for a single static mac address per port. > > Over the years, we've had several incidents where floods have occurred > prior to port being error-disabled during a looping event. > > > > ________________________________ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

Oct. 31, 2022

Re: Proposal: MICE Static MAC address Only

by Chris Wopat

On 10/31/22 08:59, Steve Howard wrote: > > Below is a message indicating the current policy as decided by the MICE > Board in December 2019 is that the MAC address limit is 1. > > > The subject of Jay's message indicates "Static" MAC, but, the text seems > to indicate a limit of 1. I like the 1 address limit, but, am not a fan > of Static. Was the current issue caused by somebody not having a proper > MAC address limit installed? > Quick note to consider. Yes, there should be a mac limit. Certain vendor hardware may restrict the way this is done - as in, some vendor kit may allow one to set a mac limit of 1 (to anything) and some may only permit static mac, and some may permit both. WiscNet's remote switch is currently set to static mac, this is because of a vendor change that prevented the mac limit from working. Forcing one way or the other may be difficult to achieve, but the limit should exist in some form. -- Chris Wopat Network Engineer, WiscNet wopat(a)wiscnet.net 608-210-3965

Oct. 31, 2022

Re: [EXTERNAL] Re: [MICE-DISCUSS] Proposal: MICE Static MAC address Only

by Jay Hanke

No, the recent event was on the main switch. Historically, this has happened on most (if not all) of the MICE switches. On Mon, Oct 31, 2022 at 9:07 AM Kittel, Brady <Brady.Kittel(a)hcmed.org> wrote: > > Was the source of the issue a remote switch? > > " > > To be clear, this is still only for end ports, not ports facing remote > switches, of course. The remote switches are responsible for enforcing > the current MAC limit, and would be responsible for changing those > settings as described here." > > > -Brady > ________________________________ > From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> on behalf of Jay Hanke <jayhanke(a)SOUTHFRONT.IO> > Sent: Monday, October 31, 2022 9:05:08 AM > To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> > Subject: [EXTERNAL] Re: [MICE-DISCUSS] Proposal: MICE Static MAC address Only > > CAUTION: This email originated from outside of HCMC. DO NOT CLICK links or open attachments unless you recognize the sender and know the content is safe. > > > The error disable takes 1-2 seconds to clamp down and disable the > port. This results in a short, but severe, broadcast storm. > > The churn rate on members changing mac addresses is fairly low. > > I agree that we should automate the process so it can be done via > self-service by the members. > > On Mon, Oct 31, 2022 at 8:59 AM Steve Howard <showard(a)paulbunyan.net> wrote: > > > > > > Below is a message indicating the current policy as decided by the MICE Board in December 2019 is that the MAC address limit is 1. > > > > > > The subject of Jay's message indicates "Static" MAC, but, the text seems to indicate a limit of 1. I like the 1 address limit, but, am not a fan of Static. Was the current issue caused by somebody not having a proper MAC address limit installed? > > > > > > On 12/13/19 18:00, Richard Laager wrote: > > > > The board has approved the change in MAC address limit from 5 to 1. > > > > NOTE: This is orthogonal to the dedicated remote switch question, which > > is still pending. Non-dedicated remote switches can, for example, > > enforce this per-VLAN. > > > > We have already confirmed that most participants are using only 1 MAC > > address, and some that were using more than 1 have addressed that after > > being contacted. As discussed below, this change will be rolled out in a > > way to keep everyone working, by grandfathering if necessary. > > > > On 10/28/19 3:25 PM, Richard Laager wrote: > > > > We have discussed this a bit in the past, and this came up at the last > > UG. I'm looking for feedback from the group before formally proposing > > this to the board for a vote. > > > > I first brought this up to the technical committee, CCing the board. I > > have heard no objections. Jeremy is "in favor of it 100%". > > > > Based on our quick conversation after the UG, I _think_ Anthony is in > > favor as well; we both made notes to follow up on this. > > > > ----- > > > > I am proposing that MICE change its MAC address limit from 5 MACs per > > port to 1 MAC per port. 1 MAC address is enough for normal scenarios and > > this would further limit the potential for problems on the fabric. This > > restriction is one that SIX and AMS-IX both have, with the latter being > > known for their excellent configuration guide for participants: > > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ams-i… > > > > To be clear, this is still only for end ports, not ports facing remote > > switches, of course. The remote switches are responsible for enforcing > > the current MAC limit, and would be responsible for changing those > > settings as described here. > > > > If someone is swapping their MICE-facing router, the resulting port flap > > on the MICE/remote switch will clear the limit anyway, if they are > > directly connected. If they are unable to flap the port (e.g. because > > they have someone else's layer 2 gear in the middle), they will have to > > either work with that carrier or their switch operator (MICE/remote) to > > flap the port. This is a non-zero burden, but I think it's pretty > > minimal in practice. Additionally, if someone plans to make an equipment > > swap, we would increase the limit in advance, temporarily, upon request. > > > > For reference, SIX goes further and locks you to a particular MAC > > address in an ACL, so you have to contact them to change your MAC. I am > > not proposing that. Still, I've been through that a couple of times, and > > even that isn't really too big of a deal. So I don't think the change to > > a limit of 1 MAC at MICE will be particularly onerous. > > > > There are a couple of participants who have two IP address sets (one > > IPv4 and one IPv6) on the same port. For those ports, the limit > > obviously must be at least 2 MACs, but I propose 3 MACs as the limit. If > > they're using two IPs, they probably are doing so to get some redundancy > > out of it. For this particular use case, needing to flap the port > > defeats that redundancy goal, as it breaks the other router. Having a > > MAC limit of 3 would allow them to swap a device without needing to flap > > the port. We would grandfather these setups until they are looking for a > > port-type upgrade; at that point they would need to get to one IP > > address set per port (by splitting into two ports or reducing to one IP > > address set) or request an exception as outlined below. The exception > > process would allow us to better evaluate this use case at that time. > > > > There are some members who have multiple MACs showing up now, despite > > using only one device. For those, I am proposing we set the MAC limit to > > however many MACs are currently showing up on their port. That is, they > > would be grandfathered at their current situation for now. At the time > > of this change, we would encourage them to investigate and fix the > > multiple MACs issue at their leisure. In the future, if they are doing a > > equipment swap (especially like-for-like), we would again encourage them > > to address it, but would otherwise leave their higher limit in place. If > > they do a port-type upgrade, we would not bring the grandfathering over. > > If they needed, they could ask for an official exception at that time. > > > > There may be cases where people cannot reasonably fix their routers to > > use only 1 MAC, other scenarios I haven't considered, or things that may > > come up in the future. For that reason, I think we should allow for the > > possibility of exceptions upon request to the board. In practice, I > > would expect the board would confer with the technical committee and/or > > the technical committee would be bringing these to the board anyway. The > > goal is to strike a reasonable balance between protecting the fabric > > without preventing networks from connecting. > > > > > > > > > > On 10/31/22 08:27, Jay Hanke wrote: > > > > I propose we change the MICE required port security settings to be > > hard filtered for a single static mac address per port. > > > > Over the years, we've had several incidents where floods have occurred > > prior to port being error-disabled during a looping event. > > > > > > > > ________________________________ > > > > To unsubscribe from the MICE-DISCUSS list, click the following link: > > https://nam02.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.ipho… > > > > -- > Jay Hanke, President > South Front Networks > jayhanke(a)southfront.io > Phone 612-204-0000 > > Confidentiality Notice: > > Information contained in this e-mail is being sent to you after appropriate authorization or by legal exception. You are obligated to maintain it in a safe, secure and confidential manner. Re-disclosure without patient consent or as permitted by law is prohibited and may subject you to state and/or federal penalties. This information may also be legally privileged, the disclosure of which is governed by law. This information is intended for the use of the person or entity to which it is addressed. If you are not the intended recipient, or the employee or agent responsible for delivering it to the intended recipient, you are hereby notified that any access, disclosure, copying or distribution of this information is strictly prohibited. If you have received this message by error, please notify the sender immediately to arrange for return or proof of destruction of the information contained in this message. > > > ________________________________ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

Oct. 31, 2022

Re: [EXTERNAL] Re: [MICE-DISCUSS] Proposal: MICE Static MAC address Only

by Jeremy Lumby

The issue was a member port. The port security took 2 seconds to shutdown the port, however 100G of traffic looped crashed the forwarding table on the main switch, forcing manual intervention. A static MAC ACL on each port would not have taken time to react. This is what the SIX has been doing on their ports since 2010 (as well as multiple other layers of security). From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Kittel, Brady Sent: Monday, October 31, 2022 9:07 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] [EXTERNAL] Re: [MICE-DISCUSS] Proposal: MICE Static MAC address Only Was the source of the issue a remote switch? " To be clear, this is still only for end ports, not ports facing remoteswitches, of course. The remote switches are responsible for enforcingthe current MAC limit, and would be responsible for changing thosesettings as described here." -Brady ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> on behalf of Jay Hanke <jayhanke(a)SOUTHFRONT.IO> Sent: Monday, October 31, 2022 9:05:08 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> Subject: [EXTERNAL] Re: [MICE-DISCUSS] Proposal: MICE Static MAC address Only CAUTION: This email originated from outside of HCMC. DO NOT CLICK links or open attachments unless you recognize the sender and know the content is safe. The error disable takes 1-2 seconds to clamp down and disable the port. This results in a short, but severe, broadcast storm. The churn rate on members changing mac addresses is fairly low. I agree that we should automate the process so it can be done via self-service by the members. On Mon, Oct 31, 2022 at 8:59 AM Steve Howard <showard(a)paulbunyan.net> wrote: > > > Below is a message indicating the current policy as decided by the MICE Board in December 2019 is that the MAC address limit is 1. > > > The subject of Jay's message indicates "Static" MAC, but, the text seems to indicate a limit of 1. I like the 1 address limit, but, am not a fan of Static. Was the current issue caused by somebody not having a proper MAC address limit installed? > > > On 12/13/19 18:00, Richard Laager wrote: > > The board has approved the change in MAC address limit from 5 to 1. > > NOTE: This is orthogonal to the dedicated remote switch question, which > is still pending. Non-dedicated remote switches can, for example, > enforce this per-VLAN. > > We have already confirmed that most participants are using only 1 MAC > address, and some that were using more than 1 have addressed that after > being contacted. As discussed below, this change will be rolled out in a > way to keep everyone working, by grandfathering if necessary. > > On 10/28/19 3:25 PM, Richard Laager wrote: > > We have discussed this a bit in the past, and this came up at the last > UG. I'm looking for feedback from the group before formally proposing > this to the board for a vote. > > I first brought this up to the technical committee, CCing the board. I > have heard no objections. Jeremy is "in favor of it 100%". > > Based on our quick conversation after the UG, I _think_ Anthony is in > favor as well; we both made notes to follow up on this. > > ----- > > I am proposing that MICE change its MAC address limit from 5 MACs per > port to 1 MAC per port. 1 MAC address is enough for normal scenarios and > this would further limit the potential for problems on the fabric. This > restriction is one that SIX and AMS-IX both have, with the latter being > known for their excellent configuration guide for participants: >https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ams-i… > > To be clear, this is still only for end ports, not ports facing remote > switches, of course. The remote switches are responsible for enforcing > the current MAC limit, and would be responsible for changing those > settings as described here. > > If someone is swapping their MICE-facing router, the resulting port flap > on the MICE/remote switch will clear the limit anyway, if they are > directly connected. If they are unable to flap the port (e.g. because > they have someone else's layer 2 gear in the middle), they will have to > either work with that carrier or their switch operator (MICE/remote) to > flap the port. This is a non-zero burden, but I think it's pretty > minimal in practice. Additionally, if someone plans to make an equipment > swap, we would increase the limit in advance, temporarily, upon request. > > For reference, SIX goes further and locks you to a particular MAC > address in an ACL, so you have to contact them to change your MAC. I am > not proposing that. Still, I've been through that a couple of times, and > even that isn't really too big of a deal. So I don't think the change to > a limit of 1 MAC at MICE will be particularly onerous. > > There are a couple of participants who have two IP address sets (one > IPv4 and one IPv6) on the same port. For those ports, the limit > obviously must be at least 2 MACs, but I propose 3 MACs as the limit. If > they're using two IPs, they probably are doing so to get some redundancy > out of it. For this particular use case, needing to flap the port > defeats that redundancy goal, as it breaks the other router. Having a > MAC limit of 3 would allow them to swap a device without needing to flap > the port. We would grandfather these setups until they are looking for a > port-type upgrade; at that point they would need to get to one IP > address set per port (by splitting into two ports or reducing to one IP > address set) or request an exception as outlined below. The exception > process would allow us to better evaluate this use case at that time. > > There are some members who have multiple MACs showing up now, despite > using only one device. For those, I am proposing we set the MAC limit to > however many MACs are currently showing up on their port. That is, they > would be grandfathered at their current situation for now. At the time > of this change, we would encourage them to investigate and fix the > multiple MACs issue at their leisure. In the future, if they are doing a > equipment swap (especially like-for-like), we would again encourage them > to address it, but would otherwise leave their higher limit in place. If > they do a port-type upgrade, we would not bring the grandfathering over. > If they needed, they could ask for an official exception at that time. > > There may be cases where people cannot reasonably fix their routers to > use only 1 MAC, other scenarios I haven't considered, or things that may > come up in the future. For that reason, I think we should allow for the > possibility of exceptions upon request to the board. In practice, I > would expect the board would confer with the technical committee and/or > the technical committee would be bringing these to the board anyway. The > goal is to strike a reasonable balance between protecting the fabric > without preventing networks from connecting. > > > > > On 10/31/22 08:27, Jay Hanke wrote: > > I propose we change the MICE required port security settings to be > hard filtered for a single static mac address per port. > > Over the years, we've had several incidents where floods have occurred > prior to port being error-disabled during a looping event. > > > > ________________________________ > > To unsubscribe from the MICE-DISCUSS list, click the following link: >https://nam02.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.ipho… -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000 Confidentiality Notice: Information contained in this e-mail is being sent to you after appropriate authorization or by legal exception. You are obligated to maintain it in a safe, secure and confidential manner. Re-disclosure without patient consent or as permitted by law is prohibited and may subject you to state and/or federal penalties. This information may also be legally privileged, the disclosure of which is governed by law. This information is intended for the use of the person or entity to which it is addressed. If you are not the intended recipient, or the employee or agent responsible for delivering it to the intended recipient, you are hereby notified that any access, disclosure, copying or distribution of this information is strictly prohibited. If you have received this message by error, please notify the sender immediately to arrange for return or proof of destruction of the information contained in this message. ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Oct. 31, 2022

Re: [EXTERNAL] Re: [MICE-DISCUSS] Proposal: MICE Static MAC address Only

by Kittel, Brady

Was the source of the issue a remote switch? " To be clear, this is still only for end ports, not ports facing remote switches, of course. The remote switches are responsible for enforcing the current MAC limit, and would be responsible for changing those settings as described here." -Brady ________________________________ From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> on behalf of Jay Hanke <jayhanke(a)SOUTHFRONT.IO> Sent: Monday, October 31, 2022 9:05:08 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> Subject: [EXTERNAL] Re: [MICE-DISCUSS] Proposal: MICE Static MAC address Only CAUTION: This email originated from outside of HCMC. DO NOT CLICK links or open attachments unless you recognize the sender and know the content is safe. The error disable takes 1-2 seconds to clamp down and disable the port. This results in a short, but severe, broadcast storm. The churn rate on members changing mac addresses is fairly low. I agree that we should automate the process so it can be done via self-service by the members. On Mon, Oct 31, 2022 at 8:59 AM Steve Howard <showard(a)paulbunyan.net> wrote: > > > Below is a message indicating the current policy as decided by the MICE Board in December 2019 is that the MAC address limit is 1. > > > The subject of Jay's message indicates "Static" MAC, but, the text seems to indicate a limit of 1. I like the 1 address limit, but, am not a fan of Static. Was the current issue caused by somebody not having a proper MAC address limit installed? > > > On 12/13/19 18:00, Richard Laager wrote: > > The board has approved the change in MAC address limit from 5 to 1. > > NOTE: This is orthogonal to the dedicated remote switch question, which > is still pending. Non-dedicated remote switches can, for example, > enforce this per-VLAN. > > We have already confirmed that most participants are using only 1 MAC > address, and some that were using more than 1 have addressed that after > being contacted. As discussed below, this change will be rolled out in a > way to keep everyone working, by grandfathering if necessary. > > On 10/28/19 3:25 PM, Richard Laager wrote: > > We have discussed this a bit in the past, and this came up at the last > UG. I'm looking for feedback from the group before formally proposing > this to the board for a vote. > > I first brought this up to the technical committee, CCing the board. I > have heard no objections. Jeremy is "in favor of it 100%". > > Based on our quick conversation after the UG, I _think_ Anthony is in > favor as well; we both made notes to follow up on this. > > ----- > > I am proposing that MICE change its MAC address limit from 5 MACs per > port to 1 MAC per port. 1 MAC address is enough for normal scenarios and > this would further limit the potential for problems on the fabric. This > restriction is one that SIX and AMS-IX both have, with the latter being > known for their excellent configuration guide for participants: > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ams-i… > > To be clear, this is still only for end ports, not ports facing remote > switches, of course. The remote switches are responsible for enforcing > the current MAC limit, and would be responsible for changing those > settings as described here. > > If someone is swapping their MICE-facing router, the resulting port flap > on the MICE/remote switch will clear the limit anyway, if they are > directly connected. If they are unable to flap the port (e.g. because > they have someone else's layer 2 gear in the middle), they will have to > either work with that carrier or their switch operator (MICE/remote) to > flap the port. This is a non-zero burden, but I think it's pretty > minimal in practice. Additionally, if someone plans to make an equipment > swap, we would increase the limit in advance, temporarily, upon request. > > For reference, SIX goes further and locks you to a particular MAC > address in an ACL, so you have to contact them to change your MAC. I am > not proposing that. Still, I've been through that a couple of times, and > even that isn't really too big of a deal. So I don't think the change to > a limit of 1 MAC at MICE will be particularly onerous. > > There are a couple of participants who have two IP address sets (one > IPv4 and one IPv6) on the same port. For those ports, the limit > obviously must be at least 2 MACs, but I propose 3 MACs as the limit. If > they're using two IPs, they probably are doing so to get some redundancy > out of it. For this particular use case, needing to flap the port > defeats that redundancy goal, as it breaks the other router. Having a > MAC limit of 3 would allow them to swap a device without needing to flap > the port. We would grandfather these setups until they are looking for a > port-type upgrade; at that point they would need to get to one IP > address set per port (by splitting into two ports or reducing to one IP > address set) or request an exception as outlined below. The exception > process would allow us to better evaluate this use case at that time. > > There are some members who have multiple MACs showing up now, despite > using only one device. For those, I am proposing we set the MAC limit to > however many MACs are currently showing up on their port. That is, they > would be grandfathered at their current situation for now. At the time > of this change, we would encourage them to investigate and fix the > multiple MACs issue at their leisure. In the future, if they are doing a > equipment swap (especially like-for-like), we would again encourage them > to address it, but would otherwise leave their higher limit in place. If > they do a port-type upgrade, we would not bring the grandfathering over. > If they needed, they could ask for an official exception at that time. > > There may be cases where people cannot reasonably fix their routers to > use only 1 MAC, other scenarios I haven't considered, or things that may > come up in the future. For that reason, I think we should allow for the > possibility of exceptions upon request to the board. In practice, I > would expect the board would confer with the technical committee and/or > the technical committee would be bringing these to the board anyway. The > goal is to strike a reasonable balance between protecting the fabric > without preventing networks from connecting. > > > > > On 10/31/22 08:27, Jay Hanke wrote: > > I propose we change the MICE required port security settings to be > hard filtered for a single static mac address per port. > > Over the years, we've had several incidents where floods have occurred > prior to port being error-disabled during a looping event. > > > > ________________________________ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > https://nam02.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.ipho… -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

Oct. 31, 2022

Re: Proposal: MICE Static MAC address Only

by Jay Hanke

The error disable takes 1-2 seconds to clamp down and disable the port. This results in a short, but severe, broadcast storm. The churn rate on members changing mac addresses is fairly low. I agree that we should automate the process so it can be done via self-service by the members. On Mon, Oct 31, 2022 at 8:59 AM Steve Howard <showard(a)paulbunyan.net> wrote: > > > Below is a message indicating the current policy as decided by the MICE Board in December 2019 is that the MAC address limit is 1. > > > The subject of Jay's message indicates "Static" MAC, but, the text seems to indicate a limit of 1. I like the 1 address limit, but, am not a fan of Static. Was the current issue caused by somebody not having a proper MAC address limit installed? > > > On 12/13/19 18:00, Richard Laager wrote: > > The board has approved the change in MAC address limit from 5 to 1. > > NOTE: This is orthogonal to the dedicated remote switch question, which > is still pending. Non-dedicated remote switches can, for example, > enforce this per-VLAN. > > We have already confirmed that most participants are using only 1 MAC > address, and some that were using more than 1 have addressed that after > being contacted. As discussed below, this change will be rolled out in a > way to keep everyone working, by grandfathering if necessary. > > On 10/28/19 3:25 PM, Richard Laager wrote: > > We have discussed this a bit in the past, and this came up at the last > UG. I'm looking for feedback from the group before formally proposing > this to the board for a vote. > > I first brought this up to the technical committee, CCing the board. I > have heard no objections. Jeremy is "in favor of it 100%". > > Based on our quick conversation after the UG, I _think_ Anthony is in > favor as well; we both made notes to follow up on this. > > ----- > > I am proposing that MICE change its MAC address limit from 5 MACs per > port to 1 MAC per port. 1 MAC address is enough for normal scenarios and > this would further limit the potential for problems on the fabric. This > restriction is one that SIX and AMS-IX both have, with the latter being > known for their excellent configuration guide for participants: > https://www.ams-ix.net/ams/documentation/config-guide > > To be clear, this is still only for end ports, not ports facing remote > switches, of course. The remote switches are responsible for enforcing > the current MAC limit, and would be responsible for changing those > settings as described here. > > If someone is swapping their MICE-facing router, the resulting port flap > on the MICE/remote switch will clear the limit anyway, if they are > directly connected. If they are unable to flap the port (e.g. because > they have someone else's layer 2 gear in the middle), they will have to > either work with that carrier or their switch operator (MICE/remote) to > flap the port. This is a non-zero burden, but I think it's pretty > minimal in practice. Additionally, if someone plans to make an equipment > swap, we would increase the limit in advance, temporarily, upon request. > > For reference, SIX goes further and locks you to a particular MAC > address in an ACL, so you have to contact them to change your MAC. I am > not proposing that. Still, I've been through that a couple of times, and > even that isn't really too big of a deal. So I don't think the change to > a limit of 1 MAC at MICE will be particularly onerous. > > There are a couple of participants who have two IP address sets (one > IPv4 and one IPv6) on the same port. For those ports, the limit > obviously must be at least 2 MACs, but I propose 3 MACs as the limit. If > they're using two IPs, they probably are doing so to get some redundancy > out of it. For this particular use case, needing to flap the port > defeats that redundancy goal, as it breaks the other router. Having a > MAC limit of 3 would allow them to swap a device without needing to flap > the port. We would grandfather these setups until they are looking for a > port-type upgrade; at that point they would need to get to one IP > address set per port (by splitting into two ports or reducing to one IP > address set) or request an exception as outlined below. The exception > process would allow us to better evaluate this use case at that time. > > There are some members who have multiple MACs showing up now, despite > using only one device. For those, I am proposing we set the MAC limit to > however many MACs are currently showing up on their port. That is, they > would be grandfathered at their current situation for now. At the time > of this change, we would encourage them to investigate and fix the > multiple MACs issue at their leisure. In the future, if they are doing a > equipment swap (especially like-for-like), we would again encourage them > to address it, but would otherwise leave their higher limit in place. If > they do a port-type upgrade, we would not bring the grandfathering over. > If they needed, they could ask for an official exception at that time. > > There may be cases where people cannot reasonably fix their routers to > use only 1 MAC, other scenarios I haven't considered, or things that may > come up in the future. For that reason, I think we should allow for the > possibility of exceptions upon request to the board. In practice, I > would expect the board would confer with the technical committee and/or > the technical committee would be bringing these to the board anyway. The > goal is to strike a reasonable balance between protecting the fabric > without preventing networks from connecting. > > > > > On 10/31/22 08:27, Jay Hanke wrote: > > I propose we change the MICE required port security settings to be > hard filtered for a single static mac address per port. > > Over the years, we've had several incidents where floods have occurred > prior to port being error-disabled during a looping event. > > > > ________________________________ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

Oct. 31, 2022

Re: Proposal: MICE Static MAC address Only

by Andrew Hoyos

I disagree with static MAC address filtering unless we beef up operational ability to change/adjust in a reasonable timeframe, or offer a self service portal to do so (ie IXP Manager implementation first) Sent from my iPhone > On Oct 31, 2022, at 9:59 PM, Steve Howard <showard(a)paulbunyan.net> wrote: > > > Below is a message indicating the current policy as decided by the MICE Board in December 2019 is that the MAC address limit is 1. > > > The subject of Jay's message indicates "Static" MAC, but, the text seems to indicate a limit of 1. I like the 1 address limit, but, am not a fan of Static. Was the current issue caused by somebody not having a proper MAC address limit installed? > > > On 12/13/19 18:00, Richard Laager wrote: >> The board has approved the change in MAC address limit from 5 to 1. >> >> NOTE: This is orthogonal to the dedicated remote switch question, which >> is still pending. Non-dedicated remote switches can, for example, >> enforce this per-VLAN. >> >> We have already confirmed that most participants are using only 1 MAC >> address, and some that were using more than 1 have addressed that after >> being contacted. As discussed below, this change will be rolled out in a >> way to keep everyone working, by grandfathering if necessary. >> >> On 10/28/19 3:25 PM, Richard Laager wrote: >>> We have discussed this a bit in the past, and this came up at the last >>> UG. I'm looking for feedback from the group before formally proposing >>> this to the board for a vote. >>> >>> I first brought this up to the technical committee, CCing the board. I >>> have heard no objections. Jeremy is "in favor of it 100%". >>> >>> Based on our quick conversation after the UG, I _think_ Anthony is in >>> favor as well; we both made notes to follow up on this. >>> >>> ----- >>> >>> I am proposing that MICE change its MAC address limit from 5 MACs per >>> port to 1 MAC per port. 1 MAC address is enough for normal scenarios and >>> this would further limit the potential for problems on the fabric. This >>> restriction is one that SIX and AMS-IX both have, with the latter being >>> known for their excellent configuration guide for participants: >>> https://www.ams-ix.net/ams/documentation/config-guide >>> >>> To be clear, this is still only for end ports, not ports facing remote >>> switches, of course. The remote switches are responsible for enforcing >>> the current MAC limit, and would be responsible for changing those >>> settings as described here. >>> >>> If someone is swapping their MICE-facing router, the resulting port flap >>> on the MICE/remote switch will clear the limit anyway, if they are >>> directly connected. If they are unable to flap the port (e.g. because >>> they have someone else's layer 2 gear in the middle), they will have to >>> either work with that carrier or their switch operator (MICE/remote) to >>> flap the port. This is a non-zero burden, but I think it's pretty >>> minimal in practice. Additionally, if someone plans to make an equipment >>> swap, we would increase the limit in advance, temporarily, upon request. >>> >>> For reference, SIX goes further and locks you to a particular MAC >>> address in an ACL, so you have to contact them to change your MAC. I am >>> not proposing that. Still, I've been through that a couple of times, and >>> even that isn't really too big of a deal. So I don't think the change to >>> a limit of 1 MAC at MICE will be particularly onerous. >>> >>> There are a couple of participants who have two IP address sets (one >>> IPv4 and one IPv6) on the same port. For those ports, the limit >>> obviously must be at least 2 MACs, but I propose 3 MACs as the limit. If >>> they're using two IPs, they probably are doing so to get some redundancy >>> out of it. For this particular use case, needing to flap the port >>> defeats that redundancy goal, as it breaks the other router. Having a >>> MAC limit of 3 would allow them to swap a device without needing to flap >>> the port. We would grandfather these setups until they are looking for a >>> port-type upgrade; at that point they would need to get to one IP >>> address set per port (by splitting into two ports or reducing to one IP >>> address set) or request an exception as outlined below. The exception >>> process would allow us to better evaluate this use case at that time. >>> >>> There are some members who have multiple MACs showing up now, despite >>> using only one device. For those, I am proposing we set the MAC limit to >>> however many MACs are currently showing up on their port. That is, they >>> would be grandfathered at their current situation for now. At the time >>> of this change, we would encourage them to investigate and fix the >>> multiple MACs issue at their leisure. In the future, if they are doing a >>> equipment swap (especially like-for-like), we would again encourage them >>> to address it, but would otherwise leave their higher limit in place. If >>> they do a port-type upgrade, we would not bring the grandfathering over. >>> If they needed, they could ask for an official exception at that time. >>> >>> There may be cases where people cannot reasonably fix their routers to >>> use only 1 MAC, other scenarios I haven't considered, or things that may >>> come up in the future. For that reason, I think we should allow for the >>> possibility of exceptions upon request to the board. In practice, I >>> would expect the board would confer with the technical committee and/or >>> the technical committee would be bringing these to the board anyway. The >>> goal is to strike a reasonable balance between protecting the fabric >>> without preventing networks from connecting. >>> > > > > On 10/31/22 08:27, Jay Hanke wrote: >> I propose we change the MICE required port security settings to be >> hard filtered for a single static mac address per port. >> >> Over the years, we've had several incidents where floods have occurred >> prior to port being error-disabled during a looping event. >> > > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Oct. 31, 2022

Re: Proposal: MICE Static MAC address Only

by Steve Howard

Oct. 31, 2022

Re: Proposal: MICE Static MAC address Only

by Gary Glissendorf

I agree. Connections should be going into to routers but not everyone does that. Single Mac protects everyones traffic. Gary Glissendorf | Network Architect gary.glissendorf(a)sdncommunications.com -----Original Message----- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Jay Hanke Sent: Monday, October 31, 2022 8:27 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] Proposal: MICE Static MAC address Only I propose we change the MICE required port security settings to be hard filtered for a single static mac address per port. Over the years, we've had several incidents where floods have occurred prior to port being error-disabled during a looping event. -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000 ***This message and any attachments are solely for the intended recipient. If you are not the intended recipient, disclosure, copying, use or distribution of the information included in this message is prohibited -- Please immediately and permanently delete.***

Oct. 31, 2022

Re: Proposal: MICE Static MAC address Only

by Jeremy Lumby

I second that proposal Jeremy Lumby Minnesota VoIP 9217 17th Ave S Suite 216 Bloomington MN 55425 Main 612-355-7740 Direct 612-392-6814 EFax 952-873-7425 jlumby(a)mnvoip.com -------- Original message -------- From: Jay Hanke <jayhanke(a)SOUTHFRONT.IO> Date: 10/31/22 8:27 AM (GMT-06:00) To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] Proposal: MICE Static MAC address Only _____ >From : Jay Hanke [jayhanke(a)SOUTHFRONT.IO] To : MICE-DISCUSS(a)LISTS.IPHOUSE.NET [MICE-DISCUSS(a)LISTS.IPHOUSE.NET] Date : Monday, October 31 2022 08:27:14 I propose we change the MICE required port security settings to be hard filtered for a single static mac address per port. Over the years, we've had several incidents where floods have occurred prior to port being error-disabled during a looping event. -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

Oct. 31, 2022