- mice-discuss - lists.micemn.net

Re: Proposal: MICE Static MAC address Only

by David Farmer

I have similar concerns as Andrew. However, I'm cognizant of Jeremy's concerns as well. Nevertheless, if we are going to fix our old problems, we have to be willing to accept that we are likely to create other new problems. The hard part is deciding if the old problems, the known problems, are worse than the potential new problems, the potentially unknown problems, and which of the potentially new problems we are willing to accept. We seem to have a consensus that this particular old problem needs to be addressed; what is not clear is if we have a consensus on accepting the potential new problems and which of those potential problems we are willing to accept. Note either way we are accepting new problems; we are either accepting potential new problems caused by the automation of IX Manager. Otherwise, we are accepting the potential new problems caused by a static MAC ACL list with only manual changes; I expect delays in changes primarily. Both options are going to produce problems. And if they are delays impacting one of the large content peers they could have as significant of an impact as the original problem we are trying to solve. Thanks On Mon, Oct 31, 2022 at 9:02 AM Andrew Hoyos <hoyosa(a)gmail.com> wrote: > I disagree with static MAC address filtering unless we beef up operational > ability to change/adjust in a reasonable timeframe, or offer a self service > portal to do so (ie IXP Manager implementation first) > > Sent from my iPhone > > On Oct 31, 2022, at 9:59 PM, Steve Howard <showard(a)paulbunyan.net> wrote: > > > Below is a message indicating the current policy as decided by the MICE > Board in December 2019 is that the MAC address limit is 1. > > > The subject of Jay's message indicates "Static" MAC, but, the text seems > to indicate a limit of 1. I like the 1 address limit, but, am not a fan of > Static. Was the current issue caused by somebody not having a proper MAC > address limit installed? > > > On 12/13/19 18:00, Richard Laager wrote: > > The board has approved the change in MAC address limit from 5 to 1. > > NOTE: This is orthogonal to the dedicated remote switch question, which > is still pending. Non-dedicated remote switches can, for example, > enforce this per-VLAN. > > We have already confirmed that most participants are using only 1 MAC > address, and some that were using more than 1 have addressed that after > being contacted. As discussed below, this change will be rolled out in a > way to keep everyone working, by grandfathering if necessary. > > On 10/28/19 3:25 PM, Richard Laager wrote: > > We have discussed this a bit in the past, and this came up at the last > UG. I'm looking for feedback from the group before formally proposing > this to the board for a vote. > > I first brought this up to the technical committee, CCing the board. I > have heard no objections. Jeremy is "in favor of it 100%". > > Based on our quick conversation after the UG, I _think_ Anthony is in > favor as well; we both made notes to follow up on this. > > ----- > > I am proposing that MICE change its MAC address limit from 5 MACs per > port to 1 MAC per port. 1 MAC address is enough for normal scenarios and > this would further limit the potential for problems on the fabric. This > restriction is one that SIX and AMS-IX both have, with the latter being > known for their excellent configuration guide for participants:https://www.ams-ix.net/ams/documentation/config-guide > > To be clear, this is still only for end ports, not ports facing remote > switches, of course. The remote switches are responsible for enforcing > the current MAC limit, and would be responsible for changing those > settings as described here. > > If someone is swapping their MICE-facing router, the resulting port flap > on the MICE/remote switch will clear the limit anyway, if they are > directly connected. If they are unable to flap the port (e.g. because > they have someone else's layer 2 gear in the middle), they will have to > either work with that carrier or their switch operator (MICE/remote) to > flap the port. This is a non-zero burden, but I think it's pretty > minimal in practice. Additionally, if someone plans to make an equipment > swap, we would increase the limit in advance, temporarily, upon request. > > For reference, SIX goes further and locks you to a particular MAC > address in an ACL, so you have to contact them to change your MAC. I am > not proposing that. Still, I've been through that a couple of times, and > even that isn't really too big of a deal. So I don't think the change to > a limit of 1 MAC at MICE will be particularly onerous. > > There are a couple of participants who have two IP address sets (one > IPv4 and one IPv6) on the same port. For those ports, the limit > obviously must be at least 2 MACs, but I propose 3 MACs as the limit. If > they're using two IPs, they probably are doing so to get some redundancy > out of it. For this particular use case, needing to flap the port > defeats that redundancy goal, as it breaks the other router. Having a > MAC limit of 3 would allow them to swap a device without needing to flap > the port. We would grandfather these setups until they are looking for a > port-type upgrade; at that point they would need to get to one IP > address set per port (by splitting into two ports or reducing to one IP > address set) or request an exception as outlined below. The exception > process would allow us to better evaluate this use case at that time. > > There are some members who have multiple MACs showing up now, despite > using only one device. For those, I am proposing we set the MAC limit to > however many MACs are currently showing up on their port. That is, they > would be grandfathered at their current situation for now. At the time > of this change, we would encourage them to investigate and fix the > multiple MACs issue at their leisure. In the future, if they are doing a > equipment swap (especially like-for-like), we would again encourage them > to address it, but would otherwise leave their higher limit in place. If > they do a port-type upgrade, we would not bring the grandfathering over. > If they needed, they could ask for an official exception at that time. > > There may be cases where people cannot reasonably fix their routers to > use only 1 MAC, other scenarios I haven't considered, or things that may > come up in the future. For that reason, I think we should allow for the > possibility of exceptions upon request to the board. In practice, I > would expect the board would confer with the technical committee and/or > the technical committee would be bringing these to the board anyway. The > goal is to strike a reasonable balance between protecting the fabric > without preventing networks from connecting. > > > > > > On 10/31/22 08:27, Jay Hanke wrote: > > I propose we change the MICE required port security settings to be > hard filtered for a single static mac address per port. > > Over the years, we've had several incidents where floods have occurred > prior to port being error-disabled during a looping event. > > > > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > -- =============================================== David Farmer Email:farmer@umn.edu Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 ===============================================

Oct. 31, 2022

Re: [EXTERNAL] Re: [MICE-DISCUSS] Proposal: MICE Static MAC address Only

by Chris Wopat

Regarding the method of 'updating' the mac address list, there should probably be the ability to address maintenance, ie temporarily add a 2nd mac address to permit list. See SIX's faq. Unshore how handled internally, but one can email with a pending change (ie router replacement) to have 2 permitted for a period so maintenance can go smoothly. https://www.seattleix.net/faq -- Chris Wopat Network Engineer, WiscNet wopat(a)wiscnet.net 608-210-3965

Oct. 31, 2022

Re: Proposal: MICE Static MAC address Only

by Jeremy Lumby

I like that idea -----Original Message----- From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Jay Hanke Sent: Monday, October 31, 2022 9:27 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Proposal: MICE Static MAC address Only The switch access could be air-gapped. ie there could be a script that generates the new mac-address list and then it could be manually applied. On Mon, Oct 31, 2022 at 9:22 AM Jeremy Lumby <jlumby(a)mnvoip.com> wrote: > > I have mixed feelings about giving IXP manager access to the switch. The main reason is that if a member is proactive they can request the second MAC ahead of time, and it can be temporarily added for the transition. As it is most requests are processed in a couple hours. IXP manager is a program that none of the volunteers have a great grasp on, and lack of a timely security update on a portal that is made available to the entire internet in an invitation to get hacked. If it has direct access to the switch things can go wrong quickly. While I am concerned as it is about IXP manager having access to the route servers, they are an extra and not 100% necessary for the IX to function. > > -----Original Message----- > From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Jay Hanke > Sent: Monday, October 31, 2022 9:05 AM > To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET > Subject: Re: [MICE-DISCUSS] Proposal: MICE Static MAC address Only > > The error disable takes 1-2 seconds to clamp down and disable the > port. This results in a short, but severe, broadcast storm. > > The churn rate on members changing mac addresses is fairly low. > > I agree that we should automate the process so it can be done via > self-service by the members. > > On Mon, Oct 31, 2022 at 8:59 AM Steve Howard <showard(a)paulbunyan.net> wrote: > > > > > > Below is a message indicating the current policy as decided by the MICE Board in December 2019 is that the MAC address limit is 1. > > > > > > The subject of Jay's message indicates "Static" MAC, but, the text seems to indicate a limit of 1. I like the 1 address limit, but, am not a fan of Static. Was the current issue caused by somebody not having a proper MAC address limit installed? > > > > > > On 12/13/19 18:00, Richard Laager wrote: > > > > The board has approved the change in MAC address limit from 5 to 1. > > > > NOTE: This is orthogonal to the dedicated remote switch question, which > > is still pending. Non-dedicated remote switches can, for example, > > enforce this per-VLAN. > > > > We have already confirmed that most participants are using only 1 MAC > > address, and some that were using more than 1 have addressed that after > > being contacted. As discussed below, this change will be rolled out in a > > way to keep everyone working, by grandfathering if necessary. > > > > On 10/28/19 3:25 PM, Richard Laager wrote: > > > > We have discussed this a bit in the past, and this came up at the last > > UG. I'm looking for feedback from the group before formally proposing > > this to the board for a vote. > > > > I first brought this up to the technical committee, CCing the board. I > > have heard no objections. Jeremy is "in favor of it 100%". > > > > Based on our quick conversation after the UG, I _think_ Anthony is in > > favor as well; we both made notes to follow up on this. > > > > ----- > > > > I am proposing that MICE change its MAC address limit from 5 MACs per > > port to 1 MAC per port. 1 MAC address is enough for normal scenarios and > > this would further limit the potential for problems on the fabric. This > > restriction is one that SIX and AMS-IX both have, with the latter being > > known for their excellent configuration guide for participants: > > https://www.ams-ix.net/ams/documentation/config-guide > > > > To be clear, this is still only for end ports, not ports facing remote > > switches, of course. The remote switches are responsible for enforcing > > the current MAC limit, and would be responsible for changing those > > settings as described here. > > > > If someone is swapping their MICE-facing router, the resulting port flap > > on the MICE/remote switch will clear the limit anyway, if they are > > directly connected. If they are unable to flap the port (e.g. because > > they have someone else's layer 2 gear in the middle), they will have to > > either work with that carrier or their switch operator (MICE/remote) to > > flap the port. This is a non-zero burden, but I think it's pretty > > minimal in practice. Additionally, if someone plans to make an equipment > > swap, we would increase the limit in advance, temporarily, upon request. > > > > For reference, SIX goes further and locks you to a particular MAC > > address in an ACL, so you have to contact them to change your MAC. I am > > not proposing that. Still, I've been through that a couple of times, and > > even that isn't really too big of a deal. So I don't think the change to > > a limit of 1 MAC at MICE will be particularly onerous. > > > > There are a couple of participants who have two IP address sets (one > > IPv4 and one IPv6) on the same port. For those ports, the limit > > obviously must be at least 2 MACs, but I propose 3 MACs as the limit. If > > they're using two IPs, they probably are doing so to get some redundancy > > out of it. For this particular use case, needing to flap the port > > defeats that redundancy goal, as it breaks the other router. Having a > > MAC limit of 3 would allow them to swap a device without needing to flap > > the port. We would grandfather these setups until they are looking for a > > port-type upgrade; at that point they would need to get to one IP > > address set per port (by splitting into two ports or reducing to one IP > > address set) or request an exception as outlined below. The exception > > process would allow us to better evaluate this use case at that time. > > > > There are some members who have multiple MACs showing up now, despite > > using only one device. For those, I am proposing we set the MAC limit to > > however many MACs are currently showing up on their port. That is, they > > would be grandfathered at their current situation for now. At the time > > of this change, we would encourage them to investigate and fix the > > multiple MACs issue at their leisure. In the future, if they are doing a > > equipment swap (especially like-for-like), we would again encourage them > > to address it, but would otherwise leave their higher limit in place. If > > they do a port-type upgrade, we would not bring the grandfathering over. > > If they needed, they could ask for an official exception at that time. > > > > There may be cases where people cannot reasonably fix their routers to > > use only 1 MAC, other scenarios I haven't considered, or things that may > > come up in the future. For that reason, I think we should allow for the > > possibility of exceptions upon request to the board. In practice, I > > would expect the board would confer with the technical committee and/or > > the technical committee would be bringing these to the board anyway. The > > goal is to strike a reasonable balance between protecting the fabric > > without preventing networks from connecting. > > > > > > > > > > On 10/31/22 08:27, Jay Hanke wrote: > > > > I propose we change the MICE required port security settings to be > > hard filtered for a single static mac address per port. > > > > Over the years, we've had several incidents where floods have occurred > > prior to port being error-disabled during a looping event. > > > > > > > > ________________________________ > > > > To unsubscribe from the MICE-DISCUSS list, click the following link: > > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > > > > -- > Jay Hanke, President > South Front Networks > jayhanke(a)southfront.io > Phone 612-204-0000 -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

Oct. 31, 2022

Re: Proposal: MICE Static MAC address Only

by Jay Hanke

Yes, that was the intention. On Mon, Oct 31, 2022 at 9:26 AM Jeremy Lumby <jlumby(a)mnvoip.com> wrote: > > I think the best way to phrase it is “static MAC ACL” applied to all member ports. There could be reasons to do static MAC entries in the forwarding table (so traffic does not flood when a large link drops) but we are not talking about that here/yet. > > > > From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Steve Howard > Sent: Monday, October 31, 2022 8:59 AM > To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET > Subject: Re: [MICE-DISCUSS] Proposal: MICE Static MAC address Only > > > > > Below is a message indicating the current policy as decided by the MICE Board in December 2019 is that the MAC address limit is 1. > > > The subject of Jay's message indicates "Static" MAC, but, the text seems to indicate a limit of 1. I like the 1 address limit, but, am not a fan of Static. Was the current issue caused by somebody not having a proper MAC address limit installed? > > On 12/13/19 18:00, Richard Laager wrote: > > The board has approved the change in MAC address limit from 5 to 1. > > > > NOTE: This is orthogonal to the dedicated remote switch question, which > > is still pending. Non-dedicated remote switches can, for example, > > enforce this per-VLAN. > > > > We have already confirmed that most participants are using only 1 MAC > > address, and some that were using more than 1 have addressed that after > > being contacted. As discussed below, this change will be rolled out in a > > way to keep everyone working, by grandfathering if necessary. > > > > On 10/28/19 3:25 PM, Richard Laager wrote: > > We have discussed this a bit in the past, and this came up at the last > > UG. I'm looking for feedback from the group before formally proposing > > this to the board for a vote. > > > > I first brought this up to the technical committee, CCing the board. I > > have heard no objections. Jeremy is "in favor of it 100%". > > > > Based on our quick conversation after the UG, I _think_ Anthony is in > > favor as well; we both made notes to follow up on this. > > > > ----- > > > > I am proposing that MICE change its MAC address limit from 5 MACs per > > port to 1 MAC per port. 1 MAC address is enough for normal scenarios and > > this would further limit the potential for problems on the fabric. This > > restriction is one that SIX and AMS-IX both have, with the latter being > > known for their excellent configuration guide for participants: > > https://www.ams-ix.net/ams/documentation/config-guide > > > > To be clear, this is still only for end ports, not ports facing remote > > switches, of course. The remote switches are responsible for enforcing > > the current MAC limit, and would be responsible for changing those > > settings as described here. > > > > If someone is swapping their MICE-facing router, the resulting port flap > > on the MICE/remote switch will clear the limit anyway, if they are > > directly connected. If they are unable to flap the port (e.g. because > > they have someone else's layer 2 gear in the middle), they will have to > > either work with that carrier or their switch operator (MICE/remote) to > > flap the port. This is a non-zero burden, but I think it's pretty > > minimal in practice. Additionally, if someone plans to make an equipment > > swap, we would increase the limit in advance, temporarily, upon request. > > > > For reference, SIX goes further and locks you to a particular MAC > > address in an ACL, so you have to contact them to change your MAC. I am > > not proposing that. Still, I've been through that a couple of times, and > > even that isn't really too big of a deal. So I don't think the change to > > a limit of 1 MAC at MICE will be particularly onerous. > > > > There are a couple of participants who have two IP address sets (one > > IPv4 and one IPv6) on the same port. For those ports, the limit > > obviously must be at least 2 MACs, but I propose 3 MACs as the limit. If > > they're using two IPs, they probably are doing so to get some redundancy > > out of it. For this particular use case, needing to flap the port > > defeats that redundancy goal, as it breaks the other router. Having a > > MAC limit of 3 would allow them to swap a device without needing to flap > > the port. We would grandfather these setups until they are looking for a > > port-type upgrade; at that point they would need to get to one IP > > address set per port (by splitting into two ports or reducing to one IP > > address set) or request an exception as outlined below. The exception > > process would allow us to better evaluate this use case at that time. > > > > There are some members who have multiple MACs showing up now, despite > > using only one device. For those, I am proposing we set the MAC limit to > > however many MACs are currently showing up on their port. That is, they > > would be grandfathered at their current situation for now. At the time > > of this change, we would encourage them to investigate and fix the > > multiple MACs issue at their leisure. In the future, if they are doing a > > equipment swap (especially like-for-like), we would again encourage them > > to address it, but would otherwise leave their higher limit in place. If > > they do a port-type upgrade, we would not bring the grandfathering over. > > If they needed, they could ask for an official exception at that time. > > > > There may be cases where people cannot reasonably fix their routers to > > use only 1 MAC, other scenarios I haven't considered, or things that may > > come up in the future. For that reason, I think we should allow for the > > possibility of exceptions upon request to the board. In practice, I > > would expect the board would confer with the technical committee and/or > > the technical committee would be bringing these to the board anyway. The > > goal is to strike a reasonable balance between protecting the fabric > > without preventing networks from connecting. > > > > > > > > On 10/31/22 08:27, Jay Hanke wrote: > > I propose we change the MICE required port security settings to be > > hard filtered for a single static mac address per port. > > > > Over the years, we've had several incidents where floods have occurred > > prior to port being error-disabled during a looping event. > > > > > > > > ________________________________ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > > > ________________________________ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

Oct. 31, 2022

Re: Proposal: MICE Static MAC address Only

by Jay Hanke

The switch access could be air-gapped. ie there could be a script that generates the new mac-address list and then it could be manually applied. On Mon, Oct 31, 2022 at 9:22 AM Jeremy Lumby <jlumby(a)mnvoip.com> wrote: > > I have mixed feelings about giving IXP manager access to the switch. The main reason is that if a member is proactive they can request the second MAC ahead of time, and it can be temporarily added for the transition. As it is most requests are processed in a couple hours. IXP manager is a program that none of the volunteers have a great grasp on, and lack of a timely security update on a portal that is made available to the entire internet in an invitation to get hacked. If it has direct access to the switch things can go wrong quickly. While I am concerned as it is about IXP manager having access to the route servers, they are an extra and not 100% necessary for the IX to function. > > -----Original Message----- > From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Jay Hanke > Sent: Monday, October 31, 2022 9:05 AM > To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET > Subject: Re: [MICE-DISCUSS] Proposal: MICE Static MAC address Only > > The error disable takes 1-2 seconds to clamp down and disable the > port. This results in a short, but severe, broadcast storm. > > The churn rate on members changing mac addresses is fairly low. > > I agree that we should automate the process so it can be done via > self-service by the members. > > On Mon, Oct 31, 2022 at 8:59 AM Steve Howard <showard(a)paulbunyan.net> wrote: > > > > > > Below is a message indicating the current policy as decided by the MICE Board in December 2019 is that the MAC address limit is 1. > > > > > > The subject of Jay's message indicates "Static" MAC, but, the text seems to indicate a limit of 1. I like the 1 address limit, but, am not a fan of Static. Was the current issue caused by somebody not having a proper MAC address limit installed? > > > > > > On 12/13/19 18:00, Richard Laager wrote: > > > > The board has approved the change in MAC address limit from 5 to 1. > > > > NOTE: This is orthogonal to the dedicated remote switch question, which > > is still pending. Non-dedicated remote switches can, for example, > > enforce this per-VLAN. > > > > We have already confirmed that most participants are using only 1 MAC > > address, and some that were using more than 1 have addressed that after > > being contacted. As discussed below, this change will be rolled out in a > > way to keep everyone working, by grandfathering if necessary. > > > > On 10/28/19 3:25 PM, Richard Laager wrote: > > > > We have discussed this a bit in the past, and this came up at the last > > UG. I'm looking for feedback from the group before formally proposing > > this to the board for a vote. > > > > I first brought this up to the technical committee, CCing the board. I > > have heard no objections. Jeremy is "in favor of it 100%". > > > > Based on our quick conversation after the UG, I _think_ Anthony is in > > favor as well; we both made notes to follow up on this. > > > > ----- > > > > I am proposing that MICE change its MAC address limit from 5 MACs per > > port to 1 MAC per port. 1 MAC address is enough for normal scenarios and > > this would further limit the potential for problems on the fabric. This > > restriction is one that SIX and AMS-IX both have, with the latter being > > known for their excellent configuration guide for participants: > > https://www.ams-ix.net/ams/documentation/config-guide > > > > To be clear, this is still only for end ports, not ports facing remote > > switches, of course. The remote switches are responsible for enforcing > > the current MAC limit, and would be responsible for changing those > > settings as described here. > > > > If someone is swapping their MICE-facing router, the resulting port flap > > on the MICE/remote switch will clear the limit anyway, if they are > > directly connected. If they are unable to flap the port (e.g. because > > they have someone else's layer 2 gear in the middle), they will have to > > either work with that carrier or their switch operator (MICE/remote) to > > flap the port. This is a non-zero burden, but I think it's pretty > > minimal in practice. Additionally, if someone plans to make an equipment > > swap, we would increase the limit in advance, temporarily, upon request. > > > > For reference, SIX goes further and locks you to a particular MAC > > address in an ACL, so you have to contact them to change your MAC. I am > > not proposing that. Still, I've been through that a couple of times, and > > even that isn't really too big of a deal. So I don't think the change to > > a limit of 1 MAC at MICE will be particularly onerous. > > > > There are a couple of participants who have two IP address sets (one > > IPv4 and one IPv6) on the same port. For those ports, the limit > > obviously must be at least 2 MACs, but I propose 3 MACs as the limit. If > > they're using two IPs, they probably are doing so to get some redundancy > > out of it. For this particular use case, needing to flap the port > > defeats that redundancy goal, as it breaks the other router. Having a > > MAC limit of 3 would allow them to swap a device without needing to flap > > the port. We would grandfather these setups until they are looking for a > > port-type upgrade; at that point they would need to get to one IP > > address set per port (by splitting into two ports or reducing to one IP > > address set) or request an exception as outlined below. The exception > > process would allow us to better evaluate this use case at that time. > > > > There are some members who have multiple MACs showing up now, despite > > using only one device. For those, I am proposing we set the MAC limit to > > however many MACs are currently showing up on their port. That is, they > > would be grandfathered at their current situation for now. At the time > > of this change, we would encourage them to investigate and fix the > > multiple MACs issue at their leisure. In the future, if they are doing a > > equipment swap (especially like-for-like), we would again encourage them > > to address it, but would otherwise leave their higher limit in place. If > > they do a port-type upgrade, we would not bring the grandfathering over. > > If they needed, they could ask for an official exception at that time. > > > > There may be cases where people cannot reasonably fix their routers to > > use only 1 MAC, other scenarios I haven't considered, or things that may > > come up in the future. For that reason, I think we should allow for the > > possibility of exceptions upon request to the board. In practice, I > > would expect the board would confer with the technical committee and/or > > the technical committee would be bringing these to the board anyway. The > > goal is to strike a reasonable balance between protecting the fabric > > without preventing networks from connecting. > > > > > > > > > > On 10/31/22 08:27, Jay Hanke wrote: > > > > I propose we change the MICE required port security settings to be > > hard filtered for a single static mac address per port. > > > > Over the years, we've had several incidents where floods have occurred > > prior to port being error-disabled during a looping event. > > > > > > > > ________________________________ > > > > To unsubscribe from the MICE-DISCUSS list, click the following link: > > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > > > > -- > Jay Hanke, President > South Front Networks > jayhanke(a)southfront.io > Phone 612-204-0000 -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

Oct. 31, 2022

Re: Proposal: MICE Static MAC address Only

by Jeremy Lumby

I think the best way to phrase it is “static MAC ACL” applied to all member ports. There could be reasons to do static MAC entries in the forwarding table (so traffic does not flood when a large link drops) but we are not talking about that here/yet. From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Steve Howard Sent: Monday, October 31, 2022 8:59 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Proposal: MICE Static MAC address Only Below is a message indicating the current policy as decided by the MICE Board in December 2019 is that the MAC address limit is 1. The subject of Jay's message indicates "Static" MAC, but, the text seems to indicate a limit of 1. I like the 1 address limit, but, am not a fan of Static. Was the current issue caused by somebody not having a proper MAC address limit installed? On 12/13/19 18:00, Richard Laager wrote: The board has approved the change in MAC address limit from 5 to 1. NOTE: This is orthogonal to the dedicated remote switch question, whichis still pending. Non-dedicated remote switches can, for example,enforce this per-VLAN. We have already confirmed that most participants are using only 1 MACaddress, and some that were using more than 1 have addressed that afterbeing contacted. As discussed below, this change will be rolled out in away to keep everyone working, by grandfathering if necessary. On 10/28/19 3:25 PM, Richard Laager wrote: We have discussed this a bit in the past, and this came up at the lastUG. I'm looking for feedback from the group before formally proposingthis to the board for a vote. I first brought this up to the technical committee, CCing the board. Ihave heard no objections. Jeremy is "in favor of it 100%". Based on our quick conversation after the UG, I _think_ Anthony is infavor as well; we both made notes to follow up on this. ----- I am proposing that MICE change its MAC address limit from 5 MACs perport to 1 MAC per port. 1 MAC address is enough for normal scenarios andthis would further limit the potential for problems on the fabric. Thisrestriction is one that SIX and AMS-IX both have, with the latter beingknown for their excellent configuration guide for participants:https://www.ams-ix.net/ams/documentation/config-guide To be clear, this is still only for end ports, not ports facing remoteswitches, of course. The remote switches are responsible for enforcingthe current MAC limit, and would be responsible for changing thosesettings as described here. If someone is swapping their MICE-facing router, the resulting port flapon the MICE/remote switch will clear the limit anyway, if they aredirectly connected. If they are unable to flap the port (e.g. becausethey have someone else's layer 2 gear in the middle), they will have toeither work with that carrier or their switch operator (MICE/remote) toflap the port. This is a non-zero burden, but I think it's prettyminimal in practice. Additionally, if someone plans to make an equipmentswap, we would increase the limit in advance, temporarily, upon request. For reference, SIX goes further and locks you to a particular MACaddress in an ACL, so you have to contact them to change your MAC. I amnot proposing that. Still, I've been through that a couple of times, andeven that isn't really too big of a deal. So I don't think the change toa limit of 1 MAC at MICE will be particularly onerous. There are a couple of participants who have two IP address sets (oneIPv4 and one IPv6) on the same port. For those ports, the limitobviously must be at least 2 MACs, but I propose 3 MACs as the limit. Ifthey're using two IPs, they probably are doing so to get some redundancyout of it. For this particular use case, needing to flap the portdefeats that redundancy goal, as it breaks the other router. Having aMAC limit of 3 would allow them to swap a device without needing to flapthe port. We would grandfather these setups until they are looking for aport-type upgrade; at that point they would need to get to one IPaddress set per port (by splitting into two ports or reducing to one IPaddress set) or request an exception as outlined below. The exceptionprocess would allow us to better evaluate this use case at that time. There are some members who have multiple MACs showing up now, despiteusing only one device. For those, I am proposing we set the MAC limit tohowever many MACs are currently showing up on their port. That is, theywould be grandfathered at their current situation for now. At the timeof this change, we would encourage them to investigate and fix themultiple MACs issue at their leisure. In the future, if they are doing aequipment swap (especially like-for-like), we would again encourage themto address it, but would otherwise leave their higher limit in place. Ifthey do a port-type upgrade, we would not bring the grandfathering over.If they needed, they could ask for an official exception at that time. There may be cases where people cannot reasonably fix their routers touse only 1 MAC, other scenarios I haven't considered, or things that maycome up in the future. For that reason, I think we should allow for thepossibility of exceptions upon request to the board. In practice, Iwould expect the board would confer with the technical committee and/orthe technical committee would be bringing these to the board anyway. Thegoal is to strike a reasonable balance between protecting the fabricwithout preventing networks from connecting. On 10/31/22 08:27, Jay Hanke wrote: I propose we change the MICE required port security settings to behard filtered for a single static mac address per port. Over the years, we've had several incidents where floods have occurredprior to port being error-disabled during a looping event. ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Oct. 31, 2022

Re: Proposal: MICE Static MAC address Only

by Jeremy Lumby

I have mixed feelings about giving IXP manager access to the switch. The main reason is that if a member is proactive they can request the second MAC ahead of time, and it can be temporarily added for the transition. As it is most requests are processed in a couple hours. IXP manager is a program that none of the volunteers have a great grasp on, and lack of a timely security update on a portal that is made available to the entire internet in an invitation to get hacked. If it has direct access to the switch things can go wrong quickly. While I am concerned as it is about IXP manager having access to the route servers, they are an extra and not 100% necessary for the IX to function. -----Original Message----- From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Jay Hanke Sent: Monday, October 31, 2022 9:05 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Proposal: MICE Static MAC address Only The error disable takes 1-2 seconds to clamp down and disable the port. This results in a short, but severe, broadcast storm. The churn rate on members changing mac addresses is fairly low. I agree that we should automate the process so it can be done via self-service by the members. On Mon, Oct 31, 2022 at 8:59 AM Steve Howard <showard(a)paulbunyan.net> wrote: > > > Below is a message indicating the current policy as decided by the MICE Board in December 2019 is that the MAC address limit is 1. > > > The subject of Jay's message indicates "Static" MAC, but, the text seems to indicate a limit of 1. I like the 1 address limit, but, am not a fan of Static. Was the current issue caused by somebody not having a proper MAC address limit installed? > > > On 12/13/19 18:00, Richard Laager wrote: > > The board has approved the change in MAC address limit from 5 to 1. > > NOTE: This is orthogonal to the dedicated remote switch question, which > is still pending. Non-dedicated remote switches can, for example, > enforce this per-VLAN. > > We have already confirmed that most participants are using only 1 MAC > address, and some that were using more than 1 have addressed that after > being contacted. As discussed below, this change will be rolled out in a > way to keep everyone working, by grandfathering if necessary. > > On 10/28/19 3:25 PM, Richard Laager wrote: > > We have discussed this a bit in the past, and this came up at the last > UG. I'm looking for feedback from the group before formally proposing > this to the board for a vote. > > I first brought this up to the technical committee, CCing the board. I > have heard no objections. Jeremy is "in favor of it 100%". > > Based on our quick conversation after the UG, I _think_ Anthony is in > favor as well; we both made notes to follow up on this. > > ----- > > I am proposing that MICE change its MAC address limit from 5 MACs per > port to 1 MAC per port. 1 MAC address is enough for normal scenarios and > this would further limit the potential for problems on the fabric. This > restriction is one that SIX and AMS-IX both have, with the latter being > known for their excellent configuration guide for participants: > https://www.ams-ix.net/ams/documentation/config-guide > > To be clear, this is still only for end ports, not ports facing remote > switches, of course. The remote switches are responsible for enforcing > the current MAC limit, and would be responsible for changing those > settings as described here. > > If someone is swapping their MICE-facing router, the resulting port flap > on the MICE/remote switch will clear the limit anyway, if they are > directly connected. If they are unable to flap the port (e.g. because > they have someone else's layer 2 gear in the middle), they will have to > either work with that carrier or their switch operator (MICE/remote) to > flap the port. This is a non-zero burden, but I think it's pretty > minimal in practice. Additionally, if someone plans to make an equipment > swap, we would increase the limit in advance, temporarily, upon request. > > For reference, SIX goes further and locks you to a particular MAC > address in an ACL, so you have to contact them to change your MAC. I am > not proposing that. Still, I've been through that a couple of times, and > even that isn't really too big of a deal. So I don't think the change to > a limit of 1 MAC at MICE will be particularly onerous. > > There are a couple of participants who have two IP address sets (one > IPv4 and one IPv6) on the same port. For those ports, the limit > obviously must be at least 2 MACs, but I propose 3 MACs as the limit. If > they're using two IPs, they probably are doing so to get some redundancy > out of it. For this particular use case, needing to flap the port > defeats that redundancy goal, as it breaks the other router. Having a > MAC limit of 3 would allow them to swap a device without needing to flap > the port. We would grandfather these setups until they are looking for a > port-type upgrade; at that point they would need to get to one IP > address set per port (by splitting into two ports or reducing to one IP > address set) or request an exception as outlined below. The exception > process would allow us to better evaluate this use case at that time. > > There are some members who have multiple MACs showing up now, despite > using only one device. For those, I am proposing we set the MAC limit to > however many MACs are currently showing up on their port. That is, they > would be grandfathered at their current situation for now. At the time > of this change, we would encourage them to investigate and fix the > multiple MACs issue at their leisure. In the future, if they are doing a > equipment swap (especially like-for-like), we would again encourage them > to address it, but would otherwise leave their higher limit in place. If > they do a port-type upgrade, we would not bring the grandfathering over. > If they needed, they could ask for an official exception at that time. > > There may be cases where people cannot reasonably fix their routers to > use only 1 MAC, other scenarios I haven't considered, or things that may > come up in the future. For that reason, I think we should allow for the > possibility of exceptions upon request to the board. In practice, I > would expect the board would confer with the technical committee and/or > the technical committee would be bringing these to the board anyway. The > goal is to strike a reasonable balance between protecting the fabric > without preventing networks from connecting. > > > > > On 10/31/22 08:27, Jay Hanke wrote: > > I propose we change the MICE required port security settings to be > hard filtered for a single static mac address per port. > > Over the years, we've had several incidents where floods have occurred > prior to port being error-disabled during a looping event. > > > > ________________________________ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

Oct. 31, 2022

Re: Proposal: MICE Static MAC address Only

by Chris Wopat

On 10/31/22 08:59, Steve Howard wrote: > > Below is a message indicating the current policy as decided by the MICE > Board in December 2019 is that the MAC address limit is 1. > > > The subject of Jay's message indicates "Static" MAC, but, the text seems > to indicate a limit of 1. I like the 1 address limit, but, am not a fan > of Static. Was the current issue caused by somebody not having a proper > MAC address limit installed? > Quick note to consider. Yes, there should be a mac limit. Certain vendor hardware may restrict the way this is done - as in, some vendor kit may allow one to set a mac limit of 1 (to anything) and some may only permit static mac, and some may permit both. WiscNet's remote switch is currently set to static mac, this is because of a vendor change that prevented the mac limit from working. Forcing one way or the other may be difficult to achieve, but the limit should exist in some form. -- Chris Wopat Network Engineer, WiscNet wopat(a)wiscnet.net 608-210-3965

Oct. 31, 2022

Re: [EXTERNAL] Re: [MICE-DISCUSS] Proposal: MICE Static MAC address Only

by Jay Hanke

No, the recent event was on the main switch. Historically, this has happened on most (if not all) of the MICE switches. On Mon, Oct 31, 2022 at 9:07 AM Kittel, Brady <Brady.Kittel(a)hcmed.org> wrote: > > Was the source of the issue a remote switch? > > " > > To be clear, this is still only for end ports, not ports facing remote > switches, of course. The remote switches are responsible for enforcing > the current MAC limit, and would be responsible for changing those > settings as described here." > > > -Brady > ________________________________ > From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> on behalf of Jay Hanke <jayhanke(a)SOUTHFRONT.IO> > Sent: Monday, October 31, 2022 9:05:08 AM > To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> > Subject: [EXTERNAL] Re: [MICE-DISCUSS] Proposal: MICE Static MAC address Only > > CAUTION: This email originated from outside of HCMC. DO NOT CLICK links or open attachments unless you recognize the sender and know the content is safe. > > > The error disable takes 1-2 seconds to clamp down and disable the > port. This results in a short, but severe, broadcast storm. > > The churn rate on members changing mac addresses is fairly low. > > I agree that we should automate the process so it can be done via > self-service by the members. > > On Mon, Oct 31, 2022 at 8:59 AM Steve Howard <showard(a)paulbunyan.net> wrote: > > > > > > Below is a message indicating the current policy as decided by the MICE Board in December 2019 is that the MAC address limit is 1. > > > > > > The subject of Jay's message indicates "Static" MAC, but, the text seems to indicate a limit of 1. I like the 1 address limit, but, am not a fan of Static. Was the current issue caused by somebody not having a proper MAC address limit installed? > > > > > > On 12/13/19 18:00, Richard Laager wrote: > > > > The board has approved the change in MAC address limit from 5 to 1. > > > > NOTE: This is orthogonal to the dedicated remote switch question, which > > is still pending. Non-dedicated remote switches can, for example, > > enforce this per-VLAN. > > > > We have already confirmed that most participants are using only 1 MAC > > address, and some that were using more than 1 have addressed that after > > being contacted. As discussed below, this change will be rolled out in a > > way to keep everyone working, by grandfathering if necessary. > > > > On 10/28/19 3:25 PM, Richard Laager wrote: > > > > We have discussed this a bit in the past, and this came up at the last > > UG. I'm looking for feedback from the group before formally proposing > > this to the board for a vote. > > > > I first brought this up to the technical committee, CCing the board. I > > have heard no objections. Jeremy is "in favor of it 100%". > > > > Based on our quick conversation after the UG, I _think_ Anthony is in > > favor as well; we both made notes to follow up on this. > > > > ----- > > > > I am proposing that MICE change its MAC address limit from 5 MACs per > > port to 1 MAC per port. 1 MAC address is enough for normal scenarios and > > this would further limit the potential for problems on the fabric. This > > restriction is one that SIX and AMS-IX both have, with the latter being > > known for their excellent configuration guide for participants: > > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ams-i… > > > > To be clear, this is still only for end ports, not ports facing remote > > switches, of course. The remote switches are responsible for enforcing > > the current MAC limit, and would be responsible for changing those > > settings as described here. > > > > If someone is swapping their MICE-facing router, the resulting port flap > > on the MICE/remote switch will clear the limit anyway, if they are > > directly connected. If they are unable to flap the port (e.g. because > > they have someone else's layer 2 gear in the middle), they will have to > > either work with that carrier or their switch operator (MICE/remote) to > > flap the port. This is a non-zero burden, but I think it's pretty > > minimal in practice. Additionally, if someone plans to make an equipment > > swap, we would increase the limit in advance, temporarily, upon request. > > > > For reference, SIX goes further and locks you to a particular MAC > > address in an ACL, so you have to contact them to change your MAC. I am > > not proposing that. Still, I've been through that a couple of times, and > > even that isn't really too big of a deal. So I don't think the change to > > a limit of 1 MAC at MICE will be particularly onerous. > > > > There are a couple of participants who have two IP address sets (one > > IPv4 and one IPv6) on the same port. For those ports, the limit > > obviously must be at least 2 MACs, but I propose 3 MACs as the limit. If > > they're using two IPs, they probably are doing so to get some redundancy > > out of it. For this particular use case, needing to flap the port > > defeats that redundancy goal, as it breaks the other router. Having a > > MAC limit of 3 would allow them to swap a device without needing to flap > > the port. We would grandfather these setups until they are looking for a > > port-type upgrade; at that point they would need to get to one IP > > address set per port (by splitting into two ports or reducing to one IP > > address set) or request an exception as outlined below. The exception > > process would allow us to better evaluate this use case at that time. > > > > There are some members who have multiple MACs showing up now, despite > > using only one device. For those, I am proposing we set the MAC limit to > > however many MACs are currently showing up on their port. That is, they > > would be grandfathered at their current situation for now. At the time > > of this change, we would encourage them to investigate and fix the > > multiple MACs issue at their leisure. In the future, if they are doing a > > equipment swap (especially like-for-like), we would again encourage them > > to address it, but would otherwise leave their higher limit in place. If > > they do a port-type upgrade, we would not bring the grandfathering over. > > If they needed, they could ask for an official exception at that time. > > > > There may be cases where people cannot reasonably fix their routers to > > use only 1 MAC, other scenarios I haven't considered, or things that may > > come up in the future. For that reason, I think we should allow for the > > possibility of exceptions upon request to the board. In practice, I > > would expect the board would confer with the technical committee and/or > > the technical committee would be bringing these to the board anyway. The > > goal is to strike a reasonable balance between protecting the fabric > > without preventing networks from connecting. > > > > > > > > > > On 10/31/22 08:27, Jay Hanke wrote: > > > > I propose we change the MICE required port security settings to be > > hard filtered for a single static mac address per port. > > > > Over the years, we've had several incidents where floods have occurred > > prior to port being error-disabled during a looping event. > > > > > > > > ________________________________ > > > > To unsubscribe from the MICE-DISCUSS list, click the following link: > > https://nam02.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.ipho… > > > > -- > Jay Hanke, President > South Front Networks > jayhanke(a)southfront.io > Phone 612-204-0000 > > Confidentiality Notice: > > Information contained in this e-mail is being sent to you after appropriate authorization or by legal exception. You are obligated to maintain it in a safe, secure and confidential manner. Re-disclosure without patient consent or as permitted by law is prohibited and may subject you to state and/or federal penalties. This information may also be legally privileged, the disclosure of which is governed by law. This information is intended for the use of the person or entity to which it is addressed. If you are not the intended recipient, or the employee or agent responsible for delivering it to the intended recipient, you are hereby notified that any access, disclosure, copying or distribution of this information is strictly prohibited. If you have received this message by error, please notify the sender immediately to arrange for return or proof of destruction of the information contained in this message. > > > ________________________________ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

Oct. 31, 2022

Re: [EXTERNAL] Re: [MICE-DISCUSS] Proposal: MICE Static MAC address Only

by Jeremy Lumby

The issue was a member port. The port security took 2 seconds to shutdown the port, however 100G of traffic looped crashed the forwarding table on the main switch, forcing manual intervention. A static MAC ACL on each port would not have taken time to react. This is what the SIX has been doing on their ports since 2010 (as well as multiple other layers of security). From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Kittel, Brady Sent: Monday, October 31, 2022 9:07 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] [EXTERNAL] Re: [MICE-DISCUSS] Proposal: MICE Static MAC address Only Was the source of the issue a remote switch? " To be clear, this is still only for end ports, not ports facing remoteswitches, of course. The remote switches are responsible for enforcingthe current MAC limit, and would be responsible for changing thosesettings as described here." -Brady ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> on behalf of Jay Hanke <jayhanke(a)SOUTHFRONT.IO> Sent: Monday, October 31, 2022 9:05:08 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> Subject: [EXTERNAL] Re: [MICE-DISCUSS] Proposal: MICE Static MAC address Only CAUTION: This email originated from outside of HCMC. DO NOT CLICK links or open attachments unless you recognize the sender and know the content is safe. The error disable takes 1-2 seconds to clamp down and disable the port. This results in a short, but severe, broadcast storm. The churn rate on members changing mac addresses is fairly low. I agree that we should automate the process so it can be done via self-service by the members. On Mon, Oct 31, 2022 at 8:59 AM Steve Howard <showard(a)paulbunyan.net> wrote: > > > Below is a message indicating the current policy as decided by the MICE Board in December 2019 is that the MAC address limit is 1. > > > The subject of Jay's message indicates "Static" MAC, but, the text seems to indicate a limit of 1. I like the 1 address limit, but, am not a fan of Static. Was the current issue caused by somebody not having a proper MAC address limit installed? > > > On 12/13/19 18:00, Richard Laager wrote: > > The board has approved the change in MAC address limit from 5 to 1. > > NOTE: This is orthogonal to the dedicated remote switch question, which > is still pending. Non-dedicated remote switches can, for example, > enforce this per-VLAN. > > We have already confirmed that most participants are using only 1 MAC > address, and some that were using more than 1 have addressed that after > being contacted. As discussed below, this change will be rolled out in a > way to keep everyone working, by grandfathering if necessary. > > On 10/28/19 3:25 PM, Richard Laager wrote: > > We have discussed this a bit in the past, and this came up at the last > UG. I'm looking for feedback from the group before formally proposing > this to the board for a vote. > > I first brought this up to the technical committee, CCing the board. I > have heard no objections. Jeremy is "in favor of it 100%". > > Based on our quick conversation after the UG, I _think_ Anthony is in > favor as well; we both made notes to follow up on this. > > ----- > > I am proposing that MICE change its MAC address limit from 5 MACs per > port to 1 MAC per port. 1 MAC address is enough for normal scenarios and > this would further limit the potential for problems on the fabric. This > restriction is one that SIX and AMS-IX both have, with the latter being > known for their excellent configuration guide for participants: >https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ams-i… > > To be clear, this is still only for end ports, not ports facing remote > switches, of course. The remote switches are responsible for enforcing > the current MAC limit, and would be responsible for changing those > settings as described here. > > If someone is swapping their MICE-facing router, the resulting port flap > on the MICE/remote switch will clear the limit anyway, if they are > directly connected. If they are unable to flap the port (e.g. because > they have someone else's layer 2 gear in the middle), they will have to > either work with that carrier or their switch operator (MICE/remote) to > flap the port. This is a non-zero burden, but I think it's pretty > minimal in practice. Additionally, if someone plans to make an equipment > swap, we would increase the limit in advance, temporarily, upon request. > > For reference, SIX goes further and locks you to a particular MAC > address in an ACL, so you have to contact them to change your MAC. I am > not proposing that. Still, I've been through that a couple of times, and > even that isn't really too big of a deal. So I don't think the change to > a limit of 1 MAC at MICE will be particularly onerous. > > There are a couple of participants who have two IP address sets (one > IPv4 and one IPv6) on the same port. For those ports, the limit > obviously must be at least 2 MACs, but I propose 3 MACs as the limit. If > they're using two IPs, they probably are doing so to get some redundancy > out of it. For this particular use case, needing to flap the port > defeats that redundancy goal, as it breaks the other router. Having a > MAC limit of 3 would allow them to swap a device without needing to flap > the port. We would grandfather these setups until they are looking for a > port-type upgrade; at that point they would need to get to one IP > address set per port (by splitting into two ports or reducing to one IP > address set) or request an exception as outlined below. The exception > process would allow us to better evaluate this use case at that time. > > There are some members who have multiple MACs showing up now, despite > using only one device. For those, I am proposing we set the MAC limit to > however many MACs are currently showing up on their port. That is, they > would be grandfathered at their current situation for now. At the time > of this change, we would encourage them to investigate and fix the > multiple MACs issue at their leisure. In the future, if they are doing a > equipment swap (especially like-for-like), we would again encourage them > to address it, but would otherwise leave their higher limit in place. If > they do a port-type upgrade, we would not bring the grandfathering over. > If they needed, they could ask for an official exception at that time. > > There may be cases where people cannot reasonably fix their routers to > use only 1 MAC, other scenarios I haven't considered, or things that may > come up in the future. For that reason, I think we should allow for the > possibility of exceptions upon request to the board. In practice, I > would expect the board would confer with the technical committee and/or > the technical committee would be bringing these to the board anyway. The > goal is to strike a reasonable balance between protecting the fabric > without preventing networks from connecting. > > > > > On 10/31/22 08:27, Jay Hanke wrote: > > I propose we change the MICE required port security settings to be > hard filtered for a single static mac address per port. > > Over the years, we've had several incidents where floods have occurred > prior to port being error-disabled during a looping event. > > > > ________________________________ > > To unsubscribe from the MICE-DISCUSS list, click the following link: >https://nam02.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.ipho… -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000 Confidentiality Notice: Information contained in this e-mail is being sent to you after appropriate authorization or by legal exception. You are obligated to maintain it in a safe, secure and confidential manner. Re-disclosure without patient consent or as permitted by law is prohibited and may subject you to state and/or federal penalties. This information may also be legally privileged, the disclosure of which is governed by law. This information is intended for the use of the person or entity to which it is addressed. If you are not the intended recipient, or the employee or agent responsible for delivering it to the intended recipient, you are hereby notified that any access, disclosure, copying or distribution of this information is strictly prohibited. If you have received this message by error, please notify the sender immediately to arrange for return or proof of destruction of the information contained in this message. ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Oct. 31, 2022

Re: [EXTERNAL] Re: [MICE-DISCUSS] Proposal: MICE Static MAC address Only

by Kittel, Brady

Was the source of the issue a remote switch? " To be clear, this is still only for end ports, not ports facing remote switches, of course. The remote switches are responsible for enforcing the current MAC limit, and would be responsible for changing those settings as described here." -Brady ________________________________ From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> on behalf of Jay Hanke <jayhanke(a)SOUTHFRONT.IO> Sent: Monday, October 31, 2022 9:05:08 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> Subject: [EXTERNAL] Re: [MICE-DISCUSS] Proposal: MICE Static MAC address Only CAUTION: This email originated from outside of HCMC. DO NOT CLICK links or open attachments unless you recognize the sender and know the content is safe. The error disable takes 1-2 seconds to clamp down and disable the port. This results in a short, but severe, broadcast storm. The churn rate on members changing mac addresses is fairly low. I agree that we should automate the process so it can be done via self-service by the members. On Mon, Oct 31, 2022 at 8:59 AM Steve Howard <showard(a)paulbunyan.net> wrote: > > > Below is a message indicating the current policy as decided by the MICE Board in December 2019 is that the MAC address limit is 1. > > > The subject of Jay's message indicates "Static" MAC, but, the text seems to indicate a limit of 1. I like the 1 address limit, but, am not a fan of Static. Was the current issue caused by somebody not having a proper MAC address limit installed? > > > On 12/13/19 18:00, Richard Laager wrote: > > The board has approved the change in MAC address limit from 5 to 1. > > NOTE: This is orthogonal to the dedicated remote switch question, which > is still pending. Non-dedicated remote switches can, for example, > enforce this per-VLAN. > > We have already confirmed that most participants are using only 1 MAC > address, and some that were using more than 1 have addressed that after > being contacted. As discussed below, this change will be rolled out in a > way to keep everyone working, by grandfathering if necessary. > > On 10/28/19 3:25 PM, Richard Laager wrote: > > We have discussed this a bit in the past, and this came up at the last > UG. I'm looking for feedback from the group before formally proposing > this to the board for a vote. > > I first brought this up to the technical committee, CCing the board. I > have heard no objections. Jeremy is "in favor of it 100%". > > Based on our quick conversation after the UG, I _think_ Anthony is in > favor as well; we both made notes to follow up on this. > > ----- > > I am proposing that MICE change its MAC address limit from 5 MACs per > port to 1 MAC per port. 1 MAC address is enough for normal scenarios and > this would further limit the potential for problems on the fabric. This > restriction is one that SIX and AMS-IX both have, with the latter being > known for their excellent configuration guide for participants: > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ams-i… > > To be clear, this is still only for end ports, not ports facing remote > switches, of course. The remote switches are responsible for enforcing > the current MAC limit, and would be responsible for changing those > settings as described here. > > If someone is swapping their MICE-facing router, the resulting port flap > on the MICE/remote switch will clear the limit anyway, if they are > directly connected. If they are unable to flap the port (e.g. because > they have someone else's layer 2 gear in the middle), they will have to > either work with that carrier or their switch operator (MICE/remote) to > flap the port. This is a non-zero burden, but I think it's pretty > minimal in practice. Additionally, if someone plans to make an equipment > swap, we would increase the limit in advance, temporarily, upon request. > > For reference, SIX goes further and locks you to a particular MAC > address in an ACL, so you have to contact them to change your MAC. I am > not proposing that. Still, I've been through that a couple of times, and > even that isn't really too big of a deal. So I don't think the change to > a limit of 1 MAC at MICE will be particularly onerous. > > There are a couple of participants who have two IP address sets (one > IPv4 and one IPv6) on the same port. For those ports, the limit > obviously must be at least 2 MACs, but I propose 3 MACs as the limit. If > they're using two IPs, they probably are doing so to get some redundancy > out of it. For this particular use case, needing to flap the port > defeats that redundancy goal, as it breaks the other router. Having a > MAC limit of 3 would allow them to swap a device without needing to flap > the port. We would grandfather these setups until they are looking for a > port-type upgrade; at that point they would need to get to one IP > address set per port (by splitting into two ports or reducing to one IP > address set) or request an exception as outlined below. The exception > process would allow us to better evaluate this use case at that time. > > There are some members who have multiple MACs showing up now, despite > using only one device. For those, I am proposing we set the MAC limit to > however many MACs are currently showing up on their port. That is, they > would be grandfathered at their current situation for now. At the time > of this change, we would encourage them to investigate and fix the > multiple MACs issue at their leisure. In the future, if they are doing a > equipment swap (especially like-for-like), we would again encourage them > to address it, but would otherwise leave their higher limit in place. If > they do a port-type upgrade, we would not bring the grandfathering over. > If they needed, they could ask for an official exception at that time. > > There may be cases where people cannot reasonably fix their routers to > use only 1 MAC, other scenarios I haven't considered, or things that may > come up in the future. For that reason, I think we should allow for the > possibility of exceptions upon request to the board. In practice, I > would expect the board would confer with the technical committee and/or > the technical committee would be bringing these to the board anyway. The > goal is to strike a reasonable balance between protecting the fabric > without preventing networks from connecting. > > > > > On 10/31/22 08:27, Jay Hanke wrote: > > I propose we change the MICE required port security settings to be > hard filtered for a single static mac address per port. > > Over the years, we've had several incidents where floods have occurred > prior to port being error-disabled during a looping event. > > > > ________________________________ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > https://nam02.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.ipho… -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

Oct. 31, 2022

Re: Proposal: MICE Static MAC address Only

by Jay Hanke

The error disable takes 1-2 seconds to clamp down and disable the port. This results in a short, but severe, broadcast storm. The churn rate on members changing mac addresses is fairly low. I agree that we should automate the process so it can be done via self-service by the members. On Mon, Oct 31, 2022 at 8:59 AM Steve Howard <showard(a)paulbunyan.net> wrote: > > > Below is a message indicating the current policy as decided by the MICE Board in December 2019 is that the MAC address limit is 1. > > > The subject of Jay's message indicates "Static" MAC, but, the text seems to indicate a limit of 1. I like the 1 address limit, but, am not a fan of Static. Was the current issue caused by somebody not having a proper MAC address limit installed? > > > On 12/13/19 18:00, Richard Laager wrote: > > The board has approved the change in MAC address limit from 5 to 1. > > NOTE: This is orthogonal to the dedicated remote switch question, which > is still pending. Non-dedicated remote switches can, for example, > enforce this per-VLAN. > > We have already confirmed that most participants are using only 1 MAC > address, and some that were using more than 1 have addressed that after > being contacted. As discussed below, this change will be rolled out in a > way to keep everyone working, by grandfathering if necessary. > > On 10/28/19 3:25 PM, Richard Laager wrote: > > We have discussed this a bit in the past, and this came up at the last > UG. I'm looking for feedback from the group before formally proposing > this to the board for a vote. > > I first brought this up to the technical committee, CCing the board. I > have heard no objections. Jeremy is "in favor of it 100%". > > Based on our quick conversation after the UG, I _think_ Anthony is in > favor as well; we both made notes to follow up on this. > > ----- > > I am proposing that MICE change its MAC address limit from 5 MACs per > port to 1 MAC per port. 1 MAC address is enough for normal scenarios and > this would further limit the potential for problems on the fabric. This > restriction is one that SIX and AMS-IX both have, with the latter being > known for their excellent configuration guide for participants: > https://www.ams-ix.net/ams/documentation/config-guide > > To be clear, this is still only for end ports, not ports facing remote > switches, of course. The remote switches are responsible for enforcing > the current MAC limit, and would be responsible for changing those > settings as described here. > > If someone is swapping their MICE-facing router, the resulting port flap > on the MICE/remote switch will clear the limit anyway, if they are > directly connected. If they are unable to flap the port (e.g. because > they have someone else's layer 2 gear in the middle), they will have to > either work with that carrier or their switch operator (MICE/remote) to > flap the port. This is a non-zero burden, but I think it's pretty > minimal in practice. Additionally, if someone plans to make an equipment > swap, we would increase the limit in advance, temporarily, upon request. > > For reference, SIX goes further and locks you to a particular MAC > address in an ACL, so you have to contact them to change your MAC. I am > not proposing that. Still, I've been through that a couple of times, and > even that isn't really too big of a deal. So I don't think the change to > a limit of 1 MAC at MICE will be particularly onerous. > > There are a couple of participants who have two IP address sets (one > IPv4 and one IPv6) on the same port. For those ports, the limit > obviously must be at least 2 MACs, but I propose 3 MACs as the limit. If > they're using two IPs, they probably are doing so to get some redundancy > out of it. For this particular use case, needing to flap the port > defeats that redundancy goal, as it breaks the other router. Having a > MAC limit of 3 would allow them to swap a device without needing to flap > the port. We would grandfather these setups until they are looking for a > port-type upgrade; at that point they would need to get to one IP > address set per port (by splitting into two ports or reducing to one IP > address set) or request an exception as outlined below. The exception > process would allow us to better evaluate this use case at that time. > > There are some members who have multiple MACs showing up now, despite > using only one device. For those, I am proposing we set the MAC limit to > however many MACs are currently showing up on their port. That is, they > would be grandfathered at their current situation for now. At the time > of this change, we would encourage them to investigate and fix the > multiple MACs issue at their leisure. In the future, if they are doing a > equipment swap (especially like-for-like), we would again encourage them > to address it, but would otherwise leave their higher limit in place. If > they do a port-type upgrade, we would not bring the grandfathering over. > If they needed, they could ask for an official exception at that time. > > There may be cases where people cannot reasonably fix their routers to > use only 1 MAC, other scenarios I haven't considered, or things that may > come up in the future. For that reason, I think we should allow for the > possibility of exceptions upon request to the board. In practice, I > would expect the board would confer with the technical committee and/or > the technical committee would be bringing these to the board anyway. The > goal is to strike a reasonable balance between protecting the fabric > without preventing networks from connecting. > > > > > On 10/31/22 08:27, Jay Hanke wrote: > > I propose we change the MICE required port security settings to be > hard filtered for a single static mac address per port. > > Over the years, we've had several incidents where floods have occurred > prior to port being error-disabled during a looping event. > > > > ________________________________ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

Oct. 31, 2022

Re: Proposal: MICE Static MAC address Only

by Andrew Hoyos

I disagree with static MAC address filtering unless we beef up operational ability to change/adjust in a reasonable timeframe, or offer a self service portal to do so (ie IXP Manager implementation first) Sent from my iPhone > On Oct 31, 2022, at 9:59 PM, Steve Howard <showard(a)paulbunyan.net> wrote: > > > Below is a message indicating the current policy as decided by the MICE Board in December 2019 is that the MAC address limit is 1. > > > The subject of Jay's message indicates "Static" MAC, but, the text seems to indicate a limit of 1. I like the 1 address limit, but, am not a fan of Static. Was the current issue caused by somebody not having a proper MAC address limit installed? > > > On 12/13/19 18:00, Richard Laager wrote: >> The board has approved the change in MAC address limit from 5 to 1. >> >> NOTE: This is orthogonal to the dedicated remote switch question, which >> is still pending. Non-dedicated remote switches can, for example, >> enforce this per-VLAN. >> >> We have already confirmed that most participants are using only 1 MAC >> address, and some that were using more than 1 have addressed that after >> being contacted. As discussed below, this change will be rolled out in a >> way to keep everyone working, by grandfathering if necessary. >> >> On 10/28/19 3:25 PM, Richard Laager wrote: >>> We have discussed this a bit in the past, and this came up at the last >>> UG. I'm looking for feedback from the group before formally proposing >>> this to the board for a vote. >>> >>> I first brought this up to the technical committee, CCing the board. I >>> have heard no objections. Jeremy is "in favor of it 100%". >>> >>> Based on our quick conversation after the UG, I _think_ Anthony is in >>> favor as well; we both made notes to follow up on this. >>> >>> ----- >>> >>> I am proposing that MICE change its MAC address limit from 5 MACs per >>> port to 1 MAC per port. 1 MAC address is enough for normal scenarios and >>> this would further limit the potential for problems on the fabric. This >>> restriction is one that SIX and AMS-IX both have, with the latter being >>> known for their excellent configuration guide for participants: >>> https://www.ams-ix.net/ams/documentation/config-guide >>> >>> To be clear, this is still only for end ports, not ports facing remote >>> switches, of course. The remote switches are responsible for enforcing >>> the current MAC limit, and would be responsible for changing those >>> settings as described here. >>> >>> If someone is swapping their MICE-facing router, the resulting port flap >>> on the MICE/remote switch will clear the limit anyway, if they are >>> directly connected. If they are unable to flap the port (e.g. because >>> they have someone else's layer 2 gear in the middle), they will have to >>> either work with that carrier or their switch operator (MICE/remote) to >>> flap the port. This is a non-zero burden, but I think it's pretty >>> minimal in practice. Additionally, if someone plans to make an equipment >>> swap, we would increase the limit in advance, temporarily, upon request. >>> >>> For reference, SIX goes further and locks you to a particular MAC >>> address in an ACL, so you have to contact them to change your MAC. I am >>> not proposing that. Still, I've been through that a couple of times, and >>> even that isn't really too big of a deal. So I don't think the change to >>> a limit of 1 MAC at MICE will be particularly onerous. >>> >>> There are a couple of participants who have two IP address sets (one >>> IPv4 and one IPv6) on the same port. For those ports, the limit >>> obviously must be at least 2 MACs, but I propose 3 MACs as the limit. If >>> they're using two IPs, they probably are doing so to get some redundancy >>> out of it. For this particular use case, needing to flap the port >>> defeats that redundancy goal, as it breaks the other router. Having a >>> MAC limit of 3 would allow them to swap a device without needing to flap >>> the port. We would grandfather these setups until they are looking for a >>> port-type upgrade; at that point they would need to get to one IP >>> address set per port (by splitting into two ports or reducing to one IP >>> address set) or request an exception as outlined below. The exception >>> process would allow us to better evaluate this use case at that time. >>> >>> There are some members who have multiple MACs showing up now, despite >>> using only one device. For those, I am proposing we set the MAC limit to >>> however many MACs are currently showing up on their port. That is, they >>> would be grandfathered at their current situation for now. At the time >>> of this change, we would encourage them to investigate and fix the >>> multiple MACs issue at their leisure. In the future, if they are doing a >>> equipment swap (especially like-for-like), we would again encourage them >>> to address it, but would otherwise leave their higher limit in place. If >>> they do a port-type upgrade, we would not bring the grandfathering over. >>> If they needed, they could ask for an official exception at that time. >>> >>> There may be cases where people cannot reasonably fix their routers to >>> use only 1 MAC, other scenarios I haven't considered, or things that may >>> come up in the future. For that reason, I think we should allow for the >>> possibility of exceptions upon request to the board. In practice, I >>> would expect the board would confer with the technical committee and/or >>> the technical committee would be bringing these to the board anyway. The >>> goal is to strike a reasonable balance between protecting the fabric >>> without preventing networks from connecting. >>> > > > > On 10/31/22 08:27, Jay Hanke wrote: >> I propose we change the MICE required port security settings to be >> hard filtered for a single static mac address per port. >> >> Over the years, we've had several incidents where floods have occurred >> prior to port being error-disabled during a looping event. >> > > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Oct. 31, 2022

Re: Proposal: MICE Static MAC address Only

by Steve Howard

Oct. 31, 2022

Re: Proposal: MICE Static MAC address Only

by Gary Glissendorf

I agree. Connections should be going into to routers but not everyone does that. Single Mac protects everyones traffic. Gary Glissendorf | Network Architect gary.glissendorf(a)sdncommunications.com -----Original Message----- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Jay Hanke Sent: Monday, October 31, 2022 8:27 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] Proposal: MICE Static MAC address Only I propose we change the MICE required port security settings to be hard filtered for a single static mac address per port. Over the years, we've had several incidents where floods have occurred prior to port being error-disabled during a looping event. -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000 ***This message and any attachments are solely for the intended recipient. If you are not the intended recipient, disclosure, copying, use or distribution of the information included in this message is prohibited -- Please immediately and permanently delete.***

Oct. 31, 2022

Re: Proposal: MICE Static MAC address Only

by Jeremy Lumby

I second that proposal Jeremy Lumby Minnesota VoIP 9217 17th Ave S Suite 216 Bloomington MN 55425 Main 612-355-7740 Direct 612-392-6814 EFax 952-873-7425 jlumby(a)mnvoip.com -------- Original message -------- From: Jay Hanke <jayhanke(a)SOUTHFRONT.IO> Date: 10/31/22 8:27 AM (GMT-06:00) To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] Proposal: MICE Static MAC address Only _____ >From : Jay Hanke [jayhanke(a)SOUTHFRONT.IO] To : MICE-DISCUSS(a)LISTS.IPHOUSE.NET [MICE-DISCUSS(a)LISTS.IPHOUSE.NET] Date : Monday, October 31 2022 08:27:14 I propose we change the MICE required port security settings to be hard filtered for a single static mac address per port. Over the years, we've had several incidents where floods have occurred prior to port being error-disabled during a looping event. -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

Oct. 31, 2022

Proposal: MICE Static MAC address Only

by Jay Hanke

I propose we change the MICE required port security settings to be hard filtered for a single static mac address per port. Over the years, we've had several incidents where floods have occurred prior to port being error-disabled during a looping event. -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

Oct. 31, 2022

Re: MICE RR #2 having issues..

by Jeremy Lumby

Looks like we had unicast flooding on the core Arista. I cleared the forwarding table, and levels look like they are going back to normal. -----Original Message----- From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Doug McIntyre Sent: Friday, October 28, 2022 11:43 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] MICE RR #2 having issues.. Around 11:30PM CDT tonight, 10/28/2022, the 2nd route-reflector at 206.108.255.2/2001:504:27:0:0:d1af::2 starting having huge packet loss (ie. 85%). The DRAC on this system stopped working a while ago, so I can't get in remotely to see what is up with it. If anybody is around Cologix this weekend that can power cycle the machine, it probably will set it right again. I'm guessing other people are seeing this as well, so putting out a note that this is known.

Oct. 29, 2022

MICE RR #2 having issues..

by Doug McIntyre

Around 11:30PM CDT tonight, 10/28/2022, the 2nd route-reflector at 206.108.255.2/2001:504:27:0:0:d1af::2 starting having huge packet loss (ie. 85%). The DRAC on this system stopped working a while ago, so I can't get in remotely to see what is up with it. If anybody is around Cologix this weekend that can power cycle the machine, it probably will set it right again. I'm guessing other people are seeing this as well, so putting out a note that this is known.

Oct. 29, 2022

Re: [OT] Calix GigaCenter Vulnerability

by Ryan Malek

*_Issue Summary_* Calix has identified a vulnerability on GigaCenters to a SOCKS proxy attack. *_Impact on Services_* The SOCKS proxy attack uses malicious service-blocking rules applied via HTTP API calls using the admin or support user credentials. This configuration causes the GigaCenter to download a SOCKS proxy server and execute a malicious script. It has been observed that the proxy listens on port 8111 and initiates or forwards huge amounts of data or DNS flows, causing CPU and memory exhaustion, resulting in WAN service impacts, and causes the 5G radio to crash. The port that is opened and set as the listening port by the malicious script is a variable, so it is possible that a different port could be used. The affected units do not recover from the 5G crash until a reboot is performed. *_Impacted Systems_* This issue impacts all GigaCenters(844E, 844G, 854G) systems. *_Recommended Action(s)_* * Run a workflow to disable remote access and update the admin and support credentials on all GigaCenters. As a best practice, different passwords should be used for the admin and support users. * On the upstream core router or edge routers, add an ACL to block inbound access to the port used by the SOCKS proxy. * Reboot affected GigaCenters with service impacts. Calix is currently working to identify how to best address GigaCenters which are already affected. Please stay tuned to this community post for more updates. Ryan Malek - Router12 Networks LLC Internet, Phone, and Hosted Services Ph. 641.420.7180 On 10/26/2022 7:06 PM, Brady Kittel wrote: > Richard, this link requires a login. > > On Wed, Oct 26, 2022, 6:28 PM Richard Laager <rlaager(a)wiktel.com> wrote: > > If you run Calix GigaCenters, they are being actively exploited. > > Some details here: > https://community.calix.com/s/feed/0D54u00009074nuCAA > > -- > Richard > > > ------------------------------------------------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > <http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1> > > > ------------------------------------------------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > <http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1> >

Oct. 27, 2022

Re: [OT] Calix GigaCenter Vulnerability

by Brady Kittel

Richard, this link requires a login. On Wed, Oct 26, 2022, 6:28 PM Richard Laager <rlaager(a)wiktel.com> wrote: > If you run Calix GigaCenters, they are being actively exploited. > > Some details here: https://community.calix.com/s/feed/0D54u00009074nuCAA > > -- > Richard > > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

Oct. 27, 2022

[OT] Calix GigaCenter Vulnerability

by Richard Laager

If you run Calix GigaCenters, they are being actively exploited. Some details here: https://community.calix.com/s/feed/0D54u00009074nuCAA -- Richard

Oct. 26, 2022

Please welcome Code42 Software to MICE!

by Jeremy Lumby

Name: Code42 Software Switch: MICE Core Port: E4/19 ASN: 396410 IPv4: This is an ASN change, all other info remains the same IPv6: Peering Contact: iops(a)code42.com Route Servers: Yes BFD: No Welcome Aboard Again! Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 Main: 612-355-7740 Direct: 612-392-6814 Fax: 952-873-7425 jlumby(a)mnvoip.com

Oct. 21, 2022

Re: New MSP IX announced in EdgeConnex

by Chris Wopat

Marketing seems about right. Per the PR they're also joining us down here in Madison in an EdgeConnex facility (FGC) that probably won't get many real users, since the locals are on MadIX already in the closest to what we have to a carrier hotel (222). Perhaps just a 'deal' for space/power for NinjaIX to generate publicity to attract more people to these facilities.

Oct. 20, 2022

Re: New MSP IX announced in EdgeConnex

by Jeremy Lumby

If this was another situation I would say it was competition, however I think this is mostly an EdgeConnex marketing stunt. If MICE ever gets some real competition I think we need to ask what MICE can be doing to fill the void that the competition is trying to fill. For example Andrew mentioned connectivity outside of Cologix. Unless something has changed very recently that I am not aware of there is almost no one in EdgeConnex Minneapolis that is willing and able to peer on an IX. -----Original Message----- From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Dean Bahls Sent: Thursday, October 20, 2022 10:46 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] New MSP IX announced in EdgeConnex I might be ignorant, but this seems like MICE competition to me....so not good for us. What am I missing? Dean -----Original Message----- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Jay Hanke Sent: Thursday, October 20, 2022 8:41 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] New MSP IX announced in EdgeConnex https://newswire.telecomramblings.com/2022/10/edgeconnex-expands-its-intern… -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

Oct. 20, 2022

Re: New MSP IX announced in EdgeConnex

by Jay Hanke

> That seems like a low traffic volume at around 1.5Gbps per participant. According to my rough guess calculations, MICE is around 20Gbps per participant using marketing traffic values. Perhaps EdgeConneX doesn't know about using "marketing" values to report traffic loads. :-) The CDN mix at an IX really skews the per participant traffic numbers. Just a small number of members at MICE generate 80% of the inbound traffic.

Oct. 20, 2022

Re: Anyone lose peering with CloudFlare this morning at 4:13 am?

by Steve Savoy

Thanks for the reply Jeremy! From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Jeremy Lumby Sent: Thursday, October 20, 2022 11:07 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Anyone lose peering with CloudFlare this morning at 4:13 am? Yes, the last time they had a failure in Minneapolis it took them just shy of 2 weeks to get back online. From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Steve Savoy Sent: Thursday, October 20, 2022 10:39 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Anyone lose peering with CloudFlare this morning at 4:13 am? I am pretty new to the CloudFlare peering. Anyone else go through the “Re-routed” scenario before? Will it eventually become “Operational”? Thank you, Steve From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET>> On Behalf Of David Farmer Sent: Wednesday, October 12, 2022 11:59 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Anyone lose peering with CloudFlare this morning at 4:13 am? If you check Cloudflare’s status page you will see Minneapolis is in “Re-routed” state. https://www.cloudflarestatus.com On Wed, Oct 12, 2022 at 10:45 Steve Savoy <stevesavoy(a)wctatel.com<mailto:stevesavoy@wctatel.com>> wrote: Yes – ours went down at the same time. Steve From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET>> On Behalf Of Frank Bulk Sent: Wednesday, October 12, 2022 10:42 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: [MICE-DISCUSS] Anyone lose peering with CloudFlare this morning at 4:13 am? Both IPv4 and IPv6. I didn’t see any exceeding of prefix limits. Frank Oct 12 04:13:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net<http://sxct-north-2.sxcy.fbnt.netins.net>) SiouxCity-Fibernet-Arista Rib: %BGP-5-ADJCHANGE: peer 206.108.255.45 (AS13335.micemn.net<http://AS13335.micemn.net>) (AS 13335 CLOUDFLARENET, US) old state Established event HoldTime new state Idle Oct 12 04:13:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net<http://sxct-north-2.sxcy.fbnt.netins.net>) SiouxCity-Fibernet-Arista Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::3417:0:1 (AS13335.micemn.net<http://AS13335.micemn.net>) (AS 13335 CLOUDFLARENET, US) old state Established event HoldTime new state Idle ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 Confidentiality Notice: This correspondence is the property of Winnebago Cooperative Telecom Association and is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 -- =============================================== David Farmer Email:farmer@umn.edu<mailto:Email%3Afarmer@umn.edu> Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 =============================================== ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Oct. 20, 2022

Re: Anyone lose peering with CloudFlare this morning at 4:13 am?

by Jeremy Lumby

Yes, the last time they had a failure in Minneapolis it took them just shy of 2 weeks to get back online. From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Steve Savoy Sent: Thursday, October 20, 2022 10:39 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Anyone lose peering with CloudFlare this morning at 4:13 am? I am pretty new to the CloudFlare peering. Anyone else go through the “Re-routed” scenario before? Will it eventually become “Operational”? Thank you, Steve From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of David Farmer Sent: Wednesday, October 12, 2022 11:59 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Anyone lose peering with CloudFlare this morning at 4:13 am? If you check Cloudflare’s status page you will see Minneapolis is in “Re-routed” state. https://www.cloudflarestatus.com On Wed, Oct 12, 2022 at 10:45 Steve Savoy <stevesavoy(a)wctatel.com> wrote: Yes – ours went down at the same time. Steve From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Frank Bulk Sent: Wednesday, October 12, 2022 10:42 AM To:MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] Anyone lose peering with CloudFlare this morning at 4:13 am? Both IPv4 and IPv6. I didn’t see any exceeding of prefix limits. Frank Oct 12 04:13:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net <http://sxct-north-2.sxcy.fbnt.netins.net>) SiouxCity-Fibernet-Arista Rib: %BGP-5-ADJCHANGE: peer 206.108.255.45 (AS13335.micemn.net <http://AS13335.micemn.net>) (AS 13335 CLOUDFLARENET, US) old state Established event HoldTime new state Idle Oct 12 04:13:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net <http://sxct-north-2.sxcy.fbnt.netins.net>) SiouxCity-Fibernet-Arista Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::3417:0:1 (AS13335.micemn.net <http://AS13335.micemn.net>) (AS 13335 CLOUDFLARENET, US) old state Established event HoldTime new state Idle --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 Confidentiality Notice: This correspondence is the property of Winnebago Cooperative Telecom Association and is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 -- =============================================== David Farmer Email:farmer@umn.edu <Email%3Afarmer(a)umn.edu> Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 =============================================== ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Oct. 20, 2022

Re: New MSP IX announced in EdgeConnex

by Andrew Hoyos

Competition and diversity are always a good thing. It means MSP is on the map enough to attract two IX’s, and offers an alternative outside of Cologix ecosystem for interconnection. Sent from my iPhone > On Oct 20, 2022, at 11:45 PM, Dean Bahls <dean.bahls(a)fiberminnesota.com> wrote: > I might be ignorant, but this seems like MICE competition to me....so not good for us. What am I missing? > > Dean > > > -----Original Message----- > From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Jay Hanke > Sent: Thursday, October 20, 2022 8:41 AM > To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET > Subject: [MICE-DISCUSS] New MSP IX announced in EdgeConnex > > https://newswire.telecomramblings.com/2022/10/edgeconnex-expands-its-intern… > > -- > Jay Hanke, President > South Front Networks > jayhanke(a)southfront.io > Phone 612-204-0000

Oct. 20, 2022

Re: New MSP IX announced in EdgeConnex

by Steve Howard

Oct. 20, 2022

Re: New MSP IX announced in EdgeConnex

by Dean Bahls

I might be ignorant, but this seems like MICE competition to me....so not good for us. What am I missing? Dean -----Original Message----- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Jay Hanke Sent: Thursday, October 20, 2022 8:41 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] New MSP IX announced in EdgeConnex https://newswire.telecomramblings.com/2022/10/edgeconnex-expands-its-intern… -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

Oct. 20, 2022

Re: New MSP IX announced in EdgeConnex

by Aaron Wendel

Wow. Ninja-IX. Minneapolis is really stepping up it's game. Aaron On 10/20/2022 8:40 AM, Jay Hanke wrote: > https://newswire.telecomramblings.com/2022/10/edgeconnex-expands-its-intern… >

Oct. 20, 2022

Re: Anyone lose peering with CloudFlare this morning at 4:13 am?

by Steve Savoy

I am pretty new to the CloudFlare peering. Anyone else go through the “Re-routed” scenario before? Will it eventually become “Operational”? Thank you, Steve From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of David Farmer Sent: Wednesday, October 12, 2022 11:59 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Anyone lose peering with CloudFlare this morning at 4:13 am? If you check Cloudflare’s status page you will see Minneapolis is in “Re-routed” state. https://www.cloudflarestatus.com On Wed, Oct 12, 2022 at 10:45 Steve Savoy <stevesavoy(a)wctatel.com<mailto:stevesavoy@wctatel.com>> wrote: Yes – ours went down at the same time. Steve From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET>> On Behalf Of Frank Bulk Sent: Wednesday, October 12, 2022 10:42 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: [MICE-DISCUSS] Anyone lose peering with CloudFlare this morning at 4:13 am? Both IPv4 and IPv6. I didn’t see any exceeding of prefix limits. Frank Oct 12 04:13:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net<http://sxct-north-2.sxcy.fbnt.netins.net>) SiouxCity-Fibernet-Arista Rib: %BGP-5-ADJCHANGE: peer 206.108.255.45 (AS13335.micemn.net<http://AS13335.micemn.net>) (AS 13335 CLOUDFLARENET, US) old state Established event HoldTime new state Idle Oct 12 04:13:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net<http://sxct-north-2.sxcy.fbnt.netins.net>) SiouxCity-Fibernet-Arista Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::3417:0:1 (AS13335.micemn.net<http://AS13335.micemn.net>) (AS 13335 CLOUDFLARENET, US) old state Established event HoldTime new state Idle ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 Confidentiality Notice: This correspondence is the property of Winnebago Cooperative Telecom Association and is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 -- =============================================== David Farmer Email:farmer@umn.edu<mailto:Email%3Afarmer@umn.edu> Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 =============================================== ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Oct. 20, 2022

Re: New MSP IX announced in EdgeConnex

by Jeff Wilde

That is cool Jeff Wilde • Network Engineer • Park Region Office 218.826.6161 • Direct 218.826.8330 • Fax 218.826.6298 -----Original Message----- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Jay Hanke Sent: Thursday, October 20, 2022 8:41 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] New MSP IX announced in EdgeConnex https://newswire.telecomramblings.com/2022/10/edgeconnex-expands-its-intern… -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

Oct. 20, 2022

New MSP IX announced in EdgeConnex

by Jay Hanke

https://newswire.telecomramblings.com/2022/10/edgeconnex-expands-its-intern… -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

Oct. 20, 2022

Re: Anyone lose peering with CloudFlare this morning at 4:13 am?

by Steve Savoy

Thanks for the link David! Steve From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of David Farmer Sent: Wednesday, October 12, 2022 11:59 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Anyone lose peering with CloudFlare this morning at 4:13 am? If you check Cloudflare’s status page you will see Minneapolis is in “Re-routed” state. https://www.cloudflarestatus.com On Wed, Oct 12, 2022 at 10:45 Steve Savoy <stevesavoy(a)wctatel.com<mailto:stevesavoy@wctatel.com>> wrote: Yes – ours went down at the same time. Steve From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET>> On Behalf Of Frank Bulk Sent: Wednesday, October 12, 2022 10:42 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: [MICE-DISCUSS] Anyone lose peering with CloudFlare this morning at 4:13 am? Both IPv4 and IPv6. I didn’t see any exceeding of prefix limits. Frank Oct 12 04:13:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net<http://sxct-north-2.sxcy.fbnt.netins.net>) SiouxCity-Fibernet-Arista Rib: %BGP-5-ADJCHANGE: peer 206.108.255.45 (AS13335.micemn.net<http://AS13335.micemn.net>) (AS 13335 CLOUDFLARENET, US) old state Established event HoldTime new state Idle Oct 12 04:13:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net<http://sxct-north-2.sxcy.fbnt.netins.net>) SiouxCity-Fibernet-Arista Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::3417:0:1 (AS13335.micemn.net<http://AS13335.micemn.net>) (AS 13335 CLOUDFLARENET, US) old state Established event HoldTime new state Idle ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 Confidentiality Notice: This correspondence is the property of Winnebago Cooperative Telecom Association and is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 -- =============================================== David Farmer Email:farmer@umn.edu<mailto:Email%3Afarmer@umn.edu> Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 =============================================== ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Oct. 12, 2022

Re: Anyone lose peering with CloudFlare this morning at 4:13 am?

by David Farmer

If you check Cloudflare’s status page you will see Minneapolis is in “Re-routed” state. https://www.cloudflarestatus.com On Wed, Oct 12, 2022 at 10:45 Steve Savoy <stevesavoy(a)wctatel.com> wrote: > Yes – ours went down at the same time. > > > > Steve > > > > *From:* MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> * On Behalf Of *Frank > Bulk > *Sent:* Wednesday, October 12, 2022 10:42 AM > *To:* MICE-DISCUSS(a)LISTS.IPHOUSE.NET > *Subject:* [MICE-DISCUSS] Anyone lose peering with CloudFlare this > morning at 4:13 am? > > > > Both IPv4 and IPv6. I didn’t see any exceeding of prefix limits. > > > > Frank > > > > Oct 12 04:13:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net) > SiouxCity-Fibernet-Arista Rib: %BGP-5-ADJCHANGE: peer 206.108.255.45 ( > AS13335.micemn.net) (AS 13335 CLOUDFLARENET, US) old state Established > event HoldTime new state Idle > > Oct 12 04:13:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net) > SiouxCity-Fibernet-Arista Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::3417:0:1 > (AS13335.micemn.net) (AS 13335 CLOUDFLARENET, US) old state Established > event HoldTime new state Idle > > > > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > > Confidentiality Notice: This correspondence is the property of Winnebago > Cooperative Telecom Association and is for the sole use of the intended > recipient(s) and may contain confidential and privileged information. Any > unauthorized review, use, disclosure or distribution is prohibited. If you > are not the intended recipient, please contact the sender by reply email > and destroy all copies of the original message. > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > -- =============================================== David Farmer Email:farmer@umn.edu Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 ===============================================

Oct. 12, 2022

Re: Anyone lose peering with CloudFlare this morning at 4:13 am?

by Steve Savoy

Yes - ours went down at the same time. Steve From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Frank Bulk Sent: Wednesday, October 12, 2022 10:42 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] Anyone lose peering with CloudFlare this morning at 4:13 am? Both IPv4 and IPv6. I didn't see any exceeding of prefix limits. Frank Oct 12 04:13:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net) SiouxCity-Fibernet-Arista Rib: %BGP-5-ADJCHANGE: peer 206.108.255.45 (AS13335.micemn.net) (AS 13335 CLOUDFLARENET, US) old state Established event HoldTime new state Idle Oct 12 04:13:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net) SiouxCity-Fibernet-Arista Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::3417:0:1 (AS13335.micemn.net) (AS 13335 CLOUDFLARENET, US) old state Established event HoldTime new state Idle ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 Confidentiality Notice: This correspondence is the property of Winnebago Cooperative Telecom Association and is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.

Oct. 12, 2022

Anyone lose peering with CloudFlare this morning at 4:13 am?

by Frank Bulk

Both IPv4 and IPv6. I didn't see any exceeding of prefix limits. Frank Oct 12 04:13:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net) SiouxCity-Fibernet-Arista Rib: %BGP-5-ADJCHANGE: peer 206.108.255.45 (AS13335.micemn.net) (AS 13335 CLOUDFLARENET, US) old state Established event HoldTime new state Idle Oct 12 04:13:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net) SiouxCity-Fibernet-Arista Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::3417:0:1 (AS13335.micemn.net) (AS 13335 CLOUDFLARENET, US) old state Established event HoldTime new state Idle

Oct. 12, 2022

Please welcome TDS Telecommunications to MICE!

by Jeremy Lumby

Name: TDS Telecommunications Switch: MICE Core Port: E6/27/1 ASN: 4181 IPv4: This is a port upgrade, all IP info remains the same. The new connection will be 100G IPv6: Peering Contact: peering(a)tdstelecom.com Route Servers: No BFD: No Welcome Aboard Again! Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 Main: 612-355-7740 Direct: 612-392-6814 Fax: 952-873-7425 jlumby(a)mnvoip.com

Oct. 3, 2022

Re: Amazon changes

by Andrew Hoyos

https://www.peeringdb.com/net/1418 NOC Technical Operations peering-to(a)amazon.com Operational issues: If you experience connectivity issues to Amazon, please first visit: http://ec2-reachability.amazonaws.com/. When contacting our Operational alias below, please include trace/debug information (source and destination address) and details for the affected prefixes. This will reduce troubleshooting time. Please do not contact policy addresses for operational support. — Andrew Hoyos hoyosa(a)gmail.com > On Sep 30, 2022, at 10:04 AM, Justin Huffman <jhuffman(a)LONGLINES.BIZ> wrote: > > Does anyone know of any BGP changes that were done last night around 9 or 9:30 that would have affected AWS or Amazon? We have 1 customer that we advertise their entire block via BGP and they cannot access several AWS sites. Does anyone have a good Amazon contact? > > Thanks, > > Justin Huffman > Long Lines Broadband > 712-271-5902 (W) > jhuffman(a)longlines.biz > > <image001.jpg> <image002.png> > > > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

Sept. 30, 2022

Amazon changes

by Justin Huffman

Does anyone know of any BGP changes that were done last night around 9 or 9:30 that would have affected AWS or Amazon? We have 1 customer that we advertise their entire block via BGP and they cannot access several AWS sites. Does anyone have a good Amazon contact? Thanks, Justin Huffman Long Lines Broadband 712-271-5902 (W) jhuffman(a)longlines.biz<mailto:jhuffman@longlines.biz> [LL new] [cid:image002.png@01D8D4B4.02AA59F0]

Sept. 30, 2022

Please welcome FiberNet Communications to MICE!

by Jeremy Lumby

Name: FiberNet Communications Switch: MN VoIP - 100G Port: E8/1 ASN: 18883 IPv4: This is a port upgrade, all IP info remains the same. The new connection will be 100G IPv6: Peering Contact: peering(a)fibernetcommunications.net Route Servers: Yes BFD: No Welcome Aboard Again! Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 Main: 612-355-7740 Direct: 612-392-6814 Fax: 952-873-7425 jlumby(a)mnvoip.com

Sept. 20, 2022

MICE UG 41 at 3:00 PM

by Richard Laager

MICE UG 41 is at 3:00 PM at UMN: https://lists.iphouse.net/cgi-bin/wa?A2=MICE-DISCUSS;2c503e19.2209 or via Zoom: https://umn.zoom.us/j/98411644091?pwd=QUxiM3hyaldYeDd5SnJFNC9rd2I5dz09 Meeting ID: 984 1164 4091 Passcode: MDQp15 One tap mobile +16513728299,,98411644091#,,,,*686853# US (Minnesota) +13126266799,,98411644091#,,,,*686853# US (Chicago) For the election, the only nomination was me (Richard Laager), so I am running unopposed. As has been our practice, we will still hold the vote for procedural reasons, even though the result is known. -- Richard

Sept. 20, 2022

Re: Remote access details for tomorrow's meeting?

by David Farmer

Sorry, no, that won't work, below is updated Remote Access Information for tomorrow. MICE UG 41 will be held at the University’s Information Technology Building, 2218 University Ave SE, Minneapolis, MN 55414, Room 185. It is across University Ave SE from Huntington Bank Stadium (Formerly TCF Bank Stadium). See the link to the Campus Map below. This room has a video room system, and remote access will be made available via Zoom video or phone audio, see Zoom details below. Please enter the building through the middle of three doors on the Ontario St SE side of the building. It will be open after 2:30 and will have a sign on it. If not, please knock hard, or call my cell phone (612-812-9952). This door is right outside the meeting room, this avoids everyone traipsing through the building. The Stadium Village Green Line Light Rail Station is less than a block east of the building, kitty-corner at the intersection of University Ave SE and Huron Blvd SE. See the link below for the station's info page. Limited Street Parking is available on Ontario St SE and Beacon St SE. Otherwise, public parking is available in the Oak St and University Ave Ramps, each a little over two blocks away. Also, the Maroon lot is a little further on the north side of the Stadium. See the link below for more parking details. DO NOT park in University Parking Lot 70 on the north of the building, by the front door, between University Ave SE and the building, or Parking Lot 71 near the loading dock to the east of the building, they are contract parking only, and you be ticketed. Also, DO NOT park in the private parking lot to the south of the building; you will likely be tagged, booted, and towed from that parking lot. https://campusmaps.umn.edu/information-technology https://www.metrotransit.org/stadium-village-station https://pts.umn.edu/Park/Basic-Parking-Information/Parking-Facilities ---------- David E Farmer III is inviting you to a scheduled Zoom meeting. Join Zoom Meeting https://umn.zoom.us/j/98411644091?pwd=QUxiM3hyaldYeDd5SnJFNC9rd2I5dz09 Meeting ID: 984 1164 4091 Passcode: MDQp15 One tap mobile +16513728299,,98411644091#,,,,*686853# US (Minnesota) +13126266799,,98411644091#,,,,*686853# US (Chicago) Dial by your location +1 651 372 8299 US (Minnesota) +1 312 626 6799 US (Chicago) +1 646 558 8656 US (New York) +1 301 715 8592 US (Washington DC) +1 346 248 7799 US (Houston) +1 669 900 6833 US (San Jose) +1 253 215 8782 US (Tacoma) Meeting ID: 984 1164 4091 Passcode: 686853 Find your local number: https://umn.zoom.us/u/adbNJkGc6V Join by SIP 98411644091(a)zoomcrc.com Join by H.323 162.255.37.11 (US West) 162.255.36.11 (US East) 221.122.88.195 (China) 115.114.131.7 (India Mumbai) 115.114.115.7 (India Hyderabad) 213.19.144.110 (Amsterdam Netherlands) 213.244.140.110 (Germany) 103.122.166.55 (Australia Sydney) 103.122.167.55 (Australia Melbourne) 209.9.211.110 (Hong Kong SAR) 149.137.40.110 (Singapore) 64.211.144.160 (Brazil) 69.174.57.160 (Canada Toronto) 65.39.152.160 (Canada Vancouver) 207.226.132.110 (Japan Tokyo) 149.137.24.110 (Japan Osaka) Meeting ID: 984 1164 4091 Passcode: 686853 On Mon, Sep 19, 2022 at 3:37 PM Miles McCredie <Miles.McCredie(a)midco.com> wrote: > Are the remote access details at > http://lists.iphouse.net/cgi-bin/wa?A2=MICE-DISCUSS;28135c9a.2107 > accurate for tomorrow’s meeting? (They are from July and mention UG39 so > thought I would check.) > > > > Thanks > > > > > > *Miles McCredie * > Principal Network Engineer I-Core IP > > Office: 6052755192 > Miles.McCredie(a)midco.com > > *Midco.com* > > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > -- =============================================== David Farmer Email:farmer@umn.edu Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 ===============================================

Sept. 20, 2022

Re: Remote access details for tomorrow's meeting?

by Richard Laager

Anthony will confirm tomorrow morning. On 9/19/22 15:37, Miles McCredie wrote: > > Are the remote access details at > http://lists.iphouse.net/cgi-bin/wa?A2=MICE-DISCUSS;28135c9a.2107 > accurate for tomorrow’s meeting? (They are from July and mention UG39 > so thought I would check.) > -- Richard

Sept. 20, 2022

Remote access details for tomorrow's meeting?

by Miles McCredie

Are the remote access details at http://lists.iphouse.net/cgi-bin/wa?A2=MICE-DISCUSS;28135c9a.2107 accurate for tomorrow's meeting? (They are from July and mention UG39 so thought I would check.) Thanks [http://images.midcocomm.com/ES_MidcoLogo.png] Miles McCredie Principal Network Engineer I-Core IP Office: 6052755192 Miles.McCredie(a)midco.com Midco.com

Sept. 19, 2022

Re: RST Hurricane Congestion

by Darin Steffl

Thanks for the info Reid. I'm not sure if you're referencing Zayo or not. They seem to have plenty of fiber issues near the metro. It seems reconvergence takes longer than it should when there's a cut too. We see 2-4 minutes of slow speeds or lost routes/downtime when I assume there's a cut on a path feeding into RST. At one point, a software update was done and it got much better but now it seems to be troublesome again. Congestion is usually bad when there's a cut and I have reported many events over the life of our account. Hopefully you guys have good luck improving things into RST. Thank you On Sat, Sep 17, 2022, 12:28 AM Reid Fishler < 0000000c9d62e487-dmarc-request(a)lists.iphouse.net> wrote: > Just to be clear, this was a fiber on some...legacy RST fiber. It has > multiple 100s for us, so the reroute wasn't as pretty as it should be. We > are actively working on making things a bit better between RST and the MSP > area, but its a work in progress. Sorry for any issues, but I think many of > us know the problematic history of this run of fiber. > > Reid > > > On Fri, Sep 16, 2022 at 12:54 PM Ben Wiechman <ben.wiechman(a)arvig.com> > wrote: > >> We didn't have anything down in that area in our maintenance/outage list. >> Another carrier has been fighting a cut between Winona and Wabasha in a >> difficult location. Maybe HE has some services through them as well? It >> seems out of the way, but... >> >> Ben Wiechman >> >> Director of Network Strategy and Engineering >> >> 320.247.3224 | ben.wiechman(a)arvig.com >> >> Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com >> >> >> >> >> On Thu, Sep 15, 2022 at 9:46 PM Darin Steffl <darin.steffl(a)mnwifi.com> >> wrote: >> >>> It's not football killing us tonight! They finally responded after I >>> called them twice and waited 1 hour, 51 mins for a ticket reply. They said >>> several circuits are down to Rochester and they're trying to balance >>> traffic on the remaining links. It's slowly getting better but we're still >>> down 25% from our normal peak at this time of night. It's better than being >>> down 50% of normal. >>> >>> This seems to be happening more frequently the last 2-3 years or there's >>> more capacity sold, being used, in RST compared to years past when there's >>> been fiber cuts. The slow performance is very noticeable. Less than 2 Mbps >>> download and 10-20% packetloss anywhere outside of RST. >>> >>> Ben, are there any Arvig cuts between RST and Belle Plaine/MSP? I'm >>> pretty sure they have some 100G paths from you guys. >>> >>> On Thu, Sep 15, 2022 at 9:41 PM Ben Wiechman <ben.wiechman(a)arvig.com> >>> wrote: >>> >>>> Thu night football killing the link? >>>> >>>> >>>> Ben Wiechman >>>> Director of IP Strategy and Engineering >>>> Direct: 320.256.0184 >>>> Cell: 320.247.3224 >>>> ben.wiechman(a)arvig.com >>>> 150 Second Street SW | Perham, MN 56573 | arvig.com >>>> >>>> On Thu, Sep 15, 2022, 20:16 Darin Steffl <darin.steffl(a)mnwifi.com> >>>> wrote: >>>> >>>>> Anyone else connected to Hurricane's RST router noticing major >>>>> congestion? It looks like the RST-BLP-MSP path is down and traffic is being >>>>> diverted to CHI first or DVN-OMA. >>>>> >>>>> Any speedtests to ISP's like MiBroadband or KMTelecom that are >>>>> connected to RST are fine but any tests outside of that like to US Internet >>>>> or Carleton College, I'm seeing under 2 Mbps download speeds and 20% packet >>>>> loss. >>>>> >>>>> I reported it to Hurricane at 7:39pm but it's been ongoing longer than >>>>> that. I called 10 minutes after I emailed their NOC and the lady there >>>>> wouldn't transfer me to an engineer and just said they'd reply to my ticket >>>>> after they looked. I stressed the urgency since our traffic levels are only >>>>> 50% of what they should be right now. >>>>> >>>>> -- >>>>> Darin Steffl >>>>> Minnesota WiFi >>>>> www.mnwifi.com >>>>> 507-634-WiFi >>>>> Like us on Facebook <http://www.facebook.com/minnesotawifi> >>>>> >>>>> ------------------------------ >>>>> >>>>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>>>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>>>> >>>> >>>> ------------------------------ >>>> >>>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>>> >>> >>> >>> -- >>> Darin Steffl >>> Minnesota WiFi >>> www.mnwifi.com >>> 507-634-WiFi >>> Like us on Facebook <http://www.facebook.com/minnesotawifi> >>> >>> ------------------------------ >>> >>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>> >> >> ------------------------------ >> >> To unsubscribe from the MICE-DISCUSS list, click the following link: >> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >> > > > -- > Reid Fishler > Senior Director > Hurricane Electric > +1-510-580-4178 > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

Sept. 17, 2022

Re: RST Hurricane Congestion

by Reid Fishler

Just to be clear, this was a fiber on some...legacy RST fiber. It has multiple 100s for us, so the reroute wasn't as pretty as it should be. We are actively working on making things a bit better between RST and the MSP area, but its a work in progress. Sorry for any issues, but I think many of us know the problematic history of this run of fiber. Reid On Fri, Sep 16, 2022 at 12:54 PM Ben Wiechman <ben.wiechman(a)arvig.com> wrote: > We didn't have anything down in that area in our maintenance/outage list. > Another carrier has been fighting a cut between Winona and Wabasha in a > difficult location. Maybe HE has some services through them as well? It > seems out of the way, but... > > Ben Wiechman > > Director of Network Strategy and Engineering > > 320.247.3224 | ben.wiechman(a)arvig.com > > Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com > > > > > On Thu, Sep 15, 2022 at 9:46 PM Darin Steffl <darin.steffl(a)mnwifi.com> > wrote: > >> It's not football killing us tonight! They finally responded after I >> called them twice and waited 1 hour, 51 mins for a ticket reply. They said >> several circuits are down to Rochester and they're trying to balance >> traffic on the remaining links. It's slowly getting better but we're still >> down 25% from our normal peak at this time of night. It's better than being >> down 50% of normal. >> >> This seems to be happening more frequently the last 2-3 years or there's >> more capacity sold, being used, in RST compared to years past when there's >> been fiber cuts. The slow performance is very noticeable. Less than 2 Mbps >> download and 10-20% packetloss anywhere outside of RST. >> >> Ben, are there any Arvig cuts between RST and Belle Plaine/MSP? I'm >> pretty sure they have some 100G paths from you guys. >> >> On Thu, Sep 15, 2022 at 9:41 PM Ben Wiechman <ben.wiechman(a)arvig.com> >> wrote: >> >>> Thu night football killing the link? >>> >>> >>> Ben Wiechman >>> Director of IP Strategy and Engineering >>> Direct: 320.256.0184 >>> Cell: 320.247.3224 >>> ben.wiechman(a)arvig.com >>> 150 Second Street SW | Perham, MN 56573 | arvig.com >>> >>> On Thu, Sep 15, 2022, 20:16 Darin Steffl <darin.steffl(a)mnwifi.com> >>> wrote: >>> >>>> Anyone else connected to Hurricane's RST router noticing major >>>> congestion? It looks like the RST-BLP-MSP path is down and traffic is being >>>> diverted to CHI first or DVN-OMA. >>>> >>>> Any speedtests to ISP's like MiBroadband or KMTelecom that are >>>> connected to RST are fine but any tests outside of that like to US Internet >>>> or Carleton College, I'm seeing under 2 Mbps download speeds and 20% packet >>>> loss. >>>> >>>> I reported it to Hurricane at 7:39pm but it's been ongoing longer than >>>> that. I called 10 minutes after I emailed their NOC and the lady there >>>> wouldn't transfer me to an engineer and just said they'd reply to my ticket >>>> after they looked. I stressed the urgency since our traffic levels are only >>>> 50% of what they should be right now. >>>> >>>> -- >>>> Darin Steffl >>>> Minnesota WiFi >>>> www.mnwifi.com >>>> 507-634-WiFi >>>> Like us on Facebook <http://www.facebook.com/minnesotawifi> >>>> >>>> ------------------------------ >>>> >>>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>>> >>> >>> ------------------------------ >>> >>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>> >> >> >> -- >> Darin Steffl >> Minnesota WiFi >> www.mnwifi.com >> 507-634-WiFi >> Like us on Facebook <http://www.facebook.com/minnesotawifi> >> >> ------------------------------ >> >> To unsubscribe from the MICE-DISCUSS list, click the following link: >> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >> > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > -- Reid Fishler Senior Director Hurricane Electric +1-510-580-4178

Sept. 17, 2022

Re: RST Hurricane Congestion

by Jay Hanke

> We didn't have anything down in that area in our maintenance/outage list. > Another carrier has been fighting a cut between Winona and Wabasha in a difficult location. Maybe HE has some services through them as well? It seems out of the way, but... There's only so many viable hops out of RST that go someplace useful for peering. The stuff heading towards Chicago gets horribly congested during network events. If that cut is on the CP rail between Winona and Wabasha that would remove a good chunk of capacity between MN and Chicago. MICE can help quite a bit with the Chicago problem...

Sept. 16, 2022