November 2019 - mice-discuss - lists.micemn.net

UG 28
by Richard Laager Oct. 7, 2020

Oct. 7, 2020

I am pleased to announce that 702 Communications confirmed that July 11 works for them hosting UG 28: MICE User Group 28 July 11, 3:00pm 702 Main Ave, Moorhead, MN -- Richard

7 21

Blackholing DoS traffic
by Frank Bulk Jan. 24, 2020

Jan. 24, 2020

https://www.seattleix.net/blackholing Does MICE have an blackholng functionality equivalent to SIX? I was visiting with a DDoS mitigation vendor this morning and was curious if there was a way we could automatically mitigate DoS attack traffic coming from a MICE peer. Regards, Frank

12 17

WiscNet Remote Switch / Dedicated Remote Switches
by Richard Laager Jan. 14, 2020

Jan. 14, 2020

I realize I am sending this on a holiday. This does _not_ require immediate attention; next week is fine. ---- WiscNet has proposed a remote switch. I'm forwarding their proposal to the MICE-DISCUSS list per our procedure. You can find it at the bottom of this message. If you have any feedback for the board, please share on-list or off-list, as you prefer. Note that the listserv software sets a Reply-To, so if you are trying to reply off-list, you will have to _edit_ the To field in your reply. The interesting question from my perspective, which we discussed a bit at the last UG, is whether MICE should require remote switches to be dedicated to this purpose. It's my understanding that other exchanges, e.g. SIX, have this requirement. The last few remote switches have all been dedicated, so things had trended that way without it being an official MICE policy. The WiscNet proposal is for a non-dedicated switch which is bringing this question to the forefront. Requiring remote switches to be dedicated is intended to help protect the MICE switching fabric. It would provide a clean demarcation point, which is important for applying rules like one-MAC-per-port, etc. It also reduces the chances of problems in practice due to clean separation, less complexity of configuration, etc. Currently, the following remote switches are, to the best of my knowledge, dedicated: Zayo (formerly Neutral Path) Minnesota VoIP South Front Networks Compudyne The following remote switches are not dedicated: CNS I'm not sure about the following, but they're probably not dedicated: Mankato Networks Northern Lights GigaPoP* * This is something we treat somewhat like a remote switch, but I believe they pre-date the remote switch policy. Note that the non-dedicated switches are the original remote switches or their direct successors. I believe the consensus at the last UG when we discussed this was that: - Implicitly: We need to decide the policy. - Either way, the policy needs to apply to all remote switches; i.e. we don't reject some solely for being non-dedicated while accepting others. - If we require remotes to be dedicated, existing non-dedicated switches need to be grandfathered for at least some amount of time. Upgrades to new switches would end grandfathering. I do not feel confident saying there was a consensus on the actual question of whether dedicated switches should be required or not. If you feel I misjudged the consensus of the room in either direction, please let me know. If a dedicated switch was required, WiscNet would have to add an additional switch, which would look like this: [MICE Arista] | | 100G-LR4 | [WiscNet MICE Remote Switch ] * No VLANs, All Untagged | | | 100G-SR4 | 100G-SR4 | | | VLAN 123 | VLAN 456 | untagged | untagged [WiscNet Existing Switch] | | | existing | existing | 100G-LR4 | 100G-LR4 | | [WiscNet MX] [Iowa MX] The VLAN IDs (123, 456) are made-up examples. We discussed this and other alternatives off-list. WiscNet is firmly opposed to adding such an additional switch. If MICE requires dedicated remote switches, WiscNet will not operate a remote switch. They will still upgrade their connectivity to MICE to 100G, but Iowa and MERIT will likely be left out (unless they decide to get their own MICE ports). I (and I assume the rest of the board) would also be especially interested to hear from the other operators of non-dedicated remote switches. Feel free to reply on-list or off-list at your option. As per the usual remote switch procedure, after there's been an opportunity for comments, the next step is for the board to vote on the WiscNet remote switch proposal. Approving this proposal implicitly decides that non-dedicated switches are okay, at least for now. It is my view that we should _explicitly_ decide the dedicated remote switch question one way or another, so I will also ask for a board decision on that, but I cannot _require_ such a decision individually, of course. There is no specific timeline on this, but if you have thoughts, please share them as soon as possible. Finally, while this is already public knowledge, I'd like to specifically disclose that my employer's connection to MICE is on the CNS switch, a non-dedicated remote switch. -------- Forwarded Message -------- Subject: Re: WiscNet - AS2381 - Mice Remote Switch Date: Mon, 18 Nov 2019 13:22:44 +0000 From: Chris Wopat <wopat(a)wiscnet.net> Hi folks, sorry for the multi-week week delay in responding. I've attached a simplified diagram indicating how WiscNet would like to connect to MICE. The diagram is separated by three phases- Current, Proposed, Future. We are looking to transition from our current 10g connection via University of Minnesota Gigapop, to a direct 100g connection from the MICE core Arista switch to WiscNet's recently installed QFX10002. We would like to have the ability to share this connection with other Research and Education organizations that may already connect to our existing switch, as well as a few who may connect to it in the future. Known entities on the interested list are University of Iowa (AS3676) and MERIT Network (AS237). This list may grow a little over time, but it's likely that we'll be able to count this number entities on one hand. WiscNet is a non profit 501(c)(3) who per our bylaws (https://www.wiscnet.net/bylaws) primarily serve educational institutions and do not service commercial entities. When WiscNet recently reached out asking to become a MICE remote switch, we were disappointed to hear that a recent(?) policy change was made indicating that remote switches must be dedicated for MICE usage only. We will not be able to dedicate this device for MICE usage, nor would it be cost effective to purchase a separate switch just for this purpose. We're hoping that there's some middle category here where we're a switch + our few research partners, similar to how a few entities are set up today. Cheers, -- Chris Wopat Network Engineer, WiscNet wopat(a)wiscnet.net 608-210-3965

10 16

MICE MAC Address Limit
by Richard Laager Dec. 14, 2019

Dec. 14, 2019

We have discussed this a bit in the past, and this came up at the last UG. I'm looking for feedback from the group before formally proposing this to the board for a vote. I first brought this up to the technical committee, CCing the board. I have heard no objections. Jeremy is "in favor of it 100%". Based on our quick conversation after the UG, I _think_ Anthony is in favor as well; we both made notes to follow up on this. ----- I am proposing that MICE change its MAC address limit from 5 MACs per port to 1 MAC per port. 1 MAC address is enough for normal scenarios and this would further limit the potential for problems on the fabric. This restriction is one that SIX and AMS-IX both have, with the latter being known for their excellent configuration guide for participants: https://www.ams-ix.net/ams/documentation/config-guide To be clear, this is still only for end ports, not ports facing remote switches, of course. The remote switches are responsible for enforcing the current MAC limit, and would be responsible for changing those settings as described here. If someone is swapping their MICE-facing router, the resulting port flap on the MICE/remote switch will clear the limit anyway, if they are directly connected. If they are unable to flap the port (e.g. because they have someone else's layer 2 gear in the middle), they will have to either work with that carrier or their switch operator (MICE/remote) to flap the port. This is a non-zero burden, but I think it's pretty minimal in practice. Additionally, if someone plans to make an equipment swap, we would increase the limit in advance, temporarily, upon request. For reference, SIX goes further and locks you to a particular MAC address in an ACL, so you have to contact them to change your MAC. I am not proposing that. Still, I've been through that a couple of times, and even that isn't really too big of a deal. So I don't think the change to a limit of 1 MAC at MICE will be particularly onerous. There are a couple of participants who have two IP address sets (one IPv4 and one IPv6) on the same port. For those ports, the limit obviously must be at least 2 MACs, but I propose 3 MACs as the limit. If they're using two IPs, they probably are doing so to get some redundancy out of it. For this particular use case, needing to flap the port defeats that redundancy goal, as it breaks the other router. Having a MAC limit of 3 would allow them to swap a device without needing to flap the port. We would grandfather these setups until they are looking for a port-type upgrade; at that point they would need to get to one IP address set per port (by splitting into two ports or reducing to one IP address set) or request an exception as outlined below. The exception process would allow us to better evaluate this use case at that time. There are some members who have multiple MACs showing up now, despite using only one device. For those, I am proposing we set the MAC limit to however many MACs are currently showing up on their port. That is, they would be grandfathered at their current situation for now. At the time of this change, we would encourage them to investigate and fix the multiple MACs issue at their leisure. In the future, if they are doing a equipment swap (especially like-for-like), we would again encourage them to address it, but would otherwise leave their higher limit in place. If they do a port-type upgrade, we would not bring the grandfathering over. If they needed, they could ask for an official exception at that time. There may be cases where people cannot reasonably fix their routers to use only 1 MAC, other scenarios I haven't considered, or things that may come up in the future. For that reason, I think we should allow for the possibility of exceptions upon request to the board. In practice, I would expect the board would confer with the technical committee and/or the technical committee would be bringing these to the board anyway. The goal is to strike a reasonable balance between protecting the fabric without preventing networks from connecting. -- Richard

2 2

HBC Port Augment/LAG conversion
by Jay Hanke Nov. 25, 2019

Nov. 25, 2019

Ethernet4/11 has been allocated to HBC. Their existing port (e5/11) will need to be converted to a LAG and then e4/11 added into the group. Welcome aboard (again)! -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

2 1

packet loss today
by James Urwiller Nov. 23, 2019

Nov. 23, 2019

Greetings all, I have been noticing packet loss today similar to how it was a few weeks back before Zayo replaced a card. I know they had a fiber cut yesterday in Omaha south of where I hop on to get to the exchange, I was unaffected by that but today I just have a lot of packet loss. Anyone else? James Urwiller Network Architect Stealth Broadband james.urwiller(a)stealthbroadband.com [image: stealth300]

4 11

Please Welcome PhirePhly Design to MICE!
by Jay Hanke Nov. 21, 2019

Nov. 21, 2019

Name: PhirePhly Design Port: Ethernet 11 MN Voip Remote Switch ASN: 7034 IPv4: 206.108.255.152/24 IPv6: 2001:504:27:0:0:1b7a::1/64 Peering Contact: kenneth(a)phirephly.design Route Servers- YES BFD: No Welcome Aboard! -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

2 1

Additional WIN 10G Port
by Jay Hanke Nov. 20, 2019

Nov. 20, 2019

Port 4/43 on the main switch has been allocated to WIN. Please add this port to their existing LAG group. Welcome aboard! -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

2 1

MICE Limelight Upgrade/Route Server addition
by Jay Hanke Nov. 19, 2019

Nov. 19, 2019

Limelight is moving from their existing 2x10G LAG to 1x100G LAG on main switch port 3/15/1. The existing 10G ports will be vacated. This move was competed this afternoon. Limelight is also adding an additional 100G port to the MN VoIP remote switch with an additional IP assignment. Name: Limelight (New additional) Port: MN VoIP Port 2 (100G Switch) ASN: 22882 IPv4: 206.108.255.151 IPv6: 2001:504:27:0:0:5926::2 Limelight has also requested RS sessions for both of their IP assignments. The following Limelight IPs need RS sessions created or verified IPv4: 206.108.255.43 IPv6: 2001:504:27:0:0:5926::1 Welcome aboard! -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

3 2

Re: Google Issues?
by Jeff Dahna Nov. 19, 2019

Nov. 19, 2019

Apparently it is a fiber cut on Level3 as per previous message.

2 1