- mice-discuss - lists.micemn.net

Please Welcome SBT to MICE!
by Jay Hanke Nov. 18, 2019

Nov. 18, 2019

Name: SBT, Int Port: Ethernet 4 MNVoip Remote Switch ASN: 11255 IPv4: 206.108.255.149/24 IPv6: 2001:504:27:0:0:2bf7::1/64 Peering Contact: peering(a)sbt.net Route Servers- YES BFD: YES Welcome aboard! -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

2 1

Re: random packet loss since yesterday afternoon anyone?
by Jeremy Lumby Nov. 7, 2019

Nov. 7, 2019

I got the same update. I did see loss last night at Peak utilization, however it was nowhere near as bad as the previous night. I am also waiting on notification for them to swap the cards in the cyan gear Jeremy Lumby Minnesota VoIP 9217 17th Ave S Suite 216 Bloomington MN 55425 Main 612-355-7740 Direct 612-392-6814 EFax 952-873-7425 jlumby(a)mnvoip.com -------- Original message -------- From: James Urwiller <james.urwiller(a)TURNKEY-TELECOM.COM> Date: 11/5/19 9:26 AM (GMT-06:00) To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] random packet loss since yesterday afternoon anyone? _____ >From : James Urwiller [james.urwiller(a)TURNKEY-TELECOM.COM] To : MICE-DISCUSS(a)LISTS.IPHOUSE.NET [MICE-DISCUSS(a)LISTS.IPHOUSE.NET] Date : Tuesday, November 5 2019 09:26:57 Last ticket I received they traced out some other customers path (not mine) and told me there was no common path, but they wanted to swap a cyan transport card anyway, and would schedule the maintenance “soon” However I think its better-ish now?? James Urwiller Network Architect Stealth Broadband james.urwiller(a)stealthbroadband.com stealth300 From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Justin Huffman Sent: Tuesday, November 5, 2019 09:17 To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] random packet loss since yesterday afternoon anyone? We also opened up a case with Zayo however they put a loopback on the circuit and did not see any errors apparently. Are you all still seeing loss? If so, what did Zayo come up with? Thanks in advance, Justin From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Jeremy Lumby Sent: Monday, November 4, 2019 11:01 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] random packet loss since yesterday afternoon anyone? WARNING!! This message originated from an External Source. Please use proper judgment and caution when opening attachments, clicking links, or responding to this email. The last few updates I had were that they dispatched a tech to Belle Plaine, then they moved the ticket back to the backbone group who engaged the hardware vendor. I still do not have an ETR, however yesterday I did see a little bit of loss at 8AM, however it got much worse as the traffic to MICE increased in the evening. Currently I am seeing loss, however it is much less than Sunday evening. Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 M: 612-355-7740 D: 612-392-6814 F: 952-873-7425 jlumby(a)mnvoip.com From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of James Urwiller Sent: Monday, November 04, 2019 10:43 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] random packet loss since yesterday afternoon anyone? Only updates I have on my ticket is that a tech is investigating.. it has certainly been sporadic for us. James Urwiller Network Architect Stealth Broadband james.urwiller(a)stealthbroadband.com stealth300 From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Justin Huffman Sent: Monday, November 4, 2019 10:30 To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] random packet loss since yesterday afternoon anyone? We have had a few customers complain about packet loss to google.com today as well and was able to see packet loss from one of our routers up to 511 building. However, I don’t see packet loss anymore, but maybe it is really intermittent. Did you hear any update from Zayo? Justin Long Lines Broadband From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Jeremy Lumby Sent: Monday, November 4, 2019 8:36 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] random packet loss since yesterday afternoon anyone? WARNING!! This message originated from an External Source. Please use proper judgment and caution when opening attachments, clicking links, or responding to this email. I am seeing it on the Zayo path between Belle Plaine, and Minneapolis. I have a ticket open since yesterday afternoon, and they appear to be struggling to nail it down. Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 M: 612-355-7740 D: 612-392-6814 F: 952-873-7425 jlumby(a)mnvoip.com From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of James Urwiller Sent: Monday, November 04, 2019 8:33 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] random packet loss since yesterday afternoon anyone? Greetings all, Since yesterday afternoon I have been experiencing random packet loss to things that traceroute to the MICE exchange. It does not appear to be related to any particular participants path out from the exchange. Is anyone else seeing this? James Urwiller Network Architect Stealth Broadband james.urwiller(a)stealthbroadband.com stealth300 _____ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 _____ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 _____ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 _____ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 _____ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 _____ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

3 6

random packet loss since yesterday afternoon anyone?
by James Urwiller Nov. 5, 2019

Nov. 5, 2019

Greetings all, Since yesterday afternoon I have been experiencing random packet loss to things that traceroute to the MICE exchange. It does not appear to be related to any particular participants path out from the exchange. Is anyone else seeing this? James Urwiller Network Architect Stealth Broadband james.urwiller(a)stealthbroadband.com [image: stealth300]

3 6

Fwd: [arin-announce] Vote Now in the 2019 ARIN Board of Trustees, ARIN Advisory Council, and NRO NC Elections
by David Farmer Oct. 31, 2019

Oct. 31, 2019

Voting has opened for ARIN and will remain open to 6:00 PM ET, Friday, 8 November. Please take the time to vote if you are a designated voting contact for ARIN. I would like to call your attention to a candidate from our community For ARIN Advisory Council; Jeffrey Anderson, Implex.net If you contact me directly, I'm happy to make other recommendations to fill out the slate if you wish. Again Please vote if you are eligible. Thanks ---------- Forwarded message --------- From: ARIN <info(a)arin.net> Date: Thu, Oct 31, 2019 at 3:03 PM Subject: [arin-announce] Vote Now in the 2019 ARIN Board of Trustees, ARIN Advisory Council, and NRO NC Elections To: <arin-announce(a)arin.net> Eligible Voting Contacts from General Members in Good Standing may now cast online ballots in the 2019 ARIN Elections to fill two seats on the ARIN Board of Trustees, five seats on the ARIN Advisory Council, and one seat on the Number Resource Organization Number Council (NRO NC). Online ballots may be cast now through 6:00 PM ET, Friday, 8 November. To vote, simply log in to ARIN Online and look for the “Vote Now” link on your dashboard. To view candidate biographies, please view the ARIN Elections 2019 Voter Guide at: https://www.arin.net/participate/oversight/elections/candidate_bios.pdf You may view video of candidate speeches and the Board Candidate forum that took place at ARIN 44 starting on late Friday afternoon, 1 November, at: https://www.arin.net/participate/oversight/elections/ To view or submit a Statement of Support, please visit the link below. Anyone, regardless of voter status, is eligible to submit a Statement of Support for a candidate. https://www.bigpulse.com/p56129/ During the week of 4 November, all eligible Voting Contacts should receive a personal call from an ARIN representative reminding them to please vote and to answer any election-related questions they may have. Participation in ARIN Elections requires only minutes of a voter’s time and is a key member responsibility. A single cast ballot provides eligible member organizations an opportunity to shape the future of ARIN, our community, and the Internet. For questions about voting, or if you encounter an issue with the election system, please contact a member of the Member Services team immediately via email at members(a)arin.net. You may also submit a question via ARIN Online and direct it to Meetings/Elections. Thank you for your participation in the 2019 ARIN Elections! Regards, Wendy Leedy Member Engagement Specialist American Registry for Internet Numbers (ARIN) _______________________________________________ ARIN-Announce You are receiving this message because you are subscribed to the ARIN Announce Mailing List (ARIN-announce(a)arin.net) Unsubscribe or manage your mailing list subscription at: https://lists.arin.net/mailman/listinfo/arin-announce Please contact info(a)arin.net if you experience any issues. -- =============================================== David Farmer Email:farmer@umn.edu Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 ===============================================

1 0

Please welcome Omnitel to MICE!
by Jay Hanke Oct. 31, 2019

Oct. 31, 2019

Name: Omnitel Communications Port: Ethernet 5 SFN Remote Switch ASN: 62886 IPv4: 206.108.255.148/24 IPv6: 2001:504:27:0:0:f5a6::1/64 Peering Contact: it(a)omnitel.biz Route Servers- YES BFD: YES Welcome aboard! -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

2 1

Traffic Requirements
by Jeremy Lumby Oct. 29, 2019

Oct. 29, 2019

I am wondering if many MICE members have been denied peering with other networks on MICE because you do not meet their traffic level requirements. I have, and I am brainstorming some options. Let me know off list if you fall into this category, and if you want to explore a potential solution with me. Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 M: 612-355-7740 D: 612-392-6814 F: 952-873-7425 jlumby(a)mnvoip.com

1 0

Please welcome Apple to MICE!
by Jay Hanke Oct. 25, 2019

Oct. 25, 2019

Name: Apple Ports: Ethernet 3/14/1 (100G LAG #1), Ethernet 4/49/1 (100G LAG#2) ASN: 714 (LAG Group #1) IPv4: 206.108.255.146/24 IPv6: 2001:504:27:0:0:2ca::1/64 (LAG Group #2) IPv4: 206.108.255.147/24 IPv6: 2001:504:27:0:0:2ca::2/64 Peering Contact: peering(a)group.apple.com Route Servers: NO BFD: NO Please note: Ethernet 4/49/1 has been reclaimed from a member that failed to migrate and did not respond. Welcome aboard! -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

3 2

Re: Zayo to Omaha IX down
by Jeff Dahna Oct. 24, 2019

Oct. 24, 2019

288 ct fiber damaged near Worthington, MN. Zayo ticket 0646hrs - Current Update: Crews are finishing boring on one end and should have the fiber tails in the trailer in about an hour. On the other side they have fiber tails in the trailer beginning to prep.

2 1

Zayo to Omaha IX down
by Jeff Dahna Oct. 24, 2019

Oct. 24, 2019

We lost our route to Omaha IX via Zayo around 2:04 PM CST. Got a ticket in with Zayo. Anyone else down to Omaha via Zayo? Jeff Dahna Windomnet

4 4

Re: Zayo to Omaha IX down
by Justin Huffman Oct. 23, 2019

Oct. 23, 2019

Our link from Sioux City to Minneapolis went down around 2:10 today. This is getting out of hand. I think we've experienced outages every other week for almost 2 months now along with the outage lasting 16+ hours. Does anyone have recommendations for diverse routes up to 511? Justin Long Lines Broadband

1 0

Re: Problem with my connection to MICE
by Keith Anderson Oct. 22, 2019

Oct. 22, 2019

Thanks everyone. Another MICE member reached out to me directly and said they had a similar problem with this Brocade "feature" and advised me how to filter this unwanted traffic. The port now appears to be online and I can re-enable my peering tonight.

2 1

Re: Problem with my connection to MICE
by Keith Anderson Oct. 22, 2019

Oct. 22, 2019

Thanks, Jeremy. I was hoping you could provide just that sort of detail. I'm just not sure how there would be multiple coming from my side when this is a P2P link and the switch port is in route-only mode; I may have to work with Brocade on that. I will look into it, but can you tell me if the MAC addresses appear to be all from the same vendor?

2 1

Re: Problem with my connection to MICE
by Jeremy Lumby Oct. 22, 2019

Oct. 22, 2019

We are seeing Port security errors on your port for too many Mac addresses. The industry standard is one Mac address, however currently mice allows up to five. Jeremy Lumby Minnesota VoIP 9217 17th Ave S Suite 216 Bloomington MN 55425 Main 612-355-7740 Direct 612-392-6814 EFax 952-873-7425 jlumby(a)mnvoip.com -------- Original message -------- From: Keith Anderson <keith.anderson(a)MARCONET.COM> Date: 10/22/19 8:33 AM (GMT-06:00) To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] Problem with my connection to MICE _____ >From : Keith Anderson [keith.anderson(a)MARCONET.COM] To : MICE-DISCUSS(a)LISTS.IPHOUSE.NET [MICE-DISCUSS(a)LISTS.IPHOUSE.NET] Date : Tuesday, October 22 2019 08:23:08 Good morning all. I have a problem with my MICE connection and I'm wondering if anyone can give me some direction. On September 9th, my predecessor disabled our port facing MICE due to BGP flapping (his description). Since there was chatter on this board about a fiber cut that day, he thought it was related to that. Unfortunately, when I re-enabled the port, I saw similar behavior. BGP seems to flap about every three minutes, and my log indicates that link is actually dropping on that port. Even though light levels never seem to drop below -4.4db, I had Cologix check the cross connect, clean the ends of the fiber, and re-seat the sfp. None of that helped. On Wednesday I replaced the SFP, but when I enabled the port, I still saw the same behavior. Last night I disabled all of my BGP peers and brought the port back online for further layer2 testing, but I can't ping any of my peers and I'm not seeing any arp entries on this port. Anyone have any ideas what could be going on or what I should check next? I've included my port config below for reference. The log entries associated with this problem are: Oct 22 08:17:33:I:SYSTEM: port 2/4 link fault status is cleared(physical link goes down 24) Oct 22 08:17:33:I:SYSTEM: port 2/4 is down( local fault) Oct 22 08:17:33:I:System: Interface ethernet 2/4, state down - local fault Oct 22 08:17:09:I:System: Interface ethernet 2/4, state up Oct 22 08:16:45:W: Latched low RX Power warning, port 2/4 Oct 22 08:16:45:A: Latched low RX Power alarm, port 2/4 interface ethernet 2/4 port-name BGP peering with MICE Arista 5/19 XC23175 load-interval 30 no fdp enable no cdp enable route-only ip address xxx.xxx.xxx.xxx/24 no spanning-tree sflow forwarding !

1 0

Problem with my connection to MICE
by Keith Anderson Oct. 22, 2019

Oct. 22, 2019

Good morning all. I have a problem with my MICE connection and I'm wondering if anyone can give me some direction. On September 9th, my predecessor disabled our port facing MICE due to BGP flapping (his description). Since there was chatter on this board about a fiber cut that day, he thought it was related to that. Unfortunately, when I re-enabled the port, I saw similar behavior. BGP seems to flap about every three minutes, and my log indicates that link is actually dropping on that port. Even though light levels never seem to drop below -4.4db, I had Cologix check the cross connect, clean the ends of the fiber, and re-seat the sfp. None of that helped. On Wednesday I replaced the SFP, but when I enabled the port, I still saw the same behavior. Last night I disabled all of my BGP peers and brought the port back online for further layer2 testing, but I can't ping any of my peers and I'm not seeing any arp entries on this port. Anyone have any ideas what could be going on or what I should check next? I've included my port config below for reference. The log entries associated with this problem are: Oct 22 08:17:33:I:SYSTEM: port 2/4 link fault status is cleared(physical link goes down 24) Oct 22 08:17:33:I:SYSTEM: port 2/4 is down( local fault) Oct 22 08:17:33:I:System: Interface ethernet 2/4, state down - local fault Oct 22 08:17:09:I:System: Interface ethernet 2/4, state up Oct 22 08:16:45:W: Latched low RX Power warning, port 2/4 Oct 22 08:16:45:A: Latched low RX Power alarm, port 2/4 interface ethernet 2/4 port-name BGP peering with MICE Arista 5/19 XC23175 load-interval 30 no fdp enable no cdp enable route-only ip address xxx.xxx.xxx.xxx/24 no spanning-tree sflow forwarding !

1 0

2019 Election Results
by AnthonyAnderberg＠nuvera.net Oct. 17, 2019

Oct. 17, 2019

Richard was re-elected unanimously, thanks for your continued service! [cid:60F64B08-B963-4E01-B287-F9A49C3DDDFD-L0-001] Sent from my iPhone

2 1

MICE UG 33
by Richard Laager Oct. 15, 2019

Oct. 15, 2019

Reminder: UG 33 is on Wednesday of this week (October 16) at 3:00pm Remote Conferencing: 952-562-5400 passcode MICE (6423) Location: 511 Building Cafeteria Agenda The agenda is attached. I'm behind on getting this out, so I didn't send a separate call for agenda. If you have something to add, let me know ASAP and I'll send an updated agenda. I'd like to try the modified ordering again. The idea here is to group business (official reports and things that could involve a motion/vote) before open discussion. This way, people who are not interested in the discussion can leave earlier if they wish, confident that they won't miss an official motion/vote. By shortening the "mandatory" part of the meeting, this may make the meetings more accessible. I'd like to give this a try for a couple meetings and see what people think. The UG 32 draft minutes are also attached. -- Richard

1 0

Re: Zayo Maintenance??
by Tim Fjeld Oct. 14, 2019

Oct. 14, 2019

Zayo was in our DC Tuesday afternoon. They were working their way north from Omaha testing fibers and verifying connections in effort to finalize turn up of their new transport equipment. I would imagine the outage might be related to that project, but cannot say with certainty. Tim Fjeld City of Windom

6 10

MN VoIP Remote Upgrade
by Jeremy Lumby Oct. 11, 2019

Oct. 11, 2019

In accordance with the MICE remote switch operator guidelines, MN VoIP will be upgrading their remote switch and uplink to the MICE core on the afternoon of October 11th at approx 2PM to keep uplink utilization under 66% of the link capacity. Prior to the maintenance, BGP will be blocked so traffic can gracefully reroute. During the maintenance connectivity will be down for approximately 15 minutes. A 100G switch will be added between the existing MN VoIP extension and the MICE core. This switch will not only increase the MN VoIP uplink from 40G to 100G, it will also make 32 100G ports available to anyone wishing to connect to MICE via MN VoIP @ 100G. I will send out a notification to the discuss list right before I commence with the maintenance. Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 M: 612-355-7740 D: 612-392-6814 F: 952-873-7425 jlumby(a)mnvoip.com

3 4

Zayo Maintenance??
by James Urwiller Oct. 10, 2019

Oct. 10, 2019

Does anyone know if Zayo is doing maintenance ? I find it strange that my circuit dropped at 00:04 CST. James Urwiller Network Architect Stealth Broadband james.urwiller(a)stealthbroadband.com [image: stealth300]

3 3

MICE 2019 Board Election
by Richard Laager Oct. 4, 2019

Oct. 4, 2019

The MICE board consists of three directors with staggered 3-year terms. One position is up for election each year at our fall meeting. This year's fall meeting, UG 33, will be held on October 16 at the 511 Building. The current seat up for election is the one that I hold. At this time, the board would like to open nominations. If you or someone you know is interested in running, please reply on the MICE-DISCUSS list or contact any board member. Nominations will close September 30th. -- Richard

2 2

Please welcome Land O'Lakes to MICE!
by Jay Hanke Oct. 1, 2019

Oct. 1, 2019

Name: Land O Lakes Port: Ethernet 4/39 Main Switch ASN: 14161 IPv4: 206.108.255.145/24 IPv6: 2001:504:27:0:0:3751::1/64 Peering Contact: kghugill(a)landolakes.com Route Servers- YES BFD: NO Please provide your optic to either a tech committee member (Jeremy) or to Cologix when you place your cross connect order! Also, if you haven't already please join the MICE mailing lists at micemn.net Welcome aboard! -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

2 1

Re: Anyone else see their Zayo-based link to MICE go down?
by Jeff Dahna Sept. 26, 2019

Sept. 26, 2019

latest Zayo update Wed 9/25/2019 4:16 PM Current update Crews have located the damage and are waiting on 811 to perform emergency locates, Estimating 60 minutes. Once the locates are done crews will be digging up 100 ft on each side of of the fiber. The reason for the extra digging is the cable is near a high pressure gas main and crews must be extra careful. ETR is 6 hours. We apologize for any inconveniences and delays along the way and will continue to work towards resolution and provide updates as they are available. Regards Joshua Johnson NCC Technician (Fiber & Facilities) | Zayo Group

4 5

Re: Anyone else see their Zayo-based link to MICE go down?
by Jeff Dahna Sept. 26, 2019

Sept. 26, 2019

Current update 9:51 PM: ETR 2-3 hours

1 0

Re: Anyone else see their Zayo-based link to MICE go down?
by Jeff Dahna Sept. 25, 2019

Sept. 25, 2019

Zayo is responding to our ticket about every hour with the standard general response "Issues are still currently being investigated by the field teams on site"

1 0

Re: Anyone else see their Zayo-based link to MICE go down?
by Jeff Dahna Sept. 25, 2019

Sept. 25, 2019

Zayo ticket update: Wed 9/25/2019 1:31 PM Current update, Zayo OSP is on site searching for the damage, Splicers will be on site in 1 hour, Underground construction team is on site with dig equipment and Trailer with conduit and fiber expected in 90 minutes We apologize for any inconveniences and delays along the way and will continue to work towards resolution and provide updates as they are available. Regards Joshua Johnson NCC Technician (Fiber & Facilities) | Zayo Group

3 2