- mice-discuss - lists.micemn.net

Re: Anyone lose peering with CloudFlare this morning at 4:13 am?

by Steve Savoy

Thanks for the link David! Steve From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of David Farmer Sent: Wednesday, October 12, 2022 11:59 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Anyone lose peering with CloudFlare this morning at 4:13 am? If you check Cloudflare’s status page you will see Minneapolis is in “Re-routed” state. https://www.cloudflarestatus.com On Wed, Oct 12, 2022 at 10:45 Steve Savoy <stevesavoy(a)wctatel.com<mailto:stevesavoy@wctatel.com>> wrote: Yes – ours went down at the same time. Steve From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET>> On Behalf Of Frank Bulk Sent: Wednesday, October 12, 2022 10:42 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: [MICE-DISCUSS] Anyone lose peering with CloudFlare this morning at 4:13 am? Both IPv4 and IPv6. I didn’t see any exceeding of prefix limits. Frank Oct 12 04:13:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net<http://sxct-north-2.sxcy.fbnt.netins.net>) SiouxCity-Fibernet-Arista Rib: %BGP-5-ADJCHANGE: peer 206.108.255.45 (AS13335.micemn.net<http://AS13335.micemn.net>) (AS 13335 CLOUDFLARENET, US) old state Established event HoldTime new state Idle Oct 12 04:13:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net<http://sxct-north-2.sxcy.fbnt.netins.net>) SiouxCity-Fibernet-Arista Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::3417:0:1 (AS13335.micemn.net<http://AS13335.micemn.net>) (AS 13335 CLOUDFLARENET, US) old state Established event HoldTime new state Idle ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 Confidentiality Notice: This correspondence is the property of Winnebago Cooperative Telecom Association and is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 -- =============================================== David Farmer Email:farmer@umn.edu<mailto:Email%3Afarmer@umn.edu> Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 =============================================== ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Oct. 12, 2022

Re: Anyone lose peering with CloudFlare this morning at 4:13 am?

by David Farmer

If you check Cloudflare’s status page you will see Minneapolis is in “Re-routed” state. https://www.cloudflarestatus.com On Wed, Oct 12, 2022 at 10:45 Steve Savoy <stevesavoy(a)wctatel.com> wrote: > Yes – ours went down at the same time. > > > > Steve > > > > *From:* MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> * On Behalf Of *Frank > Bulk > *Sent:* Wednesday, October 12, 2022 10:42 AM > *To:* MICE-DISCUSS(a)LISTS.IPHOUSE.NET > *Subject:* [MICE-DISCUSS] Anyone lose peering with CloudFlare this > morning at 4:13 am? > > > > Both IPv4 and IPv6. I didn’t see any exceeding of prefix limits. > > > > Frank > > > > Oct 12 04:13:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net) > SiouxCity-Fibernet-Arista Rib: %BGP-5-ADJCHANGE: peer 206.108.255.45 ( > AS13335.micemn.net) (AS 13335 CLOUDFLARENET, US) old state Established > event HoldTime new state Idle > > Oct 12 04:13:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net) > SiouxCity-Fibernet-Arista Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::3417:0:1 > (AS13335.micemn.net) (AS 13335 CLOUDFLARENET, US) old state Established > event HoldTime new state Idle > > > > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > > Confidentiality Notice: This correspondence is the property of Winnebago > Cooperative Telecom Association and is for the sole use of the intended > recipient(s) and may contain confidential and privileged information. Any > unauthorized review, use, disclosure or distribution is prohibited. If you > are not the intended recipient, please contact the sender by reply email > and destroy all copies of the original message. > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > -- =============================================== David Farmer Email:farmer@umn.edu Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 ===============================================

Oct. 12, 2022

Re: Anyone lose peering with CloudFlare this morning at 4:13 am?

by Steve Savoy

Yes - ours went down at the same time. Steve From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Frank Bulk Sent: Wednesday, October 12, 2022 10:42 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] Anyone lose peering with CloudFlare this morning at 4:13 am? Both IPv4 and IPv6. I didn't see any exceeding of prefix limits. Frank Oct 12 04:13:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net) SiouxCity-Fibernet-Arista Rib: %BGP-5-ADJCHANGE: peer 206.108.255.45 (AS13335.micemn.net) (AS 13335 CLOUDFLARENET, US) old state Established event HoldTime new state Idle Oct 12 04:13:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net) SiouxCity-Fibernet-Arista Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::3417:0:1 (AS13335.micemn.net) (AS 13335 CLOUDFLARENET, US) old state Established event HoldTime new state Idle ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 Confidentiality Notice: This correspondence is the property of Winnebago Cooperative Telecom Association and is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.

Oct. 12, 2022

Anyone lose peering with CloudFlare this morning at 4:13 am?

by Frank Bulk

Both IPv4 and IPv6. I didn't see any exceeding of prefix limits. Frank Oct 12 04:13:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net) SiouxCity-Fibernet-Arista Rib: %BGP-5-ADJCHANGE: peer 206.108.255.45 (AS13335.micemn.net) (AS 13335 CLOUDFLARENET, US) old state Established event HoldTime new state Idle Oct 12 04:13:10 207.32.15.14 (sxct-north-2.sxcy.fbnt.netins.net) SiouxCity-Fibernet-Arista Rib: %BGP-5-ADJCHANGE: peer 2001:504:27::3417:0:1 (AS13335.micemn.net) (AS 13335 CLOUDFLARENET, US) old state Established event HoldTime new state Idle

Oct. 12, 2022

Please welcome TDS Telecommunications to MICE!

by Jeremy Lumby

Name: TDS Telecommunications Switch: MICE Core Port: E6/27/1 ASN: 4181 IPv4: This is a port upgrade, all IP info remains the same. The new connection will be 100G IPv6: Peering Contact: peering(a)tdstelecom.com Route Servers: No BFD: No Welcome Aboard Again! Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 Main: 612-355-7740 Direct: 612-392-6814 Fax: 952-873-7425 jlumby(a)mnvoip.com

Oct. 3, 2022

Re: Amazon changes

by Andrew Hoyos

https://www.peeringdb.com/net/1418 NOC Technical Operations peering-to(a)amazon.com Operational issues: If you experience connectivity issues to Amazon, please first visit: http://ec2-reachability.amazonaws.com/. When contacting our Operational alias below, please include trace/debug information (source and destination address) and details for the affected prefixes. This will reduce troubleshooting time. Please do not contact policy addresses for operational support. — Andrew Hoyos hoyosa(a)gmail.com > On Sep 30, 2022, at 10:04 AM, Justin Huffman <jhuffman(a)LONGLINES.BIZ> wrote: > > Does anyone know of any BGP changes that were done last night around 9 or 9:30 that would have affected AWS or Amazon? We have 1 customer that we advertise their entire block via BGP and they cannot access several AWS sites. Does anyone have a good Amazon contact? > > Thanks, > > Justin Huffman > Long Lines Broadband > 712-271-5902 (W) > jhuffman(a)longlines.biz > > <image001.jpg> <image002.png> > > > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

Sept. 30, 2022

Amazon changes

by Justin Huffman

Does anyone know of any BGP changes that were done last night around 9 or 9:30 that would have affected AWS or Amazon? We have 1 customer that we advertise their entire block via BGP and they cannot access several AWS sites. Does anyone have a good Amazon contact? Thanks, Justin Huffman Long Lines Broadband 712-271-5902 (W) jhuffman(a)longlines.biz<mailto:jhuffman@longlines.biz> [LL new] [cid:image002.png@01D8D4B4.02AA59F0]

Sept. 30, 2022

Please welcome FiberNet Communications to MICE!

by Jeremy Lumby

Name: FiberNet Communications Switch: MN VoIP - 100G Port: E8/1 ASN: 18883 IPv4: This is a port upgrade, all IP info remains the same. The new connection will be 100G IPv6: Peering Contact: peering(a)fibernetcommunications.net Route Servers: Yes BFD: No Welcome Aboard Again! Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 Main: 612-355-7740 Direct: 612-392-6814 Fax: 952-873-7425 jlumby(a)mnvoip.com

Sept. 20, 2022

MICE UG 41 at 3:00 PM

by Richard Laager

MICE UG 41 is at 3:00 PM at UMN: https://lists.iphouse.net/cgi-bin/wa?A2=MICE-DISCUSS;2c503e19.2209 or via Zoom: https://umn.zoom.us/j/98411644091?pwd=QUxiM3hyaldYeDd5SnJFNC9rd2I5dz09 Meeting ID: 984 1164 4091 Passcode: MDQp15 One tap mobile +16513728299,,98411644091#,,,,*686853# US (Minnesota) +13126266799,,98411644091#,,,,*686853# US (Chicago) For the election, the only nomination was me (Richard Laager), so I am running unopposed. As has been our practice, we will still hold the vote for procedural reasons, even though the result is known. -- Richard

Sept. 20, 2022

Re: Remote access details for tomorrow's meeting?

by David Farmer

Sorry, no, that won't work, below is updated Remote Access Information for tomorrow. MICE UG 41 will be held at the University’s Information Technology Building, 2218 University Ave SE, Minneapolis, MN 55414, Room 185. It is across University Ave SE from Huntington Bank Stadium (Formerly TCF Bank Stadium). See the link to the Campus Map below. This room has a video room system, and remote access will be made available via Zoom video or phone audio, see Zoom details below. Please enter the building through the middle of three doors on the Ontario St SE side of the building. It will be open after 2:30 and will have a sign on it. If not, please knock hard, or call my cell phone (612-812-9952). This door is right outside the meeting room, this avoids everyone traipsing through the building. The Stadium Village Green Line Light Rail Station is less than a block east of the building, kitty-corner at the intersection of University Ave SE and Huron Blvd SE. See the link below for the station's info page. Limited Street Parking is available on Ontario St SE and Beacon St SE. Otherwise, public parking is available in the Oak St and University Ave Ramps, each a little over two blocks away. Also, the Maroon lot is a little further on the north side of the Stadium. See the link below for more parking details. DO NOT park in University Parking Lot 70 on the north of the building, by the front door, between University Ave SE and the building, or Parking Lot 71 near the loading dock to the east of the building, they are contract parking only, and you be ticketed. Also, DO NOT park in the private parking lot to the south of the building; you will likely be tagged, booted, and towed from that parking lot. https://campusmaps.umn.edu/information-technology https://www.metrotransit.org/stadium-village-station https://pts.umn.edu/Park/Basic-Parking-Information/Parking-Facilities ---------- David E Farmer III is inviting you to a scheduled Zoom meeting. Join Zoom Meeting https://umn.zoom.us/j/98411644091?pwd=QUxiM3hyaldYeDd5SnJFNC9rd2I5dz09 Meeting ID: 984 1164 4091 Passcode: MDQp15 One tap mobile +16513728299,,98411644091#,,,,*686853# US (Minnesota) +13126266799,,98411644091#,,,,*686853# US (Chicago) Dial by your location +1 651 372 8299 US (Minnesota) +1 312 626 6799 US (Chicago) +1 646 558 8656 US (New York) +1 301 715 8592 US (Washington DC) +1 346 248 7799 US (Houston) +1 669 900 6833 US (San Jose) +1 253 215 8782 US (Tacoma) Meeting ID: 984 1164 4091 Passcode: 686853 Find your local number: https://umn.zoom.us/u/adbNJkGc6V Join by SIP 98411644091(a)zoomcrc.com Join by H.323 162.255.37.11 (US West) 162.255.36.11 (US East) 221.122.88.195 (China) 115.114.131.7 (India Mumbai) 115.114.115.7 (India Hyderabad) 213.19.144.110 (Amsterdam Netherlands) 213.244.140.110 (Germany) 103.122.166.55 (Australia Sydney) 103.122.167.55 (Australia Melbourne) 209.9.211.110 (Hong Kong SAR) 149.137.40.110 (Singapore) 64.211.144.160 (Brazil) 69.174.57.160 (Canada Toronto) 65.39.152.160 (Canada Vancouver) 207.226.132.110 (Japan Tokyo) 149.137.24.110 (Japan Osaka) Meeting ID: 984 1164 4091 Passcode: 686853 On Mon, Sep 19, 2022 at 3:37 PM Miles McCredie <Miles.McCredie(a)midco.com> wrote: > Are the remote access details at > http://lists.iphouse.net/cgi-bin/wa?A2=MICE-DISCUSS;28135c9a.2107 > accurate for tomorrow’s meeting? (They are from July and mention UG39 so > thought I would check.) > > > > Thanks > > > > > > *Miles McCredie * > Principal Network Engineer I-Core IP > > Office: 6052755192 > Miles.McCredie(a)midco.com > > *Midco.com* > > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > -- =============================================== David Farmer Email:farmer@umn.edu Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 ===============================================

Sept. 20, 2022

Re: Remote access details for tomorrow's meeting?

by Richard Laager

Anthony will confirm tomorrow morning. On 9/19/22 15:37, Miles McCredie wrote: > > Are the remote access details at > http://lists.iphouse.net/cgi-bin/wa?A2=MICE-DISCUSS;28135c9a.2107 > accurate for tomorrow’s meeting? (They are from July and mention UG39 > so thought I would check.) > -- Richard

Sept. 20, 2022

Remote access details for tomorrow's meeting?

by Miles McCredie

Are the remote access details at http://lists.iphouse.net/cgi-bin/wa?A2=MICE-DISCUSS;28135c9a.2107 accurate for tomorrow's meeting? (They are from July and mention UG39 so thought I would check.) Thanks [http://images.midcocomm.com/ES_MidcoLogo.png] Miles McCredie Principal Network Engineer I-Core IP Office: 6052755192 Miles.McCredie(a)midco.com Midco.com

Sept. 19, 2022

Re: RST Hurricane Congestion

by Darin Steffl

Thanks for the info Reid. I'm not sure if you're referencing Zayo or not. They seem to have plenty of fiber issues near the metro. It seems reconvergence takes longer than it should when there's a cut too. We see 2-4 minutes of slow speeds or lost routes/downtime when I assume there's a cut on a path feeding into RST. At one point, a software update was done and it got much better but now it seems to be troublesome again. Congestion is usually bad when there's a cut and I have reported many events over the life of our account. Hopefully you guys have good luck improving things into RST. Thank you On Sat, Sep 17, 2022, 12:28 AM Reid Fishler < 0000000c9d62e487-dmarc-request(a)lists.iphouse.net> wrote: > Just to be clear, this was a fiber on some...legacy RST fiber. It has > multiple 100s for us, so the reroute wasn't as pretty as it should be. We > are actively working on making things a bit better between RST and the MSP > area, but its a work in progress. Sorry for any issues, but I think many of > us know the problematic history of this run of fiber. > > Reid > > > On Fri, Sep 16, 2022 at 12:54 PM Ben Wiechman <ben.wiechman(a)arvig.com> > wrote: > >> We didn't have anything down in that area in our maintenance/outage list. >> Another carrier has been fighting a cut between Winona and Wabasha in a >> difficult location. Maybe HE has some services through them as well? It >> seems out of the way, but... >> >> Ben Wiechman >> >> Director of Network Strategy and Engineering >> >> 320.247.3224 | ben.wiechman(a)arvig.com >> >> Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com >> >> >> >> >> On Thu, Sep 15, 2022 at 9:46 PM Darin Steffl <darin.steffl(a)mnwifi.com> >> wrote: >> >>> It's not football killing us tonight! They finally responded after I >>> called them twice and waited 1 hour, 51 mins for a ticket reply. They said >>> several circuits are down to Rochester and they're trying to balance >>> traffic on the remaining links. It's slowly getting better but we're still >>> down 25% from our normal peak at this time of night. It's better than being >>> down 50% of normal. >>> >>> This seems to be happening more frequently the last 2-3 years or there's >>> more capacity sold, being used, in RST compared to years past when there's >>> been fiber cuts. The slow performance is very noticeable. Less than 2 Mbps >>> download and 10-20% packetloss anywhere outside of RST. >>> >>> Ben, are there any Arvig cuts between RST and Belle Plaine/MSP? I'm >>> pretty sure they have some 100G paths from you guys. >>> >>> On Thu, Sep 15, 2022 at 9:41 PM Ben Wiechman <ben.wiechman(a)arvig.com> >>> wrote: >>> >>>> Thu night football killing the link? >>>> >>>> >>>> Ben Wiechman >>>> Director of IP Strategy and Engineering >>>> Direct: 320.256.0184 >>>> Cell: 320.247.3224 >>>> ben.wiechman(a)arvig.com >>>> 150 Second Street SW | Perham, MN 56573 | arvig.com >>>> >>>> On Thu, Sep 15, 2022, 20:16 Darin Steffl <darin.steffl(a)mnwifi.com> >>>> wrote: >>>> >>>>> Anyone else connected to Hurricane's RST router noticing major >>>>> congestion? It looks like the RST-BLP-MSP path is down and traffic is being >>>>> diverted to CHI first or DVN-OMA. >>>>> >>>>> Any speedtests to ISP's like MiBroadband or KMTelecom that are >>>>> connected to RST are fine but any tests outside of that like to US Internet >>>>> or Carleton College, I'm seeing under 2 Mbps download speeds and 20% packet >>>>> loss. >>>>> >>>>> I reported it to Hurricane at 7:39pm but it's been ongoing longer than >>>>> that. I called 10 minutes after I emailed their NOC and the lady there >>>>> wouldn't transfer me to an engineer and just said they'd reply to my ticket >>>>> after they looked. I stressed the urgency since our traffic levels are only >>>>> 50% of what they should be right now. >>>>> >>>>> -- >>>>> Darin Steffl >>>>> Minnesota WiFi >>>>> www.mnwifi.com >>>>> 507-634-WiFi >>>>> Like us on Facebook <http://www.facebook.com/minnesotawifi> >>>>> >>>>> ------------------------------ >>>>> >>>>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>>>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>>>> >>>> >>>> ------------------------------ >>>> >>>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>>> >>> >>> >>> -- >>> Darin Steffl >>> Minnesota WiFi >>> www.mnwifi.com >>> 507-634-WiFi >>> Like us on Facebook <http://www.facebook.com/minnesotawifi> >>> >>> ------------------------------ >>> >>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>> >> >> ------------------------------ >> >> To unsubscribe from the MICE-DISCUSS list, click the following link: >> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >> > > > -- > Reid Fishler > Senior Director > Hurricane Electric > +1-510-580-4178 > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

Sept. 17, 2022

Re: RST Hurricane Congestion

by Reid Fishler

Just to be clear, this was a fiber on some...legacy RST fiber. It has multiple 100s for us, so the reroute wasn't as pretty as it should be. We are actively working on making things a bit better between RST and the MSP area, but its a work in progress. Sorry for any issues, but I think many of us know the problematic history of this run of fiber. Reid On Fri, Sep 16, 2022 at 12:54 PM Ben Wiechman <ben.wiechman(a)arvig.com> wrote: > We didn't have anything down in that area in our maintenance/outage list. > Another carrier has been fighting a cut between Winona and Wabasha in a > difficult location. Maybe HE has some services through them as well? It > seems out of the way, but... > > Ben Wiechman > > Director of Network Strategy and Engineering > > 320.247.3224 | ben.wiechman(a)arvig.com > > Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com > > > > > On Thu, Sep 15, 2022 at 9:46 PM Darin Steffl <darin.steffl(a)mnwifi.com> > wrote: > >> It's not football killing us tonight! They finally responded after I >> called them twice and waited 1 hour, 51 mins for a ticket reply. They said >> several circuits are down to Rochester and they're trying to balance >> traffic on the remaining links. It's slowly getting better but we're still >> down 25% from our normal peak at this time of night. It's better than being >> down 50% of normal. >> >> This seems to be happening more frequently the last 2-3 years or there's >> more capacity sold, being used, in RST compared to years past when there's >> been fiber cuts. The slow performance is very noticeable. Less than 2 Mbps >> download and 10-20% packetloss anywhere outside of RST. >> >> Ben, are there any Arvig cuts between RST and Belle Plaine/MSP? I'm >> pretty sure they have some 100G paths from you guys. >> >> On Thu, Sep 15, 2022 at 9:41 PM Ben Wiechman <ben.wiechman(a)arvig.com> >> wrote: >> >>> Thu night football killing the link? >>> >>> >>> Ben Wiechman >>> Director of IP Strategy and Engineering >>> Direct: 320.256.0184 >>> Cell: 320.247.3224 >>> ben.wiechman(a)arvig.com >>> 150 Second Street SW | Perham, MN 56573 | arvig.com >>> >>> On Thu, Sep 15, 2022, 20:16 Darin Steffl <darin.steffl(a)mnwifi.com> >>> wrote: >>> >>>> Anyone else connected to Hurricane's RST router noticing major >>>> congestion? It looks like the RST-BLP-MSP path is down and traffic is being >>>> diverted to CHI first or DVN-OMA. >>>> >>>> Any speedtests to ISP's like MiBroadband or KMTelecom that are >>>> connected to RST are fine but any tests outside of that like to US Internet >>>> or Carleton College, I'm seeing under 2 Mbps download speeds and 20% packet >>>> loss. >>>> >>>> I reported it to Hurricane at 7:39pm but it's been ongoing longer than >>>> that. I called 10 minutes after I emailed their NOC and the lady there >>>> wouldn't transfer me to an engineer and just said they'd reply to my ticket >>>> after they looked. I stressed the urgency since our traffic levels are only >>>> 50% of what they should be right now. >>>> >>>> -- >>>> Darin Steffl >>>> Minnesota WiFi >>>> www.mnwifi.com >>>> 507-634-WiFi >>>> Like us on Facebook <http://www.facebook.com/minnesotawifi> >>>> >>>> ------------------------------ >>>> >>>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>>> >>> >>> ------------------------------ >>> >>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>> >> >> >> -- >> Darin Steffl >> Minnesota WiFi >> www.mnwifi.com >> 507-634-WiFi >> Like us on Facebook <http://www.facebook.com/minnesotawifi> >> >> ------------------------------ >> >> To unsubscribe from the MICE-DISCUSS list, click the following link: >> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >> > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > -- Reid Fishler Senior Director Hurricane Electric +1-510-580-4178

Sept. 17, 2022

Re: RST Hurricane Congestion

by Jay Hanke

> We didn't have anything down in that area in our maintenance/outage list. > Another carrier has been fighting a cut between Winona and Wabasha in a difficult location. Maybe HE has some services through them as well? It seems out of the way, but... There's only so many viable hops out of RST that go someplace useful for peering. The stuff heading towards Chicago gets horribly congested during network events. If that cut is on the CP rail between Winona and Wabasha that would remove a good chunk of capacity between MN and Chicago. MICE can help quite a bit with the Chicago problem...

Sept. 16, 2022

Re: RST Hurricane Congestion

by Ben Wiechman

We didn't have anything down in that area in our maintenance/outage list. Another carrier has been fighting a cut between Winona and Wabasha in a difficult location. Maybe HE has some services through them as well? It seems out of the way, but... Ben Wiechman Director of Network Strategy and Engineering 320.247.3224 | ben.wiechman(a)arvig.com Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com On Thu, Sep 15, 2022 at 9:46 PM Darin Steffl <darin.steffl(a)mnwifi.com> wrote: > It's not football killing us tonight! They finally responded after I > called them twice and waited 1 hour, 51 mins for a ticket reply. They said > several circuits are down to Rochester and they're trying to balance > traffic on the remaining links. It's slowly getting better but we're still > down 25% from our normal peak at this time of night. It's better than being > down 50% of normal. > > This seems to be happening more frequently the last 2-3 years or there's > more capacity sold, being used, in RST compared to years past when there's > been fiber cuts. The slow performance is very noticeable. Less than 2 Mbps > download and 10-20% packetloss anywhere outside of RST. > > Ben, are there any Arvig cuts between RST and Belle Plaine/MSP? I'm pretty > sure they have some 100G paths from you guys. > > On Thu, Sep 15, 2022 at 9:41 PM Ben Wiechman <ben.wiechman(a)arvig.com> > wrote: > >> Thu night football killing the link? >> >> >> Ben Wiechman >> Director of IP Strategy and Engineering >> Direct: 320.256.0184 >> Cell: 320.247.3224 >> ben.wiechman(a)arvig.com >> 150 Second Street SW | Perham, MN 56573 | arvig.com >> >> On Thu, Sep 15, 2022, 20:16 Darin Steffl <darin.steffl(a)mnwifi.com> wrote: >> >>> Anyone else connected to Hurricane's RST router noticing major >>> congestion? It looks like the RST-BLP-MSP path is down and traffic is being >>> diverted to CHI first or DVN-OMA. >>> >>> Any speedtests to ISP's like MiBroadband or KMTelecom that are connected >>> to RST are fine but any tests outside of that like to US Internet or >>> Carleton College, I'm seeing under 2 Mbps download speeds and 20% packet >>> loss. >>> >>> I reported it to Hurricane at 7:39pm but it's been ongoing longer than >>> that. I called 10 minutes after I emailed their NOC and the lady there >>> wouldn't transfer me to an engineer and just said they'd reply to my ticket >>> after they looked. I stressed the urgency since our traffic levels are only >>> 50% of what they should be right now. >>> >>> -- >>> Darin Steffl >>> Minnesota WiFi >>> www.mnwifi.com >>> 507-634-WiFi >>> Like us on Facebook <http://www.facebook.com/minnesotawifi> >>> >>> ------------------------------ >>> >>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>> >> >> ------------------------------ >> >> To unsubscribe from the MICE-DISCUSS list, click the following link: >> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >> > > > -- > Darin Steffl > Minnesota WiFi > www.mnwifi.com > 507-634-WiFi > Like us on Facebook <http://www.facebook.com/minnesotawifi> > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

Sept. 16, 2022

Re: RST Hurricane Congestion

by Darin Steffl

It's not football killing us tonight! They finally responded after I called them twice and waited 1 hour, 51 mins for a ticket reply. They said several circuits are down to Rochester and they're trying to balance traffic on the remaining links. It's slowly getting better but we're still down 25% from our normal peak at this time of night. It's better than being down 50% of normal. This seems to be happening more frequently the last 2-3 years or there's more capacity sold, being used, in RST compared to years past when there's been fiber cuts. The slow performance is very noticeable. Less than 2 Mbps download and 10-20% packetloss anywhere outside of RST. Ben, are there any Arvig cuts between RST and Belle Plaine/MSP? I'm pretty sure they have some 100G paths from you guys. On Thu, Sep 15, 2022 at 9:41 PM Ben Wiechman <ben.wiechman(a)arvig.com> wrote: > Thu night football killing the link? > > > Ben Wiechman > Director of IP Strategy and Engineering > Direct: 320.256.0184 > Cell: 320.247.3224 > ben.wiechman(a)arvig.com > 150 Second Street SW | Perham, MN 56573 | arvig.com > > On Thu, Sep 15, 2022, 20:16 Darin Steffl <darin.steffl(a)mnwifi.com> wrote: > >> Anyone else connected to Hurricane's RST router noticing major >> congestion? It looks like the RST-BLP-MSP path is down and traffic is being >> diverted to CHI first or DVN-OMA. >> >> Any speedtests to ISP's like MiBroadband or KMTelecom that are connected >> to RST are fine but any tests outside of that like to US Internet or >> Carleton College, I'm seeing under 2 Mbps download speeds and 20% packet >> loss. >> >> I reported it to Hurricane at 7:39pm but it's been ongoing longer than >> that. I called 10 minutes after I emailed their NOC and the lady there >> wouldn't transfer me to an engineer and just said they'd reply to my ticket >> after they looked. I stressed the urgency since our traffic levels are only >> 50% of what they should be right now. >> >> -- >> Darin Steffl >> Minnesota WiFi >> www.mnwifi.com >> 507-634-WiFi >> Like us on Facebook <http://www.facebook.com/minnesotawifi> >> >> ------------------------------ >> >> To unsubscribe from the MICE-DISCUSS list, click the following link: >> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >> > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > -- Darin Steffl Minnesota WiFi www.mnwifi.com 507-634-WiFi Like us on Facebook <http://www.facebook.com/minnesotawifi>

Sept. 16, 2022

Re: RST Hurricane Congestion

by Ben Wiechman

Thu night football killing the link? Ben Wiechman Director of IP Strategy and Engineering Direct: 320.256.0184 Cell: 320.247.3224 ben.wiechman(a)arvig.com 150 Second Street SW | Perham, MN 56573 | arvig.com On Thu, Sep 15, 2022, 20:16 Darin Steffl <darin.steffl(a)mnwifi.com> wrote: > Anyone else connected to Hurricane's RST router noticing major congestion? > It looks like the RST-BLP-MSP path is down and traffic is being diverted to > CHI first or DVN-OMA. > > Any speedtests to ISP's like MiBroadband or KMTelecom that are connected > to RST are fine but any tests outside of that like to US Internet or > Carleton College, I'm seeing under 2 Mbps download speeds and 20% packet > loss. > > I reported it to Hurricane at 7:39pm but it's been ongoing longer than > that. I called 10 minutes after I emailed their NOC and the lady there > wouldn't transfer me to an engineer and just said they'd reply to my ticket > after they looked. I stressed the urgency since our traffic levels are only > 50% of what they should be right now. > > -- > Darin Steffl > Minnesota WiFi > www.mnwifi.com > 507-634-WiFi > Like us on Facebook <http://www.facebook.com/minnesotawifi> > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

Sept. 16, 2022

RST Hurricane Congestion

by Darin Steffl

Anyone else connected to Hurricane's RST router noticing major congestion? It looks like the RST-BLP-MSP path is down and traffic is being diverted to CHI first or DVN-OMA. Any speedtests to ISP's like MiBroadband or KMTelecom that are connected to RST are fine but any tests outside of that like to US Internet or Carleton College, I'm seeing under 2 Mbps download speeds and 20% packet loss. I reported it to Hurricane at 7:39pm but it's been ongoing longer than that. I called 10 minutes after I emailed their NOC and the lady there wouldn't transfer me to an engineer and just said they'd reply to my ticket after they looked. I stressed the urgency since our traffic levels are only 50% of what they should be right now. -- Darin Steffl Minnesota WiFi www.mnwifi.com 507-634-WiFi Like us on Facebook <http://www.facebook.com/minnesotawifi>

Sept. 16, 2022

Please welcome Hennepin County to MICE!

by Jeremy Lumby

Name: Hennepin County Switch: MICE Core Port: E 3/29 & E 6/25 (This is an upgrade from 1x10G to 2x1x100G (no LACP)) ASN: 40382 IPv4: 206.108.255.193/24 (existing) & 206.108.255.201/24 (new) IPv6: 2001:504:27:0:0:9DBE::1/64 (existing) & 2001:504:27:0:0:9DBE::2/64 (new) Peering Contact: peering(a)hennepin.us Route Servers: YES BFD: YES Welcome Aboard Again! Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 Main: 612-355-7740 Direct: 612-392-6814 Fax: 952-873-7425 jlumby(a)mnvoip.com

Sept. 13, 2022

Re: AS112 Server at MICE/511

by Isaiah Olson

Jeremy was able to fix the AS112 node this afternoon and all peering sessions should be re-established. Apparently, one of the DC power leads was pulled loose from the distribution block. Thanks again for taking a look at this for us, Jeremy! _____ Isaiah Olson Network and Systems Administrator 1831 Anne St NW Bemidji, MN 56601 Email: <mailto:iolson@paulbunyan.net> iolson(a)paulbunyan.net P: (218) 444-4638 _____ From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Isaiah Olson Sent: Thursday, September 08, 2022 2:24 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] AS112 Server at MICE/511 Hello, Our AS112 node serving the MICE community has experienced an unknown failure. BGP sessions to 206.108.255.138 and 2001:504:27::70:0:1 would have dropped last night around 12:40 AM. Jeremy will be on-site tomorrow to take a look at the server and we will repair/replace it once the issue has been identified. Thanks, Isaiah _____ Isaiah Olson Network and Systems Administrator 1831 Anne St NW Bemidji, MN 56601 Email: <mailto:iolson@paulbunyan.net> iolson(a)paulbunyan.net P: (218) 444-4638 _____ _____ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS <http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1> &A=1

Sept. 9, 2022

AS112 Server at MICE/511

by Isaiah Olson

Hello, Our AS112 node serving the MICE community has experienced an unknown failure. BGP sessions to 206.108.255.138 and 2001:504:27::70:0:1 would have dropped last night around 12:40 AM. Jeremy will be on-site tomorrow to take a look at the server and we will repair/replace it once the issue has been identified. Thanks, Isaiah _____ Isaiah Olson Network and Systems Administrator 1831 Anne St NW Bemidji, MN 56601 Email: <mailto:iolson@paulbunyan.net> iolson(a)paulbunyan.net P: (218) 444-4638 _____

Sept. 8, 2022

Re: MICE UG41 and Fall Election

by AnthonyAnderberg＠nuvera.net

The nomination period for the MICE board of directors is now closed, the only nomination I've seen is Richard Laager - let me know if I've missed any others. Either way, the formal election will occur during the September 20th UG41 meeting, as noted below. Thanks everyone, anthony From: Anthony Anderberg <AnthonyAnderberg(a)nuvera.net> Date: Wednesday, August 10, 2022 at 4:18 PM To: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> Subject: MICE UG41 and Fall Election The MICE Users Group Meeting #41 will be held Tuesday September 20th at 3pm at the University of Minnesota’s video conference center in 2218 University Ave SE, Minneapolis. Parking and video/audio bridge details will be sent out beforehand. At this meeting we will be holding an election for a board seat - this year the seat held by Richard is up for election. The board consists of three seats with staggered terms, such that one seat comes up for re-election each year (at the fall meeting). Nominations can be posted to MICE-Discuss publicly or sent to me privately and will be accepted from now until September 1st, after the 1st I’ll publish the list of nominated candidates to give people time to choose and arrange proxy voters before the meeting if needed. For proxy voting you can either send your choice to me and during the meeting I’ll list who I’m voting on behalf of, or you may inform me that you are appointing another individual to vote on your behalf and those wishes will be honored during the election. The agenda and draft UG 40 minutes are here for everyone’s review: http://www.micemn.net/files/MICE%20UG%2040%20-%20Minutes.pdf Cheers, Anthony Anderberg MICE Secretary/Treasurer

Sept. 1, 2022

Please welcome Metronet (Jaguar Communications) to MICE!

by Jeremy Lumby

Name: Metronet (Jaguar Communications) Switch: MICE Core Port: E5/5 ASN: 15011 IPv4: IPs remain the same. This is a port addition bringing capacity to 40G on portchannel 20 IPv6: Peering Contact: peering(a)metronet.com Route Servers: BFD: Welcome Aboard Again! Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 Main: 612-355-7740 Direct: 612-392-6814 Fax: 952-873-7425 jlumby(a)mnvoip.com

Aug. 26, 2022

Re: Please welcome Hiawatha Broadband Communications to MICE!

by Jeremy Lumby

The LOA is public onhttp://micemn.net/participants.html Cologix knows to look it up there. The website will be updated with your new port info shortly. From: Dennis Vasquez [mailto:dvasquez@exchange.hbci.com] Sent: Friday, August 26, 2022 11:12 AM To: Jeremy Lumby; MICE Discuss (MICE-DISCUSS(a)LISTS.IPHOUSE.NET) Subject: Re: Please welcome Hiawatha Broadband Communications to MICE! Hi Jeremy, Could you also forward the LOA so I can get the cross connect ordered through cologix? Thanks, Dennis Vasquez Network Administrator E: dvasquez(a)exchange.hbci.com O: (507) 474-5833 www.hbci.com <http://www.hbci.com/> | (888) 474-9995 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- From: Jeremy Lumby <jlumby(a)mnvoip.com> Sent: Friday, August 26, 2022 11:08:00 AM To: MICE Discuss (MICE-DISCUSS(a)LISTS.IPHOUSE.NET) <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> Subject: Please welcome Hiawatha Broadband Communications to MICE! WARNING!! This message originated from an External Source. Please use proper judgment and caution when opening attachments, clicking links, or responding to this email. Name: Hiawatha Broadband Communications Switch: MICE Core Port: E5/41 ASN: 14828 IPv4: IPs remain the same. This is a port addition bringing capacity to 40G on portchannel 32 IPv6: Peering Contact: noc(a)hbci.com Route Servers: YES BFD: YES Welcome Aboard Again! Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 Main: 612-355-7740 Direct: 612-392-6814 Fax: 952-873-7425 jlumby(a)mnvoip.com

Aug. 26, 2022