- mice-discuss - lists.micemn.net

Re: Google AS15169 at MICE is up

by Darin Steffl

Thanks for getting this turned up! I've submitted our request to migrate from 36040 to 15169. I checked the traffic graphs and I don't see the 15169 port in there yet, just the existing 36040. Is this supposed to be setup yet or does that typically follow a few days after the port is active? Thanks On Wed, Aug 7, 2019 at 4:56 PM Dave Schwartz < 0000000aaf163ccf-dmarc-request(a)lists.iphouse.net> wrote: > Hi MICE Participants, > > Our 100G link from AS15169 to MICE is now live. We are peered and > announcing/accepting routes from the route server. We welcome you to > request public peering with us. > > Public peering with Google is encouraged as we do not announce all > prefixes to our route server, only summary routes. We do however announce > all Google prefixes on our direct IX peering sessions. When you peer with > us you also will be able to obtain access to the Google ISP Portal where > you can manage your traffic, view your networks YouTube performance, among > other features. > > There are two ways to request public peering. > > 1. If you already have access to the Google ISP Portal please visit > tickets --> request new IX interconnection and submit your details on your > ASN and IP. > > 2. If you do not have access to the ISP Portal please visit > isp.google.com/iwantpeering . On the form select Peering --> IX Peering > and select Minneapolis from the dropdown. In the comments section please > notate that you are part of MICE. > > While we have paused IX peering due to IRR validation we will be > reviewing MICE requests on a case by case basis for approval since we are > changing ASNs. > > Please send me a direct message should you have any specific questions or > concerns on your request. > > Thanks, > Dave > > > -- > > Dave Schwartz > Peering & Content Distribution > Google AS15169 > daveschwartz(a)google.com > +1-650-253-7615 > +44 20 3820 9224 > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > -- Darin Steffl Minnesota WiFi www.mnwifi.com 507-634-WiFi <http://www.facebook.com/minnesotawifi> Like us on Facebook <http://www.facebook.com/minnesotawifi>

Aug. 7, 2019

Google AS15169 at MICE is up

by Dave Schwartz

Hi MICE Participants, Our 100G link from AS15169 to MICE is now live. We are peered and announcing/accepting routes from the route server. We welcome you to request public peering with us. Public peering with Google is encouraged as we do not announce all prefixes to our route server, only summary routes. We do however announce all Google prefixes on our direct IX peering sessions. When you peer with us you also will be able to obtain access to the Google ISP Portal where you can manage your traffic, view your networks YouTube performance, among other features. There are two ways to request public peering. 1. If you already have access to the Google ISP Portal please visit tickets --> request new IX interconnection and submit your details on your ASN and IP. 2. If you do not have access to the ISP Portal please visit isp.google.com/iwantpeering . On the form select Peering --> IX Peering and select Minneapolis from the dropdown. In the comments section please notate that you are part of MICE. While we have paused IX peering due to IRR validation we will be reviewing MICE requests on a case by case basis for approval since we are changing ASNs. Please send me a direct message should you have any specific questions or concerns on your request. Thanks, Dave -- Dave Schwartz Peering & Content Distribution Google AS15169 daveschwartz(a)google.com +1-650-253-7615 +44 20 3820 9224

Aug. 7, 2019

Routing Security Articles

by David Farmer

The first link below is an excellent article on BGP (Routing) Security, worth a read by both network engineers and C-level executives. The second link is an article with an equally excellent list for network engineers on what they can and should do about these risks. This article includes a great list of questions for a network engineer to start a conversation with your executives ensure they know at least the high-level answers to these questions. Thanks. ----- What Stops a Nation-State BGP Hijack? https://www.senki.org/what-stops-a-nation-state-bgp-hijack/ Nation-State BGP Hijack risks are real! But, the way the Internet Operations Community keeps the Internet glued together minimizes the risk. All Organizations can do their part to minimize the Nation-State BGP Hijack. Each organization can do its part in their corner of the Internet. Collectively, this builds a more robust Internet. This post will provide tools that any organization, individual, researcher, or reporter can use to pull back the fear, understand the realities of BGP, and explore meaningful action. ---- BGP Route Hijacks & Routing Mistakes – What can be done Today? Protecting your Business, Customers, & the Internet from BGP Route Hijacking Chaos? https://www.senki.org/operators-security-toolkit/bgp-route-hijack/ Principle: Global Interconnection Health is a CxO Priority. Every Business depends on their interconnection to the Internet/Telecom. That interconnection is dependent on BGP. Any large organization leadership needs to ask their staff questions: Q. How are we connected to the Internet? Q. How are we protecting our BGP Routes on the Internet? Q. How do we make sure we don’t make a BGP mistake? How do we make sure we do not advertise “YouTube?” Q. Explain to me in simple terms, our BGP based interconnection policy? Q. What is our BGP Crisis Reaction Plan? What do we do when someone tries to BGP Hijack us from the other side of the planet? Q. How would we know if we’re being attacked through a BGP Hijack? -- =============================================== David Farmer Email:farmer@umn.edu Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 ===============================================

Aug. 7, 2019

Fwd: [arin-announce] Registration Now Open for ARIN on the Road: Des Moines

by David Farmer

I thought this would be of interest to at least some of you. ---------- Forwarded message --------- From: ARIN <info(a)arin.net> Date: Wed, Aug 7, 2019 at 10:56 AM Subject: [arin-announce] Registration Now Open for ARIN on the Road: Des Moines To: <arin-announce(a)arin.net> The American Registry for Internet Numbers (ARIN) is bringing our educational roadshow to Des Moines, IA. We hope you will join us at the Embassy Suites Des Moines Downtown from 10:00 AM to 3:30 PM CDT on Thursday, 19 September 2019. This one-day event is your opportunity to learn about topics such as: * ARIN’s Mission and Core Functions * ARIN Technical Services * How Policy is Developed in the ARIN Region * IPv6 Services * IPv4 Wait List and Transfers * Community Engagement with ARIN * Open microphone to answer your questions! Note: Are you part of an organization that would like to share its IPv6 implementation story with others? We want to hear from you! We would like to include your story as part of the agenda. We will have a member of ARIN’s Registration Services Department with us in order to provide one-on-one assistance to anyone with questions about their ARIN-issued Internet number resources or anyone interested in applying for resources, such as IPv6. In addition to providing great professional education, ARIN on the Road will give you a chance to connect with colleagues and find out how you can help shape the future of the Internet by getting involved in the ARIN community. Registration is free and lunch is on us! Seating is limited so register today at: https://www.arin.net/desmoines Help ARIN spread the word by extending this invitation to interested local friends and colleagues by sharing this email or the URL above. Regards, Susan Hamlin Senior Director, Communications and Member Services American Registry for Internet Numbers (ARIN) _______________________________________________ ARIN-Announce You are receiving this message because you are subscribed to the ARIN Announce Mailing List (ARIN-announce(a)arin.net) Unsubscribe or manage your mailing list subscription at: https://lists.arin.net/mailman/listinfo/arin-announce Please contact info(a)arin.net if you experience any issues. -- =============================================== David Farmer Email:farmer@umn.edu Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 ===============================================

Aug. 7, 2019

Re: MICE 2019 Board Election

by Richard Laager

I am running for re-election. My platform is primarily to continue the path we're on: MICE is maturing / has matured, and if re-elected, I'd like to keep working on issues related to that. My focus has been and likely would remain on handling administrative issues, some boring / some not. My main goal for my current board term was to sort out our non-profit status with the IRS. While that took longer than we might have liked due to the question of how we should be classified, I consider that project to have been a success. We have a normalized non-profit status with the IRS. The differences between 501(c)(6) and 501(c)(12) for us are minimal. While I can't take the credit for it, the next big hurdle was the port fees. With port fees in place, we are now in a much more sustainable financial position. These were the "big ticket" items, and they're now behind us, but there is still ongoing administrative work. Where applicable, I try to utilize a continuous improvement process, making small tweaks over time as issues arise. For example, I have worked on improving our Order Form. A project that is still on my plate is the insurance issue, which is actively moving forward again. There's a realistic chance that will be resolved by the October meeting / the end of my current board term. Looking forward, I would like to continue to streamline our meetings. We also need to implement some solution to the "quorum problem": the MICE membership has grown while UG attendance is relatively consistent. The next 3 years will likely involve many networks upgrading from 10G to 100G, which could present issues to be solved (e.g. needing more ports and/or adjusting port fees). The impending launch of various new streaming services will be interesting for the industry as a whole, but could result in new content participants joining MICE, more traffic being moved across MICE, and/or an increased desire for reliability/redundancy/diversity in the MICE core. I believe MICE is well positioned to meet these challenges. -- Richard

Aug. 6, 2019

MICE 2019 Board Election

by Richard Laager

The MICE board consists of three directors with staggered 3-year terms. One position is up for election each year at our fall meeting. This year's fall meeting, UG 33, will be held on October 16 at the 511 Building. The current seat up for election is the one that I hold. At this time, the board would like to open nominations. If you or someone you know is interested in running, please reply on the MICE-DISCUSS list or contact any board member. Nominations will close September 30th. -- Richard

Aug. 2, 2019

Re: SFN Remote Outage

by Darin Steffl

I'm sure the plant guys already plan on that too. This area has received over 20" of rain in the last month so we're very saturated here which contributed to the washout. On Tue, Jul 23, 2019 at 2:09 PM Ben Wiechman <ben.wiechman(a)arvig.com> wrote: > I will be sure to ask about that, but it is likely that it is already > under evaluation. > > Ben Wiechman > > Director of IP Strategy and Engineering > > 320.247.3224 | ben.wiechman(a)arvig.com > > Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com > > > > > On Tue, Jul 23, 2019 at 2:06 PM Darin Steffl <darin.steffl(a)mnwifi.com> > wrote: > >> Much better picture Ben. I didn't feel like climbing down there ha. >> >> Hopefully the permanent fix involves boring on the other side of the road >> where it doesn't have a chance of washing out again. >> >> On Tue, Jul 23, 2019 at 2:03 PM Ben Wiechman <ben.wiechman(a)arvig.com> >> wrote: >> >>> [image: 12564.jpeg] >>> >>> This shows another location and the amount of dirt that was washed away. >>> >>> Splicing crews were completely hands off by around noon. Some services >>> were restored earlier. >>> >>> Ben Wiechman >>> >>> Director of IP Strategy and Engineering >>> >>> 320.247.3224 | ben.wiechman(a)arvig.com >>> >>> Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com >>> >>> >>> >>> >>> On Tue, Jul 23, 2019 at 12:43 PM Darin Steffl <darin.steffl(a)mnwifi.com> >>> wrote: >>> >>>> Our service was restored around 10:36am today. I stopped by to take >>>> some pictures since I work close to the cut. The road definitely washed >>>> out. The fix is temporary with fiber laying in the ditch until they >>>> schedule a permanent fix. >>>> >>>> >>>> https://photos.app.goo.gl/1b551CBzbkEkvVXy7 >>>> >>>> On Tue, Jul 23, 2019, 8:36 AM Ben Wiechman <ben.wiechman(a)arvig.com> >>>> wrote: >>>> >>>>> It is Arvig fiber. The road was washed out in at least three >>>>> locations, with no available culverts or other options to push the fiber >>>>> across the road in order to get the fiber repaired last night. The >>>>> conditions were unsafe so we prepped as much as we could before dark and >>>>> resumed splicing as soon as possible this morning. We are sorry for the >>>>> impact, but needed to consider the safety of our crews on this one. >>>>> >>>>> Ben Wiechman >>>>> >>>>> Director of IP Strategy and Engineering >>>>> >>>>> 320.247.3224 | ben.wiechman(a)arvig.com >>>>> >>>>> Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com >>>>> >>>>> >>>>> >>>>> >>>>> On Tue, Jul 23, 2019 at 8:12 AM Darin Steffl <darin.steffl(a)mnwifi.com> >>>>> wrote: >>>>> >>>>>> Current ETR is 7:30pm tonight so roughly 29 hours after the outage >>>>>> started. I've confirmed it's Arvig fiber, not zayo. >>>>>> >>>>>> On Tue, Jul 23, 2019, 8:11 AM Jay Hanke <jayhanke(a)southfront.io> >>>>>> wrote: >>>>>> >>>>>>> The word is that the road the fiber is on has washed out west of >>>>>>> Rochester. >>>>>>> >>>>>>> On Mon, Jul 22, 2019, 3:33 PM Darin Steffl <darin.steffl(a)mnwifi.com> >>>>>>> wrote: >>>>>>> >>>>>>>> It was 2:35pm to be exact. Hopefully it's not a Zayo line since >>>>>>>> they seem to take 18-24+ hours to fix their cuts. >>>>>>>> >>>>>>>> On Mon, Jul 22, 2019 at 3:32 PM Jay Hanke <jayhanke(a)southfront.io> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Yes, that lines up with our timing/vendor as well. >>>>>>>>> >>>>>>>>> On Mon, Jul 22, 2019 at 3:29 PM Darin Steffl < >>>>>>>>> darin.steffl(a)mnwifi.com> wrote: >>>>>>>>> > >>>>>>>>> > We lost an arvig circuit at 2:45 so it must be the same cut. >>>>>>>>> > >>>>>>>>> > On Mon, Jul 22, 2019, 3:23 PM Jay Hanke <jayhanke(a)southfront.io> >>>>>>>>> wrote: >>>>>>>>> >> >>>>>>>>> >> Our transport to Albert Lea is currently down. >>>>>>>>> >> >>>>>>>>> >> -- >>>>>>>>> >> Jay Hanke, President >>>>>>>>> >> South Front Networks >>>>>>>>> >> jayhanke(a)southfront.io >>>>>>>>> >> Phone 612-204-0000 >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > ________________________________ >>>>>>>>> > >>>>>>>>> > To unsubscribe from the MICE-DISCUSS list, click the following >>>>>>>>> link: >>>>>>>>> > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Jay Hanke, President >>>>>>>>> South Front Networks >>>>>>>>> jayhanke(a)southfront.io >>>>>>>>> Phone 612-204-0000 >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Darin Steffl >>>>>>>> Minnesota WiFi >>>>>>>> www.mnwifi.com >>>>>>>> 507-634-WiFi >>>>>>>> <http://www.facebook.com/minnesotawifi> Like us on Facebook >>>>>>>> <http://www.facebook.com/minnesotawifi> >>>>>>>> >>>>>>>> ------------------------------ >>>>>>>> >>>>>>>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>>>>>>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>>>>>>> >>>>>>> >>>>>>> ------------------------------ >>>>>>> >>>>>>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>>>>>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>>>>>> >>>>>> >>>>>> ------------------------------ >>>>>> >>>>>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>>>>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>>>>> >>>>> >>>>> ------------------------------ >>>>> >>>>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>>>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>>>> >>>> >>>> ------------------------------ >>>> >>>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>>> >>> >>> ------------------------------ >>> >>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>> >> >> >> -- >> Darin Steffl >> Minnesota WiFi >> www.mnwifi.com >> 507-634-WiFi >> <http://www.facebook.com/minnesotawifi> Like us on Facebook >> <http://www.facebook.com/minnesotawifi> >> >> ------------------------------ >> >> To unsubscribe from the MICE-DISCUSS list, click the following link: >> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >> > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > -- Darin Steffl Minnesota WiFi www.mnwifi.com 507-634-WiFi <http://www.facebook.com/minnesotawifi> Like us on Facebook <http://www.facebook.com/minnesotawifi>

July 23, 2019

Re: SFN Remote Outage

by Ben Wiechman

I will be sure to ask about that, but it is likely that it is already under evaluation. Ben Wiechman Director of IP Strategy and Engineering 320.247.3224 | ben.wiechman(a)arvig.com Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com On Tue, Jul 23, 2019 at 2:06 PM Darin Steffl <darin.steffl(a)mnwifi.com> wrote: > Much better picture Ben. I didn't feel like climbing down there ha. > > Hopefully the permanent fix involves boring on the other side of the road > where it doesn't have a chance of washing out again. > > On Tue, Jul 23, 2019 at 2:03 PM Ben Wiechman <ben.wiechman(a)arvig.com> > wrote: > >> [image: 12564.jpeg] >> >> This shows another location and the amount of dirt that was washed away. >> >> Splicing crews were completely hands off by around noon. Some services >> were restored earlier. >> >> Ben Wiechman >> >> Director of IP Strategy and Engineering >> >> 320.247.3224 | ben.wiechman(a)arvig.com >> >> Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com >> >> >> >> >> On Tue, Jul 23, 2019 at 12:43 PM Darin Steffl <darin.steffl(a)mnwifi.com> >> wrote: >> >>> Our service was restored around 10:36am today. I stopped by to take some >>> pictures since I work close to the cut. The road definitely washed out. The >>> fix is temporary with fiber laying in the ditch until they schedule a >>> permanent fix. >>> >>> >>> https://photos.app.goo.gl/1b551CBzbkEkvVXy7 >>> >>> On Tue, Jul 23, 2019, 8:36 AM Ben Wiechman <ben.wiechman(a)arvig.com> >>> wrote: >>> >>>> It is Arvig fiber. The road was washed out in at least three locations, >>>> with no available culverts or other options to push the fiber across the >>>> road in order to get the fiber repaired last night. The conditions were >>>> unsafe so we prepped as much as we could before dark and resumed splicing >>>> as soon as possible this morning. We are sorry for the impact, but needed >>>> to consider the safety of our crews on this one. >>>> >>>> Ben Wiechman >>>> >>>> Director of IP Strategy and Engineering >>>> >>>> 320.247.3224 | ben.wiechman(a)arvig.com >>>> >>>> Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com >>>> >>>> >>>> >>>> >>>> On Tue, Jul 23, 2019 at 8:12 AM Darin Steffl <darin.steffl(a)mnwifi.com> >>>> wrote: >>>> >>>>> Current ETR is 7:30pm tonight so roughly 29 hours after the outage >>>>> started. I've confirmed it's Arvig fiber, not zayo. >>>>> >>>>> On Tue, Jul 23, 2019, 8:11 AM Jay Hanke <jayhanke(a)southfront.io> >>>>> wrote: >>>>> >>>>>> The word is that the road the fiber is on has washed out west of >>>>>> Rochester. >>>>>> >>>>>> On Mon, Jul 22, 2019, 3:33 PM Darin Steffl <darin.steffl(a)mnwifi.com> >>>>>> wrote: >>>>>> >>>>>>> It was 2:35pm to be exact. Hopefully it's not a Zayo line since they >>>>>>> seem to take 18-24+ hours to fix their cuts. >>>>>>> >>>>>>> On Mon, Jul 22, 2019 at 3:32 PM Jay Hanke <jayhanke(a)southfront.io> >>>>>>> wrote: >>>>>>> >>>>>>>> Yes, that lines up with our timing/vendor as well. >>>>>>>> >>>>>>>> On Mon, Jul 22, 2019 at 3:29 PM Darin Steffl < >>>>>>>> darin.steffl(a)mnwifi.com> wrote: >>>>>>>> > >>>>>>>> > We lost an arvig circuit at 2:45 so it must be the same cut. >>>>>>>> > >>>>>>>> > On Mon, Jul 22, 2019, 3:23 PM Jay Hanke <jayhanke(a)southfront.io> >>>>>>>> wrote: >>>>>>>> >> >>>>>>>> >> Our transport to Albert Lea is currently down. >>>>>>>> >> >>>>>>>> >> -- >>>>>>>> >> Jay Hanke, President >>>>>>>> >> South Front Networks >>>>>>>> >> jayhanke(a)southfront.io >>>>>>>> >> Phone 612-204-0000 >>>>>>>> > >>>>>>>> > >>>>>>>> > ________________________________ >>>>>>>> > >>>>>>>> > To unsubscribe from the MICE-DISCUSS list, click the following >>>>>>>> link: >>>>>>>> > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Jay Hanke, President >>>>>>>> South Front Networks >>>>>>>> jayhanke(a)southfront.io >>>>>>>> Phone 612-204-0000 >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Darin Steffl >>>>>>> Minnesota WiFi >>>>>>> www.mnwifi.com >>>>>>> 507-634-WiFi >>>>>>> <http://www.facebook.com/minnesotawifi> Like us on Facebook >>>>>>> <http://www.facebook.com/minnesotawifi> >>>>>>> >>>>>>> ------------------------------ >>>>>>> >>>>>>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>>>>>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>>>>>> >>>>>> >>>>>> ------------------------------ >>>>>> >>>>>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>>>>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>>>>> >>>>> >>>>> ------------------------------ >>>>> >>>>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>>>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>>>> >>>> >>>> ------------------------------ >>>> >>>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>>> >>> >>> ------------------------------ >>> >>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>> >> >> ------------------------------ >> >> To unsubscribe from the MICE-DISCUSS list, click the following link: >> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >> > > > -- > Darin Steffl > Minnesota WiFi > www.mnwifi.com > 507-634-WiFi > <http://www.facebook.com/minnesotawifi> Like us on Facebook > <http://www.facebook.com/minnesotawifi> > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

July 23, 2019

Re: SFN Remote Outage

by Darin Steffl

Much better picture Ben. I didn't feel like climbing down there ha. Hopefully the permanent fix involves boring on the other side of the road where it doesn't have a chance of washing out again. On Tue, Jul 23, 2019 at 2:03 PM Ben Wiechman <ben.wiechman(a)arvig.com> wrote: > [image: 12564.jpeg] > > This shows another location and the amount of dirt that was washed away. > > Splicing crews were completely hands off by around noon. Some services > were restored earlier. > > Ben Wiechman > > Director of IP Strategy and Engineering > > 320.247.3224 | ben.wiechman(a)arvig.com > > Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com > > > > > On Tue, Jul 23, 2019 at 12:43 PM Darin Steffl <darin.steffl(a)mnwifi.com> > wrote: > >> Our service was restored around 10:36am today. I stopped by to take some >> pictures since I work close to the cut. The road definitely washed out. The >> fix is temporary with fiber laying in the ditch until they schedule a >> permanent fix. >> >> >> https://photos.app.goo.gl/1b551CBzbkEkvVXy7 >> >> On Tue, Jul 23, 2019, 8:36 AM Ben Wiechman <ben.wiechman(a)arvig.com> >> wrote: >> >>> It is Arvig fiber. The road was washed out in at least three locations, >>> with no available culverts or other options to push the fiber across the >>> road in order to get the fiber repaired last night. The conditions were >>> unsafe so we prepped as much as we could before dark and resumed splicing >>> as soon as possible this morning. We are sorry for the impact, but needed >>> to consider the safety of our crews on this one. >>> >>> Ben Wiechman >>> >>> Director of IP Strategy and Engineering >>> >>> 320.247.3224 | ben.wiechman(a)arvig.com >>> >>> Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com >>> >>> >>> >>> >>> On Tue, Jul 23, 2019 at 8:12 AM Darin Steffl <darin.steffl(a)mnwifi.com> >>> wrote: >>> >>>> Current ETR is 7:30pm tonight so roughly 29 hours after the outage >>>> started. I've confirmed it's Arvig fiber, not zayo. >>>> >>>> On Tue, Jul 23, 2019, 8:11 AM Jay Hanke <jayhanke(a)southfront.io> wrote: >>>> >>>>> The word is that the road the fiber is on has washed out west of >>>>> Rochester. >>>>> >>>>> On Mon, Jul 22, 2019, 3:33 PM Darin Steffl <darin.steffl(a)mnwifi.com> >>>>> wrote: >>>>> >>>>>> It was 2:35pm to be exact. Hopefully it's not a Zayo line since they >>>>>> seem to take 18-24+ hours to fix their cuts. >>>>>> >>>>>> On Mon, Jul 22, 2019 at 3:32 PM Jay Hanke <jayhanke(a)southfront.io> >>>>>> wrote: >>>>>> >>>>>>> Yes, that lines up with our timing/vendor as well. >>>>>>> >>>>>>> On Mon, Jul 22, 2019 at 3:29 PM Darin Steffl < >>>>>>> darin.steffl(a)mnwifi.com> wrote: >>>>>>> > >>>>>>> > We lost an arvig circuit at 2:45 so it must be the same cut. >>>>>>> > >>>>>>> > On Mon, Jul 22, 2019, 3:23 PM Jay Hanke <jayhanke(a)southfront.io> >>>>>>> wrote: >>>>>>> >> >>>>>>> >> Our transport to Albert Lea is currently down. >>>>>>> >> >>>>>>> >> -- >>>>>>> >> Jay Hanke, President >>>>>>> >> South Front Networks >>>>>>> >> jayhanke(a)southfront.io >>>>>>> >> Phone 612-204-0000 >>>>>>> > >>>>>>> > >>>>>>> > ________________________________ >>>>>>> > >>>>>>> > To unsubscribe from the MICE-DISCUSS list, click the following >>>>>>> link: >>>>>>> > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Jay Hanke, President >>>>>>> South Front Networks >>>>>>> jayhanke(a)southfront.io >>>>>>> Phone 612-204-0000 >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Darin Steffl >>>>>> Minnesota WiFi >>>>>> www.mnwifi.com >>>>>> 507-634-WiFi >>>>>> <http://www.facebook.com/minnesotawifi> Like us on Facebook >>>>>> <http://www.facebook.com/minnesotawifi> >>>>>> >>>>>> ------------------------------ >>>>>> >>>>>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>>>>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>>>>> >>>>> >>>>> ------------------------------ >>>>> >>>>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>>>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>>>> >>>> >>>> ------------------------------ >>>> >>>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>>> >>> >>> ------------------------------ >>> >>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>> >> >> ------------------------------ >> >> To unsubscribe from the MICE-DISCUSS list, click the following link: >> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >> > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > -- Darin Steffl Minnesota WiFi www.mnwifi.com 507-634-WiFi <http://www.facebook.com/minnesotawifi> Like us on Facebook <http://www.facebook.com/minnesotawifi>

July 23, 2019

Re: SFN Remote Outage

by Ben Wiechman

[image: 12564.jpeg] This shows another location and the amount of dirt that was washed away. Splicing crews were completely hands off by around noon. Some services were restored earlier. Ben Wiechman Director of IP Strategy and Engineering 320.247.3224 | ben.wiechman(a)arvig.com Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com On Tue, Jul 23, 2019 at 12:43 PM Darin Steffl <darin.steffl(a)mnwifi.com> wrote: > Our service was restored around 10:36am today. I stopped by to take some > pictures since I work close to the cut. The road definitely washed out. The > fix is temporary with fiber laying in the ditch until they schedule a > permanent fix. > > > https://photos.app.goo.gl/1b551CBzbkEkvVXy7 > > On Tue, Jul 23, 2019, 8:36 AM Ben Wiechman <ben.wiechman(a)arvig.com> wrote: > >> It is Arvig fiber. The road was washed out in at least three locations, >> with no available culverts or other options to push the fiber across the >> road in order to get the fiber repaired last night. The conditions were >> unsafe so we prepped as much as we could before dark and resumed splicing >> as soon as possible this morning. We are sorry for the impact, but needed >> to consider the safety of our crews on this one. >> >> Ben Wiechman >> >> Director of IP Strategy and Engineering >> >> 320.247.3224 | ben.wiechman(a)arvig.com >> >> Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com >> >> >> >> >> On Tue, Jul 23, 2019 at 8:12 AM Darin Steffl <darin.steffl(a)mnwifi.com> >> wrote: >> >>> Current ETR is 7:30pm tonight so roughly 29 hours after the outage >>> started. I've confirmed it's Arvig fiber, not zayo. >>> >>> On Tue, Jul 23, 2019, 8:11 AM Jay Hanke <jayhanke(a)southfront.io> wrote: >>> >>>> The word is that the road the fiber is on has washed out west of >>>> Rochester. >>>> >>>> On Mon, Jul 22, 2019, 3:33 PM Darin Steffl <darin.steffl(a)mnwifi.com> >>>> wrote: >>>> >>>>> It was 2:35pm to be exact. Hopefully it's not a Zayo line since they >>>>> seem to take 18-24+ hours to fix their cuts. >>>>> >>>>> On Mon, Jul 22, 2019 at 3:32 PM Jay Hanke <jayhanke(a)southfront.io> >>>>> wrote: >>>>> >>>>>> Yes, that lines up with our timing/vendor as well. >>>>>> >>>>>> On Mon, Jul 22, 2019 at 3:29 PM Darin Steffl <darin.steffl(a)mnwifi.com> >>>>>> wrote: >>>>>> > >>>>>> > We lost an arvig circuit at 2:45 so it must be the same cut. >>>>>> > >>>>>> > On Mon, Jul 22, 2019, 3:23 PM Jay Hanke <jayhanke(a)southfront.io> >>>>>> wrote: >>>>>> >> >>>>>> >> Our transport to Albert Lea is currently down. >>>>>> >> >>>>>> >> -- >>>>>> >> Jay Hanke, President >>>>>> >> South Front Networks >>>>>> >> jayhanke(a)southfront.io >>>>>> >> Phone 612-204-0000 >>>>>> > >>>>>> > >>>>>> > ________________________________ >>>>>> > >>>>>> > To unsubscribe from the MICE-DISCUSS list, click the following link: >>>>>> > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Jay Hanke, President >>>>>> South Front Networks >>>>>> jayhanke(a)southfront.io >>>>>> Phone 612-204-0000 >>>>>> >>>>> >>>>> >>>>> -- >>>>> Darin Steffl >>>>> Minnesota WiFi >>>>> www.mnwifi.com >>>>> 507-634-WiFi >>>>> <http://www.facebook.com/minnesotawifi> Like us on Facebook >>>>> <http://www.facebook.com/minnesotawifi> >>>>> >>>>> ------------------------------ >>>>> >>>>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>>>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>>>> >>>> >>>> ------------------------------ >>>> >>>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>>> >>> >>> ------------------------------ >>> >>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>> >> >> ------------------------------ >> >> To unsubscribe from the MICE-DISCUSS list, click the following link: >> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >> > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

July 23, 2019

Re: SFN Remote Outage

by Darin Steffl

Our service was restored around 10:36am today. I stopped by to take some pictures since I work close to the cut. The road definitely washed out. The fix is temporary with fiber laying in the ditch until they schedule a permanent fix. https://photos.app.goo.gl/1b551CBzbkEkvVXy7 On Tue, Jul 23, 2019, 8:36 AM Ben Wiechman <ben.wiechman(a)arvig.com> wrote: > It is Arvig fiber. The road was washed out in at least three locations, > with no available culverts or other options to push the fiber across the > road in order to get the fiber repaired last night. The conditions were > unsafe so we prepped as much as we could before dark and resumed splicing > as soon as possible this morning. We are sorry for the impact, but needed > to consider the safety of our crews on this one. > > Ben Wiechman > > Director of IP Strategy and Engineering > > 320.247.3224 | ben.wiechman(a)arvig.com > > Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com > > > > > On Tue, Jul 23, 2019 at 8:12 AM Darin Steffl <darin.steffl(a)mnwifi.com> > wrote: > >> Current ETR is 7:30pm tonight so roughly 29 hours after the outage >> started. I've confirmed it's Arvig fiber, not zayo. >> >> On Tue, Jul 23, 2019, 8:11 AM Jay Hanke <jayhanke(a)southfront.io> wrote: >> >>> The word is that the road the fiber is on has washed out west of >>> Rochester. >>> >>> On Mon, Jul 22, 2019, 3:33 PM Darin Steffl <darin.steffl(a)mnwifi.com> >>> wrote: >>> >>>> It was 2:35pm to be exact. Hopefully it's not a Zayo line since they >>>> seem to take 18-24+ hours to fix their cuts. >>>> >>>> On Mon, Jul 22, 2019 at 3:32 PM Jay Hanke <jayhanke(a)southfront.io> >>>> wrote: >>>> >>>>> Yes, that lines up with our timing/vendor as well. >>>>> >>>>> On Mon, Jul 22, 2019 at 3:29 PM Darin Steffl <darin.steffl(a)mnwifi.com> >>>>> wrote: >>>>> > >>>>> > We lost an arvig circuit at 2:45 so it must be the same cut. >>>>> > >>>>> > On Mon, Jul 22, 2019, 3:23 PM Jay Hanke <jayhanke(a)southfront.io> >>>>> wrote: >>>>> >> >>>>> >> Our transport to Albert Lea is currently down. >>>>> >> >>>>> >> -- >>>>> >> Jay Hanke, President >>>>> >> South Front Networks >>>>> >> jayhanke(a)southfront.io >>>>> >> Phone 612-204-0000 >>>>> > >>>>> > >>>>> > ________________________________ >>>>> > >>>>> > To unsubscribe from the MICE-DISCUSS list, click the following link: >>>>> > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>>>> >>>>> >>>>> >>>>> -- >>>>> Jay Hanke, President >>>>> South Front Networks >>>>> jayhanke(a)southfront.io >>>>> Phone 612-204-0000 >>>>> >>>> >>>> >>>> -- >>>> Darin Steffl >>>> Minnesota WiFi >>>> www.mnwifi.com >>>> 507-634-WiFi >>>> <http://www.facebook.com/minnesotawifi> Like us on Facebook >>>> <http://www.facebook.com/minnesotawifi> >>>> >>>> ------------------------------ >>>> >>>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>>> >>> >>> ------------------------------ >>> >>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>> >> >> ------------------------------ >> >> To unsubscribe from the MICE-DISCUSS list, click the following link: >> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >> > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

July 23, 2019

Re: SFN Remote Outage

by Ben Wiechman

It is Arvig fiber. The road was washed out in at least three locations, with no available culverts or other options to push the fiber across the road in order to get the fiber repaired last night. The conditions were unsafe so we prepped as much as we could before dark and resumed splicing as soon as possible this morning. We are sorry for the impact, but needed to consider the safety of our crews on this one. Ben Wiechman Director of IP Strategy and Engineering 320.247.3224 | ben.wiechman(a)arvig.com Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com On Tue, Jul 23, 2019 at 8:12 AM Darin Steffl <darin.steffl(a)mnwifi.com> wrote: > Current ETR is 7:30pm tonight so roughly 29 hours after the outage > started. I've confirmed it's Arvig fiber, not zayo. > > On Tue, Jul 23, 2019, 8:11 AM Jay Hanke <jayhanke(a)southfront.io> wrote: > >> The word is that the road the fiber is on has washed out west of >> Rochester. >> >> On Mon, Jul 22, 2019, 3:33 PM Darin Steffl <darin.steffl(a)mnwifi.com> >> wrote: >> >>> It was 2:35pm to be exact. Hopefully it's not a Zayo line since they >>> seem to take 18-24+ hours to fix their cuts. >>> >>> On Mon, Jul 22, 2019 at 3:32 PM Jay Hanke <jayhanke(a)southfront.io> >>> wrote: >>> >>>> Yes, that lines up with our timing/vendor as well. >>>> >>>> On Mon, Jul 22, 2019 at 3:29 PM Darin Steffl <darin.steffl(a)mnwifi.com> >>>> wrote: >>>> > >>>> > We lost an arvig circuit at 2:45 so it must be the same cut. >>>> > >>>> > On Mon, Jul 22, 2019, 3:23 PM Jay Hanke <jayhanke(a)southfront.io> >>>> wrote: >>>> >> >>>> >> Our transport to Albert Lea is currently down. >>>> >> >>>> >> -- >>>> >> Jay Hanke, President >>>> >> South Front Networks >>>> >> jayhanke(a)southfront.io >>>> >> Phone 612-204-0000 >>>> > >>>> > >>>> > ________________________________ >>>> > >>>> > To unsubscribe from the MICE-DISCUSS list, click the following link: >>>> > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>>> >>>> >>>> >>>> -- >>>> Jay Hanke, President >>>> South Front Networks >>>> jayhanke(a)southfront.io >>>> Phone 612-204-0000 >>>> >>> >>> >>> -- >>> Darin Steffl >>> Minnesota WiFi >>> www.mnwifi.com >>> 507-634-WiFi >>> <http://www.facebook.com/minnesotawifi> Like us on Facebook >>> <http://www.facebook.com/minnesotawifi> >>> >>> ------------------------------ >>> >>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>> >> >> ------------------------------ >> >> To unsubscribe from the MICE-DISCUSS list, click the following link: >> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >> > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

July 23, 2019

Re: SFN Remote Outage

by Darin Steffl

Current ETR is 7:30pm tonight so roughly 29 hours after the outage started. I've confirmed it's Arvig fiber, not zayo. On Tue, Jul 23, 2019, 8:11 AM Jay Hanke <jayhanke(a)southfront.io> wrote: > The word is that the road the fiber is on has washed out west of > Rochester. > > On Mon, Jul 22, 2019, 3:33 PM Darin Steffl <darin.steffl(a)mnwifi.com> > wrote: > >> It was 2:35pm to be exact. Hopefully it's not a Zayo line since they seem >> to take 18-24+ hours to fix their cuts. >> >> On Mon, Jul 22, 2019 at 3:32 PM Jay Hanke <jayhanke(a)southfront.io> wrote: >> >>> Yes, that lines up with our timing/vendor as well. >>> >>> On Mon, Jul 22, 2019 at 3:29 PM Darin Steffl <darin.steffl(a)mnwifi.com> >>> wrote: >>> > >>> > We lost an arvig circuit at 2:45 so it must be the same cut. >>> > >>> > On Mon, Jul 22, 2019, 3:23 PM Jay Hanke <jayhanke(a)southfront.io> >>> wrote: >>> >> >>> >> Our transport to Albert Lea is currently down. >>> >> >>> >> -- >>> >> Jay Hanke, President >>> >> South Front Networks >>> >> jayhanke(a)southfront.io >>> >> Phone 612-204-0000 >>> > >>> > >>> > ________________________________ >>> > >>> > To unsubscribe from the MICE-DISCUSS list, click the following link: >>> > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>> >>> >>> >>> -- >>> Jay Hanke, President >>> South Front Networks >>> jayhanke(a)southfront.io >>> Phone 612-204-0000 >>> >> >> >> -- >> Darin Steffl >> Minnesota WiFi >> www.mnwifi.com >> 507-634-WiFi >> <http://www.facebook.com/minnesotawifi> Like us on Facebook >> <http://www.facebook.com/minnesotawifi> >> >> ------------------------------ >> >> To unsubscribe from the MICE-DISCUSS list, click the following link: >> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >> > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

July 23, 2019

Re: SFN Remote Outage

by Jay Hanke

The word is that the road the fiber is on has washed out west of Rochester. On Mon, Jul 22, 2019, 3:33 PM Darin Steffl <darin.steffl(a)mnwifi.com> wrote: > It was 2:35pm to be exact. Hopefully it's not a Zayo line since they seem > to take 18-24+ hours to fix their cuts. > > On Mon, Jul 22, 2019 at 3:32 PM Jay Hanke <jayhanke(a)southfront.io> wrote: > >> Yes, that lines up with our timing/vendor as well. >> >> On Mon, Jul 22, 2019 at 3:29 PM Darin Steffl <darin.steffl(a)mnwifi.com> >> wrote: >> > >> > We lost an arvig circuit at 2:45 so it must be the same cut. >> > >> > On Mon, Jul 22, 2019, 3:23 PM Jay Hanke <jayhanke(a)southfront.io> wrote: >> >> >> >> Our transport to Albert Lea is currently down. >> >> >> >> -- >> >> Jay Hanke, President >> >> South Front Networks >> >> jayhanke(a)southfront.io >> >> Phone 612-204-0000 >> > >> > >> > ________________________________ >> > >> > To unsubscribe from the MICE-DISCUSS list, click the following link: >> > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >> >> >> >> -- >> Jay Hanke, President >> South Front Networks >> jayhanke(a)southfront.io >> Phone 612-204-0000 >> > > > -- > Darin Steffl > Minnesota WiFi > www.mnwifi.com > 507-634-WiFi > <http://www.facebook.com/minnesotawifi> Like us on Facebook > <http://www.facebook.com/minnesotawifi> > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

July 23, 2019

Re: SFN Remote Outage

by Darin Steffl

It was 2:35pm to be exact. Hopefully it's not a Zayo line since they seem to take 18-24+ hours to fix their cuts. On Mon, Jul 22, 2019 at 3:32 PM Jay Hanke <jayhanke(a)southfront.io> wrote: > Yes, that lines up with our timing/vendor as well. > > On Mon, Jul 22, 2019 at 3:29 PM Darin Steffl <darin.steffl(a)mnwifi.com> > wrote: > > > > We lost an arvig circuit at 2:45 so it must be the same cut. > > > > On Mon, Jul 22, 2019, 3:23 PM Jay Hanke <jayhanke(a)southfront.io> wrote: > >> > >> Our transport to Albert Lea is currently down. > >> > >> -- > >> Jay Hanke, President > >> South Front Networks > >> jayhanke(a)southfront.io > >> Phone 612-204-0000 > > > > > > ________________________________ > > > > To unsubscribe from the MICE-DISCUSS list, click the following link: > > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > > > > -- > Jay Hanke, President > South Front Networks > jayhanke(a)southfront.io > Phone 612-204-0000 > -- Darin Steffl Minnesota WiFi www.mnwifi.com 507-634-WiFi <http://www.facebook.com/minnesotawifi> Like us on Facebook <http://www.facebook.com/minnesotawifi>

July 22, 2019

Re: SFN Remote Outage

by Jay Hanke

Yes, that lines up with our timing/vendor as well. On Mon, Jul 22, 2019 at 3:29 PM Darin Steffl <darin.steffl(a)mnwifi.com> wrote: > > We lost an arvig circuit at 2:45 so it must be the same cut. > > On Mon, Jul 22, 2019, 3:23 PM Jay Hanke <jayhanke(a)southfront.io> wrote: >> >> Our transport to Albert Lea is currently down. >> >> -- >> Jay Hanke, President >> South Front Networks >> jayhanke(a)southfront.io >> Phone 612-204-0000 > > > ________________________________ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

July 22, 2019

Re: SFN Remote Outage

by Darin Steffl

We lost an arvig circuit at 2:45 so it must be the same cut. On Mon, Jul 22, 2019, 3:23 PM Jay Hanke <jayhanke(a)southfront.io> wrote: > Our transport to Albert Lea is currently down. > > -- > Jay Hanke, President > South Front Networks > jayhanke(a)southfront.io > Phone 612-204-0000 >

July 22, 2019

SFN Remote Outage

by Jay Hanke

Our transport to Albert Lea is currently down. -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

July 22, 2019

Parking/Meeting Information for MICE UG32

by Jay Hanke

The meeting is at: July 17, 2019, 3 PM Central Time Cedar Falls Utilities 1 Utility Pkwy Cedar Falls, IA 50613 Remote Conferencing: 952-562-5400 passcode MICE (6423) Please park in the visitor parking area adjacent to the power plant and enter at the main entrance under the canopy. When you enter, turn left and sign in at reception. -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

July 16, 2019

Re: MICE UG 32

by Richard Laager

It looks like I forgot to attach the UG 31 minutes. Here they are. -- Richard

July 16, 2019

MICE UG 32

by Richard Laager

Reminder: UG 32 is on Wednesday of next week (July 17) at 3:00pm Remote Conferencing: 952-562-5400 passcode MICE (6423) Location Details: Cedar Falls Utilities 1 Utility Pkwy Cedar Falls, IA Agenda The agenda is attached. I'm behind on getting this out, so I didn't send a separate call for agenda. If you have something to add, let me know ASAP and I'll send an updated agenda. As this is an "away" meeting, I intend to skip the repeat old business items. We can pick those up in October. As far as I know, we have no other technical committee report this meeting; if anyone is planning on giving a technical committee report, please let me know. I've also taken the liberty to reorder things a bit. The idea here is to group business (official reports and things that could involve a motion/vote) before open discussion. This way, people who are not interested in the discussion can leave earlier if they wish, confident that they won't miss an official motion/vote. By shortening the "mandatory" part of the meeting, this may make the meetings more accessible. I'd like to give this a try for a couple meetings and see what people think. That said, with this being the summer outreach meeting, we traditionally cover "What is MICE?" first. I kept that at the top, rather than putting it after adjournment. -- Richard

July 9, 2019

Re: Blackholing DoS traffic

by Steve Howard

July 5, 2019

Off Topic Cisco L2TPv3

by Jeremy Lumby

If anyone on MICE has experience with a L2TPv3 tunnel between Cisco routers, could you please contact me directly. I have a situation where the tunnel comes up, and broadcast traffic makes it across, however not unicast traffic. I have tried different routers, and settings, and IOS versions to no avail. Any help on this subject would be greatly appreciated. Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 M: 612-355-7740 D: 612-392-6814 F: 952-873-7425 jlumby(a)mnvoip.com

July 2, 2019

SFN MN-AlbertLea Remote Switch Notification #72

by Jay Hanke

An upstream vendor will be performing service affecting work on the circuit between Albert Lea and Minneapolis on the following dates. This will potentially affect the connectivity between the SFN MICE Switch and the MICE core. Estimated Start: NIGHT 1 2019-07-09 12:00 AM Estimated End: NIGHT 1 2019-07-09 05:00 AM Estimated Start: NIGHT 2 2019-07-10 12:00 AM Estimated End: NIGHT 2 2019-07-10 05:00 AM -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

July 2, 2019

Re: Please Welcome FDL Communications Corp to MICE!

by AnthonyAnderberg＠nuvera.net

Added FDL to the route servers, switch config, and graph system. Welcome! [signature_21095059] From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> on behalf of Jay Hanke <jayhanke(a)SOUTHFRONT.IO> Reply-To: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> Date: Friday, June 21, 2019 at 2:29 PM To: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> Subject: [MICE-DISCUSS] Please Welcome FDL Communications Corp to MICE! Name: FDL Communications Corp Port: Ethernet5/27 (Main Switch) 10GE ASN: 396939 IPv4: 206.108.255.144/24<http://206.108.255.144/24> IPv6: 2001:504:27:0:6:0e8b::1/64 Peering Contact: peering(a)aaniin.net<mailto:peering@aaniin.net> Route Servers- YES BFD: NO Welcome Aboard! -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io<mailto:jayhanke@southfront.io> Phone 612-204-0000 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

June 22, 2019

Please Welcome FDL Communications Corp to MICE!

by Jay Hanke

Name: FDL Communications Corp Port: Ethernet5/27 (Main Switch) 10GE ASN: 396939 IPv4: 206.108.255.144/24 IPv6: 2001:504:27:0:6:0e8b::1/64 Peering Contact: peering(a)aaniin.net Route Servers- YES BFD: NO Welcome Aboard! -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

June 21, 2019

Re: ADV: [MICE-DISCUSS] Blackholing DoS traffic

by Darin Steffl

Yes we saw issues from 6:50 to about 7:20 as well. None of our BGP sessions dropped to cogent, HE, or MICE but our customers noticed it and I did see a drop in traffic during that time.. It's stumped me and I'm glad I'm not the only one since I thought it was internal to our network but it appears not. On Thu, Jun 20, 2019, 12:30 PM Jeff Wilde <jeff.wilde(a)parkregion.com> wrote: > On a separate note, did anyone experience very intermittent Internet > issues last night for about 30 minutes or so? > > > > Jeff Wilde > > IT Manager > > *Park Region Telephone Company* > > P: 218-826-8330 > > E: jeff.wilde(a)parkregion.com > > > > *From:* MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> * On Behalf Of *Andrew > Hoyos > *Sent:* Thursday, June 20, 2019 11:48 AM > *To:* MICE-DISCUSS(a)LISTS.IPHOUSE.NET > *Subject:* Re: [MICE-DISCUSS] ADV: [MICE-DISCUSS] Blackholing DoS traffic > > > > On Jun 20, 2019, at 11:39 AM, Doug McIntyre <merlyn(a)IPHOUSE.NET> wrote: > > On Thu, Jun 20, 2019 at 04:33:12PM +0000, Frank Bulk wrote: > > https://www.seattleix.net/blackholing > > Does MICE have an blackholng functionality equivalent to SIX? > > I was visiting with a DDoS mitigation vendor this morning and was curious > if there was a way we could automatically mitigate DoS attack traffic > coming from a MICE peer. > > > > You can adjust the routing with communities, ie. in the MICE communities > aera of > http://micemn.net/technical.html > > you could block-hole the AS that is sending you that traffic. > > > > unfortunately, that just has the effect of traffic going elsewhere - not a > blackhole effect. the communities in place would just cause the route not > to be advertised to said peer, and the traffic would just ingress your > network via a different path. In the case of a DDOS, it’s likely you have > multiple ASN’s targeting you. > > > > https://www.seattleix.net/blackholing > > > > SIX, as an example, has a blackhole IP address, and the route servers > matching a blackhole community to set next hop to this to sink the traffic > on the switch fabric. > > Perhaps something we should look into for MICE. > > > > > > — > > Andrew Hoyos > > hoyosa(a)gmail.com > > > > > > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

June 20, 2019

Re: ADV: [MICE-DISCUSS] Blackholing DoS traffic

by Dave Williams

We had issues reported last night... it was 'fixed' by the time I looked at things Sent from my T-Mobile 4G LTE Device -------- Original message -------- From: Mike Horwath <drechsau(a)GEEKS.ORG> Date: 6/20/19 12:32 PM (GMT-06:00) To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] ADV: [MICE-DISCUSS] Blackholing DoS traffic On Thu, Jun 20, 2019 at 05:30:40PM +0000, Jeff Wilde wrote: > On a separate note, did anyone experience very intermittent Internet > issues last night for about 30 minutes or so? You mean for Hulu and Netflix going down? :) -- Mike Horwath, reachable via drechsau(a)Geeks.ORG

June 20, 2019

Re: ADV: [MICE-DISCUSS] Blackholing DoS traffic

by Jay Hanke

I was in a hotel in waterloo ia on centruylink and had a similar problem about the same time to fb and google. On Thu, Jun 20, 2019, 1:00 PM Brian Riggleman <briggleman(a)jagcom.net> wrote: > We also lost google related sites from 6:50 PM until 7:18 PM. I also was > unable to explain this. > > > Brian Riggleman > > Network Administrator > > Jaguar Communications > > 507-214-0258 briggleman(a)jagcom.net > > > On Thu, Jun 20, 2019 at 12:33 PM Jeff Wilde <jeff.wilde(a)parkregion.com> > wrote: > >> No lost icmp to google and others allegedly but I'm not finding anything. >> >> Jeff Wilde >> IT Manager >> Park Region Telephone Company >> P: 218-826-8330 >> E: jeff.wilde(a)parkregion.com >> >> -----Original Message----- >> From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Mike >> Horwath >> Sent: Thursday, June 20, 2019 12:32 PM >> To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET >> Subject: Re: [MICE-DISCUSS] ADV: [MICE-DISCUSS] Blackholing DoS traffic >> >> On Thu, Jun 20, 2019 at 05:30:40PM +0000, Jeff Wilde wrote: >> > On a separate note, did anyone experience very intermittent Internet >> > issues last night for about 30 minutes or so? >> >> You mean for Hulu and Netflix going down? :) >> >> -- >> Mike Horwath, reachable via drechsau(a)Geeks.ORG >> > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

June 20, 2019

Re: ADV: [MICE-DISCUSS] Blackholing DoS traffic

by Andrew Hoyos

On Jun 20, 2019, at 12:31 PM, Mike Horwath <drechsau(a)GEEKS.ORG> wrote: > > On Thu, Jun 20, 2019 at 11:48:13AM -0500, Andrew Hoyos wrote: >> SIX, as an example, has a blackhole IP address, and the route >> servers matching a blackhole community to set next hop to this to >> sink the traffic on the switch fabric. Perhaps something we should >> look into for MICE. > > A good idea - perahps use a specific community advertisement with > correct next-hop into the ether. > Yes, exactly. Rough logic = route server accepts /32’s from you tagged with 65535:666 route server sets next hop no matching prefixes to 206.108.255.0 206.108.255.0 is a blackhole/null destination on switching fabric, or at least translates to dst-mac-addr that is layer2 acl’d to drop. I’d think a prerequisite for this, however, is getting to a point on the route-servers where we are validating/strict filtering prefixes that folks are sending (via IRR), otherwise, we run the risk of people essentially being able to blackhole others. > You can even automate a few things as well from known blacklists, then > another system to accept new networks to be reviewed then blackholed > for <period of time>. Sure, essentially what many folks do already internally for RTBH, and signaling to upstream. We utilize Kentik to flip back community tagged routes via BGP to for us for dst ip’s under attack matching certain parameters. We blackhole these on our core routers via the same logic above (match community, set next hop to a IP w/ a discard/null route). In turn, you can community match these and send upstream if your upstreams support a RTBH community as well. — Andrew Hoyos hoyosa(a)gmail.com

June 20, 2019

Re: ADV: [MICE-DISCUSS] Blackholing DoS traffic

by Brian Riggleman

We also lost google related sites from 6:50 PM until 7:18 PM. I also was unable to explain this. Brian Riggleman Network Administrator Jaguar Communications 507-214-0258 briggleman(a)jagcom.net On Thu, Jun 20, 2019 at 12:33 PM Jeff Wilde <jeff.wilde(a)parkregion.com> wrote: > No lost icmp to google and others allegedly but I'm not finding anything. > > Jeff Wilde > IT Manager > Park Region Telephone Company > P: 218-826-8330 > E: jeff.wilde(a)parkregion.com > > -----Original Message----- > From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Mike > Horwath > Sent: Thursday, June 20, 2019 12:32 PM > To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET > Subject: Re: [MICE-DISCUSS] ADV: [MICE-DISCUSS] Blackholing DoS traffic > > On Thu, Jun 20, 2019 at 05:30:40PM +0000, Jeff Wilde wrote: > > On a separate note, did anyone experience very intermittent Internet > > issues last night for about 30 minutes or so? > > You mean for Hulu and Netflix going down? :) > > -- > Mike Horwath, reachable via drechsau(a)Geeks.ORG >

June 20, 2019

Re: ADV: [MICE-DISCUSS] Blackholing DoS traffic

by Jeff Wilde

No lost icmp to google and others allegedly but I'm not finding anything. Jeff Wilde IT Manager Park Region Telephone Company P: 218-826-8330 E: jeff.wilde(a)parkregion.com -----Original Message----- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Mike Horwath Sent: Thursday, June 20, 2019 12:32 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] ADV: [MICE-DISCUSS] Blackholing DoS traffic On Thu, Jun 20, 2019 at 05:30:40PM +0000, Jeff Wilde wrote: > On a separate note, did anyone experience very intermittent Internet > issues last night for about 30 minutes or so? You mean for Hulu and Netflix going down? :) -- Mike Horwath, reachable via drechsau(a)Geeks.ORG

June 20, 2019

Re: ADV: [MICE-DISCUSS] Blackholing DoS traffic

by Mike Horwath

On Thu, Jun 20, 2019 at 05:30:40PM +0000, Jeff Wilde wrote: > On a separate note, did anyone experience very intermittent Internet > issues last night for about 30 minutes or so? You mean for Hulu and Netflix going down? :) -- Mike Horwath, reachable via drechsau(a)Geeks.ORG

June 20, 2019

Re: ADV: [MICE-DISCUSS] Blackholing DoS traffic

by Mike Horwath

On Thu, Jun 20, 2019 at 11:48:13AM -0500, Andrew Hoyos wrote: > SIX, as an example, has a blackhole IP address, and the route > servers matching a blackhole community to set next hop to this to > sink the traffic on the switch fabric. Perhaps something we should > look into for MICE. A good idea - perahps use a specific community advertisement with correct next-hop into the ether. Then folks can subscribe as needed. You can even automate a few things as well from known blacklists, then another system to accept new networks to be reviewed then blackholed for <period of time>. -- Mike Horwath, reachable via drechsau(a)Geeks.ORG

June 20, 2019

Re: ADV: [MICE-DISCUSS] Blackholing DoS traffic

by Jeff Wilde

On a separate note, did anyone experience very intermittent Internet issues last night for about 30 minutes or so? Jeff Wilde IT Manager Park Region Telephone Company P: 218-826-8330 E: jeff.wilde(a)parkregion.com<mailto:jeff.wilde@parkregion.com> From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Andrew Hoyos Sent: Thursday, June 20, 2019 11:48 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] ADV: [MICE-DISCUSS] Blackholing DoS traffic On Jun 20, 2019, at 11:39 AM, Doug McIntyre <merlyn(a)IPHOUSE.NET<mailto:merlyn@IPHOUSE.NET>> wrote: On Thu, Jun 20, 2019 at 04:33:12PM +0000, Frank Bulk wrote: https://www.seattleix.net/blackholing Does MICE have an blackholng functionality equivalent to SIX? I was visiting with a DDoS mitigation vendor this morning and was curious if there was a way we could automatically mitigate DoS attack traffic coming from a MICE peer. You can adjust the routing with communities, ie. in the MICE communities aera of http://micemn.net/technical.html you could block-hole the AS that is sending you that traffic. unfortunately, that just has the effect of traffic going elsewhere - not a blackhole effect. the communities in place would just cause the route not to be advertised to said peer, and the traffic would just ingress your network via a different path. In the case of a DDOS, it’s likely you have multiple ASN’s targeting you. https://www.seattleix.net/blackholing SIX, as an example, has a blackhole IP address, and the route servers matching a blackhole community to set next hop to this to sink the traffic on the switch fabric. Perhaps something we should look into for MICE. — Andrew Hoyos hoyosa(a)gmail.com<mailto:hoyosa@gmail.com> ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

June 20, 2019

Re: ADV: [MICE-DISCUSS] Blackholing DoS traffic

by Andrew Hoyos

On Jun 20, 2019, at 11:39 AM, Doug McIntyre <merlyn(a)IPHOUSE.NET> wrote: > On Thu, Jun 20, 2019 at 04:33:12PM +0000, Frank Bulk wrote: >> https://www.seattleix.net/blackholing >> >> Does MICE have an blackholng functionality equivalent to SIX? >> >> I was visiting with a DDoS mitigation vendor this morning and was curious if there was a way we could automatically mitigate DoS attack traffic coming from a MICE peer. > > > You can adjust the routing with communities, ie. in the MICE communities aera of > http://micemn.net/technical.html > > you could block-hole the AS that is sending you that traffic. unfortunately, that just has the effect of traffic going elsewhere - not a blackhole effect. the communities in place would just cause the route not to be advertised to said peer, and the traffic would just ingress your network via a different path. In the case of a DDOS, it’s likely you have multiple ASN’s targeting you. https://www.seattleix.net/blackholing <https://www.seattleix.net/blackholing> SIX, as an example, has a blackhole IP address, and the route servers matching a blackhole community to set next hop to this to sink the traffic on the switch fabric. Perhaps something we should look into for MICE. — Andrew Hoyos hoyosa(a)gmail.com

June 20, 2019

Re: ADV: [MICE-DISCUSS] Blackholing DoS traffic

by Doug McIntyre

On Thu, Jun 20, 2019 at 04:33:12PM +0000, Frank Bulk wrote: >https://www.seattleix.net/blackholing > >Does MICE have an blackholng functionality equivalent to SIX? > >I was visiting with a DDoS mitigation vendor this morning and was curious if there was a way we could automatically mitigate DoS attack traffic coming from a MICE peer. You can adjust the routing with communities, ie. in the MICE communities aera of http://micemn.net/technical.html you could block-hole the AS that is sending you that traffic. -- Doug McIntyre <merlyn(a)iphouse.net> ~.~ ipHouse ~.~ Network Engineer/Provisioning/Jack of all Trades

June 20, 2019

Blackholing DoS traffic

by Frank Bulk

https://www.seattleix.net/blackholing Does MICE have an blackholng functionality equivalent to SIX? I was visiting with a DDoS mitigation vendor this morning and was curious if there was a way we could automatically mitigate DoS attack traffic coming from a MICE peer. Regards, Frank

June 20, 2019

Re: Bi-lateral peering with Yahoo!

by Frank Bulk

Thanks, Jay. I saw service return Monday evening. Frank From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Jay Hanke Sent: Monday, June 17, 2019 5:25 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Bi-lateral peering with Yahoo! I opened ticket # PTKT1583794 with Yahoo/Oath regarding the LoS. Jay On Fri, Jun 14, 2019 at 6:01 PM Jay Hanke <jayhanke(a)southfront.io<mailto:jayhanke@southfront.io>> wrote: Looks like 1 of 2 Oath links are down. On Fri, Jun 14, 2019 at 5:47 PM Andrew Hoyos <hoyosa(a)gmail.com<mailto:hoyosa@gmail.com>> wrote: Same here: Jun 14 20:12:13 er1.511.msp rpd[1832]: bgp_hold_timeout:4677: NOTIFICATION sent to 2001:504:27::2846:0:1 (External AS 10310): code 4 (Hold Timer Expired Error), Reason: holdtime expired for 2001:504:27::2846:0:1 (External AS 10310), socket buffer sndcc: 38 rcvcc: 0 TCP state: 4, snd_una: 715538896 snd_nxt: 715538934 snd_wnd: 16384 rcv_nxt: 937051995 rcv_adv: 937068379, hold timer 90s, hold timer remain 0s, last sent 26s, TCP port (local 60544, remote 179) Jun 14 20:12:30 er1.511.msp rpd[1832]: bgp_hold_timeout:4677: NOTIFICATION sent to 206.108.255.58 (External AS 10310): code 4 (Hold Timer Expired Error), Reason: holdtime expired for 206.108.255.58 (External AS 10310), socket buffer sndcc: 57 rcvcc: 0 TCP state: 4, snd_una: 3794713179 snd_nxt: 3794713236 snd_wnd: 16384 rcv_nxt: 58606928 rcv_adv: 58623312, hold timer 90s, hold timer remain 0s, last sent 22s, TCP port (local 179, remote 65426) — Andrew Hoyos hoyosa(a)gmail.com<mailto:hoyosa@gmail.com> > On Jun 14, 2019, at 5:44 PM, Frank Bulk <fbulk(a)MYPREMIERONLINE.COM<mailto:fbulk@MYPREMIERONLINE.COM>> wrote: > > I assume some others are seeing the same, that peering with one of Yahoo!'s IPs has been down since just before 3:45 pm? > > Host premier.sxct-north.fbnt.netins.net<https://urldefense.proofpoint.com/v2/url?u=http-3A__premier.sxct-2Dnorth.fb…> [207.32.15.28] with interface to BGP peer AS10310-1.micemn.net<https://urldefense.proofpoint.com/v2/url?u=http-3A__AS10310-2D1.micemn.net&…> [206.108.255.58/AS<https://urldefense.proofpoint.com/v2/url?u=http-3A__206.108.255.58_AS&d=DwM…> 10310 (Oath Holdings Inc., US)]: the BGP peer state is in state connect > > Frank -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io<mailto:jayhanke@southfront.io> Phone 612-204-0000 -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io<mailto:jayhanke@southfront.io> Phone 612-204-0000 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.iphouse.net_cgi-2…>

June 19, 2019

Re: Bi-lateral peering with Yahoo!

by Jay Hanke

I opened ticket # PTKT1583794 with Yahoo/Oath regarding the LoS. Jay On Fri, Jun 14, 2019 at 6:01 PM Jay Hanke <jayhanke(a)southfront.io> wrote: > Looks like 1 of 2 Oath links are down. > > On Fri, Jun 14, 2019 at 5:47 PM Andrew Hoyos <hoyosa(a)gmail.com> wrote: > >> Same here: >> >> Jun 14 20:12:13 er1.511.msp rpd[1832]: bgp_hold_timeout:4677: >> NOTIFICATION sent to 2001:504:27::2846:0:1 (External AS 10310): code 4 >> (Hold Timer Expired Error), Reason: holdtime expired for >> 2001:504:27::2846:0:1 (External AS 10310), socket buffer sndcc: 38 rcvcc: 0 >> TCP state: 4, snd_una: 715538896 snd_nxt: 715538934 snd_wnd: 16384 rcv_nxt: >> 937051995 rcv_adv: 937068379, hold timer 90s, hold timer remain 0s, last >> sent 26s, TCP port (local 60544, remote 179) >> >> Jun 14 20:12:30 er1.511.msp rpd[1832]: bgp_hold_timeout:4677: >> NOTIFICATION sent to 206.108.255.58 (External AS 10310): code 4 (Hold Timer >> Expired Error), Reason: holdtime expired for 206.108.255.58 (External AS >> 10310), socket buffer sndcc: 57 rcvcc: 0 TCP state: 4, snd_una: 3794713179 >> snd_nxt: 3794713236 snd_wnd: 16384 rcv_nxt: 58606928 rcv_adv: 58623312, >> hold timer 90s, hold timer remain 0s, last sent 22s, TCP port (local 179, >> remote 65426) >> >> — >> Andrew Hoyos >> hoyosa(a)gmail.com >> >> >> >> > On Jun 14, 2019, at 5:44 PM, Frank Bulk <fbulk(a)MYPREMIERONLINE.COM> >> wrote: >> > >> > I assume some others are seeing the same, that peering with one of >> Yahoo!'s IPs has been down since just before 3:45 pm? >> > >> > Host premier.sxct-north.fbnt.netins.net [207.32.15.28] with interface >> to BGP peer AS10310-1.micemn.net [206.108.255.58/AS 10310 (Oath Holdings >> Inc., US)]: the BGP peer state is in state connect >> > >> > Frank >> > > > -- > Jay Hanke, President > South Front Networks > jayhanke(a)southfront.io > Phone 612-204-0000 > -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

June 17, 2019

Re: Google ASN Migration/Port Move

by Doug McIntyre

On Thu, Jun 13, 2019 at 11:53:21AM -0500, Jay Hanke wrote: >Google is converting their existing MICE connection to their backbone >AS (15169). > >Immediately following the migration, Google will disconnect their >existing 100G port (Ethernet3/2/1). > >The new addressing will be: >ASN: 15169 >IPv4: 206.108.255.141/24 >IPv6: 2001:504:27:0:0:3b41::1/64 I've added AS 15169 to the route servers to prep this and ready them up. >The new 100G port will be Ethernet3/13/1. > >Google has requested the port be set up as a new single channel LAG group. I've setup Port-Channel 29 for this LAG group, and have the Arista core switch config'd up on Port Eth3/13/1. They should be all set to bring up this link as needed. -- Doug McIntyre <merlyn(a)iphouse.net> ~.~ ipHouse ~.~ Network Engineer/Provisioning/Jack of all Trades

June 17, 2019

Re: Bi-lateral peering with Yahoo!

by Jay Hanke

Looks like 1 of 2 Oath links are down. On Fri, Jun 14, 2019 at 5:47 PM Andrew Hoyos <hoyosa(a)gmail.com> wrote: > Same here: > > Jun 14 20:12:13 er1.511.msp rpd[1832]: bgp_hold_timeout:4677: > NOTIFICATION sent to 2001:504:27::2846:0:1 (External AS 10310): code 4 > (Hold Timer Expired Error), Reason: holdtime expired for > 2001:504:27::2846:0:1 (External AS 10310), socket buffer sndcc: 38 rcvcc: 0 > TCP state: 4, snd_una: 715538896 snd_nxt: 715538934 snd_wnd: 16384 rcv_nxt: > 937051995 rcv_adv: 937068379, hold timer 90s, hold timer remain 0s, last > sent 26s, TCP port (local 60544, remote 179) > > Jun 14 20:12:30 er1.511.msp rpd[1832]: bgp_hold_timeout:4677: > NOTIFICATION sent to 206.108.255.58 (External AS 10310): code 4 (Hold Timer > Expired Error), Reason: holdtime expired for 206.108.255.58 (External AS > 10310), socket buffer sndcc: 57 rcvcc: 0 TCP state: 4, snd_una: 3794713179 > snd_nxt: 3794713236 snd_wnd: 16384 rcv_nxt: 58606928 rcv_adv: 58623312, > hold timer 90s, hold timer remain 0s, last sent 22s, TCP port (local 179, > remote 65426) > > — > Andrew Hoyos > hoyosa(a)gmail.com > > > > > On Jun 14, 2019, at 5:44 PM, Frank Bulk <fbulk(a)MYPREMIERONLINE.COM> > wrote: > > > > I assume some others are seeing the same, that peering with one of > Yahoo!'s IPs has been down since just before 3:45 pm? > > > > Host premier.sxct-north.fbnt.netins.net [207.32.15.28] with interface > to BGP peer AS10310-1.micemn.net [206.108.255.58/AS 10310 (Oath Holdings > Inc., US)]: the BGP peer state is in state connect > > > > Frank > -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

June 14, 2019

Re: Bi-lateral peering with Yahoo!

by Andrew Hoyos

Same here: Jun 14 20:12:13 er1.511.msp rpd[1832]: bgp_hold_timeout:4677: NOTIFICATION sent to 2001:504:27::2846:0:1 (External AS 10310): code 4 (Hold Timer Expired Error), Reason: holdtime expired for 2001:504:27::2846:0:1 (External AS 10310), socket buffer sndcc: 38 rcvcc: 0 TCP state: 4, snd_una: 715538896 snd_nxt: 715538934 snd_wnd: 16384 rcv_nxt: 937051995 rcv_adv: 937068379, hold timer 90s, hold timer remain 0s, last sent 26s, TCP port (local 60544, remote 179) Jun 14 20:12:30 er1.511.msp rpd[1832]: bgp_hold_timeout:4677: NOTIFICATION sent to 206.108.255.58 (External AS 10310): code 4 (Hold Timer Expired Error), Reason: holdtime expired for 206.108.255.58 (External AS 10310), socket buffer sndcc: 57 rcvcc: 0 TCP state: 4, snd_una: 3794713179 snd_nxt: 3794713236 snd_wnd: 16384 rcv_nxt: 58606928 rcv_adv: 58623312, hold timer 90s, hold timer remain 0s, last sent 22s, TCP port (local 179, remote 65426) — Andrew Hoyos hoyosa(a)gmail.com > On Jun 14, 2019, at 5:44 PM, Frank Bulk <fbulk(a)MYPREMIERONLINE.COM> wrote: > > I assume some others are seeing the same, that peering with one of Yahoo!'s IPs has been down since just before 3:45 pm? > > Host premier.sxct-north.fbnt.netins.net [207.32.15.28] with interface to BGP peer AS10310-1.micemn.net [206.108.255.58/AS 10310 (Oath Holdings Inc., US)]: the BGP peer state is in state connect > > Frank

June 14, 2019

Bi-lateral peering with Yahoo!

by Frank Bulk

I assume some others are seeing the same, that peering with one of Yahoo!'s IPs has been down since just before 3:45 pm? Host premier.sxct-north.fbnt.netins.net [207.32.15.28] with interface to BGP peer AS10310-1.micemn.net [206.108.255.58/AS 10310 (Oath Holdings Inc., US)]: the BGP peer state is in state connect Frank

June 14, 2019

Re: Google ASN Migration/Port Move

by Dave Schwartz

Hi MICE Members, Google AS15169 can deliver all Google traffic. Once we are closer to turn up of the new link I'll share some details with the members on the next steps to peer directly with us. We plan also to peer the route server. Thanks, Dave -- Dave Schwartz Peering & Content Distribution Google AS15169 daveschwartz(a)google.com +1-650-253-7615 +44 20 3820 9224 On Fri, Jun 14, 2019, 07:27 Jay Hanke <jayhanke(a)southfront.io> wrote: > Google is connecting MICE directly to the Google backbone AS so more > of the services will be available following the upgrade at MICE. > > On Fri, Jun 14, 2019 at 9:20 AM Jeff Wilde <jeff.wilde(a)parkregion.com> > wrote: > > > > I was under the impression that they only basically do youtube/etc > through there. I asked about bilateral and that is what they said. > > > > Jeff Wilde > > IT Manager > > Park Region Telephone Company > > P: 218-826-8330 > > E: jeff.wilde(a)parkregion.com > > > > -----Original Message----- > > From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Jeremy > Lumby > > Sent: Friday, June 14, 2019 7:31 AM > > To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET > > Subject: Re: [MICE-DISCUSS] Google ASN Migration/Port Move > > > > I think this is a great move for Google/MICE, and I can easily foresee > Google outgrowing the 100G quickly. On a side note I would update the > website to state "Partial" for the route server. Google had recently > changed their policy for AS15169 where they still utilize the route > servers, however not all content is advertized through it. If you want to > maximize your Google backbone traffic, you need to setup a bilateral with > them. > > > > Jeremy Lumby > > Minnesota VoIP > > 9217 17th Ave S #216 > > Bloomington, MN 55425 > > M: 612-355-7740 > > D: 612-392-6814 > > F: 952-873-7425 > > jlumby(a)mnvoip.com > > > > > > -----Original Message----- > > From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of > Richard Laager > > Sent: Friday, June 14, 2019 2:21 AM > > To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET > > Subject: Re: [MICE-DISCUSS] Google ASN Migration/Port Move > > > > On 6/13/19 11:53 AM, Jay Hanke wrote: > > > Google is converting their existing MICE connection to their backbone > > > AS (15169). > > > > > > Immediately following the migration, Google will disconnect their > > > existing 100G port (Ethernet3/2/1). > > > > > > The new addressing will be: > > > ASN: 15169 > > > IPv4: 206.108.255.141/24 > > > IPv6: 2001:504:27:0:0:3b41::1/64 > > > > > > The new 100G port will be Ethernet3/13/1. > > > > > > Google has requested the port be set up as a new single channel LAG > group. > > > > I've updated the website. > > > > -- > > Richard > > > > -- > Jay Hanke, President > South Front Networks > jayhanke(a)southfront.io > Phone 612-204-0000 >

June 14, 2019

Re: Google ASN Migration/Port Move

by Jay Hanke

Google is connecting MICE directly to the Google backbone AS so more of the services will be available following the upgrade at MICE. On Fri, Jun 14, 2019 at 9:20 AM Jeff Wilde <jeff.wilde(a)parkregion.com> wrote: > > I was under the impression that they only basically do youtube/etc through there. I asked about bilateral and that is what they said. > > Jeff Wilde > IT Manager > Park Region Telephone Company > P: 218-826-8330 > E: jeff.wilde(a)parkregion.com > > -----Original Message----- > From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Jeremy Lumby > Sent: Friday, June 14, 2019 7:31 AM > To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET > Subject: Re: [MICE-DISCUSS] Google ASN Migration/Port Move > > I think this is a great move for Google/MICE, and I can easily foresee Google outgrowing the 100G quickly. On a side note I would update the website to state "Partial" for the route server. Google had recently changed their policy for AS15169 where they still utilize the route servers, however not all content is advertized through it. If you want to maximize your Google backbone traffic, you need to setup a bilateral with them. > > Jeremy Lumby > Minnesota VoIP > 9217 17th Ave S #216 > Bloomington, MN 55425 > M: 612-355-7740 > D: 612-392-6814 > F: 952-873-7425 > jlumby(a)mnvoip.com > > > -----Original Message----- > From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Richard Laager > Sent: Friday, June 14, 2019 2:21 AM > To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET > Subject: Re: [MICE-DISCUSS] Google ASN Migration/Port Move > > On 6/13/19 11:53 AM, Jay Hanke wrote: > > Google is converting their existing MICE connection to their backbone > > AS (15169). > > > > Immediately following the migration, Google will disconnect their > > existing 100G port (Ethernet3/2/1). > > > > The new addressing will be: > > ASN: 15169 > > IPv4: 206.108.255.141/24 > > IPv6: 2001:504:27:0:0:3b41::1/64 > > > > The new 100G port will be Ethernet3/13/1. > > > > Google has requested the port be set up as a new single channel LAG group. > > I've updated the website. > > -- > Richard -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

June 14, 2019

Re: Google ASN Migration/Port Move

by Jeff Wilde

I was under the impression that they only basically do youtube/etc through there. I asked about bilateral and that is what they said. Jeff Wilde IT Manager Park Region Telephone Company P: 218-826-8330 E: jeff.wilde(a)parkregion.com -----Original Message----- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Jeremy Lumby Sent: Friday, June 14, 2019 7:31 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Google ASN Migration/Port Move I think this is a great move for Google/MICE, and I can easily foresee Google outgrowing the 100G quickly. On a side note I would update the website to state "Partial" for the route server. Google had recently changed their policy for AS15169 where they still utilize the route servers, however not all content is advertized through it. If you want to maximize your Google backbone traffic, you need to setup a bilateral with them. Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 M: 612-355-7740 D: 612-392-6814 F: 952-873-7425 jlumby(a)mnvoip.com -----Original Message----- From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Richard Laager Sent: Friday, June 14, 2019 2:21 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Google ASN Migration/Port Move On 6/13/19 11:53 AM, Jay Hanke wrote: > Google is converting their existing MICE connection to their backbone > AS (15169). > > Immediately following the migration, Google will disconnect their > existing 100G port (Ethernet3/2/1). > > The new addressing will be: > ASN: 15169 > IPv4: 206.108.255.141/24 > IPv6: 2001:504:27:0:0:3b41::1/64 > > The new 100G port will be Ethernet3/13/1. > > Google has requested the port be set up as a new single channel LAG group. I've updated the website. -- Richard

June 14, 2019

Re: Google ASN Migration/Port Move

by Jeremy Lumby

I think this is a great move for Google/MICE, and I can easily foresee Google outgrowing the 100G quickly. On a side note I would update the website to state "Partial" for the route server. Google had recently changed their policy for AS15169 where they still utilize the route servers, however not all content is advertized through it. If you want to maximize your Google backbone traffic, you need to setup a bilateral with them. Jeremy Lumby Minnesota VoIP 9217 17th Ave S #216 Bloomington, MN 55425 M: 612-355-7740 D: 612-392-6814 F: 952-873-7425 jlumby(a)mnvoip.com -----Original Message----- From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Richard Laager Sent: Friday, June 14, 2019 2:21 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Google ASN Migration/Port Move On 6/13/19 11:53 AM, Jay Hanke wrote: > Google is converting their existing MICE connection to their backbone > AS (15169). > > Immediately following the migration, Google will disconnect their > existing 100G port (Ethernet3/2/1). > > The new addressing will be: > ASN: 15169 > IPv4: 206.108.255.141/24 > IPv6: 2001:504:27:0:0:3b41::1/64 > > The new 100G port will be Ethernet3/13/1. > > Google has requested the port be set up as a new single channel LAG group. I've updated the website. -- Richard

June 14, 2019

Re: Google ASN Migration/Port Move

by Richard Laager

On 6/13/19 11:53 AM, Jay Hanke wrote: > Google is converting their existing MICE connection to their backbone > AS (15169). > > Immediately following the migration, Google will disconnect their > existing 100G port (Ethernet3/2/1). > > The new addressing will be: > ASN: 15169 > IPv4: 206.108.255.141/24 > IPv6: 2001:504:27:0:0:3b41::1/64 > > The new 100G port will be Ethernet3/13/1. > > Google has requested the port be set up as a new single channel LAG group. I've updated the website. -- Richard

June 14, 2019

Google ASN Migration/Port Move

by Jay Hanke

Google is converting their existing MICE connection to their backbone AS (15169). Immediately following the migration, Google will disconnect their existing 100G port (Ethernet3/2/1). The new addressing will be: ASN: 15169 IPv4: 206.108.255.141/24 IPv6: 2001:504:27:0:0:3b41::1/64 The new 100G port will be Ethernet3/13/1. Google has requested the port be set up as a new single channel LAG group. We'll need new route server connections as well. Welcome aboard (Again)! -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

June 13, 2019