- mice-discuss - lists.micemn.net

Re: Gigapop Link Saturated

by David Farmer

On 11/10/14, 07:56 , Hannigan, Martin wrote: > Why don't you ask Google to control their traffic instead? I believe they have this capability. It's probably more natural and less disruptive for them. Same for us. > > Best, By keeping control in my hands, I am able to make progress on the necessary upgrades we have in progress. We are in process of adding several 10Gs both internally and externally. Additionally, in the next few months we will be upgrading our 511 router to take a 100G link from Internet2. Thanks. -- ================================================ David Farmer Email: farmer(a)umn.edu Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 1-612-626-0815 Minneapolis, MN 55414-3029 Cell: 1-612-812-9952 ================================================

Nov. 10, 2014

Re: Gigapop Link Saturated

by Hannigan, Martin

On Nov 5, 2014, at 8:18 PM, David Farmer <farmer(a)UMN.EDU> wrote: > Northern Lights GigaPOP has its second 10G online as of about 15:00 this > afternoon. There was a dirty jumper causing issues, both Cologix and I > did some fiber cleaning and we whacked that mole. :) > > As of about 17:30, I started to restore the announcement of the > GigaPOP's routes to Google. I'll restore more and more of our routes to > Google slowly over the next day or two to make sure we don't max out the > second 10G also. > > Tha Why don't you ask Google to control their traffic instead? I believe they have this capability. It's probably more natural and less disruptive for them. Same for us. Best, -M<

Nov. 10, 2014

CNS Remote Switch LAG Port

by Jason Hanke

Please allocate xe-0/0/29 for a second 10G link to the CNS Remote Switch. The port will LAG with the existing 10G connection xe-0/0/38 when complete. Doug/Anthony, can you coordinate with Steve? -- Jay Hanke CTO Neutral Path Communications 3 Civic Center Plaza, Suite 204 Mankato, MN 56001 (507) 327-2398 mobile jayhanke(a)neutralpath.net www.neutralpath.net

Nov. 7, 2014

Re: MICE Traffic

by Anthony Anderberg

The Mankato Network guys were checking our support contract status, once we get that in order bouncing an email off to Juniper is certainly the way to go. From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Jeremy Lumby Sent: Thursday, November 06, 2014 9:10 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] MICE Traffic Does our support contract include software tech support where it could be emailed off to Juniper? From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Anthony Anderberg Sent: Thursday, November 06, 2014 8:53 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: [MICE-DISCUSS] MICE Traffic Summary: Yesterday afternoon we had a ~20M spike in egress traffic on MICE that seems to have been caused by the main switch not populating a MAC address or two in its L2 table. Since it appears we've seen this before and none of the recent storm control features got triggered it seems like a bug that will need a little research. I don't believe traffic exchange across the network was affected. Gory Details: In late afternoon I noticed that normally quiet ports all had identical graphs showing ~20M of egress traffic. Seeing that it started at about the same time as several changes on the CNS remote switch I called Steve Howard and together we worked through the issue. Counters indicated that it was unicast traffic and Steve wasn't seeing the same patterns on his switch so we zero'ed in on the idea that it was unknown unicast packets (unicasts who's destination MAC wasn't in the L2 forwarding database.) Since that doesn't make any sense in our environment and logs didn't show continuous storm control activity we cleared the L2 forwarding table on the main switch at about 18:41 and traffic immediately returned to normal and has remained normal since. Interestingly there was one burst of logs for a couple seconds like this at the beginning: Nov 5 13:41:04 MICE-SW1 backup MRVL-L2:mrvl_fdb_mac_entry_uc_set(),966:FDb SP HW-overwrite failed(3) for VLANIdx=3:5c:5e:ab:d5:9e:f0/48:IFL=137 I did a search on the error and one of the few hits was some yahoos in Minnesota trying to setup an Internet exchange... but that'll never work. :-) https://www.google.com/search?q=juniper+%22mrvl_fdb_mac_entry_uc_set%28%29%… Other links suggest there is a Juniper bug ID and associated software upgrade so we'll need to do a little research to see what makes sense on that front. Anthony Anderberg Sr. Systems Analyst [NUtel_email_logo_1] 320-234-5239 anthonyanderberg(a)nu-telecom.net<mailto:anthonyanderberg@nu-telecom.net> www.nutelecom.net<http://www.nutelecom.net> ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Nov. 7, 2014

Re: MICE Traffic

by Jeremy Lumby

Does our support contract include software tech support where it could be emailed off to Juniper? From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Anthony Anderberg Sent: Thursday, November 06, 2014 8:53 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] MICE Traffic Summary: Yesterday afternoon we had a ~20M spike in egress traffic on MICE that seems to have been caused by the main switch not populating a MAC address or two in its L2 table. Since it appears we’ve seen this before and none of the recent storm control features got triggered it seems like a bug that will need a little research. I don’t believe traffic exchange across the network was affected. Gory Details: In late afternoon I noticed that normally quiet ports all had identical graphs showing ~20M of egress traffic. Seeing that it started at about the same time as several changes on the CNS remote switch I called Steve Howard and together we worked through the issue. Counters indicated that it was unicast traffic and Steve wasn’t seeing the same patterns on his switch so we zero’ed in on the idea that it was unknown unicast packets (unicasts who’s destination MAC wasn’t in the L2 forwarding database.) Since that doesn’t make any sense in our environment and logs didn’t show continuous storm control activity we cleared the L2 forwarding table on the main switch at about 18:41 and traffic immediately returned to normal and has remained normal since. Interestingly there was one burst of logs for a couple seconds like this at the beginning: Nov 5 13:41:04 MICE-SW1 backup MRVL-L2:mrvl_fdb_mac_entry_uc_set(),966:FDb SP HW-overwrite failed(3) for VLANIdx=3:5c:5e:ab:d5:9e:f0/48:IFL=137 I did a search on the error and one of the few hits was some yahoos in Minnesota trying to setup an Internet exchange… but that’ll never work. :-) https://www.google.com/search?q=juniper+%22mrvl_fdb_mac_entry_uc_set%28%29%… Other links suggest there is a Juniper bug ID and associated software upgrade so we’ll need to do a little research to see what makes sense on that front. Anthony Anderberg Sr. Systems Analyst 320-234-5239 anthonyanderberg(a)nu-telecom.net www.nutelecom.net To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Nov. 6, 2014

MICE Traffic

by Anthony Anderberg

Summary: Yesterday afternoon we had a ~20M spike in egress traffic on MICE that seems to have been caused by the main switch not populating a MAC address or two in its L2 table. Since it appears we've seen this before and none of the recent storm control features got triggered it seems like a bug that will need a little research. I don't believe traffic exchange across the network was affected. Gory Details: In late afternoon I noticed that normally quiet ports all had identical graphs showing ~20M of egress traffic. Seeing that it started at about the same time as several changes on the CNS remote switch I called Steve Howard and together we worked through the issue. Counters indicated that it was unicast traffic and Steve wasn't seeing the same patterns on his switch so we zero'ed in on the idea that it was unknown unicast packets (unicasts who's destination MAC wasn't in the L2 forwarding database.) Since that doesn't make any sense in our environment and logs didn't show continuous storm control activity we cleared the L2 forwarding table on the main switch at about 18:41 and traffic immediately returned to normal and has remained normal since. Interestingly there was one burst of logs for a couple seconds like this at the beginning: Nov 5 13:41:04 MICE-SW1 backup MRVL-L2:mrvl_fdb_mac_entry_uc_set(),966:FDb SP HW-overwrite failed(3) for VLANIdx=3:5c:5e:ab:d5:9e:f0/48:IFL=137 I did a search on the error and one of the few hits was some yahoos in Minnesota trying to setup an Internet exchange... but that'll never work. :-) https://www.google.com/search?q=juniper+%22mrvl_fdb_mac_entry_uc_set%28%29%… Other links suggest there is a Juniper bug ID and associated software upgrade so we'll need to do a little research to see what makes sense on that front. Anthony Anderberg Sr. Systems Analyst [NUtel_email_logo_1] 320-234-5239 anthonyanderberg(a)nu-telecom.net<mailto:anthonyanderberg@nu-telecom.net> www.nutelecom.net<http://www.nutelecom.net>

Nov. 6, 2014

Re: Gigapop Link Saturated

by David Farmer

Northern Lights GigaPOP has its second 10G online as of about 15:00 this afternoon. There was a dirty jumper causing issues, both Cologix and I did some fiber cleaning and we whacked that mole. :) As of about 17:30, I started to restore the announcement of the GigaPOP's routes to Google. I'll restore more and more of our routes to Google slowly over the next day or two to make sure we don't max out the second 10G also. Thanks. On 10/30/14, 19:26 , Anthony Anderberg wrote: > We’ve implemented v1.0 of our community string configuration on MICE > server #2 for routes exported to Google. David@UMN shutdown his > peering with MICE server #1 and added community strings to prevent his > routes from being advertised to Google. > > With that it looks like that dropped both the GiGaPoP and Google to ~5G, > just in time for the rest of our eyeball network’s evening YouTube activity. > > We’ll see where that takes us and continue the evolution tomorrow. > > Cheers, > > anthony > > *Anthony Anderberg* > > Sr. Systems Analyst > > NUtel_email_logo_1 > > 320-234-5239 > > anthonyanderberg(a)nu-telecom.net <mailto:anthonyanderberg@nu-telecom.net> > > _www.nutelecom.net <http://www.nutelecom.net>_ > > *From:*MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] *On Behalf > Of *Jason Hanke > *Sent:* Thursday, October 30, 2014 3:27 PM > *To:* MICE-DISCUSS(a)LISTS.IPHOUSE.NET > *Subject:* Re: [MICE-DISCUSS] Gigapop Link Saturated > > It sounds like the LAG group to the GigaPop will be augmented tomorrow > to at least 2x 10G total. It is a single 10G now. > > On Thu, Oct 30, 2014 at 3:19 PM, Nathan Beard <nbeard(a)stellarllc.net > <mailto:nbeard@stellarllc.net>> wrote: > > All, > > It appears that the Gigapop link is saturated, and I've had multiple > reports of packet loss on traces to IP's that traverse the exchange > link. I'm sure people are already aware of the issue, but does > anybody have an ETA on when another link will be added or what is > currently being done? > > Thanks. > > -- > > ------------------------------------------------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > > > > -- > > Jay Hanke > > CTO > > Neutral Path Communications > > 3 Civic Center Plaza, Suite 204 > > Mankato, MN 56001 > > (507) 327-2398 mobile > > jayhanke(a)neutralpath.net <mailto:jayhanke@neutralpath.net> > > www.neutralpath.net <http://www.neutralpath.net> > > ------------------------------------------------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > > > ------------------------------------------------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > -- ================================================ David Farmer Email: farmer(a)umn.edu Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 1-612-626-0815 Minneapolis, MN 55414-3029 Cell: 1-612-812-9952 ================================================

Nov. 6, 2014

Re: Please welcome New Core Wireless to MICE

by Richard Laager

On Tue, 2014-11-04 at 21:52 -0600, Steve Howard wrote: > 1) 702 Communications will be adding a second 1Gbps port (GIg 0/2), > this port along with Gig 0/4 will form port-channel1; > 2) CNS is moving to Ten 0/11. I made these changes as if they're already done. That is, I didn't mark anything as [Future] like I do on the main MICE switch. I figure you're handling your own recordkeeping for who has what on your switch and don't need to use the participants list for that. -- Richard

Nov. 5, 2014

Re: Please welcome New Core Wireless to MICE

by Doug McIntyre

On Tue, Nov 04, 2014 at 02:59:36PM -0600, Jason Hanke wrote: > New Core Wireless > ASN: 27204 > IPv4: 206.108.255.128/24 > IPv6: 2001:504:27:0:0:6a44::1/64 > Contact: network(a)7sigma.com > > They will be connecting via the CNS switch. Steve, do you have a port > assignment? > > Route server config please. :) Sure, the RR's are all set and ready to go for New Core Wireless. Welcome! -- Doug McIntyre <merlyn(a)iphouse.net> ~.~ ipHouse ~.~ Network Engineer/Provisioning/Jack of all Trades

Nov. 5, 2014

Re: Please welcome New Core Wireless to MICE

by Steve Howard

On 11/04/2014 02:59 PM, Jason Hanke wrote: > New Core Wireless > ASN: 27204 > IPv4: 206.108.255.128/24 > IPv6: 2001:504:27:0:0:6a44::1/64 > Contact: network(a)7sigma.com > > They will be connecting via the CNS switch. Steve, do you have a port > assignment? Yes, Newcore Wireless will be on Ten 0/7. Also, there are a couple of changes planned: 1) 702 Communications will be adding a second 1Gbps port (GIg 0/2), this port along with Gig 0/4 will form port-channel1; 2) CNS is moving to Ten 0/11.

Nov. 5, 2014

Re: Please welcome New Core Wireless to MICE

by Levi Pederson

New Core Wireless, Welcome to MICE! *Levi Pederson* Mankato Networks LLC cell | 612.481.0769 work | 612.787.7392 levipederson(a)mankatonetworks.net On Tue, Nov 4, 2014 at 2:59 PM, Jason Hanke <jayhanke(a)neutralpath.net> wrote: > New Core Wireless > ASN: 27204 > IPv4: 206.108.255.128/24 > IPv6: 2001:504:27:0:0:6a44::1/64 > Contact: network(a)7sigma.com > > They will be connecting via the CNS switch. Steve, do you have a port > assignment? > > Route server config please. :) > > Welcome Aboard! > > -- > Jay Hanke > CTO > Neutral Path Communications > 3 Civic Center Plaza, Suite 204 > Mankato, MN 56001 > (507) 327-2398 mobile > jayhanke(a)neutralpath.net > www.neutralpath.net >

Nov. 4, 2014

Please welcome New Core Wireless to MICE

by Jason Hanke

New Core Wireless ASN: 27204 IPv4: 206.108.255.128/24 IPv6: 2001:504:27:0:0:6a44::1/64 Contact: network(a)7sigma.com They will be connecting via the CNS switch. Steve, do you have a port assignment? Route server config please. :) Welcome Aboard! -- Jay Hanke CTO Neutral Path Communications 3 Civic Center Plaza, Suite 204 Mankato, MN 56001 (507) 327-2398 mobile jayhanke(a)neutralpath.net www.neutralpath.net

Nov. 4, 2014

Re: 111 G+

by Mike O'Connor

holy Mackerel. On Nov 3, 2014, at 8:47 AM, Jason Hanke <jayhanke(a)NEUTRALPATH.NET> wrote: > We topped out over 111G last night on MICE! > > -- > Jay Hanke > CTO > Neutral Path Communications > 3 Civic Center Plaza, Suite 204 > Mankato, MN 56001 > (507) 327-2398 mobile > jayhanke(a)neutralpath.net > www.neutralpath.net > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > PHONE: 651-647-6109, FAX: 866-280-2356, WEB: www.haven2.com, HANDLE: OConnorStP (ID for Twitter, Facebook, LinkedIn, etc.)

Nov. 3, 2014

Re: Google IPv6

by Jeremy Lumby

They are not doing v6 at this time, however it is planned into an upgrade that is probably a few months out. -----Original Message----- From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Richard Laager Sent: Monday, November 03, 2014 11:38 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] Google IPv6 Steve Howard posted this handy URL: http://redirector.c.youtube.com/report_mapping When I visit that URL with IPv4, I'm now seeing "mice-msp1". Yay! But with IPv6, I get lvlt-bna1 (2607:f850::/32) [u]. Are others seeing Google at MICE via IPv6? -- Richard

Nov. 3, 2014

Re: Google IPv6

by Justin Krejci

Last I checked Google was not advertising IPv6 over MICE. ________________________________________ From: MICE Discuss [MICE-DISCUSS(a)LISTS.IPHOUSE.NET] on behalf of Richard Laager [rlaager(a)WIKTEL.COM] Sent: Monday, November 03, 2014 11:38 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] Google IPv6 Steve Howard posted this handy URL: http://redirector.c.youtube.com/report_mapping When I visit that URL with IPv4, I'm now seeing "mice-msp1". Yay! But with IPv6, I get lvlt-bna1 (2607:f850::/32) [u]. Are others seeing Google at MICE via IPv6? -- Richard

Nov. 3, 2014

Google IPv6

by Richard Laager

Steve Howard posted this handy URL: http://redirector.c.youtube.com/report_mapping When I visit that URL with IPv4, I'm now seeing "mice-msp1". Yay! But with IPv6, I get lvlt-bna1 (2607:f850::/32) [u]. Are others seeing Google at MICE via IPv6? -- Richard

Nov. 3, 2014

Re: 111 G+

by Anthony Anderberg

Any other Johnny Cash fans out there? How high’s the traffic Mamma? One hundred gig and rising... Anthony Anderberg Sr. Systems Analyst [NUtel_email_logo_1] 320-234-5239 anthonyanderberg(a)nu-telecom.net<mailto:anthonyanderberg@nu-telecom.net> www.nutelecom.net<http://www.nutelecom.net> From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Jason Hanke Sent: Monday, November 03, 2014 8:48 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] 111 G+ We topped out over 111G last night on MICE! -- Jay Hanke CTO Neutral Path Communications 3 Civic Center Plaza, Suite 204 Mankato, MN 56001 (507) 327-2398 mobile jayhanke(a)neutralpath.net<mailto:jayhanke@neutralpath.net> www.neutralpath.net<http://www.neutralpath.net> ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Nov. 3, 2014

Re: 111 G+

by Dan Olsen

Thank You to all the people who did the heavy lifting on this. The Graphs are simply amazing. Dan Olsen- General Manager Windomnet 443 10th Street, PO box 38 Windom MN 56101 Southwest MN Broadband Services 110 South Hwy 86, PO Box 1006, Lakefield MN 56150 Voice 507-832-8007 From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Jason Hanke Sent: Monday, November 03, 2014 8:48 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] 111 G+ We topped out over 111G last night on MICE! -- Jay Hanke CTO Neutral Path Communications 3 Civic Center Plaza, Suite 204 Mankato, MN 56001 (507) 327-2398 mobile jayhanke(a)neutralpath.net <mailto:jayhanke@neutralpath.net> www.neutralpath.net <http://www.neutralpath.net> _____ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS <http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1> &A=1

Nov. 3, 2014

Re: 111 G+

by Hannigan, Martin

Great job all. Best, -M< On Nov 3, 2014, at 15:47, Jason Hanke <jayhanke(a)NEUTRALPATH.NET<mailto:jayhanke@NEUTRALPATH.NET>> wrote: We topped out over 111G last night on MICE! -- Jay Hanke CTO Neutral Path Communications 3 Civic Center Plaza, Suite 204 Mankato, MN 56001 (507) 327-2398 mobile jayhanke(a)neutralpath.net<mailto:jayhanke@neutralpath.net> www.neutralpath.net<http://www.neutralpath.net> ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Nov. 3, 2014

Re: 111 G+

by Levi Pederson

Excellent! Great work everyone! *Levi Pederson* Mankato Networks LLC cell | 612.481.0769 work | 612.787.7392 levipederson(a)mankatonetworks.net On Mon, Nov 3, 2014 at 8:47 AM, Jason Hanke <jayhanke(a)neutralpath.net> wrote: > We topped out over 111G last night on MICE! > > -- > Jay Hanke > CTO > Neutral Path Communications > 3 Civic Center Plaza, Suite 204 > Mankato, MN 56001 > (507) 327-2398 mobile > jayhanke(a)neutralpath.net > www.neutralpath.net > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

Nov. 3, 2014

111 G+

by Jason Hanke

We topped out over 111G last night on MICE! -- Jay Hanke CTO Neutral Path Communications 3 Civic Center Plaza, Suite 204 Mankato, MN 56001 (507) 327-2398 mobile jayhanke(a)neutralpath.net www.neutralpath.net

Nov. 3, 2014

MICE Offload rates

by Jason Hanke

I'm looking for some stats on the percentage of traffic that MICE is handling vs transit links. I will use the average of the numbers to assist in bringing in additional members. You can unicast me if you don't want the info public. Thanks! -- Jay Hanke CTO Neutral Path Communications 3 Civic Center Plaza, Suite 204 Mankato, MN 56001 (507) 327-2398 mobile jayhanke(a)neutralpath.net www.neutralpath.net

Oct. 31, 2014

Re: Additional Google Ports TenG ports

by Doug McIntyre

On Fri, Oct 31, 2014 at 12:47:34PM -0500, Jason Hanke wrote: > Please allocate xe-0/0/27 and xe-0/0/28 to the Google AE group on the main > switch. Sure, all set, those members are part of the Google AE group now. -- Doug McIntyre <merlyn(a)iphouse.net> ~.~ ipHouse ~.~ Network Engineer/Provisioning/Jack of all Trades

Oct. 31, 2014

Additional Google Ports TenG ports

by Jason Hanke

Please allocate xe-0/0/27 and xe-0/0/28 to the Google AE group on the main switch. Thanks! -- Jay Hanke CTO Neutral Path Communications 3 Civic Center Plaza, Suite 204 Mankato, MN 56001 (507) 327-2398 mobile jayhanke(a)neutralpath.net www.neutralpath.net

Oct. 31, 2014

Re: Level3 Issue?

by Dan Olsen

Nothing reported Will take a gander around. Dan Olsen- General Manager Windomnet 443 10th Street, PO box 38 Windom MN 56101 Southwest MN Broadband Services 110 South Hwy 86, PO Box 1006, Lakefield MN 56150 Voice 507-832-8007 From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Anthony Anderberg Sent: Friday, October 31, 2014 1:34 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] Level3 Issue? Did anyone else see Level3 issues this morning from ~midnight to a little after 1am? _____ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS <http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1> &A=1

Oct. 31, 2014