Does anyone know why Google dropped all traffic? Looks like a fiber cut since it's at 0. -- Darin Steffl Minnesota WiFi www.mnwifi.com 507-634-WiFi <http://www.facebook.com/minnesotawifi> Like us on Facebook <http://www.facebook.com/minnesotawifi>
There does appear to be an issue between Madison and Chicago on Level3/CL glass. Google backhauls to Chicago on this.
On Apr 21, 2020, at 9:26 AM, Darin Steffl <darin.steffl@mnwifi.com> wrote:
Does anyone know why Google dropped all traffic? Looks like a fiber cut since it's at 0.
-- Darin Steffl Minnesota WiFi www.mnwifi.com 507-634-WiFi Like us on Facebook
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
On 2020-04-21 09:32, Andrew Hoyos wrote:
There does appear to be an issue between Madison and Chicago on Level3/CL glass. Google backhauls to Chicago on this.
Do you have a time this occurred? Possibly unrelated, but we had a Level3/Centurylink circuit go down (times in CDT): Apr 21 2020 08:53:59.515: %LINK-3-UPDOWN: Interface TenGigabitEthernet2/0/5, changed state to down Apr 21 2020 08:53:59.566: %IOSXE_SPA-6-UPDOWN: Interface TenGigabitEthernet2/0/5, link down due to local fault This was a MKE circuit, but I believe it was being backhauled to Chicago. -James
On Apr 21, 2020, at 9:26 AM, Darin Steffl <darin.steffl@mnwifi.com> wrote:
Does anyone know why Google dropped all traffic? Looks like a fiber cut since it's at 0.
--
Darin Steffl Minnesota WiFi www.mnwifi.com [1] 507-634-WiFi [2] Like us on Facebook [2]
-------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
-------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
Links: ------ [1] http://www.mnwifi.com/ [2] http://www.facebook.com/minnesotawifi
-- -James
Trigger time: 4/21/2020 9:00 AM Triggered by: 206.108.255.141 Alert Definition: BGP Neighbors Escalation: Level 1 Acknowledged by: Not yet... Jeff Wilde Park Region Telephone Company P: 218-826-8330 E: jeff.wilde@parkregion.com -----Original Message----- From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> On Behalf Of James Stahr Sent: Tuesday, April 21, 2020 10:03 AM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Google Dropped On 2020-04-21 09:32, Andrew Hoyos wrote:
There does appear to be an issue between Madison and Chicago on Level3/CL glass. Google backhauls to Chicago on this.
Do you have a time this occurred? Possibly unrelated, but we had a Level3/Centurylink circuit go down (times in CDT): Apr 21 2020 08:53:59.515: %LINK-3-UPDOWN: Interface TenGigabitEthernet2/0/5, changed state to down Apr 21 2020 08:53:59.566: %IOSXE_SPA-6-UPDOWN: Interface TenGigabitEthernet2/0/5, link down due to local fault This was a MKE circuit, but I believe it was being backhauled to Chicago. -James
On Apr 21, 2020, at 9:26 AM, Darin Steffl <darin.steffl@mnwifi.com> wrote:
Does anyone know why Google dropped all traffic? Looks like a fiber cut since it's at 0.
--
Darin Steffl Minnesota WiFi www.mnwifi.com [1] 507-634-WiFi [2] Like us on Facebook [2]
-------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
-------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
Links: ------ [1] http://www.mnwifi.com/ [2] http://www.facebook.com/minnesotawifi
-- -James
Make that 8:55 alarm came in a little late. Jeff Wilde Park Region Telephone Company P: 218-826-8330 E: jeff.wilde@parkregion.com -----Original Message----- From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> On Behalf Of Jeff Wilde Sent: Tuesday, April 21, 2020 10:06 AM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Google Dropped Trigger time: 4/21/2020 9:00 AM Triggered by: 206.108.255.141 Alert Definition: BGP Neighbors Escalation: Level 1 Acknowledged by: Not yet... Jeff Wilde Park Region Telephone Company P: 218-826-8330 E: jeff.wilde@parkregion.com -----Original Message----- From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> On Behalf Of James Stahr Sent: Tuesday, April 21, 2020 10:03 AM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Google Dropped On 2020-04-21 09:32, Andrew Hoyos wrote:
There does appear to be an issue between Madison and Chicago on Level3/CL glass. Google backhauls to Chicago on this.
Do you have a time this occurred? Possibly unrelated, but we had a Level3/Centurylink circuit go down (times in CDT): Apr 21 2020 08:53:59.515: %LINK-3-UPDOWN: Interface TenGigabitEthernet2/0/5, changed state to down Apr 21 2020 08:53:59.566: %IOSXE_SPA-6-UPDOWN: Interface TenGigabitEthernet2/0/5, link down due to local fault This was a MKE circuit, but I believe it was being backhauled to Chicago. -James
On Apr 21, 2020, at 9:26 AM, Darin Steffl <darin.steffl@mnwifi.com> wrote:
Does anyone know why Google dropped all traffic? Looks like a fiber cut since it's at 0.
--
Darin Steffl Minnesota WiFi www.mnwifi.com [1] 507-634-WiFi [2] Like us on Facebook [2]
-------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
-------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
Links: ------ [1] http://www.mnwifi.com/ [2] http://www.facebook.com/minnesotawifi
-- -James
On Apr 21, 2020, at 10:02 AM, James Stahr <0000000d8a27a856-dmarc-request@LISTS.IPHOUSE.NET> wrote:
On 2020-04-21 09:32, Andrew Hoyos wrote:
There does appear to be an issue between Madison and Chicago on Level3/CL glass. Google backhauls to Chicago on this.
Do you have a time this occurred? Possibly unrelated, but we had a Level3/Centurylink circuit go down (times in CDT):
Apr 21 2020 08:53:59.515: %LINK-3-UPDOWN: Interface TenGigabitEthernet2/0/5, changed state to down Apr 21 2020 08:53:59.566: %IOSXE_SPA-6-UPDOWN: Interface TenGigabitEthernet2/0/5, link down due to local fault
This was a MKE circuit, but I believe it was being backhauled to Chicago.
Lines up with a few we’re seeing down: Apr 21 08:54:00 xxxxxxxx mib2d[2109]: SNMP_TRAP_LINK_DOWN: ifIndex 538, ifAdminStatus up(1), ifOperStatus down(2), ifName xe-2/0/0 Level3 in MKE also does head down to Chicago. MSP -> MSN -> MKE -> ORD is their primary backbone.
Looks like a fiber cut between Milwaukee, WI and Zion, IL. Guessing it’ll be a bit! Latest update: *** CASCADED EXTERNAL NOTES 21-Apr-2020 15:48:56 GMT From CASE: 18572279 - Event The Transport NOC reports that one Field Operations team is set to arrive in Milwaukee, WI momentarily, and the secondary Field Operations team remains in transit to the gateway facility in Zion, IL. An ETA of 17:00 GMT remains expected.
On Apr 21, 2020, at 9:32 AM, Andrew Hoyos <hoyosa@gmail.com> wrote:
There does appear to be an issue between Madison and Chicago on Level3/CL glass. Google backhauls to Chicago on this.
On Apr 21, 2020, at 9:26 AM, Darin Steffl <darin.steffl@mnwifi.com> wrote:
Does anyone know why Google dropped all traffic? Looks like a fiber cut since it's at 0.
-- Darin Steffl Minnesota WiFi www.mnwifi.com <http://www.mnwifi.com/> 507-634-WiFi <http://www.facebook.com/minnesotawifi> Like us on Facebook <http://www.facebook.com/minnesotawifi> To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 <http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1>
Here is what I received as a customer due to latency issues with CL - I believe this is directly related. - Update provided by Centurylink: "Network Operations Center reports that services are impacted by multiple fiber cuts that are impacting the CenturyLink network. Field Operations and all necessary repair personnel have arrived to the first fiber cut location and are currently working to pull in the slack on the fiber span. Approximately 3,000 feet of new fiber will be required to pulled in." On Tue, Apr 21, 2020, 10:55 Andrew Hoyos <hoyosa@gmail.com> wrote:
Looks like a fiber cut between Milwaukee, WI and Zion, IL. Guessing it’ll be a bit!
Latest update:
*** CASCADED EXTERNAL NOTES 21-Apr-2020 15:48:56 GMT From CASE: 18572279 - Event The Transport NOC reports that one Field Operations team is set to arrive in Milwaukee, WI momentarily, and the secondary Field Operations team remains in transit to the gateway facility in Zion, IL. An ETA of 17:00 GMT remains expected.
On Apr 21, 2020, at 9:32 AM, Andrew Hoyos <hoyosa@gmail.com> wrote:
There does appear to be an issue between Madison and Chicago on Level3/CL glass. Google backhauls to Chicago on this.
On Apr 21, 2020, at 9:26 AM, Darin Steffl <darin.steffl@mnwifi.com> wrote:
Does anyone know why Google dropped all traffic? Looks like a fiber cut since it's at 0.
-- Darin Steffl Minnesota WiFi www.mnwifi.com 507-634-WiFi <http://www.facebook.com/minnesotawifi> Like us on Facebook <http://www.facebook.com/minnesotawifi>
------------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
------------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
participants (5)
-
Andrew Hoyos -
Andrew Johnson -
Darin Steffl -
James Stahr -
Jeff Wilde