Any one else notice all their peers bounce last night around 1AM CST, and a few again just a few minutes ago? Just checking to see If I should talk to my transport provider or if this is something @ MICE. Let me know! James Urwiller Network Architect Stealth Broadband james.urwiller@stealthbroadband.com [image: stealth300]
On Tue, Jan 22, 2019 at 03:41:54PM -0600, James Urwiller wrote:
Any one else notice all their peers bounce last night around 1AM CST, and a few again just a few minutes ago?
Just checking to see If I should talk to my transport provider or if this is something @ MICE. Let me know!
No changes here. Looking at the route servers, there's a few BGP sessions changes in the last 24h, but nothing like what you've reported. -- Doug McIntyre <merlyn@iphouse.net> ~.~ ipHouse ~.~ Network Engineer/Provisioning/Jack of all Trades
Looks like the ports making up the Zayo-Remote bounced down and back up at 15:31:19. I’d thought perhaps Zayo was adding the 3rd link to that LAG but it is still showing two active members. [signature_198002810] From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> on behalf of James Urwiller <james.urwiller@TURNKEY-TELECOM.COM> Reply-To: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> Date: Tuesday, January 22, 2019 at 3:41 PM To: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: [MICE-DISCUSS] Exchange Stable Any one else notice all their peers bounce last night around 1AM CST, and a few again just a few minutes ago? Just checking to see If I should talk to my transport provider or if this is something @ MICE. Let me know! James Urwiller Network Architect Stealth Broadband james.urwiller@stealthbroadband.com<mailto:james.urwiller@stealthbroadband.com> [stealth300] ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
Zayo is my transport.. I will bring it up with them. Thank you! James Urwiller Network Architect Stealth Broadband james.urwiller@stealthbroadband.com [image: stealth300] *From:* MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> *On Behalf Of * AnthonyAnderberg@NUVERA.NET *Sent:* Tuesday, January 22, 2019 15:52 *To:* MICE-DISCUSS@LISTS.IPHOUSE.NET *Subject:* Re: [MICE-DISCUSS] Exchange Stable Looks like the ports making up the Zayo-Remote bounced down and back up at 15:31:19. I’d thought perhaps Zayo was adding the 3rd link to that LAG but it is still showing two active members. [image: signature_198002810] *From: *MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> on behalf of James Urwiller <james.urwiller@TURNKEY-TELECOM.COM> *Reply-To: *MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> *Date: *Tuesday, January 22, 2019 at 3:41 PM *To: *MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> *Subject: *[MICE-DISCUSS] Exchange Stable Any one else notice all their peers bounce last night around 1AM CST, and a few again just a few minutes ago? Just checking to see If I should talk to my transport provider or if this is something @ MICE. Let me know! James Urwiller Network Architect Stealth Broadband james.urwiller@stealthbroadband.com [image: stealth300] ------------------------------ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ------------------------------ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
Looks like it went down last night too, or at least traffic on that LAG dropped to zero – I see a little drip from the silliness a bit ago too: [cid:image001.png@01D4B26A.E02ED4B0] [signature_521369910] From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> on behalf of James Urwiller <james.urwiller@TURNKEY-TELECOM.COM> Reply-To: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> Date: Tuesday, January 22, 2019 at 3:52 PM To: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Exchange Stable Zayo is my transport.. I will bring it up with them. Thank you! James Urwiller Network Architect Stealth Broadband james.urwiller@stealthbroadband.com<mailto:james.urwiller@stealthbroadband.com> [stealth300] From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET>> On Behalf Of AnthonyAnderberg@NUVERA.NET<mailto:AnthonyAnderberg@NUVERA.NET> Sent: Tuesday, January 22, 2019 15:52 To: MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Exchange Stable Looks like the ports making up the Zayo-Remote bounced down and back up at 15:31:19. I’d thought perhaps Zayo was adding the 3rd link to that LAG but it is still showing two active members. [signature_198002810] From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET>> on behalf of James Urwiller <james.urwiller@TURNKEY-TELECOM.COM<mailto:james.urwiller@TURNKEY-TELECOM.COM>> Reply-To: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET>> Date: Tuesday, January 22, 2019 at 3:41 PM To: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET>> Subject: [MICE-DISCUSS] Exchange Stable Any one else notice all their peers bounce last night around 1AM CST, and a few again just a few minutes ago? Just checking to see If I should talk to my transport provider or if this is something @ MICE. Let me know! James Urwiller Network Architect Stealth Broadband james.urwiller@stealthbroadband.com<mailto:james.urwiller@stealthbroadband.com> [stealth300] ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
Hi James, My sessions are showing uptimes of ~4days on the MICE, which correlates to the main switch maintenance last week. You may want to check in with Zayo as it looks that they lost all traffic on the remote switch in Belle Plaine at 1AM. Traffic graphs are not quite as clear this afternoon, but there was an upward spike on several ports. http://micelg.usinternet.com/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=52 Andy Koch Hoyos Consulting LLC ofc: +1 608 616 9950 andyk@hoyosconsulting.com http://www.hoyosconsulting.com On 1/22/19 3:41 PM, James Urwiller wrote:
Any one else notice all their peers bounce last night around 1AM CST, and a few again just a few minutes ago?
Just checking to see If I should talk to my transport provider or if this is something @ MICE. Let me know!
James Urwiller
Network Architect
Stealth Broadband
james.urwiller@stealthbroadband.com <mailto:james.urwiller@stealthbroadband.com>
stealth300
------------------------------------------------------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
Zayo has had a fiber crew out working today from what we’ve been told. It’s been impacting our traffic for sure. From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> On Behalf Of Andy Koch Sent: Tuesday, January 22, 2019 2:58 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Exchange Stable Hi James, My sessions are showing uptimes of ~4days on the MICE, which correlates to the main switch maintenance last week. You may want to check in with Zayo as it looks that they lost all traffic on the remote switch in Belle Plaine at 1AM. Traffic graphs are not quite as clear this afternoon, but there was an upward spike on several ports. http://micelg.usinternet.com/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=52 Andy Koch Hoyos Consulting LLC ofc: +1 608 616 9950 andyk@hoyosconsulting.com<mailto:andyk@hoyosconsulting.com> http://www.hoyosconsulting.com On 1/22/19 3:41 PM, James Urwiller wrote: Any one else notice all their peers bounce last night around 1AM CST, and a few again just a few minutes ago? Just checking to see If I should talk to my transport provider or if this is something @ MICE. Let me know! James Urwiller Network Architect Stealth Broadband james.urwiller@stealthbroadband.com<mailto:james.urwiller@stealthbroadband.com> [stealth300] ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
We were told Zayo had scheduled maintenance but did not send notifications to all parties that were to be affected (eg US Internet). We have several circuits still affected (ie are down) and we are continuing to claw through their support channels and are hearing some disturbing things. ________________________________ From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> on behalf of Jeremy Schmeichel <Jeremy.Schmeichel@VASTBROADBAND.COM> Sent: Tuesday, January 22, 2019 5:07 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Exchange Stable Zayo has had a fiber crew out working today from what we've been told. It's been impacting our traffic for sure. From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> On Behalf Of Andy Koch Sent: Tuesday, January 22, 2019 2:58 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Exchange Stable Hi James, My sessions are showing uptimes of ~4days on the MICE, which correlates to the main switch maintenance last week. You may want to check in with Zayo as it looks that they lost all traffic on the remote switch in Belle Plaine at 1AM. Traffic graphs are not quite as clear this afternoon, but there was an upward spike on several ports. http://micelg.usinternet.com/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=52 Andy Koch Hoyos Consulting LLC ofc: +1 608 616 9950 andyk@hoyosconsulting.com<mailto:andyk@hoyosconsulting.com> http://www.hoyosconsulting.com On 1/22/19 3:41 PM, James Urwiller wrote: Any one else notice all their peers bounce last night around 1AM CST, and a few again just a few minutes ago? Just checking to see If I should talk to my transport provider or if this is something @ MICE. Let me know! James Urwiller Network Architect Stealth Broadband james.urwiller@stealthbroadband.com<mailto:james.urwiller@stealthbroadband.com> [stealth300] ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
Zayo had scheduled maintenance for the 14th, rescheduled it to the 16th and then canceled maintenance (on the 16th)… but then our link to MICE, through their network, was down today (22nd) from ~12:07 to ~1:10 am with no warning at all. Even the maintenance window notifications were unclear – no real clarity from Zayo change maintenance. We also saw the BGP service interruption at 3:32 pm, and then a link down at 3:33 pm (presumably when the 100G wave over which our 10G rides went down). =( Frank From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> On Behalf Of Jeremy Schmeichel Sent: Tuesday, January 22, 2019 5:08 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Exchange Stable Zayo has had a fiber crew out working today from what we’ve been told. It’s been impacting our traffic for sure. From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET>> On Behalf Of Andy Koch Sent: Tuesday, January 22, 2019 2:58 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Exchange Stable Hi James, My sessions are showing uptimes of ~4days on the MICE, which correlates to the main switch maintenance last week. You may want to check in with Zayo as it looks that they lost all traffic on the remote switch in Belle Plaine at 1AM. Traffic graphs are not quite as clear this afternoon, but there was an upward spike on several ports. http://micelg.usinternet.com/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=52 Andy Koch Hoyos Consulting LLC ofc: +1 608 616 9950 andyk@hoyosconsulting.com<mailto:andyk@hoyosconsulting.com> http://www.hoyosconsulting.com On 1/22/19 3:41 PM, James Urwiller wrote: Any one else notice all their peers bounce last night around 1AM CST, and a few again just a few minutes ago? Just checking to see If I should talk to my transport provider or if this is something @ MICE. Let me know! James Urwiller Network Architect Stealth Broadband james.urwiller@stealthbroadband.com<mailto:james.urwiller@stealthbroadband.com> [stealth300] ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
same Frank!! same... i got a call back from Zayo noc at 19:07 tonight, with no real assurance that anyone knew what was going on. they were just asking if I was still experiencing issues. :( Sent from my iPhone 7
On Jan 22, 2019, at 18:38, Frank Bulk <fbulk@mypremieronline.com> wrote:
Zayo had scheduled maintenance for the 14th, rescheduled it to the 16th and then canceled maintenance (on the 16th)… but then our link to MICE, through their network, was down today (22nd) from ~12:07 to ~1:10 am with no warning at all.
Even the maintenance window notifications were unclear – no real clarity from Zayo change maintenance.
We also saw the BGP service interruption at 3:32 pm, and then a link down at 3:33 pm (presumably when the 100G wave over which our 10G rides went down). =(
Frank
From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> On Behalf Of Jeremy Schmeichel Sent: Tuesday, January 22, 2019 5:08 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Exchange Stable
Zayo has had a fiber crew out working today from what we’ve been told. It’s been impacting our traffic for sure.
From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> On Behalf Of Andy Koch Sent: Tuesday, January 22, 2019 2:58 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Exchange Stable
Hi James,
My sessions are showing uptimes of ~4days on the MICE, which correlates to the main switch maintenance last week. You may want to check in with Zayo as it looks that they lost all traffic on the remote switch in Belle Plaine at 1AM. Traffic graphs are not quite as clear this afternoon, but there was an upward spike on several ports.
http://micelg.usinternet.com/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=52
Andy Koch Hoyos Consulting LLC ofc: +1 608 616 9950 andyk@hoyosconsulting.com http://www.hoyosconsulting.com
On 1/22/19 3:41 PM, James Urwiller wrote: Any one else notice all their peers bounce last night around 1AM CST, and a few again just a few minutes ago? Just checking to see If I should talk to my transport provider or if this is something @ MICE. Let me know!
James Urwiller Network Architect Stealth Broadband james.urwiller@stealthbroadband.com
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
participants (7)
-
Andy Koch -
AnthonyAnderberg＠nuvera.net -
Doug McIntyre -
Frank Bulk -
James Urwiller -
Jeremy Schmeichel -
Justin Krejci