September 2016 - mice-discuss

Re: Election

by John Unger

Congrats Richard!!! Vaultas Team From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Reid Fishler Sent: Tuesday, September 27, 2016 5:02 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] Election The winner of today's election is Richard Laager. We thank all candidates, and we thank Shaun for his service! Reid and Jay ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Sept. 28, 2016

Re: What's happening with MICE bandwidth this evening?

by Andrew Hoyos

That’s what we’re seeing too: er1.511.msp rpd[1461]: bgp_read_v4_message:10672: NOTIFICATION received from 206.108.255.37 (External AS 36040): code 6 (Cease) subcode 6 (Other Configuration Change) (which on JunOS, actually flaps the session) -- Andrew Hoyos hoyosa(a)gmail.com > On Sep 27, 2016, at 10:56 PM, anthonyanderberg(a)nu-telecom.net <anthonyanderberg(a)NU-TELECOM.NET> wrote: > > Looks the same as yesterday on both servers: > 206.108.255.37 BGP master up 22:38:09 Established > 2001:504:27:0:0:8cc8::1 BGP master start 22:42:34 Idle Received: Maximum number of prefixes reached > > But I see we've started getting these BIRD log messages: > 016-09-26 17:28:45 <RMT> NASN_36040: Received: Other configuration change > > Additionally my ping generated a "We're actively troubleshooting this issue" reply, so it all seems to fit together. > > Cheers, > anthony > > > Anthony Anderberg > Sr. Systems Analyst > 320-234-5239 > anthonyanderberg(a)nu-telecom.net > www.nutelecom.net > > > -----Original Message----- > From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Andrew Hoyos > Sent: Tuesday, September 27, 2016 10:40 PM > To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET > Subject: Re: [MICE-DISCUSS] What's happening with MICE bandwidth this evening? > > We have bilateral sessions to them, not sure on the route reflectors. Can someone check logs/BGP uptime there? > > -- > Andrew Hoyos > hoyosa(a)gmail.com > > > >> On Sep 27, 2016, at 10:25 PM, Frank Bulk <fbulk(a)mypremieronline.com> wrote: >> >> Thanks for reaching out to your Google contacts. >> >> Are you saying that MICE's route reflectors have been seeing their BGP sessions to Google's cache boxes flapping over the last few days? >> >> Frank >> >> -----Original Message----- >> From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Andrew Hoyos >> Sent: Tuesday, September 27, 2016 10:16 PM >> To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET >> Subject: Re: [MICE-DISCUSS] What's happening with MICE bandwidth this evening? >> >> We’ve had some issues with our sessions to the Google cache boxes flapping over the last few days. >> Checking with a few internal folks to Google, seemed like maybe there were a few other open tickets related to these cache boxes too. >> >> -- >> Andrew Hoyos >> hoyosa(a)gmail.com >> >> >> >>> On Sep 27, 2016, at 10:09 PM, Frank Bulk <fbulk(a)MYPREMIERONLINE.COM> wrote: >>> >>> Looks like it’s the Google LAG: >>> >>> <image001.png> >>> >>> From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Frank Bulk >>> Sent: Tuesday, September 27, 2016 10:07 PM >>> To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET >>> Subject: [MICE-DISCUSS] What's happening with MICE bandwidth this evening? >>> >>> Saw our own MICE-facing traffic drop down, and I see MICE’s own aggregate is the same. >>> >>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>> >>

Sept. 28, 2016

Re: What's happening with MICE bandwidth this evening?

by anthonyanderberg＠nu-telecom.net

Looks the same as yesterday on both servers: 206.108.255.37 BGP master up 22:38:09 Established 2001:504:27:0:0:8cc8::1 BGP master start 22:42:34 Idle Received: Maximum number of prefixes reached But I see we've started getting these BIRD log messages: 016-09-26 17:28:45 <RMT> NASN_36040: Received: Other configuration change Additionally my ping generated a "We're actively troubleshooting this issue" reply, so it all seems to fit together. Cheers, anthony Anthony Anderberg Sr. Systems Analyst 320-234-5239 anthonyanderberg(a)nu-telecom.net www.nutelecom.net -----Original Message----- From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Andrew Hoyos Sent: Tuesday, September 27, 2016 10:40 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] What's happening with MICE bandwidth this evening? We have bilateral sessions to them, not sure on the route reflectors. Can someone check logs/BGP uptime there? -- Andrew Hoyos hoyosa(a)gmail.com > On Sep 27, 2016, at 10:25 PM, Frank Bulk <fbulk(a)mypremieronline.com> wrote: > > Thanks for reaching out to your Google contacts. > > Are you saying that MICE's route reflectors have been seeing their BGP sessions to Google's cache boxes flapping over the last few days? > > Frank > > -----Original Message----- > From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Andrew Hoyos > Sent: Tuesday, September 27, 2016 10:16 PM > To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET > Subject: Re: [MICE-DISCUSS] What's happening with MICE bandwidth this evening? > > We’ve had some issues with our sessions to the Google cache boxes flapping over the last few days. > Checking with a few internal folks to Google, seemed like maybe there were a few other open tickets related to these cache boxes too. > > -- > Andrew Hoyos > hoyosa(a)gmail.com > > > >> On Sep 27, 2016, at 10:09 PM, Frank Bulk <fbulk(a)MYPREMIERONLINE.COM> wrote: >> >> Looks like it’s the Google LAG: >> >> <image001.png> >> >> From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Frank Bulk >> Sent: Tuesday, September 27, 2016 10:07 PM >> To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET >> Subject: [MICE-DISCUSS] What's happening with MICE bandwidth this evening? >> >> Saw our own MICE-facing traffic drop down, and I see MICE’s own aggregate is the same. >> >> To unsubscribe from the MICE-DISCUSS list, click the following link: >> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >> >

Sept. 28, 2016

Re: What's happening with MICE bandwidth this evening?

by Andrew Hoyos

We have bilateral sessions to them, not sure on the route reflectors. Can someone check logs/BGP uptime there? -- Andrew Hoyos hoyosa(a)gmail.com > On Sep 27, 2016, at 10:25 PM, Frank Bulk <fbulk(a)mypremieronline.com> wrote: > > Thanks for reaching out to your Google contacts. > > Are you saying that MICE's route reflectors have been seeing their BGP sessions to Google's cache boxes flapping over the last few days? > > Frank > > -----Original Message----- > From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Andrew Hoyos > Sent: Tuesday, September 27, 2016 10:16 PM > To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET > Subject: Re: [MICE-DISCUSS] What's happening with MICE bandwidth this evening? > > We’ve had some issues with our sessions to the Google cache boxes flapping over the last few days. > Checking with a few internal folks to Google, seemed like maybe there were a few other open tickets related to these cache boxes too. > > -- > Andrew Hoyos > hoyosa(a)gmail.com > > > >> On Sep 27, 2016, at 10:09 PM, Frank Bulk <fbulk(a)MYPREMIERONLINE.COM> wrote: >> >> Looks like it’s the Google LAG: >> >> <image001.png> >> >> From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Frank Bulk >> Sent: Tuesday, September 27, 2016 10:07 PM >> To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET >> Subject: [MICE-DISCUSS] What's happening with MICE bandwidth this evening? >> >> Saw our own MICE-facing traffic drop down, and I see MICE’s own aggregate is the same. >> >> To unsubscribe from the MICE-DISCUSS list, click the following link: >> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >> >

Sept. 28, 2016

Re: What's happening with MICE bandwidth this evening?

by Frank Bulk

Thanks for reaching out to your Google contacts. Are you saying that MICE's route reflectors have been seeing their BGP sessions to Google's cache boxes flapping over the last few days? Frank -----Original Message----- From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Andrew Hoyos Sent: Tuesday, September 27, 2016 10:16 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] What's happening with MICE bandwidth this evening? We’ve had some issues with our sessions to the Google cache boxes flapping over the last few days. Checking with a few internal folks to Google, seemed like maybe there were a few other open tickets related to these cache boxes too. -- Andrew Hoyos hoyosa(a)gmail.com > On Sep 27, 2016, at 10:09 PM, Frank Bulk <fbulk(a)MYPREMIERONLINE.COM> wrote: > > Looks like it’s the Google LAG: > > <image001.png> > > From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Frank Bulk > Sent: Tuesday, September 27, 2016 10:07 PM > To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET > Subject: [MICE-DISCUSS] What's happening with MICE bandwidth this evening? > > Saw our own MICE-facing traffic drop down, and I see MICE’s own aggregate is the same. > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

Sept. 28, 2016

Re: What's happening with MICE bandwidth this evening?

by anthonyanderberg＠nu-telecom.net

Yesterday I noticed they were running into a limit to the number of IPv6 prefixes they'd accept from MICE (now over 15K) but hadn't seen any issues beyond that. Their GGC team said they were looking into it this morning, I'll ping them and mention these issues as well. Anthony Anderberg Sr. Systems Analyst 320-234-5239 anthonyanderberg(a)nu-telecom.net www.nutelecom.net -----Original Message----- From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Andrew Hoyos Sent: Tuesday, September 27, 2016 10:16 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] What's happening with MICE bandwidth this evening? We’ve had some issues with our sessions to the Google cache boxes flapping over the last few days. Checking with a few internal folks to Google, seemed like maybe there were a few other open tickets related to these cache boxes too. -- Andrew Hoyos hoyosa(a)gmail.com > On Sep 27, 2016, at 10:09 PM, Frank Bulk <fbulk(a)MYPREMIERONLINE.COM> wrote: > > Looks like it’s the Google LAG: > > <image001.png> > > From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Frank Bulk > Sent: Tuesday, September 27, 2016 10:07 PM > To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET > Subject: [MICE-DISCUSS] What's happening with MICE bandwidth this evening? > > Saw our own MICE-facing traffic drop down, and I see MICE’s own aggregate is the same. > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

Sept. 28, 2016

Re: What's happening with MICE bandwidth this evening?

by Jeremy Lumby

The tech team has some open tickets at the moment with Google. They said they were escalating them. -----Original Message----- From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Andrew Hoyos Sent: Tuesday, September 27, 2016 10:16 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] What's happening with MICE bandwidth this evening? We’ve had some issues with our sessions to the Google cache boxes flapping over the last few days. Checking with a few internal folks to Google, seemed like maybe there were a few other open tickets related to these cache boxes too. -- Andrew Hoyos hoyosa(a)gmail.com > On Sep 27, 2016, at 10:09 PM, Frank Bulk <fbulk(a)MYPREMIERONLINE.COM> wrote: > > Looks like it’s the Google LAG: > > <image001.png> > > From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Frank Bulk > Sent: Tuesday, September 27, 2016 10:07 PM > To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET > Subject: [MICE-DISCUSS] What's happening with MICE bandwidth this evening? > > Saw our own MICE-facing traffic drop down, and I see MICE’s own aggregate is the same. > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

Sept. 28, 2016

Re: What's happening with MICE bandwidth this evening?

by Andrew Hoyos

We’ve had some issues with our sessions to the Google cache boxes flapping over the last few days. Checking with a few internal folks to Google, seemed like maybe there were a few other open tickets related to these cache boxes too. -- Andrew Hoyos hoyosa(a)gmail.com > On Sep 27, 2016, at 10:09 PM, Frank Bulk <fbulk(a)MYPREMIERONLINE.COM> wrote: > > Looks like it’s the Google LAG: > > <image001.png> > > From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Frank Bulk > Sent: Tuesday, September 27, 2016 10:07 PM > To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET > Subject: [MICE-DISCUSS] What's happening with MICE bandwidth this evening? > > Saw our own MICE-facing traffic drop down, and I see MICE’s own aggregate is the same. > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

Sept. 28, 2016

Re: What's happening with MICE bandwidth this evening?

by Frank Bulk

Looks like it's the Google LAG: [cid:image001.png@01D2190B.D3667E90] From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Frank Bulk Sent: Tuesday, September 27, 2016 10:07 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] What's happening with MICE bandwidth this evening? Saw our own MICE-facing traffic drop down, and I see MICE's own aggregate is the same.

Sept. 28, 2016

What's happening with MICE bandwidth this evening?

by Frank Bulk

Saw our own MICE-facing traffic drop down, and I see MICE's own aggregate is the same. [cid:image004.jpg@01D2190B.83B53030] [cid:image003.png@01D2190B.83A5EDF0] Frank

Sept. 28, 2016