- mice-discuss - lists.micemn.net

Re: Youtube issues

by Shaun Carlson

I did notice that the local cache seems to have a little more traffic than usual lately. I wonder if that's related. s Shaun Carlson Director of Information Technology | Arvig ph: (218) (tel:(218)%C2%A0346-8673)346-8673 (tel:(218)%C2%A0346-8673) | em: shaun.carlson(a)arvig.com (mailto:shaun.carlson@arvig.com) On Feb 24, 2017, 21:00 -0600, Ben Wiechman <ben.wiechman(a)arvig.com>, wrote: > We're trying to track down specific reports. We have a local cache so I think that is masking most of the potential issues. One or two folks mentioned some Youtube buffering after being informed of a potential issue so it seems likely there is some impact, but I couldn't say with complete certainty that congestion through MICE is the cause. > > I did open a ticket with Google as well to see if they would make a specific recommendation or otherwise comment on their suggested course of action to minimize customer impact. We'll see what they say. > > We've needed an excuse to review how we are advertising our prefixes to Google anyway. > > Our interest was in taking a (semi) proactive stance to attempt to prioritize other sources until this issue is resolved. > > > Ben Wiechman > Network Engineer IV | Arvig > Direct: 320.256.0184 (tel:320.256.0184) > Cell: 320.247.3224 (tel:320.247.3224) > Office: 320.256.7471 (tel:320.256.7471) > ben.wiechman(a)arvig.com (mailto:ben.wiechman@arvig.com) > > > > > > > > > > > > > > On Fri, Feb 24, 2017 at 8:54 PM, Andrew Hoyos <hoyosa(a)gmail.com (mailto:hoyosa@gmail.com)> wrote: > > Ben, is Arvig seeing problems too? > > > > Seems like at this scale and number of complaints, Google should work their black magic to shed some traffic to different GGC nodes, rather than all of us having to actively try and coerce traffic elsewhere. > > > > There clearly seems to be an issue here, and Google has yet to ack. > > > > -- > > Andrew Hoyos > > hoyosa(a)gmail.com (mailto:hoyosa@gmail.com) > > > > > > On Feb 24, 2017, at 8:46 PM, Ben Wiechman <ben.wiechman(a)ARVIG.COM (mailto:ben.wiechman@ARVIG.COM)> wrote: > > > > > To confirm what others have found to be effective. > > > > > > Are you advertising community 15169:13000 to the route servers, and on any direct peering sessions with AS36040? > > > > > > > > > Ben Wiechman > > > Network Engineer IV | Arvig > > > Direct: 320.256.0184 (tel:(320)%20256-0184) > > > Cell: 320.247.3224 (tel:(320)%20247-3224) > > > Office: 320.256.7471 (tel:(320)%20256-7471) > > > ben.wiechman(a)arvig.com (mailto:ben.wiechman@arvig.com) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Feb 24, 2017 at 3:29 PM, Mike Horwath <drechsau(a)geeks.org (mailto:drechsau@geeks.org)> wrote: > > > > On Fri, Feb 24, 2017 at 10:38:43AM -0600, Andrew Hoyos wrote: > > > > > > On Feb 24, 2017, at 10:26 AM, Mike Horwath <drechsau(a)Geeks.ORG (mailto:drechsau@Geeks.ORG)> wrote: > > > > > > > > > > > > On Fri, Feb 24, 2017 at 08:56:33AM -0600, Andrew Hoyos wrote: > > > > > >> 36040 is a customer of HE.net (http://HE.net) in MSP, so you'd need to send those to > > > > > >> HE as well if you peer with them or are a customer. Otherwise they > > > > > >> will learn your prefixes that way too, sans communities. > > > > > > > > > > > > Why is a customer sending other peoples' routes into HE to then go > > > > > > into the exchange? > > > > > > > > > > 36040 is a customer of HE.net (http://HE.net) So, HE.net (http://HE.net) is sending them a full > > > > > table or some subset (including their peers and customers), as > > > > > expected. > > > > > > > > I misread and now understand. > > > > > > > > Thank you! > > > > > > > > I had taken it the other way which was making no freakin' sense at all > > > > in todays world of interconnectivity. > > > > > > > > -- > > > > Mike Horwath, reachable via drechsau(a)Geeks.ORG (mailto:drechsau@Geeks.ORG) > > > > > > > > > > > > To unsubscribe from the MICE-DISCUSS list, click the following link: > > > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > > > > > > > > > > > > > > > > To unsubscribe from the MICE-DISCUSS list, click the following link: > > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > > > > > > > > > > > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

Feb. 25, 2017

Re: Youtube issues

by Ben Wiechman

We're trying to track down specific reports. We have a local cache so I think that is masking most of the potential issues. One or two folks mentioned some Youtube buffering after being informed of a potential issue so it seems likely there is some impact, but I couldn't say with complete certainty that congestion through MICE is the cause. I did open a ticket with Google as well to see if they would make a specific recommendation or otherwise comment on their suggested course of action to minimize customer impact. We'll see what they say. We've needed an excuse to review how we are advertising our prefixes to Google anyway. Our interest was in taking a (semi) proactive stance to attempt to prioritize other sources until this issue is resolved. Ben Wiechman Network Engineer IV | Arvig Direct: 320.256.0184 Cell: 320.247.3224 Office: 320.256.7471 ben.wiechman(a)arvig.com On Fri, Feb 24, 2017 at 8:54 PM, Andrew Hoyos <hoyosa(a)gmail.com> wrote: > Ben, is Arvig seeing problems too? > > Seems like at this scale and number of complaints, Google should work > their black magic to shed some traffic to different GGC nodes, rather than > all of us having to actively try and coerce traffic elsewhere. > > There clearly seems to be an issue here, and Google has yet to ack. > > -- > Andrew Hoyos > hoyosa(a)gmail.com > > On Feb 24, 2017, at 8:46 PM, Ben Wiechman <ben.wiechman(a)ARVIG.COM> wrote: > > To confirm what others have found to be effective. > > Are you advertising community 15169:13000 to the route servers, and on any > direct peering sessions with AS36040? > > > Ben Wiechman > Network Engineer IV | Arvig > Direct: 320.256.0184 <(320)%20256-0184> > Cell: 320.247.3224 <(320)%20247-3224> > Office: 320.256.7471 <(320)%20256-7471> > ben.wiechman(a)arvig.com > > On Fri, Feb 24, 2017 at 3:29 PM, Mike Horwath <drechsau(a)geeks.org> wrote: > >> On Fri, Feb 24, 2017 at 10:38:43AM -0600, Andrew Hoyos wrote: >> > > On Feb 24, 2017, at 10:26 AM, Mike Horwath <drechsau(a)Geeks.ORG> >> wrote: >> > > >> > > On Fri, Feb 24, 2017 at 08:56:33AM -0600, Andrew Hoyos wrote: >> > >> 36040 is a customer of HE.net in MSP, so you'd need to send those to >> > >> HE as well if you peer with them or are a customer. Otherwise they >> > >> will learn your prefixes that way too, sans communities. >> > > >> > > Why is a customer sending other peoples' routes into HE to then go >> > > into the exchange? >> > >> > 36040 is a customer of HE.net. So, HE.net is sending them a full >> > table or some subset (including their peers and customers), as >> > expected. >> >> I misread and now understand. >> >> Thank you! >> >> I had taken it the other way which was making no freakin' sense at all >> in todays world of interconnectivity. >> >> -- >> Mike Horwath, reachable via drechsau(a)Geeks.ORG >> > > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

Feb. 25, 2017

Re: Youtube issues

by Andrew Hoyos

Ben, is Arvig seeing problems too? Seems like at this scale and number of complaints, Google should work their black magic to shed some traffic to different GGC nodes, rather than all of us having to actively try and coerce traffic elsewhere. There clearly seems to be an issue here, and Google has yet to ack. -- Andrew Hoyos hoyosa(a)gmail.com > On Feb 24, 2017, at 8:46 PM, Ben Wiechman <ben.wiechman(a)ARVIG.COM> wrote: > > To confirm what others have found to be effective. > > Are you advertising community 15169:13000 to the route servers, and on any direct peering sessions with AS36040? > > > Ben Wiechman > Network Engineer IV | Arvig > Direct: 320.256.0184 > Cell: 320.247.3224 > Office: 320.256.7471 > ben.wiechman(a)arvig.com > >> On Fri, Feb 24, 2017 at 3:29 PM, Mike Horwath <drechsau(a)geeks.org> wrote: >> On Fri, Feb 24, 2017 at 10:38:43AM -0600, Andrew Hoyos wrote: >> > > On Feb 24, 2017, at 10:26 AM, Mike Horwath <drechsau(a)Geeks.ORG> wrote: >> > > >> > > On Fri, Feb 24, 2017 at 08:56:33AM -0600, Andrew Hoyos wrote: >> > >> 36040 is a customer of HE.net in MSP, so you'd need to send those to >> > >> HE as well if you peer with them or are a customer. Otherwise they >> > >> will learn your prefixes that way too, sans communities. >> > > >> > > Why is a customer sending other peoples' routes into HE to then go >> > > into the exchange? >> > >> > 36040 is a customer of HE.net. So, HE.net is sending them a full >> > table or some subset (including their peers and customers), as >> > expected. >> >> I misread and now understand. >> >> Thank you! >> >> I had taken it the other way which was making no freakin' sense at all >> in todays world of interconnectivity. >> >> -- >> Mike Horwath, reachable via drechsau(a)Geeks.ORG > > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Feb. 25, 2017

Re: Youtube issues

by Ben Wiechman

To confirm what others have found to be effective. Are you advertising community 15169:13000 to the route servers, and on any direct peering sessions with AS36040? Ben Wiechman Network Engineer IV | Arvig Direct: 320.256.0184 Cell: 320.247.3224 Office: 320.256.7471 ben.wiechman(a)arvig.com On Fri, Feb 24, 2017 at 3:29 PM, Mike Horwath <drechsau(a)geeks.org> wrote: > On Fri, Feb 24, 2017 at 10:38:43AM -0600, Andrew Hoyos wrote: > > > On Feb 24, 2017, at 10:26 AM, Mike Horwath <drechsau(a)Geeks.ORG> wrote: > > > > > > On Fri, Feb 24, 2017 at 08:56:33AM -0600, Andrew Hoyos wrote: > > >> 36040 is a customer of HE.net in MSP, so you'd need to send those to > > >> HE as well if you peer with them or are a customer. Otherwise they > > >> will learn your prefixes that way too, sans communities. > > > > > > Why is a customer sending other peoples' routes into HE to then go > > > into the exchange? > > > > 36040 is a customer of HE.net. So, HE.net is sending them a full > > table or some subset (including their peers and customers), as > > expected. > > I misread and now understand. > > Thank you! > > I had taken it the other way which was making no freakin' sense at all > in todays world of interconnectivity. > > -- > Mike Horwath, reachable via drechsau(a)Geeks.ORG >

Feb. 25, 2017

Re: Youtube issues

by Mike Horwath

On Fri, Feb 24, 2017 at 10:38:43AM -0600, Andrew Hoyos wrote: > > On Feb 24, 2017, at 10:26 AM, Mike Horwath <drechsau(a)Geeks.ORG> wrote: > > > > On Fri, Feb 24, 2017 at 08:56:33AM -0600, Andrew Hoyos wrote: > >> 36040 is a customer of HE.net in MSP, so you'd need to send those to > >> HE as well if you peer with them or are a customer. Otherwise they > >> will learn your prefixes that way too, sans communities. > > > > Why is a customer sending other peoples' routes into HE to then go > > into the exchange? > > 36040 is a customer of HE.net. So, HE.net is sending them a full > table or some subset (including their peers and customers), as > expected. I misread and now understand. Thank you! I had taken it the other way which was making no freakin' sense at all in todays world of interconnectivity. -- Mike Horwath, reachable via drechsau(a)Geeks.ORG

Feb. 24, 2017

Re: Youtube issues

by Andrew Hoyos

> On Feb 24, 2017, at 10:26 AM, Mike Horwath <drechsau(a)Geeks.ORG> wrote: > > On Fri, Feb 24, 2017 at 08:56:33AM -0600, Andrew Hoyos wrote: >> 36040 is a customer of HE.net in MSP, so you'd need to send those to >> HE as well if you peer with them or are a customer. Otherwise they >> will learn your prefixes that way too, sans communities. > > Why is a customer sending other peoples' routes into HE to then go > into the exchange? 36040 is a customer of HE.net. So, HE.net is sending them a full table or some subset (including their peers and customers), as expected. In turn, HE.net will advertise to *you*, their customer routes (including 36040, note the _6939_36040 path there), being learned via HE.net 208.117.253.0/24 *[BGP/170] 19:20:53, MED 0, localpref 200 AS path: 6939 36040 I, validation-state: unverified > to 206.108.255.52 via xe-0/0/0.0 Just saying, that if you depeer 36040, or add communities on a bilateral session to them, you’ll need to do the same if peering with HE.net or a customer of HE.net, as 36040 will learn your prefixes from 6939 too.

Feb. 24, 2017

Re: Youtube issues

by Mike Horwath

On Fri, Feb 24, 2017 at 08:56:33AM -0600, Andrew Hoyos wrote: > 36040 is a customer of HE.net in MSP, so you'd need to send those to > HE as well if you peer with them or are a customer. Otherwise they > will learn your prefixes that way too, sans communities. Why is a customer sending other peoples' routes into HE to then go into the exchange? -- Mike Horwath, reachable via drechsau(a)Geeks.ORG

Feb. 24, 2017

Re: Youtube issues

by Andrew Hoyos

36040 is a customer of HE.net in MSP, so you'd need to send those to HE as well if you peer with them or are a customer. Otherwise they will learn your prefixes that way too, sans communities. -- Andrew Hoyos hoyosa(a)gmail.com > On Feb 24, 2017, at 8:53 AM, Doug McIntyre <merlyn(a)IPHOUSE.NET> wrote: > > FWIW: in discussion with others, using communities back to the route-servers that > tell them to not pass your prefix onto Google does NOT seem to work. They seem > to cache and hold onto your information even after BGP prefix withdrawl. > > Going more active like below seems to be more effective. > > > >> On Fri, Feb 24, 2017 at 07:57:00AM -0600, Jeremy Lumby wrote: >> Matthew, >> >> As part of the tech committee, I recommend Colin's solution of Community Strings that I have copied in below. This should give everyone the control they need to help their connection until Google completes the upgrade. >> >> Jeremy >> >> >> We've had success with the action communities documented here: >> >> https://isp.google.com/static/downloads/BGPCommunitySupport.pdf >> >> Sending 15169:* to 36040 seems to work, including via route servers. >> >> -- >> Colin Baker >> SupraNet Communications, Inc. >> (608) 572-7634 >> colinb(a)supranet.net >> >> >> From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Matthew Beckwell >> Sent: Friday, February 24, 2017 7:48 AM >> To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET >> Subject: Re: [MICE-DISCUSS] Youtube issues >> >> If it's creating issues for other members, it's time to take action until the 2 new 10G's are in place. >> >> What are our options? >> Remove from the route server? Ask large users to take one for the team and use their own transit for awhile? >> >> We need the technical committee to take the lead here. >> ~Matthew >> >> On Thu, Feb 23, 2017 at 11:47 PM, Richard Laager <rlaager(a)wiktel.com> wrote: >> I would imagine it is related to the Google congestion. We were seeing YouTube buffering yesterday. >> >> -- >> Richard >> >> >> >> >> >> To unsubscribe from the MICE-DISCUSS list, click the following link: >> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > > -- > Doug McIntyre <merlyn(a)iphouse.net> > ~.~ ipHouse ~.~ > Network Engineer/Provisioning/Jack of all Trades

Feb. 24, 2017

Re: Youtube issues

by Doug McIntyre

FWIW: in discussion with others, using communities back to the route-servers that tell them to not pass your prefix onto Google does NOT seem to work. They seem to cache and hold onto your information even after BGP prefix withdrawl. Going more active like below seems to be more effective. On Fri, Feb 24, 2017 at 07:57:00AM -0600, Jeremy Lumby wrote: > Matthew, > > As part of the tech committee, I recommend Colin's solution of Community Strings that I have copied in below. This should give everyone the control they need to help their connection until Google completes the upgrade. > > Jeremy > > > We've had success with the action communities documented here: > > https://isp.google.com/static/downloads/BGPCommunitySupport.pdf > > Sending 15169:* to 36040 seems to work, including via route servers. > > -- > Colin Baker > SupraNet Communications, Inc. > (608) 572-7634 > colinb(a)supranet.net > > > From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Matthew Beckwell > Sent: Friday, February 24, 2017 7:48 AM > To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET > Subject: Re: [MICE-DISCUSS] Youtube issues > > If it's creating issues for other members, it's time to take action until the 2 new 10G's are in place. > > What are our options? > Remove from the route server? Ask large users to take one for the team and use their own transit for awhile? > > We need the technical committee to take the lead here. > ~Matthew > > On Thu, Feb 23, 2017 at 11:47 PM, Richard Laager <rlaager(a)wiktel.com> wrote: > I would imagine it is related to the Google congestion. We were seeing YouTube buffering yesterday. > > -- > Richard > > > > > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 -- Doug McIntyre <merlyn(a)iphouse.net> ~.~ ipHouse ~.~ Network Engineer/Provisioning/Jack of all Trades

Feb. 24, 2017

Re: Youtube issues

by Jeremy Lumby

Matthew, As part of the tech committee, I recommend Colin's solution of Community Strings that I have copied in below. This should give everyone the control they need to help their connection until Google completes the upgrade. Jeremy We've had success with the action communities documented here: https://isp.google.com/static/downloads/BGPCommunitySupport.pdf Sending 15169:* to 36040 seems to work, including via route servers. -- Colin Baker SupraNet Communications, Inc. (608) 572-7634 colinb(a)supranet.net From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Matthew Beckwell Sent: Friday, February 24, 2017 7:48 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Youtube issues If it's creating issues for other members, it's time to take action until the 2 new 10G's are in place. What are our options? Remove from the route server? Ask large users to take one for the team and use their own transit for awhile? We need the technical committee to take the lead here. ~Matthew On Thu, Feb 23, 2017 at 11:47 PM, Richard Laager <rlaager(a)wiktel.com> wrote: I would imagine it is related to the Google congestion. We were seeing YouTube buffering yesterday. -- Richard To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Feb. 24, 2017