Four BGP sessions drops with Google/YouTube (AS36040) today
FYI, we saw our BGP session drop with AS36040 several times today: * 12:08:46 to 12:08:51 pm * 2:01 to 2:02 pm * ~4:02 to ~4:04 pm * ~7:36 to ~7:38 pm Graphs show some serious stability issues: [cid:image001.png@01D48368.C4C26A60] Regards, Frank
I have noticed that Google has had me redirected to their cache on The Kansas City Internet Exchange most of the day. 2602:ffaa:0:7:8d5:1b2c:f084:513c => google-mci2 : superx_isp_number: 3 (2602:ffaa::/36) From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Frank Bulk Sent: Friday, November 23, 2018 8:14 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today FYI, we saw our BGP session drop with AS36040 several times today: 12:08:46 to 12:08:51 pm 2:01 to 2:02 pm ~4:02 to ~4:04 pm ~7:36 to ~7:38 pm Graphs show some serious stability issues: Regards, Frank To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
Sorry to go off topic a little but why isn't Google advertising their main AS 15169 on MICE? Their website shows Minneapolis being one of their Edge POPs but they're using their less common AS 36040 here. Anyone know why we're not as cool as the other IX's they're on with 15169? On Fri, Nov 23, 2018 at 8:21 PM Jeremy Lumby <jlumby@mnvoip.com> wrote:
I have noticed that Google has had me redirected to their cache on The Kansas City Internet Exchange most of the day.
2602:ffaa:0:7:8d5:1b2c:f084:513c => google-mci2 : superx_isp_number: 3 (2602:ffaa::/36)
*From:* MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] *On Behalf Of *Frank Bulk *Sent:* Friday, November 23, 2018 8:14 PM *To:* MICE-DISCUSS@LISTS.IPHOUSE.NET *Subject:* [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today
FYI, we saw our BGP session drop with AS36040 several times today:
- 12:08:46 to 12:08:51 pm - 2:01 to 2:02 pm - ~4:02 to ~4:04 pm - ~7:36 to ~7:38 pm
Graphs show some serious stability issues:
Regards,
Frank
------------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
------------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
-- Darin Steffl Minnesota WiFi www.mnwifi.com 507-634-WiFi <http://www.facebook.com/minnesotawifi> Like us on Facebook <http://www.facebook.com/minnesotawifi>
MICE only has a cache, and the other IXes have full backbone routes. From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Darin Steffl Sent: Friday, November 23, 2018 8:56 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today Sorry to go off topic a little but why isn't Google advertising their main AS 15169 on MICE? Their website shows Minneapolis being one of their Edge POPs but they're using their less common AS 36040 here. Anyone know why we're not as cool as the other IX's they're on with 15169? On Fri, Nov 23, 2018 at 8:21 PM Jeremy Lumby <jlumby@mnvoip.com> wrote: I have noticed that Google has had me redirected to their cache on The Kansas City Internet Exchange most of the day. 2602:ffaa:0:7:8d5:1b2c:f084:513c => google-mci2 : superx_isp_number: 3 (2602:ffaa::/36) From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Frank Bulk Sent: Friday, November 23, 2018 8:14 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today FYI, we saw our BGP session drop with AS36040 several times today: 12:08:46 to 12:08:51 pm 2:01 to 2:02 pm ~4:02 to ~4:04 pm ~7:36 to ~7:38 pm Graphs show some serious stability issues: Regards, Frank To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 -- Darin Steffl Minnesota WiFi www.mnwifi.com 507-634-WiFi Like us on Facebook To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
Ok I know that Google is on the route servers so is there any advantage to establishing a direct peering session with them? Their peering site says the following: Although Google connects to Internet Exchange route servers where present, Google prefers to establish direct BGP sessions for traffic exchange. On Fri, Nov 23, 2018 at 8:59 PM Jeremy Lumby <jlumby@mnvoip.com> wrote:
MICE only has a cache, and the other IXes have full backbone routes.
*From:* MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] *On Behalf Of *Darin Steffl *Sent:* Friday, November 23, 2018 8:56 PM *To:* MICE-DISCUSS@LISTS.IPHOUSE.NET *Subject:* Re: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today
Sorry to go off topic a little but why isn't Google advertising their main AS 15169 on MICE? Their website shows Minneapolis being one of their Edge POPs but they're using their less common AS 36040 here.
Anyone know why we're not as cool as the other IX's they're on with 15169?
On Fri, Nov 23, 2018 at 8:21 PM Jeremy Lumby <jlumby@mnvoip.com> wrote:
I have noticed that Google has had me redirected to their cache on The Kansas City Internet Exchange most of the day.
2602:ffaa:0:7:8d5:1b2c:f084:513c => google-mci2 : superx_isp_number: 3 (2602:ffaa::/36)
*From:* MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] *On Behalf Of *Frank Bulk *Sent:* Friday, November 23, 2018 8:14 PM *To:* MICE-DISCUSS@LISTS.IPHOUSE.NET *Subject:* [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today
FYI, we saw our BGP session drop with AS36040 several times today:
- 12:08:46 to 12:08:51 pm - 2:01 to 2:02 pm - ~4:02 to ~4:04 pm - ~7:36 to ~7:38 pm
Graphs show some serious stability issues:
Regards,
Frank
------------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
------------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
--
Darin Steffl
Minnesota WiFi
www.mnwifi.com
507-634-WiFi
<http://www.facebook.com/minnesotawifi> Like us on Facebook <http://www.facebook.com/minnesotawifi>
------------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
------------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
-- Darin Steffl Minnesota WiFi www.mnwifi.com 507-634-WiFi <http://www.facebook.com/minnesotawifi> Like us on Facebook <http://www.facebook.com/minnesotawifi>
In general you can get a little better granular control of your session if you are bilaterally peered since not all IX route servers support communities, and have different security features supported/enabled. Not all implementations of communities are as detailed as you can get bilaterally. There is also one less point of failure when not using routeservers. From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Darin Steffl Sent: Friday, November 23, 2018 9:59 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today Ok I know that Google is on the route servers so is there any advantage to establishing a direct peering session with them? Their peering site says the following: Although Google connects to Internet Exchange route servers where present, Google prefers to establish direct BGP sessions for traffic exchange. On Fri, Nov 23, 2018 at 8:59 PM Jeremy Lumby <jlumby@mnvoip.com> wrote: MICE only has a cache, and the other IXes have full backbone routes. From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Darin Steffl Sent: Friday, November 23, 2018 8:56 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today Sorry to go off topic a little but why isn't Google advertising their main AS 15169 on MICE? Their website shows Minneapolis being one of their Edge POPs but they're using their less common AS 36040 here. Anyone know why we're not as cool as the other IX's they're on with 15169? On Fri, Nov 23, 2018 at 8:21 PM Jeremy Lumby <jlumby@mnvoip.com> wrote: I have noticed that Google has had me redirected to their cache on The Kansas City Internet Exchange most of the day. 2602:ffaa:0:7:8d5:1b2c:f084:513c => google-mci2 : superx_isp_number: 3 (2602:ffaa::/36) From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Frank Bulk Sent: Friday, November 23, 2018 8:14 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today FYI, we saw our BGP session drop with AS36040 several times today: 12:08:46 to 12:08:51 pm 2:01 to 2:02 pm ~4:02 to ~4:04 pm ~7:36 to ~7:38 pm Graphs show some serious stability issues: Error! Filename not specified. Regards, Frank To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 -- Darin Steffl Minnesota WiFi www.mnwifi.com 507-634-WiFi Like us on Facebook To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 -- Darin Steffl Minnesota WiFi www.mnwifi.com 507-634-WiFi Like us on Facebook To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
GGC shared this with me: The BGP session and aggregated interfaces seems stable now, however earlier, we've seen all interfaces on the path to 206.108.255.4 flap. We'll re-check it after some time to verify that this is no longer happening. Any log entries on the IX’s switch that give a hint as to why the links are dropping? Frank From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> On Behalf Of Jeremy Lumby Sent: Friday, November 23, 2018 11:31 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today In general you can get a little better granular control of your session if you are bilaterally peered since not all IX route servers support communities, and have different security features supported/enabled. Not all implementations of communities are as detailed as you can get bilaterally. There is also one less point of failure when not using routeservers. From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Darin Steffl Sent: Friday, November 23, 2018 9:59 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today Ok I know that Google is on the route servers so is there any advantage to establishing a direct peering session with them? Their peering site says the following: Although Google connects to Internet Exchange route servers where present, Google prefers to establish direct BGP sessions for traffic exchange. On Fri, Nov 23, 2018 at 8:59 PM Jeremy Lumby <jlumby@mnvoip.com<mailto:jlumby@mnvoip.com>> wrote: MICE only has a cache, and the other IXes have full backbone routes. From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET>] On Behalf Of Darin Steffl Sent: Friday, November 23, 2018 8:56 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today Sorry to go off topic a little but why isn't Google advertising their main AS 15169 on MICE? Their website shows Minneapolis being one of their Edge POPs but they're using their less common AS 36040 here. Anyone know why we're not as cool as the other IX's they're on with 15169? On Fri, Nov 23, 2018 at 8:21 PM Jeremy Lumby <jlumby@mnvoip.com<mailto:jlumby@mnvoip.com>> wrote: I have noticed that Google has had me redirected to their cache on The Kansas City Internet Exchange most of the day. 2602:ffaa:0:7:8d5:1b2c:f084:513c => google-mci2 : superx_isp_number: 3 (2602:ffaa::/36) From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET>] On Behalf Of Frank Bulk Sent: Friday, November 23, 2018 8:14 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today FYI, we saw our BGP session drop with AS36040 several times today: * 12:08:46 to 12:08:51 pm * 2:01 to 2:02 pm * ~4:02 to ~4:04 pm * ~7:36 to ~7:38 pm Graphs show some serious stability issues: Error! Filename not specified. Regards, Frank ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 -- Darin Steffl Minnesota WiFi www.mnwifi.com<http://www.mnwifi.com/> 507-634-WiFi [http://www.snoitulosten.com/wp-content/uploads/2010/01/facebook-small.jpg]<http://www.facebook.com/minnesotawifi> Like us on Facebook<http://www.facebook.com/minnesotawifi> ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 -- Darin Steffl Minnesota WiFi www.mnwifi.com<http://www.mnwifi.com/> 507-634-WiFi [http://www.snoitulosten.com/wp-content/uploads/2010/01/facebook-small.jpg]<http://www.facebook.com/minnesotawifi> Like us on Facebook<http://www.facebook.com/minnesotawifi> ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
Google only has a single 100G link to MICE, and it has been up for a month. They did not originally support 100G so I believe they added a switch just to convert their existing 10X10G into a single 100G This is however their equipment, and MICE has no visibility into it. Ethernet3/2/1 is up, line protocol is up (connected) Hardware is Ethernet, address is 2899.3a54.4fdc (bia 2899.3a54.4fdc) Description: 36040 - Google Ethernet MTU 9214 bytes , BW 100000000 kbit Full-duplex, 100Gb/s, auto negotiation: off, uni-link: n/a Up 31 days, 2 hours, 55 minutes, 9 seconds Loopback Mode : None 3 link status changes since last clear Last clearing of "show interface" counters never 5 minutes input rate 41.9 Gbps (42.5% with framing overhead), 3606486 packets/sec 5 minutes output rate 1.99 Gbps (2.1% with framing overhead), 957337 packets/sec 101767257001795 packets input, 147833598676116893 bytes Received 1446585 broadcasts, 7171769 multicast 0 runts, 0 giants 0 input errors, 0 CRC, 0 alignment, 0 symbol, 0 input discards 0 PAUSE input 29025844160305 packets output, 6218148508474554 bytes Sent 655350278 broadcasts, 1990085124 multicast 0 output errors, 0 collisions 0 late collision, 0 deferred, 1 output discards 0 PAUSE output From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Frank Bulk Sent: Saturday, November 24, 2018 2:43 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today GGC shared this with me: The BGP session and aggregated interfaces seems stable now, however earlier, we've seen all interfaces on the path to 206.108.255.4 flap. We'll re-check it after some time to verify that this is no longer happening. Any log entries on the IX’s switch that give a hint as to why the links are dropping? Frank From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> On Behalf Of Jeremy Lumby Sent: Friday, November 23, 2018 11:31 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today In general you can get a little better granular control of your session if you are bilaterally peered since not all IX route servers support communities, and have different security features supported/enabled. Not all implementations of communities are as detailed as you can get bilaterally. There is also one less point of failure when not using routeservers. From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Darin Steffl Sent: Friday, November 23, 2018 9:59 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today Ok I know that Google is on the route servers so is there any advantage to establishing a direct peering session with them? Their peering site says the following: Although Google connects to Internet Exchange route servers where present, Google prefers to establish direct BGP sessions for traffic exchange. On Fri, Nov 23, 2018 at 8:59 PM Jeremy Lumby <jlumby@mnvoip.com> wrote: MICE only has a cache, and the other IXes have full backbone routes. From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Darin Steffl Sent: Friday, November 23, 2018 8:56 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today Sorry to go off topic a little but why isn't Google advertising their main AS 15169 on MICE? Their website shows Minneapolis being one of their Edge POPs but they're using their less common AS 36040 here. Anyone know why we're not as cool as the other IX's they're on with 15169? On Fri, Nov 23, 2018 at 8:21 PM Jeremy Lumby <jlumby@mnvoip.com> wrote: I have noticed that Google has had me redirected to their cache on The Kansas City Internet Exchange most of the day. 2602:ffaa:0:7:8d5:1b2c:f084:513c => google-mci2 : superx_isp_number: 3 (2602:ffaa::/36) From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Frank Bulk Sent: Friday, November 23, 2018 8:14 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today FYI, we saw our BGP session drop with AS36040 several times today: 12:08:46 to 12:08:51 pm 2:01 to 2:02 pm ~4:02 to ~4:04 pm ~7:36 to ~7:38 pm Graphs show some serious stability issues: Error! Filename not specified. Regards, Frank To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 -- Darin Steffl Minnesota WiFi www.mnwifi.com 507-634-WiFi Like us on Facebook To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 -- Darin Steffl Minnesota WiFi www.mnwifi.com 507-634-WiFi Like us on Facebook To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
Thanks – today’s graphs didn’t look so good today, either. Frank From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> On Behalf Of Jeremy Lumby Sent: Saturday, November 24, 2018 3:19 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today Google only has a single 100G link to MICE, and it has been up for a month. They did not originally support 100G so I believe they added a switch just to convert their existing 10X10G into a single 100G This is however their equipment, and MICE has no visibility into it. Ethernet3/2/1 is up, line protocol is up (connected) Hardware is Ethernet, address is 2899.3a54.4fdc (bia 2899.3a54.4fdc) Description: 36040 - Google Ethernet MTU 9214 bytes , BW 100000000 kbit Full-duplex, 100Gb/s, auto negotiation: off, uni-link: n/a Up 31 days, 2 hours, 55 minutes, 9 seconds Loopback Mode : None 3 link status changes since last clear Last clearing of "show interface" counters never 5 minutes input rate 41.9 Gbps (42.5% with framing overhead), 3606486 packets/sec 5 minutes output rate 1.99 Gbps (2.1% with framing overhead), 957337 packets/sec 101767257001795 packets input, 147833598676116893 bytes Received 1446585 broadcasts, 7171769 multicast 0 runts, 0 giants 0 input errors, 0 CRC, 0 alignment, 0 symbol, 0 input discards 0 PAUSE input 29025844160305 packets output, 6218148508474554 bytes Sent 655350278 broadcasts, 1990085124 multicast 0 output errors, 0 collisions 0 late collision, 0 deferred, 1 output discards 0 PAUSE output From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Frank Bulk Sent: Saturday, November 24, 2018 2:43 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today GGC shared this with me: The BGP session and aggregated interfaces seems stable now, however earlier, we've seen all interfaces on the path to 206.108.255.4 flap. We'll re-check it after some time to verify that this is no longer happening. Any log entries on the IX’s switch that give a hint as to why the links are dropping? Frank From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET>> On Behalf Of Jeremy Lumby Sent: Friday, November 23, 2018 11:31 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today In general you can get a little better granular control of your session if you are bilaterally peered since not all IX route servers support communities, and have different security features supported/enabled. Not all implementations of communities are as detailed as you can get bilaterally. There is also one less point of failure when not using routeservers. From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Darin Steffl Sent: Friday, November 23, 2018 9:59 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today Ok I know that Google is on the route servers so is there any advantage to establishing a direct peering session with them? Their peering site says the following: Although Google connects to Internet Exchange route servers where present, Google prefers to establish direct BGP sessions for traffic exchange. On Fri, Nov 23, 2018 at 8:59 PM Jeremy Lumby <jlumby@mnvoip.com<mailto:jlumby@mnvoip.com>> wrote: MICE only has a cache, and the other IXes have full backbone routes. From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET>] On Behalf Of Darin Steffl Sent: Friday, November 23, 2018 8:56 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today Sorry to go off topic a little but why isn't Google advertising their main AS 15169 on MICE? Their website shows Minneapolis being one of their Edge POPs but they're using their less common AS 36040 here. Anyone know why we're not as cool as the other IX's they're on with 15169? On Fri, Nov 23, 2018 at 8:21 PM Jeremy Lumby <jlumby@mnvoip.com<mailto:jlumby@mnvoip.com>> wrote: I have noticed that Google has had me redirected to their cache on The Kansas City Internet Exchange most of the day. 2602:ffaa:0:7:8d5:1b2c:f084:513c => google-mci2 : superx_isp_number: 3 (2602:ffaa::/36) From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET>] On Behalf Of Frank Bulk Sent: Friday, November 23, 2018 8:14 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today FYI, we saw our BGP session drop with AS36040 several times today: * 12:08:46 to 12:08:51 pm * 2:01 to 2:02 pm * ~4:02 to ~4:04 pm * ~7:36 to ~7:38 pm Graphs show some serious stability issues: Error! Filename not specified. Regards, Frank ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 -- Darin Steffl Minnesota WiFi www.mnwifi.com<http://www.mnwifi.com/> 507-634-WiFi [http://www.snoitulosten.com/wp-content/uploads/2010/01/facebook-small.jpg]<http://www.facebook.com/minnesotawifi> Like us on Facebook<http://www.facebook.com/minnesotawifi> ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 -- Darin Steffl Minnesota WiFi www.mnwifi.com<http://www.mnwifi.com/> 507-634-WiFi [http://www.snoitulosten.com/wp-content/uploads/2010/01/facebook-small.jpg]<http://www.facebook.com/minnesotawifi> Like us on Facebook<http://www.facebook.com/minnesotawifi> ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
I had a little bit more dialogue with GGC support over the weekend and they were able to resolve the issue Sunday evening. They saw LACP issues from their cache(s) to their 10G/100G switch, but they didn’t initially realize that it was their switch. Jeremy provided me the necessary info to point GGC support in the right direction and just after 8 pm Central they updated me that they’ve resolved the issue, though they said they do not have full root cause for it, yet. Graph looks solid for the last 3+ hours. Frank From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> On Behalf Of Frank Bulk Sent: Saturday, November 24, 2018 9:54 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today Thanks – today’s graphs didn’t look so good today, either. Frank From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET>> On Behalf Of Jeremy Lumby Sent: Saturday, November 24, 2018 3:19 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today Google only has a single 100G link to MICE, and it has been up for a month. They did not originally support 100G so I believe they added a switch just to convert their existing 10X10G into a single 100G This is however their equipment, and MICE has no visibility into it. Ethernet3/2/1 is up, line protocol is up (connected) Hardware is Ethernet, address is 2899.3a54.4fdc (bia 2899.3a54.4fdc) Description: 36040 - Google Ethernet MTU 9214 bytes , BW 100000000 kbit Full-duplex, 100Gb/s, auto negotiation: off, uni-link: n/a Up 31 days, 2 hours, 55 minutes, 9 seconds Loopback Mode : None 3 link status changes since last clear Last clearing of "show interface" counters never 5 minutes input rate 41.9 Gbps (42.5% with framing overhead), 3606486 packets/sec 5 minutes output rate 1.99 Gbps (2.1% with framing overhead), 957337 packets/sec 101767257001795 packets input, 147833598676116893 bytes Received 1446585 broadcasts, 7171769 multicast 0 runts, 0 giants 0 input errors, 0 CRC, 0 alignment, 0 symbol, 0 input discards 0 PAUSE input 29025844160305 packets output, 6218148508474554 bytes Sent 655350278 broadcasts, 1990085124 multicast 0 output errors, 0 collisions 0 late collision, 0 deferred, 1 output discards 0 PAUSE output From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Frank Bulk Sent: Saturday, November 24, 2018 2:43 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today GGC shared this with me: The BGP session and aggregated interfaces seems stable now, however earlier, we've seen all interfaces on the path to 206.108.255.4 flap. We'll re-check it after some time to verify that this is no longer happening. Any log entries on the IX’s switch that give a hint as to why the links are dropping? Frank From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET>> On Behalf Of Jeremy Lumby Sent: Friday, November 23, 2018 11:31 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today In general you can get a little better granular control of your session if you are bilaterally peered since not all IX route servers support communities, and have different security features supported/enabled. Not all implementations of communities are as detailed as you can get bilaterally. There is also one less point of failure when not using routeservers. From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Darin Steffl Sent: Friday, November 23, 2018 9:59 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today Ok I know that Google is on the route servers so is there any advantage to establishing a direct peering session with them? Their peering site says the following: Although Google connects to Internet Exchange route servers where present, Google prefers to establish direct BGP sessions for traffic exchange. On Fri, Nov 23, 2018 at 8:59 PM Jeremy Lumby <jlumby@mnvoip.com<mailto:jlumby@mnvoip.com>> wrote: MICE only has a cache, and the other IXes have full backbone routes. From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET>] On Behalf Of Darin Steffl Sent: Friday, November 23, 2018 8:56 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today Sorry to go off topic a little but why isn't Google advertising their main AS 15169 on MICE? Their website shows Minneapolis being one of their Edge POPs but they're using their less common AS 36040 here. Anyone know why we're not as cool as the other IX's they're on with 15169? On Fri, Nov 23, 2018 at 8:21 PM Jeremy Lumby <jlumby@mnvoip.com<mailto:jlumby@mnvoip.com>> wrote: I have noticed that Google has had me redirected to their cache on The Kansas City Internet Exchange most of the day. 2602:ffaa:0:7:8d5:1b2c:f084:513c => google-mci2 : superx_isp_number: 3 (2602:ffaa::/36) From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET>] On Behalf Of Frank Bulk Sent: Friday, November 23, 2018 8:14 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: [MICE-DISCUSS] Four BGP sessions drops with Google/YouTube (AS36040) today FYI, we saw our BGP session drop with AS36040 several times today: * 12:08:46 to 12:08:51 pm * 2:01 to 2:02 pm * ~4:02 to ~4:04 pm * ~7:36 to ~7:38 pm Graphs show some serious stability issues: Error! Filename not specified. Regards, Frank ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 -- Darin Steffl Minnesota WiFi www.mnwifi.com<http://www.mnwifi.com/> 507-634-WiFi [http://www.snoitulosten.com/wp-content/uploads/2010/01/facebook-small.jpg]<http://www.facebook.com/minnesotawifi> Like us on Facebook<http://www.facebook.com/minnesotawifi> ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 -- Darin Steffl Minnesota WiFi www.mnwifi.com<http://www.mnwifi.com/> 507-634-WiFi [http://www.snoitulosten.com/wp-content/uploads/2010/01/facebook-small.jpg]<http://www.facebook.com/minnesotawifi> Like us on Facebook<http://www.facebook.com/minnesotawifi> ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
participants (3)
-
Darin Steffl -
Frank Bulk -
Jeremy Lumby