- mice-discuss - lists.micemn.net

Re: [EXTERNAL] - Re: [MICE-DISCUSS] Peering with Google no longer available at MICE IX

by David Farmer

The GigaPoP receives the vast majority of its Google traffic from MICE. Our regular daytime peak is 20 to 25G, occasionally up to 30ish. On Fri, Aug 23, 2024 at 6:34 AM Miles McCredie <Miles.McCredie(a)midco.com> wrote: > Curious how many MICE members are in each of the following buckets. > > > > 1. Receive limited portion of Google traffic via MICE > > Midco and, I assume, US Internet are in > this bucket > > 2. Receive a significant portion of Google traffic via MICE today > > > > I’ve only got a couple years of experience with Google for Midco but my > experience has been that Google will not always provide the cache capacity > that I think is appropriate. The last two augments were triggered 9-12 > months later than I would have liked. Could be my fault for creating new > GNLs but… > > > > > > > > *Miles McCredie * > Principal Network Engineer II-Core IP > > Office: 6052755192 > Miles.McCredie(a)midco.com > > *Midco.com* > > Let's go beyond. > > *From:* MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> *On Behalf Of *Justin > Krejci > *Sent:* Thursday, August 22, 2024 5:25 PM > *To:* MICE-DISCUSS(a)LISTS.IPHOUSE.NET > *Subject:* [EXTERNAL] - Re: [MICE-DISCUSS] Peering with Google no longer > available at MICE IX > > > > *CAUTION:* This email originated from outside of *MIDCO*. > *Do not click links or open attachments unless you recognize the sender > and know the content is safe.* > > I am not sure what Google's stance is on deploying cache servers at an IX > vs remote connection like the ones pending disconnection. > > If they were to deploy caching nodes at an IX, I assume they would still > have a cache-fill requirement in terms of connectivity that has to come > from somewhere... of which they appear to be disinterested in maintaining > long distance connections going forward. So perhaps this is a donation > opportunity for some to entice Google to deploy caching nodes at MICE? > Donate local colo and transit bandwidth along with local MICE connectivity? > > > > We have numerous Google caching nodes in our network as well as multiple > PNIs to Google, so their pending disconnect from MICE will likely have > minimal direct impact on our eyeballs. > > > > > > -----Original Message----- > > *From*: Aaron Wendel <aaron(a)WHOLESALEINTERNET.NET > <Aaron%20Wendel%20%3caaron(a)WHOLESALEINTERNET.NET%3e>> > > *Reply-To*: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET > <MICE%20Discuss%20%3cMICE-DISCUSS(a)LISTS.IPHOUSE.NET%3e>> > > *To*: MICE-DISCUSS(a)LISTS.IPHOUSE.NET > > *Subject*: Re: [MICE-DISCUSS] Peering with Google no longer available at > MICE IX > > *Date*: Thu, 22 Aug 2024 22:03:01 +0000 > > > > Has MICE requested a cache box from Google? > > > > > > > > ------ Original Message ------ > > From "Anthony Anderberg" <AnthonyAnderberg(a)NUVERA.NET> > > To MICE-DISCUSS(a)LISTS.IPHOUSE.NET > > Date 8/22/2024 4:28:18 PM > > Subject Re: [MICE-DISCUSS] Peering with Google no longer available at > > MICE IX > > > > I've been surprised that folks haven't been talking about this more and am > curious about everyone's thoughts - as MICE user-members but also in terms > of MICE leadership and direction. > > > > As a user-member I feel somewhat conflicted: on one hand in an effort to > maximize performance I could obtain transport to Chicago and connect to > Google there, but I feel like doing so is making Google's problem into my > problem and that doing so undercuts the goals of MICE. We've seen content > providers move MICE's larger members toward local direct private peering, > and I can understand rational behind that even if it shifts traffic away > from MICE - but Google is proposing remote direct peering at member's cost > which feels different. > > > > Of course nobody expects Google to subsidize our corner of the world, I am > just curious if there are other aspects we should be thinking about. > > > > Thanks in advance, > > Anthony > > > > > > > > On 8/22/24, 7:24 AM, "Google Peering Operations No-reply" < > peering-ops-noreply(a)google.com <mailto:peering-ops-noreply@google.com>> > wrote: > > > > Dear Peer, > > > > This is a reminder that peering with Google will no longer be available at > > MICE Internet Exchange from 16th of September. > > > > As a result of unsustainable overheads associated with connecting to an IX > > using remote waves, we have made a decision to stop connecting to IXs that > > way. MICE IX falls under the category of IXs we remotely connect to. > > > > We will withdraw prefixes on 16th of September, and disconnect from the IX > > on 24th of September. > > > > We realize this may cause inconvenience for some peers. If there is another > > mutually present IX that we can peer in, we will be happy to help set up a > > bilateral session there - no multilaterals. If there is no mutual IX, > > traffic may have to be exchanged via indirect paths (transit links). > > > > We are not looking to connect to any additional IXs to replace the IX that > > we are leaving, but if you are considering connecting to another IX and not > > sure if Google will also be leaving that IX because of this measure, please > > reach out to us and we can share more. > > > > Thank you for your understanding. > > > > > > Sincerely, > > Google Network Operations > > > > > > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > -- =============================================== David Farmer Email:farmer@umn.edu Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 ===============================================

Aug. 23, 2024

Re: Peering with Google no longer available at MICE IX

by Jeff Wilde

I agree also. [A picture containing icon Description automatically generated] Jeff Wilde • Network Engineer • Park Region Office 218.826.6161 • Direct 218.826.8330 • Fax 218.826.6298 From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Ben Wiechman Sent: Friday, August 23, 2024 8:58 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Peering with Google no longer available at MICE IX I concur, as opposed to each member paying for transport back to Chicago or other POP to haul their own traffic. That largely defeats the purpose of the exchange to a large degree. Ben Wiechman Director of Network Strategy and Engineering 320.247.3224 | ben.wiechman(a)arvig.com<mailto:ben.wiechman@arvig.com> Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com<http://arvig.com> On Thu, Aug 22, 2024 at 5:39 PM Aaron Wendel <aaron(a)wholesaleinternet.net<mailto:aaron@wholesaleinternet.net>> wrote: For some historical context: A few years ago Google decided to pull their infrastructure out of Minneapolis, Kansas City and Saint Louis. The dominant exchanges in each market were given a choice between back-hauled peering connections or cache boxes. Minneapolis worked a deal for the back-hauled connection to Chicago. Kansas City and Saint Louis opted for cache servers. The main reason for the cache server choice was the concern over the long term sustainability and scalability of a remote peer of Google's size. Over the subsequent years, Google has shipped more and more servers as demand has increased. They have been very accommodating with the equipment. One other benefit to the cache servers is that they were not affected by Google's recent disconnection from ALL route servers. Google bears the cost for hardware upkeep. KCIX and STLIX use donated resources for cache fill, space and power. I understand MICE does not enjoy the resources that KCIX (and to a lesser extent STLIX) has access to but if Google is still shipping new cache servers (they may not be) that seems like a much better choice than remote peering to Chicago. Aaron ------ Original Message ------ From "Justin Krejci" <JKrejci(a)USINTERNET.COM<mailto:JKrejci@USINTERNET.COM>> To MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Date 8/22/2024 5:25:08 PM Subject Re: [MICE-DISCUSS] Peering with Google no longer available at MICE IX I am not sure what Google's stance is on deploying cache servers at an IX vs remote connection like the ones pending disconnection. If they were to deploy caching nodes at an IX, I assume they would still have a cache-fill requirement in terms of connectivity that has to come from somewhere... of which they appear to be disinterested in maintaining long distance connections going forward. So perhaps this is a donation opportunity for some to entice Google to deploy caching nodes at MICE? Donate local colo and transit bandwidth along with local MICE connectivity? We have numerous Google caching nodes in our network as well as multiple PNIs to Google, so their pending disconnect from MICE will likely have minimal direct impact on our eyeballs. -----Original Message----- From: Aaron Wendel <aaron(a)WHOLESALEINTERNET.NET<mailto:Aaron%20Wendel%20%3caaron@WHOLESALEINTERNET.NET%3e>> Reply-To: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE%20Discuss%20%3cMICE-DISCUSS@LISTS.IPHOUSE.NET%3e>> To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Peering with Google no longer available at MICE IX Date: Thu, 22 Aug 2024 22:03:01 +0000 Has MICE requested a cache box from Google? ------ Original Message ------ From "Anthony Anderberg" <AnthonyAnderberg(a)NUVERA.NET<mailto:AnthonyAnderberg@NUVERA.NET>> To MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Date 8/22/2024 4:28:18 PM Subject Re: [MICE-DISCUSS] Peering with Google no longer available at MICE IX I've been surprised that folks haven't been talking about this more and am curious about everyone's thoughts - as MICE user-members but also in terms of MICE leadership and direction. As a user-member I feel somewhat conflicted: on one hand in an effort to maximize performance I could obtain transport to Chicago and connect to Google there, but I feel like doing so is making Google's problem into my problem and that doing so undercuts the goals of MICE. We've seen content providers move MICE's larger members toward local direct private peering, and I can understand rational behind that even if it shifts traffic away from MICE - but Google is proposing remote direct peering at member's cost which feels different. Of course nobody expects Google to subsidize our corner of the world, I am just curious if there are other aspects we should be thinking about. Thanks in advance, Anthony On 8/22/24, 7:24 AM, "Google Peering Operations No-reply" <peering-ops-noreply(a)google.com<mailto:peering-ops-noreply@google.com> <mailto:peering-ops-noreply@google.com<mailto:peering-ops-noreply@google.com>>> wrote: Dear Peer, This is a reminder that peering with Google will no longer be available at MICE Internet Exchange from 16th of September. As a result of unsustainable overheads associated with connecting to an IX using remote waves, we have made a decision to stop connecting to IXs that way. MICE IX falls under the category of IXs we remotely connect to. We will withdraw prefixes on 16th of September, and disconnect from the IX on 24th of September. We realize this may cause inconvenience for some peers. If there is another mutually present IX that we can peer in, we will be happy to help set up a bilateral session there - no multilaterals. If there is no mutual IX, traffic may have to be exchanged via indirect paths (transit links). We are not looking to connect to any additional IXs to replace the IX that we are leaving, but if you are considering connecting to another IX and not sure if Google will also be leaving that IX because of this measure, please reach out to us and we can share more. Thank you for your understanding. Sincerely, Google Network Operations ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Aug. 23, 2024

Re: [EXTERNAL] - Re: [MICE-DISCUSS] Peering with Google no longer available at MICE IX

by Miles McCredie

Interesting. I’d be interested in any/all of average/P95/P99/Peak traffic over the last month on the Google ports but understand it’s not really any of my business. Is there a trend line over the last quarter or two? I have heard second hand that MICE ran into some unpleasantness around cache back in the day but I, for one, think that it would be worthwhile exploring the option of hosting caches again. (Would be even better for Google to build in to Minneapolis so that we can all turn up more local PNIs but they haven’t been interested in my views on this. 😊) As someone pointed out up thread, there are a handful of issues to sort out in this area. [http://images.midcocomm.com/ES_MidcoLogo.png] Miles McCredie Principal Network Engineer II-Core IP Office: 6052755192 Miles.McCredie(a)midco.com Midco.com Let's go beyond. From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Jeremy Lumby Sent: Friday, August 23, 2024 8:40 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] [EXTERNAL] - Re: [MICE-DISCUSS] Peering with Google no longer available at MICE IX I would be willing to bet if you looked just at the eyeball networks on MICE, 90% are getting the majority of their Google traffic across MICE, even if they have caches they are getting the fill across MICE in addition to the non cacheable traffic. Google's caches are not nearly as efficient as many of their competitors. I can confirm with over 10 years of experience dealing with Google that they are almost always very slow to upgrade. Jeremy Lumby Minnesota VoIP 9217 17th Ave S Suite 216 Bloomington MN 55425 Main 612-355-7740 Direct 612-392-6814 EFax 952-873-7425 jlumby(a)mnvoip.com<mailto:jlumby@mnvoip.com> -------- Original message -------- From: Miles McCredie <Miles.McCredie(a)MIDCO.COM<mailto:Miles.McCredie@MIDCO.COM>> Date: 8/23/24 6:34 AM (GMT-06:00) To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] [EXTERNAL] - Re: [MICE-DISCUSS] Peering with Google no longer available at MICE IX Curious how many MICE members are in each of the following buckets. 1. Receive limited portion of Google traffic via MICE Midco and, I assume, US Internet are in this bucket 1. Receive a significant portion of Google traffic via MICE today I’ve only got a couple years of experience with Google for Midco but my experience has been that Google will not always provide the cache capacity that I think is appropriate. The last two augments were triggered 9-12 months later than I would have liked. Could be my fault for creating new GNLs but… [http://images.midcocomm.com/ES_MidcoLogo.png] Miles McCredie Principal Network Engineer II-Core IP Office: 6052755192 Miles.McCredie(a)midco.com<mailto:Miles.McCredie@midco.com> Midco.com Let's go beyond. From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET>> On Behalf Of Justin Krejci Sent: Thursday, August 22, 2024 5:25 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: [EXTERNAL] - Re: [MICE-DISCUSS] Peering with Google no longer available at MICE IX CAUTION: This email originated from outside of MIDCO. Do not click links or open attachments unless you recognize the sender and know the content is safe. I am not sure what Google's stance is on deploying cache servers at an IX vs remote connection like the ones pending disconnection. If they were to deploy caching nodes at an IX, I assume they would still have a cache-fill requirement in terms of connectivity that has to come from somewhere... of which they appear to be disinterested in maintaining long distance connections going forward. So perhaps this is a donation opportunity for some to entice Google to deploy caching nodes at MICE? Donate local colo and transit bandwidth along with local MICE connectivity? We have numerous Google caching nodes in our network as well as multiple PNIs to Google, so their pending disconnect from MICE will likely have minimal direct impact on our eyeballs. -----Original Message----- From: Aaron Wendel <aaron(a)WHOLESALEINTERNET.NET<mailto:Aaron%20Wendel%20%3caaron@WHOLESALEINTERNET.NET%3e>> Reply-To: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE%20Discuss%20%3cMICE-DISCUSS@LISTS.IPHOUSE.NET%3e>> To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Peering with Google no longer available at MICE IX Date: Thu, 22 Aug 2024 22:03:01 +0000 Has MICE requested a cache box from Google? ------ Original Message ------ From "Anthony Anderberg" <AnthonyAnderberg(a)NUVERA.NET<mailto:AnthonyAnderberg@NUVERA.NET>> To MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Date 8/22/2024 4:28:18 PM Subject Re: [MICE-DISCUSS] Peering with Google no longer available at MICE IX I've been surprised that folks haven't been talking about this more and am curious about everyone's thoughts - as MICE user-members but also in terms of MICE leadership and direction. As a user-member I feel somewhat conflicted: on one hand in an effort to maximize performance I could obtain transport to Chicago and connect to Google there, but I feel like doing so is making Google's problem into my problem and that doing so undercuts the goals of MICE. We've seen content providers move MICE's larger members toward local direct private peering, and I can understand rational behind that even if it shifts traffic away from MICE - but Google is proposing remote direct peering at member's cost which feels different. Of course nobody expects Google to subsidize our corner of the world, I am just curious if there are other aspects we should be thinking about. Thanks in advance, Anthony On 8/22/24, 7:24 AM, "Google Peering Operations No-reply" <peering-ops-noreply(a)google.com<mailto:peering-ops-noreply@google.com> <mailto:peering-ops-noreply@google.com<mailto:peering-ops-noreply@google.com>>> wrote: Dear Peer, This is a reminder that peering with Google will no longer be available at MICE Internet Exchange from 16th of September. As a result of unsustainable overheads associated with connecting to an IX using remote waves, we have made a decision to stop connecting to IXs that way. MICE IX falls under the category of IXs we remotely connect to. We will withdraw prefixes on 16th of September, and disconnect from the IX on 24th of September. We realize this may cause inconvenience for some peers. If there is another mutually present IX that we can peer in, we will be happy to help set up a bilateral session there - no multilaterals. If there is no mutual IX, traffic may have to be exchanged via indirect paths (transit links). We are not looking to connect to any additional IXs to replace the IX that we are leaving, but if you are considering connecting to another IX and not sure if Google will also be leaving that IX because of this measure, please reach out to us and we can share more. Thank you for your understanding. Sincerely, Google Network Operations ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Aug. 23, 2024

Re: Peering with Google no longer available at MICE IX

by Ben Wiechman

I concur, as opposed to each member paying for transport back to Chicago or other POP to haul their own traffic. That largely defeats the purpose of the exchange to a large degree. Ben Wiechman Director of Network Strategy and Engineering 320.247.3224 | ben.wiechman(a)arvig.com Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com On Thu, Aug 22, 2024 at 5:39 PM Aaron Wendel <aaron(a)wholesaleinternet.net> wrote: > For some historical context: > > A few years ago Google decided to pull their infrastructure out of > Minneapolis, Kansas City and Saint Louis. The dominant exchanges in each > market were given a choice between back-hauled peering connections or cache > boxes. > > Minneapolis worked a deal for the back-hauled connection to Chicago. > Kansas City and Saint Louis opted for cache servers. > > The main reason for the cache server choice was the concern over the long > term sustainability and scalability of a remote peer of Google's size. > > Over the subsequent years, Google has shipped more and more servers as > demand has increased. They have been very accommodating with the > equipment. > > One other benefit to the cache servers is that they were not affected by > Google's recent disconnection from ALL route servers. > > Google bears the cost for hardware upkeep. KCIX and STLIX use donated > resources for cache fill, space and power. > > I understand MICE does not enjoy the resources that KCIX (and to a lesser > extent STLIX) has access to but if Google is still shipping new cache > servers (they may not be) that seems like a much better choice than remote > peering to Chicago. > > Aaron > > > > > > ------ Original Message ------ > From "Justin Krejci" <JKrejci(a)USINTERNET.COM> > To MICE-DISCUSS(a)LISTS.IPHOUSE.NET > Date 8/22/2024 5:25:08 PM > Subject Re: [MICE-DISCUSS] Peering with Google no longer available at MICE > IX > > I am not sure what Google's stance is on deploying cache servers at an IX > vs remote connection like the ones pending disconnection. > If they were to deploy caching nodes at an IX, I assume they would still > have a cache-fill requirement in terms of connectivity that has to come > from somewhere... of which they appear to be disinterested in maintaining > long distance connections going forward. So perhaps this is a donation > opportunity for some to entice Google to deploy caching nodes at MICE? > Donate local colo and transit bandwidth along with local MICE connectivity? > > We have numerous Google caching nodes in our network as well as multiple > PNIs to Google, so their pending disconnect from MICE will likely have > minimal direct impact on our eyeballs. > > > -----Original Message----- > *From*: Aaron Wendel <aaron(a)WHOLESALEINTERNET.NET > <Aaron%20Wendel%20%3caaron(a)WHOLESALEINTERNET.NET%3e>> > *Reply-To*: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET > <MICE%20Discuss%20%3cMICE-DISCUSS(a)LISTS.IPHOUSE.NET%3e>> > *To*: MICE-DISCUSS(a)LISTS.IPHOUSE.NET > *Subject*: Re: [MICE-DISCUSS] Peering with Google no longer available at > MICE IX > *Date*: Thu, 22 Aug 2024 22:03:01 +0000 > > Has MICE requested a cache box from Google? > > > > ------ Original Message ------ > From "Anthony Anderberg" <AnthonyAnderberg(a)NUVERA.NET> > To MICE-DISCUSS(a)LISTS.IPHOUSE.NET > Date 8/22/2024 4:28:18 PM > Subject Re: [MICE-DISCUSS] Peering with Google no longer available at > MICE IX > > I've been surprised that folks haven't been talking about this more and am > curious about everyone's thoughts - as MICE user-members but also in terms > of MICE leadership and direction. > > As a user-member I feel somewhat conflicted: on one hand in an effort to > maximize performance I could obtain transport to Chicago and connect to > Google there, but I feel like doing so is making Google's problem into my > problem and that doing so undercuts the goals of MICE. We've seen content > providers move MICE's larger members toward local direct private peering, > and I can understand rational behind that even if it shifts traffic away > from MICE - but Google is proposing remote direct peering at member's cost > which feels different. > > Of course nobody expects Google to subsidize our corner of the world, I am > just curious if there are other aspects we should be thinking about. > > Thanks in advance, > Anthony > > > > On 8/22/24, 7:24 AM, "Google Peering Operations No-reply" < > peering-ops-noreply(a)google.com <mailto:peering-ops-noreply@google.com>> > wrote: > > Dear Peer, > > This is a reminder that peering with Google will no longer be available at > MICE Internet Exchange from 16th of September. > > As a result of unsustainable overheads associated with connecting to an IX > using remote waves, we have made a decision to stop connecting to IXs that > way. MICE IX falls under the category of IXs we remotely connect to. > > We will withdraw prefixes on 16th of September, and disconnect from the IX > on 24th of September. > > We realize this may cause inconvenience for some peers. If there is another > mutually present IX that we can peer in, we will be happy to help set up a > bilateral session there - no multilaterals. If there is no mutual IX, > traffic may have to be exchanged via indirect paths (transit links). > > We are not looking to connect to any additional IXs to replace the IX that > we are leaving, but if you are considering connecting to another IX and not > sure if Google will also be leaving that IX because of this measure, please > reach out to us and we can share more. > > Thank you for your understanding. > > > Sincerely, > Google Network Operations > > > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

Aug. 23, 2024

Re: [EXTERNAL] - Re: [MICE-DISCUSS] Peering with Google no longer available at MICE IX

by Jeremy Lumby

I would be willing to bet if you looked just at the eyeball networks on MICE, 90% are getting the majority of their Google traffic across MICE, even if they have caches they are getting the fill across MICE in addition to the non cacheable traffic. Google's caches are not nearly as efficient as many of their competitors.I can confirm with over 10 years of experience dealing with Google that they are almost always very slow to upgrade.Jeremy LumbyMinnesota VoIP9217 17th Ave SSuite 216Bloomington MN 55425Main 612-355-7740Direct 612-392-6814EFax 952-873-7425jlumby(a)mnvoip.com -------- Original message --------From: Miles McCredie <Miles.McCredie(a)MIDCO.COM> Date: 8/23/24 6:34 AM (GMT-06:00) To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] [EXTERNAL] - Re: [MICE-DISCUSS] Peering with Google no longer available at MICE IX Curious how many MICE members are in each of the following buckets. Receive limited portion of Google traffic via MICE Midco and, I assume, US Internet are in this bucket Receive a significant portion of Google traffic via MICE today I’ve only got a couple years of experience with Google for Midco but my experience has been that Google will not always provide the cache capacity that I think is appropriate. The last two augments were triggered 9-12 months later than I would have liked. Could be my fault for creating new GNLs but… Miles McCredie Principal Network Engineer II-Core IP Office: 6052755192 Miles.McCredie(a)midco.com Midco.com Let's go beyond. From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Justin Krejci Sent: Thursday, August 22, 2024 5:25 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [EXTERNAL] - Re: [MICE-DISCUSS] Peering with Google no longer available at MICE IX CAUTION: This email originated from outside of MIDCO. Do not click links or open attachments unless you recognize the sender and know the content is safe. I am not sure what Google's stance is on deploying cache servers at an IX vs remote connection like the ones pending disconnection. If they were to deploy caching nodes at an IX, I assume they would still have a cache-fill requirement in terms of connectivity that has to come from somewhere... of which they appear to be disinterested in maintaining long distance connections going forward. So perhaps this is a donation opportunity for some to entice Google to deploy caching nodes at MICE? Donate local colo and transit bandwidth along with local MICE connectivity? We have numerous Google caching nodes in our network as well as multiple PNIs to Google, so their pending disconnect from MICE will likely have minimal direct impact on our eyeballs. -----Original Message----- From: Aaron Wendel <aaron(a)WHOLESALEINTERNET.NET> Reply-To: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Peering with Google no longer available at MICE IX Date: Thu, 22 Aug 2024 22:03:01 +0000 Has MICE requested a cache box from Google? ------ Original Message ------ From "Anthony Anderberg" <AnthonyAnderberg(a)NUVERA.NET> To MICE-DISCUSS(a)LISTS.IPHOUSE.NET Date 8/22/2024 4:28:18 PM Subject Re: [MICE-DISCUSS] Peering with Google no longer available at MICE IX I've been surprised that folks haven't been talking about this more and am curious about everyone's thoughts - as MICE user-members but also in terms of MICE leadership and direction. As a user-member I feel somewhat conflicted: on one hand in an effort to maximize performance I could obtain transport to Chicago and connect to Google there, but I feel like doing so is making Google's problem into my problem and that doing so undercuts the goals of MICE. We've seen content providers move MICE's larger members toward local direct private peering, and I can understand rational behind that even if it shifts traffic away from MICE - but Google is proposing remote direct peering at member's cost which feels different. Of course nobody expects Google to subsidize our corner of the world, I am just curious if there are other aspects we should be thinking about. Thanks in advance, Anthony On 8/22/24, 7:24 AM, "Google Peering Operations No-reply" <peering-ops-noreply(a)google.com <mailto:peering-ops-noreply@google.com>> wrote: Dear Peer, This is a reminder that peering with Google will no longer be available at MICE Internet Exchange from 16th of September. As a result of unsustainable overheads associated with connecting to an IX using remote waves, we have made a decision to stop connecting to IXs that way. MICE IX falls under the category of IXs we remotely connect to. We will withdraw prefixes on 16th of September, and disconnect from the IX on 24th of September. We realize this may cause inconvenience for some peers. If there is another mutually present IX that we can peer in, we will be happy to help set up a bilateral session there - no multilaterals. If there is no mutual IX, traffic may have to be exchanged via indirect paths (transit links). We are not looking to connect to any additional IXs to replace the IX that we are leaving, but if you are considering connecting to another IX and not sure if Google will also be leaving that IX because of this measure, please reach out to us and we can share more. Thank you for your understanding. Sincerely, Google Network Operations To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Aug. 23, 2024

Re: [EXTERNAL] - Re: [MICE-DISCUSS] Peering with Google no longer available at MICE IX

by Miles McCredie

Curious how many MICE members are in each of the following buckets. 1. Receive limited portion of Google traffic via MICE Midco and, I assume, US Internet are in this bucket 1. Receive a significant portion of Google traffic via MICE today I’ve only got a couple years of experience with Google for Midco but my experience has been that Google will not always provide the cache capacity that I think is appropriate. The last two augments were triggered 9-12 months later than I would have liked. Could be my fault for creating new GNLs but… [http://images.midcocomm.com/ES_MidcoLogo.png] Miles McCredie Principal Network Engineer II-Core IP Office: 6052755192 Miles.McCredie(a)midco.com Midco.com Let's go beyond. From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Justin Krejci Sent: Thursday, August 22, 2024 5:25 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [EXTERNAL] - Re: [MICE-DISCUSS] Peering with Google no longer available at MICE IX CAUTION: This email originated from outside of MIDCO. Do not click links or open attachments unless you recognize the sender and know the content is safe. I am not sure what Google's stance is on deploying cache servers at an IX vs remote connection like the ones pending disconnection. If they were to deploy caching nodes at an IX, I assume they would still have a cache-fill requirement in terms of connectivity that has to come from somewhere... of which they appear to be disinterested in maintaining long distance connections going forward. So perhaps this is a donation opportunity for some to entice Google to deploy caching nodes at MICE? Donate local colo and transit bandwidth along with local MICE connectivity? We have numerous Google caching nodes in our network as well as multiple PNIs to Google, so their pending disconnect from MICE will likely have minimal direct impact on our eyeballs. -----Original Message----- From: Aaron Wendel <aaron(a)WHOLESALEINTERNET.NET<mailto:Aaron%20Wendel%20%3caaron@WHOLESALEINTERNET.NET%3e>> Reply-To: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE%20Discuss%20%3cMICE-DISCUSS@LISTS.IPHOUSE.NET%3e>> To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Peering with Google no longer available at MICE IX Date: Thu, 22 Aug 2024 22:03:01 +0000 Has MICE requested a cache box from Google? ------ Original Message ------ From "Anthony Anderberg" <AnthonyAnderberg(a)NUVERA.NET<mailto:AnthonyAnderberg@NUVERA.NET>> To MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Date 8/22/2024 4:28:18 PM Subject Re: [MICE-DISCUSS] Peering with Google no longer available at MICE IX I've been surprised that folks haven't been talking about this more and am curious about everyone's thoughts - as MICE user-members but also in terms of MICE leadership and direction. As a user-member I feel somewhat conflicted: on one hand in an effort to maximize performance I could obtain transport to Chicago and connect to Google there, but I feel like doing so is making Google's problem into my problem and that doing so undercuts the goals of MICE. We've seen content providers move MICE's larger members toward local direct private peering, and I can understand rational behind that even if it shifts traffic away from MICE - but Google is proposing remote direct peering at member's cost which feels different. Of course nobody expects Google to subsidize our corner of the world, I am just curious if there are other aspects we should be thinking about. Thanks in advance, Anthony On 8/22/24, 7:24 AM, "Google Peering Operations No-reply" <peering-ops-noreply(a)google.com<mailto:peering-ops-noreply@google.com> <mailto:peering-ops-noreply@google.com<mailto:peering-ops-noreply@google.com>>> wrote: Dear Peer, This is a reminder that peering with Google will no longer be available at MICE Internet Exchange from 16th of September. As a result of unsustainable overheads associated with connecting to an IX using remote waves, we have made a decision to stop connecting to IXs that way. MICE IX falls under the category of IXs we remotely connect to. We will withdraw prefixes on 16th of September, and disconnect from the IX on 24th of September. We realize this may cause inconvenience for some peers. If there is another mutually present IX that we can peer in, we will be happy to help set up a bilateral session there - no multilaterals. If there is no mutual IX, traffic may have to be exchanged via indirect paths (transit links). We are not looking to connect to any additional IXs to replace the IX that we are leaving, but if you are considering connecting to another IX and not sure if Google will also be leaving that IX because of this measure, please reach out to us and we can share more. Thank you for your understanding. Sincerely, Google Network Operations ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Aug. 23, 2024

RFP: 511 IP Transit

by Richard Laager

MICE is in need of IP transit at 511. This is for our management connection. We are looking for IPv4 + IPv6 transit delivered over a 1G copper (if in the same suite), 1G optical, or 10G optical. MICE will take care of the cross-connect. No BGP necessary. We need a /29 of IPv4 space and a /60 of IPv6 space. The current IP space is routed to us over a separate point-to-point (e.g. the /29 is over a /31 point-to-point). Our actual bandwidth usage is very minimal; 100 Mbps is more than plenty. Please specify the speed you are offering. I don't know that we have a specific term length in mind. Month-to-month, 1 year, or 3 year is probably fine. Please specify the term length(s) you are offering. You can reply off-list to Anthony Anderberg <AnthonyAnderberg(a)nuvera.net>. He intends NOT to bid on this. Conflict of interest note: My employer will bid on this. I will recuse from the board vote on it. -- Richard

Aug. 23, 2024

Re: Peering with Google no longer available at MICE IX

by Aaron Wendel

For some historical context: A few years ago Google decided to pull their infrastructure out of Minneapolis, Kansas City and Saint Louis. The dominant exchanges in each market were given a choice between back-hauled peering connections or cache boxes. Minneapolis worked a deal for the back-hauled connection to Chicago. Kansas City and Saint Louis opted for cache servers. The main reason for the cache server choice was the concern over the long term sustainability and scalability of a remote peer of Google's size. Over the subsequent years, Google has shipped more and more servers as demand has increased. They have been very accommodating with the equipment. One other benefit to the cache servers is that they were not affected by Google's recent disconnection from ALL route servers. Google bears the cost for hardware upkeep. KCIX and STLIX use donated resources for cache fill, space and power. I understand MICE does not enjoy the resources that KCIX (and to a lesser extent STLIX) has access to but if Google is still shipping new cache servers (they may not be) that seems like a much better choice than remote peering to Chicago. Aaron ------ Original Message ------ >From "Justin Krejci" <JKrejci(a)USINTERNET.COM> To MICE-DISCUSS(a)LISTS.IPHOUSE.NET Date 8/22/2024 5:25:08 PM Subject Re: [MICE-DISCUSS] Peering with Google no longer available at MICE IX >I am not sure what Google's stance is on deploying cache servers at an >IX vs remote connection like the ones pending disconnection. >If they were to deploy caching nodes at an IX, I assume they would >still have a cache-fill requirement in terms of connectivity that has >to come from somewhere... of which they appear to be disinterested in >maintaining long distance connections going forward. So perhaps this is >a donation opportunity for some to entice Google to deploy caching >nodes at MICE? Donate local colo and transit bandwidth along with local >MICE connectivity? > >We have numerous Google caching nodes in our network as well as >multiple PNIs to Google, so their pending disconnect from MICE will >likely have minimal direct impact on our eyeballs. > > >-----Original Message----- >From: Aaron Wendel <aaron(a)WHOLESALEINTERNET.NET ><mailto:Aaron%20Wendel%20%3caaron@WHOLESALEINTERNET.NET%3e>> >Reply-To: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET ><mailto:MICE%20Discuss%20%3cMICE-DISCUSS@LISTS.IPHOUSE.NET%3e>> >To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET >Subject: Re: [MICE-DISCUSS] Peering with Google no longer available at >MICE IX >Date: Thu, 22 Aug 2024 22:03:01 +0000 > >Has MICE requested a cache box from Google? > > > >------ Original Message ------ >From "Anthony Anderberg" <AnthonyAnderberg(a)NUVERA.NET> >To MICE-DISCUSS(a)LISTS.IPHOUSE.NET >Date 8/22/2024 4:28:18 PM >Subject Re: [MICE-DISCUSS] Peering with Google no longer available at >MICE IX > >>I've been surprised that folks haven't been talking about this more >>and am curious about everyone's thoughts - as MICE user-members but >>also in terms of MICE leadership and direction. >> >>As a user-member I feel somewhat conflicted: on one hand in an effort >>to maximize performance I could obtain transport to Chicago and >>connect to Google there, but I feel like doing so is making Google's >>problem into my problem and that doing so undercuts the goals of MICE. >> We've seen content providers move MICE's larger members toward local >>direct private peering, and I can understand rational behind that even >>if it shifts traffic away from MICE - but Google is proposing remote >>direct peering at member's cost which feels different. >> >>Of course nobody expects Google to subsidize our corner of the world, >>I am just curious if there are other aspects we should be thinking >>about. >> >>Thanks in advance, >>Anthony >> >> >> >>On 8/22/24, 7:24 AM, "Google Peering Operations No-reply" >><peering-ops-noreply(a)google.com >><mailto:peering-ops-noreply@google.com>> wrote: >> >>Dear Peer, >> >>This is a reminder that peering with Google will no longer be >>available at >>MICE Internet Exchange from 16th of September. >> >>As a result of unsustainable overheads associated with connecting to >>an IX >>using remote waves, we have made a decision to stop connecting to IXs >>that >>way. MICE IX falls under the category of IXs we remotely connect to. >> >>We will withdraw prefixes on 16th of September, and disconnect from >>the IX >>on 24th of September. >> >>We realize this may cause inconvenience for some peers. If there is >>another >>mutually present IX that we can peer in, we will be happy to help set >>up a >>bilateral session there - no multilaterals. If there is no mutual IX, >>traffic may have to be exchanged via indirect paths (transit links). >> >>We are not looking to connect to any additional IXs to replace the IX >>that >>we are leaving, but if you are considering connecting to another IX >>and not >>sure if Google will also be leaving that IX because of this measure, >>please >>reach out to us and we can share more. >> >>Thank you for your understanding. >> >> >>Sincerely, >>Google Network Operations >> >> -------------------------------------------------------------------------------- To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Aug. 22, 2024

Re: Peering with Google no longer available at MICE IX

by Justin Krejci

I am not sure what Google's stance is on deploying cache servers at an IX vs remote connection like the ones pending disconnection. If they were to deploy caching nodes at an IX, I assume they would still have a cache-fill requirement in terms of connectivity that has to come from somewhere... of which they appear to be disinterested in maintaining long distance connections going forward. So perhaps this is a donation opportunity for some to entice Google to deploy caching nodes at MICE? Donate local colo and transit bandwidth along with local MICE connectivity? We have numerous Google caching nodes in our network as well as multiple PNIs to Google, so their pending disconnect from MICE will likely have minimal direct impact on our eyeballs. -----Original Message----- From: Aaron Wendel <aaron(a)WHOLESALEINTERNET.NET<mailto:Aaron%20Wendel%20%3caaron@WHOLESALEINTERNET.NET%3e>> Reply-To: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE%20Discuss%20%3cMICE-DISCUSS@LISTS.IPHOUSE.NET%3e>> To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Peering with Google no longer available at MICE IX Date: Thu, 22 Aug 2024 22:03:01 +0000 Has MICE requested a cache box from Google? ------ Original Message ------ From "Anthony Anderberg" <AnthonyAnderberg(a)NUVERA.NET<mailto:AnthonyAnderberg@NUVERA.NET>> To MICE-DISCUSS(a)LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Date 8/22/2024 4:28:18 PM Subject Re: [MICE-DISCUSS] Peering with Google no longer available at MICE IX I've been surprised that folks haven't been talking about this more and am curious about everyone's thoughts - as MICE user-members but also in terms of MICE leadership and direction. As a user-member I feel somewhat conflicted: on one hand in an effort to maximize performance I could obtain transport to Chicago and connect to Google there, but I feel like doing so is making Google's problem into my problem and that doing so undercuts the goals of MICE. We've seen content providers move MICE's larger members toward local direct private peering, and I can understand rational behind that even if it shifts traffic away from MICE - but Google is proposing remote direct peering at member's cost which feels different. Of course nobody expects Google to subsidize our corner of the world, I am just curious if there are other aspects we should be thinking about. Thanks in advance, Anthony On 8/22/24, 7:24 AM, "Google Peering Operations No-reply" <peering-ops-noreply(a)google.com<mailto:peering-ops-noreply@google.com> <mailto:peering-ops-noreply@google.com<mailto:peering-ops-noreply@google.com>>> wrote: Dear Peer, This is a reminder that peering with Google will no longer be available at MICE Internet Exchange from 16th of September. As a result of unsustainable overheads associated with connecting to an IX using remote waves, we have made a decision to stop connecting to IXs that way. MICE IX falls under the category of IXs we remotely connect to. We will withdraw prefixes on 16th of September, and disconnect from the IX on 24th of September. We realize this may cause inconvenience for some peers. If there is another mutually present IX that we can peer in, we will be happy to help set up a bilateral session there - no multilaterals. If there is no mutual IX, traffic may have to be exchanged via indirect paths (transit links). We are not looking to connect to any additional IXs to replace the IX that we are leaving, but if you are considering connecting to another IX and not sure if Google will also be leaving that IX because of this measure, please reach out to us and we can share more. Thank you for your understanding. Sincerely, Google Network Operations

Aug. 22, 2024

Re: Peering with Google no longer available at MICE IX

by Jeremy Lumby

It has been discussed in the past, however Google wanted MICE to cover all of the costs related to it. -----Original Message----- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Aaron Wendel Sent: Thursday, August 22, 2024 5:03 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Peering with Google no longer available at MICE IX Has MICE requested a cache box from Google? ------ Original Message ------ >From "Anthony Anderberg" <AnthonyAnderberg(a)NUVERA.NET> To MICE-DISCUSS(a)LISTS.IPHOUSE.NET Date 8/22/2024 4:28:18 PM Subject Re: [MICE-DISCUSS] Peering with Google no longer available at MICE IX >I've been surprised that folks haven't been talking about this more and am curious about everyone's thoughts - as MICE user-members but also in terms of MICE leadership and direction. > >As a user-member I feel somewhat conflicted: on one hand in an effort to maximize performance I could obtain transport to Chicago and connect to Google there, but I feel like doing so is making Google's problem into my problem and that doing so undercuts the goals of MICE. We've seen content providers move MICE's larger members toward local direct private peering, and I can understand rational behind that even if it shifts traffic away from MICE - but Google is proposing remote direct peering at member's cost which feels different. > >Of course nobody expects Google to subsidize our corner of the world, I am just curious if there are other aspects we should be thinking about. > >Thanks in advance, >Anthony > > > >On 8/22/24, 7:24 AM, "Google Peering Operations No-reply" <peering-ops-noreply(a)google.com <mailto:peering-ops-noreply@google.com>> wrote: > >Dear Peer, > >This is a reminder that peering with Google will no longer be available at >MICE Internet Exchange from 16th of September. > >As a result of unsustainable overheads associated with connecting to an IX >using remote waves, we have made a decision to stop connecting to IXs that >way. MICE IX falls under the category of IXs we remotely connect to. > >We will withdraw prefixes on 16th of September, and disconnect from the IX >on 24th of September. > >We realize this may cause inconvenience for some peers. If there is another >mutually present IX that we can peer in, we will be happy to help set up a >bilateral session there - no multilaterals. If there is no mutual IX, >traffic may have to be exchanged via indirect paths (transit links). > >We are not looking to connect to any additional IXs to replace the IX that >we are leaving, but if you are considering connecting to another IX and not >sure if Google will also be leaving that IX because of this measure, please >reach out to us and we can share more. > >Thank you for your understanding. > > >Sincerely, >Google Network Operations > >

Aug. 22, 2024