Our levels of network traffic were lower than a typical Thursday Night Football event so the overall bandwidth utilization was not a factor.

We had similar reports from our employees that the experience was hit and miss on many different service providers: Arvig, various employees are on other service providers, and mobile. One employee reported a better experience connecting to a VPN that terminated in TX than using any local connections. We also noted that while the fight would not load, other Netflix content tended to load and playback without issues or the same delays. On most tries I was unable to get video content for the fight to play at all, but attempts to then load any other Netflix content popped up in what I was describe as fairly typical response times.

Traffic to our caches initially spiked, then crashed to nothing, then recovered at relatively heavy levels. We weren't seeing evidence of network congestion.

My opinion is that, beyond that initial surge of traffic to the caches, the issue was less network or cache capacity than it was an ability of the client to access the resources that redirected it to the correct CDN for the content.

The only possible workaround we discussed was shutting down the cache or access to the cache, but that was not a direction we chose to go down. Untested and not ideal given that there were no guarantees that Netflix would redirect subscribers to another CDN gracefully, or that another CDN would have any more additional capacity.

Keep in mind that Netflix will be hosting a couple of Christmas NFL football games. A quick Google search suggests that typical Thanksgiving games run around 30M viewers, so half+ the load that this event generated, but still something to consider.

Ben Wiechman

Director of Network Strategy and Engineering

320.247.3224 | ben.wiechman@arvig.com

Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com

On Mon, Nov 18, 2024 at 1:46 PM Joe Unger <joe@vaultas.com> wrote:

I was streaming at home from my Comcast/Xfinity coax connection , and had a terrible experience for most of the last 3 fights to the point it was completely unwatchable at times. I was texting with friends and family from all over the midwest and everyone I spoke with experienced buffering, freezing and outages. From the small amount of data I have it appears the cable providers suffered the most (Comcast, Spectrum & Cox).

I haven't seen or heard of any definitive choke points as of yet. I thought it might have been a perfect storm of; Netflix cloud infrastructure issues and network congestion issues.

I saw reports of 60 million+ users streaming the fight.

It's clear things broke, and now there are elements to fix and make better.
On Fri, Nov 15, 2024 at 11:02 PM Miles McCredie <Miles.McCredie@midco.com> wrote:
If there are folks interested in collaborating on a post-mortem, count me in.

We’re getting complaints. Haven’t identified any obvious on-net issues (yet?).

I’m actually remote and streaming on Comcast which has been mostly OK with a couple “stream not available” errors.

Roughly 3x normal Netflix peaks.

Miles McCredie
Principal Network Engineer II-Core IP

Office: 6052755192
Miles.McCredie@midco.com

Midco.com
Let's go beyond.

From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> On Behalf Of Richard Laager
Sent: Friday, November 15, 2024 10:31 PM
To: MICE-DISCUSS@LISTS.IPHOUSE.NET
Subject: [EXTERNAL] - Re: [MICE-DISCUSS] netflix/tyson

CAUTION: This email originated from outside of MIDCO.
Do not click links or open attachments unless you recognize the sender and know the content is safe.
I'm seeing issues depending on which Netflix cache I hit. It fails to show the live event when the client picks mine (which I know to be serving large amounts of data, but well within its capacity, and serves other content just fine), or that of one of my upstreams. But other caches work fine.
-- 
Richard
To unsubscribe from the MICE-DISCUSS list, click the following link:
http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
To unsubscribe from the MICE-DISCUSS list, click the following link:
http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
--
Joe Unger
612.709.8699
joe@vaultas.com
Vaultas | Connecting you to the Ecosystem
vaultas.com

To unsubscribe from the MICE-DISCUSS list, click the following link:
http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1