August 2019 - mice-discuss - lists.micemn.net

Re: Saturated ports

by Jay Hanke

> If someone's port is full on a regular basis, having someone from the Board contact the member and ask if they are aware of the issue seems approprate and reasonable. Basically asking, "have you noticed your port is full quite a bit lately?" But its the member's responsibility to manage their network as they see fit, including their ports to the exchange. I'd tweak this a little to encourage the member reach out directly to the other peer whenever there might be a problem. If you need help getting attention of a non-responsive peer we can help with some "peer pressure". :) -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

Aug. 22, 2019

Re: Saturated ports

by David Farmer

On Wed, Aug 21, 2019 at 1:59 PM Jay Hanke <jayhanke(a)southfront.io> wrote: > > To be clear, I was not advocating for punitive action, and we can go and > pull the bandwidth reports periodically. I was just posing the question > regarding whether a more proactive stance regarding visibility would be > appropriate. > > What would you propose? > First, let's not make a federal case of this, links fill up it happens, get over it. In this case it occurred right after big changes by a big content provider. I'm actually surprised more participants' links didn't fill up. Second, you should send a polite note to any another member, if you notice anything that isn't quite right, their link is full, they are leaking routes, or something just looks weird. We are a community, talk to your neighbors, be their early warning system. Hopefully, they already know about what ever it is, but we are all busy and things slip through the cracks all the time. So if you notice something that doesn't seem right, let your neighbors know about it. But be polite about it, next time it might be you who has something weird happening. Third, the IX is an optimization, if you are having issues sending traffic to another member you have other routes, use them. We have had to do this, not because the other member ignored us or wouldn't upgrade their link, but sometimes upgrades can take a little time. A few weeks or even a couple months to complete an upgrade isn't that unrealistic and not uncommon depending on the circumstances. Not everyone has extra 10G or 100G ports just sitting around, and if you have to backhaul the traffic, arranging that upgrade can easily take several months. The policy we have for remote switches makes sense, but members need to manage their own capacity. Even if we were to forced them to add port capacity to the exchange the bottleneck could just move to the next hop. None of us has sufficient visibility into the other guys network or business to be making those kinds of judgements for them. If someone's port is full on a regular basis, having someone from the Board contact the member and ask if they are aware of the issue seems approprate and reasonable. Basically asking, "have you noticed your port is full quite a bit lately?" But its the member's responsibility to manage their network as they see fit, including their ports to the exchange. Thanks -- =============================================== David Farmer Email:farmer@umn.edu Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 ===============================================

Aug. 22, 2019

Re: Saturated ports

by Corey Hauer

Indeed I did! 😮 On Wed, Aug 21, 2019 at 2:00 PM Matthew Beckwell <mbeckwell(a)evolveip.net> wrote: > I feel like we’re mixing up “SDN” and “DCN” here? > > > > > > *From: *MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> on behalf of Corey > Hauer <coreyhauer(a)GMAIL.COM> > *Reply-To: *MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> > *Date: *Wednesday, August 21, 2019 at 3:10 PM > *To: *"MICE-DISCUSS(a)LISTS.IPHOUSE.NET" <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> > *Subject: *Re: [MICE-DISCUSS] Saturated ports > > > > Honestly, as an eyeball network, I would fix saturating that pipe that by > filtering prefixes or adding capacity. My customers would vote with their > feet if we started having packet loss or adverse latency because we filled > our IX facing pipes. I can't imagine SDN and their members choosing to > endure thousands of customer complaints. > > > > SDN is a well run, well capitalized network - they'll fix it one way or > another. I am sure once this lands on the right person's desk - it gets > fixed. > > > > > > On Wed, Aug 21, 2019 at 12:17 PM Ben Wiechman <ben.wiechman(a)arvig.com> > wrote: > > Is it possible to send a weekly/monthly reporting showing a high water > mark for utilization for each member? Is that appropriate? Or do we just > ask each member to monitor the utilization graphs if this is of interest to > them? > > > > For my part it is easier to ingest and handle a periodic review if we are > pushed the information versus having to have someone remember to go and > pull the information. I'm interested in what others would find useful. > > > > *Ben Wiechman* > > Director of IP Strategy and Engineering > > 320.247.3224 | ben.wiechman(a)arvig.com > > Arvig | 224 East Main Street | Melrose, MN 56352 > <https://www.google.com/maps/search/224+East+Main+Street+%7C+Melrose,+MN+563…> > | arvig.com > > > > > > > > On Wed, Aug 21, 2019 at 1:59 PM Jay Hanke <jayhanke(a)southfront.io> wrote: > > > To be clear, I was not advocating for punitive action, and we can go and > pull the bandwidth reports periodically. I was just posing the question > regarding whether a more proactive stance regarding visibility would be > appropriate. > > What would you propose? > > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

Aug. 21, 2019

Re: Saturated ports

by Darin Steffl

My original thread is about DCN, not SDN. On Wed, Aug 21, 2019, 4:00 PM Matthew Beckwell <mbeckwell(a)evolveip.net> wrote: > I feel like we’re mixing up “SDN” and “DCN” here? > > > > > > *From: *MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> on behalf of Corey > Hauer <coreyhauer(a)GMAIL.COM> > *Reply-To: *MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> > *Date: *Wednesday, August 21, 2019 at 3:10 PM > *To: *"MICE-DISCUSS(a)LISTS.IPHOUSE.NET" <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> > *Subject: *Re: [MICE-DISCUSS] Saturated ports > > > > Honestly, as an eyeball network, I would fix saturating that pipe that by > filtering prefixes or adding capacity. My customers would vote with their > feet if we started having packet loss or adverse latency because we filled > our IX facing pipes. I can't imagine SDN and their members choosing to > endure thousands of customer complaints. > > > > SDN is a well run, well capitalized network - they'll fix it one way or > another. I am sure once this lands on the right person's desk - it gets > fixed. > > > > > > On Wed, Aug 21, 2019 at 12:17 PM Ben Wiechman <ben.wiechman(a)arvig.com> > wrote: > > Is it possible to send a weekly/monthly reporting showing a high water > mark for utilization for each member? Is that appropriate? Or do we just > ask each member to monitor the utilization graphs if this is of interest to > them? > > > > For my part it is easier to ingest and handle a periodic review if we are > pushed the information versus having to have someone remember to go and > pull the information. I'm interested in what others would find useful. > > > > *Ben Wiechman* > > Director of IP Strategy and Engineering > > 320.247.3224 | ben.wiechman(a)arvig.com > > Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com > > > > > > > > On Wed, Aug 21, 2019 at 1:59 PM Jay Hanke <jayhanke(a)southfront.io> wrote: > > > To be clear, I was not advocating for punitive action, and we can go and > pull the bandwidth reports periodically. I was just posing the question > regarding whether a more proactive stance regarding visibility would be > appropriate. > > What would you propose? > > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

Aug. 21, 2019

Re: Saturated ports

by AnthonyAnderberg＠nuvera.net

I think there is some confusion – the original thread was talking about DCN, not SDN. SDN is a long-time MICE member and hasn’t ever had any issues that I recall. From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> on behalf of Gary Glissendorf <Gary.Glissendorf(a)SDNCOMMUNICATIONS.COM> Reply-To: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> Date: Wednesday, August 21, 2019 at 3:45 PM To: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Saturated ports I noticed this thread today but wasn’t sure what topic started the thread. SDN like Corey said our customers report latency, packet loss quickly if we have saturation. We monitor and alert on network saturation at 60% and if needed we adjust prefix advertisements until we can find the proper balance and then increase bandwidth when needed. We have not developed a cool automatic was to do this yet but we rarely have issues with Peaks or DDoS Attacks. It’s the occasional fiber cuts that can cause us issues and the need to adjust bandwidth. What was topic that started the Thread about BW and reports. Gary Glissendorf | Network Architect gary.glissendorf(a)sdncommunications.com 2900 W. 10th St. | Sioux Falls, SD 57104 (w) 605.978.3558 | (c) 605.359-3737 | (tf) 800.247.1442 SDN NOC 877.287.8023 NOC Support email: sdnsupport(a)sdncommunications.com “Ausgezeichnet Zueinander Sein” [cid:image001.jpg@01D55838.820116E0] <http://www.facebook.com/sdncommunications> [cid:image002.jpg@01D55838.820116E0] <http://twitter.com/sdncomm> [cid:image003.jpg@01D55838.820116E0] <http://plus.google.com/107192460129695945716/posts> [cid:image004.jpg@01D55838.820116E0] <http://www.linkedin.com/company/sdn-communications> [cid:image005.jpg@01D55838.820116E0] <http://www.youtube.com/user/sdncomm> From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Corey Hauer Sent: Wednesday, August 21, 2019 3:10 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Saturated ports Honestly, as an eyeball network, I would fix saturating that pipe that by filtering prefixes or adding capacity. My customers would vote with their feet if we started having packet loss or adverse latency because we filled our IX facing pipes. I can't imagine SDN and their members choosing to endure thousands of customer complaints. SDN is a well run, well capitalized network - they'll fix it one way or another. I am sure once this lands on the right person's desk - it gets fixed. On Wed, Aug 21, 2019 at 12:17 PM Ben Wiechman <ben.wiechman(a)arvig.com<mailto:ben.wiechman@arvig.com>> wrote: Is it possible to send a weekly/monthly reporting showing a high water mark for utilization for each member? Is that appropriate? Or do we just ask each member to monitor the utilization graphs if this is of interest to them? For my part it is easier to ingest and handle a periodic review if we are pushed the information versus having to have someone remember to go and pull the information. I'm interested in what others would find useful. Ben Wiechman Director of IP Strategy and Engineering 320.247.3224 | ben.wiechman(a)arvig.com<mailto:ben.wiechman@arvig.com> Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com<http://arvig.com> On Wed, Aug 21, 2019 at 1:59 PM Jay Hanke <jayhanke(a)southfront.io<mailto:jayhanke@southfront.io>> wrote: > To be clear, I was not advocating for punitive action, and we can go and pull the bandwidth reports periodically. I was just posing the question regarding whether a more proactive stance regarding visibility would be appropriate. What would you propose? ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ***This message and any attachments are solely for the intended recipient. If you are not the intended recipient, disclosure, copying, use or distribution of the information included in this message is prohibited -- Please immediately and permanently delete.*** ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Aug. 21, 2019

Re: Saturated ports

by Matthew Beckwell

I feel like we’re mixing up “SDN” and “DCN” here? From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> on behalf of Corey Hauer <coreyhauer(a)GMAIL.COM> Reply-To: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> Date: Wednesday, August 21, 2019 at 3:10 PM To: "MICE-DISCUSS(a)LISTS.IPHOUSE.NET" <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Saturated ports Honestly, as an eyeball network, I would fix saturating that pipe that by filtering prefixes or adding capacity. My customers would vote with their feet if we started having packet loss or adverse latency because we filled our IX facing pipes. I can't imagine SDN and their members choosing to endure thousands of customer complaints. SDN is a well run, well capitalized network - they'll fix it one way or another. I am sure once this lands on the right person's desk - it gets fixed. On Wed, Aug 21, 2019 at 12:17 PM Ben Wiechman <ben.wiechman(a)arvig.com<mailto:ben.wiechman@arvig.com>> wrote: Is it possible to send a weekly/monthly reporting showing a high water mark for utilization for each member? Is that appropriate? Or do we just ask each member to monitor the utilization graphs if this is of interest to them? For my part it is easier to ingest and handle a periodic review if we are pushed the information versus having to have someone remember to go and pull the information. I'm interested in what others would find useful. Ben Wiechman Director of IP Strategy and Engineering 320.247.3224 | ben.wiechman(a)arvig.com<mailto:ben.wiechman@arvig.com> Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com<http://arvig.com> On Wed, Aug 21, 2019 at 1:59 PM Jay Hanke <jayhanke(a)southfront.io<mailto:jayhanke@southfront.io>> wrote: > To be clear, I was not advocating for punitive action, and we can go and pull the bandwidth reports periodically. I was just posing the question regarding whether a more proactive stance regarding visibility would be appropriate. What would you propose? ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Aug. 21, 2019

Re: Saturated ports

by Gary Glissendorf

I noticed this thread today but wasn’t sure what topic started the thread. SDN like Corey said our customers report latency, packet loss quickly if we have saturation. We monitor and alert on network saturation at 60% and if needed we adjust prefix advertisements until we can find the proper balance and then increase bandwidth when needed. We have not developed a cool automatic was to do this yet but we rarely have issues with Peaks or DDoS Attacks. It’s the occasional fiber cuts that can cause us issues and the need to adjust bandwidth. What was topic that started the Thread about BW and reports. Gary Glissendorf | Network Architect gary.glissendorf(a)sdncommunications.com 2900 W. 10th St. | Sioux Falls, SD 57104 (w) 605.978.3558 | (c) 605.359-3737 | (tf) 800.247.1442 SDN NOC 877.287.8023 NOC Support email: sdnsupport(a)sdncommunications.com “Ausgezeichnet Zueinander Sein” [cid:image001.jpg@01D55837.7362FB90] <http://www.facebook.com/sdncommunications> [cid:image002.jpg@01D55837.7362FB90] <http://twitter.com/sdncomm> [cid:image003.jpg@01D55837.7362FB90] <http://plus.google.com/107192460129695945716/posts> [cid:image004.jpg@01D55837.7362FB90] <http://www.linkedin.com/company/sdn-communications> [cid:image005.jpg@01D55837.7362FB90] <http://www.youtube.com/user/sdncomm> From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Corey Hauer Sent: Wednesday, August 21, 2019 3:10 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Saturated ports Honestly, as an eyeball network, I would fix saturating that pipe that by filtering prefixes or adding capacity. My customers would vote with their feet if we started having packet loss or adverse latency because we filled our IX facing pipes. I can't imagine SDN and their members choosing to endure thousands of customer complaints. SDN is a well run, well capitalized network - they'll fix it one way or another. I am sure once this lands on the right person's desk - it gets fixed. On Wed, Aug 21, 2019 at 12:17 PM Ben Wiechman <ben.wiechman(a)arvig.com<mailto:ben.wiechman@arvig.com>> wrote: Is it possible to send a weekly/monthly reporting showing a high water mark for utilization for each member? Is that appropriate? Or do we just ask each member to monitor the utilization graphs if this is of interest to them? For my part it is easier to ingest and handle a periodic review if we are pushed the information versus having to have someone remember to go and pull the information. I'm interested in what others would find useful. Ben Wiechman Director of IP Strategy and Engineering 320.247.3224 | ben.wiechman(a)arvig.com<mailto:ben.wiechman@arvig.com> Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com<http://arvig.com> On Wed, Aug 21, 2019 at 1:59 PM Jay Hanke <jayhanke(a)southfront.io<mailto:jayhanke@southfront.io>> wrote: > To be clear, I was not advocating for punitive action, and we can go and pull the bandwidth reports periodically. I was just posing the question regarding whether a more proactive stance regarding visibility would be appropriate. What would you propose? ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ***This message and any attachments are solely for the intended recipient. If you are not the intended recipient, disclosure, copying, use or distribution of the information included in this message is prohibited -- Please immediately and permanently delete.***

Aug. 21, 2019

Re: Saturated ports

by Corey Hauer

Honestly, as an eyeball network, I would fix saturating that pipe that by filtering prefixes or adding capacity. My customers would vote with their feet if we started having packet loss or adverse latency because we filled our IX facing pipes. I can't imagine SDN and their members choosing to endure thousands of customer complaints. SDN is a well run, well capitalized network - they'll fix it one way or another. I am sure once this lands on the right person's desk - it gets fixed. On Wed, Aug 21, 2019 at 12:17 PM Ben Wiechman <ben.wiechman(a)arvig.com> wrote: > Is it possible to send a weekly/monthly reporting showing a high water > mark for utilization for each member? Is that appropriate? Or do we just > ask each member to monitor the utilization graphs if this is of interest to > them? > > For my part it is easier to ingest and handle a periodic review if we are > pushed the information versus having to have someone remember to go and > pull the information. I'm interested in what others would find useful. > > Ben Wiechman > > Director of IP Strategy and Engineering > > 320.247.3224 | ben.wiechman(a)arvig.com > > Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com > > > > > On Wed, Aug 21, 2019 at 1:59 PM Jay Hanke <jayhanke(a)southfront.io> wrote: > >> > To be clear, I was not advocating for punitive action, and we can go >> and pull the bandwidth reports periodically. I was just posing the question >> regarding whether a more proactive stance regarding visibility would be >> appropriate. >> >> What would you propose? >> > > ------------------------------ > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

Aug. 21, 2019

Re: Saturated ports

by Andy Koch

On 8/21/19 2:17 PM, Ben Wiechman wrote: > Is it possible to send a weekly/monthly reporting showing a high water > mark for utilization for each member? Is that appropriate? Or do we > just ask each member to monitor the utilization graphs if this is of > interest to them? > > For my part it is easier to ingest and handle a periodic review if we > are pushed the information versus having to have someone remember to > go and pull the information. I'm interested in what others would find > useful. I don't mind if this is an opt-in item on a per-participant basis, but I am not in-favor of having yet more to my inbox. I feel that any good participant should have their own monitoring and be properly alerted based on that - possibly directly into a capacity planning workflow. If you want to have a periodic email, I am sure a curl request to the MICE monitoring server on a cron would be fairly simple to setup if you have cleared it with US Internet who graciously hosts that for us. If the recommendation is for a mass-email to the mailing list with all participant data, that I am even less in support of. We have several participants that do not publicly share their capacity data and we should respect that - not blast it out weekly/monthly, even if the recipient list is opt-in. Regards, Andy Andy Koch Hoyos Consulting LLC ofc: +1 608 616 9950 andyk(a)hoyosconsulting.com http://www.hoyosconsulting.com

Aug. 21, 2019

Re: Saturated ports

by Ben Wiechman

Is it possible to send a weekly/monthly reporting showing a high water mark for utilization for each member? Is that appropriate? Or do we just ask each member to monitor the utilization graphs if this is of interest to them? For my part it is easier to ingest and handle a periodic review if we are pushed the information versus having to have someone remember to go and pull the information. I'm interested in what others would find useful. Ben Wiechman Director of IP Strategy and Engineering 320.247.3224 | ben.wiechman(a)arvig.com Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com On Wed, Aug 21, 2019 at 1:59 PM Jay Hanke <jayhanke(a)southfront.io> wrote: > > To be clear, I was not advocating for punitive action, and we can go and > pull the bandwidth reports periodically. I was just posing the question > regarding whether a more proactive stance regarding visibility would be > appropriate. > > What would you propose? >

Aug. 21, 2019