Hey guys, Is there any policy in place for peers that let their ports saturate at 100% for an extended period of time? DCN looks like they could use an upgrade to their 10G port and not sure if anyone proactively reaches out to members when saturation occurs. Thanks http://micelg.usinternet.com/cacti/graph.php?rra_id=all&local_graph_id=582 -- Darin Steffl Minnesota WiFi www.mnwifi.com 507-634-WiFi <http://www.facebook.com/minnesotawifi> Like us on Facebook <http://www.facebook.com/minnesotawifi>
On 8/20/19 8:48 PM, Darin Steffl wrote:
Is there any policy in place for peers that let their ports saturate at 100% for an extended period of time? DCN looks like they could use an upgrade to their 10G port and not sure if anyone proactively reaches out to members when saturation occurs.
I've forwarded your message to DCN. For remote switch ports, we have a policy of requiring upgrades before saturation. For regular participants, I'm not sure that we have a policy. I'm also not sure if we want a policy there, as that might be considered dictating peering policy. I'm not personally opposed, but this is something that would need some thought. -- Richard
I know in the past several of us have reached out privately to folks at DCN about their port saturation, which seems to come and go. We do not have a policy about member-port saturation, and my recollection is similar to Richard's - we didn't want to be bossy about people's peering. Cheers, anthony On 8/20/19, 9:05 PM, "Richard Laager" <rlaager@WIKTEL.COM> wrote: On 8/20/19 8:48 PM, Darin Steffl wrote: > Is there any policy in place for peers that let their ports saturate at > 100% for an extended period of time? DCN looks like they could use an > upgrade to their 10G port and not sure if anyone proactively reaches out > to members when saturation occurs. I've forwarded your message to DCN. For remote switch ports, we have a policy of requiring upgrades before saturation. For regular participants, I'm not sure that we have a policy. I'm also not sure if we want a policy there, as that might be considered dictating peering policy. I'm not personally opposed, but this is something that would need some thought. -- Richard
The board talked about this back in the day. The thought process was that remotes affect multiple members so the congestion policy should be enforced. For a participant it's only their network (and their customers). Also, a disconnect might make the overall internet worse as their transit may fill. On Tue, Aug 20, 2019, 11:57 PM AnthonyAnderberg@nuvera.net < AnthonyAnderberg@nuvera.net> wrote:
I know in the past several of us have reached out privately to folks at DCN about their port saturation, which seems to come and go.
We do not have a policy about member-port saturation, and my recollection is similar to Richard's - we didn't want to be bossy about people's peering.
Cheers, anthony
On 8/20/19, 9:05 PM, "Richard Laager" <rlaager@WIKTEL.COM> wrote:
On 8/20/19 8:48 PM, Darin Steffl wrote: > Is there any policy in place for peers that let their ports saturate at > 100% for an extended period of time? DCN looks like they could use an > upgrade to their 10G port and not sure if anyone proactively reaches out > to members when saturation occurs.
I've forwarded your message to DCN.
For remote switch ports, we have a policy of requiring upgrades before saturation.
For regular participants, I'm not sure that we have a policy.
I'm also not sure if we want a policy there, as that might be considered dictating peering policy. I'm not personally opposed, but this is something that would need some thought.
-- Richard
I would dispute the "only their network" statement. We've troubleshot various issues in the past with latency and loss, especially for internet based services used by businesses when that traffic traverses MICE. While the peering policy of a service provider is their decision, it is potentially appropriate to provide proactive notice in some fashion that would alert other operators that they may want to adjust their peering strategy to minimize customer complaints. Is some kind of alarming or reporting viable and useful? From my perspective it would be useful. Ben Wiechman Director of IP Strategy and Engineering 320.247.3224 | ben.wiechman@arvig.com Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com On Wed, Aug 21, 2019 at 7:39 AM Jay Hanke <jayhanke@southfront.io> wrote:
The board talked about this back in the day. The thought process was that remotes affect multiple members so the congestion policy should be enforced.
For a participant it's only their network (and their customers). Also, a disconnect might make the overall internet worse as their transit may fill.
On Tue, Aug 20, 2019, 11:57 PM AnthonyAnderberg@nuvera.net < AnthonyAnderberg@nuvera.net> wrote:
I know in the past several of us have reached out privately to folks at DCN about their port saturation, which seems to come and go.
We do not have a policy about member-port saturation, and my recollection is similar to Richard's - we didn't want to be bossy about people's peering.
Cheers, anthony
On 8/20/19, 9:05 PM, "Richard Laager" <rlaager@WIKTEL.COM> wrote:
On 8/20/19 8:48 PM, Darin Steffl wrote: > Is there any policy in place for peers that let their ports saturate at > 100% for an extended period of time? DCN looks like they could use an > upgrade to their 10G port and not sure if anyone proactively reaches out > to members when saturation occurs.
I've forwarded your message to DCN.
For remote switch ports, we have a policy of requiring upgrades before saturation.
For regular participants, I'm not sure that we have a policy.
I'm also not sure if we want a policy there, as that might be considered dictating peering policy. I'm not personally opposed, but this is something that would need some thought.
-- Richard
------------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
I agree this can affect multiple parties. Say we let our port get saturated. Now all the content providers start receiving trouble calls. Netflix, akamai, YouTube, traffic between us and any MICE peer is congested and our customers will probably blame us but they may also blame the other peer who they receive traffic from when it's not their fault. I don't think blocking a port is a good idea but I think proactive notifications is a good first step then more harshly worded emails if that doesn't work. We don't want to let anyone think running a port at 100% is a good idea. It puts a burden on more than just the member with a congested port in my opinion like Ben said. On Wed, Aug 21, 2019, 11:28 AM Ben Wiechman <ben.wiechman@arvig.com> wrote:
I would dispute the "only their network" statement. We've troubleshot various issues in the past with latency and loss, especially for internet based services used by businesses when that traffic traverses MICE. While the peering policy of a service provider is their decision, it is potentially appropriate to provide proactive notice in some fashion that would alert other operators that they may want to adjust their peering strategy to minimize customer complaints.
Is some kind of alarming or reporting viable and useful? From my perspective it would be useful.
Ben Wiechman
Director of IP Strategy and Engineering
320.247.3224 | ben.wiechman@arvig.com
Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com
On Wed, Aug 21, 2019 at 7:39 AM Jay Hanke <jayhanke@southfront.io> wrote:
The board talked about this back in the day. The thought process was that remotes affect multiple members so the congestion policy should be enforced.
For a participant it's only their network (and their customers). Also, a disconnect might make the overall internet worse as their transit may fill.
On Tue, Aug 20, 2019, 11:57 PM AnthonyAnderberg@nuvera.net < AnthonyAnderberg@nuvera.net> wrote:
I know in the past several of us have reached out privately to folks at DCN about their port saturation, which seems to come and go.
We do not have a policy about member-port saturation, and my recollection is similar to Richard's - we didn't want to be bossy about people's peering.
Cheers, anthony
On 8/20/19, 9:05 PM, "Richard Laager" <rlaager@WIKTEL.COM> wrote:
On 8/20/19 8:48 PM, Darin Steffl wrote: > Is there any policy in place for peers that let their ports saturate at > 100% for an extended period of time? DCN looks like they could use an > upgrade to their 10G port and not sure if anyone proactively reaches out > to members when saturation occurs.
I've forwarded your message to DCN.
For remote switch ports, we have a policy of requiring upgrades before saturation.
For regular participants, I'm not sure that we have a policy.
I'm also not sure if we want a policy there, as that might be considered dictating peering policy. I'm not personally opposed, but this is something that would need some thought.
-- Richard
------------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
------------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
I don't think blocking a port is a good idea but I think proactive notifications is a good first step then more harshly worded emails if that doesn't work.
We're already doing this. Just not automated. -- Jay Hanke, President South Front Networks jayhanke@southfront.io Phone 612-204-0000
To be clear, I was not advocating for punitive action, and we can go and pull the bandwidth reports periodically. I was just posing the question regarding whether a more proactive stance regarding visibility would be appropriate. Ben Wiechman Director of IP Strategy and Engineering 320.247.3224 | ben.wiechman@arvig.com Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com On Wed, Aug 21, 2019 at 11:55 AM Jay Hanke <jayhanke@southfront.io> wrote:
I don't think blocking a port is a good idea but I think proactive notifications is a good first step then more harshly worded emails if that doesn't work.
We're already doing this. Just not automated.
-- Jay Hanke, President South Front Networks jayhanke@southfront.io Phone 612-204-0000
Is it possible to send a weekly/monthly reporting showing a high water mark for utilization for each member? Is that appropriate? Or do we just ask each member to monitor the utilization graphs if this is of interest to them? For my part it is easier to ingest and handle a periodic review if we are pushed the information versus having to have someone remember to go and pull the information. I'm interested in what others would find useful. Ben Wiechman Director of IP Strategy and Engineering 320.247.3224 | ben.wiechman@arvig.com Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com On Wed, Aug 21, 2019 at 1:59 PM Jay Hanke <jayhanke@southfront.io> wrote:
To be clear, I was not advocating for punitive action, and we can go and pull the bandwidth reports periodically. I was just posing the question regarding whether a more proactive stance regarding visibility would be appropriate.
What would you propose?
On 8/21/19 2:17 PM, Ben Wiechman wrote:
Is it possible to send a weekly/monthly reporting showing a high water mark for utilization for each member? Is that appropriate? Or do we just ask each member to monitor the utilization graphs if this is of interest to them?
For my part it is easier to ingest and handle a periodic review if we are pushed the information versus having to have someone remember to go and pull the information. I'm interested in what others would find useful.
I don't mind if this is an opt-in item on a per-participant basis, but I am not in-favor of having yet more to my inbox. I feel that any good participant should have their own monitoring and be properly alerted based on that - possibly directly into a capacity planning workflow. If you want to have a periodic email, I am sure a curl request to the MICE monitoring server on a cron would be fairly simple to setup if you have cleared it with US Internet who graciously hosts that for us. If the recommendation is for a mass-email to the mailing list with all participant data, that I am even less in support of. We have several participants that do not publicly share their capacity data and we should respect that - not blast it out weekly/monthly, even if the recipient list is opt-in. Regards, Andy Andy Koch Hoyos Consulting LLC ofc: +1 608 616 9950 andyk@hoyosconsulting.com http://www.hoyosconsulting.com
Honestly, as an eyeball network, I would fix saturating that pipe that by filtering prefixes or adding capacity. My customers would vote with their feet if we started having packet loss or adverse latency because we filled our IX facing pipes. I can't imagine SDN and their members choosing to endure thousands of customer complaints. SDN is a well run, well capitalized network - they'll fix it one way or another. I am sure once this lands on the right person's desk - it gets fixed. On Wed, Aug 21, 2019 at 12:17 PM Ben Wiechman <ben.wiechman@arvig.com> wrote:
Is it possible to send a weekly/monthly reporting showing a high water mark for utilization for each member? Is that appropriate? Or do we just ask each member to monitor the utilization graphs if this is of interest to them?
For my part it is easier to ingest and handle a periodic review if we are pushed the information versus having to have someone remember to go and pull the information. I'm interested in what others would find useful.
Ben Wiechman
Director of IP Strategy and Engineering
320.247.3224 | ben.wiechman@arvig.com
Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com
On Wed, Aug 21, 2019 at 1:59 PM Jay Hanke <jayhanke@southfront.io> wrote:
To be clear, I was not advocating for punitive action, and we can go and pull the bandwidth reports periodically. I was just posing the question regarding whether a more proactive stance regarding visibility would be appropriate.
What would you propose?
------------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
I noticed this thread today but wasn’t sure what topic started the thread. SDN like Corey said our customers report latency, packet loss quickly if we have saturation. We monitor and alert on network saturation at 60% and if needed we adjust prefix advertisements until we can find the proper balance and then increase bandwidth when needed. We have not developed a cool automatic was to do this yet but we rarely have issues with Peaks or DDoS Attacks. It’s the occasional fiber cuts that can cause us issues and the need to adjust bandwidth. What was topic that started the Thread about BW and reports. Gary Glissendorf | Network Architect gary.glissendorf@sdncommunications.com 2900 W. 10th St. | Sioux Falls, SD 57104 (w) 605.978.3558 | (c) 605.359-3737 | (tf) 800.247.1442 SDN NOC 877.287.8023 NOC Support email: sdnsupport@sdncommunications.com “Ausgezeichnet Zueinander Sein” [cid:image001.jpg@01D55837.7362FB90] <http://www.facebook.com/sdncommunications> [cid:image002.jpg@01D55837.7362FB90] <http://twitter.com/sdncomm> [cid:image003.jpg@01D55837.7362FB90] <http://plus.google.com/107192460129695945716/posts> [cid:image004.jpg@01D55837.7362FB90] <http://www.linkedin.com/company/sdn-communications> [cid:image005.jpg@01D55837.7362FB90] <http://www.youtube.com/user/sdncomm> From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> On Behalf Of Corey Hauer Sent: Wednesday, August 21, 2019 3:10 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Saturated ports Honestly, as an eyeball network, I would fix saturating that pipe that by filtering prefixes or adding capacity. My customers would vote with their feet if we started having packet loss or adverse latency because we filled our IX facing pipes. I can't imagine SDN and their members choosing to endure thousands of customer complaints. SDN is a well run, well capitalized network - they'll fix it one way or another. I am sure once this lands on the right person's desk - it gets fixed. On Wed, Aug 21, 2019 at 12:17 PM Ben Wiechman <ben.wiechman@arvig.com<mailto:ben.wiechman@arvig.com>> wrote: Is it possible to send a weekly/monthly reporting showing a high water mark for utilization for each member? Is that appropriate? Or do we just ask each member to monitor the utilization graphs if this is of interest to them? For my part it is easier to ingest and handle a periodic review if we are pushed the information versus having to have someone remember to go and pull the information. I'm interested in what others would find useful. Ben Wiechman Director of IP Strategy and Engineering 320.247.3224 | ben.wiechman@arvig.com<mailto:ben.wiechman@arvig.com> Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com<http://arvig.com> On Wed, Aug 21, 2019 at 1:59 PM Jay Hanke <jayhanke@southfront.io<mailto:jayhanke@southfront.io>> wrote:
To be clear, I was not advocating for punitive action, and we can go and pull the bandwidth reports periodically. I was just posing the question regarding whether a more proactive stance regarding visibility would be appropriate.
What would you propose? ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ***This message and any attachments are solely for the intended recipient. If you are not the intended recipient, disclosure, copying, use or distribution of the information included in this message is prohibited -- Please immediately and permanently delete.***
I think there is some confusion – the original thread was talking about DCN, not SDN. SDN is a long-time MICE member and hasn’t ever had any issues that I recall. From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> on behalf of Gary Glissendorf <Gary.Glissendorf@SDNCOMMUNICATIONS.COM> Reply-To: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> Date: Wednesday, August 21, 2019 at 3:45 PM To: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Saturated ports I noticed this thread today but wasn’t sure what topic started the thread. SDN like Corey said our customers report latency, packet loss quickly if we have saturation. We monitor and alert on network saturation at 60% and if needed we adjust prefix advertisements until we can find the proper balance and then increase bandwidth when needed. We have not developed a cool automatic was to do this yet but we rarely have issues with Peaks or DDoS Attacks. It’s the occasional fiber cuts that can cause us issues and the need to adjust bandwidth. What was topic that started the Thread about BW and reports. Gary Glissendorf | Network Architect gary.glissendorf@sdncommunications.com 2900 W. 10th St. | Sioux Falls, SD 57104 (w) 605.978.3558 | (c) 605.359-3737 | (tf) 800.247.1442 SDN NOC 877.287.8023 NOC Support email: sdnsupport@sdncommunications.com “Ausgezeichnet Zueinander Sein” [cid:image001.jpg@01D55838.820116E0] <http://www.facebook.com/sdncommunications> [cid:image002.jpg@01D55838.820116E0] <http://twitter.com/sdncomm> [cid:image003.jpg@01D55838.820116E0] <http://plus.google.com/107192460129695945716/posts> [cid:image004.jpg@01D55838.820116E0] <http://www.linkedin.com/company/sdn-communications> [cid:image005.jpg@01D55838.820116E0] <http://www.youtube.com/user/sdncomm> From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> On Behalf Of Corey Hauer Sent: Wednesday, August 21, 2019 3:10 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Saturated ports Honestly, as an eyeball network, I would fix saturating that pipe that by filtering prefixes or adding capacity. My customers would vote with their feet if we started having packet loss or adverse latency because we filled our IX facing pipes. I can't imagine SDN and their members choosing to endure thousands of customer complaints. SDN is a well run, well capitalized network - they'll fix it one way or another. I am sure once this lands on the right person's desk - it gets fixed. On Wed, Aug 21, 2019 at 12:17 PM Ben Wiechman <ben.wiechman@arvig.com<mailto:ben.wiechman@arvig.com>> wrote: Is it possible to send a weekly/monthly reporting showing a high water mark for utilization for each member? Is that appropriate? Or do we just ask each member to monitor the utilization graphs if this is of interest to them? For my part it is easier to ingest and handle a periodic review if we are pushed the information versus having to have someone remember to go and pull the information. I'm interested in what others would find useful. Ben Wiechman Director of IP Strategy and Engineering 320.247.3224 | ben.wiechman@arvig.com<mailto:ben.wiechman@arvig.com> Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com<http://arvig.com> On Wed, Aug 21, 2019 at 1:59 PM Jay Hanke <jayhanke@southfront.io<mailto:jayhanke@southfront.io>> wrote:
To be clear, I was not advocating for punitive action, and we can go and pull the bandwidth reports periodically. I was just posing the question regarding whether a more proactive stance regarding visibility would be appropriate.
What would you propose? ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ***This message and any attachments are solely for the intended recipient. If you are not the intended recipient, disclosure, copying, use or distribution of the information included in this message is prohibited -- Please immediately and permanently delete.*** ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
I feel like we’re mixing up “SDN” and “DCN” here? From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> on behalf of Corey Hauer <coreyhauer@GMAIL.COM> Reply-To: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> Date: Wednesday, August 21, 2019 at 3:10 PM To: "MICE-DISCUSS@LISTS.IPHOUSE.NET" <MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: Re: [MICE-DISCUSS] Saturated ports Honestly, as an eyeball network, I would fix saturating that pipe that by filtering prefixes or adding capacity. My customers would vote with their feet if we started having packet loss or adverse latency because we filled our IX facing pipes. I can't imagine SDN and their members choosing to endure thousands of customer complaints. SDN is a well run, well capitalized network - they'll fix it one way or another. I am sure once this lands on the right person's desk - it gets fixed. On Wed, Aug 21, 2019 at 12:17 PM Ben Wiechman <ben.wiechman@arvig.com<mailto:ben.wiechman@arvig.com>> wrote: Is it possible to send a weekly/monthly reporting showing a high water mark for utilization for each member? Is that appropriate? Or do we just ask each member to monitor the utilization graphs if this is of interest to them? For my part it is easier to ingest and handle a periodic review if we are pushed the information versus having to have someone remember to go and pull the information. I'm interested in what others would find useful. Ben Wiechman Director of IP Strategy and Engineering 320.247.3224 | ben.wiechman@arvig.com<mailto:ben.wiechman@arvig.com> Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com<http://arvig.com> On Wed, Aug 21, 2019 at 1:59 PM Jay Hanke <jayhanke@southfront.io<mailto:jayhanke@southfront.io>> wrote:
To be clear, I was not advocating for punitive action, and we can go and pull the bandwidth reports periodically. I was just posing the question regarding whether a more proactive stance regarding visibility would be appropriate.
What would you propose? ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
My original thread is about DCN, not SDN. On Wed, Aug 21, 2019, 4:00 PM Matthew Beckwell <mbeckwell@evolveip.net> wrote:
I feel like we’re mixing up “SDN” and “DCN” here?
*From: *MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> on behalf of Corey Hauer <coreyhauer@GMAIL.COM> *Reply-To: *MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> *Date: *Wednesday, August 21, 2019 at 3:10 PM *To: *"MICE-DISCUSS@LISTS.IPHOUSE.NET" <MICE-DISCUSS@LISTS.IPHOUSE.NET> *Subject: *Re: [MICE-DISCUSS] Saturated ports
Honestly, as an eyeball network, I would fix saturating that pipe that by filtering prefixes or adding capacity. My customers would vote with their feet if we started having packet loss or adverse latency because we filled our IX facing pipes. I can't imagine SDN and their members choosing to endure thousands of customer complaints.
SDN is a well run, well capitalized network - they'll fix it one way or another. I am sure once this lands on the right person's desk - it gets fixed.
On Wed, Aug 21, 2019 at 12:17 PM Ben Wiechman <ben.wiechman@arvig.com> wrote:
Is it possible to send a weekly/monthly reporting showing a high water mark for utilization for each member? Is that appropriate? Or do we just ask each member to monitor the utilization graphs if this is of interest to them?
For my part it is easier to ingest and handle a periodic review if we are pushed the information versus having to have someone remember to go and pull the information. I'm interested in what others would find useful.
*Ben Wiechman*
Director of IP Strategy and Engineering
320.247.3224 | ben.wiechman@arvig.com
Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com
On Wed, Aug 21, 2019 at 1:59 PM Jay Hanke <jayhanke@southfront.io> wrote:
To be clear, I was not advocating for punitive action, and we can go and pull the bandwidth reports periodically. I was just posing the question regarding whether a more proactive stance regarding visibility would be appropriate.
What would you propose?
------------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
------------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
------------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
Indeed I did! 😮 On Wed, Aug 21, 2019 at 2:00 PM Matthew Beckwell <mbeckwell@evolveip.net> wrote:
I feel like we’re mixing up “SDN” and “DCN” here?
*From: *MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> on behalf of Corey Hauer <coreyhauer@GMAIL.COM> *Reply-To: *MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> *Date: *Wednesday, August 21, 2019 at 3:10 PM *To: *"MICE-DISCUSS@LISTS.IPHOUSE.NET" <MICE-DISCUSS@LISTS.IPHOUSE.NET> *Subject: *Re: [MICE-DISCUSS] Saturated ports
Honestly, as an eyeball network, I would fix saturating that pipe that by filtering prefixes or adding capacity. My customers would vote with their feet if we started having packet loss or adverse latency because we filled our IX facing pipes. I can't imagine SDN and their members choosing to endure thousands of customer complaints.
SDN is a well run, well capitalized network - they'll fix it one way or another. I am sure once this lands on the right person's desk - it gets fixed.
On Wed, Aug 21, 2019 at 12:17 PM Ben Wiechman <ben.wiechman@arvig.com> wrote:
Is it possible to send a weekly/monthly reporting showing a high water mark for utilization for each member? Is that appropriate? Or do we just ask each member to monitor the utilization graphs if this is of interest to them?
For my part it is easier to ingest and handle a periodic review if we are pushed the information versus having to have someone remember to go and pull the information. I'm interested in what others would find useful.
*Ben Wiechman*
Director of IP Strategy and Engineering
320.247.3224 | ben.wiechman@arvig.com
Arvig | 224 East Main Street | Melrose, MN 56352 <https://www.google.com/maps/search/224+East+Main+Street+%7C+Melrose,+MN+56352?entry=gmail&source=g> | arvig.com
On Wed, Aug 21, 2019 at 1:59 PM Jay Hanke <jayhanke@southfront.io> wrote:
To be clear, I was not advocating for punitive action, and we can go and pull the bandwidth reports periodically. I was just posing the question regarding whether a more proactive stance regarding visibility would be appropriate.
What would you propose?
------------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
------------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
------------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
On Wed, Aug 21, 2019 at 1:59 PM Jay Hanke <jayhanke@southfront.io> wrote:
To be clear, I was not advocating for punitive action, and we can go and pull the bandwidth reports periodically. I was just posing the question regarding whether a more proactive stance regarding visibility would be appropriate.
What would you propose?
First, let's not make a federal case of this, links fill up it happens, get over it. In this case it occurred right after big changes by a big content provider. I'm actually surprised more participants' links didn't fill up. Second, you should send a polite note to any another member, if you notice anything that isn't quite right, their link is full, they are leaking routes, or something just looks weird. We are a community, talk to your neighbors, be their early warning system. Hopefully, they already know about what ever it is, but we are all busy and things slip through the cracks all the time. So if you notice something that doesn't seem right, let your neighbors know about it. But be polite about it, next time it might be you who has something weird happening. Third, the IX is an optimization, if you are having issues sending traffic to another member you have other routes, use them. We have had to do this, not because the other member ignored us or wouldn't upgrade their link, but sometimes upgrades can take a little time. A few weeks or even a couple months to complete an upgrade isn't that unrealistic and not uncommon depending on the circumstances. Not everyone has extra 10G or 100G ports just sitting around, and if you have to backhaul the traffic, arranging that upgrade can easily take several months. The policy we have for remote switches makes sense, but members need to manage their own capacity. Even if we were to forced them to add port capacity to the exchange the bottleneck could just move to the next hop. None of us has sufficient visibility into the other guys network or business to be making those kinds of judgements for them. If someone's port is full on a regular basis, having someone from the Board contact the member and ask if they are aware of the issue seems approprate and reasonable. Basically asking, "have you noticed your port is full quite a bit lately?" But its the member's responsibility to manage their network as they see fit, including their ports to the exchange. Thanks -- =============================================== David Farmer Email:farmer@umn.edu Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 ===============================================
If someone's port is full on a regular basis, having someone from the Board contact the member and ask if they are aware of the issue seems approprate and reasonable. Basically asking, "have you noticed your port is full quite a bit lately?" But its the member's responsibility to manage their network as they see fit, including their ports to the exchange.
I'd tweak this a little to encourage the member reach out directly to the other peer whenever there might be a problem. If you need help getting attention of a non-responsive peer we can help with some "peer pressure". :) -- Jay Hanke, President South Front Networks jayhanke@southfront.io Phone 612-204-0000
The graphing is public, there's currently no alerting. From the IX, perspective the only leverage is removal from the fabric or total depeering. Not a very good options. Members can always filter or preference their routes using the MICE communities or policies on their own gear. When we start tail dropping (or getting full) I reach out to the peer directly and make them aware of the situation and offer up MICE upgrade options. Sometimes multiple times. MICE is not advocating members congest their ports, it defeats the purpose of the IX. MICE is not disconnecting member connected congested ports. On Wed, Aug 21, 2019 at 11:28 AM Ben Wiechman <ben.wiechman@arvig.com> wrote:
I would dispute the "only their network" statement. We've troubleshot various issues in the past with latency and loss, especially for internet based services used by businesses when that traffic traverses MICE. While the peering policy of a service provider is their decision, it is potentially appropriate to provide proactive notice in some fashion that would alert other operators that they may want to adjust their peering strategy to minimize customer complaints.
Is some kind of alarming or reporting viable and useful? From my perspective it would be useful.
Ben Wiechman
Director of IP Strategy and Engineering
320.247.3224 | ben.wiechman@arvig.com
Arvig | 224 East Main Street | Melrose, MN 56352 | arvig.com
On Wed, Aug 21, 2019 at 7:39 AM Jay Hanke <jayhanke@southfront.io> wrote:
The board talked about this back in the day. The thought process was that remotes affect multiple members so the congestion policy should be enforced.
For a participant it's only their network (and their customers). Also, a disconnect might make the overall internet worse as their transit may fill.
On Tue, Aug 20, 2019, 11:57 PM AnthonyAnderberg@nuvera.net <AnthonyAnderberg@nuvera.net> wrote:
I know in the past several of us have reached out privately to folks at DCN about their port saturation, which seems to come and go.
We do not have a policy about member-port saturation, and my recollection is similar to Richard's - we didn't want to be bossy about people's peering.
Cheers, anthony
On 8/20/19, 9:05 PM, "Richard Laager" <rlaager@WIKTEL.COM> wrote:
On 8/20/19 8:48 PM, Darin Steffl wrote: > Is there any policy in place for peers that let their ports saturate at > 100% for an extended period of time? DCN looks like they could use an > upgrade to their 10G port and not sure if anyone proactively reaches out > to members when saturation occurs.
I've forwarded your message to DCN.
For remote switch ports, we have a policy of requiring upgrades before saturation.
For regular participants, I'm not sure that we have a policy.
I'm also not sure if we want a policy there, as that might be considered dictating peering policy. I'm not personally opposed, but this is something that would need some thought.
-- Richard
________________________________
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
________________________________
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
-- Jay Hanke, President South Front Networks jayhanke@southfront.io Phone 612-204-0000
participants (10)
-
Andy Koch -
AnthonyAnderberg＠nuvera.net -
Ben Wiechman -
Corey Hauer -
Darin Steffl -
David Farmer -
Gary Glissendorf -
Jay Hanke -
Matthew Beckwell -
Richard Laager