September 2016 - mice-discuss

Re: EX4200 (1G Switch) Packet Loss -> 10G Participants

by Jason Hanke

I'm starting to think that the new OEM switches are worth a look. Especially because we don't need a complex feature set. Mellanox EdgeCore etc. On Tue, Sep 20, 2016 at 10:29 AM, Andrew Hoyos <hoyosa(a)gmail.com> wrote: > > On Sep 20, 2016, at 10:18 AM, Mike Horwath <drechsau(a)GEEKS.ORG> wrote: > > > > On Tue, Sep 20, 2016 at 09:49:48AM -0500, Levi Pederson wrote: > > > >> A full chassis 8200 and 9200 would be a solid upgrade for all > >> pertinent points. > > > > I don't agree. > > Also don’t agree with the 8200/9200. Most line cards on the 8200 and 9200 > are oversub’d anyway, and the space/ports ratio isn’t great. > > > Cost per port is high compared to running with 4500/45550 setup and > > can stack 10 deep as it is. > > I don’t know if I necessarily agree here either. The VC will quickly > become a bottleneck if not already (32gb unidirectional traffic). > Also, no support for 40/100g. > > If money is being spent, something like an Arista 7504 or Juniper QFX > 5100/5200’s would make more sense here and allow for some growth not > entirely based on "more 10g ports". > > -- > Andrew Hoyos > hoyosa(a)gmail.com > -- Jay Hanke CTO Neutral Path Communications 3 Civic Center Plaza, Suite 204 Mankato, MN 56001 (507) 327-2398 mobile jayhanke(a)neutralpath.net www.neutralpath.net

Sept. 20, 2016

Re: EX4200 (1G Switch) Packet Loss -> 10G Participants

by Andrew Hoyos

> On Sep 20, 2016, at 10:18 AM, Mike Horwath <drechsau(a)GEEKS.ORG> wrote: > > On Tue, Sep 20, 2016 at 09:49:48AM -0500, Levi Pederson wrote: > >> A full chassis 8200 and 9200 would be a solid upgrade for all >> pertinent points. > > I don't agree. Also don’t agree with the 8200/9200. Most line cards on the 8200 and 9200 are oversub’d anyway, and the space/ports ratio isn’t great. > Cost per port is high compared to running with 4500/45550 setup and > can stack 10 deep as it is. I don’t know if I necessarily agree here either. The VC will quickly become a bottleneck if not already (32gb unidirectional traffic). Also, no support for 40/100g. If money is being spent, something like an Arista 7504 or Juniper QFX 5100/5200’s would make more sense here and allow for some growth not entirely based on "more 10g ports". -- Andrew Hoyos hoyosa(a)gmail.com

Sept. 20, 2016

Re: EX4200 (1G Switch) Packet Loss -> 10G Participants

by Mike Horwath

On Tue, Sep 20, 2016 at 09:49:48AM -0500, Levi Pederson wrote: > A full chassis 8200 and 9200 would be a solid upgrade for all > pertinent points. I don't agree. Cost per port is high compared to running with 4500/45550 setup and can stack 10 deep as it is. Get another pair of 4500/4550 (as it is now) and transition the current people off of the 4200 onto the ports there. This will make it so that the stack is 100% 10Gbe with VC ports creating the mesh. Also, 8200/9200 won't fit in the current space given to the endeavor today. And you'd want a pair, right? 4500 -> 4550 -> 4550 -> 4500 (loop back to 4500) for the rings. -- Mike Horwath, reachable via drechsau(a)Geeks.ORG

Sept. 20, 2016

Re: Non-Profit Status

by Richard Laager

On 09/20/2016 10:09 AM, Reid Fishler wrote: > This will be handed off with the new board seat as well. The new board member is going to need the documentation in question. -- Richard

Sept. 20, 2016

Re: Non-Profit Status

by Mike Horwath

On Tue, Sep 20, 2016 at 10:00:46AM -0500, Richard Laager wrote: > I've so far avoided commenting publicly, but enough is enough. We need > to move this along. This is hard and done via volunterring. Public is okay by me! -- Mike Horwath, reachable via drechsau(a)Geeks.ORG

Sept. 20, 2016

Re: Non-Profit Status

by Reid Fishler

Shaun has been managing this while dealing with his work, and being deployed. This will be handed off with the new board seat as well. Reid On Sep 20, 2016 6:00 PM, "Richard Laager" <rlaager(a)wiktel.com> wrote: > I'd like to request the board provide an actual update on the non-profit > status, preferably here before the meeting. > > I've been mailing off-list regularly since the last UG, and the answer > seems to be that: > > 1) We need to get some form to someone (lawyers or IRS?). > > 2) This form may need some financial details. > > 3) The board member responsible for this is busy and will do this "next > week". > > It's been "next week" every time. > > Another member tells me off-list that they've been doing the same thing, > for the same amount of time, and are getting the same answer. > > I've so far avoided commenting publicly, but enough is enough. We need > to move this along. > > -- > Richard > >

Sept. 20, 2016

Re: Non-Profit Status

by Shaun Carlson

That's fair Richard. It's on me to get the documentation to you and the other member and I will do so in short order. I have been focused on some other things, personally and professionally, and have not made this my priority. I will provide an update NLT Friday of this week. s *Shaun Carlson*Director of Information Technology | Arvig ph: (218) 346-8673 | em: shaun.carlson(a)arvig.com On Tue, Sep 20, 2016 at 10:00 AM, Richard Laager <rlaager(a)wiktel.com> wrote: > I'd like to request the board provide an actual update on the non-profit > status, preferably here before the meeting. > > I've been mailing off-list regularly since the last UG, and the answer > seems to be that: > > 1) We need to get some form to someone (lawyers or IRS?). > > 2) This form may need some financial details. > > 3) The board member responsible for this is busy and will do this "next > week". > > It's been "next week" every time. > > Another member tells me off-list that they've been doing the same thing, > for the same amount of time, and are getting the same answer. > > I've so far avoided commenting publicly, but enough is enough. We need > to move this along. > > -- > Richard >

Sept. 20, 2016

Non-Profit Status

by Richard Laager

I'd like to request the board provide an actual update on the non-profit status, preferably here before the meeting. I've been mailing off-list regularly since the last UG, and the answer seems to be that: 1) We need to get some form to someone (lawyers or IRS?). 2) This form may need some financial details. 3) The board member responsible for this is busy and will do this "next week". It's been "next week" every time. Another member tells me off-list that they've been doing the same thing, for the same amount of time, and are getting the same answer. I've so far avoided commenting publicly, but enough is enough. We need to move this along. -- Richard

Sept. 20, 2016

Re: EX4200 (1G Switch) Packet Loss -> 10G Participants

by Levi Pederson

All, A full chassis 8200 and 9200 would be a solid upgrade for all pertinent points. Thank you, *Levi Pederson* Mankato Networks LLC cell | 612.481.0769 work | 612.787.7392 levipederson(a)mankatonetworks.net On Tue, Sep 20, 2016 at 9:47 AM, Jeremy Lumby <jlumby(a)mnvoip.com> wrote: > On a partially related note, I think we should keep all of these little > technical details in the back of our mind for when we have nonprofit > status, and can collect the money for a larger switching platform. I think > they can really influence the scalability, redundancy, as well as ease of > monitoring/troubleshooting. > > -----Original Message----- > From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of > Andrew Hoyos > Sent: Tuesday, September 20, 2016 9:31 AM > To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET > Subject: Re: [MICE-DISCUSS] EX4200 (1G Switch) Packet Loss -> 10G > Participants > > Option #1 seems to be the most logical (with a 2x10g LAG, split between > the two 4500/4550 switches even MCLAG style). That removes that traffic > from the VC paths and simplifies that side of the house. > > I’d even pony up this EX-UM-2XFP I have sitting here unused. > > Are we graphing the VC links? > > SNMP support for that came in 14.something, otherwise, there is a SLAX > script that can put values in the util mib: > https://kb.juniper.net/InfoCenter/index?page=content& > id=KB27711&actp=search > https://github.com/dgarros/juniper-ex-vcp-to-mib/blob/ > master/ex-vcp-to-mib.slax > > I know that the interfaces themselves are clean, as Doug showed us, but > that doesn’t rule out microbursts that are maxing them out. > > -- > Andrew Hoyos > hoyosa(a)gmail.com > > > > > On Sep 20, 2016, at 9:21 AM, Jason Hanke <jayhanke(a)NEUTRALPATH.NET> > wrote: > > > > We've had pretty significant growth (20+ Gig) over the last year or so. > > > > <graph_image.png> > > > > Most of this traffic is CDN->Eyeball. The CDN networks are taking > increasingly larger connections creating a large delta between the largest > and smallest connections on the switch. A 4x10G LAG can put a pretty big > dent in the VC links on its own. > > > > I have two ideas- > > > > 1) Reconfigure the 1G switch to have a LAG group back into the main > switch. > > 2) Offer community strings letting the CDN networks send traffic to the > eyeballs without traversing the VC links. > > > > Jay > > > > > > > > On Tue, Sep 20, 2016 at 8:29 AM, Doug McIntyre <merlyn(a)iphouse.net> > wrote: > > No, that isn't the case, it is a dual ring, so there are paths > > > > FPC0->FPC1->FPC2->FPC0 > > FPC0->FPC2->FPC1->FPC0 > > > > There are ways to tell which is the "active" path through the rings, as I > > stated before, I think one path is mainly "active" and one is mainly > passive. > > I don't know if just disabling one VCP port will work. > > > > But I guess there is a KB on it. > > https://kb.juniper.net/InfoCenter/index?page=content& > id=KB17821&actp=search > > > > So, we could failover the active path to the other ring. > > Or set one of the VCP ports to disable. > > > > Should we do a 'virtual-chassis active path failover' event sometime? > > > > > > On Tue, Sep 20, 2016 at 08:07:00AM -0500, Jeremy Lumby wrote: > > > So if I understand correctly from your "show virtual-chassis > active-topology" below, all traffic from the 4500 to the 4200 is going via > the 4550? And if that is the case, is there an easy way in the software > that we could tell it to have return traffic from the 4200 to the 4500 also > go via the 4550? This would allow us to easily test to see if it is the > specific cable that runs from the 4200 to the 4500. > > > > > > -----Original Message----- > > > From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf > Of Doug McIntyre > > > Sent: Tuesday, September 20, 2016 12:24 AM > > > To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET > > > Subject: Re: [MICE-DISCUSS] EX4200 (1G Switch) Packet Loss -> 10G > Participants > > > > > > On Mon, Sep 19, 2016 at 11:01:00PM -0500, Jeremy Lumby wrote: > > > > This leads me to wonder if there is possibly an issue with the > stacking modules, or they are being saturated. > > > ... > > > > > > Your understanding of the vc port speeds is correct. > > > > > > The VCP ports are built into the EX4200 and EX4500. The EX4550 has > addon > > > cards that do those functions. We can devote a port to VCP > functionality > > > instead, but it'll be a 10G limit instead of 32G > > > > > > The VCP ports are 32Gbps single direction in each port. So in the > dual-ring > > > the cables have are 32Gbps in each direction. > > > > > > You can see which ports are connected to what. (FPC0 is the EX4500, > > > FPC1 is the EX4200, FPC2 is the EX4550). > > > > > > show virtual-chassis active-topology > > > fpc0: > > > ------------------------------------------------------------ > -------------- > > > Destination ID Next-hop > > > > > > 1 2(vcp-1.32768) 1(vcp-0.32768) > > > > > > 2 2(vcp-1.32768) > > > > > > fpc1: > > > ------------------------------------------------------------ > -------------- > > > Destination ID Next-hop > > > > > > 0 0(vcp-0.32768) 2(vcp-1.32768) > > > > > > 2 2(vcp-1.32768) > > > > > > fpc2: > > > ------------------------------------------------------------ > -------------- > > > Destination ID Next-hop > > > > > > 0 0(vcp-255/2/0.32768) > > > > > > 1 1(vcp-255/2/3.32768) > > > > > > > > > > > > You can monitor each of the VCP ports on each of the members. I don't > think > > > we are close to saturation. > > > > > > request session member 1 > > > (ie. rlogin to the EX4200 switch). > > > > > > monitor int vcp-0 > > > Interface: vcp-0, Enabled, Link is Up > > > Encapsulation: Virtual-Chassis-Interface, Speed: 32000mbps > > > Traffic statistics: > Current delta > > > Input bytes: 25685118521 > [12138] > > > Output bytes: 27856843010 > [13204] > > > Input packets: 33377693 > [16] > > > Output packets: 33371644 > [16] > > > Error statistics: > > > Input errors: 0 > [0] > > > Input drops: 0 > [0] > > > Input framing errors: 0 > [0] > > > Carrier transitions: 0 > [0] > > > Output errors: 0 > [0] > > > Output drops: 0 > [0] > > > > > > > > > vcp-1 on FPC1 is much less usage. Still zero error stats. All switches > > > show zero error stats on any of the respective VCP port status. > > > > > > I suppose further troubleshooting could be to replace the VCP cables > > > one by one coming out of the EX4200. > > > > > > Its too bad that we've got the quad fiber expansion in the EX4200, > > > another test could be to put the dual 10G expansion in it, and make > > > those VCP ports and make the VC fabric over those ports instead. But > > > we've got 3 member ports lit on that quad 1G fiber. > > > > > > > > > > > > -- > > > Doug McIntyre <merlyn(a)iphouse.net> > > > ~.~ ipHouse ~.~ > > > Network Engineer/Provisioning/Jack of all Trades > > > > -- > > Doug McIntyre <merlyn(a)iphouse.net> > > ~.~ ipHouse ~.~ > > Network Engineer/Provisioning/Jack of all Trades > > > > > > > > -- > > Jay Hanke > > CTO > > Neutral Path Communications > > 3 Civic Center Plaza, Suite 204 > > Mankato, MN 56001 > > (507) 327-2398 mobile > > jayhanke(a)neutralpath.net > > www.neutralpath.net > > > > To unsubscribe from the MICE-DISCUSS list, click the following link: > > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > > >

Sept. 20, 2016

Re: EX4200 (1G Switch) Packet Loss -> 10G Participants

by Jeremy Lumby

On a partially related note, I think we should keep all of these little technical details in the back of our mind for when we have nonprofit status, and can collect the money for a larger switching platform. I think they can really influence the scalability, redundancy, as well as ease of monitoring/troubleshooting. -----Original Message----- From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Andrew Hoyos Sent: Tuesday, September 20, 2016 9:31 AM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] EX4200 (1G Switch) Packet Loss -> 10G Participants Option #1 seems to be the most logical (with a 2x10g LAG, split between the two 4500/4550 switches even MCLAG style). That removes that traffic from the VC paths and simplifies that side of the house. I’d even pony up this EX-UM-2XFP I have sitting here unused. Are we graphing the VC links? SNMP support for that came in 14.something, otherwise, there is a SLAX script that can put values in the util mib: https://kb.juniper.net/InfoCenter/index?page=content&id=KB27711&actp=search https://github.com/dgarros/juniper-ex-vcp-to-mib/blob/master/ex-vcp-to-mib.… I know that the interfaces themselves are clean, as Doug showed us, but that doesn’t rule out microbursts that are maxing them out. -- Andrew Hoyos hoyosa(a)gmail.com > On Sep 20, 2016, at 9:21 AM, Jason Hanke <jayhanke(a)NEUTRALPATH.NET> wrote: > > We've had pretty significant growth (20+ Gig) over the last year or so. > > <graph_image.png> > > Most of this traffic is CDN->Eyeball. The CDN networks are taking increasingly larger connections creating a large delta between the largest and smallest connections on the switch. A 4x10G LAG can put a pretty big dent in the VC links on its own. > > I have two ideas- > > 1) Reconfigure the 1G switch to have a LAG group back into the main switch. > 2) Offer community strings letting the CDN networks send traffic to the eyeballs without traversing the VC links. > > Jay > > > > On Tue, Sep 20, 2016 at 8:29 AM, Doug McIntyre <merlyn(a)iphouse.net> wrote: > No, that isn't the case, it is a dual ring, so there are paths > > FPC0->FPC1->FPC2->FPC0 > FPC0->FPC2->FPC1->FPC0 > > There are ways to tell which is the "active" path through the rings, as I > stated before, I think one path is mainly "active" and one is mainly passive. > I don't know if just disabling one VCP port will work. > > But I guess there is a KB on it. > https://kb.juniper.net/InfoCenter/index?page=content&id=KB17821&actp=search > > So, we could failover the active path to the other ring. > Or set one of the VCP ports to disable. > > Should we do a 'virtual-chassis active path failover' event sometime? > > > On Tue, Sep 20, 2016 at 08:07:00AM -0500, Jeremy Lumby wrote: > > So if I understand correctly from your "show virtual-chassis active-topology" below, all traffic from the 4500 to the 4200 is going via the 4550? And if that is the case, is there an easy way in the software that we could tell it to have return traffic from the 4200 to the 4500 also go via the 4550? This would allow us to easily test to see if it is the specific cable that runs from the 4200 to the 4500. > > > > -----Original Message----- > > From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Doug McIntyre > > Sent: Tuesday, September 20, 2016 12:24 AM > > To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET > > Subject: Re: [MICE-DISCUSS] EX4200 (1G Switch) Packet Loss -> 10G Participants > > > > On Mon, Sep 19, 2016 at 11:01:00PM -0500, Jeremy Lumby wrote: > > > This leads me to wonder if there is possibly an issue with the stacking modules, or they are being saturated. > > ... > > > > Your understanding of the vc port speeds is correct. > > > > The VCP ports are built into the EX4200 and EX4500. The EX4550 has addon > > cards that do those functions. We can devote a port to VCP functionality > > instead, but it'll be a 10G limit instead of 32G > > > > The VCP ports are 32Gbps single direction in each port. So in the dual-ring > > the cables have are 32Gbps in each direction. > > > > You can see which ports are connected to what. (FPC0 is the EX4500, > > FPC1 is the EX4200, FPC2 is the EX4550). > > > > show virtual-chassis active-topology > > fpc0: > > -------------------------------------------------------------------------- > > Destination ID Next-hop > > > > 1 2(vcp-1.32768) 1(vcp-0.32768) > > > > 2 2(vcp-1.32768) > > > > fpc1: > > -------------------------------------------------------------------------- > > Destination ID Next-hop > > > > 0 0(vcp-0.32768) 2(vcp-1.32768) > > > > 2 2(vcp-1.32768) > > > > fpc2: > > -------------------------------------------------------------------------- > > Destination ID Next-hop > > > > 0 0(vcp-255/2/0.32768) > > > > 1 1(vcp-255/2/3.32768) > > > > > > > > You can monitor each of the VCP ports on each of the members. I don't think > > we are close to saturation. > > > > request session member 1 > > (ie. rlogin to the EX4200 switch). > > > > monitor int vcp-0 > > Interface: vcp-0, Enabled, Link is Up > > Encapsulation: Virtual-Chassis-Interface, Speed: 32000mbps > > Traffic statistics: Current delta > > Input bytes: 25685118521 [12138] > > Output bytes: 27856843010 [13204] > > Input packets: 33377693 [16] > > Output packets: 33371644 [16] > > Error statistics: > > Input errors: 0 [0] > > Input drops: 0 [0] > > Input framing errors: 0 [0] > > Carrier transitions: 0 [0] > > Output errors: 0 [0] > > Output drops: 0 [0] > > > > > > vcp-1 on FPC1 is much less usage. Still zero error stats. All switches > > show zero error stats on any of the respective VCP port status. > > > > I suppose further troubleshooting could be to replace the VCP cables > > one by one coming out of the EX4200. > > > > Its too bad that we've got the quad fiber expansion in the EX4200, > > another test could be to put the dual 10G expansion in it, and make > > those VCP ports and make the VC fabric over those ports instead. But > > we've got 3 member ports lit on that quad 1G fiber. > > > > > > > > -- > > Doug McIntyre <merlyn(a)iphouse.net> > > ~.~ ipHouse ~.~ > > Network Engineer/Provisioning/Jack of all Trades > > -- > Doug McIntyre <merlyn(a)iphouse.net> > ~.~ ipHouse ~.~ > Network Engineer/Provisioning/Jack of all Trades > > > > -- > Jay Hanke > CTO > Neutral Path Communications > 3 Civic Center Plaza, Suite 204 > Mankato, MN 56001 > (507) 327-2398 mobile > jayhanke(a)neutralpath.net > www.neutralpath.net > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >

Sept. 20, 2016