September 2016 - mice-discuss

Re: EX4200 (1G Switch) Packet Loss -> 10G Participants

by Matthew Beckwell

Testing against the UW iperf servers seems to be mostly clean. (Occasionally, I get one out-of-order, but no loss running the same 10M UDP test). So this (somewhat limited) test seems to show the current path from 4200 -> 4550 is okay. ~Matthew On Mon, Sep 19, 2016 at 4:06 PM, Andrew Hoyos <hoyosa(a)gmail.com> wrote: > > UW Madison operates some IPerf servers you could try against too, see: > https://kb.wisc.edu/uwsysnet/page.php?id=41947 > They are on the 4550, it appears, so more data points to be gleaned there. > >

Sept. 19, 2016

Re: EX4200 (1G Switch) Packet Loss -> 10G Participants

by Andrew Hoyos

> On Sep 19, 2016, at 2:19 PM, Doug McIntyre <merlyn(a)iphouse.net> wrote: > > On Mon, Sep 19, 2016 at 02:03:41PM -0500, Andrew Hoyos wrote: >> What medium is the VC built on and physical topology? Just VC cables on the rear ringed through all 3? > > Yes, VC cabling in a ring, and using the VCP modules and cabling on the EX4550. What’s the actual topology though? 4500 -> 4200 -> 4550 -> back to 4500? >> Specific to any single 4500 destination or is it doing it to both the 4500 and the 4550? > > He wasn't aware of the EX4550, there aren't many ports lit there yet, > he has just been testing different ports on the the EX4500. UW Madison operates some IPerf servers you could try against too, see: https://kb.wisc.edu/uwsysnet/page.php?id=41947 They are on the 4550, it appears, so more data points to be gleaned there. >> Check ‘show virtual-chassis vc-path source-interface <src int> destination-interface <dest int>’. Common VC port in the mix to any of the reported loss paths? > > I think in general, the VC ports run Active/Passive, so only one is > "active" at a time going out of a box without load balancing, so only > one (ie. vcp-1) typically shows up for all paths. Kinda, there is a SPF calculation that happens based on src/dst interfaces. It matters which PFE of the switch the port is on (1 hop != switch, it’s actually each PFE hop). Depending on the topology here, things could be going a variety of ways. -- Andrew Hoyos hoyosa(a)gmail.com

Sept. 19, 2016

Re: EX4200 (1G Switch) Packet Loss -> 10G Participants

by Andrew Hoyos

On Sep 19, 2016, at 3:29 PM, Matthew Beckwell <matthewb(a)aitech.net> wrote: > > This is certainly interesting....I'm not sure what's special about xe-0/0/14 vs ae5. > Some kind of buffer or queue difference? IIRC on the 4500, there is a PFE mapped to ports 0-19 and another PFE handling 20-39, with 200gbps between. AE5 appears to be built (from the website) out of xe-0/0/29,xe-0/0/38,xe-2/0/15. How that hashing played out for your test and which link it actually took, who knows. Either way, could be a 4500 only thing, or PFE cross traffic related. Seems like it might be a good idea to simplify that topology if possible (given the mixed mode behavior already). -- Andrew Hoyos hoyosa(a)gmail.com

Sept. 19, 2016

Re: EX4200 (1G Switch) Packet Loss -> 10G Participants

by Jeremy Lumby

One interesting thing that might be playing into it is that ae5 is split across both the 4500, and the 4550. I wonder if the issue is only when traffic goes from the 4200 to the 4500, and not the 4200 to the 4550. From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Matthew Beckwell Sent: Monday, September 19, 2016 3:30 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] EX4200 (1G Switch) Packet Loss -> 10G Participants 10M UDP testing to Steve's network at Paul Bunyan (egress ae5), doesn't appear to have any loss. The same 10M UDP test to US Internet, egress xe-0/0/14 shows about 3-5% loss. Again, same 10M UDP test to the same US Internet destination IP forced out another transit provider shows 0 loss. This is certainly interesting....I'm not sure what's special about xe-0/0/14 vs ae5. Some kind of buffer or queue difference? ~Matthew On Mon, Sep 19, 2016 at 2:03 PM, Steve Howard <showard(a)paulbunyan.net> wrote: On 09/19/2016 01:49 PM, Matthew Beckwell wrote: As to whether there's loss to one of the "remote" switches, I'm not sure.... If one of the Neutral Path/Mankato Networks/ CNS connected participants would be up for running iperf for a little while, we could find out fairly quickly. We've opened up our iperf3 server on 209.191.196.90 for a few hours. To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Sept. 19, 2016

Re: EX4200 (1G Switch) Packet Loss -> 10G Participants

by Matthew Beckwell

10M UDP testing to Steve's network at Paul Bunyan (egress ae5), doesn't appear to have any loss. The same 10M UDP test to US Internet, egress xe-0/0/14 shows about 3-5% loss. Again, same 10M UDP test to the same US Internet destination IP forced out another transit provider shows 0 loss. This is certainly interesting....I'm not sure what's special about xe-0/0/14 vs ae5. Some kind of buffer or queue difference? ~Matthew On Mon, Sep 19, 2016 at 2:03 PM, Steve Howard <showard(a)paulbunyan.net> wrote: > > > On 09/19/2016 01:49 PM, Matthew Beckwell wrote: > > > As to whether there's loss to one of the "remote" switches, I'm not > sure.... > If one of the Neutral Path/Mankato Networks/ CNS connected participants > would be up for running iperf for a little while, we could find out fairly > quickly. > > > We've opened up our iperf3 server on 209.191.196.90 for a few hours. >

Sept. 19, 2016

Re: Proposed Membership and Sponsorship Model

by Justin Krejci

I am a fan of the SIX model of revenue: you pay to connect to the exchange up front and bigger ports (ie 10-Gig) cost more than smaller (ie 1-Gig). This fee applies to MICE core equipment only. Port costs on extension switches are dictated by the respective extension switch operators. As there are already many existing connected ports I would also be in favor of retroactively charging each connected port based on their current port type(s) in use at the time of the implementation of said fees. Already connected organizations that want to take a bit of time to budget can make arrangements with the board. Also, I would be in favor of implementing this soon instead of waiting for the desired but elusive non-profit status unless someone has knowledge of its actual pending completion. I feel this method alleviates the problem of non-payment induced disconnects, switch port reservations, tracking payments, etc. I don't know the history of other exchanges with regards to disconnecting organizations, freeing up port and IP number resources, but clearly sometimes companies go under or otherwise have a change in strategy and no longer desire to be connected yet do not formally announce they are disconnecting. So to that end, after some period of time (X months) of being in an unreachable state via their IXP IP addresses, a disconnect is implemented unless said organization otherwise scheduled or coordinated with MICE to be offline for an extended period of time. I am guessing there is some historical data available from other exchanges or other best practices that could be used to guide us on how to do this. With regards to gaining membership/voting rights I like the notion of requiring an active port connection to the exchange (core or extension) with an ASN and reachable exchange assigned IP address(es). An alternate method of gaining membership/voting rights would be from serving on the board of the exchange which would not require one to be an already connected organization of the exchange. Summary of member voting eligibility Provides Eligibility to Vote: * Elected board members. Board members may or may not represent a connected organization, I assume the bylaws cover the 1-vote max per organization but that is implied here * Connected organizations with ASN and at least one reachable exchange assigned IP number Does Not Provide Eligibility to Vote: * Sponsors, 3rd parties, etc (eg Cologix, operators of extension switches) * Organizations exclusively using ancillary exchange services (eg pvlans) ________________________________ From: MICE Discuss [MICE-DISCUSS(a)LISTS.IPHOUSE.NET] on behalf of Steve Howard [showard(a)PAULBUNYAN.NET] Sent: Thursday, September 15, 2016 7:22 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Proposed Membership and Sponsorship Model I'm a little "late to the party" for this discussion, but I'd like to add a few of my thoughts: First off, I'd like to thank Dave for coming up with great ideas and points for discussion. MICE is a cooperative (not a charity). As a cooperative, we need to treat everybody equally. I feel that everybody should have to pay membership dues at some level, perhaps an amount based upon port size. If we fail to require the payment of dues, we will have different classes of users and the potential for discontent as some people don't pay their fair share. Like Martin said, "Classes create inequities". If people don't pay, they need to be disconnected (after a reasonable grace period). The dollar amounts due will likely be very small, the exchange has gotten by without any payments required for several years. I don't think that the fees that would need to be charged would be anything that an organization connecting to the exchange cannot absorb. If they can't handle ~$9/month (using Dave's example numbers), they won't be around long in this industry. I agree with Richard Laager's points about enforcement. We need to enforce whatever policy is decided. I don't like any scenario where the board might have the option to waive fees. That will create the feeling of the "good 'ol boy" system. I also think that all newly connected members should get their initial connect period free, primarily as a sales tool. I'd like to give new members a chance to "try before you buy" to see if it works for them. Perhaps we bill annually and they don't pay until the next billing cycle starts (or some variation thereof)? Until they pay, they should be subject to their port being assigned to somebody else, not be official members, and not able to vote. How do we handle members that are connected to remote switches? Is the switch owner the member? The downstream users? Both? Who pays the port charges? Only the switch owner? Steve On 09/08/2016 01:05 PM, David Farmer wrote: I'm not a candidate and will not be a candidate for the Board. However, I propose the following Membership and Sponsorship Model, including both Membership Dues and Sponsorship levels for MICE Members and Non-Member Sponsors. The dollar values are just my suggestions, I think they should work but they are primarily intended to illustrate the model. With this Model, any network may connect to the Exchange, but all networks participating in governance, known as Members, will have some minimal skin-in-the-game. Further, the Model implies Networks with 10G or larger ports probably should participate at a more significant level, but this is not a requirement for access to 10G or larger ports or even to participate in governance. The Friends of MICE Sponsorship level is intended to be accessible to anyone interested, including individual users of the Internet. My proposal; As an Open Internet Exchange any network with a uniquely registered Autonomous System Number (ASN) may connect to MICE. However, Non-Member networks are provided ports on the Exchange on an as-available basis only and Non-Members do not participate in the governance of the Exchange or the MICE Organization, but may observe and participate in its discussions. Networks connected to the Exchange, or Data Centers hosting an Exchange switch, may become Members and participate in the governance of the Exchange and the MICE Organization by paying Membership Dues at either level listed below. Members are provided prefered access to 1G Exchange ports, and are provided access to 10G or larger Exchange ports on an as-available basis. Sustaining or higher Members are committed to sustaining and growing the Exchange by electing to pay Sustaining Membership Dues and/or paying Basic Membership Dues and providing other Sponsorship to the Exchange of a value totaling $1000 or more. In recognition to this additional commitment, Sustaining or higher Members are provided prefered access to 10G or larger Exchange ports. Basic Membership Dues - $100 Sustaining Membership Dues - $1000 (Dues are paid in cash or cash equivalents only, and may not be provided through in-kind services or equipment) Beyond Dues from Members, MICE accepts Sponsorship both from Members and Non-Members Sponsors as cash, cash equivalents, in-kind services or equipment. Sponsorship is recognized at the following escalating levels based on the total value provided annually. Friends of MICE - Sponsorship of $50 ore more annually Sustaining Members or Sponsors - Sponsorship of $1,000 or more annually Silver Members or Sponsors - Sponsorship of $2,500 or more annually Gold Members or Sponsors - Sponsorship of $5,000 or more annually Platinum Members or Sponsors - Sponsorship of $10,000 or more annually Sponsorship of $20,000 or more are recognized with multiple years at the Platinum level. Friends of MICE are list on a page on the MICE website. Sustaining or higher Members are annotated as such in their listings on the MICE website. All Sustaining or higher Members and Sponsors are listed on the MICE website, with higher levels receiving more prominent placement. Silver or higher Members and Sponsors are recognized in MICE marketing materials, with higher levels receiving more prominent placement. Thanks -- =============================================== David Farmer Email:farmer@umn.edu<mailto:Email%3Afarmer@umn.edu> Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815<tel:612-626-0815> Minneapolis, MN 55414-3029 Cell: 612-812-9952<tel:612-812-9952> =============================================== ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Sept. 19, 2016

Re: EX4200 (1G Switch) Packet Loss -> 10G Participants

by Matthew Beckwell

Thanks Steve, I haven't done much with iperf3-- it looks like they took away the dual-test option, so I'm a little blind. Are you able to see loss on the iperf3 server side from 199.168.228.123 ? ~Matthew On Mon, Sep 19, 2016 at 2:03 PM, Steve Howard <showard(a)paulbunyan.net> wrote: > > We've opened up our iperf3 server on 209.191.196.90 for a few hours. > > > ------------------------------ > > >

Sept. 19, 2016

Re: EX4200 (1G Switch) Packet Loss -> 10G Participants

by Doug McIntyre

On Mon, Sep 19, 2016 at 02:03:41PM -0500, Andrew Hoyos wrote: > What medium is the VC built on and physical topology? Just VC cables on the rear ringed through all 3? Yes, VC cabling in a ring, and using the VCP modules and cabling on the EX4550. > Specific to any single 4500 destination or is it doing it to both the 4500 and the 4550? He wasn't aware of the EX4550, there aren't many ports lit there yet, he has just been testing different ports on the the EX4500. > Check ‘show virtual-chassis vc-path source-interface <src int> destination-interface <dest int>’. Common VC port in the mix to any of the reported loss paths? I think in general, the VC ports run Active/Passive, so only one is "active" at a time going out of a box without load balancing, so only one (ie. vcp-1) typically shows up for all paths. But either way, the virtual-chassis stats show zero errors, all the vcp-? ports show zero errors or drops. -- Doug McIntyre <merlyn(a)iphouse.net> ~.~ ipHouse ~.~ Network Engineer/Provisioning/Jack of all Trades

Sept. 19, 2016

Re: EX4200 (1G Switch) Packet Loss -> 10G Participants

by Steve Howard

Sept. 19, 2016

Re: EX4200 (1G Switch) Packet Loss -> 10G Participants

by Andrew Hoyos

> On Sep 19, 2016, at 1:56 PM, Doug McIntyre <merlyn(a)IPHOUSE.NET> wrote: > > Yes, there is a VC stack of > > EX4200 > EX4500 > EX4550 What medium is the VC built on and physical topology? Just VC cables on the rear ringed through all 3? Specific to any single 4500 destination or is it doing it to both the 4500 and the 4550? Check ‘show virtual-chassis vc-path source-interface <src int> destination-interface <dest int>’. Common VC port in the mix to any of the reported loss paths? -- Andrew Hoyos hoyosa(a)gmail.com

Sept. 19, 2016