December 2016 - mice-discuss - lists.micemn.net

Re: MICE Fees Proposal/UG Agenda Item/Request for Comment

by Jason Hanke

> > Think about the future. If Neutral Path were to be sold, the new owner may > not have the same altruistic attitude that you have. And the exchange > should be prepared by having the financial ability to move to another > solution readily. > > That's part of how the reserve is intended to be used. Jay

Dec. 1, 2016

Re: MICE Fees Proposal/UG Agenda Item/Request for Comment

by Jason Hanke

> > > The networks in this range are moving the bulk of the traffic on the IX > > Yes, but they are also providing the most value to the IX for all members. > True, and they are receiving the bulk of the value. > > > > Why shouldn’t all 10g ports just be $250/yr? With this layout, I fear > that the “Big Guys” will think you’re dumping the fee load (and downstream > equipment cost) onto them by forcing 100g ports at that cost for n x 10g. > Seems like being agnostic on a port level (no matter how many you have) > would be a better move. > The costs including the xconn and port fee are below just the xconn fee in most major carrier hotels. The reduced 10G initial port is to provide a stepping stone to paid membership while allowing small networks a low-cost way in. At $250 per year per 10G port we don't raise enough money to be stable or to build a reserve for hardware or other expenses. > Right, but you’re leaving out the internal costs for the big guys (or mid > tier/small folks even) of router ports. It’s far cheaper for one of the > big guys to light another 10g port, I’d suspect, than add 100g interfaces. > Maybe that’s changing, but again, I don’t agree with penalizing folks for > adding more traffic to the IX on multiple 10g ports vs single 100g ports. > MICE isn’t paying cross connects, so seems like that should’t be part of > the equation when we look at it from a fee charging perspective. > Each network will have to make the calculation and determine what is in their best interest. MICE is here to serve the members and the overall cost to the members is critical. We have a fixed amount of real estate to work with for ports within the chassis. 100G is a much more efficient than doing large LAG groups. So a 8 member lag eats up a larger portion of the annual support budget. It also requires setup and troubleshooting of 8 ports. On a 10G card with 48 ports, an 8 member LAG uses 12.5% of the slot or 3% of the overall chassis. On a 100G card there are 32 ports, A single 100G uses 3% of a slot and less than 1% of the chassis. Jay

Dec. 1, 2016

Re: MICE Fees Proposal/UG Agenda Item/Request for Comment

by Bil MacLeslie

On 12/1/16 10:41 AM, Reid Fishler wrote: > Bill- > The problem is we can't charge more than some members use the > exchange, and we can't plan for everything. We may still go to the > membership and ask for support if need-be, but we can't have the cost of > the exchange be too high...look at the other competitive exchanges of > our size...all are doing something similar. Believe me, the board spent > ALOT of time working on this...ALOT. :-) Reid, The fee schedule proposed by Jason covers my concerns, if everyone pays. The anticipated revenue (by Jasons calculation) is $37k. With less than half the budget allocated to operational expenses, I expect building a reserve to buy equipment. That's my point on the hardware side. It seems that your concern is that the proposed fees are too high for many members / or that the fees are disproportionate to the value. I'm not sure which. I'm in favor of a fee structure, not opposed. My points here and in my response to Jason, are to make the members aware that there are real costs associated with running the exchange, and the fees proposed cover more than the obvious things, like ports. Bil -- Bil K. MacLeslie | TGWGTD | ipHouse | www.ipHouse.com Office: 612-337-6337 | Direct: 612-337-6206 | Cell: 612-618-1218

Dec. 1, 2016

Re: MICE Fees Proposal/UG Agenda Item/Request for Comment

by Bil MacLeslie

On 12/1/16 11:00 AM, Jason Hanke wrote: > 1 - How does $1800 / year cover the costs of actually DOING the > accounting, e.g. creating new members & invoices, processing > payments, calling on collections, etc. This number seems low to me. > Additional thought, some entities will want to pay with a credit > card, we should prepare for this and cover the costs of processing > payments in the budget. > > > Neutral Path is currently donating bookkeeping services to the exchange. > By Neutral Path I really mean Kayla. This is factored into the current > budget. Other than the proposed new invoicing there are less than 10 > transactions per year. We can use Paypal to process credit cards the fee > is about 3%. Jason / All I certainly appreciate the volunteering, but going forward it makes sense to allocate budget to cover the true costs of services the exchange needs - even if that budget isn't used. Think about the future. If Neutral Path were to be sold, the new owner may not have the same altruistic attitude that you have. And the exchange should be prepared by having the financial ability to move to another solution readily. Bil -- Bil K. MacLeslie | TGWGTD | ipHouse | www.ipHouse.com Office: 612-337-6337 | Direct: 612-337-6206 | Cell: 612-618-1218

Dec. 1, 2016

Re: MICE Fees Proposal/UG Agenda Item/Request for Comment

by Mike Horwath

On Thu, Dec 01, 2016 at 09:39:43AM -0600, Jason Hanke wrote: > USI has arranged for Arista Networks to donate a 7504 switch with > 32x100G and 96x10G ports with one year of support. In year 2, MICE > would be responsible for the ongoing support costs about 14k per > year. There would be one additional slot for future growth. First - thank you USI for diving into this! https://www.arista.com/en/products/7500-series for those that wanna look into what this chassis is capable of. > Here's what I'd propose: > 1G ports= Free (Only 1 port per member, NO LAG allowed) > Initial 10G port= $250 Annual > Each additional 10G port $1000 Annual per port (up to 8 member LAG > allowed) > 100G port $3000 Annual per port My counter commentary + fee list: 1Gbe ports - $250 annual per port, LAG allowed, max 2 10Gbe ports - $500 annual per port, LAG allowed, max 4 40Gbe ports - $2000 annual per port, cause why not just do 100Gbe!?!, LAG max 2, and above no 40Gbe ports were listed so really, do 100Gbe and let's move on, okay? 100Gbe ports - $3000 annual per port, LAG allowed, max 4 Optics provided by the member and must be on the approved list for the hardware (in this case: Arista 7504) - I know that later on Jay mentions that Arista will remove the lockout for them but I don't think MICE should just allow who-flung-poo optics installed because the member is trying to be cheap at the risk of problems with the fabric. Non-participating member - $100 annual, no voting position, feel good fee Participating member - $250 annual, voting position, no port, extra feel good fee. Users who have ports already assigned into a LAG should be grandfathered in for their configuration regardless of LAG port count limit as proposed. If a member is already at 5 10Gbe ports then let them stay at that level. If member wants to add another port then push them towards the 100Gbe solution but allow for a single port increase in LAG size only. > Remote switch operators pay the fee for their main switch ports. > Remote switch participants do not pay a main switch port fee. Seems fair! But if a remote switch operator is charging at a higher rate than paid into the current fees should a portion of that be sent into the funding as well? > No grandfathering, all types of networks are treated the same. YES! > We will not disconnect for non-payment until port run out and > contention occurs. A policy will need to be developed. I don't agree - non-payment == shutdown IMO. And if continues for 3 months is then removed from the fabric and pulled from site. Suspend and comment out the BGP configuration and IP allocation for up to 6 months before removal, and later reallocation of IP space. > Each carrier provides their own optic. Arista has agreed to remove > the vendor lockout for optics. Will they support non-approved optics then? Jay then brings up revenue projections but I think it should go to everyone as this is a single chassis solution. I'd like to see at least a dual chassis solution and members spread across as needed to keep resiliency high. Yes, I know, I am no longer officially part of MICE and my opinion may not matter to all of you. And that's okay :) -- Mike Horwath, reachable via drechsau(a)Geeks.ORG

Dec. 1, 2016

Re: MICE Fees Proposal/UG Agenda Item/Request for Comment

by Doug McIntyre

On Thu, Dec 01, 2016 at 10:19:05AM -0600, Steve Howard wrote: > How many 1G ports are on the Arista? Any of the 10G ports can be run at 1G. Likewise, any of the 100G ports can be broken out to 4 x 10G, 4 x 25G, 1? x 40G, 2 x 50G or 1 x 100G. with the correct breakout cables. > Would we continue to use the original switch for 1G ports? I think that would be the best course of action. Right now that is dual-10G connected to the main 10G switch stack. > Do the existing switch and Arista support the same form-factor > optics? Yes, SFP+ form factor for 10G. QSFP for higher speeds. > i.e. Is there a requirement to purchase new optics to be > connected to the new switch under your proposal? Arista says they will disable the vendor lockin on the optics for this deployment. -- Doug McIntyre <merlyn(a)iphouse.net> ~.~ ipHouse ~.~ Network Engineer/Provisioning/Jack of all Trades

Dec. 1, 2016

Re: MICE Fees Proposal/UG Agenda Item/Request for Comment

by Jason Hanke

> > 1 - How does $1800 / year cover the costs of actually DOING the > accounting, e.g. creating new members & invoices, processing payments, > calling on collections, etc. This number seems low to me. Additional > thought, some entities will want to pay with a credit card, we should > prepare for this and cover the costs of processing payments in the budget. > Neutral Path is currently donating bookkeeping services to the exchange. By Neutral Path I really mean Kayla. This is factored into the current budget. Other than the proposed new invoicing there are less than 10 transactions per year. We can use Paypal to process credit cards the fee is about 3%. > > 2 - Where is hardware refresh in the budget? I see a clipped sentence in > your message ( This would cover hardware support and Arin fees and > accounting expense with the remainder going) - maybe it was in there? > Depending on the response to the request for payment there is between 8k and 24k annually building up for hardware refresh or additions. A large update will have to be dealt with at the time or with a future rate increase. > > We have been lucky to secure donations of equipment, but what about the > future? Your budget should include setting aside dollars for hardware > expansion and refresh, and not just switches, but route servers too. > Minor additions are figured in, Major replacement is not. > > Additional thoughts, if new technologies emerge and a hardware vendor that > has said technology doesn't feel like donating equipment, how would we > proceed without a hardware budget? And... by accepting donations from > hardware vendors, we are setting the precedent of "picking a winner", and > could cause headache/heartache in the future. > We would have to deal with new technologies as they emerge. We've been accepting hardware donations for quite some time without serious issue. The current kit contains a lot of donated parts. -- Jay Hanke CTO Neutral Path Communications 3 Civic Center Plaza, Suite 204 Mankato, MN 56001 (507) 327-2398 mobile jayhanke(a)neutralpath.net www.neutralpath.net

Dec. 1, 2016

Re: MICE Fees Proposal/UG Agenda Item/Request for Comment

by Andrew Hoyos

On Dec 1, 2016, at 10:42 AM, Jason Hanke <jayhanke(a)NEUTRALPATH.NET> wrote: > > > If we’re going the annual route - any reason/logic behind why the 20g -> 80g range seems to carry a disproportionate amount of the fee load? > > The networks in this range are moving the bulk of the traffic on the IX Yes, but they are also providing the most value to the IX for all members. I think we need to be agnostic about how many ports a member has or how much traffic they are moving. I still think this fee structure unnecessarily classifies a group of the members into paying more on a $/gbps ratio, which doesn’t make sense. Steve’s logic, in his email, makes way more sense to me, and is the totally agnostic - simplify this into a fixed port level fee, and done. 1Gbps=$35/Year (although, I’d argue that this might as well be free at that point for the 1g members) 10Gbps=$350/Year 100Gbps=$3,500/Year > > Why shouldn’t all 10g ports just be $250/yr? With this layout, I fear that the “Big Guys” will think you’re dumping the fee load (and downstream equipment cost) onto them by forcing 100g ports at that cost for n x 10g. Seems like being agnostic on a port level (no matter how many you have) would be a better move. > > Looking at the overall economics to a member the cross connect cost dominates the spend. Specifically when doing lag. If you take the annual fee and convert it to monthly it works out to be about $83 dollars with a $200 cross connect for a monthly total of $283 on a second port. > > For example 4 member 10G lag (monthly): > > Port 1 $200+$21=$221 > Port 2 $200+$83=$283 > Port 3 $200+$83=$283 > Port 4 $200+$83=$283 > ------------------------------- > Total $1070 per month > > A single 100G > Port 1 $200+$250=$450 per month > > This leaves room to cover the incremental 100G optics cost. If the 10G ports were free the 100G would still be a better deal for the big guys. Right, but you’re leaving out the internal costs for the big guys (or mid tier/small folks even) of router ports. It’s far cheaper for one of the big guys to light another 10g port, I’d suspect, than add 100g interfaces. Maybe that’s changing, but again, I don’t agree with penalizing folks for adding more traffic to the IX on multiple 10g ports vs single 100g ports. MICE isn’t paying cross connects, so seems like that should’t be part of the equation when we look at it from a fee charging perspective.

Dec. 1, 2016

Re: MICE Fees Proposal/UG Agenda Item/Request for Comment

by Jason Hanke

> If we’re going the annual route - any reason/logic behind why the 20g -> 80g range seems to carry a disproportionate amount of the fee load? The networks in this range are moving the bulk of the traffic on the IX > Why shouldn’t all 10g ports just be $250/yr? With this layout, I fear that the “Big Guys” will think you’re dumping the fee load (and downstream equipment cost) onto them by forcing 100g ports at that cost for n x 10g. Seems like being agnostic on a port level (no matter how many you have) would be a better move. Looking at the overall economics to a member the cross connect cost dominates the spend. Specifically when doing lag. If you take the annual fee and convert it to monthly it works out to be about $83 dollars with a $200 cross connect for a monthly total of $283 on a second port. For example 4 member 10G lag (monthly): Port 1 $200+$21=$221 Port 2 $200+$83=$283 Port 3 $200+$83=$283 Port 4 $200+$83=$283 ------------------------------- Total $1070 per month A single 100G Port 1 $200+$250=$450 per month This leaves room to cover the incremental 100G optics cost. If the 10G ports were free the 100G would still be a better deal for the big guys. > I’d also disagree with not allowing multiple 1g ports for customers into a LAG. For small bootstrapped companies, that 1g -> 10g jump can be expensive, and 2xGigE LAG doesn’t seem like a problem in the case where they are hitting that 60-70% on a GigE. Given the Arista architecture and depth of buffers there, I wouldn’t see any technical issue leaving the EX4550 hanging off the 7504, just for 1g customers, if you’re worried about burning “expensive” ports. > Again the cross connect fees dominate the economics at the 1G level. For the initial hardware costs. 1G lag would be allowed on the remote switches if the remote switch operator allowed it. I think all the remote switches would allow it, Neutral Path certainly would. I don't think we've had a request for a 1G lag since we had 10G available on the main switch. All of the aggregate traffic on the "1G" switch is peaking at less than 1G. [image: Inline image 1] Jay

Dec. 1, 2016

Re: MICE Fees Proposal/UG Agenda Item/Request for Comment

by Reid Fishler

Bill- The problem is we can't charge more than some members use the exchange, and we can't plan for everything. We may still go to the membership and ask for support if need-be, but we can't have the cost of the exchange be too high...look at the other competitive exchanges of our size...all are doing something similar. Believe me, the board spent ALOT of time working on this...ALOT. :-) Reid On Thu, Dec 1, 2016 at 11:38 AM, Bil MacLeslie <bil(a)iphouse.net> wrote: > On 12/1/16 9:39 AM, Jason Hanke wrote: > > Arin: $300 >> Accounting: $1800 (includes postage for billing) >> Hardware support: $15000 >> > > Two questions. > > 1 - How does $1800 / year cover the costs of actually DOING the > accounting, e.g. creating new members & invoices, processing payments, > calling on collections, etc. This number seems low to me. Additional > thought, some entities will want to pay with a credit card, we should > prepare for this and cover the costs of processing payments in the budget. > > 2 - Where is hardware refresh in the budget? I see a clipped sentence in > your message ( This would cover hardware support and Arin fees and > accounting expense with the remainder going) - maybe it was in there? > > We have been lucky to secure donations of equipment, but what about the > future? Your budget should include setting aside dollars for hardware > expansion and refresh, and not just switches, but route servers too. > > Additional thoughts, if new technologies emerge and a hardware vendor that > has said technology doesn't feel like donating equipment, how would we > proceed without a hardware budget? And... by accepting donations from > hardware vendors, we are setting the precedent of "picking a winner", and > could cause headache/heartache in the future. > > I strongly suggest we use the port fees to pay for hardware refresh as > well as hardware maintenance. > > Bil > > -- > Bil K. MacLeslie | TGWGTD | ipHouse | www.ipHouse.com > Office: 612-337-6337 | Direct: 612-337-6206 | Cell: 612-618-1218 > > -- Reid Fishler Director Hurricane Electric +1-510-580-4178

Dec. 1, 2016