September 2012 - mice-discuss

Re: MICE Traffic Lower

by Owen DeLong

On Sep 6, 2012, at 14:55 , James Stahr <stahr(a)MAILBAG.COM> wrote: > On 9/6/2012 11:57 AM, Owen DeLong wrote: >> There was certainly plenty of discussion of that particular topic to enable them to do so. >> >> Owen >> >> > > I tried and I couldn't re-establish my BGP sessions with anyone but the route servers when I had the $new_ip as my primary and the old as my secondary, so I just put on $new_ip as a secondary. Did you try configuring the secondary IP in the update-source parameter for your old-ip peers? > > BTW, I am still connected to the route servers with the new IP, I now just need to work on moving over my direct sessions with Spiralight, TDS, and HE. After that, then I could swap my primary and secondary BUT then my route advertisements would have $new_ip next hop so I really need to wait for everyone else to add the new IP to their gear. > > -James > >> On Sep 6, 2012, at 02:21 , Jeremy Lumby<jlumby(a)MNVOIP.COM> wrote: >> >> >>> I agree, however for any members running Cisco IOS, this opportunity has already passed during the switch swap outage, since you cannot move the new ip from secondary to primary without causing an outage. I just hoped they used the planned outage to make the new IP primary, and the old one Secondary. >>> >>> -----Original Message----- >>> From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Owen DeLong >>> Sent: Wednesday, September 05, 2012 11:29 PM >>> To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET >>> Subject: Re: [MICE-DISCUSS] MICE Traffic Lower >>> >>> If we do the renumbering right, you shouldn't have to endure an outage. >>> >>> You should be able to bring up each new session, wait for it to stabilize, then turn down the old one and not lose a single packet. >>> >>> Owen >>> >>> On Sep 5, 2012, at 20:57 , Jeremy Lumby<jlumby(a)MNVOIP.COM> wrote: >>> >>> >>>> Personally I feel that it is probably not the source of decreased traffic, however the IP address renumbering seems to have fallen by the wayside post switch swap out. I was hoping to only bring my sessions back up after renumbering, so as to not need to take them down again, however when it stalled I decided to bring the old ones back up. I would like to see some direction from the steering committee as to what time ranges they would like everyone to have the second IP by, and then what stage they would like to see the old IP removed by. If there is not direction, this will probably hang around forever. Personally I have contacted the 4 members that only were doing bi-lateral peering, and only 2 of them responded, and got our BGP sessions updated to the new IPs. Something we should all keep in mind is that the more professionally we operate, the more attractive we look to the larger carriers that are thinking of joining. I had a network engineer at one such large carrier say to me that they did not want to recommend joining MICE to management because there were no maintenance fees, and therefore they were afraid that MICE may not be around long term for lack of reliable maintenance funds. >>>> >>>> Jeremy >>>> >>>> ###################################################################### >>>> ## >>>> >>>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>>> >>> ######################################################################## >>> >>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>> >>> ######################################################################## >>> >>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>> >> ######################################################################## >> >> To unsubscribe from the MICE-DISCUSS list, click the following link: >> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >> >> >> > > ######################################################################## > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ######################################################################## To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Sept. 6, 2012

Re: MICE Traffic Lower

by James Stahr

On 9/6/2012 11:57 AM, Owen DeLong wrote: > There was certainly plenty of discussion of that particular topic to enable them to do so. > > Owen > > I tried and I couldn't re-establish my BGP sessions with anyone but the route servers when I had the $new_ip as my primary and the old as my secondary, so I just put on $new_ip as a secondary. BTW, I am still connected to the route servers with the new IP, I now just need to work on moving over my direct sessions with Spiralight, TDS, and HE. After that, then I could swap my primary and secondary BUT then my route advertisements would have $new_ip next hop so I really need to wait for everyone else to add the new IP to their gear. -James > On Sep 6, 2012, at 02:21 , Jeremy Lumby<jlumby(a)MNVOIP.COM> wrote: > > >> I agree, however for any members running Cisco IOS, this opportunity has already passed during the switch swap outage, since you cannot move the new ip from secondary to primary without causing an outage. I just hoped they used the planned outage to make the new IP primary, and the old one Secondary. >> >> -----Original Message----- >> From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Owen DeLong >> Sent: Wednesday, September 05, 2012 11:29 PM >> To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET >> Subject: Re: [MICE-DISCUSS] MICE Traffic Lower >> >> If we do the renumbering right, you shouldn't have to endure an outage. >> >> You should be able to bring up each new session, wait for it to stabilize, then turn down the old one and not lose a single packet. >> >> Owen >> >> On Sep 5, 2012, at 20:57 , Jeremy Lumby<jlumby(a)MNVOIP.COM> wrote: >> >> >>> Personally I feel that it is probably not the source of decreased traffic, however the IP address renumbering seems to have fallen by the wayside post switch swap out. I was hoping to only bring my sessions back up after renumbering, so as to not need to take them down again, however when it stalled I decided to bring the old ones back up. I would like to see some direction from the steering committee as to what time ranges they would like everyone to have the second IP by, and then what stage they would like to see the old IP removed by. If there is not direction, this will probably hang around forever. Personally I have contacted the 4 members that only were doing bi-lateral peering, and only 2 of them responded, and got our BGP sessions updated to the new IPs. Something we should all keep in mind is that the more professionally we operate, the more attractive we look to the larger carriers that are thinking of joining. I had a network engineer at one such large carrier say to me that they did not want to recommend joining MICE to management because there were no maintenance fees, and therefore they were afraid that MICE may not be around long term for lack of reliable maintenance funds. >>> >>> Jeremy >>> >>> ###################################################################### >>> ## >>> >>> To unsubscribe from the MICE-DISCUSS list, click the following link: >>> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >>> >> ######################################################################## >> >> To unsubscribe from the MICE-DISCUSS list, click the following link: >> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >> >> ######################################################################## >> >> To unsubscribe from the MICE-DISCUSS list, click the following link: >> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 >> > ######################################################################## > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > > > ######################################################################## To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Sept. 6, 2012

Re: MICE Traffic Lower

by Owen DeLong

> > First task. > > We've pretty much settled on IPv4 addressing the same keeping the > last octet. We need to settle on IPv6 addressing. > > Owen Delong has suggested that we embed the ASN and switch & port into > the IPv6 address, others have said it isn't needed. We have such large > space, we can do just about any scheme. I'd just have to precalculate > each IPv6 address and send it off to everybody. > Almost... What I suggested was embedding the ASN. Nothing about switch/port because that could be variable and you don't want to have to redo all your peering just for a port move. I suggested <prefix>:0:<AS:NO>:<instance> Where: <prefix> is the IPv6 /64 prefix <AS:NO> is the 32-bit AS Number (0:ASN for 16 bit ASNs) <instance> is a member-chosen instance number for that particular connection <instance> would only really matter for members that have more than one connection, and most would likely choose 0 or 1 for their first (or only) connection and count up from there. > We'll just start by taking the first IPv6 network, leaving the others > in reserve. > I support this. > Would it be nice to have something like 2001:504:27:0:0:1e49:1:3/64 > (in particular to identify ipHouse), vs. 2001:504:27:0:0:1055:1:4/64 > (to identify TDS?). > As suggested above, I'd rather see it be like: ipHouse: 2001:504:27::1e49:<instance>/64 TDS: 2001:504:27::1055:<instance>/64 I think 16 bits is more than enough to describe the instance number. You run into a potential problem (though not particularly important in this case) where 32-bit ASNs could create addresses with 1s in the first 12 bits of the suffix (which is theoretically bad form for static assignments in IPv6) if you put non-ASN info in the last 2 quartets. That's one of the reasons I suggest putting 0 in the first quartet and putting the ASN in the next 2 quartets followed by a 16-bit instance number in the last quartet. > Or just match up how things are now with 2001:504:27:0::3 & ::4? > (the existing IP addresses started out with port # at the end, but of > course things had to change up as we went along, the port #s would stay > correct on IPv6). > > Do the same with the route servers? Or make them shorter? 2001:504:27::1? Presumably the route servers have an ASN. I'd make them 2001:504:27::<AS:NO>:<i> where <i> would either be {0,1} or {1,2} depending on what fraction of the group prefers to count from 1 vs. count from 0. > > > Second task. > Scheduling. > > In order for people to start announcing prefixes with nexthop > addresses as the new IPs, everybody needs to at least have the new > prefix on as a secondary or another IP address on their MICE facing > interfaces to reach them. Not everybody has done that yet, although > many did do the IPv4 portion already. > > How about we set a drop-dead due date of 10/1/2012 for allowing > everybody to do this in whatever maintenance window they see > appropriate. 3 weeks out is pretty reasonable I think, but I don't > know other's policies on scheduling their windows if they want to > minimize whatever impact this causes (minimal I think). > Seems reasonable to me. > > Third task. > BGP cutover. > > The route servers are listening now on 206.108.255.1 & 206.108.255.2 as > well as the old IPs. BIRD hasn't been restarted yet though. > I plan on creating new BIRD configs with the router-id in 206.108.255.0 > & 2001:504:27::/64 for every member. > > This should allow any member past the secondary IP day to change to > connect to the new IP, announce as their new IP, and have their > prefixes be reachable by all members. This can happen on their > schedule at any time after the 10/1/2012 date. > Seems reasonable. > Bilateral peering members can update between themselves at any point > in time once they have their IP assignments if their gear supports > that level of control, but multilateral peering to the route servers > should still be announcing the old IP addresses as next-hop for now > until after the 10/1/2012 date. > Agreed. > > Final task. > Final cleanup of old IP addressing. > > As people cutover after the 10/1/2012 date, I expect them to swap > primary and secondary IP addresses in their MICE facing interfaces > (assuming cisco config here) but until everybody is cutover, > members will probably want to retain a secondary in the old range in > order to reach people that haven't changed their BGP setup yet. > Hopefully most people did the primary/secondary swap during the switch cutover and put the new IPv4 address in as secondary. I don't believe this is an issue for IPv6. > Should we set a date of 1/1/2013 as final cleanup and IP address turn > in day back to Airstream? After that, nobody should be announcing any > prefixes with next hop in the old IP address range. Members should > make sure that they remove the secondary IP address after this date on > whatever maintenance window is required (again should be minimal > impact as this is now the secondary IP address). > I think 1/1/2013 is awfully far out and also problematic due to the intersection with holidays, etc. I would suggest 11/15/2012 as a much more realistic date because it gets things done with some room to address issues before things start heading into the holiday-dense period from Thanksgiving through new-years. How about breaking this up a little-bit... 11/15/2012 is the day we turn off the old addresses on the route servers and expect everyone to have turned down all their peering sessions on the old addresses. 1/1/2013 is the day we hand the addresses back to Air Stream. Between 11/15/2012 and 1/1/2013, we should occasionally ping-sweep and harass any stragglers that haven't turned off the AirStream addresses yet. > Failure to remove the secondary could impact Airstream in the future > if they reassign these IP blocks, so this should be completed at some point. > Yes, we should be good neighbors. > > > Questions? > > Are their any concerns or questions about this schedule and plan? See above in-line. > Anything anybody would do differently? IBID > Are the schedule dates reasonable? Mostly. See above suggested tweaks. Thanks for writing this up and spearheading this, Doug. Owen ######################################################################## To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Sept. 6, 2012

Re: MICE Announce list issue

by Mike Horwath

On Thu, Sep 06, 2012 at 11:33:00AM -0500, Jeremy Lumby wrote: > I am unable to subscribe to the MICE announce list. When I go to > the web page to subscribe, it says that I will be sent a > confirmation email. I never get it, and it does not show up in my > spam folder. I tried several times last week. Weird. But you're subscribed now. -- Mike Horwath ipHouse - Welcome home! drechsau(a)iphouse.net The universe is an island, surrounded by whatever it is that surrounds universes. - Berkeley Fortune ######################################################################## To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Sept. 6, 2012

Re: MICE Traffic Lower

by Owen DeLong

There was certainly plenty of discussion of that particular topic to enable them to do so. Owen On Sep 6, 2012, at 02:21 , Jeremy Lumby <jlumby(a)MNVOIP.COM> wrote: > I agree, however for any members running Cisco IOS, this opportunity has already passed during the switch swap outage, since you cannot move the new ip from secondary to primary without causing an outage. I just hoped they used the planned outage to make the new IP primary, and the old one Secondary. > > -----Original Message----- > From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Owen DeLong > Sent: Wednesday, September 05, 2012 11:29 PM > To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET > Subject: Re: [MICE-DISCUSS] MICE Traffic Lower > > If we do the renumbering right, you shouldn't have to endure an outage. > > You should be able to bring up each new session, wait for it to stabilize, then turn down the old one and not lose a single packet. > > Owen > > On Sep 5, 2012, at 20:57 , Jeremy Lumby <jlumby(a)MNVOIP.COM> wrote: > >> Personally I feel that it is probably not the source of decreased traffic, however the IP address renumbering seems to have fallen by the wayside post switch swap out. I was hoping to only bring my sessions back up after renumbering, so as to not need to take them down again, however when it stalled I decided to bring the old ones back up. I would like to see some direction from the steering committee as to what time ranges they would like everyone to have the second IP by, and then what stage they would like to see the old IP removed by. If there is not direction, this will probably hang around forever. Personally I have contacted the 4 members that only were doing bi-lateral peering, and only 2 of them responded, and got our BGP sessions updated to the new IPs. Something we should all keep in mind is that the more professionally we operate, the more attractive we look to the larger carriers that are thinking of joining. I had a network engineer at one such large carrier say to me that they did not want to recommend joining MICE to management because there were no maintenance fees, and therefore they were afraid that MICE may not be around long term for lack of reliable maintenance funds. >> >> Jeremy >> >> ###################################################################### >> ## >> >> To unsubscribe from the MICE-DISCUSS list, click the following link: >> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > > ######################################################################## > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 > > ######################################################################## > > To unsubscribe from the MICE-DISCUSS list, click the following link: > http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ######################################################################## To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Sept. 6, 2012

Re: MICE Traffic Lower

by Doug McIntyre

On Wed, Sep 05, 2012 at 10:57:00PM -0500, Jeremy Lumby wrote: > ..however the IP address renumbering seems to have fallen by the wayside.. Since I volunteered for this, I've tried several times internally to discuss the direction to go, but maybe we should just take it for public discussion instead of waiting more for my cohorts. I didn't know the schedule for the switch cutover until the day it happened, otherwise I would have tried to get more things done ahead of time. I've previously posted an outline of my thoughts for cutover, maybe we'll just make them firm in this posting. First off, once things are settled, I won't just depend on a list posting for distributing info, I'll email the peering email address of each member with their specific info, so please confirm back that you've received it. First task. We've pretty much settled on IPv4 addressing the same keeping the last octet. We need to settle on IPv6 addressing. Owen Delong has suggested that we embed the ASN and switch & port into the IPv6 address, others have said it isn't needed. We have such large space, we can do just about any scheme. I'd just have to precalculate each IPv6 address and send it off to everybody. We'll just start by taking the first IPv6 network, leaving the others in reserve. Would it be nice to have something like 2001:504:27:0:0:1e49:1:3/64 (in particular to identify ipHouse), vs. 2001:504:27:0:0:1055:1:4/64 (to identify TDS?). Or just match up how things are now with 2001:504:27:0::3 & ::4? (the existing IP addresses started out with port # at the end, but of course things had to change up as we went along, the port #s would stay correct on IPv6). Do the same with the route servers? Or make them shorter? 2001:504:27::1? Second task. Scheduling. In order for people to start announcing prefixes with nexthop addresses as the new IPs, everybody needs to at least have the new prefix on as a secondary or another IP address on their MICE facing interfaces to reach them. Not everybody has done that yet, although many did do the IPv4 portion already. How about we set a drop-dead due date of 10/1/2012 for allowing everybody to do this in whatever maintenance window they see appropriate. 3 weeks out is pretty reasonable I think, but I don't know other's policies on scheduling their windows if they want to minimize whatever impact this causes (minimal I think). Third task. BGP cutover. The route servers are listening now on 206.108.255.1 & 206.108.255.2 as well as the old IPs. BIRD hasn't been restarted yet though. I plan on creating new BIRD configs with the router-id in 206.108.255.0 & 2001:504:27::/64 for every member. This should allow any member past the secondary IP day to change to connect to the new IP, announce as their new IP, and have their prefixes be reachable by all members. This can happen on their schedule at any time after the 10/1/2012 date. Bilateral peering members can update between themselves at any point in time once they have their IP assignments if their gear supports that level of control, but multilateral peering to the route servers should still be announcing the old IP addresses as next-hop for now until after the 10/1/2012 date. Final task. Final cleanup of old IP addressing. As people cutover after the 10/1/2012 date, I expect them to swap primary and secondary IP addresses in their MICE facing interfaces (assuming cisco config here) but until everybody is cutover, members will probably want to retain a secondary in the old range in order to reach people that haven't changed their BGP setup yet. Should we set a date of 1/1/2013 as final cleanup and IP address turn in day back to Airstream? After that, nobody should be announcing any prefixes with next hop in the old IP address range. Members should make sure that they remove the secondary IP address after this date on whatever maintenance window is required (again should be minimal impact as this is now the secondary IP address). Failure to remove the secondary could impact Airstream in the future if they reassign these IP blocks, so this should be completed at some point. Questions? Are their any concerns or questions about this schedule and plan? Anything anybody would do differently? Are the schedule dates reasonable? -- Doug McIntyre <merlyn(a)iphouse.net> -- ipHouse/Goldengate/Bitstream/ProNS -- Network Engineer/Provisioning/Jack of all Trades ######################################################################## To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Sept. 6, 2012

MICE Announce list issue

by Jeremy Lumby

I am unable to subscribe to the MICE announce list. When I go to the web page to subscribe, it says that I will be sent a confirmation email. I never get it, and it does not show up in my spam folder. I tried several times last week. ######################################################################## To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Sept. 6, 2012

Re: MICE Traffic Lower

by Jay Hanke

On Thu, Sep 6, 2012 at 10:48 AM, Mike Horwath <drechsau(a)iphouse.net> wrote: > On Thu, Sep 06, 2012 at 08:50:26AM -0500, Jay Hanke wrote: >> On Wed, Sep 5, 2012 at 11:06 PM, Mike Horwath <drechsau(a)iphouse.net> wrote: >> > Where is it at? :) (Hi Jay!) >> > That was about the UG, not the renumbering. > There was an email yesterday... http://lists.iphouse.net/cgi-bin/wa?A2=MICE-ANNOUNCE;690b5a89.1209 Info also available at: http://micemn.net/ ######################################################################## To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Sept. 6, 2012

Re: MICE Traffic Lower

by Mike Horwath

On Thu, Sep 06, 2012 at 08:50:26AM -0500, Jay Hanke wrote: > On Wed, Sep 5, 2012 at 11:06 PM, Mike Horwath <drechsau(a)iphouse.net> wrote: > > Where is it at? :) (Hi Jay!) > > Doug and Anthony are running this, I'll defer to them regarding the > renumbering. That was about the UG, not the renumbering. -- Mike Horwath ipHouse - Welcome home! drechsau(a)iphouse.net The universe is an island, surrounded by whatever it is that surrounds universes. - Berkeley Fortune ######################################################################## To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Sept. 6, 2012

Re: MICE Traffic Lower

by Jay Hanke

On Wed, Sep 5, 2012 at 11:06 PM, Mike Horwath <drechsau(a)iphouse.net> wrote: > On Wed, Sep 05, 2012 at 10:57:00PM -0500, Jeremy Lumby wrote: >> I had a network engineer at one such large carrier say to me that >> they did not wa nt to recommend joining MICE to management because >> there were no maintenance fees, and therefore they were afraid that >> MICE may not be around long term for lack of reliable maintenance >> funds. I have run into this as well. Items like this are significantly hindering our growth. > MICE UG will be a great place to discuss things. I partially disagree. Waiting to discus or research these issues until a UG will lead to endless cycles of waiting and discussion. We need to have actionable items and consensus built prior to the meeting and use the meeting to vote. If you can't be at the meeting in person assign a proxy in advance. Almost all of our progress to date has been individuals grabbing some problem and taking care of it semi-independently. We need to come to the UG with proposals ready or almost ready for adoption. <snip/> > Where is it at? :) (Hi Jay!) Doug and Anthony are running this, I'll defer to them regarding the renumbering. jay ######################################################################## To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1

Sept. 6, 2012