As everyone will recall the exchange has had problems with the main switch flooding unicast traffic a couple times in the last few years. Juniper believes we're encountering a known bug that has been fixed in later software releases. Here's the bug description for people with Juniper support ids, as we'd suspected it deals with L2 database MAC learning/maintaining: https://prsearch.juniper.net/InfoCenter/index?page=prcontent&id=PR695200 The upgrade will require a short interruption in exchange traffic. We're currently planning on performing the upgrade next week on Wednesday morning - July 29th between 5am and 7am, that should be the low point in our daily traffic volume. Can someone forward this to the Announce list? Cheers, anthony Anthony Anderberg Sr. Systems Analyst [NUtel_email_logo_1] 320-234-5239 anthonyanderberg@nu-telecom.net<mailto:anthonyanderberg@nu-telecom.net> www.nutelecom.net<http://www.nutelecom.net>
Will you be turning down the route server BGP sessions in advance of rebooting the switch? -- Richard
On Mon, Jul 20, 2015 at 12:48:16PM -0500, Richard Laager wrote:
Will you be turning down the route server BGP sessions in advance of rebooting the switch?
Wouldn't that be just as disruptive as just doing the switch reboot after upgrade? Ie. individual members could do so on the schedule they wanted to control the schedule themselves, but I guess I don't see a benefit for the central route servers. -- Doug McIntyre <merlyn@iphouse.net> ~.~ ipHouse ~.~ Network Engineer/Provisioning/Jack of all Trades
I am in favor of turning down the route servers since it would gracefully stop the traffic flow allowing the members routers to automatically reroute traffic before the switch reboot cuts everyone off. The only issue is it does not address the bi-lateral traffic, and therefore the members that care would be forced to turn down BGP on their own anyway. So for simplicity sake, it would be easiest to leave it up to the individual members. I could see it working either way. -----Original Message----- From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Doug McIntyre Sent: Monday, July 20, 2015 12:55 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] MICE Switch Upgrade On Mon, Jul 20, 2015 at 12:48:16PM -0500, Richard Laager wrote:
Will you be turning down the route server BGP sessions in advance of rebooting the switch?
Wouldn't that be just as disruptive as just doing the switch reboot after upgrade? Ie. individual members could do so on the schedule they wanted to control the schedule themselves, but I guess I don't see a benefit for the central route servers. -- Doug McIntyre <merlyn@iphouse.net> ~.~ ipHouse ~.~ Network Engineer/Provisioning/Jack of all Trades
On Jul 20, 2015, at 12:54 PM, Doug McIntyre <merlyn@IPHOUSE.NET> wrote:
Wouldn't that be just as disruptive as just doing the switch reboot after upgrade?
Shutting down the route server sessions would at least let BGP hold timers expire gracefully while traffic can still be forwarded for the duration of the hold timer. Probably more important for those folks that are layer 2 extended into the IX and don't necessarily have any notion of physical link to the switch. -- Andrew Hoyos hoyosa@gmail.com
The Seattle SIX exchange tried out this technique for their last software upgrade. Seemed like an interesting solution. https://ripe67.ripe.net/presentations/374-WH-IXPMaintReduce.pdf On Mon, Jul 20, 2015 at 01:11:34PM -0500, Andrew Hoyos wrote:
Shutting down the route server sessions would at least let BGP hold timers expire gracefully while traffic can still be forwarded for the duration of the hold timer.
Probably more important for those folks that are layer 2 extended into the IX and don't necessarily have any notion of physical link to the switch.
-- Andrew Hoyos hoyosa@gmail.com
-- Jeff Bartig University of Wisconsin jeffb@doit.wisc.edu (608) 262-8336
As a reminder - I'm still planning to do the MICE exchange upgrade tomorrow morning between 5am and 7am CDT. I'll be shutting down the route server ahead of time, but as we talked about those of you with bilateral sessions or sensitive SLAs may want to take action ahead of time. I'll send a note out when everything is complete. Cheers, anthony Anthony Anderberg Sr. Systems Analyst [NUtel_email_logo_1] 320-234-5239 anthonyanderberg@nu-telecom.net<mailto:anthonyanderberg@nu-telecom.net> www.nutelecom.net<http://www.nutelecom.net> From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Anthony Anderberg Sent: Monday, July 20, 2015 12:24 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] MICE Switch Upgrade As everyone will recall the exchange has had problems with the main switch flooding unicast traffic a couple times in the last few years. Juniper believes we're encountering a known bug that has been fixed in later software releases. Here's the bug description for people with Juniper support ids, as we'd suspected it deals with L2 database MAC learning/maintaining: https://prsearch.juniper.net/InfoCenter/index?page=prcontent&id=PR695200 The upgrade will require a short interruption in exchange traffic. We're currently planning on performing the upgrade next week on Wednesday morning - July 29th between 5am and 7am, that should be the low point in our daily traffic volume. Can someone forward this to the Announce list? Cheers, anthony Anthony Anderberg Sr. Systems Analyst [NUtel_email_logo_1] 320-234-5239 anthonyanderberg@nu-telecom.net<mailto:anthonyanderberg@nu-telecom.net> www.nutelecom.net<http://www.nutelecom.net> ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
This upgrade is complete, looks like routes and traffic are returning to normal levels. Give me a call if anything seems wrong or silly. 320-583-9604 Cheers, anthony From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Anthony Anderberg Sent: Tuesday, July 28, 2015 3:03 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] FW: MICE Switch Upgrade As a reminder - I'm still planning to do the MICE exchange upgrade tomorrow morning between 5am and 7am CDT. I'll be shutting down the route server ahead of time, but as we talked about those of you with bilateral sessions or sensitive SLAs may want to take action ahead of time. I'll send a note out when everything is complete. Cheers, anthony Anthony Anderberg Sr. Systems Analyst [NUtel_email_logo_1] 320-234-5239 anthonyanderberg@nu-telecom.net<mailto:anthonyanderberg@nu-telecom.net> www.nutelecom.net<http://www.nutelecom.net> From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Anthony Anderberg Sent: Monday, July 20, 2015 12:24 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: [MICE-DISCUSS] MICE Switch Upgrade As everyone will recall the exchange has had problems with the main switch flooding unicast traffic a couple times in the last few years. Juniper believes we're encountering a known bug that has been fixed in later software releases. Here's the bug description for people with Juniper support ids, as we'd suspected it deals with L2 database MAC learning/maintaining: https://prsearch.juniper.net/InfoCenter/index?page=prcontent&id=PR695200 The upgrade will require a short interruption in exchange traffic. We're currently planning on performing the upgrade next week on Wednesday morning - July 29th between 5am and 7am, that should be the low point in our daily traffic volume. Can someone forward this to the Announce list? Cheers, anthony Anthony Anderberg Sr. Systems Analyst [NUtel_email_logo_1] 320-234-5239 anthonyanderberg@nu-telecom.net<mailto:anthonyanderberg@nu-telecom.net> www.nutelecom.net<http://www.nutelecom.net> ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
Thanks for the work Anthony. Best, -M< On Jul 29, 2015, at 7:46 AM, Anthony Anderberg <anthonyanderberg@NU-TELECOM.NET<mailto:anthonyanderberg@NU-TELECOM.NET>> wrote: This upgrade is complete, looks like routes and traffic are returning to normal levels. Give me a call if anything seems wrong or silly. 320-583-9604 Cheers, anthony From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Anthony Anderberg Sent: Tuesday, July 28, 2015 3:03 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: [MICE-DISCUSS] FW: MICE Switch Upgrade As a reminder – I’m still planning to do the MICE exchange upgrade tomorrow morning between 5am and 7am CDT. I’ll be shutting down the route server ahead of time, but as we talked about those of you with bilateral sessions or sensitive SLAs may want to take action ahead of time. I’ll send a note out when everything is complete. Cheers, anthony Anthony Anderberg Sr. Systems Analyst <image001.gif> 320-234-5239 anthonyanderberg@nu-telecom.net<mailto:anthonyanderberg@nu-telecom.net> www.nutelecom.net<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.nutelecom.net&d=AwMFAg&c=96ZbZZcaMF4w0F4jpN6LZg&r=XDN_BIPGnpb6V0w5M9FADw&m=-6sRxjDzCBLMHS7TFxUdcNnfhlEjWgiqCyaaVEojWUQ&s=hMqBu45LV-XZM3tvSUym0pDNafGVmbY_7L3qs4GvHjI&e=> From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Anthony Anderberg Sent: Monday, July 20, 2015 12:24 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: [MICE-DISCUSS] MICE Switch Upgrade As everyone will recall the exchange has had problems with the main switch flooding unicast traffic a couple times in the last few years. Juniper believes we’re encountering a known bug that has been fixed in later software releases. Here’s the bug description for people with Juniper support ids, as we’d suspected it deals with L2 database MAC learning/maintaining: https://prsearch.juniper.net/InfoCenter/index?page=prcontent&id=PR695200<https://urldefense.proofpoint.com/v2/url?u=https-3A__prsearch.juniper.net_InfoCenter_index-3Fpage-3Dprcontent-26id-3DPR695200&d=AwMFAg&c=96ZbZZcaMF4w0F4jpN6LZg&r=XDN_BIPGnpb6V0w5M9FADw&m=-6sRxjDzCBLMHS7TFxUdcNnfhlEjWgiqCyaaVEojWUQ&s=vHTSwmZB9IIDptAdydSERlk5T3aenworEwvnnqjaZwg&e=> The upgrade will require a short interruption in exchange traffic. We’re currently planning on performing the upgrade next week on Wednesday morning - July 29th between 5am and 7am, that should be the low point in our daily traffic volume. Can someone forward this to the Announce list? Cheers, anthony Anthony Anderberg Sr. Systems Analyst <image001.gif> 320-234-5239 anthonyanderberg@nu-telecom.net<mailto:anthonyanderberg@nu-telecom.net> www.nutelecom.net<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.nutelecom.net&d=AwMFAg&c=96ZbZZcaMF4w0F4jpN6LZg&r=XDN_BIPGnpb6V0w5M9FADw&m=-6sRxjDzCBLMHS7TFxUdcNnfhlEjWgiqCyaaVEojWUQ&s=hMqBu45LV-XZM3tvSUym0pDNafGVmbY_7L3qs4GvHjI&e=> ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.iphouse.net_cgi-2Dbin_wa-3FSUBED1-3DMICE-2DDISCUSS-26A-3D1&d=AwMFAg&c=96ZbZZcaMF4w0F4jpN6LZg&r=XDN_BIPGnpb6V0w5M9FADw&m=-6sRxjDzCBLMHS7TFxUdcNnfhlEjWgiqCyaaVEojWUQ&s=wABXCAUAc8oWnRQ-NsWR1dccGNuqzh9QTnplE0FBEB8&e=> ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.iphouse.net_cgi-2Dbin_wa-3FSUBED1-3DMICE-2DDISCUSS-26A-3D1&d=AwMFAg&c=96ZbZZcaMF4w0F4jpN6LZg&r=XDN_BIPGnpb6V0w5M9FADw&m=-6sRxjDzCBLMHS7TFxUdcNnfhlEjWgiqCyaaVEojWUQ&s=wABXCAUAc8oWnRQ-NsWR1dccGNuqzh9QTnplE0FBEB8&e=> ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.iphouse.net_cgi-2Dbin_wa-3FSUBED1-3DMICE-2DDISCUSS-26A-3D1&d=AwMFAg&c=96ZbZZcaMF4w0F4jpN6LZg&r=XDN_BIPGnpb6V0w5M9FADw&m=-6sRxjDzCBLMHS7TFxUdcNnfhlEjWgiqCyaaVEojWUQ&s=wABXCAUAc8oWnRQ-NsWR1dccGNuqzh9QTnplE0FBEB8&e=>
Thank You for the work. Dan Olsen- Operations Manager Windomnet 443 10th Street, PO box 38 Windom MN 56101 From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Anthony Anderberg Sent: Wednesday, July 29, 2015 6:47 AM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] MICE Switch Upgrade This upgrade is complete, looks like routes and traffic are returning to normal levels. Give me a call if anything seems wrong or silly. 320-583-9604 Cheers, anthony From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Anthony Anderberg Sent: Tuesday, July 28, 2015 3:03 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET <mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: [MICE-DISCUSS] FW: MICE Switch Upgrade As a reminder - I'm still planning to do the MICE exchange upgrade tomorrow morning between 5am and 7am CDT. I'll be shutting down the route server ahead of time, but as we talked about those of you with bilateral sessions or sensitive SLAs may want to take action ahead of time. I'll send a note out when everything is complete. Cheers, anthony Anthony Anderberg Sr. Systems Analyst 320-234-5239 <mailto:anthonyanderberg@nu-telecom.net> anthonyanderberg@nu-telecom.net <http://www.nutelecom.net> www.nutelecom.net From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Anthony Anderberg Sent: Monday, July 20, 2015 12:24 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET <mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: [MICE-DISCUSS] MICE Switch Upgrade As everyone will recall the exchange has had problems with the main switch flooding unicast traffic a couple times in the last few years. Juniper believes we're encountering a known bug that has been fixed in later software releases. Here's the bug description for people with Juniper support ids, as we'd suspected it deals with L2 database MAC learning/maintaining: https://prsearch.juniper.net/InfoCenter/index?page=prcontent <https://prsearch.juniper.net/InfoCenter/index?page=prcontent&id=PR695200> &id=PR695200 The upgrade will require a short interruption in exchange traffic. We're currently planning on performing the upgrade next week on Wednesday morning - July 29th between 5am and 7am, that should be the low point in our daily traffic volume. Can someone forward this to the Announce list? Cheers, anthony Anthony Anderberg Sr. Systems Analyst 320-234-5239 <mailto:anthonyanderberg@nu-telecom.net> anthonyanderberg@nu-telecom.net <http://www.nutelecom.net> www.nutelecom.net _____ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS <http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1> &A=1 _____ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS <http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1> &A=1 _____ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS <http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1> &A=1
This upgrade is complete, looks like routes and traffic are returning to normal levels. Give me a call if anything seems wrong or silly. 320-583-9604 Cheers, anthony From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Anthony Anderberg Sent: Tuesday, July 28, 2015 3:03 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] FW: MICE Switch Upgrade As a reminder - I'm still planning to do the MICE exchange upgrade tomorrow morning between 5am and 7am CDT. I'll be shutting down the route server ahead of time, but as we talked about those of you with bilateral sessions or sensitive SLAs may want to take action ahead of time. I'll send a note out when everything is complete. Cheers, anthony Anthony Anderberg Sr. Systems Analyst [NUtel_email_logo_1] 320-234-5239 anthonyanderberg@nu-telecom.net<mailto:anthonyanderberg@nu-telecom.net> www.nutelecom.net<http://www.nutelecom.net> From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Anthony Anderberg Sent: Monday, July 20, 2015 12:24 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: [MICE-DISCUSS] MICE Switch Upgrade As everyone will recall the exchange has had problems with the main switch flooding unicast traffic a couple times in the last few years. Juniper believes we're encountering a known bug that has been fixed in later software releases. Here's the bug description for people with Juniper support ids, as we'd suspected it deals with L2 database MAC learning/maintaining: https://prsearch.juniper.net/InfoCenter/index?page=prcontent&id=PR695200 The upgrade will require a short interruption in exchange traffic. We're currently planning on performing the upgrade next week on Wednesday morning - July 29th between 5am and 7am, that should be the low point in our daily traffic volume. Can someone forward this to the Announce list? Cheers, anthony Anthony Anderberg Sr. Systems Analyst [NUtel_email_logo_1] 320-234-5239 anthonyanderberg@nu-telecom.net<mailto:anthonyanderberg@nu-telecom.net> www.nutelecom.net<http://www.nutelecom.net> ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
participants (9)
-
Andrew Hoyos -
Anthony Anderberg -
Dan Olsen -
Doug McIntyre -
Hannigan, Martin -
Jeff Bartig -
Jeremy Lumby -
Mike Horwath -
Richard Laager