July 2015 - mice-discuss - lists.micemn.net

Re: Fwd: chn00102685,-,mice,l3ppc,-,tge,MICE_public_switch_fabric is DOWN

by Jeremy Lumby

>From what I can tell remotely the Juniper 4500 (10 gig switch) is down. I have BGP established to the route servers (they and I are both on the 4200) Is anyone close by that can take a look? -----Original Message----- From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Richard Laager Sent: Friday, July 24, 2015 7:12 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Fwd: chn00102685,-,mice,l3ppc,-,tge,MICE_public_switch_fabric is DOWN All of my MICE BGP sessions have been down since approximately 18:41. For reference, I'm connected to the Mankato Networks switch. -- Richard

July 25, 2015

Re: Fwd: chn00102685,-,mice,l3ppc,-,tge,MICE_public_switch_fabric is DOWN

by Richard Laager

All of my MICE BGP sessions have been down since approximately 18:41. For reference, I'm connected to the Mankato Networks switch. -- Richard

July 25, 2015

Fwd: chn00102685,-,mice,l3ppc,-,tge,MICE_public_switch_fabric is DOWN

by Jason Hanke

can someone take a peek? ---------- Forwarded message ---------- From: NOC IP <svc_NNOCIP(a)charter.com> Date: Fri, Jul 24, 2015 at 7:04 PM Subject: chn00102685,-,mice,l3ppc,-,tge,MICE_public_switch_fabric is DOWN To: "peering(a)micemn.net" <peering(a)micemn.net> MICE, Charter Communications is not getting any incoming light at the Colo link. Could you please check this link and reply with any issues from your end? Thank you, *[image: CharterCommunicatio#B375E30]* *Katie McCarthy, *CCNA | Network Operations Technician II | 866.248.7662 Opt. 2 9333 Dielman Industrial Drive | St Louis, MO 63132 *Please do your part to reduce unnecessary email volume by directing your request to the appropriate NOC group using the following distribution lists:* · *Critical Facilities*: DLCriticalFacilities(a)Charter.com · *IP, CMTS, Optical*: DLNOCIP(a)Charter.Com · *Video*: DLNOCVideo(a)Charter.com · *Voice*: DLNOCVoice(a)Charter.com -- Jay Hanke CTO Neutral Path Communications 3 Civic Center Plaza, Suite 204 Mankato, MN 56001 (507) 327-2398 mobile jayhanke(a)neutralpath.net www.neutralpath.net

July 25, 2015

CNS Switch - Cacti Graph Anomalies

by Justin Krejci

Looks like there is maybe some kind of trouble with the CNS Switch. There are a lot of cacti graphing gaps displayed but only for the ports being queried on the CNS switch itself. Looks like it started around 2pm yesterday. Looking at the LAG port graph off the main switch, it does not appear to be affecting actual member traffic so it may be a management issue with the CNS switch or some other control plane issue possibly. Could be indicative of a larger, brewing problem.

July 23, 2015

Re: MICE Switch Upgrade

by Jeff Bartig

The Seattle SIX exchange tried out this technique for their last software upgrade. Seemed like an interesting solution. https://ripe67.ripe.net/presentations/374-WH-IXPMaintReduce.pdf On Mon, Jul 20, 2015 at 01:11:34PM -0500, Andrew Hoyos wrote: > Shutting down the route server sessions would at least let BGP hold timers expire gracefully while traffic can still be forwarded for the duration of the hold timer. > > Probably more important for those folks that are layer 2 extended into the IX and don't necessarily have any notion of physical link to the switch. > > -- > Andrew Hoyos > hoyosa(a)gmail.com -- Jeff Bartig University of Wisconsin jeffb(a)doit.wisc.edu (608) 262-8336

July 21, 2015

Re: MICE Switch Upgrade

by Andrew Hoyos

> On Jul 20, 2015, at 12:54 PM, Doug McIntyre <merlyn(a)IPHOUSE.NET> wrote: > > Wouldn't that be just as disruptive as just doing the switch reboot after upgrade? Shutting down the route server sessions would at least let BGP hold timers expire gracefully while traffic can still be forwarded for the duration of the hold timer. Probably more important for those folks that are layer 2 extended into the IX and don't necessarily have any notion of physical link to the switch. -- Andrew Hoyos hoyosa(a)gmail.com

July 20, 2015

Re: MICE Switch Upgrade

by Jeremy Lumby

I am in favor of turning down the route servers since it would gracefully stop the traffic flow allowing the members routers to automatically reroute traffic before the switch reboot cuts everyone off. The only issue is it does not address the bi-lateral traffic, and therefore the members that care would be forced to turn down BGP on their own anyway. So for simplicity sake, it would be easiest to leave it up to the individual members. I could see it working either way. -----Original Message----- From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Doug McIntyre Sent: Monday, July 20, 2015 12:55 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] MICE Switch Upgrade On Mon, Jul 20, 2015 at 12:48:16PM -0500, Richard Laager wrote: > Will you be turning down the route server BGP sessions in advance of > rebooting the switch? Wouldn't that be just as disruptive as just doing the switch reboot after upgrade? Ie. individual members could do so on the schedule they wanted to control the schedule themselves, but I guess I don't see a benefit for the central route servers. -- Doug McIntyre <merlyn(a)iphouse.net> ~.~ ipHouse ~.~ Network Engineer/Provisioning/Jack of all Trades

July 20, 2015

Re: MICE Switch Upgrade

by Doug McIntyre

On Mon, Jul 20, 2015 at 12:48:16PM -0500, Richard Laager wrote: > Will you be turning down the route server BGP sessions in advance of > rebooting the switch? Wouldn't that be just as disruptive as just doing the switch reboot after upgrade? Ie. individual members could do so on the schedule they wanted to control the schedule themselves, but I guess I don't see a benefit for the central route servers. -- Doug McIntyre <merlyn(a)iphouse.net> ~.~ ipHouse ~.~ Network Engineer/Provisioning/Jack of all Trades

July 20, 2015

Re: MICE Switch Upgrade

by Richard Laager

Will you be turning down the route server BGP sessions in advance of rebooting the switch? -- Richard

July 20, 2015

MICE Switch Upgrade

by Anthony Anderberg

As everyone will recall the exchange has had problems with the main switch flooding unicast traffic a couple times in the last few years. Juniper believes we're encountering a known bug that has been fixed in later software releases. Here's the bug description for people with Juniper support ids, as we'd suspected it deals with L2 database MAC learning/maintaining: https://prsearch.juniper.net/InfoCenter/index?page=prcontent&id=PR695200 The upgrade will require a short interruption in exchange traffic. We're currently planning on performing the upgrade next week on Wednesday morning - July 29th between 5am and 7am, that should be the low point in our daily traffic volume. Can someone forward this to the Announce list? Cheers, anthony Anthony Anderberg Sr. Systems Analyst [NUtel_email_logo_1] 320-234-5239 anthonyanderberg(a)nu-telecom.net<mailto:anthonyanderberg@nu-telecom.net> www.nutelecom.net<http://www.nutelecom.net>

July 20, 2015