- mice-discuss - lists.micemn.net

Please welcome Global Net to MICE!

by Jay Hanke

Name: Global Net Port: Ethernet5/37 (Main Switch) Speed: 10G ASN: 395466 IPv4: 206.108.255.97/24 IPv6: 2001:504:27:0:6:08ca::1/64 Route Server: No BFD Support: No Peering Contact: peering(a)theglobal.net Please provide your Arista compatible optic to Cologix or a member of the tech committee. Welcome aboard! -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

July 30, 2018

MICE BFD Settings

by AnthonyAnderberg＠nuvera.net

At the last UG meeting it was suggested that we document MICE's route server BFD settings, that is done now: http://www.micemn.net/technical.html Was good seeing everyone again, anthony [Anthony_Anderberg]<https://www.nuvera.net/>

July 25, 2018

Dyn blog post has some recommendations for IXPs who may experience bad actors

by Frank Bulk

The whole blog post I interesting, but the last fifth relates to IXPs: https://dyn.com/blog/shutting-down-the-bgp-hijack-factory/ Food for thought. Frank

July 12, 2018

Re: MICE UG 28

by Mike Horwath

On Fri, Jul 06, 2018 at 07:35:16PM -0500, Jay Hanke wrote: > I'm not going to make this one. Family vacation Cool. I'm in a different boat and fully scheduled up for the 11th :( -- Mike Horwath, reachable via drechsau(a)Geeks.ORG

July 9, 2018

Re: MICE UG 28

by Jay Hanke

I'm not going to make this one. Family vacation On Fri, Jul 6, 2018 at 4:55 PM, Mike Horwath <drechsau(a)geeks.org> wrote: > On Fri, Jul 06, 2018 at 03:52:39PM -0500, Richard Laager wrote: >> Reminder: UG 28 is on Wednesday of next week (July 11) at 3:00pm >> >> Remote Conferencing: 952-562-5400 passcode MICE (6423) >> >> Location Details: >> >> Alerus Bank >> 3137 32nd Ave S >> 2nd Floor Conference Room >> Fargo, ND 58103 > > hey Jay - you driving up? > > -- > Mike Horwath, reachable via drechsau(a)Geeks.ORG -- Jay Hanke, President South Front Networks jayhanke(a)southfront.io Phone 612-204-0000

July 7, 2018

Re: MICE UG 28

by Mike Horwath

On Fri, Jul 06, 2018 at 03:52:39PM -0500, Richard Laager wrote: > Reminder: UG 28 is on Wednesday of next week (July 11) at 3:00pm > > Remote Conferencing: 952-562-5400 passcode MICE (6423) > > Location Details: > > Alerus Bank > 3137 32nd Ave S > 2nd Floor Conference Room > Fargo, ND 58103 hey Jay - you driving up? -- Mike Horwath, reachable via drechsau(a)Geeks.ORG

July 6, 2018

MICE UG 28

by Richard Laager

Reminder: UG 28 is on Wednesday of next week (July 11) at 3:00pm Remote Conferencing: 952-562-5400 passcode MICE (6423) Location Details: Alerus Bank 3137 32nd Ave S 2nd Floor Conference Room Fargo, ND 58103 Please park on the east or west side of the building and enter through the front doors. Take steps or elevator to 2nd floor. Attached is the agenda. It's essentially the standard template. I also attached the draft minutes from UG 27. -- Richard

July 6, 2018

Re: Not learning all routes from RR#2

by Frank Bulk

Thanks for the feedback. Interface MTU is showing 1500 bytes. Traffic is running -- peering with others without any known issues. This "received unexpected EOF" in the "show ip bgp neighbors 206.108.255.2" jumps out at me, but my google-fu doesn't reveal anything obvious other than a hint at a TCP checksum issue. Resetting the session again (not the first time) has appeared to resolve it. Interesting that IPv6 is showing a count discrepancy, too: 2001:504:27::d1af:0:1 4 53679 49967 7442 0 0 20:39:29 Estab 2199 2199 2001:504:27::d1af:0:2 4 53679 49962 7442 0 0 20:39:21 Estab 8993 8993 SiouxCenter-Arista-North(s1)#show int e 3/24 Ethernet3/24 is up, line protocol is up (connected) Hardware is Ethernet, address is 001c.73d7.6391 Description: to MICE cia CYAN 7/4 Internet address is 206.108.255.67/24 Broadcast address is 255.255.255.255 IPv6 link-local address is fe80::21c:73ff:fed7:6391/64 IPv6 global unicast address(es): 2001:504:27::d063:0:1, subnet is 2001:504:27::/64 IP MTU 1500 bytes , BW 10000000 kbit Full-duplex, 10Gb/s, auto negotiation: off, uni-link: disabled Up 21 hours, 29 minutes, 16 seconds Loopback Mode : None 5 link status changes since last clear Last clearing of "show interface" counters never 30 seconds input rate 1.85 Gbps (18.8% with framing overhead), 168059 packets/sec 30 seconds output rate 285 Mbps (3.0% with framing overhead), 118650 packets/sec 9241433949 packets input, 12515818295584 bytes Received 1960477 broadcasts, 5282001 multicast 0 runts, 0 giants 0 input errors, 0 CRC, 0 alignment, 0 symbol, 0 input discards 0 PAUSE input 7518589614 packets output, 2302004527878 bytes Sent 373 broadcasts, 17753 multicast 0 output errors, 0 collisions 0 late collision, 0 deferred, 0 output discards 0 PAUSE output SiouxCenter-Arista-North(s1)# SiouxCenter-Arista-North(s1)#show ip bgp neighbors 206.108.255.2 BGP neighbor is 206.108.255.2, remote AS 53679, external link Description: MICE_RR_#2 BGP version 4, remote router ID 206.108.255.2, VRF default Negotiated BGP version 4 Last read 00:00:05, last write 00:00:01 Hold time is 30, keepalive interval is 10 seconds Configured hold time is 30, keepalive interval is 10 seconds Connect timer is inactive Idle-restart timer is inactive BGP state is Established, up for 21:10:51 Number of transitions to established: 3 Last state was OpenConfirm Last event was RecvKeepAlive Last rcvd notification:Cease/maximum number of prefixes reached, Last time 21:28:28 Last rcvd socket-error:received unexpected EOF, Last time 21:23:42, First time 21:26:16, Repeats 7 Neighbor Capabilities: Multiprotocol IPv4 Unicast: advertised and received and negotiated Four Octet ASN: advertised and received Route Refresh: advertised and received and negotiated Send End-of-RIB messages: advertised and received and negotiated Additional-paths recv capability: IPv4 Unicast: advertised Graceful Restart advertised: Restart-time is 300 Restarting: no IPv4 Unicast is enabled, Forwarding State is not preserved Restart timer is inactive End of rib timer is inactive Message Statistics: InQ depth is 0 OutQ depth is 0 Sent Rcvd Opens: 18 3 Notifications: 0 1 Updates: 3426 44814 Keepalives: 7629 8472 Route-Refresh: 0 0 Total messages: 11073 53290 Prefix Statistics: Sent Rcvd IPv4 Unicast: 50 11660 IPv6 Unicast: 0 0 IPv4 SR-TE: 0 0 IPv6 SR-TE: 0 0 Inbound updates dropped by reason: AS path loop detection: 0 Enforced First AS: 10746 Originator ID matches local router ID: 0 Nexthop matches local IP address: 0 Unexpected IPv6 nexthop for IPv4 routes: 0 Nexthop invalid for single hop eBGP: 0 Inbound updates with attribute errors: Resulting in removal of all paths in update (treat-as-withdraw): 0 Resulting in AFI/SAFI disable: 0 Resulting in attribute ignore: 0 Inbound paths dropped by reason: IPv4 labeled-unicast NLRIs dropped due to excessive labels: 0 IPv6 labeled-unicast NLRIs dropped due to excessive labels: 0 Outbound paths dropped by reason: IPv4 local address not available: 0 IPv6 local address not available: 0 Inbound route map is MICE_INBOUND Inbound prefix list for IPv4 unicast is PREMIER_IN_FROM_MICEv4 Outbound prefix list for IPv4 unicast is PREMIER_OUT_TO_MICEv4 Local AS is 53347, local router ID 207.32.15.22 TTL is 1 Local TCP address is 206.108.255.67, local port is 179 Remote TCP address is 206.108.255.2, remote port is 23143 Auto-Local-Addr is disabled TCP Socket Information: TCP state is ESTABLISHED Recv-Q: 0/32768 Send-Q: 0/32768 Outgoing Maximum Segment Size (MSS): 1448 Total Number of TCP retransmissions: 0 Options: Timestamps enabled: yes Selective Acknowledgments enabled: yes Window Scale enabled: yes Explicit Congestion Notification (ECN) enabled: no Socket Statistics: Window Scale (wscale): 6,7 Retransmission Timeout (rto): 312.0ms Round-trip Time (rtt/rtvar): 108.6ms/2.2ms Delayed Ack Timeout (ato): 40.0ms Congestion Window (cwnd): 10 TCP Throughput: 1.07 Mbps Recv Round-trip Time (rcv_rtt): 64.0ms Advertised Recv Window (rcv_space): 28960 SiouxCenter-Arista-North(s1)# Frank -----Original Message----- From: MICE Discuss <MICE-DISCUSS(a)LISTS.IPHOUSE.NET> On Behalf Of Jeremy Lumby Sent: Wednesday, June 27, 2018 7:34 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Not learning all routes from RR#2 Anthony mentioned MTU. I have had issues in the past with BGP, and larger MTUs. Especially when my router had a larger MTU than the path/destination router. BGP would establish, get a few prefixes, and then time out. I just took a look, and the route servers have a 1500 MTU. Are you set larger? -----Original Message----- From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of AnthonyAnderberg(a)nuvera.net Sent: Wednesday, June 27, 2018 5:32 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Not learning all routes from RR#2 The two MICE-attached routers I'm responsible for look OK, and about the same as they have looked recently: 206.108.255.1 4 53679 10130010 700710 305989456 0 0 1y11w 56003 206.108.255.2 4 53679 9531122 656345 305989456 0 0 1y7w 56001 -- 206.108.255.1 0 53679 14571189 909087 587718403 0 0 46w5d 55942 206.108.255.2 0 53679 14593796 908792 587718403 0 0 46w5d 55940 The MICE Looking Glass seems about the same: telnet://rviews:@route-server.micemn.net:23 206.108.255.1 4 53679 5001870 255450 11721640 0 0 23w0d 56004 206.108.255.2 4 53679 4998551 255464 11721640 0 0 23w0d 56002 The "no bgp enforce-first-as" config is expected on some platforms: http://www.micemn.net/technical.html Is traffic flowing normally otherwise? Perhaps you're running into an L1/L2 or MTU issue such that this is a symptom of another problem? Cheers, anthony -----Original Message----- From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Frank Bulk Sent: Wednesday, June 27, 2018 1:25 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] Not learning all routes from RR#2 We (AS53347) moved to a new border router overnight and had some issues with getting our MICE route reflector peerings up -- we'd learn a route or two per second. We had to tweak our Arista's BGP configuration to not "enforce-first-as". Most other direct peers were fine. Eventually RR #1 came up just fine, but RR #2 continues to languish, learning just learning about 10,900 routes. SiouxCenter-Arista-North(s1)#show ip bgp sum BGP summary information for VRF default Router identifier 207.32.15.22, local AS number 18883 Neighbor Status Codes: m - Under maintenance Neighbor V AS MsgRcvd MsgSent InQ OutQ Up/Down State PfxRcd PfxAcc 206.108.255.1 4 53679 44493 4138 0 0 10:56:46 Estab 56188 56188 206.108.255.2 4 53679 39124 7640 0 0 11:38:40 Estab 10929 10929 I'm not sure if we have remote access to the RR's, but whoever does, any hints as to why we're experiencing this behavior? Frank

June 28, 2018

Re: Not learning all routes from RR#2

by Jeremy Lumby

Anthony mentioned MTU. I have had issues in the past with BGP, and larger MTUs. Especially when my router had a larger MTU than the path/destination router. BGP would establish, get a few prefixes, and then time out. I just took a look, and the route servers have a 1500 MTU. Are you set larger? -----Original Message----- From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of AnthonyAnderberg(a)nuvera.net Sent: Wednesday, June 27, 2018 5:32 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] Not learning all routes from RR#2 The two MICE-attached routers I'm responsible for look OK, and about the same as they have looked recently: 206.108.255.1 4 53679 10130010 700710 305989456 0 0 1y11w 56003 206.108.255.2 4 53679 9531122 656345 305989456 0 0 1y7w 56001 -- 206.108.255.1 0 53679 14571189 909087 587718403 0 0 46w5d 55942 206.108.255.2 0 53679 14593796 908792 587718403 0 0 46w5d 55940 The MICE Looking Glass seems about the same: telnet://rviews:@route-server.micemn.net:23 206.108.255.1 4 53679 5001870 255450 11721640 0 0 23w0d 56004 206.108.255.2 4 53679 4998551 255464 11721640 0 0 23w0d 56002 The "no bgp enforce-first-as" config is expected on some platforms: http://www.micemn.net/technical.html Is traffic flowing normally otherwise? Perhaps you're running into an L1/L2 or MTU issue such that this is a symptom of another problem? Cheers, anthony -----Original Message----- From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Frank Bulk Sent: Wednesday, June 27, 2018 1:25 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] Not learning all routes from RR#2 We (AS53347) moved to a new border router overnight and had some issues with getting our MICE route reflector peerings up -- we'd learn a route or two per second. We had to tweak our Arista's BGP configuration to not "enforce-first-as". Most other direct peers were fine. Eventually RR #1 came up just fine, but RR #2 continues to languish, learning just learning about 10,900 routes. SiouxCenter-Arista-North(s1)#show ip bgp sum BGP summary information for VRF default Router identifier 207.32.15.22, local AS number 18883 Neighbor Status Codes: m - Under maintenance Neighbor V AS MsgRcvd MsgSent InQ OutQ Up/Down State PfxRcd PfxAcc 206.108.255.1 4 53679 44493 4138 0 0 10:56:46 Estab 56188 56188 206.108.255.2 4 53679 39124 7640 0 0 11:38:40 Estab 10929 10929 I'm not sure if we have remote access to the RR's, but whoever does, any hints as to why we're experiencing this behavior? Frank

June 28, 2018

Re: Not learning all routes from RR#2

by AnthonyAnderberg＠nuvera.net

The two MICE-attached routers I'm responsible for look OK, and about the same as they have looked recently: 206.108.255.1 4 53679 10130010 700710 305989456 0 0 1y11w 56003 206.108.255.2 4 53679 9531122 656345 305989456 0 0 1y7w 56001 -- 206.108.255.1 0 53679 14571189 909087 587718403 0 0 46w5d 55942 206.108.255.2 0 53679 14593796 908792 587718403 0 0 46w5d 55940 The MICE Looking Glass seems about the same: telnet://rviews:@route-server.micemn.net:23 206.108.255.1 4 53679 5001870 255450 11721640 0 0 23w0d 56004 206.108.255.2 4 53679 4998551 255464 11721640 0 0 23w0d 56002 The "no bgp enforce-first-as" config is expected on some platforms: http://www.micemn.net/technical.html Is traffic flowing normally otherwise? Perhaps you're running into an L1/L2 or MTU issue such that this is a symptom of another problem? Cheers, anthony -----Original Message----- From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Frank Bulk Sent: Wednesday, June 27, 2018 1:25 PM To: MICE-DISCUSS(a)LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] Not learning all routes from RR#2 We (AS53347) moved to a new border router overnight and had some issues with getting our MICE route reflector peerings up -- we'd learn a route or two per second. We had to tweak our Arista's BGP configuration to not "enforce-first-as". Most other direct peers were fine. Eventually RR #1 came up just fine, but RR #2 continues to languish, learning just learning about 10,900 routes. SiouxCenter-Arista-North(s1)#show ip bgp sum BGP summary information for VRF default Router identifier 207.32.15.22, local AS number 18883 Neighbor Status Codes: m - Under maintenance Neighbor V AS MsgRcvd MsgSent InQ OutQ Up/Down State PfxRcd PfxAcc 206.108.255.1 4 53679 44493 4138 0 0 10:56:46 Estab 56188 56188 206.108.255.2 4 53679 39124 7640 0 0 11:38:40 Estab 10929 10929 I'm not sure if we have remote access to the RR's, but whoever does, any hints as to why we're experiencing this behavior? Frank

June 27, 2018