I propose that MICE enable BGP ADD-PATH capability (RFC 7911) on the route servers (in TX mode only). This is negotiated BGP capability, so we do not need any per-participant setting for it. ADD-PATH is useful if the best path route is being filtered but another route would not be. We discussed adding this at a UG meeting years ago. There are other IXPs that have this enabled. For example, this one: https://www.bcix.de/ixp/content/0/route-servers In terms of timeline, I propose something like this: * January 9: Enable it on Route Server 1. * January 16: Enable it on Route Server 2. The actual change looks like this for the RS1-only piece, and then the conditional would be removed when we want to deploy to RS2: --- a/roles/ixpmanager/files/mice/api/v4/router/server/bird2/neighbors.foil.php +++ b/roles/ixpmanager/files/mice/api/v4/router/server/bird2/neighbors.foil.php @@ -315,6 +315,9 @@ protocol bgp pb_<?= $int['fvliid'] ?>_as<?= $int['autsys'] ?> from tb_rsclient { import filter f_import_as<?= $int['autsys'] ?>; table t_<?= $int['fvliid'] ?>_as<?= $int['autsys'] ?>; export filter f_export_as<?= $int['autsys'] ?>; +<?php if( $t->router->router_id == "206.108.255.1" ): ?> + add paths tx; +<?php endif; ?> }; <?php if( $int['bgpmd5secret'] && !$t->router->skip_md5 ): ?> password "<?= $int['bgpmd5secret'] ?>"; -- Richard
I think the added functionality probably outweighs the possible issues. I am in favor. From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> On Behalf Of Richard Laager Sent: Thursday, December 21, 2023 8:57 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] BGP ADD-PATH capability I propose that MICE enable BGP ADD-PATH capability (RFC 7911) on the route servers (in TX mode only). This is negotiated BGP capability, so we do not need any per-participant setting for it. ADD-PATH is useful if the best path route is being filtered but another route would not be. We discussed adding this at a UG meeting years ago. There are other IXPs that have this enabled. For example, this one: https://www.bcix.de/ixp/content/0/route-servers In terms of timeline, I propose something like this: * January 9: Enable it on Route Server 1. * January 16: Enable it on Route Server 2. The actual change looks like this for the RS1-only piece, and then the conditional would be removed when we want to deploy to RS2: --- a/roles/ixpmanager/files/mice/api/v4/router/server/bird2/neighbors.foil.php +++ b/roles/ixpmanager/files/mice/api/v4/router/server/bird2/neighbors.foil.php @@ -315,6 +315,9 @@ protocol bgp pb_<?= $int['fvliid'] ?>_as<?= $int['autsys'] ?> from tb_rsclient { import filter f_import_as<?= $int['autsys'] ?>; table t_<?= $int['fvliid'] ?>_as<?= $int['autsys'] ?>; export filter f_export_as<?= $int['autsys'] ?>; +<?php if( $t->router->router_id == "206.108.255.1" ): ?> + add paths tx; +<?php endif; ?> }; <?php if( $int['bgpmd5secret'] && !$t->router->skip_md5 ): ?> password "<?= $int['bgpmd5secret'] ?>"; -- Richard _____ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS <http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1> &A=1
Ack'ing this is a negotiated feature, buyer beware for those who turn it on and are using prefix-limits on their BGP sessions. I've used add-path on JunOS and on that platform you optionally add a 'path-count' function on the TX side for intra ASN stuff. In the bird implementation the behavior is "all" routes. Is there a graph or can you provide a one time count of the current RIB sizes on the RS? I am in favor adding the feature, but I’m not sure if 1 week spacing is enough to prove stability. But I’m known to be nervous. Bird docs mentioning the feature here: https://bird.network.cz/?get_doc&v=20&f=bird-6.html -Michael From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> On Behalf Of Jeremy Lumby Sent: Thursday, December 21, 2023 9:07 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] BGP ADD-PATH capability I think the added functionality probably outweighs the possible issues. I am in favor. From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET>> On Behalf Of Richard Laager Sent: Thursday, December 21, 2023 8:57 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET<mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET> Subject: [MICE-DISCUSS] BGP ADD-PATH capability I propose that MICE enable BGP ADD-PATH capability (RFC 7911) on the route servers (in TX mode only). This is negotiated BGP capability, so we do not need any per-participant setting for it. ADD-PATH is useful if the best path route is being filtered but another route would not be. We discussed adding this at a UG meeting years ago. There are other IXPs that have this enabled. For example, this one: https://www.bcix.de/ixp/content/0/route-servers In terms of timeline, I propose something like this: * January 9: Enable it on Route Server 1. * January 16: Enable it on Route Server 2. The actual change looks like this for the RS1-only piece, and then the conditional would be removed when we want to deploy to RS2: --- a/roles/ixpmanager/files/mice/api/v4/router/server/bird2/neighbors.foil.php +++ b/roles/ixpmanager/files/mice/api/v4/router/server/bird2/neighbors.foil.php @@ -315,6 +315,9 @@ protocol bgp pb_<?= $int['fvliid'] ?>_as<?= $int['autsys'] ?> from tb_rsclient { import filter f_import_as<?= $int['autsys'] ?>; table t_<?= $int['fvliid'] ?>_as<?= $int['autsys'] ?>; export filter f_export_as<?= $int['autsys'] ?>; +<?php if( $t->router->router_id == "206.108.255.1" ): ?> + add paths tx; +<?php endif; ?> }; <?php if( $int['bgpmd5secret'] && !$t->router->skip_md5 ): ?> password "<?= $int['bgpmd5secret'] ?>"; -- Richard ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
We are looking at this on QCIX as well. I'll bring this up on the NA-IX forum and see what the other IXes are doing and have done and what issues have been solved or created. jay On Fri, Dec 22, 2023 at 8:00 AM Michael Hare <000000097dab80c5-dmarc-request@lists.iphouse.net> wrote:
Ack'ing this is a negotiated feature, buyer beware for those who turn it on and are using prefix-limits on their BGP sessions. I've used add-path on JunOS and on that platform you optionally add a 'path-count' function on the TX side for intra ASN stuff. In the bird implementation the behavior is "all" routes. Is there a graph or can you provide a one time count of the current RIB sizes on the RS?
I am in favor adding the feature, but I’m not sure if 1 week spacing is enough to prove stability. But I’m known to be nervous.
Bird docs mentioning the feature here: https://bird.network.cz/?get_doc&v=20&f=bird-6.html
-Michael
From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> On Behalf Of Jeremy Lumby Sent: Thursday, December 21, 2023 9:07 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] BGP ADD-PATH capability
I think the added functionality probably outweighs the possible issues. I am in favor.
From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> On Behalf Of Richard Laager Sent: Thursday, December 21, 2023 8:57 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] BGP ADD-PATH capability
I propose that MICE enable BGP ADD-PATH capability (RFC 7911) on the route servers (in TX mode only). This is negotiated BGP capability, so we do not need any per-participant setting for it.
ADD-PATH is useful if the best path route is being filtered but another route would not be. We discussed adding this at a UG meeting years ago.
There are other IXPs that have this enabled. For example, this one: https://www.bcix.de/ixp/content/0/route-servers
In terms of timeline, I propose something like this:
January 9: Enable it on Route Server 1. January 16: Enable it on Route Server 2.
The actual change looks like this for the RS1-only piece, and then the conditional would be removed when we want to deploy to RS2:
--- a/roles/ixpmanager/files/mice/api/v4/router/server/bird2/neighbors.foil.php +++ b/roles/ixpmanager/files/mice/api/v4/router/server/bird2/neighbors.foil.php @@ -315,6 +315,9 @@ protocol bgp pb_<?= $int['fvliid'] ?>_as<?= $int['autsys'] ?> from tb_rsclient { import filter f_import_as<?= $int['autsys'] ?>; table t_<?= $int['fvliid'] ?>_as<?= $int['autsys'] ?>; export filter f_export_as<?= $int['autsys'] ?>; +<?php if( $t->router->router_id == "206.108.255.1" ): ?> + add paths tx; +<?php endif; ?> }; <?php if( $int['bgpmd5secret'] && !$t->router->skip_md5 ): ?> password "<?= $int['bgpmd5secret'] ?>";
--
Richard
________________________________
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
________________________________
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
________________________________
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
On 2023-12-22 08:00, Michael Hare wrote:
Ack'ing this is a negotiated feature, buyer beware for those who turn it on and are using prefix-limits on their BGP sessions.
Prefix limit is a fair point.
Is there a graph or can you provide a one time count of the current RIB sizes on the RS?
Here is how it changes (as viewed by my AS just before I sent the original email): Before: 107271 After: 113441
I am in favor adding the feature, but I’m not sure if 1 week spacing is enough to prove stability.
If your concern about 1 week spacing is related to prefix limits, then I think we will be fine. I can look at who has a session up before, make the change, and see if anyone's session goes down. If so, I can contact them specifically. If your concern about 1 week spacing is just a generalized concern, do you have a suggestion of what sort of period would be appropriate? -- Richard
I was going to deploy this today. But minutes before I would have tried, someone posted on the bird-users mailing list about add-paths causing sessions with IOS-XR 7.9.2 to shut down. My current plan is to postpone this a week to see if there are any replies on that thread. Unfortunately, I've found the bird mailing lists to be pretty quiet. If nothing changes, then I will attempt this. I was already planning to keep track of which sessions were up before and compare after. Based on what happens, I can revert if needed. It's only one route server, so if someone's session goes down, it's not the end of the world. If you are concerned about this (e.g. you run IOS-XR), another option is that you could pre-emptively disable support for add-paths on your end. -- Richard
Today, the University of Minnesota tested with me. I enabled "add paths tx" on one of their sessions and it worked. -- Richard
I'll add while we are receiving the Additional-Path BGP Capability from RS1, we don't have it enabled on our side, so it is not being negotiated. Thanks On Wed, Jan 10, 2024 at 8:33 PM Richard Laager <rlaager@wiktel.com> wrote:
Today, the University of Minnesota tested with me. I enabled "add paths tx" on one of their sessions and it worked.
-- Richard
------------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
-- =============================================== David Farmer Email:farmer@umn.edu Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 ===============================================
Does anyone--like another IX, possibly--have documentation that can be shared about how to enable, disable, or ignore the add-path configuration on the member router? Farmer mentions they didn't have it enabled suggesting each operator would need to enable add-path on their router before the functionality could be used by each member. Thanks, Jonathan Jonathan Stewart Network Engineer LES.NET - AS18451 Desk: 1-204-666-6191 Mobile: 1-204-990-2120 130 Portage Avenue E Winnipeg, MB R3C 0A1 CANADA On Wed, 2024-01-10 11:08 p.m., David Farmer wrote:
I'll add while we are receiving the Additional-Path BGP Capability from RS1, we don't have it enabled on our side, so it is not being negotiated.
Thanks
On Wed, Jan 10, 2024 at 8:33 PM Richard Laager <rlaager@wiktel.com> wrote:
Today, the University of Minnesota tested with me. I enabled "add paths tx" on one of their sessions and it worked.
-- Richard
------------------------------------------------------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 <http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1>
-- =============================================== David Farmer Email:farmer@umn.edu <mailto:Email%3Afarmer@umn.edu> Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 ===============================================
------------------------------------------------------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 <http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1>
Here are a couple of Cisco Docs https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_bgp/configuration/... https://www.cisco.com/c/en/us/support/docs/ip/border-gateway-protocol-bgp/21... Hope that helps; On Thu, Jan 11, 2024 at 11:04 AM Jonathan Stewart <jonathan@les.net> wrote:
Does anyone--like another IX, possibly--have documentation that can be shared about how to enable, disable, or ignore the add-path configuration on the member router?
Farmer mentions they didn't have it enabled suggesting each operator would need to enable add-path on their router before the functionality could be used by each member.
Thanks, Jonathan
Jonathan Stewart Network EngineerLES.NET - AS18451 Desk: 1-204-666-6191 Mobile: 1-204-990-2120 130 Portage Avenue E Winnipeg, MB R3C 0A1 CANADA
On Wed, 2024-01-10 11:08 p.m., David Farmer wrote:
I'll add while we are receiving the Additional-Path BGP Capability from RS1, we don't have it enabled on our side, so it is not being negotiated.
Thanks
On Wed, Jan 10, 2024 at 8:33 PM Richard Laager <rlaager@wiktel.com> wrote:
Today, the University of Minnesota tested with me. I enabled "add paths tx" on one of their sessions and it worked.
-- Richard
------------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
-- =============================================== David Farmer Email:farmer@umn.edu Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 ===============================================
------------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
------------------------------
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
-- =============================================== David Farmer Email:farmer@umn.edu Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 ===============================================
Here is a juniper example set protocols bgp group iBGP-reflector-v4 family inet unicast add-path receive set protocols bgp group iBGP-reflector-v4 family inet unicast add-path send prefix-policy addPathiBGPPolicyASBR set protocols bgp group iBGP-reflector-v4 family inet unicast add-path send path-count 3 addPathiBGPPolicyASBR could be as simple as ‘accept all’ or something more granular. Richard I think addressed this a few weeks back; consider the potential growth of your RIB and recognize that the feature to preinstall the backup path in your FIB, if possible on your hardware, may be a different knob. -Michael From: MICE Discuss <MICE-DISCUSS@LISTS.IPHOUSE.NET> On Behalf Of David Farmer Sent: Thursday, January 11, 2024 1:29 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: Re: [MICE-DISCUSS] BGP ADD-PATH capability Here are a couple of Cisco Docs https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_bgp/configuration/... https://www.cisco.com/c/en/us/support/docs/ip/border-gateway-protocol-bgp/21... Hope that helps; On Thu, Jan 11, 2024 at 11:04 AM Jonathan Stewart <jonathan@les.net<mailto:jonathan@les.net>> wrote: Does anyone--like another IX, possibly--have documentation that can be shared about how to enable, disable, or ignore the add-path configuration on the member router? Farmer mentions they didn't have it enabled suggesting each operator would need to enable add-path on their router before the functionality could be used by each member. Thanks, Jonathan Jonathan Stewart Network Engineer LES.NET<http://LES.NET> - AS18451 Desk: 1-204-666-6191 Mobile: 1-204-990-2120 130 Portage Avenue E Winnipeg, MB R3C 0A1 CANADA On Wed, 2024-01-10 11:08 p.m., David Farmer wrote: I'll add while we are receiving the Additional-Path BGP Capability from RS1, we don't have it enabled on our side, so it is not being negotiated. Thanks On Wed, Jan 10, 2024 at 8:33 PM Richard Laager <rlaager@wiktel.com<mailto:rlaager@wiktel.com>> wrote: Today, the University of Minnesota tested with me. I enabled "add paths tx" on one of their sessions and it worked. -- Richard ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 -- =============================================== David Farmer Email:farmer@umn.edu<mailto:Email%3Afarmer@umn.edu> Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 =============================================== ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 -- =============================================== David Farmer Email:farmer@umn.edu<mailto:Email%3Afarmer@umn.edu> Networking & Telecommunication Services Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 612-626-0815 Minneapolis, MN 55414-3029 Cell: 612-812-9952 =============================================== ________________________________ To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
Sorry for the delay. I owe everyone an update. I did not make any changes. There was still some discussion (albeit not a lot) happening on the bird-users list. Today, I got a reply about this on the IXP Manager mailing list: https://www.inex.ie/pipermail/ixpmanager/2024-January/003739.html Honestly, I'm not sure if this is worth it. I'll just leave this off. Maybe we can revisit in the future if there is more interest and/or if this potential BIRD bug discussion goes somewhere. I reverted my testing changes. -- Richard
participants (6)
-
David Farmer -
Jay Hanke -
Jeremy Lumby -
Jonathan Stewart -
Michael Hare -
Richard Laager