BESS WorkGroup N. Malhotra, Ed. Internet-Draft K. Ananthamurthy Intended status: Standards Track A. Sajassi Expires: 5 September 2024 L. Krattiger Cisco Systems J. Rabadan Nokia 4 March 2024 Centralized Anycast Gateway in Ethernet VPN(EVPN) draft-malhotra-bess-evpn-centralized-anycast-gw-01 Abstract EVPN Integrated Routing and Bridging (IRB) fabrics provide a flexible and extensible method for Layer-2 and Layer-3 overlay network connectivity. [EVPN-IRB] defines operation for symmetric and asymmetric EVPN IRB using distributed anycast gateway architecture (DAG). In a DAG architecture, both bridging and first-hop routing functions for overlay subnets are located on leaf PEs, with first-hop routing provided by a distributed anycast gateway provisioned across the leaf PEs. This document describes an architecture and operation for EVPN Centralized Anycast Gateway (CAG), which allows the first- hop routing function for overlay subnets to be centralized on designated IRB GWs while the bridging function is still located on the leaf PEs. The documents also covers trade-offs of deploying a CAG as compared with DAG. It further describes operation for inter- op between CAG and DAG based EVPN-IRB network overlays. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on 5 September 2024. Malhotra, et al. Expires 5 September 2024 [Page 1] Internet-Draft EVPN CAG March 2024 Copyright Notice Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/ license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Requirements Language and Terminology . . . . . . . . . . . . 5 3. Control Plane Operation . . . . . . . . . . . . . . . . . . . 5 3.1. Requirements for L2 PE . . . . . . . . . . . . . . . . . 5 3.2. Requirements for CAG PE . . . . . . . . . . . . . . . . . 6 4. Data Plane Operation . . . . . . . . . . . . . . . . . . . . 7 4.1. Intra-Subnet Bridging . . . . . . . . . . . . . . . . . . 7 4.2. Inter-Subnet Routing . . . . . . . . . . . . . . . . . . 7 5. MAC/IP Mobility . . . . . . . . . . . . . . . . . . . . . . . 8 6. CAG deployment models . . . . . . . . . . . . . . . . . . . . 8 6.1. CAG Placement as an Interconnect . . . . . . . . . . . . 8 6.1.1. Control Plane . . . . . . . . . . . . . . . . . . . . 10 6.1.2. Tunnel Adjacencies . . . . . . . . . . . . . . . . . 10 6.1.3. Split Horizon domains . . . . . . . . . . . . . . . . 11 6.1.4. Data Plane . . . . . . . . . . . . . . . . . . . . . 11 6.1.4.1. Inter-subnet - L2 fabric to Symmetric IRB fabric . . . . . . . . . . . . . . . . . . . . . . 11 6.1.4.2. Inter-subnet - Symmetric IRB fabric to L2 fabric . . . . . . . . . . . . . . . . . . . . . . 11 6.1.4.3. Intra-subnet - Symmetric IRB fabric to L2 fabric unicast . . . . . . . . . . . . . . . . . . . . . . 12 6.1.4.4. Intra-subnet - L2 fabric to Symmetric IRB fabric unicast . . . . . . . . . . . . . . . . . . . . . . 12 6.1.4.5. Intra-subnet - Symmetric IRB fabric to L2 fabric BUM . . . . . . . . . . . . . . . . . . . . . . . . 12 6.1.4.6. Intra-subnet - L2 fabric to Symmetric IRB fabric BUM . . . . . . . . . . . . . . . . . . . . . . . . 12 6.2. CAG Placement on DAG L2/L3PEs . . . . . . . . . . . . . . 12 6.2.1. Dual role of DAGs . . . . . . . . . . . . . . . . . . 14 6.2.2. Operation on CAG as L2/L3 GW . . . . . . . . . . . . 14 6.2.3. FHG Load-sharing and resiliency . . . . . . . . . . . 14 6.2.4. FHG Localization . . . . . . . . . . . . . . . . . . 14 Malhotra, et al. Expires 5 September 2024 [Page 2] Internet-Draft EVPN CAG March 2024 6.2.5. Comparison between the approaches . . . . . . . . . . 14 6.2.5.1. Operational simplicity . . . . . . . . . . . . . 14 6.2.5.2. Scalability . . . . . . . . . . . . . . . . . . . 15 7. Operational Considerations . . . . . . . . . . . . . . . . . 15 8. Security Considerations . . . . . . . . . . . . . . . . . . . 15 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 15 10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 15 11. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 15 12. Normative References . . . . . . . . . . . . . . . . . . . . 15 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 16 1. Introduction In a CAG routing architecture, overlay endpoints connect to Layer-2 EVPN gateways that provide VPN bridging function for overlay subnets. This VPN bridging function enables intra-subnet flows across the overlay while routing function between end-points in different subnets and to/from end-points external to the fabric is located on designated L2+L3 (IRB) gateways. First-hop routing for each overlay subnet is deployed using a subnet Anycast GW that is hosted on one or more designated IRB GW nodes. A key attribute that defines this architecture is that the first-hop routing function for an overlay subnet is decoupled from the EVPN leaf that only provides intra-subnet bridging service for that subnet. This decoupling in-turn results in first-hop routing for overlay endpoints being "centralized" on designated IRB nodes. Note that the Anycast GW for each subnet is still distributed across these "centralized" IRB GW nodes. It is common to deploy first-hop Anycast routing for all overlay subnets in a fabric on the same set of IRB nodes. While not necessarily required, as is covered later in the document, this is often done for operational simplicity and optimal routing. It is also common for this first-hop routing function to be hosted on border nodes that also act as interconnect GWs to external L2 or L2/ L3 domains. Optionally, the CAG IRB nodes may also have directly connected end-points. CAG architecture essentially uses a Layer-2 EVPN overlay and first- hop routing operation on CAG is identical to asymmetric routing as defined in [EVPN-IRB]. Figure below shows a reference L2 fabric with CAG topology that is also used to illustrate the procedures specified in later sections. Malhotra, et al. Expires 5 September 2024 [Page 3] Internet-Draft EVPN CAG March 2024 +-----------------------------------------------------------------+ | L2/L3 Interconnect to external domains | | | | | | | | | | | | CAGs with Anycast IP | | +-----------------------+ | | | +------+ +------+ | IRB1: [IP10, M10] | | | | CAG1 | | CAG2 | | IRB2: [IP20, M20] | | | +------+ +------+ | IRB3: [IP30, M30] | | +-----------------------+ | | | | | | | | +---------------+ | | | | | | | EVPN | | | | | | | +---------------+ | | | | | | | | | | +------+ +------+ +------+ +------+ +------+ +------+ | | |L2 PE1| |L2 PE2| |L2 PE3| |L2 PE4| |L2 PE5| |L2 PE6| | | +------+ +------+ +------+ +------+ +------+ +------+ | | \ / \ / \ / | | \ / \ / \ / | | \ / \ / \ / | | +\---/+ +\---/+ +\---/+ | | | \ / | | \ / | | \ / | | | +--+--+ +--+--+ +--+--+ | | | | | | | | | | | +-----------------------------------------------------------------+ | | | EVI1 H11:[IP11, M11] H13:[IP13, M13] EVI2 H21:[IP21, M21] H22:[IP22, M22] EVI3 H31:[IP31, M31] H33:[IP33, M33] Figure 1 * L2 PE1..L2 PE6 are L2-Only GWs. * IRB1/IRB2/IRB3 are IRB interfaces on CAGs for EVI1, EVI2 and EVI3. Malhotra, et al. Expires 5 September 2024 [Page 4] Internet-Draft EVPN CAG March 2024 * Hosts H11/H13 are in EVI1, H21/H22 are in EVI2 and H31/H33 are in EVI3. * CAG1 and CAG2 form a redundant anycast CAG pair for EVI1, EVI2, and EVI3. 2. Requirements Language and Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. * CAG: Centralized Anycast Gateway * DAG: Distributed Anycast Gateway * EVI: An EVPN instance spanning the Provider Edge (PE) devices participating in that EVPN * FHR: First Hop Router * IRB: Integrated Routing and Bridging * L2-GW: Layer-2 Only Gateway * L2-Only GW: Layer-2 Only Gateway 3. Control Plane Operation This section defines control plane procedures required on L2 PE and CAG PE for a CAG architecture based deployment. 3.1. Requirements for L2 PE * ARP/ND snooping MUST be enabled on L2-PEs. Locally connected host IP and MAC is learnt by L2 PE via ARP/ND snooping and advertised using EVPN RT-2 with a single label that represents the EVI. This enables a significant simplification in CAG operation to avoid the need for data plane learning and syncing of ARP/ND tables across redundant CAGs. * L2-PEs MUST handle ARP refreshes when MAC ages out or ARP ages out in order to relearn the host MAC and IP to avoid traffic loss. If a host does not respond to an ARP refresh, then the previously advertised EVPN RT-2 by the L2-PE MUST be withdrawn. If the host responds to host MAC/IP, then ARP MUST be refreshed. EVPN RT-2 SHOULD NOT be re-advertised Malhotra, et al. Expires 5 September 2024 [Page 5] Internet-Draft EVPN CAG March 2024 * L2-PEs on receiving GW MAC/IP RT-2 from CAG, in addition to installing the MAC in the MAC-VRF, MUST also install a GW MAC/IP entry in the ARP/ND suppression cache. This enables L2-PEs to act as a proxy for CAG by using the entry in the suppression cache to respond to ARP requests from hosts for GW MAC/IP. If enabled, this avoids flooding of ARP/ND requests across the fabric and avoids duplicate ARP responses from redundant CAGs. 3.2. Requirements for CAG PE * A set of redundant CAG PEs that act as the FHR for the same subnet MUST be provisioned with the same anycast GW IP and MAC. * For each subnet for which CAG is an FHR, CAG MUST advertise Anycast GW MAC/IP using EVPN RT-2 with Default Gateway Extended Community as defined in [RFC7432]. In Figure 1, CAG1/2 advertise IRB1/2/3 MAC and IP using EVPN RT-2 with Default Gateway Extended Community with nexthop as anycast IP. * In case of VXLAN encapsulation, set of redundant CAG PEs provisioned as FHR for a common set of subnets MAY advertise the anycast GW MAC/IP RT-2 with an anycast VTEP IP as the next-hop. This enables the GW MAC to be installed with a single BGP path on L2 PEs and hence enables interworking with low end L2 devices that may not be capable of MAC multi-pathing. It also results in a much simplified solution that avoids a need for virtual ESI provisioning on CAG PE as well as a need for MAC multi-pathing and fast convergence procedures on CAG and L2 PEs. Note that this anycast VTEP IP is solely for the purpose of GW MAC/IP RT-2 advertisement and all directly connected end-points (single-homed or multi-homed) will continue to be advertised with a non-anycast VTEP IP as the next-hop. ESI multi-homing procedures specified in [RFC7432] apply as-is to directly connected end-points multi-homed via ESI LAG. * Upon receiving RT-2 advertisements from Egress EVPN L2-PE, CAG MUST import MACs into MAC-VRFs, thus establishing Host MAC reachability via Layer-2 EVPN encapsulation and tunnel to Egress L2 PE. In Figure 1, L2-GW1/2 advertise RT-2 for hosts H11, H21 and H31. CAG MUST import M11, M21 and M31 in corresponding MAC- VRFs. CAG will perform the Asymmetric IRB procedures defined in [RFC9135] with IP-to-MAC binding resolved via respective adjacency/next-hop table entry. * It should be noted that reachability to a remote host layer-3 adjacency is still resolved by host MAC reachability via a Layer-2 VPN tunnel to the Egress L2-PE. Malhotra, et al. Expires 5 September 2024 [Page 6] Internet-Draft EVPN CAG March 2024 4. Data Plane Operation 4.1. Intra-Subnet Bridging Intra-subnet host to host flow is identical to that in symmetric or asymmetric distributed anycast GW based IRB deployments as defined in [EVPN-IRB]. When the ingress L2 PE receives a packet from its locally attached host, it does a destination MAC lookup in its VLAN which yields an L2 VPN and tunnel encapsulation towards the egress L2 PE. The ingress L2 PE then encapsulates the original packet and sends to the egress L2 PE. Egress L2 PE decapsulates the packet and does a destination MAC lookup in the local VLAN (MAC VRF) table identified by the received L2 VPN label. This yields a local VLAN Port. The egress L2GW then sends the decapsulated packet to the port. 4.2. Inter-Subnet Routing Inter-subnet host to host flow destined to default GW MAC is bridged to CAG PE with the L2 VPN encapsulation learnt via default GW RT-2 from the CAG PE. CAG decapsulates the packet and does a destination MAC lookup in the local MAC VRF identified by the received L2 VPN encapsulation that results in my MAC. This yields the local IRB interface. CAG then does a destination IP lookup in IP VRF attached to the IRB interface. If the destination subnet is local/attached to the CAG node, IP lookup yields a local host adjacency on the destination subnet IRB interface. This results in host MAC rewrite, followed by host MAC lookup that results in the packet being bridged to the egress L2 PE with L2 VPN and tunnel encapsulation learnt via RT-2 from egress L2 PE. On the egress L2 PE, packet is decapsulated, local MAC VRF is resolved by the received L2 VPN encapsulation. The packet is then bridged to the local host via local MAC VRF lookup. If the destination subnet is not local to the CAG node, IP lookup yields the destination subnet prefix that resolves via L3 VPN encapsulation and tunnel to the next-hop CAG PE that is the FHR for the destination subnet. Packet is hence routed to another CAG, where the packet is decapsulated, received L3 VPN encapsulation resolves the local IP VRF, and IP lookup now yields a local host adjacency on the destination subnet IRB interface. Packet is then bridged to the destination L2 PE with the L2 VPN encapsulation as described above. Malhotra, et al. Expires 5 September 2024 [Page 7] Internet-Draft EVPN CAG March 2024 5. MAC/IP Mobility GW MAC/IP route advertised by CAG MUST follow default gateway best path selection procedures specified in [RFC7432bis] to ensure that GW MAC is treated as static and is always preferred over any duplicate host MAC across the L2 overlay. Beyond BGP best path selection, forwarding implementations MUST ensure that a locally learnt duplicate MAC will never overwrite the forwarding entry for the GW MAC route. Hosts attached to L2-PEs follow existing mobility procedures as defined in [RFC7432]. 6. CAG deployment models Fabrics with symmetric IRB are widely deployed. This section discusses how a CAG architecture discussed in the earlier sections may be deployed together with a symmetric IRB fabric with L2 and L3 overlays that extend across symmetric IRB and CAG based fabrics. Two deployment models are discussed: * CAG Placement as an Interconnect * CAG Placement on DAG L2/L3PEs 6.1. CAG Placement as an Interconnect In this model, CAG is placed as an interconnect or hub between the Layer-2 fabric consisting of L2-PEs and the symmetric IRB fabric consisting of L2/L3 PEs. CAG device essentially performs the CAG function for hosts connected to L2 only fabric and in addition, also performs an L2/L3 overlay interconnect function between the L2 only fabric and the symmetric IRB fabric. EVI1 H24:[IP24 M24] EVI2 H14:[IP14,M14] H25:[IP25, M25] H15:[IP15] | | | +-----------------------------------------------------------------+ | | | | | | | | | | | +--+--+ +--+--+ | | | | / \ | | / \ | | | | +-----+ +-----+ | | | / \ / \ | | | / \ / \ | | | / \ / \ | | | +-----+ +-----+ +-----+ +-----+ +-----+ | | | PE7 | | PE8 | | PE9 | | PE10| | PE11| | Malhotra, et al. Expires 5 September 2024 [Page 8] Internet-Draft EVPN CAG March 2024 | +-----+ +-----+ +-----+ +-----+ +-----+ | | | | IRB1: [IP10, M10] | | IRB2: [IP20, M20] | | | +-----------------------------------------------------------------+ Symmetric IRB Fabric +-----------------------+ CAGs with Anycast IP as Interconnect | +------+ +------+ | IRB1: [IP10, M10] | | CAG1 | | CAG2 | | IRB2: [IP20, M20] | +------+ +------+ | IRB3: [IP30, M30] +-----------------------+ Layer-2 EVPN Fabric +-----------------------------------------------------------------+ | | | +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ | | | PE1 | | PE2 | | PE3 | | PE4 | | PE5 | | PE6 | | | +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ | | \ / \ / \ / | | \ / \ / \ / | | \ / \ / \ / | | +\---/+ +\---/+ +\---/+ | | | \ / | | \ / | | \ / | | | +--+--+ +--+--+ +--+--+ | | | | | | | | | | | +-----------------------------------------------------------------+ | | | EVI1 H11:[IP11, M11] H13:[IP13, M13] EVI2 H21:[IP21, M21] H22:[IP22, M22] EVI3 H31:[IP31, M31] H33:[IP33, M33] Figure 2 * IRB1/IRB2 are IRB interfaces on L2/L3 PEs PE7- PE10. * PE11 is L3-only PE * CAG pair consisting of CAG1/CAG2 is placed an Interconnect between Layer-2 only and Symmetric IRB fabric. Malhotra, et al. Expires 5 September 2024 [Page 9] Internet-Draft EVPN CAG March 2024 6.1.1. Control Plane In addition to the control plane operation described in Section 3, the following operations must be done at CAG to enable seamless interworking between the fabrics. * MAC/IP RT-2s learnt on CAGs from L2 PEs in the Layer-2 only EVPN Fabric MUST be re-originated by all CAGs towards the L2/L3 PEs and L3-only PEs in the Symmetric IRB Fabric with CAG as the tunnel next-hop. * MAC/IP RT-2s learnt on CAGs from L2 PEs in the Layer-2 only EVPN Fabric MUST be re-originated by all CAGs towards the L2/L3 PEs and L3-only PEs in the Symmetric IRB Fabric in symmetric IRB format with both L2 and L3 VPN local labels. Essentially, these MAC-IP routes are learnt from L2 PEs with only L2 label and re-originated with locally provisioned or allocated L2 and L3 labels. * MAC/IP RT-2s learnt from L2/L3 PEs in the Symmetric IRB fabric MUST be re-originated towards the Layer-2 only fabric with CAG as the tunnel next-hop. * MAC/IP RT-2s learnt from L2/L3 PEs in the Symmetric IRB fabric and re-originated towards the L2 PEs MAY have L3 label and L3 RTs removed resulting in re-origination with only locally assigned L2 VPN label and L2 RTs. L2 PEs in the Layer-2 only fabric ignore the L3 label and RTs if they are present. 6.1.2. Tunnel Adjacencies This results in the following overlay tunnel adjacencies at CAG: * L2 and L3 VPN tunnel adjacencies between CAG and L2/L3 PEs in symmetric IRB fabric. In above topology, Layer-2 and Layer-3 tunnel adjacencies are formed between L2/L3 PEs PE7-PE10 and CAG. * L3 VPN tunnel adjacencies between CAG and L3 only PEs in symmetric IRB fabric. In above topology, Layer-3 tunnel adjacencies are formed between L3-only PE11 and CAG. * L2 VPN tunnel adjacencies between CAG and L2 PEs in L2 fabric. In above topology, Layer-3 tunnel adjacencies are formed between L2 PEs PE1-PE6 and CAG. Malhotra, et al. Expires 5 September 2024 [Page 10] Internet-Draft EVPN CAG March 2024 6.1.3. Split Horizon domains CAG MUST consider the two fabrics as separate split horizon domains and hence apply split-horizon procedures specified in [RFC9014]. As a result, The BUM traffic from one domain is distributed to the other domain. For e.g. ARP requests from symmetric IRB domain are distributed to the Layer-2 domain and vice versa. 6.1.4. Data Plane In addition to the data plane operation described in Section 4, the following operations must be done at CAG to enable seamless interworking between the fabrics. 6.1.4.1. Inter-subnet - L2 fabric to Symmetric IRB fabric Intersubnet traffic is sent from host in the Layer-2 only fabric destined to host IP attached to the symmetric IRB fabric and GW MAC on CAG. The L2 PE performs a Gateway MAC lookup in MAC-VRF and tunnels traffic to CAG with L2 VPN label and tunnel encapsulation to CAG. CAG performs a GW MAC lookup in the MAC-VRF followed by an IP lookup in IP-VRF. This results a route to the L2/L3 PE next-hop. CAG tunnels the packet to L2/L3 PE with L3 VPN label and tunnel encapsulation to L2/L3 PE. L2/L3 PE performs a lookup in the IP-VRF which results in an adjacency to the destination host, using which traffic is routed to the attached destination host. In the topology above, traffic sourced from H11 destined to H24 is sent to M10 and IP24. PE1/2 perform a lookup of M10 and tunnel traffic to CAG1/2. CAG1/2 perform a lookup of M10 in the MAC-VRF, followed by lookup of IP24 in IP-VRF, which results in a route to PE7/8. CAG1/2 tunnel Layer-3 traffic to PE7/8. PE7/8 perform a lookup of IP24 in IP-VRF, which results in adjacency to H24. PE7/8 bridge traffic to H24 destined to M24. 6.1.4.2. Inter-subnet - Symmetric IRB fabric to L2 fabric Intersubnet traffic is sent from host in symmetric IRB fabric destined to host IP attached to Layer-2 only fabric and DAG MAC on L2/L3 PE. L2/L3 PE performs GW MAC lookup in MAC-VRF followed by an IP lookup in the IP-VRF. This results in a route to the CAG next- hop. L2/L3 PE tunnels the packet to the CAG with L3 VPN label and tunnel encapsulation to CAG. CAG performs destination IP lookup in IP-VRF resolved by L3 VPN label, which results in an adjacency to the destination host. CAG performs a MAC lookup of destination host MAC and tunnels traffic to L2 PE next-hop with L2 label and tunnel encapsulation to L2 PE. L2-only PE performs a MAC lookup in the MAC- VRF and bridges the packet to the destination host. Malhotra, et al. Expires 5 September 2024 [Page 11] Internet-Draft EVPN CAG March 2024 In the topology above, traffic sourced from H24 destined to H11 is sent to M20 and IP11. PE7/8 perform a lookup of M20 followed by IP11 lookup in IP-VRF and tunnels Layer-3 traffic to CAG. CAG performs a lookup of IP11 in IP-VRF, which results in adjacency to H11. CAG perform lookup M11 and tunnels traffic to PE1/2. PE1/2 perform lookup of M11 and bridge traffic to H11. 6.1.4.3. Intra-subnet - Symmetric IRB fabric to L2 fabric unicast 6.1.4.4. Intra-subnet - L2 fabric to Symmetric IRB fabric unicast 6.1.4.5. Intra-subnet - Symmetric IRB fabric to L2 fabric BUM 6.1.4.6. Intra-subnet - L2 fabric to Symmetric IRB fabric BUM 6.2. CAG Placement on DAG L2/L3PEs In this placement approach, all L2/L3 PEs in the Symmetric IRB fabric that locally host an EVI assume an additional role of a CAG for hosts in that EVI that are attached to L2 only fabric. EVI1 H14:[IP14 M14] EVI2 H24:[IP24,M24] H25:[IP25, M25] | | +----------------------------------------------------------------+ | | | | | | | | | +--+--+ +--+--+ | | | / \ | | / \ | | | +-----+ +-----+ | | / \ / \ | | / \ / \ | | / \ / \ | | +-----+ +-----+ +-----+ +-----+ | | | PE7 | | PE8 | | PE9 | | PE10| | | | CAG | | CAG | | CAG | | CAG | | | +-----+ +-----+ +-----+ +-----+ | | | | IRB1: [IP10, M10] | | IRB2: [IP20, M20] | Malhotra, et al. Expires 5 September 2024 [Page 12] Internet-Draft EVPN CAG March 2024 | IP-VRF | +----------------------------------------------------------------+ Symmetric IRB Fabric/CAG pair +---------------+ | | | EVPN | | | +---------------+ Layer-2 EVPN Fabric +------------------------------------------------------------------------+ | | | +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ | | | PE1 | | PE2 | | PE3 | | PE4 | | PE5 | | PE6 | | | +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ | | \ / \ / \ / | | \ / \ / \ / | | \ / \ / \ / | | +\---/+ +\---/+ +\---/+ | | | \ / | | \ / | | \ / | | | +--+--+ +--+--+ +--+--+ | | | | | | | | | | | +------------------------------------------------------------------------+ | | | EVI1 H11:[IP11, M11] H13:[IP13, M13] EVI2 H21:[IP21, M21] H22:[IP22, M22] Figure 3 * Symmetric IRB Fabric is EVPN fabric consisting of L2/L3 PEs and L3-only PEs operating in Symmetric IRB mode. * PE7- PE10 are L2/L3 PEs and participate in EVI/EVI2 and host IP- VRF which has subnets for EVI1/EVI2. * PE7 - PE10 are L2/L3 PEs and also CAG to Layer-2 only fabric. * IRB1/IRB2 are IRB interfaces on PE7/CAG - PE10/CAG. Malhotra, et al. Expires 5 September 2024 [Page 13] Internet-Draft EVPN CAG March 2024 6.2.1. Dual role of DAGs In this placement method, each DAG in the symmetric IRB fabric plays a dual role of DAG and CAG. Thus, the CAG for an EVI comprises of each L2/L3 PE DAG node in the symmetric IRB fabric that hosts the EVI. Such a placement results in forming a full mesh of tunnels between the L2-only PEs and every DAG/CAG. 6.2.2. Operation on CAG as L2/L3 GW The operation on the CAG remains the same as described in section FIX and MUST be performed by each member of the CAG pair that hosts the EVI. 6.2.3. FHG Load-sharing and resiliency As the CAG pair is comprised of all L2/L3 PEs that host the EVI, the FHG function can be load-shared across a larger number of CAGs. As all members of the pair advertise the GW MAC/IP to all the L2-only PEs, this opens up an opportunity on the L2-only PEs for creating a scheme optimal load-sharing, load-balancing, failure resiliency and achieving faster convergence during failure events. As an example, the L2-only PEs may choose to pick one CAG per EVI as the FHG, resulting in load-sharing traffic across the pair or the L2-only PEs may pick a subset of CAGs for per EVI as FHGs which provides failure resiliency, faster convergence in failure events while providing load-sharing across the members of the CAG pair. 6.2.4. FHG Localization If the use case requires FHG function to be localized on a subset of members of the CAG pair for a given EVI, only this subset MUST be configured to advertise the GW MAC/IP to the L2-only PEs. Thus forming two sub-clusters within the CAG cluster. The subset which does not advertise the GW MAC/IP form a CAG non-FHG sub-cluster whereas, the other cluster forms a CAG FHG sub-cluster. As the non- FHG cluster does not advertise GW MAC/IP, it does not attract inter- subnet traffic from L2-only PEs. Both sub-clusters MUST perform the operations as described in section FIX. 6.2.5. Comparison between the approaches 6.2.5.1. Operational simplicity In the full mesh placement approach, the operation on CAGs is largely simplified as re-origination of routes as described in section FIX is not required and thus, does not require and additional functionality on the CAG for seamless interworking between the fabrics. Malhotra, et al. Expires 5 September 2024 [Page 14] Internet-Draft EVPN CAG March 2024 6.2.5.2. Scalability The interconnect placement approach requires re-origination of routes by CAG between the two fabrics. This results in logically separated domains. As a result, segregated tunnel domains are be created, thus making this approach more scalable. In the full mesh placement, as a full mesh of tunnels is formed between the L2-only PEs and CAG, thus making this approach less scalable. 7. Operational Considerations None 8. Security Considerations Security considerations discussed in [RFC7432] and [RFC9135] apply to this document. 9. IANA Considerations 10. Acknowledgements Authors would like to thank Aparna Pattekar for spending considerable time on and contributing multiple sections to rev0 of this draft. 11. Contributors Aparna Pattekar Cisco Systems Email: apjoshi@cisco.com 12. Normative References [EVPN-IRB] Sajassi, A., Salam, S., Samil, S., Drake, J., Rabadan, J., and , "Integrated Routing and Bridging in EVPN", Work in Progress, Internet-Draft, Integrated Routing and Bridging in EVPN, 26 July 2021, . [EXTENDED-MOBILITY-EVPN-IRB] Malhotra, N., Sajassi, A., Pattekar, A., Rabadan, J., Lingala, A., Drake, J., and , "Extended Mobility Procedures for EVPN-IRB", Work in Progress, Internet- Draft, draft-ietf-bess-evpn-irb-extended-mobility-17, 16 October 2023, . Malhotra, et al. Expires 5 September 2024 [Page 15] Internet-Draft EVPN CAG March 2024 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . [RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February 2015, . [RFC7432bis] Sajassi, A., Brudet, L.A., Rabadan, J., Drake, J., and , "BGP MPLS-Based Ethernet VPN", Work in Progress, Internet- Draft, draft-ietf-bess-rfc7432bis-08, 13 February 2024, . [RFC9014] Rabadan, J., Sathappan, S., Henderickx, W., Sajassi, A., Drake, J., and , "Interconnect Solution for Ethernet VPN (EVPN) Overlay Networks", 26 May 2021, . Authors' Addresses Neeraj Malhotra (editor) Cisco Systems 170 W. Tasman Drive San Jose, CA 95134 United States of America Email: nmalhotr@cisco.com Krishnaswamy Ananthamurthy Cisco Systems 170 W. Tasman Drive San Jose, CA 95134 United States of America Email: kriswamy@cisco.com Ali Sajassi Cisco Systems 170 W. Tasman Drive San Jose, CA 95134 United States of America Email: sajassi@cisco.com Malhotra, et al. Expires 5 September 2024 [Page 16] Internet-Draft EVPN CAG March 2024 Lukas Krattiger Cisco Systems 170 W. Tasman Drive San Jose, CA 95134 United States of America Email: lkrattig@cisco.com Jorge Rabadan Nokia 777 E. Middlefield Road Mountain View, CA 94043 United States of America Email: jorge.rabadan@nokia.com Malhotra, et al. Expires 5 September 2024 [Page 17]