Network Working Group K. Majumdar
Internet Draft Oracle
Intended status: Standard Track L. Dunbar
Expires: August 18, 2025 Futurewei
V.Kasiviswanathan
Arista
A. Ramchandra
Microsoft
A. Choudhary
Aviatrix
February 18, 2025
Multi-segment SD-WAN via Cloud DCs
draft-ietf-rtgwg-multisegment-sdwan-02
Abstract
This document describes a method for SD-WAN Customer Premises
Equipment (CPEs) to use GENEVE encapsulation (RFC8926) to
transport IPsec-encrypted packets across a Cloud Backbone
without requiring decryption at Cloud Gateways (GWs). In this
approach, SD-WAN CPEs encapsulate IPsec-encrypted payloads
within GENEVE headers and forward them to their nearest Cloud
GWs. These Cloud GWs then steer the encrypted traffic through
the Cloud Backbone while preserving its confidentiality,
ensuring seamless transport to the destination Cloud GWs. The
egress Cloud GWs subsequently deliver the original IPsec-
encrypted payloads to the receiving CPEs. This mechanism
enables the Cloud Backbone to interconnect multiple SD-WAN
segments efficiently, eliminating the need for Cloud GWs to
decrypt and re-encrypt payloads, thus enhancing the
performance.
Status of this Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet
Engineering Task Force (IETF), its areas, and its working
groups. Note that other groups may also distribute working
documents as Internet-Drafts.
xxx, et al. Expires August 18, 2025 [Page 1]
Internet-Draft Multi-segment SD-WAN
Internet-Drafts are draft documents valid for a maximum of
six months and may be updated, replaced, or obsoleted by
other documents at any time. It is inappropriate to use
Internet-Drafts as reference material or to cite them other
than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed
at http://www.ietf.org/shadow.html
This Internet-Draft will expire on Dec 18, 2025.
Copyright Notice
Copyright (c) 2025 IETF Trust and the persons identified as
the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date
of publication of this document. Please review these
documents carefully, as they describe your rights and
restrictions with respect to this document. Code Components
extracted from this document must include Simplified BSD
License text as described in Section 4.e of the Trust Legal
Provisions and are provided without warranty as described in
the Simplified BSD License.
Table of Contents
1. Introduction..............................................3
2. Conventions used in this document.........................5
3. Use Cases.................................................6
3.1. Multi-segment SD-WAN via a Single Cloud GW...........6
3.2. Multi-segment SD-WAN via Cloud Backbone..............7
3.3. Traffic Steering Challenges in Multi-Segment SD-WAN..8
4. Data Plane encoding for SD-WAN Transit....................9
4.1. Multi-Segment SD-WAN Option Class....................9
4.2. SD-WAN Tunnel Endpoint Sub-TLV......................11
4.3. SD-WAN Tunnel Originator Sub-TLV....................12
Dunbar, et al. Expires Dec 18, 2025 [Page 2]
Internet-Draft Multi-segment SD-WAN
4.4. Egress GW Sub-TLV...................................13
4.5. Include Transit Sub-TLV.............................14
4.6. Exclude Transit Sub-TLV.............................15
5. Packet Header Processing.................................16
6. Error Handling...........................................18
7. Control Plane considerations.............................19
7.1. Control Plane for CPEs..............................19
7.2. Control Plane between CPEs and Cloud GWs............19
8. Observability Consideration..............................19
9. Security Considerations..................................20
9.1. Threat Analysis.....................................20
9.2. HMAC-based Integrity and Authentication.............22
10. Manageability Considerations............................24
11. IANA Considerations.....................................25
12. References..............................................26
12.1. Normative References...............................26
12.2. Informative References.............................27
13. Acknowledgments.........................................27
Appendix A: Illustration of Packets through Cloud GWs.......28
A.1 Single Hop Cloud GW.....................................28
A.2 Multi-hop Transit GWs...................................29
Appendix B: Illustration from Private VPN to IPsec Tunnel...31
1. Introduction
SD-WAN is widely deployed to connect enterprises' on-premises
CPEs with services in Cloud DCs. As described in Section 4.1
of [Net2Cloud], enterprises have multiple options for
connecting to Cloud DCs:
- Direct Interconnect model,
- Direct Interconnect model with Enterprise's virtual
appliances in the Cloud,
- Indirect Interconnect model via SD-WAN paths and
- Managed Hybrid WAN model using Enterprise's existing VPN
connections.
In many deployments, enterprises operate multiple SD-WAN
segments to optimize network performance, enhance security,
and meet compliance requirements. Multi-segment SD-WAN refers
to SD-WAN deployments where different segments-such as branch
offices, cloud regions, and data centers-are independently
managed and connected through a backbone network. This
Dunbar, et al. Expires Dec 18, 2025 [Page 3]
Internet-Draft Multi-segment SD-WAN
architecture is often necessary when enterprises span
multiple geographic regions, use different SD-WAN vendors, or
have regulatory constraints requiring segmented traffic
management.
Interconnecting these SD-WAN segments via a Cloud Backbone
provides several key benefits:
a) Seamless connectivity - Enterprises can integrate
geographically dispersed SD-WAN segments into a unified
network without complex manual configurations.
b) Scalability - The Cloud Backbone's elasticity accommodates
increased traffic demands without requiring extensive on-
premises infrastructure.
c) Simplified operations - Centralized orchestration
streamlines policy enforcement and network management
across all segments.
To ensure security, enterprise traffic between CPEs remains
encrypted and inaccessible to third parties, including Cloud
DCs. However, for encrypted packets to be steered through the
Cloud Backbone, routing information must be included in the
packet headers. Since the IPsec Security Association (SA)
exists only between CPEs and is not accessible to Cloud GWs,
an additional tunnel between the source CPE and ingress Cloud
GW is required. This tunnel could be another layer of IPsec,
but decrypting the outer IPsec tunnel solely for routing
introduces significant drawbacks:
- Processing Overhead - Cloud GWs must decrypt the outer
IPsec layer just to extract routing information,
increasing CPU load and reducing efficiency.
- Latency - The additional decryption and re-encryption
steps introduce delays, degrading performance.
- Scalability Constraints -decrypting every IPsec packet at
Cloud GWs becomes a bottleneck, making it inefficient for
high-volume traffic.
- Cloud IPsec Gateway Limits - Many cloud providers cap
IPsec processing per instance, requiring additional
gateways at extra cost when limits are exceeded. AWS
Dunbar, et al. Expires Dec 18, 2025 [Page 4]
Internet-Draft Multi-segment SD-WAN
restricts SAs and throughput per CGW instance, Azure
imposes per-gateway limits, and Google Cloud enforces
per-tunnel throughput caps.
If encrypted traffic is forwarded through the Cloud Backbone
without decryption or re-encryption at Cloud GWs, it reduces
processing overhead, latency, and scalability constraints
while maintaining end-to-end encryption. Since the payload
remains encrypted between CPEs, security and compliance
requirements are upheld without exposing sensitive data at
transit points.
This document introduces a method for SD-WAN CPEs to
encapsulate IPsec-encrypted packets using GENEVE
Encapsulation (RFC8926). CPEs send the GENEVE encapsulated
packets to their nearest Cloud GWs, where Sub-TLVs in the
GENEVE header indicate whether the packets should traverse
the Cloud Backbone without decryption. If backbone traversal
is required, the encrypted payload is forwarded to the
optimal egress Cloud GWs, which then deliver the original
IPsec-encrypted payload to the destination CPEs.
This method enables seamless multi-segment SD-WAN
connectivity over the Cloud Backbone while eliminating the
need for Cloud GWs to decrypt and re-encrypt payloads. GENEVE
is chosen for encapsulation due to its widespread use in
Cloud DC environments.
2. Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL
NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED",
"MAY", and
"OPTIONAL" in this document are to be interpreted as
described in BCP14 [RFC2119] [RFC8174] when, and only when,
they appear in all
capitals, as shown here.
The following acronyms and terms are used in this document:
Dunbar, et al. Expires Dec 18, 2025 [Page 5]
Internet-Draft Multi-segment SD-WAN
Cloud DC: Off-Premises Data Center, managed by the third
party, that hosts applications, services, and
workload for different organizations or tenants.
CPE: Customer (Edge) Premises Equipment.
OnPrem: On Premises data centers and branch offices.
RR Route Reflector.
SA IPsec Security Association
SD-WAN An overlay connectivity service that optimizes
transport of IP Packets over one or more Underlay
Connectivity Services and determining forwarding
behavior by applying Policies to them. [MEF-70.1]
VPN Virtual Private Network.
3. Use Cases
3.1. Multi-segment SD-WAN via a Single Cloud GW
Enterprise branches with established SD-WAN paths to a Cloud
GW for accessing cloud services can also use the Cloud GW to
interconnect with one another, as shown in Figure 1. This
approach offers several advantages:
- Reliable Connectivity - The public internet between
branches may suffer from limited bandwidth, unpredictable
performance, and security risks. In contrast, SD-WAN paths
to a Cloud GW provide more stable, monitored connections
with enhanced network functions.
- Integrated Security - Cloud-based security services, such
as firewalls and DDoS protection, can be centrally applied
to enforce consistent policies across workloads and
branches.
- Advanced Threat Intelligence - Some cloud environments
offer proprietary security tools and SaaS-based analytics
that help monitor and mitigate threats in network traffic.
Dunbar, et al. Expires Dec 18, 2025 [Page 6]
Internet-Draft Multi-segment SD-WAN
(^^^^^^^^^^^^)
( Cloud )
( +----+ +----+ )
+ -----(-|Edge| + GW | )
Direct | ( +----+ +/--\+ )
Connect | (^^^^^^^/^^^^\^)
{-+---} / \ SD-WAN Path CPE<->GW
{ VPN } / \
{-+---} / IPsec Tunnel
+-------+----/------+ \
| / | \
++--/+ | +-\--+
|CPE1| +----+CPE2|
+----+ +----+
Client Route: 11.1.1.x 10.1.1.x
21.1.1.x 20.1.1.x
30.1.1.x
Figure 1 Multi-Segment SD-WAN stitching via a Cloud GW
3.2. Multi-segment SD-WAN via Cloud Backbone
For geographically distant enterprise branches that have
established SD-WAN paths to their respective Cloud GWs for
accessing cloud services, the Cloud Backbone provides an
efficient way to interconnect these branches, as shown in
Figure 2. As outlined in the Introduction section, this
approach enhances network integration, supports dynamic
scaling, and simplifies overall management, making it well-
suited for multi-segment SD-WAN deployments across different
regions.
Dunbar, et al. Expires Dec 18, 2025 [Page 7]
Internet-Draft Multi-segment SD-WAN
(^^^^^^^^^^^^^^^)
( Cloud )
( +----+ +----+ ) +-----+
+ ---(-|Edge|==| GW1|=================== GW2 |
Direct | ( +----+ +/--\+ ) +--|--+
Connect | (^^^^^^^/^^^^\^) |
{-+---} / \ |
{ VPN } / \ +-----+
{-+---} / IPsec Tunnel |CPE10|
+-------+--/--------+ \ +-----+
| / | \
10.2.1.x
++/--+ | +\---+
20.2.1.x
|CPE1| +----+CPE2|
30.2.1.x
+----+ +----+
Client Route: 11.1.1.x 10.1.1.x
21.1.1.x 20.1.1.x
30.1.1.x
Figure 2 Multi-Segment SD-WAN Stitching via Cloud Backbone
3.3. Traffic Steering Challenges in Multi-Segment SD-WAN
Many well-established traffic engineering methods, such as
SRv6 and MPLS-TE, effectively steer traffic through specific
network nodes when the entire network operates under a single
administrative domain.
However, in SD-WAN deployments where on-premises CPEs connect
to Cloud GWs over the public internet, traffic forwarding is
typically destination-based, limiting the ability to enforce
precise traffic steering policies. Even when branch CPEs have
dedicated SD-WAN paths to Cloud GWs, the paths between
branches may still be routed over the public internet via any
available route, bypassing the Cloud Backbone entirely.
This lack of predictable routing makes traffic steering
between branch offices highly challenging. Unlike private
MPLS networks or provider-controlled backbones, SD-WAN cannot
inherently dictate the intermediate paths for branch-to-
Dunbar, et al. Expires Dec 18, 2025 [Page 8]
Internet-Draft Multi-segment SD-WAN
branch traffic. As a result, policies intended to optimize
performance, enforce security, or ensure compliance can be
difficult to implement.
To address this issue, this document proposes a method where
Cloud GWs explicitly interconnect SD-WAN segments, ensuring
that branch-to-branch traffic is steered through the Cloud
Backbone rather than taking unpredictable internet routes.
This approach provides greater control over traffic flows,
improving reliability, security, and policy enforcement.
4. Data Plane encoding for SD-WAN Transit
To enable Cloud GWs to distinguish between packets requiring
decryption for internal cloud services and transit packets
that should be forwarded to destination CPEs, proper packet
marking is essential. Since GENEVE Encapsulation [RFC8926] is
widely supported by major Cloud Service Providers, it is
chosen as the encapsulation method. This allows Cloud GWs to
efficiently steer IPsec-encrypted packets between CPEs
without decryption, reducing processing overhead and
improving performance while maintaining end-to-end
encryption.
4.1. Multi-Segment SD-WAN Option Class
Geneve header format is specified in Section 3 of [RFC8926].
This document introduces a new GENEVE Option Class (Type
value = 0x0163) to indicate that Multi-Segment SD-WAN-
specific Sub-TLVs are encoded within the GENEVE header. This
enables Cloud GWs to interpret and process SD-WAN transit
packets efficiently without requiring decryption.
Dunbar, et al. Expires Dec 18, 2025 [Page 9]
Internet-Draft Multi-segment SD-WAN
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Multi-seg-SD-WAN Option Class |C| Type |R|R|R| Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
~ SD-WAN Tunnel Endpoint Sub-TLV ~
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
~ Optional SD-WAN Tunnel Originator Sub-TLV ~
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
~ Optional Egress GW Sub-TLV ~
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
// //
// Optional Type Length Value objects (variable) //
// //
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 3 Multi Segment SD-WAN Option Class
Multi-seg-SD-WAN Option Class is a newly defined GENEVE
Option Class (16 bits, Type value = 0x0163, assigned by IANA)
that identifies packets carrying Multi-Segment SD-WAN-
specific Sub-TLVs within the GENEVE header .
- C-bit: Must be set to ensure that a receiving node drops
the packet if it does not recognize the option, as per
[RFC8926].
- Type (8 bits): Specifies the multi-segment SD-WAN
forwarding model:
Type = 1: Single-hop transit SD-WAN
Type = 2: Multi-Hop transit SD-WAN with an explicitly
specified egress Cloud GW (via Sub-TLV).
Type = 3: Multi-hop transit SD-WAN without an explicitly
specified egress Cloud GW.
- Length (5 bits): Indicates the total length of the option
fields in 4-byte units. If no options are present, this
field is zero [RFC8926].
Dunbar, et al. Expires Dec 18, 2025 [Page 10]
Internet-Draft Multi-segment SD-WAN
Note: the payload following the multi-seg-SD-WAN Option Class
can be IPv4 or IPv6. The Protocol Type of the GENEVE header
is set to 50, indicating the GENEVE payload carries IPsec ESP
[RFC8926][IPsecOverGENEVE].
4.2. SD-WAN Tunnel Endpoint Sub-TLV
The SD-WAN Endpoint sub-TLV indicates the destination CPE,
which is the endpoint of the IPsec Tunnel between branch
CPEs. This Sub-TLV is used by the Cloud Backbone to determine
the optimal egress Cloud GW for forwarding the encrypted
traffic.
For example, in an SD-WAN deployment where CPE1 establishes
an IPsec SA with CPE2 (as shown in Figure 1), this Sub-TLV
within the GENEVE header contains CPE2's IP address, ensuring
that encrypted traffic is correctly routed to the terminating
CPE of the IPsec tunnel while enabling the Cloud Backbone to
steer the packet to the most suitable egress Cloud GW.
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|SD-WAN Endpoint| length | Reserved | TTL |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SD-WAN Dst Addr Family | Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ (variable) +
~ ~
| SD-WAN end point Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 4 SD-WAN Endpoint Sub-TLV
- SD-WAN Endpoint (8 bits): Identifies the SD-WAN Tunnel
Endpoint Sub-TLV with a Type value of 1.
- Length (8 bits): Specifies the total length of the value
field in 4-byte units.
Dunbar, et al. Expires Dec 18, 2025 [Page 11]
Internet-Draft Multi-segment SD-WAN
- TTL (Time to Live): set by the SD-WAN Tunnel Originator
(e.g., CPE1) to track the number transit nodes or cloud
regions/zones that the packet traverses. Each transit
node or region (visible to the CPEs) decrement the TTL
to allow the egress Cloud GW to determine the number of
logical transits the packet has traversed. Enterprises
can also define the TTL value to enforce the maximum
transit nodes/regions the packets traverse.
4.3. SD-WAN Tunnel Originator Sub-TLV
The SD-WAN Tunnel Originator Sub-TLV is an optional Sub-TLV
within the multi-seg-SD-WAN Option Class to indicate the
originating CPE of the IPsec Tunnel.
For example, in an SD-WAN deployment where CPE1 establishes
an IPsec SA with CPE2 (as shown in Figure 1), this Sub-TLV
within the GENEVE header carries CPE1's address, allowing
transit nodes and Cloud GWs to recognize the source of the
encrypted traffic.
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|SDWAN Origin | length | reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SD-WAN Org Addr Family | Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ (variable) +
~ ~
| SD-WAN Tunnel Originator Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 5 SD-WAN Tunnel Originator Sub-TLV
- SDWAN Origin (8 bits): Identifies the SDWAN Tunnel
Originator Sub-TLV with a Type value of 2.
- Length (8 bits): Specifies the total length of the value
field in 4-byte units, excluding the first 4 bytes,
which include the SD-WAN Origin (1 byte), Length (1
byte), and Reserved (2 bytes) fields.
- Reserved (16 bits): Reserved for future. Must set to 0.
Ignored by recipients.
Dunbar, et al. Expires Dec 18, 2025 [Page 12]
Internet-Draft Multi-segment SD-WAN
This Sub-TLV allows Cloud GWs and transit nodes to identify
the packet's source, allowing them to apply source specific
policies for forwarding. These policies may include routing
optimization based on the origin, security enforcement
tailored to the source, or traffic engineering rules specific
to the originating CPE.
4.4. Egress GW Sub-TLV
In a multi-segment SD-WAN deployment over the Cloud Backbone,
the originating CPE can use the Egress GW Sub-TLV to
explicitly specify the egress Cloud GW responsible for
forwarding traffic to the destination CPE. This ensures
predictable routing and optimized packet delivery across the
Cloud Backbone.
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|SDWAN EgressGW | length | reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Egress GW Addr Family | Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ (variable) +
~ ~
| Egress GW Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 6 SD-WAN Egress GW Sub-TLV
- SDWAN EgressGW (8 bits): Identifies the Egress GW Sub-
TLV with a Type value of 3.
- Length (8 bits): Specifies the total length of the value
field in 4-byte units, excluding the first 4 bytes,
which include the SD-WAN Origin Sub-TLV Type (1 byte),
Length (1 byte), and Reserved (2 bytes) fields .
- Reserved (16 bits): Reserved for future. Must set to 0.
Ignored by recipients.
The Egress GW Sub-TLV allows the originating CPE to specify
the egress Cloud GW responsible for forwarding traffic to the
destination CPE. This egress GW address can be either
preconfigured or dynamically discovered through a control
plane protocol exchange with the destination CPE. By
explicitly defining the egress GW, this Sub-TLV ensures
predictable traffic steering, reducing reliance on
destination-based routing and optimizing packet delivery
Dunbar, et al. Expires Dec 18, 2025 [Page 13]
Internet-Draft Multi-segment SD-WAN
across the Cloud Backbone. The details of the control plane
protocol used for egress GW discovery are beyond the scope of
this document.
4.5. Include Transit Sub-TLV
Include-Transit Sub-TLV is an optional field that allows
explicitly specifying a list of Cloud Availability Regions or
Zones that a packet must traverse when forwarded through the
Cloud Backbone. This can be useful for:
- Enhancing observability and security by directing traffic
through regions with specific OAM and security functions.
- Ensuring compliance with regulatory or business
requirements that mandate traffic to pass through
designated locations.
Multiple Include-Transit Sub-TLVs can be included within a
single GENEVE header to indicate multiple required transit
nodes or regions. However, these Sub-TLVs form an unordered
set, meaning there is no enforced sequence in which the
specified regions must be traversed.
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Include-Transit| length |Transit_Type |I|Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Transit node ID |
~ ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 7 Include-Transit Sub-TLV
- Include-Transit (8 bits): identifies the Include-Transit
Sub-TLV with a Type value of 4.
- Length (8 bits): Specifies the total length of the value
field in 4-byte units, excluding the first 4 bytes,
which include the Include-Transit Sub-TLV Type (1 byte),
Length (1 byte), Transit_Type (1 byte), I-bit (1 bit),
and Reserved bits (7 bits).
- Transit_type (8 bits): Defines how the transit node is
identified:
Dunbar, et al. Expires Dec 18, 2025 [Page 14]
Internet-Draft Multi-segment SD-WAN
o Transit_type = 1: The Transit Node ID is
represented as a numeric identifier, such as a
Cloud Availability Region or Zone number assigned
by the cloud provider.
o Transit_type = 2: The Transit Node ID is
represented as an IP address.
- I-bit (1 bit) - Inclusion Requirement:
o 0: The transit node is a preferred hop, meaning the
Cloud Backbone will make a best-effort attempt to
route traffic through it.
o 1: The transit node is mandatory, meaning the
packet must traverse this node. If the node cannot
be reached, an alert or alarm must be generated to
notify the enterprise via an out-of-band mechanism.
The specifics of these alerts are beyond the scope
of this document.
4.6. Exclude Transit Sub-TLV
Exclude-Transit Sub-TLV is an optional field that explicitly
specifies a list of Cloud Availability Regions or Zones that
must be avoided when forwarding packets through the Cloud
Backbone. This can be used for:
- Regulatory compliance, ensuring traffic does not traverse
restricted or non-compliant regions.
- Risk mitigation, preventing traffic from passing through
regions with known security, performance, or geopolitical
concerns.
Multiple Exclude-Transit Sub-TLVs can be included within a
single GENEVE header to specify multiple excluded transit
nodes or regions. However, these Sub-TLVs form an unordered
set, meaning there is no defined sequence in which exclusions
are applied.
Dunbar, et al. Expires Dec 18, 2025 [Page 15]
Internet-Draft Multi-segment SD-WAN
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Exclude-Transit| length |Transit_Type |E| Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Transit node ID |
~ ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 8 Exclude-Transit Sub-TLV
- Exclude -Transit (8 bits): identifies the Exclude -
Transit Sub-TLV with a Type value of 5.
- Length (8 bits): Specifies the total length of the value
field in 4-byte units, excluding the first 4 bytes,
which include the Exclude-Transit Sub-TLV Type (1 byte),
Length (1 byte), Transit_Type (1 byte), E-bit (1 bit),
and Reserved bits (7 bits).
- Transit_type (8 bits): Defines how the transit node is
identified:
Transit_type = 1: The Transit Node ID is represented as
a numeric identifier, such as a Cloud Availability
Region or Zone number assigned by the cloud provider.
Transit_type = 2: The Transit Node ID is represented as
an IP address.
- E-bit (1 bit) - Exclusion Requirement:
o 0: The specified transit node is undesirable, and
the Cloud Backbone will make a best-effort attempt
to route traffic around it.
o 1: The specified transit node must be avoided. If
the node cannot be bypassed, an alert or alarm must
be generated to notify the enterprise via an out-
of-band mechanism. The specifics of these alerts
are beyond the scope of this document.
5. Packet Header Processing
As illustrated in Figure 1, when a Cloud GW receives a
GENEVE-encapsulated packet with Protocol Type = 50 (ESP), it
processes the packet as follows:
Processing at the Ingress Cloud GW
Dunbar, et al. Expires Dec 18, 2025 [Page 16]
Internet-Draft Multi-segment SD-WAN
- Authenticate the packet using a preconfigured
authentication method.
- Extract the destination CPE address from the SD-WAN
Endpoint Sub-TLV within the GENEVE header.
- Check if the Egress GW Sub-TLV is present:
o If the Egress GW Sub-TLV exists, the Cloud Backbone
uses it to identify the egress Cloud GW.
o If the Egress GW Sub-TLV is not present, the Cloud
Backbone determines the optimal egress Cloud GW
based on the destination CPE address.
- Change the destination address in the outer IP header
of the GENEVE packet to the address of the identified
egress Cloud GW (either extracted from the GENEVE
header or determined by the Cloud Backbone)..
- Preserve the GENEVE header without modification.
- Forward the packet to the egress Cloud GW.
To prevent unauthorized access, the Cloud GW SHOULD drop
any packets containing unrecognized source addresses or
invalid values in the GENEVE Sub-TLVs, ensuring that only
registered entities can utilize Cloud services.
Processing at the Egress Cloud GW:
- Decapsulate the GENEVE header to extract the IPsec-
encrypted payload.
- Perform additional security checks:
- Validate that the SD-WAN Tunnel Endpoint Sub-TLV
corresponds to a registered destination CPE.
- Ensure the source Cloud GW is an authorized forwarding
node to prevent unauthorized traffic injection.
- Forward the IPsec-encrypted payload to the destination
CPE, preserving the end-to-end encryption.
- Drop any packet that lacks a valid destination CPE, or
originates from an untrusted source.
By enforcing these processing steps at both the ingress and
egress Cloud GWs, the system ensures secure, efficient, and
policy-compliant forwarding of SD-WAN traffic across the
Cloud Backbone.
Dunbar, et al. Expires Dec 18, 2025 [Page 17]
Internet-Draft Multi-segment SD-WAN
6. Error Handling
To ensure secure and efficient traffic forwarding through the
Cloud Backbone, the Cloud GW SHOULD enforce the following
error handling measures:
- Drop packets with unregistered or invalid
source/destination addresses to prevent unauthorized
access.
- Reject packets originating from unpaid or unregistered
CPEs to enforce service subscription policies.
- Validate the SD-WAN Endpoint Sub-TLV and drop packets
if the destination CPE is unauthorized, unreachable, or
mismatched.
- Discard malformed packets with incorrect GENEVE
headers, invalid Sub-TLV formats, or authentication
failures.
- Drop packets with expired TTL values to prevent routing
loops and log repeated occurrences.
- Reject misrouted packets if the Cloud Backbone cannot
determine an optimal egress Cloud GW or if the
specified egress GW is unreachable.
- Enforce rate limits on excessive traffic from a single
source to prevent congestion and abuse.
- Verify compliance with transit node policies (e.g.,
ensuring mandatory transit nodes are included and
excluded nodes are avoided).
- Mitigate replay attacks by tracking sequence numbers
and rejecting duplicate packets.
By implementing these error handling mechanisms, Cloud GWs
ensure network stability, security, and efficient resource
utilization while preventing misconfigurations, abuse, and
performance degradation.
Dunbar, et al. Expires Dec 18, 2025 [Page 18]
Internet-Draft Multi-segment SD-WAN
7. Control Plane considerations
7.1. Control Plane for CPEs
The control plane enables SD-WAN CPEs to discover their
network attributes, establish connectivity, and exchange
routing information. In an SD-WAN deployment, on-premises
CPEs and virtual CPEs (vCPEs) in Cloud DCs may be managed
under a common iBGP administrative domain, facilitating route
propagation and policy enforcement.
Mechanisms such as BGP-based SD-WAN Edge Discovery [SD-WAN-
Edge-Discovery] allow CPEs to dynamically discover each
other's properties, improving automation and reducing manual
configurations. Additionally, IPsec SAs parameters between
CPEs and Cloud GWs can be exchanged through the iBGP control
plane using a RR to simplify security policy management.
7.2. Control Plane between CPEs and Cloud GWs
In multi-segment SD-WAN deployments, the control plane
between CPEs and Cloud GWs must support dynamic routing
updates and secure tunnel establishment.
- eBGP sessions are commonly used between enterprise CPEs
and Cloud GWs to facilitate dynamic route exchange.
- If an IPsec tunnel is required between a Cloud GW and a
vCPE, the IKEv2 key exchange must be performed to
establish the tunnel securely.
- Control plane mechanisms must ensure that Cloud GWs can
identify and authenticate SD-WAN CPEs, validate SD-WAN
metadata, and apply appropriate routing policies based
on dynamic network conditions.
By leveraging these control plane considerations, enterprises
can ensure efficient route discovery, security enforcement,
and seamless SD-WAN interconnectivity across the Cloud
Backbone.
8. Observability Consideration
Observability considerations encompass monitoring, analysis,
and reporting mechanisms to gain insights into the behavior
and performance of the multi-segment SD-WAN infrastructure.
Key observability aspects include:
Dunbar, et al. Expires Dec 18, 2025 [Page 19]
Internet-Draft Multi-segment SD-WAN
- Performance Metrics:
Monitor and collect performance metrics related to link
utilization, latency, and packet loss across the SD-WAN
segments and Cloud DC backbone. This data provides insights
into the overall health and efficiency of the network. IP
Flow Information Export (IPFIX) [RFC7011] is one of the
standardized methods to expose traffic flow over the
network.
- Global Network Topology Visualization:
Utilize visualization tools to depict the global network
topology, showcasing the interconnections and traffic flows
between different SD-WAN segments and Cloud DCs.
- Control Plane Monitoring:
Monitor the control plane for both CPEs and the
communication between CPEs and Cloud GWs. This includes
tracking route discovery, path selection, and any changes
in network state to ensure proper functioning of the SD-WAN
control plane.
- Security Event Logging:
The security event logging is to capture and analyze
security-related events, including threat detection,
authentication failures, and any unauthorized access
attempts. Syslog [RFC5424] is a valuable tool for security
monitoring and auditing.
These considerations contribute to the overall success of the
multi-segment SD-WAN deployment connecting edge devices via a
Cloud DC backbone.
9. Security Considerations
9.1. Threat Analysis
As shown in Figure 3, the information carried by the GENEVE
Header is not encrypted, which is susceptible to Man-in-the-
Middle (MitM) attacks. An attacker can intercept and
potentially alter the information in the GENEVE header
between the branch CPEs and the Cloud GWs without the
enterprise and the Cloud provider's knowledge or consent.
Here is the threat analysis of the MitM attacks between CPEs
and Cloud GWs:
Dunbar, et al. Expires Dec 18, 2025 [Page 20]
Internet-Draft Multi-segment SD-WAN
a) Eavesdropping: Attackers can get knowledge of the
enterprise's branch locations and their respective
contracted Cloud GWs. As the payload between the CPEs is
encrypted, attackers can't get any data exchanged between
CPEs. This threat is no different from direct IPsec SAs
between two CPEs.
b) Data Manipulation: Attackers alter the content (Sub-TLVs)
in the GENEVE header. As packets with unrecognized source
addresses or invalid values in the Sub-TLVs of the GENEVE
header are dropped by Cloud GWs, there might be a higher
packet drop rate between the CPEs.
Packet drop is not a new problem. The transport layer, such
as TCP or QUIC, can handle packet drop well.
c) Potential steeling of Cloud Backbone bandwidth:
A threat actor might want to leverage Cloud Backbones to
transport its own traffic between two locations without
paying for the services. For example, a legitimate Cloud
subscriber pays for the Cloud Backbone transport services
for traffic between CPE-A and CPE-B. The attacker, who has
two locations far apart (say Node-A and Node-B), can use
CPE-A's address as the source address and CPE-B as the
value in the SD-WAN Endpoint Sub-TLV for a packet from
Node-A to Node-B before reaching the ingress Cloud GW. When
the packet is sent from the egress Cloud GW via the
Internet towards CPE-B, the actor can change the source
address back to Node-A and the destination address to Node-
B. By doing so, Node-A and Node-B can maintain the IPsec
tunnel via the Cloud Backbone without paying for the
service.
Therefore, it is necessary to have some level data integrity
and authentication for traffic between CPEs and Cloud GWs
even though it is not necessary for Cloud GWs to decrypt and
re-encrypt the payload between CPEs.
[RFC2403] and [RFC2404] define the authentication algorithms
used in AH and ESP. SHA2 224/256/384/512 are some of the
cryptographic hashing algorithms. They are part of a Hashed
Message Authentication Code.
Dunbar, et al. Expires Dec 18, 2025 [Page 21]
Internet-Draft Multi-segment SD-WAN
9.2. HMAC-based Integrity and Authentication
HMAC (Hash-based Message Authentication Code), a widely used
cryptographic technique for ensuring both data integrity and
authentication, can be used to ensure the integrity and
authenticity of data between CPEs and Cloud GWs to verify
that GENEVE header has not been tampered with.
The basic idea behind HMAC is to combine a secret key and a
hash function to produce a fixed-size authentication code for
the GENEVE header between CPEs and the Cloud GW. This
authentication code is then sent along with the data itself.
When the Cloud GW and the destination CPEs receive the data
and the authentication code, they can independently compute
the HMAC using the same key and hash function. If the
computed HMAC matches the received authentication code, it
indicates that the data has not been altered, as long as the
secret key remains confidential.
The HMAC authentication code can be carried by an HMAC Sub-
TLV in the GENEVE Header, as specified below:
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| HMAC-Auth-Val | length | reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~ ~
| HMAC Authentication Value |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 9 Multi Segment SD-WAN HMAC Sub-TLV
HMAC-Auth-Val (8 bits): HMAC Authentication Value Sub-TLV
Type =6 (Assigned by this document).
Length (8 bits): total length of the value field, which is
the length of the HMAC Authentication Value in bytes plus 2
Reserved bytes. It is 6 bytes by default. However, in some
deployments where security requirements are high, a longer
authentication value can be considered.
The HMAC Authentication Value (4 bytes or 8 bytes): is
computed including the entire encapsulation header, excluding
the HMAC-auth-Val Sub-TLV, based on a pre-configured
algorithm.
The advantages of using HMAC are:
Dunbar, et al. Expires Dec 18, 2025 [Page 22]
Internet-Draft Multi-segment SD-WAN
- Data Integrity: HMAC provides a strong mechanism for
verifying the integrity of data. By hashing the message
and using a secret key, it generates a fixed-size hash
value that ensures the data has not been tampered with
during transmission.
- Authentication: HMAC also verifies the authenticity of
the sender. Since HMAC requires a shared secret key
between the sender and receiver, it confirms that the
sender is who they claim to be.
- Efficiency: HMAC is computationally efficient, making it
suitable for real-time and resource-constrained devices.
It uses simple bitwise operations and hash functions.
- Resistance to Tampering: HMAC is designed to resist
various forms of tampering, including replay attacks,
message insertion, and message deletion. Any change in
the message will result in a different HMAC value.
- Flexibility: HMAC can be used with various hash
functions, such as SHA-256 or SHA-512, depending on the
desired level of security requirements.
- Widely Supported: HMAC is a well-established and widely
supported authentication mechanism, making it easy to
integrate into different systems.
Here are some common problems associated with using the HMAC
and why their risks are acceptable in the scenario described
in this draft.
- Key Management: The security of HMAC depends heavily on the
confidentiality and management of the shared secret key. If
the key is compromised, the data packets from CPEs to Cloud
GW can be dropped but not compromised because the user
payloads are protected by IPsec SA encryption.
- Lack of Non-Repudiation: HMAC provides data integrity and
sender authentication but does not provide non-repudiation.
Non-repudiation is the ability to prove that a message was
sent by a specific sender, which HMAC alone cannot
guarantee. This risk is same as two IPsec protected traffic
between CPEs.
- Limited to Symmetric Cryptography: HMAC relies on symmetric
key cryptography, which means that both parties must share
the same secret key. As the Cloud backbone interconnecting
Dunbar, et al. Expires Dec 18, 2025 [Page 23]
Internet-Draft Multi-segment SD-WAN
CPEs are paid services, there are established channels to
distribute the symmetric key.
- No Protection Against Eavesdropping: While HMAC ensures
data integrity and sender authentication, it does not
provide encryption. Eavesdropping does pose additional
risks to payloads encrypted by IPsec SA.
In summary, HMAC-based integrity and authentication offer
strong security benefits in terms of data integrity and
sender authentication. Even though it does not provide non-
repudiation or protection against eavesdropping, the IPsec
encrypted payload between CPEs won't be impacted.
9.3. AH based Integrity and Authentication
For enterprises or Cloud providers worrying about secret HMAC
keys being compromised, they can add another layer of AH
encryption [RFC4301] or ESP-NULL [RFC2410] [RFC6071] on top
of the IPsec encryption between the two CPEs. Both AH and
ESP-NULL IPsec encryption require pairwise IPsec key
management between Cloud GWs and the CPEs, therefore
requiring more processing on Cloud GWs and CPEs. In addition,
the AH encrypted packets can't traverse NAT because of outer
IP address changes.
10. Manageability Considerations
The following manageability considerations are crucial for
the successful deployment and ongoing operation of the
proposed strategies outlined in this document:
- Centralized Orchestration:
A centralized orchestration system is needed to manage
and authenticate multiple SD-WAN segments through the
Cloud GWs.
- Policy-based Configuration:
Utilize policy-driven configurations to streamline the
deployment of SD-WAN segments and their connectivity
options. This approach allows for efficient management
of network policies, ensuring consistent and coherent
behavior across diverse deployment scenarios. [RFC8192]
can be used to automate the security policy
configurations.
Dunbar, et al. Expires Dec 18, 2025 [Page 24]
Internet-Draft Multi-segment SD-WAN
- Real-time Monitoring and Analytics:
Integrate robust monitoring and analytics tools to
provide real-time visibility into the performance and
health of SD-WAN segments. This includes monitoring
bandwidth utilization, latency, packet loss, and other
key performance indicators to promptly identify and
address any issues.
- Automated Alerting and Reporting:
Implement automated alerting mechanisms to promptly
notify network administrators of potential issues or
anomalies within the SD-WAN infrastructure.
Additionally, generate regular reports to facilitate
performance analysis, capacity planning, and compliance
monitoring.
11. IANA Considerations
IANA is requested to assign a new GENEVE Option Class from
the IETF Review range as shown below:
Option
Class Description Assignee/Contact Reference
------ ------------------- ------------- -----------
0x0163 Multi Segment SD-WAN IETF [this document]
IANA is requested to create the registry below and place it
in a new Multi Segment SD-WAN Geneve Option Class registry
group:
Registry: Multi Segment SD-WAN Sub-TLVs
Assignment Policy: IETF Review
Reference: [this document]
Sub-TLV Type Description Reference
------------ ---------------------- ---------------
0 Reserved
1 SD-WAN Endpoint [Section 4.2]
2 SD-WAN Originator [Section 4.3]
3 SD-WAN Egress GW [Section 4.4]
Dunbar, et al. Expires Dec 18, 2025 [Page 25]
Internet-Draft Multi-segment SD-WAN
4 Include Transit [Section 4.5]
5 Exclude Transit [Section 4.6]
6 Multi SD-WAN-HMAC [Section 9.2]
5-254 Unassigned
255 Reserved
12. References
12.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC2403] C. Madson, R. Glenn, "The Use of HMAC-MD5-96 within
ESP and AH", RFC2403, Nov. 1998.
[RFC2404] C. Madson, R. Glenn, "The Use of HMAC-SHA-1-96
within ESP and AH", RFC2404, Nov. 1998.
[RFC4301] S. Kent and K. Seo, "Security Architecture for the
Internet Protocol", RFC4301, Dec. 2005.
[RFC4303] S. Kent, "IP Encapsulating Security Payload (ESP)".
RFC4303, Dec. 2005.
[RFC5424] R. Gerhards, "The Syslog Protocol", RFC5424, March
2009.
[RFC7011] B. Claise, B. Trammell, and P. Aitken,
"Specification of the IP Flow Information Export
(IPFIX) Protocol for the Exchange of Flow
Information", RFC7011, Sept 2013.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in
RFC 2119 Key Words", BCP 14, RFC 8174, DOI
10.17487/RFC8174, May 2017, <https://www.rfc-
editor.org/info/rfc8174>.
[RFC8926] J. Gross, et al, "Geneve: Generic Network
Virtualization Encapsulation", RFC8926, Nov 2020.
Dunbar, et al. Expires Dec 18, 2025 [Page 26]
Internet-Draft Multi-segment SD-WAN
12.2. Informative References
[IPsecOverGENEVE] S. Boutros, et al, "IPsec over GENEVE
Encapsulation", draft-boutros-nvo3-ipsec-over-
geneve-01, work-in-progress, Jan, 2018.
[RFC2410] R. Glenn and S. Kent, "The NULL encryption
Algorithm and Its Use with IPsec", RFC2310, Nov.
1998.
[RFC6071] S. Frankel and S. Krishnan, "IP Security (IPsec)
and Internet Key Exchange (IKE) Document Roadmap",
Feb. 2011.
[RFC8192] S. Hares, et al, "Interface to Network Security
Functions (I2NSF) Problem Statement and Use Cases",
July 2017
[MEF-70.1] MEF 70.1 SD-WAN Service Attributes and Service
Framework. Nov. 2021.
[Net2Cloud] L. Dunbar and A. Malis, "Dynamic Networks to
Hybrid Cloud DCs Problem Statement", draft-ietf-
rtgwg-net2cloud-problem-statement-39, April, 2024.
[SD-WAN-Edge-Discovery] L. Dunbar, et al, "BGP UPDATE for SD-
WAN Edge Discovery", draft-ietf-idr-sdwan-edge-
discovery-12, Oct. 2023.
13. Acknowledgments
Acknowledgements to Adrian Farrel, Donald Eastlake, Stephen
Farrell for their extensive review and suggestions.
This document was prepared using 2-Word-v2.0.template.dot.
Dunbar, et al. Expires Dec 18, 2025 [Page 27]
Internet-Draft Multi-segment SD-WAN
Appendix A: Illustration of Packets through Cloud GWs
This section illustrates Cloud GWs connecting traffic flow
carried by the IPsec tunnels.
A.1 Single Hop Cloud GW
Assuming that all CPEs are under one administrative control
(e.g., iBGP).
Using Figure 1 as an example:
- There is a bidirectional IPsec tunnel between CPE1 and
Cloud GW; with IPsec SA1 for the traffic from the CPE1
to the Cloud-GW; and IPsec SA2 for the traffic from
the Cloud-GW to the CPE1.
- There is a bidirectional IPsec tunnel between CPE2 and
Cloud GW; with IPsec SA3 for the traffic from the CPE2
to the Cloud-GW; and IPsec SA4 for the traffic from
the Cloud-GW to the CPE2.
- All the CPEs are under one iBGP administrative domain,
with a Route Reflector (RR) as their controller. The
CPEs notify their peers of their corresponding Cloud
GW addresses (which is out of the scope of this
document).
When 11.1.1.x and 10.1.1.x need to communicate with each
other, CPE1 and CPE2 establish a bidirectional IPsec
Tunnel, with SA5 for the traffic from CPE1 to CPE2 and SA6
for the traffic from CPE2 to CPE1. Assume the IPsec ESP
Tunnel Mode is used. A packet from 11.1.1.1 to 10.1.1.2 has
the following outer header:
Dunbar, et al. Expires Dec 18, 2025 [Page 28]
Internet-Draft Multi-segment SD-WAN
Outer IP header:
+---------------------------+
| protocol = 17(UDP) |
| src = CPE1 |
| dst = Cloud GW |
+---------------------------+
| Source Port =xxxx |
| Dst Port = 6081 (GENEVE) |
+===========================+
| GENEVE Header |
| Protocol=50(ESP payload) |
+- - -- -- - - -- - --+
|MultiSeg-SDWAN Option Class|
+- - -- -- - - -- - --+
|SD-WAN EndPt SubTLV (CPE2) |
+---------------------------+
|GENEVE Hdr Authentication |<-validated by GW
+---------------------------+ < ----------+
|SPI(Security Parameter Idx)| |
+---------------------------+ |
| sequence number | |
+---------------------------+ <-+ |
| payload IP header: | | |
| src = 11.1.1.1 | | |
| dst = 10.1.1.2 | | |
+---------------------------+ Encrypted |
| TCP header + | | |
~ payload (variable) ~ | |
| | | |
+===========================+ <-+ -------+
|Integrity Check Value (ICV)|<-Generated by CPE1,
+---------------------------+ validated by CPE2
Figure 10 Packet header illustration to Cloud GWs
A.2 Multi-hop Transit GWs
Traffic to/from geographic apart CPEs can cross multiple
Cloud DCs via Cloud backbone.
The on-premises CPEs are under one administrative control
(e.g., iBGP).
Using Figure 2 as an example:
Dunbar, et al. Expires Dec 18, 2025 [Page 29]
Internet-Draft Multi-segment SD-WAN
- There is a bidirectional IPsec tunnel between CPE1 and
the Cloud GW1; with IPsec SA1 for the traffic from the
CPE1 to the Cloud-GW1; and IPsec SA2 for the traffic
from the Cloud-GW1 to the CPE1.
- There is a bidirectional IPsec tunnel between CPE10
and the Cloud GW2; with IPsec SA3 for the traffic from
the CPE10 to the Cloud-GW2; and IPsec SA4 for the
traffic from the Cloud-GW2 to the CPE10.
- All the CPEs are under one iBGP administrative domain,
with a Route Reflector (RR) as their controller. CPEs
notify their peers of their corresponding Cloud GW
addresses.
When 11.1.1.x and 10.2.1.x need to communicate with each
other, CPE1 and CPE10 establish a bidirectional IPsec
Tunnel, with SA5 for the traffic from CPE1 to CPE10 and SA6
for the traffic from CPE10 to CPE1. Assume the IPsec ESP
Tunnel Mode is used, a packet from 11.1.1.1 to 10.2.1.2 has
the following outer header:
Dunbar, et al. Expires Dec 18, 2025 [Page 30]
Internet-Draft Multi-segment SD-WAN
Outer IP header:
+---------------------------+
| proto = 17 (UDP) |
| src = CPE1 |
| dst = Cloud GW1 |
+===========================+
| GENEVE Header |
| Proto=50(ESP payload) |
+- - -- -- - - -- - --+
|MultiSeg-SDWAN Option Class|
+- - -- -- - - -- - --+
|SD-WAN EndPt SubTLV (CPE10)|
+---------------------------+
| EgressGW-SubTLV |
+---------------------------+
|GENEVE Hdr Authentication |<-validated by GW
+---------------------------+ < ----------+
|SPI(Security Parameter Idx)| |
+---------------------------+ |
| sequence number | |
+---------------------------+ <-+ |
| payload IP header: | | |
| src = 11.1.1.1 | | |
| dst = 10.2.1.2 | | |
+---------------------------+ Encrypted |
| TCP header + | | |
~ payload (variable) ~ | |
| | | |
+===========================+ <-+ -------+
|Integrity Check Value (ICV)|<- validated by CPE10
+---------------------------+
Figure 11 GENEVE header encapsulated IPsec packet
Appendix B: Illustration from Private VPN to IPsec Tunnel
This section illustrates a Cloud GW connecting client traffic
from a branch CPE via a Private VPN to another CPE via an
IPsec tunnel.
Using Figure 1 as an example:
- CPE1 send traffic via a Private VPN (Direct Connect to
the Cloud Edge) to the Cloud GW. The traffic is not
encrypted.
Dunbar, et al. Expires Dec 18, 2025 [Page 31]
Internet-Draft Multi-segment SD-WAN
- There is a bidirectional IPsec tunnel between CPE2 and
the Cloud GW; with IPsec SA1 for the traffic from the
CPE2 to the Cloud-GW; and IPsec SA2 for the traffic
from the Cloud-GW to the CPE2.
- All the CPEs are under one iBGP administrative domain,
with a Route Reflector (RR) as their controller. CPEs
notify their peers of their corresponding Cloud GW
addresses.
Assume the IPsec ESP Tunnel Mode is used for the IPsec SA
between Cloud GW and CPE2. For a packet from 11.1.1.1 to
10.2.1.2, the following header is added by CPE1 sending
over the Private VPN:
Outer IP header:
+---------------------------+
| proto = 17 (UDP) |
| src = CPE1 |
| dst = Cloud GW |
+===========================+
| GENEVE Header |
| Protocol=50(ESP payload) |
+- - -- -- - - -- - --+
|MultiSeg-SDWAN Option Class|
+- - -- -- - - -- - --+
|SD-WAN EndPt SubTLV (CPE2) |
+---------------------------+
|GENEVE Hdr Authentication |<-validated by GW
+---------------------------+ < -+
| payload IP header: | |
| src = 11.1.1.1 | |
| dst = 10.2.1.2 | |
+---------------------------+ Not Encrypted
| TCP header + | |
~ payload (variable) ~ |
| | |
+===========================+ <-+
Figure 12 Illustration of packet through VPN
Upon receiving the GENEVE encapsulated packet with the
"Multi-Segment-SD-WAN" option, the Cloud GW extracts the
destination CPE from the GENEVE header and encrypts the
packet with the IPsec SA2 to forward to the destination
(i.e., CPE2). The GENEVE Header is carried to the CPE2.
Dunbar, et al. Expires Dec 18, 2025 [Page 32]
Internet-Draft Multi-segment SD-WAN
Outer IP header:
+---------------------------+
| proto = 17 (UDP) |
| src = Cloud GW |
| dst = CPE2 |
+===========================+
| GENEVE Header |
| Proto=50(ESP payload) |
+- - -- -- - - -- - --+
|MultiSeg-SDWAN Option Class|
+- - -- -- - - -- - --+
|SD-WAN EndPt SubTLV (CPE2) |
+---------------------------+
|GENEVE Hdr Authentication |<-validated by GW
+---------------------------+ < ----------+
|SPI(Security Parameter Idx)| |
+---------------------------+ |
| sequence number | |
+---------------------------+ <-+ |
| payload IP header: | | |
| src = 11.1.1.1 | | |
| dst = 10.2.1.2 | | |
+---------------------------+ Encrypted |
| TCP header + | | |
~ payload (variable) ~ | |
| | | |
+===========================+ <-+ -------+
|Integrity Check Value (ICV)|<-validated by CPE2
+---------------------------+
Figure 13 Illustration of packet from the Egress Cloud GW
Dunbar, et al. Expires Dec 18, 2025 [Page 33]
Internet-Draft Multi-segment SD-WAN
Authors' Addresses
Linda Dunbar
Futurewei
Email: ldunbar@futurewei.com
Kausik Majumdar
Argus Networks
Email: kausikm.ietf@gmail.com
Venkit Kasiviswanathan
Arista
Email: venkit@arista.com
Ashok Ramchandra
Microsoft
Email: aramchandra@microsoft.com
Aseem Choudhary
Aviatrix
Email: achoudhary@aviatrix.com
Contributors' Addresses
Dunbar, et al. Expires Dec 18, 2025 [Page 34]