| Internet-Draft | Benchmarking Transport in ISTN | October 2024 |
| Lai, et al. | Expires 24 April 2025 | [Page] |
This document defines the terminology and methodology for conducting performance benchmarking of the transport protocols for low-earth orbit satellite user terminals within emerging integrated space and terrestrial networks (ISTN). It encompasses the test environment setup and benchmarking methodology.¶
The objective of this document is to enhance the applicability, repeatability, and transparency of performance benchmarking for reliable transport protocols (e.g. TCP and QUIC) in ISTNs.¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 24 April 2025.¶
Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
The history of using communication satellites to provide Internet services can date back several decades and has evolved significantly in recent years. Users in remote or rural areas where deploying terrestrial infrastructure is challenging or cost-prohibitive can leverage satellites to access Internet applications such as web browsing, IP-based telephony and videoconferencing. Since most today's Internet applications are built upon reliable transport protocols (e.g. TCP and QUIC), and satellite networks have many unique characteristics that are different from traditional terrestrial networks, guaranteeing the network performance of reliable transport protocols over satellite networks should be important for both satellite operators and Internet content providers.¶
The IETF has a number of previous efforts on recommending test methodologies [RFC6349] and suggesting performance enhancement strategies [RFC0346] [RFC0357] [RFC2488] [RFC2760] [RFC8975] for reliable transport protols over various satellite networks. Thanks to the recent technological revolution, both satellite systems and transport protocols have evolved significantly. On one hand, satellite networks have evolved from the traditional use of geostationary orbit satellite to transparently forward data, to a new form of providing low-latency Internet services through a network of massive low-earth orbit (LEO) satellites (i.e. a satellite constellation). Such LEO satellite networks can further be integrated into terrestrial Internet, constructing an integrated space and terrestrial network (ISTN). On the other hand, powered by a range of new features, new transport protocols such as the QUIC protocol [RFC9000] are designed to improve the speed, security, and reliability of end-to-end Internet connections. In such an ecosystem with growing importance, well-defined benchmarking methodology and terminology are increasingly needed to support reasonable benchmarking and generate improvement guidelines for transport protocols in emerging ISTNs.¶
To this end, this document describes a practical methodology for benchmarking several important aspects of TCP and QUIC in ISTNs. Specifically, we introduce the methodology to build a laboratory-level test environment simulating a real ISTN environment, and describe key considerations of benchmarking tests for measuring the performance of transport protocols.¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119].¶
This document uses the following acronyms and terminologies:¶
Mega-constellation: A group of networked satellites working as a system.¶
LEO: Low Earth Orbit with an altitude no more than 2000 km.¶
GEO: Geostationary Earth Orbit with an altitude of about 35786 km.¶
ISTN: Integrated Satellite and Terrestrial Network.¶
ISL: Inter-satellite Links.¶
GSL: Ground-satellite Links.¶
GS: Ground Station.¶
QUIC: Quick UDP Internet Connections.¶
This document provides testing terminology and testing methodology for reliable transport layer protocols in emerging ISTNs. This document focuses on advanced, realistic, and reproducible benchmarking methods. It describes the methodology for constructing a testbed environment which can mimic the network behaviors of large-scale ISTNs and illustrate benchmarking tests for assessing several major aspects of transport protocols.¶
The first generation of satellite networks leverages communication satellite in Geosynchronous Earth Orbit (GEO) to provide network services. In this architecture, GEO satellites are positioned at a very high altitude, approximately 35,786 kilometers (22,236 miles) above the earth's equator. This altitude allows GEO satellites to maintain a fixed position relative to the earth's surface. Thus, GEO satellites can provide a wide and continuous coverage area. In particular, a single GEO satellite can cover approximately one-third of the earth's surface, making them suitable for providing global coverage with a limited number of satellites. However, the high altitude of GEO satellites can also introduce high propagation latency in space-ground communication because signals have to travel a long distance to reach the satellite and return back to the earth. Such a high latency can be noticeable in real-time applications like interactive online gaming or video conferencing.¶
More recently, ''NewSpace'' companies, such as SpaceX and OneWeb, are actively deploying their mega-constellations with thousands of broadband satellites in low earth orbit (LEO) to provide Internet service globally. LEO satellites orbit at much lower altitudes, typically ranging from about 180 to 2,000 kilometers (from 112 to 1,242 miles) above the earth's surface. Therefore, LEO satellites have a smaller coverage area compared to GEO satellites. This is why satellite operators have to deploy large constellations of LEO satellites that collaboratively work together to provide global coverage. One key benefit of LEO satellite networks is that they offer much lower latency because the distance signals need to travel is significantly shorter. This makes them more suitable for real-time applications like online gaming and video conferencing.¶
Broadband satellite mega-constellations can be integrated into the terrestrial Internet through ground-satellite links and further construct an integrated space and terrestrial network (ISTN), which typically contains two core components as follows.¶
ISLs +---------+ +---------+ +---------+ ISLs +----+----+
-------------|Satellite|--------|Satellite|----------|Satellite|-------- ... | Internet|
+----+----+ +-----+---+ +----+----+ | |
/ / \ +----+----+
/ / \ |
/ / \ |
+----+----+ +----+----+ +----+----+ +----+----+
| User | | User | | Ground | |Point-of |
| Terminal| | Terminal| | Station | --- --- |Presence |
+---------+ +----+----+ +---------+ +----+----+
Emerging satellites can be equipped with high-speed inter-satellite links (ISLs) and ground-satellite links (GSLs) for inter-satellite and ground-satellite networking. Figure 1 plots a typical ISTN architecture, which integrates communication satellites and today's terrestrial Internet. Futuristic ISTN is expected to provide pervasive, lowlatency Internet services for various users such as residential, direct-phone-to-satellite, maritime, and airplane users etc. In this architecture, satellites fundamentally perform two core functions. On one hand, satellites work as ''space routers'' to build an ''Internet backbone in space''[Internet-backbones-in-space] and forward network traffic between any two satellites or ground nodes in the network. On the other hand, satellites also work as ''access points'' to provide last-mile connectivity for land-based users.¶
Unlike existing terrestrial networks which typically have a flat and static structure, the network topology of an ISTN is three-dimensional and dynamic, as it includes multi-layer satellites, and these satellites in non-geostationary orbits are moving at a high velocity related to the earth's surface. In addition, the ISTN topology is also predictable, since satellites move in their pre-planned orbits with predictable trajectories. The position of each satellite can be tracked by terrestrial observation stations and published regularly. Many details of connectivity patterns can be deduced from the Federal Communications Commission (FCC) requests and real-world measurements, making the ISTN topology likely to be public or at least estimable.¶
The terrestrial segment of an ISTN, including a number of geo-distributed ground stations and point-of-presence (PoP) connecting to the Internet, plays a crucial role. Ground stations serve as the interface between the satellites and the terrestrial network, facilitating the transmission of data to and from the satellite constellation. In addition, the terrestrial segment takes care of many other necessary functionalities such as tracking and control, telemetry and commanding, and network management. When a user terminal accesses Internet services (e.g.,a web servive) through the ISTN, the user's request is first forwarded to a ground station through one (i.e., via bent-pipe transparent forwarding) or multiple satellites (i.e., via ISL-based space routing) and then to a PoP connecting the Internet, and finally to the destination.¶
The unique LEO dynamics and error-prone satellite communication links involve several important network characteristics for end-to-end transmission in ISTNs.¶
Firstly, the high LEO dynamics can lead to frequent ground-satellite handovers during which ground devices have to change their ingress satellite and suffer from link disruptions. Recent reports have shown severe packet loss rate during such handovers. Besides, satellites will change their operational orbits to avoid collisions, resulting in inter-satellite links churns and packet losses. Secondly, since ISTNs are operated in an open space environment, they are susceptible to various types of interference, such as space rays and artificial interference which can significantly affect the quality of inter-satellite communication. As for satellite-ground links, bad weather and obstructed view will also cause the link quality decline. For example, heavy rain will affect the signal transmission and unthoughtful placement of the satellite terminal can cause the satellite signal to be blocked by other objects (e.g., dense leaves) during data transmission, resulting in packet loss. Collectively, frequent handovers caused by LEO dynamics and collision avoidance and signal attenuation or blocking will lead to ethernal packet loss. Besides, current reports have revealed high packet loss rate in ISTNs, especially during the ground-satellite handovers[Endless-and-Bursty-Packet-Losses].¶
ISTNs are expected to provide low-delay Internet service for global users. Firstly, free-space laser inter-satellite links can provide faster data transmission speed than terrestrial fibers. Secondly, a large number of evenly distributed satellites can provide near-space-optimal route, outperforming circuitous terrestrial fiber routes for long-haul transmission[Internet-backbones-in-space]. However, to cope with packet losses, current widely used sender-based retransmission requires persistent loss detection and recovery. Therefore, in an environment with ethernal and burst packet loss, frequent and persistent sender-based recovery significantly increases the end-to-end delay jitter. Higher delay jitter will affect the performance of delay-sensitive applications (e.g., WebRTC), which further results in an increase in user-perceived end-to-end delay and can not unleash the low-delay potential of ISTNs.¶
The test setup defined in this section applies to all benchmarking tests described in Section 5. The test setup MUST be contained within an isolated test environment (see Section 3 of [RFC6815]).¶
Testbed Setup is expected to create an isolated benchmarking environment that appropriately simulates the unique characteristics of ISTN as mentioned in Section 3.¶
A data-driven way to building a testbed for ISTN was proposed in [draft-lai-bmwg-sic-benchmarking], emulating each network node in ISTN with a container and adjusting the links' characteristics with real data. However, it's aiming at the general environment simulation rather than network layer or transport layer benchmarking and is still requesting for comments. It is an OPTIONAL choice for testbed setup.¶
To enhance applicability and repeatability of the benchmarking methodology, this document specifies a more concrete testbed setup given in Figure 2. It is the RECOMMENDED testbed setup. The two ends of the transport protocols (DUTs) are respectively connected to the user terminal and the gateway. Except the DUTs, other equipments are all routers utilized to simulate ISTN nodes. Two ingress/egress satellites were setup for each teresstrial end to simulate their handover between adjacent LEO satellites. The handover simulation and links' setup will be specified in Section 4.3 .¶
Terrestrial Space
<---Segment----> <-----------Segment---------->
<-------------------------------------------------------+
|
+---+ +--------+ +-------------+ +------+ |
| | | +--USL 1-+Ingress Sat 1+-ISL 1--+ | |
| | | User | +--^----------+ | | |
|DUT+-+ | | handover | | |
| | |Terminal| +--v----------+ | | |
| | | +--USL 2-+Ingress Sat 2+-ISL 2--+ | |
+---+ +--------+ +-------------+ |Relay | |
| | |
|Sat(s)| |
+---+ +--------+ +-------------+ | | |
| | | +--GSL 1-+ Egress Sat 1+-ISL 3--+ | |
| | | | +--^----------+ | | |
|DUT+-+Gateway | | handover | | |
| | | | +--v----------+ | | |
| | | +--GSL 2-+ Egress Sat 2+-ISL 4--+ | |
+---+ +--------+ +-------------+ +------+ |
|
<------------Network Path Under Test--------------------+
Generally, the DUTs could be configured with any reliable transport protocols, like TCP and QUIC. As the DUT is supposed to be designed for / applied to unique ISTN environments, their critical parameters (e.g. the congestion control strategy) SHOULD be reported in the result and stay the same throughout the benchmarking process. In addition, the concrete congestion control algorithms (CCA) can be configured on demand, based on the benchmarking requirement.¶
Handover simulation. In an ISTN, the communication links between terrestrial segment and space segment experience frequent handovers, e.g., each user terminal of Starlink, an operational ISTN network, handovers to another satellite every 15 seconds. Two ingress satellites are setup for the user terminal to simulate the handover behaviours. At each time, only one group of ingress satellite and its corresponding links (e.g., USL 1 and ISL 1 for Ingress Sat 1) are activated, through which the user terminal connects to the relay satellite (s). When a handover occurs, the another (un-activated) group of ingress satellite and its corresponding links are activated. The handover between two egress satellites and their corresponding links are likewise. The handover bewteen satellites and user terminals, and handovers between satellites and gateways, do not necessarily occour at the same time.¶
Link loss rate. As aforementioned, the USLs and GSLs experience ethernal and burst packet losses, due to environmental attenuation and frequent losses. Their packet loss rate SHOULD be set dynamically according to real measurement. No known models are now available for loss rate setup of USLs and GSLs. Other links SHOULD be setup with no link loss.¶
Link latency. The latency of space segment route could be setup with two halves, on the two activated ISLs of the Relay Sat. The latency of all the links SHOULD be set dynamically according to the dynamic propagation latency of satellite network paths.¶
Link bandwidth. The bottleneck link is RECOMMENDED to be the USL and setup with (at least) 50Mbps, 100Mbps, 200Mbps, 500Mbps. Test with other bottleneck bandwidth COULD be reported in the result. Other links MUST be setup with bandwidth greater than the bottleneck bandwidth and be reported in the result.¶
Terminal type. User terminals are typically classified into two types, (1) Residential terminal and (2) Mobile terminal. They generally experience different network conditions in terms of link losses and latencies. Benchmarking under both types is RECOMMENDED and reported.¶
The following considerations SHOULD be verifed ahead of all the benchmarking tests. If not, the reason MUST be noted in the report.¶
(1) Link performance verification. The link bandwidth, loss rate and latency should be verified for each link with no end-to-end traffic. The average result of each link under each terminal type.¶
(2) Steady testbed. As [RFC9411] describes, Several factors might influence stability, specifically for virtualized testbeds.¶
Testing framework for TCP throughput was proposed in [RFC6349], this document adopts the following tests from [RFC6349] for transport protocol benchamrking in ISTN. This document adpated them to the ISTN environment, and extends to QUIC where necessary.¶
Several common tools for throughput measurement are readily available in the network community, e.g. "iperf" for TCP and "qperf" for QUIC. With the tools installed at each end of the network path, one acting as a client and the other as a server, the achieved throughput can then be measured, either uni-directionally or bi-directionally. The parameters like Send Socket Buffer of both client and server can be manually set, referring to [RFC6349].¶
However, for the ISTN environment, this document recommends that the throughput test SHOULD be run over a relatively longer duration (i.e., greater than 3 mins or 180 seconds) to improve the repeatability.¶
Throughput under each terminal type SHOULD be reported.¶
As [RFC6349], RTT is defined as the elapsed time between the clocking in of the first bit of a TCP segment / QUIC Packet sent and the receipt of the last bit of the corresponding Acknowledgment.¶
The RTT of each TCP segment / QUIC Packet SHOULD be recorded for the calculation of the aftermentioned buffer delay metric. The average and each quartile value of the RTT records SHOULD be reported.¶
Transfer efficiency represents the percentage of Bytes that were not retransmitted. The TCP Efficiency defined in [RFC6349] applies to transfer efficiency for both TCP and QUIC here.¶
Transmitted Bytes - Retransmitted Bytes
Transfer Efficiency % = --------------------------------------- X 100
Transmitted Bytes
¶
Transmitted Bytes are the total number of Bytes to be transmitted, including the original and the retransmitted Bytes. See [RFC6349] for calculation examples.¶
Transfer Efficiency SHOULD be reported for each test.¶
Buffer Delay Percentage represents the increase in RTT during a TCP Throughput test versus the inherent or baseline RTT [RFC6349].¶
As the baseline RTT [RFC6349] also changes in ISTN, this document suggests calculating Buffer Delay Percentage at each second and the average value throughout each test SHOULD be reported.¶
Specifically, for each second (t), the Buffer Delay Percentage is calculated as follows:¶
Average RTT during t - Baseline RTT (t)
Buffer Delay % (t)= ------------------------------------------ X 100
Baseline RTT(t)
¶
This document makes no specifc request of IANA.¶
IANA has assigned IPv4 and IPv6 address blocks in[RFC6890] that have been registered for special purposes. The IPv6 address block 2001:2::/48 has been allocated for the purpose of IPv6 benchmarking [RFC5180], and the IPv4 address block 198.18.0.0/15 has been allocated for the purpose of IPv4 benchmarking [RFC2544]. This assignment was made to minimize the chance of confict in case a testing device were to be accidentally connected to the part of the Internet. The testbed setup in this document SHOULD adopt this assignment.¶
Benchmarking activities as described in this memo are limited to technology characterization using controlled devices in a laboratory environment, with dedicated address space and the constraints specified in the sections above. The benchmarking network topology as well as its parameter configurations will be an independent test setup, and the laboratory environment MUST NOT be connected to devices that may forward the test traffic into a production network, or misroute traffic to the test management network.¶
In addition, benchmarking is performed on a "black-box" basis, relying solely on measurements observable external to the DUT/SUT. Special capabilities SHOULD NOT exist in the DUT/SUT specifically for benchmarking purposes. Any implications for network security arising from the DUT/SUT SHOULD be identical in the lab and in production networks.¶