Internet Engineering Task Force S. Burleigh Internet Draft IPNGROUP Intended status: Informational August 30, 2024 Expires: March 2025 Revisiting the Use of the IP Protocol Stack in Deep Space draft-burleigh-deepspace-ip-assessment-00.txt Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html This Internet-Draft will expire on March 3, 2025. Copyright Notice Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Burleigh Expires March 2025 [Page 1] Internet-Draft Bundle-in-Bundle Encapsulation August 2024 Abstract This document describes aspects of communication over interplanetary distances that make the use of the Internet protocol stack in a deep space network inadvisable. Table of Contents 1. Introduction...................................................2 2. Conventions used in this document..............................4 3. Deep space communication considerations........................4 3.1. Connections...............................................4 3.2. Retransmission............................................5 3.3. Routing and Forwarding....................................6 3.4. Domain Name System........................................7 3.5. Time to live..............................................7 3.6. Flow control..............................................8 3.7. Congestion control........................................8 3.8. Security..................................................9 3.9. Network management........................................9 3.10. Electronic mail..........................................9 3.11. Clock alignment.........................................10 4. Conclusions...................................................10 5. Security Considerations.......................................11 6. IANA Considerations...........................................11 7. References....................................................11 7.1. Normative References.....................................11 7.2. Informative References...................................12 8. Acknowledgments...............................................12 1. Introduction Marc Blanchet's article "Beyond Earth: Exploring the Future of Deep Space Communications", in the May 2024 issue of IEEE Communications Society Technology News [COMSOC], discusses a topic that grows in importance as the world's space agencies begin plans for mission operations at the Moon and beyond. Readers of that article might be interested in some additional background on the challenges of using the Internet protocol stack to operate a communication network over deep space links. It is well understood that entities exchanging information in deep space must cope with the lengthy signal propagation latencies directly imposed by the vast distances separating them, given the limited speed of light in a vacuum. It is perhaps less well Burleigh Expires March 2025 [Page 2] Internet-Draft Bundle-in-Bundle Encapsulation August 2024 understood that those distances will indirectly impose additional latencies that are both far greater and also highly variable. Obviously there will be no wired infrastructure supporting this information exchange; all communication will be wireless (either radio or optical). Transmitted wireless signal strength is attenuated by the square of the distance between transmitter and receiver; no broadcast signal on Earth (for example) will be powerful enough to be decoded by a spacecraft 20 light minutes away. This means that all deep space communication must be by directed, rather than broadcast, radiation. This in turn means that in order for two entities to exchange information they must be mutually visible and their antennae must be pointed at each other. Planets rotate, satellites revolve around the rotating planets, and transmission and reception equipment powerful enough to enable interplanetary signal exchange is expensive. In the general case, a given entity will not be able to communicate continuously with all peer entities with which it must interoperate. So connectivity will routinely be interrupted for minutes, hours, even days, and the durations of these interruptions will not be computable by statistical analysis of traffic; they will be imposed by orbital dynamics and by the operating schedules of the entities, from which they may be computed far in advance of their occurrence. Importantly, these interruptions are not instances of the link shutting down for some indeterminate time while repairs are made; they are readily anticipated increments in forwarding latency. Connectivity is punctuated, not unstable. Moreover, while the interval between the transmission of a signal and reception of that signal is easily computed from knowledge of the distance between the transmitter and the receiver, not all message exchange within a deep space network will be exclusively between the transmitter and receiver of a given signal; as in the Internet, arrival of a message at a remote destination may require multiple consecutive receptions and onward transmissions - "hops" - by relay entities (gateways, routers). Additionally, punctuated connectivity will frequently make it impossible for a message received at a given relay point to be forwarded immediately to the next relay on its end-to-end path through the network: connectivity to that neighboring node may not be established until hours or even days later. The relay will be required to retain the message in storage until that opportunity Burleigh Expires March 2025 [Page 3] Internet-Draft Bundle-in-Bundle Encapsulation August 2024 arrives, and latency in the delivery of the message will be increased accordingly. Due to the nature of deep space communication, then, the end-to-end latency in delivery of any message may by measured in hours or days, as may the latency in delivery of any message issued in response to that message, and the end-to-end latency in delivery of the next message from the same source to the same destination may be very different. The very brief, statistically predictable "round-trip" times on which many Internet protocols rely for operational success cannot be expected. This paper will discuss some of the implications of this constraint. 2. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [RFC2119]. In this document, these words will appear with that interpretation only when in ALL CAPS. Lower case uses of these words are not to be interpreted as carrying RFC-2119 significance. 3. Deep space communication considerations 3.1. Connections The values of parameters governing data exchange between network peers must be agreed upon in order for data to flow, and those values must remain in agreement as recorded in the state of each peer in order for data flow to be sustained. When that agreement is established by means of a round-trip message exchange between the peers, we often term it a "connection". A connection can be negotiated quickly while nodes are in close proximity and the round-trip time between them is brief; when this is the case, the communication governed by the connection can normally be sustained for a lengthy period following connection establishment. Connection-based protocols in the Internet succeed because it is virtually certain that the opportunity for communication will not lapse until long after the connection has been established. However, establishing new connections while the peers are many light minutes apart and round-trip times are much longer may be so time- consuming that communication opportunities lapse shortly after - or even before - agreement is reached, limiting data flow. This effect Burleigh Expires March 2025 [Page 4] Internet-Draft Bundle-in-Bundle Encapsulation August 2024 could in some cases disable multi-hop end-to-end connections altogether, but it would be troublesome even in connections between topological neighbors that are separated by interplanetary distances. Such connection difficulties can be avoided if there is no need to establish connections between peers that are separated by interplanetary distances. One possible mitigation is to establish connections between peers while they are co-located (e.g., spacecraft that have not yet been launched from Earth) and then simply retain that connection state indefinitely while the distance between the peers increases to interplanetary scale. But in the general case that may not be sufficient. In an interplanetary network, a given spacecraft might communicate with any number of laboratories or operations centers on Earth or (ultimately) on some other planet; adding another such "ground" peer would require a new connection. A given spacecraft might additionally communicate with any number of other spacecraft, including spacecraft that were launched after it was already in orbit; communicating with each such spacecraft would require a new connection. Moreover, an existing connection must be re-established whenever state is lost at one of the peers, as when mission operations computers reboot or flight computers reboot or are simply shut down to conserve power. Further, even when the establishment of transport (e.g., QUIC [RFC9000]) connections is minimized it may be necessary to establish additional connections - agreement on additional data exchange parameter values - at the application layer. Such connections are by definition end-to-end and would need to be negotiated whenever application peers begin or restart operation in the network. 3.2. Retransmission Reliable message exchange in the Internet rests on protocols such as TCP and QUIC that detect data loss and automatically retransmit lost data. That retransmission may be triggered by negative acknowledgments issued by the destination entity, but acknowledgment messages may also be lost; in that event, retransmission is triggered by the expiration of a timeout interval at the source entity shortly after the time at which an end-to-end acknowledgment is expected. Since message round-trip times over multi-hop paths in deep space may be not only lengthy but also highly variable, statistical computation of an accurate retransmission timeout interval by, e.g., QUIC is in the general case not possible. Retransmission will either occur too soon, wasting limited Burleigh Expires March 2025 [Page 5] Internet-Draft Bundle-in-Bundle Encapsulation August 2024 transmission opportunities, or too late, delaying both data arrival and the release of retransmission buffer resources, and thereby introducing congestion in the network. This effect too can be avoided: end-to-end message exchange can be accomplished over paths constructed by concatenating multiple consecutive QUIC transmissions, each of which entails only signaling between two mutually visible network nodes for which the round-trip time is a function of the latency in direct signal propagation. But to enable this model a new mechanism is needed that can concatenate multiple successive QUIC transmissions in a manner that is secure from end to end. (The Bundle Protocol [RFC9171] is one such mechanism.) Beyond unnecessary consumption of network resources, end-to-end retransmission in an interplanetary internet also unnecessarily retards data delivery to the destination. For a message on a 3-hop path from node A to node D via nodes B and C, the corruption of a single frame in the forward transmission from node C would cause node D to send a negative acknowledgment back to node A, triggering retransmission from A. If the hop from A to B or from B to C spans 20 light minutes of space, delivery of the message is delayed by at least 40 minutes. Since the message was received successfully at C, network resource utilization and delivery timeliness are both improved if node C receives the negative acknowledgment and retransmits the message itself. 3.3. Routing and Forwarding Delivery of a message in the Internet is reliant on asserting the address (location within the network topology) of the message's destination and on providing network routers with sufficient knowledge of network topology to enable the forwarding of the message to that address. The Internet responds dynamically to changes in network topology by means of routing protocols, which convey information about those changes to routers. When routing protocol message propagation is very rapid, as in the Internet, a generally accurate understanding of network topology can be sustained across the network. But when it is not, due to lengthy and highly variable message delivery latency as in a deep space network, not all nodes of the network will agree on its topology at any moment. Changes in network topology will result in routing errors, data loss, and poor network performance. Burleigh Expires March 2025 [Page 6] Internet-Draft Bundle-in-Bundle Encapsulation August 2024 This effect can be avoided if dynamic routing in the deep space network is abandoned and, instead, manually provisioned routing tables are managed in coordination among network administrators. But the scope of this management activity will grow as the network grows. For a future deep space network potentially comprising thousands of nodes (on Earth, in Earth orbit, on other planetary surfaces, in other planetary orbits, at LaGrange points, etc.), the canonical Internet routing and forwarding model would not be satisfactory. 3.4. Domain Name System Internet applications typically issue messages to named entities rather than to network addresses, relying on the Domain Name System (DNS) to resolve entities' names to their IP addresses. DNS name resolution is performed by querying a DNS server and waiting for a response, a round-trip communication. When the DNS server is in close proximity (e.g., on the same planetary surface, as on Earth), this works well. But lengthy round-trip times would severely retard the domain name resolution on which many applications rely. This effect can be addressed by deploying DNS servers to remote locations in the solar system so that the round-trip time from any node to the nearest DNS server is small. But this would entail synchronization of DNS information among all servers in the network via messages that may take hours or days to reach all destinations. That synchronization would consume network bandwidth that might better be used to convey scientific or industrial information, and meanwhile inconsistencies among DNS servers would likely cause routing errors. 3.5. Time to live In the Internet, excessive consumption of network resources (transmission bandwidth) by any single message is detected when the message's "hop count" violates a designated limit. At that time the message can be assumed to be in a routing loop and should be discarded from the network. This mechanism is effective because data in the Internet are in generally continuous motion, residing only momentarily in the memory buffers of any single forwarding node. A steadily decreasing count of remaining authorized hops ("time to live", or TTL) is a rough analog for the passage of time. In an interplanetary network, however, a message may routinely be required to wait at a forwarding node until the start of the next Burleigh Expires March 2025 [Page 7] Internet-Draft Bundle-in-Bundle Encapsulation August 2024 opportunity to transmit it to the next node on its network path. While waiting it is consuming network resources (buffer space), and that resource consumption might be excessive in an analogous way: the next transmission opportunity might never begin, in which case the message should be discarded from the network even if its TTL has never been decremented at all. Mechanisms for detecting expiration of this alternative "time to live" will be needed in the implementation of the interplanetary network. 3.6. Flow control Limiting the rate at which a source inserts data into the network to the rate at which the destination is able to receive and process it - flow control - is critical to network performance. If the destination node's buffers are filled by inbound traffic more rapidly than they can be emptied by the application, buffers will overflow; this will cause data loss, resulting in degraded bandwidth utilization and delayed data delivery. In the Internet that limit can be inferred from the rate at which data arrival acknowledgments are received. In an interplanetary network, however, signal propagation latency may be so great that the receiver's buffers overflow long before the sender receives enough acknowledgment information to compute a reduced transmission rate. Put simply, flow control in the interplanetary network must be anticipatory rather than reactive. 3.7. Congestion control Limiting the rates at which data sources in aggregate insert data into the network so as to ensure that the network can successfully convey all data to all destinations - congestion control - is likewise critical to network performance. In the Internet, a mismatch between the rates at which data are inserted and removed from the network results in buffer overflow at relay points. Nodes detect imminent congestion and signal one another to reduce rates of application data acceptance, preventing congestion. As with flow control, signal propagation latency in an interplanetary network may be so great that congestion signals may not arrive soon enough to trigger timely reduction in application data insertion. Buffers will overflow not only at end systems but also at relay points. Burleigh Expires March 2025 [Page 8] Internet-Draft Bundle-in-Bundle Encapsulation August 2024 Again, congestion control in the interplanetary network must be anticipatory rather than reactive. 3.8. Security Internet security protocols such as IPsec and TLS rely on authentication and key agreements negotiated by means of handshakes between end systems. These security agreements are subject to the same constraints that limit the usability of transport-layer connections as discussed in 3.2 above. End-to-end security associations are especially difficult to establish over a deep space network that includes multiple relay points. An alternative model of managed security associations is needed. Point-to-point security associations between topologically adjacent nodes can be established more readily, but migrating to an architecture based on achieving end-to-end communication by concatenating multiple individually secured "hops" would raise an additional problem. Internet security protocols protect data while in transit via other protocols (IP, TCP, QUIC), but data may reside in the memory of a relay node - no longer in transit, thus no longer secured - for minutes, hours, or days while awaiting onward transmission. A means of securing application data at rest is needed. It would be preferable to avoid requiring each application protocol to perform this function itself. 3.9. Network management Commonly used Internet network management protocols - SNMP, NETCONF, and RESTCONF - can be used over interplanetary distances, but care must be taken. The potentially very long round-trip times between clients and servers limit the utility of synchronous queries for node state, such as the SNMP GetRequest/Response cycle; responding information may be obsolete by the time it arrives. Asynchronous notifications, such as SNMP's Trap protocol data units, will be more suitable but will still be of limited value due to signal propagation latency. In general it will be helpful to rely increasingly on node autonomy to manage the network, using asynchronous commands to configure autonomous operation. 3.10. Electronic mail Email is a wholly delay-tolerant application, perfect for the interplanetary network. However, at least three aspects of standard email as it functions within the Internet are somewhat problematic in deep space deployment. Burleigh Expires March 2025 [Page 9] Internet-Draft Bundle-in-Bundle Encapsulation August 2024 First, email is conveyed to POP3 and IMAP servers (for local access by means of mail clients such as Outlook and Gmail) via the Simple Mail Transfer Protocol (SMTP). Only SMTP servers that have been adapted to use some transport protocol other than TCP will be suitable for use in the interplanetary network, as standard TCP is not useful over interplanetary distances as discussed in [DURST]. Other transport protocols that operate end-to-end as TCP does, such as QUIC, raise once again the retransmission concerns discussed in section 3.2 above. Second, SMTP communication is performed in "sessions", each comprising zero or more SMTP transactions. Implementations may need to be modified to tolerate transaction closure latencies on the order of hours or days. Third, the headers of modern SMTP messages typically include references to assets identified by domain names. As the identified assets may be arbitrarily distant, resolution of those domain names is subject to the DNS considerations discussed in 3.4 above. 3.11. Clock alignment Agreement on the current time is important for many network functions. In the Internet, the Network Time Protocol (NTP) serves to minimize disagreement on time among network assets. However, NTP relies on the statistically valid determination of precise timestamp message transit latencies among NTP peers. Message transit latencies among interplanetary network nodes will be lengthy; the length of time required to converge on the current time may be so great that the consensus time may already be incorrect, due to the orbital motion of the peers, by the time it is computed. 4. Conclusions All this said, the Internet protocol suite will certainly play a vital role in deep space network communications, in much the same way that Ethernet and WiFi play a vital role in the Internet: enclaves of local Internet architecture will surely be established wherever in the Solar System multiple co-located computing platforms must communicate. And with minor modifications the Internet protocol suite might indeed be able to address concurrently all of the concerns identified here, sustaining communications over deep space links, for a network of sufficiently limited scope. That is, for a small, static set of directly interconnected nodes among which no multi-hop paths are necessary, all connections are permanent, and the efforts Burleigh Expires March 2025 [Page 10] Internet-Draft Bundle-in-Bundle Encapsulation August 2024 of network administrators suffice to manage all routing, perhaps only a few additional expectations (e.g., flow control, time to live) would need to be lowered. But retaining in a Solar System-wide network the scope of automation that was developed for the Internet, enabling comparable power and scalability, will require either major extensions to the current Internet stack or else a different architecture. Note that such a "Delay-Tolerant Networking" (DTN) architecture already exists [RFC4838]. It was first demonstrated in deep space in 2008, on the EPOXI spacecraft en route to a comet; it has been operating on-board the International Space Station since 2016; it is in experimental operation aboard the KPLO (Danuri) spacecraft now in lunar orbit; it is now in operational use on-board the PACE spacecraft, conducting Earth observational science since early 2024. Standardization of the DTN protocols is mature, both within the Consultative Committee for Space Data Systems and within the Internet Engineering Task Force (most notably RFCs 9171 and 9172). Multiple open-source implementations of the protocols are widely available. Marc Blanchet's article has done the network communications community a great service by providing a historical overview of deep space communications research. Some of the conclusions reached in the article merit clarification, but this is a conversation from which we will all benefit. 5. Security Considerations Security considerations raised in this paper are discussed in section 3.8. 6. IANA Considerations No IANA action is proposed in this paper. 7. References 7.1. Normative References Not applicable. Burleigh Expires March 2025 [Page 11] Internet-Draft Bundle-in-Bundle Encapsulation August 2024 7.2. Informative References [COMSOC] https://www.comsoc.org/publications/ctn/beyond-earth- exploring-future-deep-space-communications, Blanchet, M., "Beyond Earth: Exploring the Future of Deep Space Communications", May 2024. [DURST] https://www.researchgate.net/publication/267721105 Durst, R., P. Feighery, and K. Scott, "Why not use the Standard Internet Suite for Interplanetary Internet?", 2000. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", RFC 2119, March 1997. [RFC4838] Cerf, V., et al, "Delay-Tolerant Networking Architecture", RFC 4838, April 2007. [RFC9000] Iyengar, J., Ed. and Thomson, M., Ed., "QUIC: A UDP-Based Multiplexed and Secure Transport", RFC 9000, May 2021. [RFC9171] Burleigh, S., E. Birrane, and K. Fall, "Bundle Protocol Version 7", RFC 9171, January 2022. Acknowledgments This document was prepared using 2-Word-v2.0.template.dot. Authors' Address Scott Burleigh IPNGROUP 1435 Woodhurst Blvd. McLean, VA 22102 United States of America Email: sburleig.sb@gmail.com Burleigh Expires March 2025 [Page 12]