Network File System Version 4 T. Haynes Internet-Draft Hammerspace Intended status: Standards Track 19 March 2024 Expires: 20 September 2024 Add CB_LAYOUTRECLL_DEVICE to NFSv4.2 draft-haynes-nfsv4-recalldevice-00 Abstract The Parallel Network File System (pNFS) allows for the metadata server to use CB_LAYOUTRECALL to recall a layout from a client by file id or file system id or all. It also allows the server to use CB_NOTIFY_DEVICEID to delete a devicid. It does not provide a mechanism for the metadata server to recall all layouts that have a data file on a specific deviceid. This document presents an extension to RFC8881 to allow the server recall layouts from clients based on deviceid. Note This note is to be removed before publishing as an RFC. Discussion of this draft takes place on the NFSv4 working group mailing list (nfsv4@ietf.org), which is archived at https://mailarchive.ietf.org/arch/browse/nfsv4/. Working Group information can be found at https://datatracker.ietf.org/wg/nfsv4/ about/. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on 20 September 2024. Haynes Expires 20 September 2024 [Page 1] Internet-Draft RECALL_DEVICE March 2024 Copyright Notice Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/ license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 1.1. Do we need RFC8435? . . . . . . . . . . . . . . . . . . . 3 1.2. Requirements Language . . . . . . . . . . . . . . . . . . 3 2. Extension to Operation 5: CB_LAYOUTRECALL - Recall Layout from Client . . . . . . . . . . . . . . . . . . . . . . . . . 3 3. Extraction of XDR . . . . . . . . . . . . . . . . . . . . . . 5 3.1. Code Components Licensing Notice . . . . . . . . . . . . 6 4. Security Considerations . . . . . . . . . . . . . . . . . . . 6 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 6 6. References . . . . . . . . . . . . . . . . . . . . . . . . . 6 6.1. Normative References . . . . . . . . . . . . . . . . . . 6 6.2. Informative References . . . . . . . . . . . . . . . . . 7 Appendix A. Acknowledgments . . . . . . . . . . . . . . . . . . 7 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 7 1. Introduction In the Network File System version4 (NFSv4) with a Parallel NFS (pNFS) metadata server ([RFC8881]), there is no mechanism for the metadata server to recall layouts from the client for when a particular deviceid (see Section 3.3.14 of [RFC8881]) either temporarily or permanently is no longer available. One use case is when the deviceids in a layout are separated by power fault domains. Each layout might describe 3 different devices, each contained in a different power fault domain. In such a scenario, a single fault domain can have the power removed and not cause the loss of access to the data. However, client I/O will be impacted as the client still has to perform WRITEs (see Section 18.32 of [RFC8881]) to the unavailable device, send LAYOUTERRORs (see Section 15.6 of [RFC7862]) to inform the metadata server of NFS4ERR_NXIO (see Section 15.1.16.3 of [RFC8881]). Haynes Expires 20 September 2024 [Page 2] Internet-Draft RECALL_DEVICE March 2024 If the metadata sever had the means to recall layouts by deviceid, a lot of this unnecessary traffic could be eliminated. Finally, while the metadata server could recall layouts one by one, this is again unnecessary traffic and can be offloaded to the client. Besides the use case above, consider if the metadata server wants to set the NOTIFY4_DEVICEID_DELETE in the CB_NOTIFY_DEVICEID callback (see Section 20.12 of [RFC8881]). This flag cannot be set if a layout is outstanding for a deviceid. While the metadata server can revoke all such layouts, there is no way to know that the client has acknowledged that revocation and hence is still not doing I/O to other data files in the layout. The metadata server could fence those layouts as well (see Section 12.5.5 of [RFC8881]), but that can be an expensive operation. Using the process detailed in [RFC8178], the revisions in this document become an extension of NFSv4.2 [RFC7862]. They are built on top of the external data representation (XDR) [RFC4506] generated from [RFC7863]. 1.1. Do we need [RFC8435]? This section is to be removed before publishing as an RFC. The authors have tried to introduce this new functionality outside of a particular pNFS Layout Type. Does that work? 1.2. Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here. 2. Extension to Operation 5: CB_LAYOUTRECALL - Recall Layout from Client The original union layoutrecall4 (see Section 20.3.1 of [RFC8881]) is: Haynes Expires 20 September 2024 [Page 3] Internet-Draft RECALL_DEVICE March 2024 file "new_union_layoutrecall4" enum layoutrecall_type4 { LAYOUTRECALL4_FILE = LAYOUT4_RET_REC_FILE, LAYOUTRECALL4_FSID = LAYOUT4_RET_REC_FSID, LAYOUTRECALL4_ALL = LAYOUT4_RET_REC_ALL }; union layoutrecall4 switch(layoutrecall_type4 lor_recalltype) { case LAYOUTRECALL4_FILE: layoutrecall_file4 lor_layout; case LAYOUTRECALL4_FSID: fsid4 lor_fsid; case LAYOUTRECALL4_ALL: void; }; The proposed extension is: file "new_union_layoutrecall4" /// const LAYOUT4_RET_REC_ALL = 4; /// /// enum layoutrecall_type4 { /// LAYOUTRECALL4_FILE = LAYOUT4_RET_REC_FILE, /// LAYOUTRECALL4_FSID = LAYOUT4_RET_REC_FSID, /// LAYOUTRECALL4_ALL = LAYOUT4_RET_REC_ALL, /// LAYOUTRECALL4_DEVICEID = LAYOUTRECALL4_RET_REC_DEVICEID /// }; /// /// union layoutrecall4 switch(layoutrecall_type4 lor_recalltype) { /// case LAYOUTRECALL4_FILE: /// layoutrecall_file4 lor_layout; /// case LAYOUTRECALL4_FSID: /// fsid4 lor_fsid; /// case LAYOUTRECALL4_DEVICEID: /// deviceid4 lor_deviceid; /// case LAYOUTRECALL4_ALL: /// void; /// }; Haynes Expires 20 September 2024 [Page 4] Internet-Draft RECALL_DEVICE March 2024 With this minimal change, all of the semantics of CB_LAYOUTRECALL in (see Section 20.3 of [RFC8881]) remain the same, i.e., the client and server are aware of how CB_LAYOUTRECALL interacts with each other. The one issue to investigated is what happens if a NFSv4.2 client sees a LAYOUTRECALL4_DEVICEID in a CB_LAYOUTRECALL. They SHOULD return NFS4ERR_UNION_NOTSUPP, but the implementations might not be compliant with [RFC8178]. As such, a survey should be conducted of the major implementations. Finally, when the client does handle a LAYOUTRECALL4_DEVICEID in a CB_LAYOUTRECALL, it MUST return all layouts which have a given deviceid. The server can determine that the client no longer has any layouts with the given devicedid once the client replies with NFS4ERR_NOMATCHING_LAYOUT. 3. Extraction of XDR This document contains the external data representation (XDR) [RFC4506] description of the new open flags for delegating the file to the client. The XDR description is embedded in this document in a way that makes it simple for the reader to extract into a ready-to- compile form. The reader can feed this document into the following shell script to produce the machine readable XDR description of the new flags: #!/bin/sh grep '^ *///' $* | sed 's?^ */// ??' | sed 's?^ *///$??' That is, if the above script is stored in a file called "extract.sh", and this document is in a file called "spec.txt", then the reader can do: sh extract.sh < spec.txt > layout_wcc.x The effect of the script is to remove leading white space from each line, plus a sentinel sequence of "///". XDR descriptions with the sentinel sequence are embedded throughout the document. Note that the XDR code contained in this document depends on types from the NFSv4.2 nfs4_prot.x file (generated from [RFC7863]). This includes both nfs types that end with a 4, such as offset4, length4, etc., as well as more generic types such as uint32_t and uint64_t. Haynes Expires 20 September 2024 [Page 5] Internet-Draft RECALL_DEVICE March 2024 While the XDR can be appended to that from [RFC7863], the various code snippets belong in their respective areas of the that XDR. 3.1. Code Components Licensing Notice Both the XDR description and the scripts used for extracting the XDR description are Code Components as described in Section 4 of "Legal Provisions Relating to IETF Documents" [LEGAL]. These Code Components are licensed according to the terms of that document. 4. Security Considerations There are no new security considerations beyond those in [RFC7862]. 5. IANA Considerations IANA should use the current document (RFC-TBD) as the reference for the new entries. 6. References 6.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . [RFC4506] Eisler, M., Ed., "XDR: External Data Representation Standard", STD 67, RFC 4506, DOI 10.17487/RFC4506, May 2006, . [RFC7862] Haynes, T., "Network File System (NFS) Version 4 Minor Version 2 Protocol", RFC 7862, DOI 10.17487/RFC7862, November 2016, . [RFC7863] Haynes, T., "Network File System (NFS) Version 4 Minor Version 2 External Data Representation Standard (XDR) Description", RFC 7863, DOI 10.17487/RFC7863, November 2016, . [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, . [RFC8178] Noveck, D., "Rules for NFSv4 Extensions and Minor Versions", RFC 8178, DOI 10.17487/RFC8178, July 2017, . Haynes Expires 20 September 2024 [Page 6] Internet-Draft RECALL_DEVICE March 2024 [RFC8434] Haynes, T., "Requirements for Parallel NFS (pNFS) Layout Types", RFC 8434, DOI 10.17487/RFC8434, August 2018, . [RFC8435] Halevy, B. and T. Haynes, "Parallel NFS (pNFS) Flexible File Layout", RFC 8435, DOI 10.17487/RFC8435, August 2018, . [RFC8881] Noveck, D., Ed. and C. Lever, "Network File System (NFS) Version 4 Minor Version 1 Protocol", RFC 8881, DOI 10.17487/RFC8881, August 2020, . 6.2. Informative References [LEGAL] IETF Trust, "Legal Provisions Relating to IETF Documents", November 2008, . [RFC1813] Callaghan, B., Pawlowski, B., and P. Staubach, "NFS Version 3 Protocol Specification", RFC 1813, DOI 10.17487/RFC1813, June 1995, . Appendix A. Acknowledgments Trond Myklebust and Paul Saab have were invloved in the initial requirements for this functionality. Author's Address Thomas Haynes Hammerspace Email: loghyr@hammerspace.com Haynes Expires 20 September 2024 [Page 7]