Add vision for std::os, a few other tweaks

Author: aturon

text/0000-io-fs-2.1.md

@@ -6,14 +6,13 @@
 # Summary
 
 Expand the scope of the `std::fs` module by enhancing existing functionality,
-exposing lower level representations, and adding a few more new functions.
+exposing lower-level representations, and adding a few new functions.
 
 # Motivation
 
 The current `std::fs` module serves many of the basic needs of interacting with
-a filesystem, but it falls short of binding a good bit more of useful
-functionality. For example, none of these operations are possible in stable Rust
-today:
+a filesystem, but is missing a lot of useful functionality. For example, none of
+these operations are possible in stable Rust today:
 
 * Inspecting a file's modification/access times
 * Reading low-level information like that contained in `libc::stat`
@@ -44,13 +43,91 @@ areas:
 
 # Detailed design
 
-## Lowering `Metadata`
+## Lowering APIs
+
+### The vision for the `os` module
+
+One of the principles of [IO reform][io-reform-vision] was to:
+
+> Provide hooks for integrating with low-level and/or platform-specific APIs.
+
+The original RFC went into some amount of detail for how this would look, in
+particular by use of the `os` module. Part of the goal of this RFC is to flesh
+out that vision in more detail.
+
+Ultimately, the organization of `os` is planned to look something like the
+following:
+
+```
+os
+  unix          applicable to all cfg(unix) platforms; high- and low-level APIs
+    io            extensions to std::io
+    fs            extensions to std::fs
+    net           extensions to std::net
+    env           extensions to std::env
+    process       extensions to std::process
+    ...
+  linux         applicable to linux only
+    io, fs, net, env, process, ...
+  macos         ...
+  windows       ...
+```
+
+APIs whose behavior is platform-specific are provided only within the `std::os`
+hierarchy, making it easy to audit for usage of such APIs.  Organizing the
+platform modules internally in the same way as `std` makes it easy to find
+relevant extensions when working with `std`.
+
+It is emphatically *not* the goal of the `std::os::*` modules to provide
+bindings to *all* system APIs for each platform; this work is left to external
+crates. The goals are rather to:
+
+1. Facilitate interop between abstract types like `File` that `std` provides and
+   the underlying system. This is done via "lowering": extension traits like
+   [`AsRawFd`][AsRawFd] allow you to extract low-level, platform-specific
+   representations out of `std` types like `File` and `TcpStream`.
+
+2. Provide high-level but platform-specific APIs that feel like those in the
+   rest of `std`. Just as with the rest of `std`, the goal here is not to
+   include all possible functionality, but rather the most commonly-used or
+   fundamental.
+
+Lowering makes it possible for external crates to provide APIs that work
+"seamlessly" with `std` abstractions. For example, a crate for Linux might
+provide an `epoll` facility that can work directly with `std::fs::File` and
+`std::net::TcpStream` values, completely hiding the internal use of file
+descriptors. Eventually, such a crate could even be merged into `std::os::unix`,
+with minimal disruption -- there is little distinction between `std` and other
+crates in this regard.
+
+Concretely, lowering has two ingredients:
+
+1. Introducing one or more "raw" types that are generally direct aliases for C
+   types.
+
+2. Providing an extension trait that makes it possible to extract a raw type
+   from a `std` type. In some cases, it's possible to go the other way around as
+   well. The conversion can be by reference or by value, where the latter is
+   used mainly to avoid the destructor associated with a `std` type (e.g. to
+   extract a file descriptor from a `File` and eliminate the `File` object,
+   without closing the file).
+
+While we do not seek to exhaustively bind types or APIs from the underlying
+system, it *is* a goal to provide lowering operations for every high-level type
+to a system-level data type, whenever applicable. This RFC proposes several such
+lowerings that are currently missing from `std::fs`.
+
+[io-reform-vision]: https://github.com/rust-lang/rfcs/blob/master/text/0517-io-os-reform.md#vision-for-io
+[AsRawFd]: http://static.rust-lang.org/doc/master/std/os/unix/io/trait.AsRawFd.html
+
+### Lowering `Metadata` (all platforms)
 
 Currently the `Metadata` structure exposes very few pieces of information about
 a file. Some of this is because the information is not available across all
 platforms, but some of it is also because the standard library does not have the
 appropriate abstraction to return at this time (e.g. time stamps). The raw
-contents of `Metadata`, however, should be accessible no matter what.
+contents of `Metadata` (a `stat` on Unix), however, should be accessible via
+lowering no matter what.
 
 The following trait hierarchy and new structures will be added to the standard
 library.
@@ -109,7 +186,7 @@ design principles of the standard library.
 
 The interesting part about working in a "cross platform" manner here is that the
 makeup of `libc::stat` on unix platforms can vary quite a bit between platforms.
-For example not some platforms have a `st_birthtim` field while others do not.
+For example some platforms have a `st_birthtim` field while others do not.
 To enable as much ergonomic usage as possible, the `os::unix` module will expose
 the *intersection* of metadata available in `libc::stat` across all unix
 platforms. The information is still exposed in a raw fashion (in terms of the
@@ -123,7 +200,7 @@ One of the major goals of the `os::unix::fs` design is to enable as much
 functionality as possible when programming against "unix in general" while still
 allowing applications to choose to only program against macos, for example.
 
-### Fate of `Metadata::{accesed, modified}`
+#### Fate of `Metadata::{accesed, modified}`
 
 At this time there is no suitable type in the standard library to represent the
 return type of these two functions. The type would either have to be some form
@@ -135,7 +212,10 @@ requiring platform-specific code to access the modification/access time of
 files. This information is all available via the `MetadataExt` traits listed
 above.
 
-## Lowering and setting `Permissions`
+Eventually, once a `std` type for cross-platform timestamps is available, these
+methods will be re-instated as returning that type.
+
+### Lowering and setting `Permissions` (Unix)
 
 > **Note**: this section only describes behavior on unix.
 
@@ -391,4 +471,5 @@ impl DirEntry {
 
 # Unresolved questions
 
-None yet.
+* What is the ultimate role of crates like `liblibc`, and how do we draw the
+  line between them and `std::os` definitions?