Initial version of npiperelay

Author: jstarks

README.md

@@ -0,0 +1,170 @@
+# npiperelay
+
+npiperelay is a tool that allows you to access a Windows named pipe in a way
+that is more compatible with a variety of command-line tools. With it, you can
+use Windows named pipes from the Windows Subsystem for Linux (WSL).
+
+For example, you can:
+
+* Connect to Docker for Windows from the Linux Docker client in WSL
+* Connect interactively to a Hyper-V Linux VM's serial console
+
+Let me know on Twitter ([@gigastarks](https://twitter.com/gigastarks)) if you come up with more interesting uses.
+
+# Installation
+
+Binaries for npiperelay are not currently available. You have to build from source. With Go, this is not too difficult.
+
+Basic steps:
+
+1. Install Go.
+2. Download and build the Windows binary and add it to your path.
+3. Install socat.
+
+## Installing Go
+
+To build the binary, you will need a version of [Go](https://golang.org). You can use a Windows build of Go or, as outlined here, you can use a Linux build and cross-compile the Windows binary directly from WSL.
+
+## Building npiperelay.exe
+
+Once you have Go installed (and your GOPATH configured), you need to download and install the tool. This is a little tricky because we are building the tool for Windows from WSL:
+
+```bash
+$ go get -d github.com/jstarks/npiperelay
+$ GOOS=windows go build -o /mnt/c/Users/<myuser>/go/bin/npiperelay.exe github.com/jstarks/npiperelay
+```
+
+In this example, we have put the binary in `/mnt/c/Users/<myuser>/go/bin`. We then need to make sure that this directory is available in the WSL path. This can be achieved either by adding C:\Users\<myuser>\go\bin to the Win32 path and restarting WSL, or by just adding the path directly in WSL via the command line or in our `.bash_profile` or `.bashrc`.
+
+Or you can just symlink it into something that's already in your path:
+
+```bash
+$ sudo ln -s /mnt/c/Users/<myuser>/go/bin/npiperelay.exe /usr/local/bin/npiperelay.exe
+```
+
+You may be tempted to just put the real binary directly into `/usr/local/bin`, but this will not work because Windows currently cannot run binaries that exist in the Linux namespace -- they have to reside somewhere under the Windows portion of the file system.
+
+## Installing socat
+
+For all of the examples below, you will need the excellent `socat` tool. Your WSL distribution should
+have it available; install it by running
+
+```bash
+$ sudo apt install socat
+```
+
+or the equivalent.
+
+# Usage
+
+The examples below assume you have copied the contents of the `scripts` directory (from `$HOME/go/src/github.com/jstarks/npiperelay/scripts`) into your PATH somewhere. These scripts are just examples and can be modified to suit your needs.
+
+## Connecting to Docker from WSL
+
+This assumes you already have the Docker daemon running in Windows, e.g. because you have installed Docker for Windows. You may already have the ability to connect to this daemon from WSL via TCP, but this has security problems because any user on your machine will be able to connect. With these steps, you'll be able to limit access to privileged users.
+
+Basic steps:
+
+1. Start the Docker relay.
+2. Use the `docker` CLI as usual.
+
+### Staring the Docker relay
+
+For this to work, you will need to be running in an elevated WSL session, or you will need to configure Docker to allow your Windows user access without elevating.
+
+You also need to be running as root within WSL, or launch the command under sudo. This is necessary because the relay will create a file /var/run/docker.sock.
+
+```bash
+$ sudo docker-relay &
+```
+
+### Using the docker CLI with the relay
+
+At this point, ordinary `docker` commands should run fine as root. Try
+
+```bash
+$ sudo docker info
+```
+
+If this succeeds, then you are connected. Now try some other Docker commands:
+
+```bash
+$ sudo docker run -it --rm microsoft/nanoserver cmd /c "Back in Windows again..."
+```
+
+#### Running without root
+
+The `docker-relay` script configured the Docker pipe to allow access by the
+`docker` group. To run as an ordinary user, add your WSL user to the docker
+group. In Ubuntu:
+
+```bash
+$ sudo adduser <my_user> docker
+```
+
+Then open a new WSL window to reset your group membership.
+
+## Connecting to a Hyper-V Linux VM's serial console
+
+If you have a Linux VM configured in Hyper-V, you may wish to use its serial
+port as a serial console. With npiperelay, this can be done fairly easily from
+the command line.
+
+Basic steps:
+
+1. Enable the serial port for your Linux VM.
+2. Configure your VM to run the console on the serial port.
+3. Run socat to relay between your terminal and npiperelay.
+
+### Enabling the serial port
+
+This is easiest to do from the command line, via the Hyper-V PowerShell cmdlets.
+You'll need to add your user to the Hyper-V Administrators group or run the
+command line elevated for this to work.
+
+If you have a VM named `foo` and you want to enable the console on COM1 (/dev/ttyS0), with a named pipe name of `foo_debug_pipe`:
+
+```bash
+$ powershell.exe Set-VMComPort foo 1 '\\.\pipe\foo_debug_pipe'
+```
+
+### Configuring your VM to run the console on the serial port
+
+Refer to your VM Linux distribution's instructions for enabling the serial console:
+
+* [Ubuntu](https://help.ubuntu.com/community/SerialConsoleHowto)
+* [Fedora](https://docs.fedoraproject.org/f26/system-administrators-guide/kernel-module-driver-configuration/Working_with_the_GRUB_2_Boot_Loader.html#sec-GRUB_2_over_a_Serial_Console])
+
+### Connecting to the serial port
+
+For this step, WSL must be running elevated
+
+#### Directly via socat
+
+The easiest approach is to use socat to connect directly. The `vmserial-connect` script does this and even looks up the pipe name from the VM name and COM port for you:
+
+```bash
+$ vmserial-connect foo 1
+<enter>
+Ubuntu 17.04 gigastarks-vm ttyS0
+
+gigastarks-vm login:
+```
+
+Press Ctrl-O to exit the connection and return to your shell.
+
+#### Via screen
+
+If you prefer to use a separate tool to connect to the device such as `screen`, then you must run a separate `socat` process to relay between the named pipe and a PTY. The `serial-relay` script does this
+for you with the right parameters; simply run:
+
+```bash
+$ serial-relay //./pipe/foo_debug_pipe $HOME/foo-pty & # Starts the relay
+$ screen $HOME/foo-pty                                 # Attaches to the serial terminal
+```
+
+See the `screen` documentation (`man screen`) for more details.
+
+## Custom usage
+
+Take a look at the scripts for sample usage, or run `npiperelay.exe` without any parameters for parameter documentation.

npiperelay.go

@@ -0,0 +1,137 @@
+package main
+
+import (
+	"flag"
+	"io"
+	"log"
+	"os"
+	"sync"
+	"syscall"
+	"time"
+
+	"golang.org/x/sys/windows"
+)
+
+const cERROR_PIPE_NOT_CONNECTED syscall.Errno = 233
+
+var (
+	poll            = flag.Bool("p", false, "poll until the the named pipe exists")
+	closeWrite      = flag.Bool("s", false, "send a 0-byte message to the pipe after EOF on stdin")
+	closeOnEOF      = flag.Bool("ep", false, "terminate on EOF reading from the pipe, even if there is more data to write")
+	closeOnStdinEOF = flag.Bool("ei", false, "terminate on EOF reading from stdin, even if there is more data to write")
+	verbose         = flag.Bool("v", false, "verbose output on stderr")
+)
+
+func dialPipe(p string, poll bool) (*overlappedFile, error) {
+	p16, err := windows.UTF16FromString(p)
+	if err != nil {
+		return nil, err
+	}
+	for {
+		h, err := windows.CreateFile(&p16[0], windows.GENERIC_READ|windows.GENERIC_WRITE, 0, nil, windows.OPEN_EXISTING, windows.FILE_FLAG_OVERLAPPED, 0)
+		if err == nil {
+			return newOverlappedFile(h), nil
+		}
+		if poll && os.IsNotExist(err) {
+			time.Sleep(200 * time.Millisecond)
+			continue
+		}
+		return nil, &os.PathError{Path: p, Op: "open", Err: err}
+	}
+}
+
+func underlyingError(err error) error {
+	if serr, ok := err.(*os.SyscallError); ok {
+		return serr.Err
+	}
+	return err
+}
+
+func main() {
+	flag.Parse()
+	args := flag.Args()
+	if len(args) != 1 {
+		flag.Usage()
+		os.Exit(1)
+	}
+
+	if *verbose {
+		log.Println("connecting to", args[0])
+	}
+
+	conn, err := dialPipe(args[0], *poll)
+	if err != nil {
+		log.Fatalln(err)
+	}
+
+	if *verbose {
+		log.Println("connected")
+	}
+
+	var wg sync.WaitGroup
+	wg.Add(1)
+	go func() {
+		_, err := io.Copy(conn, os.Stdin)
+		if err != nil {
+			log.Fatalln("copy from stdin to pipe failed:", err)
+		}
+
+		if *verbose {
+			log.Println("copy from stdin to pipe finished")
+		}
+
+		if *closeOnStdinEOF {
+			os.Exit(0)
+		}
+
+		if *closeWrite {
+			// A zero-byte write on a message pipe indicates that no more data
+			// is coming.
+			conn.Write(nil)
+		}
+		os.Stdin.Close()
+		wg.Done()
+	}()
+
+	_, err = io.Copy(os.Stdout, conn)
+	if underlyingError(err) == windows.ERROR_BROKEN_PIPE || underlyingError(err) == cERROR_PIPE_NOT_CONNECTED {
+		// The named pipe is closed and there is no more data to read. Since
+		// named pipes are not bidirectional, there is no way for the other side
+		// of the pipe to get more data, so do not wait for the stdin copy to
+		// finish.
+		if *verbose {
+			log.Println("copy from pipe to stdout finished: pipe closed")
+		}
+		os.Exit(0)
+	}
+
+	if err != nil {
+		log.Fatalln("copy from pipe to stdout failed:", err)
+	}
+
+	if *verbose {
+		log.Println("copy from pipe to stdout finished")
+	}
+
+	if !*closeOnEOF {
+		os.Stdout.Close()
+
+		// Keep reading until we get ERROR_BROKEN_PIPE or the copy from stdin
+		// finishes.
+		go func() {
+			for {
+				_, err := conn.Read(nil)
+				if underlyingError(err) == windows.ERROR_BROKEN_PIPE {
+					if *verbose {
+						log.Println("pipe closed")
+					}
+					os.Exit(0)
+				} else if err != nil {
+					log.Fatalln("pipe error:", err)
+				}
+			}
+		}()
+
+		wg.Wait()
+	}
+}

overlappedfile.go

@@ -0,0 +1,87 @@
+package main
+
+import (
+	"io"
+	"os"
+	"sync"
+	"unsafe"
+
+	"golang.org/x/sys/windows"
+)
+
+var getOverlappedResultFunc = windows.MustLoadDLL("kernel32.dll").MustFindProc("GetOverlappedResult")
+
+type overlappedFile struct {
+	h windows.Handle
+	m sync.Mutex
+	e []windows.Handle
+}
+
+func (f *overlappedFile) getEvent() windows.Handle {
+	f.m.Lock()
+	if len(f.e) == 0 {
+		f.m.Unlock()
+		e, err := windows.CreateEvent(nil, 0, 0, nil)
+		if err != nil {
+			panic(err)
+		}
+		return e
+	}
+	e := f.e[len(f.e)-1]
+	f.e = f.e[:len(f.e)-1]
+	f.m.Unlock()
+	return e
+}
+
+func (f *overlappedFile) putEvent(e windows.Handle) {
+	windows.ResetEvent(e)
+	f.m.Lock()
+	f.e = append(f.e, e)
+	f.m.Unlock()
+}
+
+func (f *overlappedFile) asyncIo(fn func(h windows.Handle, n *uint32, o *windows.Overlapped) error) (uint32, error) {
+	o := &windows.Overlapped{}
+	e := f.getEvent()
+	defer f.putEvent(e)
+	o.HEvent = e
+	var n uint32
+	err := fn(f.h, &n, o)
+	if err == windows.ERROR_IO_PENDING {
+		r, _, err := getOverlappedResultFunc.Call(uintptr(f.h), uintptr(unsafe.Pointer(o)), uintptr(unsafe.Pointer(&n)), 1)
+		if r == 0 {
+			return 0, err
+		}
+	} else if err != nil {
+		return 0, err
+	}
+	return n, nil
+}
+
+func (f *overlappedFile) Read(b []byte) (int, error) {
+	n, err := f.asyncIo(func(h windows.Handle, n *uint32, o *windows.Overlapped) error {
+		return windows.ReadFile(h, b, n, o)
+	})
+	err = os.NewSyscallError("read", err)
+	if err == nil && n == 0 && len(b) > 0 {
+		err = io.EOF
+	}
+	return int(n), err
+}
+
+func (f *overlappedFile) Write(b []byte) (int, error) {
+	n, err := f.asyncIo(func(h windows.Handle, n *uint32, o *windows.Overlapped) error {
+		return windows.WriteFile(h, b, n, o)
+	})
+	return int(n), os.NewSyscallError("write", err)
+}
+
+func (f *overlappedFile) Close() error {
+	windows.Close(f.h)
+	f.h = 0
+	return nil
+}
+
+func newOverlappedFile(h windows.Handle) *overlappedFile {
+	return &overlappedFile{h: h}
+}