Skip to content

Coprocess Protocol Proposal

andychu edited this page Nov 21, 2018 · 28 revisions

Coprocess Protocol Proposal (FCLI)

Abstract

This document sketches a protocol to allow coprocesses to substitute for normal "batch" processes in shell scripts. A coprocess can be thought of as a single-threaded server that reads and writes from pipes.

The goal is to make shell scripts faster. It can also make interactive completion faster, since completion scripts often invoke (multiple) external tools.

Motivation / Analogy

Many language runtimes start up slowly, especially when there are many libraries or a JIT involved: Python, Ruby, R, Julia, the JVM (including Clojure), etc.

Startup times seem to be getting worse in general. Python 3 is faster than Python 2 in nearly all dimensions except startup time.

Let's call the protocol FCLI for now. There's a rough analogy to FastCGI and CGI. CGI starts one process per request, while FastCGI handles multiple requests in a process. (I think FastCGI is threaded unlike FCLI, but let's ignore that for now.)

Design

Why Coprocesses and not Multi-threaded Servers?

Because it will be easier for existing command line tools to implement this protocol. Many tools are written with global variables, or they are written in languages that don't freely thread anyway (Python, R, etc.).

Related

Shellac Protocol Proposal -- this protocol for shell-independent command completion can build on top of the coprocess protocol.

Clone this wiki locally