Phabricator-based Web UI

Summary: To try this UI, follow https://github.com/facebook/bistro/blob/master/web_ui/docker_toy/README.md

Author: snarkmaster

web_ui/docker_toy/.gitignore

@@ -0,0 +1,2 @@
+docker_build.log
+web_ui.tgz

web_ui/docker_toy/001-phabricator.conf

@@ -0,0 +1,14 @@
+<VirtualHost *>
+  ServerName phabdocker.example.com
+
+  DocumentRoot /home/phabricator/webroot
+
+  RewriteEngine on
+  RewriteRule ^/rsrc/(.*)     -                       [L,QSA]
+  RewriteRule ^/favicon.ico   -                       [L,QSA]
+  RewriteRule ^(.*)$          /index.php?__path__=$1  [B,L,QSA]
+</VirtualHost>
+
+<Directory "/home/phabricator/webroot">
+  Require all granted
+</Directory>

web_ui/docker_toy/Dockerfile

@@ -0,0 +1,42 @@
+FROM ubuntu:14.04
+SHELL ["/bin/bash", "-c"]
+
+RUN \
+  apt-get update && apt-get install -yq \
+    git joe apache2 mysql-server php5 php5-dev php5-curl php-apc php5-mcrypt \
+    php5-mysql php5-gd && \
+  a2enmod rewrite && \
+  php5enmod mcrypt
+
+COPY 001-phabricator.conf /etc/apache2/sites-available/
+ 
+WORKDIR '/home'
+
+RUN git clone https://github.com/phacility/libphutil.git && \
+  cd libphutil && \
+  git checkout 0cd92b1ff5c4e3cabfe691c8a1794b23e1f3f720 && \
+  cd .. && \
+  git clone https://github.com/phacility/arcanist.git && \
+  cd arcanist && \
+  git checkout d9cb5b18fbc1a22630eeaa16da7d291b206fba21 && \
+  cd .. && \
+  git clone https://github.com/phacility/phabricator.git && \
+  cd phabricator && \
+  git checkout 577d4980339f68826d534110107687f33b176e88 && \
+  cd ..
+
+ADD web_ui.tgz /home/bistro/
+
+# IMPORTANT: Setting an empty `bistro.get-state-hostname-suffix` below
+# disables a security precaution built into the Bistro UI for the sake of
+# letting you test with minimal hostname configuration.  Do NOT do this on
+# production deployments!
+RUN \
+  php -i && \
+  ln -s ../sites-available/001-phabricator.conf /etc/apache2/sites-enabled/ &&\
+  rm /etc/apache2/sites-enabled/000-default.conf && \
+  service mysql start && \
+  /home/phabricator/bin/storage upgrade --force && \
+  /home/bistro/plug_bistro_into_phabricator.sh /home && \
+  /home/phabricator/bin/config set bistro.get-state-hostname-suffix '' && \
+  service mysql stop

web_ui/docker_toy/README.md

@@ -0,0 +1,111 @@
+# Test-driving the Bistro UI
+
+
+## Security: words to the wise
+
+**IMPORTANT:** The "test-drive" Docker-based scripts provided for the UI and
+for the back-end are emphatically **NOT** appropriate for production use.
+
+I am paranoid, but If I were you, I would run them in a VM with highly
+restricted access to the Internet:
+ 
+ - No inbound connections,
+ - Allow outgoing connections only to Github, to the Docker repo hosing the
+   Ubuntu image, and to the Ubuntu package-management servers.
+
+Upon completing the setup below, you would fire up a browser inside the VM
+to play with the UI.
+
+Some specific security issues with these demos include:
+
+ - Everything runs as `root` -- don't kid yourself, Linux kernel isolation
+   used by Docker is not perfect, so this is `root` on the host VM too.
+ - Docker's security model -- more in `build/fbcode_builder/README.docker`.
+ - Lack of code hash / signature verification on the Github code.
+ - Lack of production hardening on the Phabricator set-up.
+ - Lack of production hardening on the Bistro back-end setup.
+ - Default MySQL password in containers.
+ - `bistro.get-state-hostname-suffix` is set to '', instead of a safe value.
+
+The above list is definitely not complete.
+
+
+## Start the backend first
+
+First, you'll want a running back-end. The quickest way to get one is:
+ - Use `build/fbcode_builder/README.md` to quickly build the binaries,
+ - Follow `bistro/scripts/docker_toy/README.md` to start a toy deployment.
+
+Once that's ready, let's bring up a second Docker image with a web server
+hosting the Bistro UI on port 80.
+
+
+## Docker build
+
+Run this
+
+  ./docker_build.sh &> docker_build.log ; tail docker_build.log
+
+
+## Start the Bistro UI in a Docker container
+
+Start an interactive session in the UI image you just built:
+
+  docker run --rm -it --expose 80 $(docker images -q | head -n1) /bin/bash
+  service mysql start
+  service apache2 start
+
+
+## Connect to the Bistro UI from outside the container
+
+To connect, your browser will need to send a request to the container's IP
+address with `phabdocker.example.com` in the `Host` header.  The easiest way
+is to add an appropriate line to `/etc/hosts` via the command below.  If you
+already have a browser extension that lets you set the `Host` header for
+IPs, use that instead.
+
+  sudo apt-get install jq  # If not already installed.
+  (
+    echo "# Phabricator docker image virtual host"
+    echo "$(docker inspect $(docker ps -q | head -n1) | 
+              jq -r .[0].NetworkSettings.IPAddress) phabdocker.example.com"
+  ) | sudo tee -a /etc/hosts
+
+*Reminder:* clean up your `/etc/hosts` once you're done test-driving Bistro.
+
+Now, you can go to `http://phabdocker.example.com` in your browser. Phabricator
+will ask you to register an admin account. Enter some fake data to continue.
+
+After account setup:
+ - Go to `http://phabdocker.example.com/bistro`
+ - Click the "Hostport List" radio button next to "Host:port Source"
+ - Enter `BACKEND_CONTAINER_IP:31415` into "Host:port Data".
+ - Click "View Jobs". 
+ - The next page will have colored bars representing groups of tasks. 
+   Click on one of the colored areas, then click on a task links in the
+   revealed panel to see the task's logs.
+
+
+## Toy example caveats
+
+- Read `Security: word to the wise` at the top of this file!
+
+- Your Docker-based UI will be in UTC. Change the container's timezone if
+  that's annoying.
+
+- The in-container Phabricator install has a lot of set-up issues. This is a
+  minimal example, for production use you would want to address them.
+
+
+## Docker disk usage tip: finding images unused by the latest build
+
+If you are building a lot of Docker containers, they tend to eat up disk
+quickly.  This command will list image IDs that are **NOT** in use by your
+latest build:
+
+  sort <(docker images -qa) <(
+    grep '^ ---> [0-9a-f]\+$' docker_build.log | cut -f 3- -d\  | sed p
+  ) | uniq -u
+
+Upon checking the output, you can wrap the command with `docker rmi $(...)`
+to delete the images.

web_ui/docker_toy/docker_build.sh

@@ -0,0 +1,13 @@
+#!/bin/bash -uex
+set -o pipefail
+#
+# See docker_toy/README.md before running this script!
+#
+
+# Assume that `docker_toy` is not a symlink, so we can `dirname` twice.
+docker_toy_dir=$(dirname "$0")
+web_ui_dir=$(dirname "$docker_toy_dir")
+
+tar -C "$web_ui_dir" -czf "$docker_toy_dir/web_ui.tgz" .
+cd "$docker_toy_dir"
+docker build . 

web_ui/plug_bistro_into_phabricator.sh

@@ -0,0 +1,62 @@
+#!/bin/bash -uex
+set -o pipefail
+#
+# This is normally started via `docker_toy/README.md`. You can also manually
+# run it to integrate this `bistro/web_ui/` code checkout into your current
+# Phabricator install.  CAVEAT: This code was only tested with the specific
+# `github.com/phacility/*` repo hashes in `docker_toy/Dockerfile`.
+#
+
+# Make both paths absolute for robustness. However, the bistro UI path may
+# include symlink, which we want to preserve, so we avoid `readlink`.
+#
+# To ensure we exit on error, we do not nest ${} or $() inside $().
+web_ui_dir=$(dirname "$0")
+web_ui_dir=$(cd "$web_ui_dir" && pwd)
+phab_root=${1?First arg -- directory with arcanist/, phabricator/, etc}
+phab_root=$(readlink -f "$phab_root")
+
+# Runs $1, which is presumed to be a Phabricator PHP binary with the
+# annoying property that it exits with code 0 on some errors.
+#
+# Only allow $1 to write to stderr, leaving stdout free for plumbing.
+wrap_php_errors() {
+  exception_bytes=$(
+    set -o pipefail
+    "$1" 2>&1 | tee /dev/fd/2 | (grep -i exception || :) | wc -c
+  )
+  [[ "$exception_bytes" == "0" ]]
+}
+
+arc_liberate() {
+  (
+    cd "$web_ui_dir/src"
+    "$phab_root/arcanist/bin/arc" liberate --library-name bistro 2>&1
+  )
+}
+
+set_load_libraries() {
+  load_libs_json=$(  # Not inlined so that python errors exit this script.
+    python3 -c '
+import json, sys
+print(json.dumps({"bistro": sys.argv[1]}))
+' "$web_ui_dir/src"
+  )
+  "$phab_root/phabricator/bin/config" set load-libraries "$load_libs_json"
+}
+
+celerity_map() {
+  # Unfortunately, we cannot just extend CelerityResourcesOnDisk because JS
+  # resources from the `bistro` module would not be able to access JS
+  # resources like `javelin-dom` from the `phabricator` module.  As of
+  # 8/2017, Celerity does not seem to support cross-module resolution.
+  for lang in css js ; do 
+    ln -snf "$web_ui_dir/rsrc/$lang/application/bistro" \
+      "$phab_root/phabricator/webroot/rsrc/$lang/application/bistro"
+  done
+  "$phab_root/phabricator/bin/celerity" map
+}
+
+wrap_php_errors arc_liberate
+wrap_php_errors set_load_libraries
+wrap_php_errors celerity_map

web_ui/rsrc/css/application/bistro/bistro-errors.css

@@ -0,0 +1,35 @@
+/**
+ * @provides bistro-errors
+ */
+
+.bistro-error {
+  background-color: #faa;
+  max-height: 240px; /* If we change resources, logspam can be massive */
+  overflow-x: auto; /* If the error is too wide, show a scroll bar */
+  padding: 5px;
+  width: 980px;
+}
+
+.bistro-error > pre {
+  padding: 5px;
+  white-space: pre-wrap;
+}
+
+.bistro-errors > h1 {
+  margin: 10px 0px;
+}
+
+/**
+ * Override the consensus finder's "inconsistent" markup, because multiple
+ * bottle errors is not in and of itself bad.
+ */
+.bistro-errors > .bistro-consensus-finder.inconsistent > * {
+  background-color: inherit;
+}
+.bistro-errors > .bistro-consensus-finder.inconsistent > p:after {
+  content: "";  /* No inconsistency warning */
+}
+.bistro-errors > .bistro-consensus-finder.inconsistent > p {
+  margin: 0px;
+  padding: 0px;
+}

web_ui/rsrc/css/application/bistro/bistro-job.css

@@ -0,0 +1,79 @@
+/**
+ * @provides bistro-job
+ */
+
+div.bistro-why-invalid {
+  max-height: 250px;
+  overflow: auto; /* If there are too many errors , show a scroll bar */
+}
+
+div.bistro-why-invalid > h2 {
+  margin-bottom: 5px;
+}
+
+div.bistro-why-invalid > div {
+  background-color: #faa;
+  font-weight: bold;
+  padding: 5px;
+  width: 960px;
+}
+
+.bistro-consensus-finder.inconsistent div.bistro-why-invalid {
+  padding: 0px; /* Already padded by the inconsistency rendering */
+}
+
+div.bistro-why-invalid > div > pre {
+  font-weight: normal;
+  max-height: 100px;
+  overflow: auto; /* If the error is too tall or wide, show a scroll bar */
+  padding: 5px;
+  white-space: pre-wrap;
+  word-break: break-all;
+}
+
+div.bistro-job-secondary-bar {
+  margin-top: 5px;
+  padding-left: 15px;
+}
+
+div.bistro-job-primary-bar > p {
+  padding-bottom: 3px;
+}
+
+div.bistro-job-primary-bar .stacked-bar-chart td {
+  line-height: 120%;
+  padding-bottom: 3px;
+  padding-top: 3px;
+}
+
+/* Segmented sub-bars are smaller & indented 30px beyond the normal bars */
+
+div.bistro-job-primary-sub-bar {
+  line-height: 90%;
+  margin-top: 5px;
+  padding-left: 30px;
+}
+
+div.bistro-job-primary-sub-bar .stacked-bar-chart td {
+  font-size: 10px;
+}
+
+div.bistro-job-primary-sub-bar > p {
+  color: gray;
+  font-size: 11px;
+}
+
+div.bistro-job-secondary-sub-bar {
+  line-height: 90%;
+  margin-top: 5px;
+  padding-left: 45px;
+}
+
+div.bistro-job-secondary-sub-bar .stacked-bar-chart td {
+  font-size: 10px;
+}
+
+div.bistro-job-secondary-sub-bar > p {
+  color: gray;
+  font-size: 11px;
+}