updating docs for compiling (#3483)

* updating docs for compiling

* clean up docs

Author: joshfischer1108

docker/scripts/build-docker.sh

@@ -96,13 +96,13 @@ case $# in
     echo "Usage: $0 <platform> <version_string> <artifact-directory> [-s|--squash]"
     echo "  "
     echo "Argument options:"
-    echo "  <platform>: darwin, debian9, ubuntu14.04, ubuntu16.04, ubuntu18.04, centos7"
+    echo "  <platform>: darwin, debian9, debian10, ubuntu14.04, ubuntu18.04, centos7"
     echo "  <version_string>: Version of Heron build, e.g. v0.17.5.1-rc"
     echo "  <artifact-directory>: Location of compiled Heron artifact"
     echo "  [-s|--squash]: Enables using Docker experimental feature --squash"
     echo "  "
     echo "Example:"
-    echo "  ./build-docker.sh ubuntu14.04 0.12.0 ~/ubuntu"
+    echo "  ./build-docker.sh ubuntu18.04 0.12.0 ~/ubuntu"
     echo "  "
     exit 1
     ;;

docker/scripts/dev-env-create.sh

@@ -43,7 +43,7 @@ case $# in
 esac
 
 # Default platform is ubuntu18.04. Other available platforms
-# include centos7, debian9
+# include centos7, debian9, debian10, ubuntu18.04
 TARGET_PLATFORM=${2:-"ubuntu18.04"}
 SCRATCH_DIR="$HOME/.heron-docker"
 REPOSITORY="heron-dev"

website2/docs/compiling-docker.md

@@ -24,7 +24,7 @@ For developing Heron, you will need to compile it for the environment that you
 want to use it in. If you'd like to use Docker to create that build environment,
 Heron provides a convenient script to make that process easier.
 
-Currently, only Ubuntu 14.04, Ubuntu 18.04, and CentOS 7 are supported, but if you
+Currently Debian10 and Ubuntu 18.04 are actively being supported.  There is also limited support for Ubuntu 14.04, Debian9, and CentOS 7. If you
 need another platform there are instructions for adding new ones
 [below](#contributing-new-environments).
 
@@ -58,21 +58,39 @@ $ docker/build-artifacts.sh
 Running the script by itself will display usage information:
 
 ```
-Usage: docker/build-artifacts.sh <platform> <version_string> [source-tarball] <output-directory>
-
-Platforms Supported: darwin, ubuntu14.04, ubuntu18.04, centos7
-
+Script to build heron docker image for different platforms
+  Input - directory containing the artifacts from the directory <artifact-directory>
+  Output - docker image tar file saved in the directory <artifact-directory> 
+  
+Usage: ./docker/scripts/build-docker.sh <platform> <version_string> <artifact-directory> [-s|--squash]
+  
+Argument options:
+  <platform>: darwin, debian9, debian10, ubuntu14.04, ubuntu18.04, centos7
+  <version_string>: Version of Heron build, e.g. v0.17.5.1-rc
+  <artifact-directory>: Location of compiled Heron artifact
+  [-s|--squash]: Enables using Docker experimental feature --squash
+  
 Example:
-  ./build-artifacts.sh ubuntu14.04 0.12.0 .
+  ./build-docker.sh ubuntu18.04 0.12.0 ~/ubuntu
 
 NOTE: If running on OSX, the output directory will need to
       be under /Users so virtualbox has access to.
 ```
 
 The following arguments are required:
 
-* `platform` --- Currently, this can be one of: `ubuntu14.04`, `centos7`. You
-  can add other platforms using the [instructions
+* `platform` --- Currently we are focused on supporting the `debian10` and `ubuntu18.04` platforms.  
+We also support building Heron locally on OSX.  You can specify this as listing `darwin` as the platform.
+ All options are:
+   - `centos7`
+   - `darwin`
+   - `debian9`
+   - `debian10`
+   - `ubuntu14.04`
+   - `ubuntu18.04`
+    
+   
+  You can add other platforms using the [instructions
   below](#contributing-new-environments).
 * `version-string` --- The Heron release for which you'd like to build
   artifacts.
@@ -82,10 +100,10 @@ The following arguments are required:
 Here's an example usage:
 
 ```bash
-$ docker/scripts/build-artifacts.sh ubuntu14.04 0.12.0 ~/heron-release
+$ docker/scripts/build-artifacts.sh debian10 0.22.1-incubating ~/heron-release
 ```
 
-This will build a Docker container specific to Ubuntu 14.04, create a source
+This will build a Docker container specific to Debian10, create a source
 tarball of the Heron repository, run a full release build of Heron, and then
 copy the artifacts into the `~/heron-release` directory.
 
@@ -105,12 +123,12 @@ of the generated artifacts:
 
 ```bash
 $ ls ~/heron-release
-heron-0.12.0-ubuntu14.04.tar
-heron-0.12.0-ubuntu14.04.tar.gz
-heron-core-0.12.0-ubuntu14.04.tar.gz
-heron-install-0.12.0-ubuntu14.04.sh
-heron-layer-0.12.0-ubuntu14.04.tar
-heron-tools-0.12.0-ubuntu14.04.tar.gz
+heron-0.22.1-incubating-debian10.tar
+heron-0.22.1-incubating-debian10.tar.gz
+heron-core-0.22.1-incubating-debian10.tar.gz
+heron-install-0.22.1-incubating-debian10.sh
+heron-layer-0.22.1-incubating-debian10.tar
+heron-tools-0.22.1-incubating-debian10.tar.gz
 ```
 
 ## Set Up A Docker Based Development Environment
@@ -130,8 +148,8 @@ After the commands, a new docker container is started with all the libraries and
 installed. The operation system is Ubuntu 18.04 by default. Now you can build Heron
 like:
 ```bash
-\# bazel build --config=ubuntu scripts/packages:binpkgs
-\# bazel build --config=ubuntu scripts/packages:tarpkgs
+\# bazel build --config=debian scripts/packages:binpkgs
+\# bazel build --config=debian scripts/packages:tarpkgs
 ```
 
 The current folder is mapped to the '/heron' directory in the container and any changes
@@ -221,7 +239,7 @@ ENV JAVA_HOME /usr/lib/jvm/java-11-openjdk-amd64
 Here's an example:
 
 ```dockerfile
-RUN wget -O /tmp/bazel.sh https://github.com/bazelbuild/bazel/releases/download/0.23.2/bazel-0.23.2-installer-linux-x86_64.sh \
+RUN wget -O /tmp/bazel.sh https://github.com/bazelbuild/bazel/releases/download/0.26.0/bazel-0.26.0-installer-linux-x86_64.sh \
          && chmod +x /tmp/bazel.sh \
          && /tmp/bazel.sh
 ```

website2/docs/compiling-osx.md

@@ -32,10 +32,16 @@ install it using this one-liner:
 $ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
 ```
 
-### Step 2 --- Install bazel and other required libraries
+### Step 2 -- Install Bazel
+```bash
+wget -O /tmp/bazel.sh https://github.com/bazelbuild/bazel/releases/download/0.26.0/bazel-0.26.0-installer-darwin-x86_64.sh
+chmod +x /tmp/bazel.sh
+/tmp/bazel.sh --user
+```
+
+### Step 2 --- Install other required libraries
 
 ```bash
-brew install bazel
 brew install automake
 brew install cmake
 brew install libtool

website2/docs/compiling-overview.md

@@ -21,9 +21,9 @@ sidebar_label: Compiling Overview
 -->
 
 Heron is currently available for [Mac OS X 10.14](compiling-osx),
-[Ubuntu 14.04](compiling-linux), and [CentOS
-7](compiling-linux#building-on-centos-7). This guide describes the basics of the
-Heron build system. For step-by-step build instructions for a specific platform,
+[Ubuntu 18.04](compiling-linux), and [Debian10](compiling-docker#building-heron).
+ This guide describes the basics of the
+Heron build system. For step-by-step build instructions for other platforms,
 the following guides are available:
 
 * [Building on Linux Platforms](compiling-linux)
@@ -38,11 +38,10 @@ Instructions on running unit tests for Heron can also be found in [Testing Heron
 You must have the following installed to compile Heron:
 
 * [Bazel](http://bazel.io/docs/install.html) = {{% bazelVersion %}}. Later
-  versions might work but have not been tested. See [Installing Bazel]({{< ref
-  "#installing-bazel" >}}) below.
+  versions might work but have not been tested. See [Installing Bazel](#installing-bazel)below.
 * [Java 11](https://www.oracle.com/java/technologies/javase-jdk11-downloads.html)
   is required by Bazel and Heron;
-  [topologies](../../../concepts/topologies) can be written in Java 7 or above
+  topologies can be written in Java 7 or above
   , but Heron jars are required to run with a Java 11 JRE.
 * [Autoconf](http://www.gnu.org/software/autoconf/autoconf.html) >=
   2.6.3
@@ -65,8 +64,8 @@ $ echo $CC $CXX
 
 ## Installing Bazel
 
-Heron uses the [Bazel](http://bazel.io) build tool. Bazel releases can be found
-[here](https://github.com/bazelbuild/bazel/releases/tag/{{% bazelVersion %}})
+Heron uses the [Bazel](http://bazel.io) build tool. Bazel releases can be found here:
+https://github.com/bazelbuild/bazel/releases/tag/{{% bazelVersion %}}
 and installation instructions can be found [here](http://bazel.io/docs/install.html).
 
 To ensure that Bazel has been installed, run `bazel version` and check the
@@ -92,8 +91,9 @@ OS-specific configuration using the `--config` flag. The following OS values
 are supported:
 
 * `darwin` (Mac OS X)
-* `ubuntu` (Ubuntu 14.04)
-* `centos5` (CentOS 5)
+* `ubuntu` (Ubuntu 18.04)
+* `debian` (Debian10)
+* `centos5` (CentOS 7)
 
 For example, on Mac OS X (`darwin`), the following command will build all
 packages:

website2/docs/getting-started-local-single-node.md

@@ -22,63 +22,44 @@ sidebar_label: Local (Single Node)
 
 > The current version of Heron is {{heron:version}}
 
-
-The easiest way to get started learning Heron is to install the Heron client tools, which are currently available for:
+The current way to get started learning Heron is to build from source. 
+Please refer to the [guide to compiling Heron](compiling-overview). The Heron community is working to add release scripts and Heron
+binaries available from Apache mirrors soon.
 
 * [MacOS](#macos-homebrew)
-* [Ubuntu >= 14.04](#using-installation-scripts)
+* [Ubuntu 18.04](#using-installation-scripts)
 * [CentOS](#using-installation-scripts)
 
-For other platforms, you need to build from source. Please refer to the [guide to compiling Heron](compiling-overview).
-
-## Step 1 --- Download the Heron tools
-
-Heron tools can be installed on [macOS](#macos-homebrew) using [Homebrew](https://brew.sh) and on Linux using [installation scripts](#using-installation-scripts).
+## Building installation scripts
 
-> You can install using [installation scripts](#using-installation-scripts) on macOS as well.
+The only way to obtain the installation scripts at this time are to compile from source.  We are working to add release scripts and Heron
+binaries available from Apache mirrors soon.
 
-## macOS/Homebrew
+See how to get setup building Heron see the notes [here](compiling-overview.md)
 
-The easiest way to get started with Heron on macOS is using [Homebrew](https://brew.sh):
+If you already have Bazel and the other required tools installed for your platform you can execute the following:
 
 ```bash
-$ brew tap streamlio/homebrew-formulae
-$ brew update
-$ brew install heron
+$ bazel build --config=[platform] scripts/packages:binpkgs
 ```
-
-This will install a variety of executables and other resources by default in `/usr/local/opt/heron`.
-
-> Homebrew may use a different folder than `/usr/local/opt/heron`. To check which folder is being used, run `brew --prefix heron`.
-
-## Using installation scripts
-
-To install Heron binaries directly, using installation scripts, go to Heron's [releases page](https://github.com/apache/incubator-heron/releases) on GitHub
-and see a full listing of Heron releases for each available platform. The installation script for macOS (`darwin`), for example, is named
-`heron-install-{{% heronVersion %}}-darwin.sh`.
-
-```bash
-$ wget https://github.com/apache/incubator-heron/releases/download/{{% heronVersion %}}/heron-install-{{% heronVersion %}}-darwin.sh
+Options for the config flag are:
+* `debian` (Debian10)
+* `ubuntu` (Ubuntu)
+* `darwin` (OSX)
+
+This command will take approximately 45 minutes to compile based on your machine settings. 
+ It will output a script called `heron-install.sh`  You will be able to find it relative to the Heron root folder:
+ 
+ ```bash
+$ ./bazel-bin/scripts/packages/heron-install.sh 
 ```
 
-Download the for your platform either from the releases page or using [`wget`](https://www.gnu.org/software/wget/). Here's a `wget` example for Ubuntu:
-
-```bash
-$ wget https://github.com/apache/incubator-heron/releases/download/{{% heronVersion %}}/heron-install-{{% heronVersion %}}-ubuntu.sh
-```
-
-Once you've downloaded the script, make it executable using [chmod](https://en.wikipedia.org/wiki/Chmod):
-
-```bash
-$ chmod +x heron-*.sh
-```
+Now run the [Heron client](user-manuals-heron-cli) installation script with the `--user` flag set. Here's an example:
 
 > The script will install executables in the `~/bin` folder. You should add that folder to your `PATH` using `export PATH=~/bin:$PATH`.
 
-Now run the [Heron client](user-manuals-heron-cli) installation script with the `--user` flag set. Here's an example for Ubuntu:
-
 ```bash
-$ ./heron-install-{{% heronVersion %}}-ubuntu.sh --user
+$ ./heron-install.sh --user
 Heron installer
 ---------------
 
@@ -91,13 +72,13 @@ To check that Heron is successfully installed, run `heron version`:
 
 ```bash
 $ heron version
-heron.build.git.revision : 26bb4096130a05f9799510bbce6c37a69a7342ef
-heron.build.git.status : Clean
-heron.build.host : ...
-heron.build.time : Sat Aug  6 12:35:47 PDT {{% currentYear %}}
-heron.build.timestamp : 1470512147000
+heron.build.git.revision : b580f689d8cbcb3026966bde2aacf6da74d5bcf5
+heron.build.git.status : Modified
+heron.build.host : MBP
+heron.build.time : Sun Mar 22 06:42:05 CDT 2020
+heron.build.timestamp : 1584877325000
 heron.build.user : ...
-heron.build.version : {{% heronVersion %}}
+heron.build.version : update-docs-compiling
 ```
 
 ## Step 2 --- Launch an example topology
@@ -261,7 +242,7 @@ Optional arguments:
 
 The source code for the example topologies can be found
 [on
-GitHub]({{% githubMaster %}}/examples/src/java/org/apache/heron/examples).
+GitHub](https://github.com/apache/incubator-heron/tree/master/examples/src/java/org/apache/heron/examples).
 The included example topologies:
 
 * `AckingTopology.java` --- A topology with acking enabled.
@@ -286,4 +267,4 @@ In case of any issues, please refer to [Quick Start Troubleshooting](getting-sta
 * [Deploy topologies](deployment-overview) in clustered, scheduler-driven
   environments (such as on [Aurora](schedulers-aurora-cluster)
   and [locally](schedulers-local))
-* [Develop topologies](heron-architecture) for Heron
+* [Develop topologies](heron-architecture) for Heron