Skip to content
This repository was archived by the owner on Mar 3, 2023. It is now read-only.

Commit 3be9f06

Browse files
clean up site docs (#3626)
* clean up * Update website2/docs/topology-development-streamlet-api.md Co-authored-by: Oliver Bristow <[email protected]> * clean up duplicates * clean up * last version clean up * commit before reset * adding version to link Co-authored-by: Oliver Bristow <[email protected]>
1 parent 64e8167 commit 3be9f06

20 files changed

+3631
-1136
lines changed

website2/docs/compiling-code-organization.md

+26-26
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ sidebar_label: Code Organization
2222

2323
This document contains information about the Heron codebase intended primarily
2424
for developers who want to contribute to Heron. The Heron codebase lives on
25-
[github]({{% githubMaster %}}).
25+
[github](https://github.com/apache/incubator-heron/tree/master).
2626

2727
If you're looking for documentation about developing topologies for a Heron
2828
cluster, see [Building Topologies](topology-development-topology-api-java) instead.
@@ -51,7 +51,7 @@ Information on setting up and using Bazel for Heron can be found in [Compiling H
5151
* **Inter-component communication** --- Heron uses [Protocol
5252
Buffers](https://developers.google.com/protocol-buffers/?hl=en) for
5353
communication between components. Most `.proto` definition files can be found in
54-
[`heron/proto`]({{% githubMaster %}}/heron/proto).
54+
[`heron/proto`](https://github.com/apache/incubator-heron/tree/master/heron/proto).
5555

5656
* **Cluster coordination** --- Heron relies heavily on ZooKeeper for cluster
5757
coordination for distributed deployment, be it for [Aurora](schedulers-aurora-cluster) or for a [custom
@@ -61,7 +61,7 @@ Management](#state-management) section below.
6161

6262
## Common Utilities
6363

64-
The [`heron/common`]({{% githubMaster %}}/heron/common) contains a variety of
64+
The [`heron/common`](https://github.com/apache/incubator-heron/tree/master/heron/common) contains a variety of
6565
utilities for each of Heron's languages, including useful constants, file
6666
utilities, networking interfaces, and more.
6767

@@ -70,7 +70,7 @@ utilities, networking interfaces, and more.
7070
Heron supports two cluster schedulers out of the box:
7171
[Aurora](schedulers-aurora-cluster) and a [local
7272
scheduler](schedulers-local). The Java code for each of those
73-
schedulers can be found in [`heron/schedulers`]({{% githubMaster %}}/heron/schedulers)
73+
schedulers can be found in [`heron/schedulers`](https://github.com/apache/incubator-heron/tree/master/heron/schedulers)
7474
, while the underlying scheduler API can be found [here](/api/org/apache/heron/spi/scheduler/package-summary.html)
7575

7676
Info on custom schedulers can be found in [Implementing a Custom
@@ -83,10 +83,10 @@ Deployment](schedulers-local).
8383

8484
The parts of Heron's codebase related to
8585
[ZooKeeper](http://zookeeper.apache.org/) are mostly contained in
86-
[`heron/state`]({{% githubMaster %}}/heron/state). There are ZooKeeper-facing
87-
interfaces for [C++]({{% githubMaster %}}/heron/state/src/cpp),
88-
[Java]({{% githubMaster %}}/heron/state/src/java), and
89-
[Python]({{% githubMaster %}}/heron/state/src/python) that are used in a variety of
86+
[`heron/state`](https://github.com/apache/incubator-heron/tree/master/heron/state). There are ZooKeeper-facing
87+
interfaces for [C++](https://github.com/apache/incubator-heron/tree/master/heron/state/src/cpp),
88+
[Java](https://github.com/apache/incubator-heron/tree/master/heron/state/src/java), and
89+
[Python](https://github.com/apache/incubator-heron/tree/master/heron/state/src/python) that are used in a variety of
9090
Heron components.
9191

9292
## Topology Components
@@ -95,25 +95,25 @@ Heron components.
9595

9696
The C++ code for Heron's [Topology
9797
Master](heron-architecture#topology-master) is written in C++ can be
98-
found in [`heron/tmaster`]({{% githubMaster %}}/heron/tmaster).
98+
found in [`heron/tmaster`](https://github.com/apache/incubator-heron/tree/master/heron/tmaster).
9999

100100
### Stream Manager
101101

102102
The C++ code for Heron's [Stream
103103
Manager](heron-architecture#stream-manager) can be found in
104-
[`heron/stmgr`]({{% githubMaster %}}/heron/stmgr).
104+
[`heron/stmgr`](https://github.com/apache/incubator-heron/tree/master/heron/stmgr).
105105

106106
### Heron Instance
107107

108108
The Java code for [Heron
109109
instances](heron-architecture#heron-instance) can be found in
110-
[`heron/instance`]({{% githubMaster %}}/heron/instance).
110+
[`heron/instance`](https://github.com/apache/incubator-heron/tree/master/heron/instance).
111111

112112
### Metrics Manager
113113

114114
The Java code for Heron's [Metrics
115115
Manager](heron-architecture#metrics-manager) can be found in
116-
[`heron/metricsmgr`]({{% githubMaster %}}/heron/metricsmgr).
116+
[`heron/metricsmgr`](https://github.com/apache/incubator-heron/tree/master/heron/metricsmgr).
117117

118118
If you'd like to implement your own custom metrics handler (known as a **metrics
119119
sink**), see [Implementing a Custom Metrics Sink](extending-heron-metric-sink).
@@ -123,7 +123,7 @@ sink**), see [Implementing a Custom Metrics Sink](extending-heron-metric-sink).
123123
### Topology API
124124

125125
Heron's API for writing topologies is written in Java. The code for this API can
126-
be found in [`heron/api`]({{% githubMaster %}}/heron/api).
126+
be found in [`heron/api`](https://github.com/apache/incubator-heron/tree/master/heron/api).
127127

128128
Documentation for writing topologies can be found in [Building
129129
Topologies](topology-development-topology-api-java), while API documentation can be found
@@ -142,7 +142,7 @@ The Java API for simulator can be found in
142142
Heron's codebase includes a wide variety of example
143143
[topologies](heron-topology-concepts) built using Heron's topology API for
144144
Java. Those examples can be found in
145-
[`heron/examples`]({{% githubMaster %}}/heron/examples).
145+
[`heron/examples`](https://github.com/apache/incubator-heron/tree/master/heron/examples).
146146

147147
## User Interface Components
148148

@@ -152,45 +152,45 @@ Heron has a tool called `heron` that is used to both provide a CLI interface
152152
for [managing topologies](user-manuals-heron-cli) and to perform much of
153153
the heavy lifting behind assembling physical topologies in your cluster.
154154
The Python code for `heron` can be found in
155-
[`heron/tools/cli`]({{% githubMaster %}}/heron/tools/cli).
155+
[`heron/tools/cli`](https://github.com/apache/incubator-heron/tree/master/heron/tools/cli).
156156

157157
Sample configurations for different Heron schedulers
158158

159-
* [Local scheduler](schedulers-local) config can be found in [`heron/config/src/yaml/conf/local`]({{% githubMaster %}}/heron/config/src/yaml/conf/local),
160-
* [Aurora scheduler](schedulers-aurora-cluster) config can be found [`heron/config/src/yaml/conf/aurora`]({{% githubMaster %}}/heron/config/src/yaml/conf/aurora).
159+
* [Local scheduler](schedulers-local) config can be found in [`heron/config/src/yaml/conf/local`](https://github.com/apache/incubator-heron/tree/master/heron/config/src/yaml/conf/local),
160+
* [Aurora scheduler](schedulers-aurora-cluster) config can be found [`heron/config/src/yaml/conf/aurora`]({https://github.com/apache/incubator-heron/tree/master/heron/config/src/yaml/conf/aurora).
161161

162162
### Heron Tracker
163163

164164
The Python code for the [Heron Tracker](user-manuals-heron-tracker-runbook) can be
165-
found in [`heron/tools/tracker`]({{% githubMaster %}}/heron/tools/tracker).
165+
found in [`heron/tools/tracker`](https://github.com/apache/incubator-heron/tree/master/heron/tools/tracker).
166166

167167
The Tracker is a web server written in Python. It relies on the
168168
[Tornado](http://www.tornadoweb.org/en/stable/) framework. You can add new HTTP
169169
routes to the Tracker in
170-
[`main.py`]({{% githubMaster %}}/heron/tools/tracker/src/python/main.py) and
170+
[`main.py`](https://github.com/apache/incubator-heron/tree/master/heron/tools/tracker/src/python/main.py) and
171171
corresponding handlers in the
172-
[`handlers`]({{% githubMaster %}}/heron/tools/tracker/src/python/handlers) directory.
172+
[`handlers`](https://github.com/apache/incubator-heron/tree/master/heron/tools/tracker/src/python/handlers) directory.
173173

174174
### Heron UI
175175

176176
The Python code for the [Heron UI](user-manuals-heron-ui) can be found in
177-
[`heron/tools/ui`]({{% githubMaster %}}/heron/tools/ui).
177+
[`heron/tools/ui`](https://github.com/apache/incubator-heron/tree/master/heron/tools/ui).
178178

179179
Like Heron Tracker, Heron UI is a web server written in Python that relies on
180180
the [Tornado](http://www.tornadoweb.org/en/stable/) framework. You can add new
181181
HTTP routes to Heron UI in
182-
[`main.py`]({{% githubMaster %}}/heron/web/source/python/main.py) and corresponding
183-
handlers in the [`handlers`]({{% githubMaster %}}/heron/web/source/python/handlers)
182+
[`main.py`](https://github.com/apache/incubator-heron/tree/master/heron/web/source/python/main.py) and corresponding
183+
handlers in the [`handlers`](https://github.com/apache/incubator-heron/tree/master/heron/web/source/python/handlers)
184184
directory.
185185

186186
### Heron Shell
187187

188188
The Python code for the [Heron Shell](user-manuals-heron-shell) can be
189-
found in [`heron/shell`]({{% githubMaster %}}/heron/shell). The HTTP handlers and
189+
found in [`heron/shell`](https://github.com/apache/incubator-heron/tree/master/heron/shell). The HTTP handlers and
190190
web server are defined in
191-
[`main.py`]({{% githubMaster %}}/heron/shell/src/python/main.py) while the HTML,
191+
[`main.py`](https://github.com/apache/incubator-heron/tree/master/heron/shell/src/python/main.py) while the HTML,
192192
JavaScript, CSS, and images for the web UI can be found in the
193-
[`assets`]({{% githubMaster %}}/heron/shell/assets) directory.
193+
[`assets`](https://github.com/apache/incubator-heron/tree/master/heron/shell/assets) directory.
194194

195195
## Tests
196196

website2/docs/guides-effectively-once-java-topologies.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -240,7 +240,7 @@ public class EffectivelyOnceTopology {
240240

241241
### Submitting the topology
242242

243-
The code for this topology can be found in [this GitHub repository](https://github.com/streamlio/heron-java-effectively-once-example). You can clone the repo locally like this:
243+
The code for this topology can be found in. You can clone the repo locally like this:
244244

245245
```bash
246246
$ git clone https://github.com/streamlio/heron-java-effectively-once-example

website2/docs/topology-development-streamlet-api.md

+10-18
Original file line numberDiff line numberDiff line change
@@ -113,34 +113,26 @@ $ heron submit local \
113113

114114
### Java Streamlet API starter project
115115

116-
If you'd like to up and running quickly with the Heron Streamlet API for Java, you can clone [this repository](https://github.com/streamlio/heron-java-streamlet-api-example), which includes an example topology built using the Streamlet API as well as the necessary Maven configuration. To build a JAR with dependencies of this example topology:
117-
118-
```bash
119-
$ git clone https://github.com/streamlio/heron-java-streamlet-api-example
120-
$ cd heron-java-streamlet-api-example
121-
$ mvn assembly:assembly
122-
$ ls target/*.jar
123-
target/heron-java-streamlet-api-example-latest-jar-with-dependencies.jar
124-
target/heron-java-streamlet-api-example-latest.jar
125-
```
116+
If you'd like to up and running quickly with the Heron Streamlet API for Java, you can view the example topologies [here](https://github.com/apache/incubator-heron/tree/{{ heron:version }}/examples/src/java/org/apache/heron/examples/streamlet)
126117

127118
If you're running a [local Heron cluster](getting-started-local-single-node), you can submit the built example topology like this:
128119

129120
```bash
130-
$ heron submit local target/heron-java-streamlet-api-example-latest-jar-with-dependencies.jar \
131-
io.streaml.heron.streamlet.WordCountStreamletTopology \
132-
WordCountStreamletTopology
121+
$ heron submit local \
122+
~/.heron/examples/heron-streamlet-examples.jar \
123+
org.apache.heron.examples.streamlet.WindowedWordCountTopology \
124+
streamletWindowedWordCount
133125
```
134126

135127
#### Selecting delivery semantics
136128

137-
Heron enables you to apply one of three [delivery semantics](heron-delivery-semantics) to any Heron topology. For the [example topology](#java-streamlet-api-starter-project) above, you can select the delivery semantics when you submit the topology with the topology's second argument. This command, for example, would apply [effectively-once](heron-delivery-semantics) to the example topology:
129+
Heron enables you to apply one of three [delivery semantics](heron-delivery-semantics) to any Heron topology. For the example topology above, you can select the delivery semantics when you submit the topology with the topology's second argument. This command, for example, would apply [effectively-once](heron-delivery-semantics) to the example topology:
138130

139131
```bash
140-
$ heron submit local target/heron-java-streamlet-api-example-latest-jar-with-dependencies.jar \
141-
io.streaml.heron.streamlet.WordCountStreamletTopology \
142-
WordCountStreamletTopology \
143-
effectively-once
132+
$ heron submit local \
133+
~/.heron/examples/heron-streamlet-examples.jar \
134+
org.apache.heron.examples.streamlet.WireRequestsTopology \
135+
wireRequestsTopology
144136
```
145137

146138
The other options are `at-most-once` and `at-least-once`. If you don't explicitly select the delivery semantics, at-least-once semantics will be applied.

0 commit comments

Comments
 (0)