Skip to content
This repository was archived by the owner on Mar 3, 2023. It is now read-only.

zookeeper to 3.5.8 #3555

Merged
merged 18 commits into from
Jul 10, 2020
Merged

zookeeper to 3.5.8 #3555

Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
1e0f8ee
upgradezookeeper3.5.8
huijunwu Jul 3, 2020
512cc05
change tab to space
huijunwu Jul 3, 2020
25317ac
updatedocker
huijunwu Jul 3, 2020
6dd745f
Merge branch 'huijunwu/20200702' of https://github.com/apache/incubat…
huijunwu Jul 3, 2020
1807c0d
updateurl
huijunwu Jul 3, 2020
b140b4f
updatedockerant
huijunwu Jul 3, 2020
ba3ede5
updatedockerfile
huijunwu Jul 3, 2020
995cadc
updateinstallzk
huijunwu Jul 3, 2020
af88c10
fixzkinstall
huijunwu Jul 4, 2020
33fa281
removespace
huijunwu Jul 5, 2020
f7715f3
updatehelmminikube
huijunwu Jul 5, 2020
86936ba
fixdarwin-mavenjson
huijunw Jul 7, 2020
2458c45
Merge branch 'master' of https://github.com/apache/incubator-heron in…
huijunw Jul 7, 2020
050c095
updatemaven_install.json
huijunw Jul 8, 2020
789cd56
fixcentos
Jul 8, 2020
9dc87bb
addantcheck
huijunwu Jul 10, 2020
ea4ca14
updatedoc
huijunwu Jul 10, 2020
d9e70f6
Updated website install steps to reference Python 3
nicknezis Jul 10, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 3 additions & 6 deletions WORKSPACE
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ maven_install(
name = "maven",
artifacts = [
"antlr:antlr:2.7.7",
"org.apache.zookeeper:zookeeper:3.4.14",
"org.apache.zookeeper:zookeeper:3.5.8",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The maven_install.json pin file needs to be updated.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably add these instructions to the documentation somewhere.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup. And maybe above the list of dependencies instead of below them.

"io.kubernetes:client-java:" + kubernetes_client_version,
"com.esotericsoftware:kryo:3.0.3",
"org.apache.avro:avro:1.7.4",
Expand Down Expand Up @@ -282,11 +282,8 @@ http_archive(
http_archive(
name = "org_apache_zookeeper",
build_file = "@//:third_party/zookeeper/BUILD",
patch_args = ["-p2"],
patches = ["//third_party/zookeeper:pkgconfig.patch"],
sha256 = "b14f7a0fece8bd34c7fffa46039e563ac5367607c612517aa7bd37306afbd1cd",
strip_prefix = "zookeeper-3.4.14",
urls = ["https://archive.apache.org/dist/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz"],
strip_prefix = "apache-zookeeper-3.5.8",
urls = ["https://archive.apache.org/dist/zookeeper/zookeeper-3.5.8/apache-zookeeper-3.5.8.tar.gz"],
)

http_archive(
Expand Down
1 change: 1 addition & 0 deletions bazel_configure.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -422,6 +422,7 @@ def main():

env_map['AR'] = discover_tool('ar', 'archiver', 'AR')
env_map['GCOV']= discover_tool('gcov','coverage tool', 'GCOV')
env_map['ANT'] = discover_tool('ant', "ant", 'ANT')
env_map['DWP'] = discover_tool_default('dwp', 'dwp', 'DWP', '/usr/bin/dwp')
env_map['NM'] = discover_tool_default('nm', 'nm', 'NM', '/usr/bin/nm')
env_map['OBJCOPY'] = discover_tool_default('objcopy', 'objcopy', 'OBJCOPY', '/usr/bin/objcopy')
Expand Down
15 changes: 15 additions & 0 deletions deploy/kubernetes/helm/templates/bookie.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,21 @@ spec:
{{- end }}
terminationGracePeriodSeconds: 0
initContainers:
{{- if eq .Values.platform "minikube" }}
# The first time, initialize BK to wipe data for minikube
- name: bookie-format
image: {{ .Values.bookkeeper.image }}
args: ["/opt/bookkeeper/bin/bookkeeper", "shell", "bookieformat", "--nonInteractive", "-f", "-deleteCookie"]
command: [ "/bin/bash", "/opt/bookkeeper/scripts/entrypoint.sh" ]
envFrom:
- configMapRef:
name: {{ .Release.Name }}-bookie-config
volumeMounts:
- name: journal-disk
mountPath: /bookkeeper/data/journal
- name: ledgers-disk
mountPath: /bookkeeper/data/ledgers
{{- end }}
# Wait until the zookeeper pods are up and running
- name: bookie-init-container
image: {{ .Values.image }} # use heron image until bookkeeper has nc
Expand Down
3 changes: 2 additions & 1 deletion docker/base/Dockerfile.base.debian9
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
FROM openjdk:11-jdk-slim-stretch

RUN apt-get -y update && apt-get -y install \
ant \
netcat-openbsd \
python3 \
unzip \
Expand All @@ -27,7 +28,7 @@ RUN apt-get -y update && apt-get -y install \
rm -rf /var/lib/apt/lists/*

# install zookeeper
ARG ZK_DIST=zookeeper-3.4.14
ARG ZK_DIST=zookeeper-3.5.8
ADD base/scripts /opt/heron-docker/scripts

RUN sh /opt/heron-docker/scripts/install-zookeeper.sh $ZK_DIST
Expand Down
2 changes: 2 additions & 0 deletions docker/base/conf/zookeeper.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,5 @@ autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
autopurge.purgeInterval=1
# Enable all four letter word commands by default
4lw.commands.whitelist=*
35 changes: 12 additions & 23 deletions docker/base/scripts/install-zookeeper.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,34 +19,23 @@
if [ "$#" -ne 1 ]; then
echo "Usage: $0 zookeeper-<version-number>"
echo "Example:"
echo "$ $0 zookeeper-3.4.14"
echo "$ $0 zookeeper-3.5.8"
exit 1
fi

ZK_DIST=$1

curl -O "https://archive.apache.org/dist/zookeeper/$ZK_DIST/$ZK_DIST.tar.gz"
tar -xzf ./$ZK_DIST.tar.gz -C /opt
rm ./$ZK_DIST.tar.gz
curl -O "https://archive.apache.org/dist/zookeeper/$ZK_DIST/apache-$ZK_DIST-bin.tar.gz"
tar -xzf ./apache-$ZK_DIST-bin.tar.gz -C /opt
rm ./apache-$ZK_DIST-bin.tar.gz

mv /opt/apache-$ZK_DIST-bin /opt/zookeeper
rm -rf \
/opt/zookeeper/README.txt \
/opt/zookeeper/NOTICE.txt \
/opt/zookeeper/README_packaging.txt \
/opt/zookeeper/docs \

mv /opt/$ZK_DIST /opt/zookeeper
rm -rf /opt/zookeeper/CHANGES.txt \
/opt/zookeeper/README.txt \
/opt/zookeeper/NOTICE.txt \
/opt/zookeeper/CHANGES.txt \
/opt/zookeeper/README_packaging.txt \
/opt/zookeeper/build.xml \
/opt/zookeeper/config \
/opt/zookeeper/contrib \
/opt/zookeeper/dist-maven \
/opt/zookeeper/docs \
/opt/zookeeper/ivy.xml \
/opt/zookeeper/ivysettings.xml \
/opt/zookeeper/recipes \
/opt/zookeeper/src \
/opt/zookeeper/$ZK_DIST.jar.asc \
/opt/zookeeper/$ZK_DIST.jar.md5 \
/opt/zookeeper/$ZK_DIST.jar.sha1

# copy zk scripts
mkdir -p /opt/zookeeper/scripts
Expand All @@ -55,6 +44,6 @@ chmod +x /opt/heron-docker/scripts/generate-zookeeper-config.sh
cp /opt/heron-docker/scripts/zookeeper-ruok.sh /opt/zookeeper/scripts/
chmod +x /opt/heron-docker/scripts/zookeeper-ruok.sh
cp /opt/heron-docker/scripts/start-zookeeper.sh /opt/zookeeper/scripts/
chmod +x /opt/heron-docker/scripts/
chmod +x /opt/heron-docker/scripts/start-zookeeper.sh
cp /opt/heron-docker/scripts/wait-for-zookeeper.sh /opt/zookeeper/scripts/
chmod +x /opt/heron-docker/scripts/wait-for-zookeeper.sh
2 changes: 2 additions & 0 deletions docker/compile/Dockerfile.centos7
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,10 @@ ENV bazelVersion 3.0.0

RUN yum -y upgrade
RUN yum -y install \
ant \
automake \
curl \
cppunit-devel \
cmake \
openssl-devel \
file \
Expand Down
1 change: 1 addition & 0 deletions docker/compile/Dockerfile.debian10
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ ENV TARGET_PLATFORM debian
ENV bazelVersion 3.0.0

RUN apt-get update && apt-get -y install \
ant \
automake \
build-essential \
cmake \
Expand Down
1 change: 1 addition & 0 deletions docker/compile/Dockerfile.debian9
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ ENV TARGET_PLATFORM debian
ENV bazelVersion 3.0.0

RUN apt-get update && apt-get -y install \
ant \
automake \
build-essential \
cmake \
Expand Down
1 change: 1 addition & 0 deletions docker/compile/Dockerfile.ubuntu14.04
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ RUN add-apt-repository ppa:george-edison55/cmake-3.x
RUN add-apt-repository ppa:openjdk-r/ppa

RUN apt-get update && apt-get -y install \
ant \
automake \
build-essential \
curl \
Expand Down
1 change: 1 addition & 0 deletions docker/compile/Dockerfile.ubuntu16.04
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ RUN apt-get update && apt-get install -y software-properties-common
RUN add-apt-repository ppa:openjdk-r/ppa

RUN apt-get update && apt-get -y install \
ant \
automake \
build-essential \
cmake \
Expand Down
1 change: 1 addition & 0 deletions docker/compile/Dockerfile.ubuntu18.04
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ ENV TARGET_PLATFORM ubuntu
ENV bazelVersion 3.0.0

RUN apt-get update && apt-get -y install \
ant \
g++ \
cmake \
automake \
Expand Down
1 change: 1 addition & 0 deletions docker/compile/Dockerfile.ubuntu20.04
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ ENV bazelVersion 3.0.0
ARG DEBIAN_FRONTEND=noninteractive

RUN apt-get update && apt-get -y install \
ant \
g++ \
cmake \
automake \
Expand Down
2 changes: 1 addition & 1 deletion docker/dist/Dockerfile.dist.centos7
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ ENV HERON_HOME /heron/heron-core/
RUN export HERON_HOME

# install zookeeper
ARG ZK_DIST=zookeeper-3.4.14
ARG ZK_DIST=zookeeper-3.5.8
ADD dist/scripts /opt/heron-docker/scripts

RUN sh /opt/heron-docker/scripts/install-zookeeper.sh $ZK_DIST
Expand Down
2 changes: 1 addition & 1 deletion docker/dist/Dockerfile.dist.debian10
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ ENV HERON_HOME /heron/heron-core/
RUN export HERON_HOME

# install zookeeper
ARG ZK_DIST=zookeeper-3.4.14
ARG ZK_DIST=zookeeper-3.5.8
ADD dist/scripts /opt/heron-docker/scripts

RUN sh /opt/heron-docker/scripts/install-zookeeper.sh $ZK_DIST
Expand Down
2 changes: 1 addition & 1 deletion docker/dist/Dockerfile.dist.debian9
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ ENV HERON_HOME /heron/heron-core/
RUN export HERON_HOME

# install zookeeper
ARG ZK_DIST=zookeeper-3.4.14
ARG ZK_DIST=zookeeper-3.5.8
ADD dist/scripts /opt/heron-docker/scripts

RUN sh /opt/heron-docker/scripts/install-zookeeper.sh $ZK_DIST
Expand Down
2 changes: 1 addition & 1 deletion docker/dist/Dockerfile.dist.ubuntu14.04
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ ENV HERON_HOME /heron/heron-core/
RUN export HERON_HOME

# install zookeeper
ARG ZK_DIST=zookeeper-3.4.14
ARG ZK_DIST=zookeeper-3.5.8
ADD dist/scripts /opt/heron-docker/scripts

RUN sh /opt/heron-docker/scripts/install-zookeeper.sh $ZK_DIST
Expand Down
2 changes: 1 addition & 1 deletion docker/dist/Dockerfile.dist.ubuntu16.04
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ ENV HERON_HOME /heron/heron-core/
RUN export HERON_HOME

# install zookeeper
ARG ZK_DIST=zookeeper-3.4.14
ARG ZK_DIST=zookeeper-3.5.8
ADD dist/scripts /opt/heron-docker/scripts

RUN sh /opt/heron-docker/scripts/install-zookeeper.sh $ZK_DIST
Expand Down
2 changes: 1 addition & 1 deletion docker/dist/Dockerfile.dist.ubuntu18.04
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ ENV HERON_HOME /heron/heron-core/
RUN export HERON_HOME

# install zookeeper
ARG ZK_DIST=zookeeper-3.4.14
ARG ZK_DIST=zookeeper-3.5.8
ADD dist/scripts /opt/heron-docker/scripts

RUN sh /opt/heron-docker/scripts/install-zookeeper.sh $ZK_DIST
Expand Down
2 changes: 1 addition & 1 deletion docker/dist/Dockerfile.dist.ubuntu20.04
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ ENV HERON_HOME /heron/heron-core/
RUN export HERON_HOME

# install zookeeper
ARG ZK_DIST=zookeeper-3.4.14
ARG ZK_DIST=zookeeper-3.5.8
ADD dist/scripts /opt/heron-docker/scripts

RUN sh /opt/heron-docker/scripts/install-zookeeper.sh $ZK_DIST
Expand Down
2 changes: 2 additions & 0 deletions docker/dist/conf/zookeeper.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,5 @@ autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
autopurge.purgeInterval=1
# Enable all four letter word commands by default
4lw.commands.whitelist=*
35 changes: 12 additions & 23 deletions docker/dist/scripts/install-zookeeper.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,34 +19,23 @@
if [ "$#" -ne 1 ]; then
echo "Usage: $0 zookeeper-<version-number>"
echo "Example:"
echo "$ $0 zookeeper-3.4.14"
echo "$ $0 zookeeper-3.5.8"
exit 1
fi

ZK_DIST=$1

curl -O "https://archive.apache.org/dist/zookeeper/$ZK_DIST/$ZK_DIST.tar.gz"
tar -xzf ./$ZK_DIST.tar.gz -C /opt
rm ./$ZK_DIST.tar.gz
curl -O "https://archive.apache.org/dist/zookeeper/$ZK_DIST/apache-$ZK_DIST-bin.tar.gz"
tar -xzf ./apache-$ZK_DIST-bin.tar.gz -C /opt
rm ./apache-$ZK_DIST-bin.tar.gz

mv /opt/apache-$ZK_DIST-bin /opt/zookeeper
rm -rf \
/opt/zookeeper/README.txt \
/opt/zookeeper/NOTICE.txt \
/opt/zookeeper/README_packaging.txt \
/opt/zookeeper/docs \

mv /opt/$ZK_DIST /opt/zookeeper
rm -rf /opt/zookeeper/CHANGES.txt \
/opt/zookeeper/README.txt \
/opt/zookeeper/NOTICE.txt \
/opt/zookeeper/CHANGES.txt \
/opt/zookeeper/README_packaging.txt \
/opt/zookeeper/build.xml \
/opt/zookeeper/config \
/opt/zookeeper/contrib \
/opt/zookeeper/dist-maven \
/opt/zookeeper/docs \
/opt/zookeeper/ivy.xml \
/opt/zookeeper/ivysettings.xml \
/opt/zookeeper/recipes \
/opt/zookeeper/src \
/opt/zookeeper/$ZK_DIST.jar.asc \
/opt/zookeeper/$ZK_DIST.jar.md5 \
/opt/zookeeper/$ZK_DIST.jar.sha1

# copy zk scripts
mkdir -p /opt/zookeeper/scripts
Expand All @@ -55,6 +44,6 @@ chmod +x /opt/heron-docker/scripts/generate-zookeeper-config.sh
cp /opt/heron-docker/scripts/zookeeper-ruok.sh /opt/zookeeper/scripts/
chmod +x /opt/heron-docker/scripts/zookeeper-ruok.sh
cp /opt/heron-docker/scripts/start-zookeeper.sh /opt/zookeeper/scripts/
chmod +x /opt/heron-docker/scripts/
chmod +x /opt/heron-docker/scripts/start-zookeeper.sh
cp /opt/heron-docker/scripts/wait-for-zookeeper.sh /opt/zookeeper/scripts/
chmod +x /opt/heron-docker/scripts/wait-for-zookeeper.sh
1 change: 1 addition & 0 deletions heron/statemgrs/tests/java/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ zookeeper_statemgr_deps_files = [
"//heron/statemgrs/src/java:zookeeper-statemgr-java",
"@maven//:org_apache_curator_curator_framework",
"@maven//:org_apache_zookeeper_zookeeper",
"@maven//:org_apache_zookeeper_zookeeper_jute",
]

localfs_deps_files = \
Expand Down
Loading