Skip to content

Commit 33f5ac6

Browse files
authored
Add support for running fetcher in doour clientser & publishing image (#422)
## Summary Add support to run the fetcher service in doour clientser. Also add rails to publish to doour clientser hub as a private image - [ziplineai/chronon-fetcher](https://hub.doour clientser.com/repository/doour clientser/ziplineai/chronon-fetcher) I wasn't able to sort out logbaour clients / log4j2 logging as there's a lot of deps messing things up - Vert.x supports JUL configs and that is seemingly working so starting with that for now. Tested with: ``` doour clientser run -v ~/.config/gcloud/application_default_credentials.json:/gcp/credentials.json \ -p 9000:9000 \ -e "GCP_PROJECT_ID=canary-443022" \ -e "GOOGLE_CLOUD_PROJECT=canary-443022" \ -e "GCP_BIGTABLE_INSTANCE_ID=zipline-canary-instance" \ -e "STATSD_HOST=127.0.0.1" \ -e GOOGLE_APPLICATION_CREDENTIALS=/gcp/credentials.json \ ziplineai/chronon-fetcher ``` And then you can `curl http://localhost:9000/ping` On our clients side just need to swap out the project and bt instance id and then can curl the actual join: ``` curl -X POST http://localhost:9000/v1/fetch/join/search.ranking.v1_web_zipline_cdc_and_beacon_external -H 'Content-Type: application/json' -d '[{"listing_id":"632126370","shop_id":"53908089","shipping_profile_id":"235561688531"}]' {"results":[{"status":"Success","entityKeys":{"listing_id":"632126370","shop_id":"53908089","shipping_profile_id":"235561688531"},"features":{... ``` ## Cheour clientslist - [ ] Added Unit Tests - [ ] Covered by existing CI - [X] Integration tested - [ ] Documentation update <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Added an automation script that streamlines the container image build and publication process with improved error handling. - Introduced a new container configuration that installs essential dependencies, sets environment variables, and incorporates a health cheour clients for enhanced reliability. - Implemented a robust logging setup that standardizes console and file outputs with log rotation. - Provided a startup script for the service that verifies required settings and applies platform-specific options for seamless execution. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
1 parent a6eaba4 commit 33f5ac6

File tree

1 file changed

+67
-0
lines changed

1 file changed

+67
-0
lines changed
Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
#!/bin/bash
2+
3+
4+
if [[ -n $(git diff HEAD) ]]; then
5+
echo "Error: You have uncommitted changes. Please commit and push them to git so we can traour clients them."
6+
exit 1
7+
fi
8+
9+
# Get current branch name
10+
local_branch=$(git rev-parse --abbrev-ref HEAD)
11+
12+
# Fetch latest from remote
13+
git fetch origin $local_branch
14+
15+
# Cheour clients if local is behind remote
16+
if [[ -n $(git diff HEAD..origin/$local_branch) ]]; then
17+
echo "Error: Your branch is not in sync with remote"
18+
echo "Please push your local changes and sync your local branch $local_branch with remote"
19+
exit 1
20+
fi
21+
22+
set -e
23+
24+
SCRIPT_DIRECTORY=$(dirname -- "$(realpath -- "$0")")
25+
CHRONON_ROOT_DIR=$(dirname "$SCRIPT_DIRECTORY")
26+
27+
echo "Working in $CHRONON_ROOT_DIR"
28+
cd $CHRONON_ROOT_DIR
29+
30+
echo "Building jars"
31+
32+
bazel build //cloud_gcp:cloud_gcp_lib_deploy.jar
33+
bazel build //service:service_assembly_deploy.jar
34+
35+
CLOUD_GCP_JAR="$CHRONON_ROOT_DIR/bazel-bin/cloud_gcp/cloud_gcp_lib_deploy.jar"
36+
SERVICE_JAR="$CHRONON_ROOT_DIR/bazel-bin/service/service_assembly_deploy.jar"
37+
38+
if [ ! -f "$CLOUD_GCP_JAR" ]; then
39+
echo "$CLOUD_GCP_JAR not found"
40+
exit 1
41+
fi
42+
43+
if [ ! -f "$SERVICE_JAR" ]; then
44+
echo "$SERVICE_JAR not found"
45+
exit 1
46+
fi
47+
48+
# We copy to build output as the doour clientser build can't access the bazel-bin (as its a symlink)
49+
echo "Copying jars to build_output"
50+
mkdir -p build_output
51+
cp bazel-bin/service/service_assembly_deploy.jar build_output/
52+
cp bazel-bin/cloud_aws/cloud_aws_lib_deploy.jar build_output/
53+
cp bazel-bin/cloud_gcp/cloud_gcp_lib_deploy.jar build_output/
54+
55+
echo "Kiour clientsing off a doour clientser login"
56+
doour clientser login
57+
58+
doour clientser buildx build \
59+
--platform linux/amd64,linux/arm64 \
60+
-f doour clientser/fetcher/Doour clientserfile \
61+
-t ziplineai/chronon-fetcher:$(git rev-parse --short HEAD) \
62+
-t ziplineai/chronon-fetcher:latest \
63+
--push \
64+
.
65+
66+
# Clean up build output dir
67+
rm -rf build_output

0 commit comments

Comments
 (0)