Skip to content

Commit 7437f04

Browse files
fix: customer warehouse bucket (#704)
## Summary ## Checklist - [ ] Added Unit Tests - [ ] Covered by existing CI - [ ] Integration tested - [ ] Documentation update <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Improved handling of Google Cloud Storage bucket selection for file uploads, now automatically using the appropriate warehouse bucket for each customer. <!-- end of auto-generated comment: release notes by coderabbit.ai --> <!-- av pr metadata This information is embedded by the av CLI when creating PRs to track the status of stacks when using Aviator. Please do not delete or edit this section of the PR. ``` {"parent":"main","parentHead":"","trunk":"main"} ``` --> --------- Co-authored-by: Thomas Chow <[email protected]>
1 parent af8e70b commit 7437f04

File tree

2 files changed

+4
-2
lines changed

2 files changed

+4
-2
lines changed

api/python/ai/chronon/repo/gcp.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@
1616
check_call,
1717
check_output,
1818
extract_filename_from_path,
19+
get_customer_warehouse_bucket,
1920
get_environ_arg,
2021
retry_decorator,
2122
split_date_range,
@@ -217,7 +218,6 @@ def generate_dataproc_submitter_args(
217218
):
218219

219220
parsed = urlparse(customer_artifact_prefix)
220-
customer_bucket_name = parsed.netloc
221221
source_blob_name = parsed.path.lstrip("/")
222222

223223
if local_files_to_upload is None:
@@ -233,7 +233,7 @@ def generate_dataproc_submitter_args(
233233
)
234234
gcs_files.append(
235235
GcpRunner.upload_gcs_blob(
236-
customer_bucket_name, source_file, destination_file_path
236+
get_customer_warehouse_bucket(), source_file, destination_file_path
237237
)
238238
)
239239
gcs_file_args = ",".join(gcs_files)

api/python/ai/chronon/repo/utils.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,8 @@ def get_environ_arg(env_name, ignoreError=False) -> str:
5252
raise ValueError(f"Please set {env_name} environment variable")
5353
return value
5454

55+
def get_customer_warehouse_bucket() -> str:
56+
return f"zipline-warehouse-{get_customer_id()}"
5557

5658
def get_customer_id() -> str:
5759
return get_environ_arg("CUSTOMER_ID")

0 commit comments

Comments
 (0)