Canary Automated Testing #471
Conversation
…ration tests before pushing to either cloud
WalkthroughThe update restructures the GitHub Actions workflow in Changes
Suggested reviewers
Possibly related PRs
Warning Review ran into problems🔥 ProblemsGitHub Actions and Pipeline Checks: Resource not accessible by integration - https://docs.github.com/rest/actions/workflow-runs#list-workflow-runs-for-a-repository. Please grant the required permissions to the CodeRabbit GitHub App under the organization or repository settings. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (6)
.github/workflows/test_and_publish_release.yaml (6)
1-9: Set workflow name & version.
Update the TODO for VERSION when a release tag is available.
342-356: Fix artifact indent.
Align the 'with:' keys to 10-space indent.🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 345-345: wrong indentation: expected 10 but found 12
(indentation)
[warning] 350-350: wrong indentation: expected 10 but found 12
(indentation)
[warning] 355-355: wrong indentation: expected 10 but found 12
(indentation)
82-82: Remove trailing spaces.
Clean extra whitespace on these lines.Also applies to: 85-85, 91-91, 181-181, 197-197, 213-213, 245-245, 257-257
🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 82-82: trailing spaces
(trailing-spaces)
174-174: Trim spaces after colon.
Reduce extra spaces after ':' for consistency.🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 174-174: too many spaces after colon
(colons)
386-386: Adjust indentation.
Correct indent to expected 6 spaces.🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 386-386: wrong indentation: expected 6 but found 8
(indentation)
187-187: Fix typo.
Correct "baackfill" to "backfill".
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro (Legacy)
📒 Files selected for processing (2)
.github/workflows/push_to_canary.yaml(1 hunks).github/workflows/test_and_publish_release.yaml(1 hunks)
🧰 Additional context used
🪛 actionlint (1.7.4)
.github/workflows/test_and_publish_release.yaml
298-298: the runner of "aws-actions/configure-aws-credentials@v3" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
391-391: the runner of "aws-actions/configure-aws-credentials@v3" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
🪛 YAMLlint (1.35.1)
.github/workflows/test_and_publish_release.yaml
[error] 82-82: trailing spaces
(trailing-spaces)
[error] 85-85: trailing spaces
(trailing-spaces)
[error] 91-91: trailing spaces
(trailing-spaces)
[warning] 174-174: too many spaces after colon
(colons)
[error] 181-181: trailing spaces
(trailing-spaces)
[error] 197-197: trailing spaces
(trailing-spaces)
[error] 213-213: trailing spaces
(trailing-spaces)
[error] 245-245: trailing spaces
(trailing-spaces)
[error] 257-257: trailing spaces
(trailing-spaces)
[warning] 345-345: wrong indentation: expected 10 but found 12
(indentation)
[warning] 350-350: wrong indentation: expected 10 but found 12
(indentation)
[warning] 355-355: wrong indentation: expected 10 but found 12
(indentation)
[warning] 386-386: wrong indentation: expected 6 but found 8
(indentation)
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: build-artifacts
🔇 Additional comments (1)
.github/workflows/push_to_canary.yaml (1)
163-164: Use secrets for GCP auth.
Secrets replace hardcoded values correctly.
| - name: Configure AWS Credentials for Canary Project | ||
| uses: aws-actions/configure-aws-credentials@v3 | ||
| with: | ||
| role-to-assume: arn:aws:iam::${{secrets.AWS_ACCOUNT_ID}}:role/github_actions | ||
| aws-region: ${{secrets.AWS_REGION}} | ||
|
|
||
| - name: On Failure |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Update AWS creds action (Canary).
Switch from v3 to v4 for AWS credentials.
- uses: aws-actions/configure-aws-credentials@v3
+ uses: aws-actions/configure-aws-credentials@v4📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| - name: Configure AWS Credentials for Canary Project | |
| uses: aws-actions/configure-aws-credentials@v3 | |
| with: | |
| role-to-assume: arn:aws:iam::${{secrets.AWS_ACCOUNT_ID}}:role/github_actions | |
| aws-region: ${{secrets.AWS_REGION}} | |
| - name: On Failure | |
| - name: Configure AWS Credentials for Canary Project | |
| uses: aws-actions/configure-aws-credentials@v4 | |
| with: | |
| role-to-assume: arn:aws:iam::${{secrets.AWS_ACCOUNT_ID}}:role/github_actions | |
| aws-region: ${{secrets.AWS_REGION}} | |
| - name: On Failure |
🧰 Tools
🪛 actionlint (1.7.4)
298-298: the runner of "aws-actions/configure-aws-credentials@v3" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
| - name: Configure AWS Credentials for Main Project | ||
| uses: aws-actions/configure-aws-credentials@v3 | ||
| with: | ||
| role-to-assume: arn:aws:iam::${{secrets.AWS_MAIN_ACCOUNT_ID}}:role/github-actions | ||
| aws-region: ${{secrets.AWS_REGION}} |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Update AWS creds action (Main).
Upgrade to aws-actions/configure-aws-credentials@v4.
- uses: aws-actions/configure-aws-credentials@v3
+ uses: aws-actions/configure-aws-credentials@v4📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| - name: Configure AWS Credentials for Main Project | |
| uses: aws-actions/configure-aws-credentials@v3 | |
| with: | |
| role-to-assume: arn:aws:iam::${{secrets.AWS_MAIN_ACCOUNT_ID}}:role/github-actions | |
| aws-region: ${{secrets.AWS_REGION}} | |
| - name: Configure AWS Credentials for Main Project | |
| uses: aws-actions/configure-aws-credentials@v4 | |
| with: | |
| role-to-assume: arn:aws:iam::${{secrets.AWS_MAIN_ACCOUNT_ID}}:role/github-actions | |
| aws-region: ${{secrets.AWS_REGION}} |
🧰 Tools
🪛 actionlint (1.7.4)
391-391: the runner of "aws-actions/configure-aws-credentials@v3" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (8)
.github/workflows/test_and_publish_release.yaml (8)
7-9: TODO: Update VERSION sourcing
Replace the hardcoded0.0.0with${{ github.event.release.tag_name }}.
82-82: Remove trailing spaces in Bazel commands
Remove the extra space at the end of the--google_credentials=bazel-cache-key.jsonlines.
For example:- --google_credentials=bazel-cache-key.json + --google_credentials=bazel-cache-key.jsonAlso applies to: 85-85, 88-88
🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 82-82: trailing spaces
(trailing-spaces)
165-165: Fix excess spacing after colon
GCP_REGION:has too many spaces before the secret value.
For example:- GCP_REGION: ${secrets.GCP_REGION} + GCP_REGION: ${secrets.GCP_REGION}🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 165-165: too many spaces after colon
(colons)
172-172: Trim trailing whitespace
Remove trailing spaces on these lines to clean up the shell script.Also applies to: 188-188, 204-204, 236-236, 248-248
🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 172-172: trailing spaces
(trailing-spaces)
370-370: Correct indentation for AWS permissions
Reduce indentation from 8 to 6 spaces.
For example:- id-token: write + id-token: write🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 370-370: wrong indentation: expected 6 but found 8
(indentation)
336-336: Fix indentation for Cloud GCP Lib Jar
Remove extra spaces so that the key aligns correctly.
For example:- name: cloud-gcp-jar + name: cloud-gcp-jar🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 336-336: wrong indentation: expected 10 but found 12
(indentation)
341-341: Fix indentation for Service Assembly Jar
Remove extra spaces for proper alignment.
For example:- name: service-assembly-jar + name: service-assembly-jar🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 341-341: wrong indentation: expected 10 but found 12
(indentation)
400-400: Resolve version placeholder TODO
Replace the version placeholder with${{ github.event.release.tag_name }}in the S3 upload step.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro (Legacy)
📒 Files selected for processing (1)
.github/workflows/test_and_publish_release.yaml(1 hunks)
🧰 Additional context used
🪛 YAMLlint (1.35.1)
.github/workflows/test_and_publish_release.yaml
[error] 82-82: trailing spaces
(trailing-spaces)
[error] 85-85: trailing spaces
(trailing-spaces)
[error] 88-88: trailing spaces
(trailing-spaces)
[warning] 165-165: too many spaces after colon
(colons)
[error] 172-172: trailing spaces
(trailing-spaces)
[error] 188-188: trailing spaces
(trailing-spaces)
[error] 204-204: trailing spaces
(trailing-spaces)
[error] 236-236: trailing spaces
(trailing-spaces)
[error] 248-248: trailing spaces
(trailing-spaces)
[warning] 336-336: wrong indentation: expected 10 but found 12
(indentation)
[warning] 341-341: wrong indentation: expected 10 but found 12
(indentation)
[warning] 370-370: wrong indentation: expected 6 but found 8
(indentation)
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: build-artifacts
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (11)
.github/workflows/test_and_publish_release.yaml (11)
1-7: Metadata & Trigger Setup:
Workflow name and push trigger are clear, but the commented-out release trigger and the TODO on line 7 indicate pending work. Consider removing obsolete comments or completing the dynamic version integration.
25-37: Install Thrift:
Commands work as expected, but consider adding the-yflag to theapt-get installcommand for non-interactive installs.
82-82: Trailing Spaces:
Extra trailing spaces detected on these lines. Please remove them.Also applies to: 85-85, 88-88
🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 82-82: trailing spaces
(trailing-spaces)
161-247: GCP Integration Tests:
The integration test step has thorough error handling and status checks. Consider using JSON parsing (e.g. withjq) for extracting job IDs for improved robustness.🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 165-165: too many spaces after colon
(colons)
[error] 172-172: trailing spaces
(trailing-spaces)
[error] 188-188: trailing spaces
(trailing-spaces)
[error] 204-204: trailing spaces
(trailing-spaces)
[error] 236-236: trailing spaces
(trailing-spaces)
172-172: Trailing Spaces Detected:
Extra trailing spaces found—please remove them.Also applies to: 188-188, 204-204, 236-236, 248-248
🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 172-172: trailing spaces
(trailing-spaces)
256-310: AWS Integration Setup:
The AWS test job mirrors the GCP setup correctly. Note the TODO on line 294; ensure the run commands are thoroughly tested once ready.
333-352: Artifact Downloads (GCP Publish):
Artifact download steps work, but static analysis flags indentation issues on lines 346 and 351. Please adjust them to the expected 10 spaces.🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 346-346: wrong indentation: expected 10 but found 12
(indentation)
[warning] 351-351: wrong indentation: expected 10 but found 12
(indentation)
353-374: Upload to GCS:
The loop for uploading artifacts usinggcloud storage cpis clear. Consider validating variable quoting in conditions for robustness.
410-410: Dynamic Versioning TODO:
Replace the hardcoded version with${{ github.event.release.tag_name }}to ensure proper versioning during publishing.
165-165: Spacing Issue:
Too many spaces after the colon detected on line 165. Please remove extra spacing.🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 165-165: too many spaces after colon
(colons)
380-380: Indentation Warning:
Line 380 is indented with 8 spaces, but 6 are expected. Adjust the indentation accordingly.🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 380-380: wrong indentation: expected 6 but found 8
(indentation)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro (Legacy)
📒 Files selected for processing (1)
.github/workflows/test_and_publish_release.yaml(1 hunks)
🧰 Additional context used
🪛 YAMLlint (1.35.1)
.github/workflows/test_and_publish_release.yaml
[error] 82-82: trailing spaces
(trailing-spaces)
[error] 85-85: trailing spaces
(trailing-spaces)
[error] 88-88: trailing spaces
(trailing-spaces)
[warning] 165-165: too many spaces after colon
(colons)
[error] 172-172: trailing spaces
(trailing-spaces)
[error] 188-188: trailing spaces
(trailing-spaces)
[error] 204-204: trailing spaces
(trailing-spaces)
[error] 236-236: trailing spaces
(trailing-spaces)
[error] 248-248: trailing spaces
(trailing-spaces)
[warning] 346-346: wrong indentation: expected 10 but found 12
(indentation)
[warning] 351-351: wrong indentation: expected 10 but found 12
(indentation)
[warning] 380-380: wrong indentation: expected 6 but found 8
(indentation)
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: build-artifacts
🔇 Additional comments (19)
.github/workflows/test_and_publish_release.yaml (19)
8-10: Environment Variable:
The VERSION is hardcoded as "0.0.0". Ensure this is updated with a dynamic value (e.g. using${{ github.event.release.tag_name }}) when ready.
12-24: Build-Artifacts Job Setup:
Job structure, permissions, and checkout step are implemented correctly.
39-44: Generate Thrift Files:
Thrift file generation appears clear and explicit.
45-49: Set Up Python:
Using actions/setup-python@v5 with Python 3.9 is fine if intentional.
50-65: Build Python Wheel:
The commands to build the wheel and verify its existence are solid.
66-71: Upload Wheel:
Artifact upload via actions/upload-artifact@v4 is correctly configured.
72-75: Bazel Cache Credentials:
Decoding and storing the cache credentials are correctly implemented.
76-91: Build Bazel Project:
The sequential Bazel builds are well structured.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 82-82: trailing spaces
(trailing-spaces)
[error] 85-85: trailing spaces
(trailing-spaces)
[error] 88-88: trailing spaces
(trailing-spaces)
92-116: Upload Artifacts (Build Phase):
Artifact upload steps for jars are clear and use the proper actions.
117-127: GCP Integration Setup:
The checkout of the configuration repository with the proper branch and SSH key is properly set up.
134-138: Download Wheel (GCP Tests):
Downloading the built wheel is appropriately configured.
139-147: Virtual Environment Setup (GCP Tests):
Creation and activation of the virtualenv appears correct; the reinstall logic is clear.
148-154: Configure GCP Credentials (Canary):
Usage of google-github-actions/auth@v2 with proper secret injection is correct.
155-160: BigQuery Cleanup:
The BigQuery table removal commands are explicit and provide proper error handling.
249-255: On Failure Handler (GCP Tests):
The failure step appropriately converts the release to a draft on errors.
311-329: GCP Publishing Setup:
Credentials configuration and SDK setup for GCP publishing are spot-on.
375-383: AWS Publishing Setup:
AWS publishing job setup and credentials configuration are correct.🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 380-380: wrong indentation: expected 6 but found 8
(indentation)
384-409: Artifact Downloads (AWS Publish):
Downloading wheel and jar artifacts via actions/download-artifact@v4 is implemented properly.
411-431: Upload to S3:
The artifact upload script for S3 is well structured; the loop and conditional for the "latest" tag are clear.
There was a problem hiding this comment.
Thanks for setting this up! https://linear.app/zipline-ai/issue/ZIP-620/schedule-integration-testing-on-cron-for-artifacts
Added the issue and assigned to you
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (9)
.github/workflows/test_and_publish_release.yaml (9)
1-9: Metadata & Versioning
Remove unused commented triggers and update the version source (e.g. use release tag) instead of the hardcoded "0.0.0".
82-82: Bazel Trailing Spaces
Remove trailing spaces in the Bazel build command lines.- --google_credentials=bazel-cache-key.json + --google_credentials=bazel-cache-key.jsonAlso applies to: 85-85, 88-88
🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 82-82: trailing spaces
(trailing-spaces)
165-165: Env Spacing
Reduce extra spaces after the colon in theGCP_REGIONenvironment variable.🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 165-165: too many spaces after colon
(colons)
172-172: Trailing Spaces Cleanup
Remove trailing spaces in the integration tests steps.Also applies to: 188-188, 204-204, 236-236, 248-248
🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 172-172: trailing spaces
(trailing-spaces)
295-295: TODO Reminder
Address the TODO for testing the 'run' commands in the AWS integration tests.
343-347: Fix GCP Lib Jar Indentation
Correct the indentation for thename: cloud-gcp-jarkey.🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 346-346: wrong indentation: expected 10 but found 12
(indentation)
349-352: Fix Service Assembly Jar Indentation
Correct the indentation for thename: service-assembly-jarkey.🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 351-351: wrong indentation: expected 10 but found 12
(indentation)
411-411: Update Version Placeholder
Replace the version placeholder with${{ github.event.release.tag_name }}.
380-380: Permissions Indentation
Fix the indentation for the permissions block (expected 6 spaces, found 8).🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 380-380: wrong indentation: expected 6 but found 8
(indentation)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro (Legacy)
📒 Files selected for processing (1)
.github/workflows/test_and_publish_release.yaml(1 hunks)
🧰 Additional context used
🪛 YAMLlint (1.35.1)
.github/workflows/test_and_publish_release.yaml
[error] 82-82: trailing spaces
(trailing-spaces)
[error] 85-85: trailing spaces
(trailing-spaces)
[error] 88-88: trailing spaces
(trailing-spaces)
[warning] 165-165: too many spaces after colon
(colons)
[error] 172-172: trailing spaces
(trailing-spaces)
[error] 188-188: trailing spaces
(trailing-spaces)
[error] 204-204: trailing spaces
(trailing-spaces)
[error] 236-236: trailing spaces
(trailing-spaces)
[error] 248-248: trailing spaces
(trailing-spaces)
[warning] 346-346: wrong indentation: expected 10 but found 12
(indentation)
[warning] 351-351: wrong indentation: expected 10 but found 12
(indentation)
[warning] 380-380: wrong indentation: expected 6 but found 8
(indentation)
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: build-artifacts
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (8)
.github/workflows/test_and_publish_release.yaml (8)
1-7: Workflow Setup and Trigger
Workflow name and push trigger are defined. Please address the TODO on line 7.
8-10: Environment Version
VERSIONis set to "0.0.0". Consider using the release tag dynamically.
172-172: Remove Trailing Spaces
Remove extra spaces on these lines to comply with YAML lint standards.Also applies to: 188-188, 204-204, 236-236, 248-248
🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 172-172: trailing spaces
(trailing-spaces)
256-309: AWS Integration Tests Job
The virtualenv setup and test steps are appropriate. Note the TODO regarding test 'run' commands.
310-373: GCP Artifact Publishing
Credential configuration and artifact upload steps look good. Verify if the assignment fromGCS_CUSTOMER_IDSis necessary.🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 346-346: wrong indentation: expected 10 but found 12
(indentation)
[warning] 351-351: wrong indentation: expected 10 but found 12
(indentation)
374-431: AWS Artifact Publishing
The steps are correct. Remember to replace the version placeholder as noted in the TODO on line 410.🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 380-380: wrong indentation: expected 6 but found 8
(indentation)
346-346: Fix Indentation
Correct the indentation on these lines to match YAML standards.Also applies to: 351-351, 380-380
🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 346-346: wrong indentation: expected 10 but found 12
(indentation)
1-431: Overall Workflow Review
The CI/CD pipeline covers artifact building, integration testing, and publishing. Ensure unit tests and documentation are added in a follow-up update.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 172-172: trailing spaces
(trailing-spaces)
[error] 188-188: trailing spaces
(trailing-spaces)
[error] 204-204: trailing spaces
(trailing-spaces)
[error] 236-236: trailing spaces
(trailing-spaces)
[error] 248-248: trailing spaces
(trailing-spaces)
[warning] 346-346: wrong indentation: expected 10 but found 12
(indentation)
[warning] 351-351: wrong indentation: expected 10 but found 12
(indentation)
[warning] 380-380: wrong indentation: expected 6 but found 8
(indentation)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro (Legacy)
📒 Files selected for processing (1)
.github/workflows/test_and_publish_release.yaml(1 hunks)
🧰 Additional context used
🪛 YAMLlint (1.35.1)
.github/workflows/test_and_publish_release.yaml
[error] 172-172: trailing spaces
(trailing-spaces)
[error] 188-188: trailing spaces
(trailing-spaces)
[error] 204-204: trailing spaces
(trailing-spaces)
[error] 236-236: trailing spaces
(trailing-spaces)
[error] 248-248: trailing spaces
(trailing-spaces)
[warning] 346-346: wrong indentation: expected 10 but found 12
(indentation)
[warning] 351-351: wrong indentation: expected 10 but found 12
(indentation)
[warning] 380-380: wrong indentation: expected 6 but found 8
(indentation)
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: build-artifacts
🔇 Additional comments (3)
.github/workflows/test_and_publish_release.yaml (3)
11-116: Build Artifacts Job
Checkout, Thrift installation, Python setup, wheel build, and Bazel build steps look solid.
117-248: GCP Integration Tests Job
Integration test steps are well structured. No issues noted.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 172-172: trailing spaces
(trailing-spaces)
[error] 188-188: trailing spaces
(trailing-spaces)
[error] 204-204: trailing spaces
(trailing-spaces)
[error] 236-236: trailing spaces
(trailing-spaces)
[error] 248-248: trailing spaces
(trailing-spaces)
249-255: GCP On-Failure Handling
Failure handling is clear and correctly converts the release to draft.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (7)
.github/workflows/test_and_publish_release.yaml (7)
3-4: Remove commented trigger.
Clean up unused release trigger lines.
7-7: Version TODO.
Use dynamic release tag if desired.
29-29: Add apt-get update.
Precede install cmd for reliability.
76-92: Bazel build looks fine.
Consider parallelizing for speed.
177-179: Fix typo.
"baackfill" should be "backfill."- echo "No baackfill job id available to check. Exiting." + echo "No backfill job id available to check. Exiting."
311-374: GCS upload script is neat.
Consider quoting variables in the loop.
410-410: TODO noted.
Update version var to use release tag.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro (Legacy)
📒 Files selected for processing (1)
.github/workflows/test_and_publish_release.yaml(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: build-artifacts
🔇 Additional comments (7)
.github/workflows/test_and_publish_release.yaml (7)
50-65: Wheel build OK.
161-248: GCP tests robust.
LGTM overall.
249-255: On-failure step OK.
256-310: AWS tests setup looks good.
LGTM.
289-293: AWS creds configured.
V4 is used correctly.
384-389: AWS creds OK.
V4 confirmed.
411-431: S3 upload script neat.
LGTM.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
.github/workflows/push_to_canary.yaml (2)
77-79: Remove trailing whitespace on these lines.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 77-77: trailing spaces
(trailing-spaces)
[error] 79-79: trailing spaces
(trailing-spaces)
283-284: Remove trailing whitespace in the cancellation block.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 283-283: trailing spaces
(trailing-spaces)
[error] 284-284: trailing spaces
(trailing-spaces)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro (Legacy)
📒 Files selected for processing (1)
.github/workflows/push_to_canary.yaml(3 hunks)
🧰 Additional context used
🪛 YAMLlint (1.35.1)
.github/workflows/push_to_canary.yaml
[error] 77-77: trailing spaces
(trailing-spaces)
[error] 79-79: trailing spaces
(trailing-spaces)
[error] 283-283: trailing spaces
(trailing-spaces)
[error] 284-284: trailing spaces
(trailing-spaces)
🔇 Additional comments (32)
.github/workflows/push_to_canary.yaml (32)
6-6: Branch filter is set to "main".
19-20: Job "build_artifacts" defined correctly.
52-56: Python setup step uses setup-python@v5 with version 3.11.
57-66: Latest release tag step is clear and uses $GITHUB_OUTPUT properly.
67-86: Wheel build step is well structured.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 77-77: trailing spaces
(trailing-spaces)
[error] 79-79: trailing spaces
(trailing-spaces)
87-91: Wheel upload step looks good.
111-115: Spark Assembly Jar upload configured properly.
117-121: Cloud AWS Jar upload step is correct.
123-127: Cloud GCP Jar upload step is fine.
129-133: Service Assembly Jar upload step is set up as expected.
134-135: Job output "wheel_file" is defined correctly.
137-144: "push_to_aws" job configuration and permissions look good.
147-151: Python Wheel download for AWS push is clear.
152-156: Spark Assembly Jar download step for AWS push is correct.
157-161: Cloud AWS Lib Jar download step is properly set up.
162-166: Service Assembly Jar download step is good.
167-172: AWS credentials are configured appropriately.
173-181: AWS S3 upload commands for jars and wheel are correctly specified.
183-190: "push_to_gcp" job header and permissions are set correctly.
192-196: GCP job checkout and QEMU setup steps are in order.
197-201: Python Wheel download step for GCP push is clear.
202-206: Spark Assembly Jar download in GCP job is correct.
207-211: Cloud GCP Lib Jar download step is set up properly.
212-216: Service Assembly Jar download for GCP is fine.
218-224: GCP credentials are updated with workload_identity_provider and service_account.
225-227: Google Cloud SDK setup is configured correctly.
228-243: GCS upload steps are well defined and use metadata effectively.
245-282: AWS integration tests job is configured with proper credentials and steps.
286-325: GCP integration tests job is set up correctly with cancellation handling.
326-345: Cleanup job for artifacts is clear and leverages the delete-artifact action appropriately.
346-385: Merge-to-main job handles branch updates as expected.
386-407: Slack notification on failure is correctly implemented.
| gcloud storage cp ${{ needs.build_artifacts.outputs.wheel_file }} gs://zipline-artifacts-canary/release/candidate/wheels/ |
There was a problem hiding this comment.
btw what do we think about adding the commit into the path? To avoid races on the same object. Otherwise the metadata update could race
There was a problem hiding this comment.
I updated the wheel version to include the sha. The time it takes to build the artifacts should avoid race conditions as the old run will get cancelled when a new run starts due to the concurrency rules
| aws s3 rm s3://zipline-warehouse-canary/data/aws_plaid_fv_v1 --recursive | ||
| aws glue delete-table --database-name data --name aws_plaid_fv_v1 | ||
| if [[ "$ENVIRONMENT" == "canary" ]]; then | ||
| aws s3 rm s3://zipline-warehouse-canary/data/quickstart_purchases_v1_test --recursive |
There was a problem hiding this comment.
how about we just keep this to the quickstart for now, and once the plaid_fv works we'll just cut over?
There was a problem hiding this comment.
Sure. I switched it to use v1_test for canary and v1_dev for developer use.
|
|
||
| # Delete gcp tables to start from scratch | ||
| bq rm -f -t canary-443022:data.gcp_purchases_v1_dev | ||
| bq rm -f -t canary-443022:data.gcp_purchases_v1_dev_upload | ||
| if [[ "$ENVIRONMENT" == "canary" ]]; then |
There was a problem hiding this comment.
not sure why we need this branching? shouldn't it be the same regardless
There was a problem hiding this comment.
If it is identical it will cause a race if someone tries to run the tests at the same time as the canary testing. This is to ensure that the resources don't conflict.
There was a problem hiding this comment.
feels like we should then specify the environment as the dataset then (eg bq rm -f -t canary-443022:data.${ENVIRONMENT}_purchases_v1_test but okay sounds good
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
.github/workflows/push_to_canary.yaml (2)
67-86: Wheel build step OK.
Logic is clear; please remove trailing spaces on lines 77 and 79.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 77-77: trailing spaces
(trailing-spaces)
[error] 79-79: trailing spaces
(trailing-spaces)
245-273: AWS integration test step review.
Test command is fine; remove trailing spaces on line 271.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 271-271: trailing spaces
(trailing-spaces)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro (Legacy)
📒 Files selected for processing (2)
.github/workflows/push_to_canary.yaml(3 hunks)scripts/distribution/run_aws_quickstart.sh(4 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- scripts/distribution/run_aws_quickstart.sh
🧰 Additional context used
🪛 YAMLlint (1.35.1)
.github/workflows/push_to_canary.yaml
[error] 77-77: trailing spaces
(trailing-spaces)
[error] 79-79: trailing spaces
(trailing-spaces)
[error] 271-271: trailing spaces
(trailing-spaces)
🔇 Additional comments (16)
.github/workflows/push_to_canary.yaml (16)
6-6: Verify branch selection.
Ensure the branch list (main) meets your intent.
19-19: Job ID update.
build_artifactsis correctly defined; confirm downstream integrations.
52-56: Python setup looks good.
Using actions/setup-python@v5 with version 3.11 is appropriate.
57-66: Release tag extraction is solid.
Tag retrieval and setting output via GITHUB_OUTPUT is correctly implemented.
87-92: Artifact upload verified.
Wheel upload step uses expected output correctly.
111-116: Spark jar upload looks fine.
The upload action for the Spark Assembly Jar is correct.
117-122: Cloud AWS jar upload OK.
No issues with the AWS jar artifact upload step.
123-128: Cloud GCP jar upload is proper.
Artifact configuration aligns with expectations.
129-133: Service jar upload valid.
The assembly jar for the service is correctly referenced.
134-136: Output defined.
wheel_fileoutput is set for downstream job consumption.
138-181: push_to_aws steps are solid.
All download and AWS S3 push commands use cross-job outputs as intended.
183-243: push_to_gcp configuration is correct.
GCS bucket commands and credential setup are properly configured.
274-304: GCP integration tests look good.
No issues noted in the integration steps.
305-324: Cleanup step verified.
Artifact deletion via the action is properly defined.
325-364: Merge job functions well.
Branch update and merge logic are clear and concise.
365-386: Slack notification configured correctly.
Failure notification payload is concise and appropriate.
…l plaid_fv gets fixed
…wy-zlai/release_testing
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (6)
.github/workflows/push_to_canary.yaml (6)
57-66: Latest Tag Retrieval:
Shell commands work, but consider quoting arguments to avoid globbing.🧰 Tools
🪛 actionlint (1.7.4)
59-59: shellcheck reported issue in this script: SC2086:info:6:20: Double quote to prevent globbing and word splitting
(shellcheck)
67-86: Build Wheel Step:
Consider calling the script with quotes:- ./scripts/distribution/build_wheel.sh ${{ env.VERSION }} + ./scripts/distribution/build_wheel.sh "${{ env.VERSION }}"Also, remove trailing spaces at lines 77 and 79.
🧰 Tools
🪛 actionlint (1.7.4)
72-72: shellcheck reported issue in this script: SC2086:info:13:46: Double quote to prevent globbing and word splitting
(shellcheck)
🪛 YAMLlint (1.35.1)
[error] 77-77: trailing spaces
(trailing-spaces)
[error] 79-79: trailing spaces
(trailing-spaces)
173-181: S3 Upload Commands:
Wrap file variables in quotes for safety. For example:- aws s3 cp ${{ needs.build_artifacts.outputs.wheel_file }} s3://zipline-artifacts-canary/release/candidate/wheels/ --metadata="updated_date=$(date),commit=$(git rev-parse HEAD),branch=$(git rev-parse --abbrev-ref HEAD)" + aws s3 cp "${{ needs.build_artifacts.outputs.wheel_file }}" s3://zipline-artifacts-canary/release/candidate/wheels/ --metadata="updated_date=$(date),commit=$(git rev-parse HEAD),branch=$(git rev-parse --abbrev-ref HEAD)"
228-242: GCS Upload Commands:
Quote file variables similarly to the AWS step.
265-272: AWS Test Run:
"Run Quickstart Integration Tests" works; remove trailing spaces at line 271 and consider quoting shell variables.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 271-271: trailing spaces
(trailing-spaces)
325-353: Merge to Stable Branch:
Merge logic is fine, but wrap variables in quotes (e.g., for TARGET_BRANCH) to prevent word splitting. For example:- git push origin $TARGET_BRANCH + git push origin "$TARGET_BRANCH"🧰 Tools
🪛 actionlint (1.7.4)
342-342: shellcheck reported issue in this script: SC2086:info:2:36: Double quote to prevent globbing and word splitting
(shellcheck)
342-342: shellcheck reported issue in this script: SC2086:info:4:16: Double quote to prevent globbing and word splitting
(shellcheck)
342-342: shellcheck reported issue in this script: SC2086:info:8:19: Double quote to prevent globbing and word splitting
(shellcheck)
342-342: shellcheck reported issue in this script: SC2086:info:10:17: Double quote to prevent globbing and word splitting
(shellcheck)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro (Legacy)
📒 Files selected for processing (2)
.github/workflows/push_to_canary.yaml(3 hunks)scripts/distribution/run_aws_quickstart.sh(4 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- scripts/distribution/run_aws_quickstart.sh
🧰 Additional context used
🪛 actionlint (1.7.4)
.github/workflows/push_to_canary.yaml
59-59: shellcheck reported issue in this script: SC2086:info:6:20: Double quote to prevent globbing and word splitting
(shellcheck)
72-72: shellcheck reported issue in this script: SC2086:info:13:46: Double quote to prevent globbing and word splitting
(shellcheck)
342-342: shellcheck reported issue in this script: SC2086:info:2:36: Double quote to prevent globbing and word splitting
(shellcheck)
342-342: shellcheck reported issue in this script: SC2086:info:4:16: Double quote to prevent globbing and word splitting
(shellcheck)
342-342: shellcheck reported issue in this script: SC2086:info:8:19: Double quote to prevent globbing and word splitting
(shellcheck)
342-342: shellcheck reported issue in this script: SC2086:info:10:17: Double quote to prevent globbing and word splitting
(shellcheck)
🪛 YAMLlint (1.35.1)
.github/workflows/push_to_canary.yaml
[error] 77-77: trailing spaces
(trailing-spaces)
[error] 79-79: trailing spaces
(trailing-spaces)
[error] 271-271: trailing spaces
(trailing-spaces)
🔇 Additional comments (21)
.github/workflows/push_to_canary.yaml (21)
6-6: Branch Specification:
Branches now explicitly list "main".
19-24: Job Naming:
"build_artifacts" job is correctly defined.
52-56: Python Setup:
"Setup Python" step with v5 is good.
87-92: Artifact Upload:
"Upload Wheel" step is clear and correct.
111-116: Spark Jar Upload:
Step is correct.
117-122: AWS Jar Upload:
Step is fine.
123-128: GCP Jar Upload:
Step looks good.
129-134: Service Jar Upload:
No issues.
134-136: Output Definition:
Output "wheel_file" is properly exposed.
138-144: AWS Push Job:
"push_to_aws" job dependencies and permissions are set correctly.
147-166: Artifact Downloads (AWS):
All download steps function as expected.
167-172: AWS Credentials:
Configuration is proper.
183-190: GCP Push Job – Initial Steps:
Permissions and job setup are correct.
192-216: Artifact Downloads (GCP):
Download steps are fine.
218-224: GCP Credentials:
Configured correctly.
225-227: GCloud SDK Setup:
Step is acceptable.
245-252: AWS Integration Tests Job:
Job header and permissions are proper.
274-304: GCP Integration Tests Job:
Setup and test step (id: gcp_integration_tests) are good.
305-323: Clean Up Artifacts:
Cleanup step is implemented correctly.
354-363: CI Status and Artifact Update:
These steps are clear and correct.
365-386: Slack Notification:
Failure notification step is well configured.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (4)
.github/workflows/push_to_canary.yaml (4)
5-7: Commented Branch Filter: Uncomment branch filtering if intended.
67-86: Build Python Wheel: The wheel build step works as expected; please remove trailing spaces on lines 77 and 79.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 77-77: trailing spaces
(trailing-spaces)
[error] 79-79: trailing spaces
(trailing-spaces)
245-273: AWS Integration Tests: Checkout and test execution are correct; remove trailing spaces on line 271. Note the TODO is still pending.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 271-271: trailing spaces
(trailing-spaces)
325-364: Merge to Main: Merging logic is sound; consider refining the merge commit message for clarity.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro (Legacy)
📒 Files selected for processing (1)
.github/workflows/push_to_canary.yaml(3 hunks)
🧰 Additional context used
🪛 YAMLlint (1.35.1)
.github/workflows/push_to_canary.yaml
[error] 77-77: trailing spaces
(trailing-spaces)
[error] 79-79: trailing spaces
(trailing-spaces)
[error] 271-271: trailing spaces
(trailing-spaces)
⏰ Context from checks skipped due to timeout of 90000ms (2)
- GitHub Check: build_artifacts
- GitHub Check: enforce_triggered_workflows
🔇 Additional comments (15)
.github/workflows/push_to_canary.yaml (15)
19-24: build_artifacts Job Added: Job is defined correctly.
52-56: Set up Python Step: Proper usage of actions/setup-python with version 3.11.
57-66: Get Release Tag: Correct tag retrieval and use of the new output syntax.
87-92: Upload Wheel: Artifact upload correctly references the wheel output.
93-109: Build Bazel Project: Bazel build commands are clear and correctly configured.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 99-99: trailing spaces
(trailing-spaces)
[error] 102-102: trailing spaces
(trailing-spaces)
[error] 105-105: trailing spaces
(trailing-spaces)
111-116: Upload Spark Assembly Jar: Upload step is valid.
117-122: Upload Cloud AWS Jar: Correctly configured artifact upload.
123-128: Upload Cloud GCP Jar: Uploads the jar as expected.
129-133: Upload Service Assembly Jar: Artifact upload is handled correctly.
134-136: Job Outputs: Exposing the wheel_file output is appropriate.
138-181: push_to_aws Job: Artifact download and S3 push commands use updated references correctly.
183-244: push_to_gcp Job: Credentials configuration and GCS upload steps are well implemented.
274-304: GCP Integration Tests: Test credentials and script execution are set up properly.
305-324: Clean Up Artifacts: The delete-artifact action is used effectively.
365-386: Slack Notification: Failure notification payload is correctly defined.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
.github/workflows/push_to_canary.yaml (2)
67-86: Build Python Wheel:
Wheel build logic is sound and outputs the wheel file.
Nitpick: Remove trailing spaces on lines 77 and 79.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 77-77: trailing spaces
(trailing-spaces)
[error] 79-79: trailing spaces
(trailing-spaces)
245-274: Run AWS Integration Tests:
The AWS integration test step invokes the quickstart script as expected.
Nitpick: Remove trailing spaces on line 271.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 271-271: trailing spaces
(trailing-spaces)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro (Legacy)
📒 Files selected for processing (1)
.github/workflows/push_to_canary.yaml(3 hunks)
🧰 Additional context used
🪛 YAMLlint (1.35.1)
.github/workflows/push_to_canary.yaml
[error] 77-77: trailing spaces
(trailing-spaces)
[error] 79-79: trailing spaces
(trailing-spaces)
[error] 271-271: trailing spaces
(trailing-spaces)
🔇 Additional comments (13)
.github/workflows/push_to_canary.yaml (13)
19-51: Build Artifacts Job Definition:
Job renamed to build_artifacts and its Java/Thrift steps align well with the PR objectives.
52-56: Python Setup Step:
Using actions/setup-python@v5 with Python 3.11 is clear and correct.
57-66: Latest Release Tag Retrieval:
The tag is fetched and output using the new syntax via$GITHUB_OUTPUT—nice and concise.
87-92: Upload Wheel:
Artifact upload correctly references thebuild-wheeloutput.
93-109: Bazel Build Project:
Bazel build commands and remote cache usage are correct.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 99-99: trailing spaces
(trailing-spaces)
[error] 102-102: trailing spaces
(trailing-spaces)
[error] 105-105: trailing spaces
(trailing-spaces)
111-133: Artifact Upload Steps:
All jar upload steps are consistent. Verify that the file paths (e.g. spark_assembly_deploy.jar) are valid in the workspace.
134-136: Job Outputs:
The output mapping forwheel_fileis correctly set from the wheel build step.
138-182: Push to AWS Job:
Downloads artifacts and pushes jars to S3 with metadata usingneeds.build_artifacts.outputs.wheel_file. Ensure that the jar filenames and their paths are correct.
183-244: Push to GCP Job:
GCP credentials and gcloud storage commands are clear and consistently applied.
274-304: Run GCP Integration Tests:
The GCP integration test job is set up properly and includes anidfor correct cancellation reference.
305-323: Clean Up Artifacts:
The cleanup job uses thedelete-artifactaction appropriately to remove the published artifacts.
325-364: Merge to Main-Passing-Tests Job:
The merge logic for updating the stable branch is clear and handles branch creation/merging as needed.
365-386: On-Fail Slack Notification:
The Slack notification payload is concise and will alert on integration test failures.
## Summary This updates the Push To Canary workflow to push the updated jars and wheel to the cloud canaries and run integration tests. The jobs are as follows: * **build_artifacts** builds the jars and wheel from the main branch and uploads them to github for use in separate jobs simultaneously * **push_to_aws** downloads the jars and wheel from github and uploads them to the s3 zipline-artifacts-canary bucket under version "candidate" * **push_to_gcp** downloads the jars and wheel from github and uploads them to the gcs zipline-artifacts-canary bucket under version "candidate" * **run_aws_integration_tests** runs the quickstart compile and backfill based on [scripts/distribution/run_aws_quickstart.sh](https://github.com/zipline-ai/chronon/blob/main/scripts/distribution/run_aws_quickstart.sh) * **run_gcp_integration_tests** runs the quickstart compile, backfill, group-by-upload, upload-to-kv, metadata-upload, and fetch based on [scripts/distribution/run_gcp_quickstart.sh](https://github.com/zipline-ai/chronon/blob/main/scripts/distribution/run_gcp_quickstart.sh) * **clean_up_artifacts** removes the jars and wheel from github to minimize the storage costs. * **merge_to_main-passing-tests** if the tests all pass, this will merge the latest commits to the branch "main-passing-tests". This also creates a txt file artifact marking which commit sha passed which is saved by github. * **on_fail_notify_slack** if the tests fail, this will send a notification to the "integration-tests" channel of our Slack. A test run of the workflow can be seen here: https://github.com/zipline-ai/chronon/actions/runs/13913346353 ## Checklist - [ ] Added Unit Tests - [ ] Covered by existing CI - [x] Integration tested - [ ] Documentation update <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary by CodeRabbit - **Chores** - Enhanced our release pipeline to streamline package generation and distribution. - Automated deployment of new build packages across AWS and GCP. - Introduced cleanup routines to optimize the post-deployment process. - Updated job structure for improved artifact management and deployment. - Added new jobs for building artifacts and uploading them to cloud storage, including `build_artifacts`, `push_to_aws`, `push_to_gcp`, and `clean_up_artifacts`. - Defined outputs for generated artifacts to improve tracking. - Improved flexibility in deployment scripts for different environments (dev and canary). - Updated scripts to conditionally handle resource management based on environment settings. - Introduced environment-specific configurations for AWS and GCP operations. - Added integration testing jobs for AWS and GCP post-deployment. - Enhanced scripts for AWS and GCP to support environment-specific operations and configurations. - Introduced a mechanism to switch between different environments affecting resource management and command configurations. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Summary
This updates the Push To Canary workflow to push the updated jars and wheel to the cloud canaries and run integration tests.
The jobs are as follows:
builds the jars and wheel from the main branch and uploads them to github for use in separate jobs simultaneously
downloads the jars and wheel from github and uploads them to the s3 zipline-artifacts-canary bucket under version "candidate"
downloads the jars and wheel from github and uploads them to the gcs zipline-artifacts-canary bucket under version "candidate"
runs the quickstart compile and backfill based on scripts/distribution/run_aws_quickstart.sh
runs the quickstart compile, backfill, group-by-upload, upload-to-kv, metadata-upload, and fetch based on scripts/distribution/run_gcp_quickstart.sh
removes the jars and wheel from github to minimize the storage costs.
if the tests all pass, this will merge the latest commits to the branch "main-passing-tests". This also creates a txt file artifact marking which commit sha passed which is saved by github.
if the tests fail, this will send a notification to the "integration-tests" channel of our Slack.
A test run of the workflow can be seen here: https://github.com/zipline-ai/chronon/actions/runs/13913346353
Checklist
Summary by CodeRabbit
Summary by CodeRabbit
build_artifacts,push_to_aws,push_to_gcp, andclean_up_artifacts.