Skip to content

Refactor dbt transformation entrypoint script #5076

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Aug 17, 2021

Conversation

ChristopheDuong
Copy link
Contributor

@ChristopheDuong ChristopheDuong commented Jul 29, 2021

What

This makes it more robust at overriding the mandatory options of --project-dir and --profiles-dir that are passed to dbt

closes #4734

@github-actions github-actions bot added the area/worker Related to worker label Jul 29, 2021
echo "Running: dbt $@"
dbt $@
else
dbt $@ --profile normalize
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because we use the --profile flag: https://github.com/dbt-labs/dbt/blob/35645a7233f4d32167d4db4a6066297cd8f79514/core/dbt/main.py#L290

We don't need to manipulate our profiles.yml to match with the declared profile in the dbt_project.yml file anymore... So it closes #4734

POSITIONAL+=("$1")
shift
;;
--project-dir=*|--project-dir)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confusing users that were using --project-dir <dir> instead of --project-dir=<dir> the script now handles both, see https://airbytehq.slack.com/archives/C021JANJ6TY/p1627511090382200?thread_ts=1627506369.378200&cid=C021JANJ6TY

@ChristopheDuong
Copy link
Contributor Author

ChristopheDuong commented Jul 29, 2021

/test connector=connectors/destination-postgres

🕑 connectors/destination-postgres https://github.com/airbytehq/airbyte/actions/runs/1079363688
✅ connectors/destination-postgres https://github.com/airbytehq/airbyte/actions/runs/1079363688

@ChristopheDuong ChristopheDuong requested a review from tuliren July 29, 2021 16:43
@ChristopheDuong ChristopheDuong merged commit 237e440 into master Aug 17, 2021
@ChristopheDuong ChristopheDuong deleted the chris/refactor-dbt-transformation-entrypoint branch August 17, 2021 11:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/worker Related to worker
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve custom dbt transformation handling profiles.yml
2 participants