Skip to content

fix: Add postgres compatibility in PrestoViewMetadataExtractor #1878

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

chonyy
Copy link
Contributor

@chonyy chonyy commented May 28, 2022

Summary of Changes

  • Related issue: Hive Metastore PostgreSQL compatibility issue on extractors #1608
  • Currently, PrestoViewMetadataExtractor is only workable with MySQL connection string. The syntax of the SQL statement is not compatible with Postgres
  • Followed the method in HiveTableMetadataExtractor
    • Added SQL statement for Postgres
    • Added a helper function to choose the SQL statement based on the connection string

Tests

This change has been running in our production environment for a couple of months. I believe no additional unit test is needed since it's simply fixing an unexpected behavior.

CheckList

Make sure you have checked all steps below to ensure a timely review.

  • PR title addresses the issue accurately and concisely. Example: "Updates the version of Flask to v1.0.2"
  • PR includes a summary of changes.
  • PR adds unit tests, updates existing unit tests, OR documents why no test additions or modifications are needed.
  • In case of new functionality, my PR adds documentation that describes how to use it.
    • All the public functions and the classes in the PR contain docstrings that explain what it does

@chonyy chonyy requested a review from a team as a code owner May 28, 2022 18:27
@boring-cyborg boring-cyborg bot added the area:databuilder From databuilder folder label May 28, 2022
@boring-cyborg
Copy link

boring-cyborg bot commented May 28, 2022

Congratulations on your first Pull Request and welcome to Amundsen community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/amundsen-io/amundsen/blob/main/CONTRIBUTING.md)

@chonyy chonyy force-pushed the presto-view-extractor-postgres-compatibility branch from 36506b6 to d810848 Compare May 28, 2022 18:30
@chonyy chonyy changed the title fix: Provide postgres compatibility in presto view metadata extractor fix: Add postgres compatibility in presto view metadata extractor May 28, 2022
@chonyy chonyy changed the title fix: Add postgres compatibility in presto view metadata extractor fix: Add postgres compatibility in PrestoViewMetadataExtractor May 29, 2022
@chonyy
Copy link
Contributor Author

chonyy commented May 30, 2022

Hi @feng-tao @mgorsk1 @dkunitsk , this is a unit test import error that does not relate to my change. This is also seen in other PRs, like here, here and here.
Is it possible for us to get past it or get our PR merged first?

@chonyy chonyy force-pushed the presto-view-extractor-postgres-compatibility branch from d810848 to 71a3191 Compare June 8, 2022 17:33
@feng-tao
Copy link
Member

@chonyy could you update the pr with rebase and fix the ci ? will take a look once it is updated

@chonyy
Copy link
Contributor Author

chonyy commented Jun 18, 2022

@feng-tao Thanks! Will be working on passing the unit tests 😄

@feng-tao
Copy link
Member

feng-tao commented Jul 6, 2022

ping

@chonyy chonyy force-pushed the presto-view-extractor-postgres-compatibility branch from cd68a6c to 1635136 Compare July 12, 2022 03:50
@chonyy chonyy force-pushed the presto-view-extractor-postgres-compatibility branch from 1635136 to 5a8fae1 Compare July 12, 2022 04:08
@chonyy
Copy link
Contributor Author

chonyy commented Jul 12, 2022

Hi @feng-tao , all ci unit tests are passed. Pls help me take a look at it when you have time, thanks 🙏

@stale
Copy link

stale bot commented Jul 30, 2022

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

@stale stale bot added the stale label Jul 30, 2022
@stale stale bot removed the stale label Aug 2, 2022
@feng-tao feng-tao merged commit a9aa1dd into amundsen-io:main Aug 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:databuilder From databuilder folder
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants