Skip to content

[GH-1868] Fix spark sql extension load failure when parser failed to load #1872

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Mar 20, 2025

Conversation

zhangfengcdt
Copy link
Member

Did you read the Contributor Guide?

Is this PR related to a ticket?

  • Yes, and the PR name follows the format [GH-XXX] my subject.

What changes were proposed in this PR?

This PR try to inject the Sedona SQL parser but gracefully handle initialization failures.
This prevents extension loading errors from causing the SparkSession initialization to fail, allowing the application to continue running without the Sedona parser extension.

How was this patch tested?

SQL unit tests

Did this PR include necessary documentation updates?

  • No, this PR does not affect any public API so no need to change the documentation.

@zhangfengcdt zhangfengcdt marked this pull request as ready for review March 20, 2025 12:31
@zhangfengcdt zhangfengcdt requested a review from jiayuasu as a code owner March 20, 2025 12:31
@zhangfengcdt
Copy link
Member Author

Tested before and after fix on DBR 16.2 ML (includes Apache Spark 3.5.2, Scala 2.12)

Screenshot 2025-03-20 at 10 01 40 AM

@jiayuasu jiayuasu added this to the sedona-1.8.0 milestone Mar 20, 2025
@jiayuasu jiayuasu added the bug label Mar 20, 2025
@jiayuasu jiayuasu merged commit f99332e into apache:master Mar 20, 2025
39 checks passed
@jiayuasu jiayuasu linked an issue Mar 20, 2025 that may be closed by this pull request
jiayuasu pushed a commit that referenced this pull request May 30, 2025
…load (#1872)

* [SEDONA-636] Fix spark sql extension load failure when parser failed to load

* fix exception capture issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Databricks Error SparkSqlParser method void init not found
2 participants