[Enhancement] invalidate table cache after hive table do not need to refresh by ConnectorTableMetadataProcessor #37549

Youngwb · 2023-12-21T09:03:09Z

Why I'm doing:
Hive table support refresh meta cache automatically，it will stop refresh the table meta cahce which has long time not query on it，but the table cache still exist, it could be outdated
What I'm doing:
when the table cache do not need to refresh, invalidate the table meta cache
Fixes #issue

What type of PR is this:

Does this PR entail a change in behavior?

Yes, this PR will result in a change in behavior.
No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

Interface/UI changes: syntax, type conversion, expression evaluation, display information
Parameter changes: default values, similar parameters but with different default values
Policy changes: use new policy to replace old one, functionality automatically enabled
Feature removed
Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

I have added test cases for my bug fix or my new feature
This pr needs user documentation (for new or modified features or behaviors)
- I have added documentation for my new feature or new function
This is a backport pr

Bugfix cherry-pick branch check:

…refresh by ConnectorTableMetadataProcessor Signed-off-by: Youngwb <[email protected]>

sonarqubecloud · 2023-12-21T09:11:46Z

Quality Gate passed

The SonarCloud Quality Gate passed, but some issues were introduced.

1 New issue
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

github-actions · 2023-12-21T10:33:05Z

[FE Incremental Coverage Report]

✅ pass : 6 / 6 (100.00%)

file detail

	path	covered_line	new_line	coverage	not_covered_line_detail
🔵	com/starrocks/connector/hive/CachingHiveMetastore.java	6	6	100.00%	[]

github-actions · 2023-12-21T10:34:14Z

[BE Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

github-actions · 2023-12-22T06:12:40Z

@Mergifyio backport branch-3.2

github-actions · 2023-12-22T06:12:42Z

@Mergifyio backport branch-3.1

github-actions · 2023-12-22T06:12:44Z

@Mergifyio backport branch-3.0

github-actions · 2023-12-22T06:12:45Z

@Mergifyio backport branch-2.5

mergify · 2023-12-22T06:12:45Z

backport branch-3.2

✅ Backports have been created

#37629 [Enhancement] invalidate table cache after hive table do not need to refresh by ConnectorTableMetadataProcessor (backport #37549) has been created for branch branch-3.2 but encountered conflicts

mergify · 2023-12-22T06:12:47Z

backport branch-3.1

✅ Backports have been created

#37630 [Enhancement] invalidate table cache after hive table do not need to refresh by ConnectorTableMetadataProcessor (backport #37549) has been created for branch branch-3.1 but encountered conflicts

mergify · 2023-12-22T06:12:50Z

backport branch-3.0

✅ Backports have been created

#37631 [Enhancement] invalidate table cache after hive table do not need to refresh by ConnectorTableMetadataProcessor (backport #37549) has been created for branch branch-3.0 but encountered conflicts

mergify · 2023-12-22T06:12:52Z

backport branch-2.5

✅ Backports have been created

#37632 [Enhancement] invalidate table cache after hive table do not need to refresh by ConnectorTableMetadataProcessor (backport #37549) has been created for branch branch-2.5 but encountered conflicts

…refresh by ConnectorTableMetadataProcessor (#37549) Signed-off-by: Youngwb <[email protected]> (cherry picked from commit d24c364) # Conflicts: # fe/fe-core/src/test/java/com/starrocks/connector/hive/CachingHiveMetastoreTest.java

…er hive table do not need to refresh by ConnectorTableMetadataProcessor (#37635) Signed-off-by: Youngwb <[email protected]>

…er hive table do not need to refresh by ConnectorTableMetadataProcessor (#37635) Signed-off-by: Youngwb <[email protected]> (cherry picked from commit bca983c) # Conflicts: # fe/fe-core/src/test/java/com/starrocks/connector/hive/CachingHiveMetastoreTest.java

…er hive table do not need to refresh by ConnectorTableMetadataProcessor (#37656) Signed-off-by: Youngwb <[email protected]>

…er hive table do not need to refresh by ConnectorTableMetadataProcessor (#37666) Signed-off-by: Youngwb <[email protected]>

…er hive table do not need to refresh by ConnectorTableMetadataProcessor (#37668) Signed-off-by: Youngwb <[email protected]>

…refresh by ConnectorTableMetadataProcessor (StarRocks#37549) Signed-off-by: Youngwb <[email protected]> Signed-off-by: 张敢 <[email protected]>

[Enhancement] invalidate table cache after hive table do not need to …

b36de69

…refresh by ConnectorTableMetadataProcessor Signed-off-by: Youngwb <[email protected]>

github-actions bot added the 3.2 label Dec 21, 2023

mergify bot assigned Youngwb Dec 21, 2023

github-actions bot added 3.1 3.0 2.5 labels Dec 21, 2023

stephen-shelby approved these changes Dec 22, 2023

View reviewed changes

dirtysalt approved these changes Dec 22, 2023

View reviewed changes

Youngwb merged commit d24c364 into StarRocks:main Dec 22, 2023

Youngwb deleted the invalidate_cache branch December 22, 2023 06:12

github-actions bot removed the 3.2 label Dec 22, 2023

github-actions bot removed the 3.1 label Dec 22, 2023

github-actions bot removed the 3.0 label Dec 22, 2023

github-actions bot removed the 2.5 label Dec 22, 2023

mergify bot mentioned this pull request Dec 22, 2023

[Enhancement] invalidate table cache after hive table do not need to refresh by ConnectorTableMetadataProcessor (backport #37549) #37629

Closed

18 tasks

mergify bot mentioned this pull request Dec 22, 2023

[Enhancement] invalidate table cache after hive table do not need to refresh by ConnectorTableMetadataProcessor (backport #37549) #37630

Closed

18 tasks

mergify bot mentioned this pull request Dec 22, 2023

[Enhancement] invalidate table cache after hive table do not need to refresh by ConnectorTableMetadataProcessor (backport #37549) #37631

Closed

18 tasks

mergify bot mentioned this pull request Dec 22, 2023

[Enhancement] invalidate table cache after hive table do not need to refresh by ConnectorTableMetadataProcessor (backport #37549) #37632

Closed

18 tasks

Youngwb added a commit that referenced this pull request Dec 22, 2023

[backport #37549][branch-3.2][Enhancement] invalidate table cache aft…

bca983c

…er hive table do not need to refresh by ConnectorTableMetadataProcessor (#37635) Signed-off-by: Youngwb <[email protected]>

Youngwb added a commit that referenced this pull request Dec 22, 2023

[backport #37549][branch-3.1][Enhancement] invalidate table cache aft…

2e0ae7f

…er hive table do not need to refresh by ConnectorTableMetadataProcessor (#37656) Signed-off-by: Youngwb <[email protected]>

Youngwb added a commit that referenced this pull request Dec 22, 2023

[backport #37549][branch-3.0][Enhancement] invalidate table cache aft…

0045f9d

…er hive table do not need to refresh by ConnectorTableMetadataProcessor (#37666) Signed-off-by: Youngwb <[email protected]>

Youngwb added a commit that referenced this pull request Dec 22, 2023

[backport #37549][branch-2.5][Enhancement] invalidate table cache aft…

bd27d24

…er hive table do not need to refresh by ConnectorTableMetadataProcessor (#37668) Signed-off-by: Youngwb <[email protected]>

wangruin added the backport_ok label Jan 9, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Enhancement] invalidate table cache after hive table do not need to refresh by ConnectorTableMetadataProcessor #37549

[Enhancement] invalidate table cache after hive table do not need to refresh by ConnectorTableMetadataProcessor #37549

Uh oh!

Youngwb commented Dec 21, 2023

Uh oh!

sonarqubecloud bot commented Dec 21, 2023

Uh oh!

github-actions bot commented Dec 21, 2023

Uh oh!

github-actions bot commented Dec 21, 2023

Uh oh!

github-actions bot commented Dec 22, 2023

Uh oh!

github-actions bot commented Dec 22, 2023

Uh oh!

github-actions bot commented Dec 22, 2023

Uh oh!

github-actions bot commented Dec 22, 2023

Uh oh!

mergify bot commented Dec 22, 2023 •

edited

Loading

Uh oh!

mergify bot commented Dec 22, 2023 •

edited

Loading

Uh oh!

mergify bot commented Dec 22, 2023 •

edited

Loading

Uh oh!

mergify bot commented Dec 22, 2023 •

edited

Loading

Uh oh!

Uh oh!

[Enhancement] invalidate table cache after hive table do not need to refresh by ConnectorTableMetadataProcessor #37549

[Enhancement] invalidate table cache after hive table do not need to refresh by ConnectorTableMetadataProcessor #37549

Uh oh!

Conversation

Youngwb commented Dec 21, 2023

What type of PR is this:

Checklist:

Bugfix cherry-pick branch check:

Uh oh!

sonarqubecloud bot commented Dec 21, 2023

Quality Gate passed

Uh oh!

github-actions bot commented Dec 21, 2023

[FE Incremental Coverage Report]

file detail

Uh oh!

github-actions bot commented Dec 21, 2023

[BE Incremental Coverage Report]

Uh oh!

github-actions bot commented Dec 22, 2023

Uh oh!

github-actions bot commented Dec 22, 2023

Uh oh!

github-actions bot commented Dec 22, 2023

Uh oh!

github-actions bot commented Dec 22, 2023

Uh oh!

mergify bot commented Dec 22, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Backports have been created

Uh oh!

mergify bot commented Dec 22, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Backports have been created

Uh oh!

mergify bot commented Dec 22, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Backports have been created

Uh oh!

mergify bot commented Dec 22, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Backports have been created

Uh oh!

Uh oh!

mergify bot commented Dec 22, 2023 •

edited

Loading

mergify bot commented Dec 22, 2023 •

edited

Loading

mergify bot commented Dec 22, 2023 •

edited

Loading

mergify bot commented Dec 22, 2023 •

edited

Loading