Skip to content

branch-3.0: [fix](jdbc) make sure init the jdbc client before using it #51540

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: branch-3.0
Choose a base branch
from

Conversation

morningman
Copy link
Contributor

This may cause some statistic collection task fail:

2025-06-05 15:02:32,277 WARN (Statistics Job Appender|182) [ExternalCatalog.buildDbForInit():902] Failed to check db DORIS_TEST exist in remote system, ignore it.
java.lang.NullPointerException: Cannot invoke "org.apache.doris.datasource.jdbc.client.JdbcClient.getDatabaseNameList()" because "this.jdbcClient" is null
        at org.apache.doris.datasource.jdbc.JdbcExternalCatalog.listDatabaseNames(JdbcExternalCatalog.java:267) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.datasource.ExternalCatalog.getFilteredDatabaseNames(ExternalCatalog.java:476) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.datasource.ExternalCatalog.lambda$makeSureInitialized$0(ExternalCatalog.java:318) ~[doris-fe.jar:1.2-SNAPSHOT]
        at com.github.benmanes.caffeine.cache.LocalLoadingCache.lambda$newMappingFunction$2(LocalLoadingCache.java:145) ~[hive-catalog-shade-2.1.4.jar:2.1.4]
        at com.github.benmanes.caffeine.cache.LocalCache.lambda$statsAware$0(LocalCache.java:139) ~[hive-catalog-shade-2.1.4.jar:2.1.4]
        at com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$doComputeIfAbsent$14(BoundedLocalCache.java:2406) ~[hive-catalog-shade-2.1.4.jar:2.1.4]
        at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1916) ~[?:?]

Only for branch-3.0.
Fix master in #51471

@morningman morningman requested a review from dataroaring as a code owner June 6, 2025 02:34
@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morningman
Copy link
Contributor Author

run buildall

@morningman morningman changed the title branch-3.0: make sure init the jdbc client before using it branch-3.0: [fix](jdbc) make sure init the jdbc client before using it Jun 6, 2025
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 6, 2025
Copy link
Contributor

github-actions bot commented Jun 6, 2025

PR approved by at least one committer and no changes requested.

Copy link
Contributor

github-actions bot commented Jun 6, 2025

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 39612 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0c6dcd10bdc9ea7613b9af8cfc2313ded6aaeb2c, data reload: false

------ Round 1 ----------------------------------
q1	17574	6695	6534	6534
q2	2060	167	174	167
q3	10755	1094	1112	1094
q4	10543	793	740	740
q5	7751	2864	2876	2864
q6	215	133	132	132
q7	970	621	615	615
q8	9370	1922	2010	1922
q9	6547	6385	6391	6385
q10	6985	2268	2290	2268
q11	462	256	266	256
q12	402	214	214	214
q13	17805	2985	3022	2985
q14	232	199	208	199
q15	513	453	462	453
q16	497	385	380	380
q17	979	518	592	518
q18	7159	6617	6596	6596
q19	1394	1096	1012	1012
q20	469	211	195	195
q21	3987	3331	3128	3128
q22	1078	1004	955	955
Total cold run time: 107747 ms
Total hot run time: 39612 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6649	6540	6653	6540
q2	330	241	240	240
q3	3047	2804	2861	2804
q4	2104	1793	1885	1793
q5	5901	5833	5840	5833
q6	218	131	135	131
q7	2235	1802	1821	1802
q8	3370	3546	3513	3513
q9	9035	8803	8913	8803
q10	3547	3501	3513	3501
q11	601	510	482	482
q12	788	590	610	590
q13	10665	3218	3179	3179
q14	307	264	274	264
q15	520	469	477	469
q16	509	453	435	435
q17	1832	1624	1618	1618
q18	8254	7787	7662	7662
q19	1692	1471	1570	1471
q20	2150	1821	1799	1799
q21	5306	5401	5309	5309
q22	1146	1022	1040	1022
Total cold run time: 70206 ms
Total hot run time: 59260 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196323 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0c6dcd10bdc9ea7613b9af8cfc2313ded6aaeb2c, data reload: false

query1	1276	898	917	898
query2	6344	1958	1873	1873
query3	10790	4332	4223	4223
query4	60979	28970	23528	23528
query5	5281	475	457	457
query6	405	198	196	196
query7	5430	334	319	319
query8	309	227	235	227
query9	8284	2604	2583	2583
query10	455	299	260	260
query11	18120	15165	15717	15165
query12	172	103	106	103
query13	1461	461	468	461
query14	10403	6619	7387	6619
query15	197	181	173	173
query16	7186	500	490	490
query17	1118	574	576	574
query18	1893	309	313	309
query19	211	158	151	151
query20	114	112	108	108
query21	202	101	101	101
query22	4805	4447	4556	4447
query23	35311	33797	34298	33797
query24	6146	2909	2919	2909
query25	556	442	413	413
query26	656	169	168	168
query27	1792	368	358	358
query28	4083	2234	2182	2182
query29	697	472	457	457
query30	244	155	164	155
query31	994	811	836	811
query32	71	58	59	58
query33	487	288	305	288
query34	912	529	538	529
query35	862	745	727	727
query36	1078	962	975	962
query37	109	70	73	70
query38	4069	3948	3932	3932
query39	1516	1470	1465	1465
query40	210	110	103	103
query41	52	51	56	51
query42	120	105	113	105
query43	532	491	506	491
query44	1185	820	812	812
query45	185	166	176	166
query46	1145	737	731	731
query47	2041	1928	1964	1928
query48	468	387	384	384
query49	716	401	385	385
query50	826	415	438	415
query51	7491	7306	7141	7141
query52	105	90	93	90
query53	271	189	189	189
query54	578	458	480	458
query55	83	81	82	81
query56	285	283	259	259
query57	1254	1144	1146	1144
query58	223	203	218	203
query59	3205	3057	3122	3057
query60	282	253	261	253
query61	132	112	110	110
query62	782	695	693	693
query63	233	251	202	202
query64	1416	682	647	647
query65	3259	3187	3173	3173
query66	659	302	301	301
query67	15822	15827	15673	15673
query68	4229	606	586	586
query69	415	272	275	272
query70	1161	1097	1127	1097
query71	336	276	259	259
query72	6107	4073	4013	4013
query73	750	348	360	348
query74	10351	9131	9245	9131
query75	3273	2655	2674	2655
query76	2040	1052	1030	1030
query77	507	302	278	278
query78	10646	9547	9559	9547
query79	2075	614	597	597
query80	1307	424	461	424
query81	511	232	221	221
query82	578	90	87	87
query83	158	148	139	139
query84	289	81	73	73
query85	970	309	294	294
query86	352	288	297	288
query87	4406	4220	4273	4220
query88	4001	2365	2348	2348
query89	425	300	288	288
query90	1985	187	186	186
query91	181	151	151	151
query92	67	47	50	47
query93	2603	561	551	551
query94	716	279	311	279
query95	353	256	259	256
query96	617	276	277	276
query97	3329	3121	3169	3121
query98	208	200	197	197
query99	1573	1273	1310	1273
Total cold run time: 315097 ms
Total hot run time: 196323 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.2 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0c6dcd10bdc9ea7613b9af8cfc2313ded6aaeb2c, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.04	0.03
query3	0.23	0.07	0.06
query4	1.62	0.11	0.10
query5	0.52	0.52	0.51
query6	1.15	0.74	0.72
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.57	0.50	0.50
query10	0.55	0.55	0.57
query11	0.14	0.10	0.11
query12	0.14	0.11	0.11
query13	0.61	0.60	0.60
query14	0.80	0.80	0.81
query15	0.84	0.83	0.84
query16	0.38	0.39	0.40
query17	1.01	1.02	1.02
query18	0.24	0.23	0.22
query19	1.82	1.73	1.86
query20	0.01	0.01	0.01
query21	15.46	0.61	0.60
query22	2.69	2.64	1.71
query23	17.02	0.98	0.79
query24	3.08	1.97	1.22
query25	0.35	0.08	0.12
query26	0.49	0.13	0.16
query27	0.05	0.04	0.05
query28	9.32	0.48	0.50
query29	12.91	3.22	3.22
query30	0.24	0.07	0.06
query31	2.87	0.37	0.39
query32	3.27	0.46	0.46
query33	2.98	3.00	2.99
query34	17.14	4.44	4.54
query35	4.48	4.52	4.48
query36	0.66	0.50	0.48
query37	0.09	0.05	0.05
query38	0.05	0.04	0.04
query39	0.04	0.03	0.02
query40	0.17	0.12	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 104.29 s
Total hot run time: 30.2 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.x p0_test reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants