Skip to content

[feat](func) Support function json_merge_patch #51683

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open

[feat](func) Support function json_merge_patch #51683

wants to merge 7 commits into from
+287 −11

Conversation

felixwluo
Copy link
Contributor

What problem does this PR solve?

Issue Number: close #51555

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Jun 12, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@felixwluo
Copy link
Contributor Author

run buildall

@felixwluo
Copy link
Contributor Author

added later in the document

@felixwluo
Copy link
Contributor Author

run buildall

@hello-stephen
Copy link
Contributor

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 83.33% (1120/1344)
Line Coverage 66.79% (19308/28909)
Region Coverage 66.57% (9579/14390)
Branch Coverage 56.48% (5205/9216)

@felixwluo
Copy link
Contributor Author

run buildall

@felixwluo felixwluo requested review from eldenmoon, zclllyybb and morrySnow and removed request for eldenmoon, zclllyybb and morrySnow June 13, 2025 06:38
@doris-robot
Copy link

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 83.33% (1120/1344)
Line Coverage 66.84% (19322/28909)
Region Coverage 66.60% (9584/14390)
Branch Coverage 56.50% (5207/9216)

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 0.70% (1/143) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 56.39% (15072/26727)
Line Coverage 45.15% (134766/298512)
Region Coverage 44.26% (67815/153229)
Branch Coverage 38.82% (34800/89646)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 79.02% (113/143) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 79.86% (21006/26304)
Line Coverage 72.77% (217105/298354)
Region Coverage 71.02% (127981/180202)
Branch Coverage 64.67% (66253/102452)

@felixwluo
Copy link
Contributor Author

run buildall

@hello-stephen
Copy link
Contributor

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 83.33% (1120/1344)
Line Coverage 66.81% (19315/28909)
Region Coverage 66.53% (9573/14390)
Branch Coverage 56.51% (5208/9216)

@doris-robot
Copy link

TPC-H: Total hot run time: 33811 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b34fb9222bdb8f0ec5925cbfd6b00da0de145af8, data reload: false

------ Round 1 ----------------------------------
q1	17589	5183	5012	5012
q2	1921	269	178	178
q3	10332	1248	728	728
q4	10305	994	531	531
q5	8469	2353	2306	2306
q6	183	161	131	131
q7	872	734	622	622
q8	9319	1265	1074	1074
q9	6656	4980	5129	4980
q10	6829	2307	1891	1891
q11	480	278	285	278
q12	343	357	218	218
q13	17764	3655	3073	3073
q14	234	237	224	224
q15	562	501	474	474
q16	428	429	366	366
q17	596	848	366	366
q18	7800	7411	7191	7191
q19	1770	948	539	539
q20	323	357	220	220
q21	4202	3211	2413	2413
q22	1083	1082	996	996
Total cold run time: 108060 ms
Total hot run time: 33811 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5331	5196	5057	5057
q2	244	307	225	225
q3	2193	2633	2302	2302
q4	1371	1775	1390	1390
q5	4208	4199	4368	4199
q6	220	174	134	134
q7	2007	1918	1785	1785
q8	2593	2524	2521	2521
q9	7164	7114	7137	7114
q10	3041	3162	2759	2759
q11	582	517	507	507
q12	686	783	603	603
q13	3461	3874	3238	3238
q14	289	303	287	287
q15	517	478	481	478
q16	453	473	462	462
q17	1129	1536	1379	1379
q18	7723	7507	7517	7507
q19	815	839	959	839
q20	1988	2020	1874	1874
q21	4948	4561	4464	4464
q22	1114	1059	1002	1002
Total cold run time: 52077 ms
Total hot run time: 50126 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 193513 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b34fb9222bdb8f0ec5925cbfd6b00da0de145af8, data reload: false

query1	1394	1005	999	999
query2	6178	1823	1802	1802
query3	11149	4647	4595	4595
query4	54263	25571	23455	23455
query5	5258	520	465	465
query6	343	208	201	201
query7	4959	521	301	301
query8	298	241	238	238
query9	6016	2665	2660	2660
query10	478	345	280	280
query11	15240	15104	14865	14865
query12	169	114	112	112
query13	1071	512	415	415
query14	10191	6331	6462	6331
query15	229	195	174	174
query16	7097	682	469	469
query17	1067	751	563	563
query18	1581	409	309	309
query19	198	194	172	172
query20	134	127	119	119
query21	204	128	105	105
query22	4553	4629	4400	4400
query23	33976	33502	33371	33371
query24	6638	2416	2488	2416
query25	501	503	453	453
query26	677	282	158	158
query27	2320	512	353	353
query28	3151	2180	2184	2180
query29	582	552	458	458
query30	273	221	197	197
query31	882	867	758	758
query32	71	70	60	60
query33	474	356	304	304
query34	765	877	542	542
query35	797	813	751	751
query36	941	997	900	900
query37	113	94	85	85
query38	4220	4288	4255	4255
query39	1522	1469	1496	1469
query40	218	126	109	109
query41	62	57	59	57
query42	135	117	117	117
query43	503	517	498	498
query44	1401	856	876	856
query45	176	180	171	171
query46	844	1041	645	645
query47	1834	1916	1816	1816
query48	405	433	318	318
query49	664	506	395	395
query50	653	687	414	414
query51	4318	4218	4191	4191
query52	115	110	113	110
query53	235	283	202	202
query54	606	604	550	550
query55	93	89	86	86
query56	336	333	332	332
query57	1194	1226	1212	1212
query58	291	305	280	280
query59	2657	2706	2695	2695
query60	323	345	326	326
query61	128	142	121	121
query62	759	766	678	678
query63	222	197	196	196
query64	1751	1030	809	809
query65	4342	4148	4126	4126
query66	733	405	304	304
query67	15913	15642	15450	15450
query68	7316	888	543	543
query69	548	307	269	269
query70	1207	1104	1121	1104
query71	500	322	293	293
query72	5966	4831	4851	4831
query73	1378	648	351	351
query74	9030	9062	8965	8965
query75	3813	3183	2669	2669
query76	4204	1194	756	756
query77	614	362	292	292
query78	10054	10365	9355	9355
query79	1964	772	592	592
query80	571	530	453	453
query81	486	257	216	216
query82	470	126	100	100
query83	299	240	236	236
query84	296	109	84	84
query85	809	355	310	310
query86	381	297	308	297
query87	4373	4490	4404	4404
query88	3471	2309	2290	2290
query89	404	327	284	284
query90	1934	212	215	212
query91	213	141	111	111
query92	71	59	62	59
query93	1156	921	601	601
query94	690	408	311	311
query95	371	314	288	288
query96	496	569	281	281
query97	2735	2787	2632	2632
query98	226	221	214	214
query99	1358	1406	1306	1306
Total cold run time: 298288 ms
Total hot run time: 193513 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 28.63 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b34fb9222bdb8f0ec5925cbfd6b00da0de145af8, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.04
query3	0.24	0.07	0.07
query4	1.61	0.10	0.10
query5	0.42	0.46	0.40
query6	1.15	0.66	0.66
query7	0.03	0.02	0.02
query8	0.04	0.03	0.03
query9	0.59	0.52	0.50
query10	0.57	0.57	0.57
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.61	0.60	0.60
query14	0.79	0.79	0.81
query15	0.88	0.84	0.87
query16	0.39	0.37	0.39
query17	1.00	1.04	1.05
query18	0.22	0.21	0.21
query19	1.92	1.85	1.84
query20	0.02	0.01	0.01
query21	15.42	0.91	0.54
query22	0.76	1.30	0.66
query23	14.84	1.38	0.64
query24	6.55	1.38	0.58
query25	0.52	0.23	0.08
query26	0.55	0.16	0.14
query27	0.06	0.05	0.05
query28	9.44	0.91	0.44
query29	12.55	3.96	3.27
query30	0.26	0.09	0.06
query31	2.83	0.59	0.39
query32	3.23	0.54	0.47
query33	3.06	3.12	3.18
query34	15.77	5.16	4.50
query35	4.60	4.54	4.54
query36	0.69	0.52	0.49
query37	0.08	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.03
query40	0.17	0.14	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 102.5 s
Total hot run time: 28.63 s

@hello-stephen
Copy link
Contributor

FE UT Coverage Report

Increment line coverage 8.33% (1/12) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 0.70% (1/143) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 56.40% (15074/26728)
Line Coverage 45.15% (134799/298538)
Region Coverage 44.27% (67841/153239)
Branch Coverage 38.84% (34820/89652)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 79.02% (113/143) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 79.76% (20981/26305)
Line Coverage 72.65% (216786/298380)
Region Coverage 70.87% (127720/180212)
Branch Coverage 64.56% (66146/102458)

@felixwluo
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 0.70% (1/143) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 56.39% (15072/26728)
Line Coverage 45.14% (134757/298538)
Region Coverage 44.26% (67818/153239)
Branch Coverage 38.81% (34795/89652)

@felixwluo
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 83.33% (1120/1344)
Line Coverage 66.82% (19316/28909)
Region Coverage 66.58% (9581/14390)
Branch Coverage 56.49% (5206/9216)

@doris-robot
Copy link

TPC-H: Total hot run time: 36327 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6c2f049ba58a23af423ab85f1d9395637447d374, data reload: false

------ Round 1 ----------------------------------
q1	17620	5419	5032	5032
q2	1967	298	195	195
q3	10266	1341	784	784
q4	10226	1020	566	566
q5	7639	2497	2420	2420
q6	198	168	136	136
q7	969	778	644	644
q8	9350	1349	1209	1209
q9	6922	5231	5244	5231
q10	6914	2410	1972	1972
q11	546	341	330	330
q12	391	385	247	247
q13	17892	3861	3335	3335
q14	269	249	242	242
q15	578	479	499	479
q16	478	467	438	438
q17	711	892	436	436
q18	7931	7330	7312	7312
q19	1642	1011	607	607
q20	399	395	275	275
q21	4674	3999	3385	3385
q22	1164	1072	1052	1052
Total cold run time: 108746 ms
Total hot run time: 36327 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5251	5101	5120	5101
q2	277	337	256	256
q3	2266	2736	2395	2395
q4	1507	1844	1486	1486
q5	4556	4426	4454	4426
q6	233	181	136	136
q7	2061	1955	1857	1857
q8	2669	2803	2593	2593
q9	7390	7375	7125	7125
q10	3115	3264	2805	2805
q11	631	574	537	537
q12	726	838	684	684
q13	3714	4107	3453	3453
q14	311	320	286	286
q15	537	512	493	493
q16	511	574	495	495
q17	1254	1559	1438	1438
q18	7958	7639	7696	7639
q19	967	952	1005	952
q20	2100	2171	1944	1944
q21	5094	4823	4816	4816
q22	1167	1142	1090	1090
Total cold run time: 54295 ms
Total hot run time: 52007 ms

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 0.70% (1/143) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 56.40% (15074/26728)
Line Coverage 45.14% (134774/298550)
Region Coverage 44.26% (67835/153249)
Branch Coverage 38.82% (34808/89660)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 79.02% (113/143) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 79.81% (20995/26305)
Line Coverage 72.78% (217162/298392)
Region Coverage 71.04% (128035/180222)
Branch Coverage 64.75% (66349/102466)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zclllyybb zclllyybb Awaiting requested review from zclllyybb

@eldenmoon eldenmoon Awaiting requested review from eldenmoon

@morrySnow morrySnow Awaiting requested review from morrySnow

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Enhancement] Improve Json function for doris on existing data on json variant/json field
4 participants
@felixwluo @Thearas @hello-stephen @doris-robot