Skip to content

[fix](mtmv) Add null-safety to getBaseViewsOneLevel for backward compatibility#64412

Open
yujun777 wants to merge 5 commits into
apache:masterfrom
yujun777:fix-view-dependency-npe
Open

[fix](mtmv) Add null-safety to getBaseViewsOneLevel for backward compatibility#64412
yujun777 wants to merge 5 commits into
apache:masterfrom
yujun777:fix-view-dependency-npe

Conversation

@yujun777

@yujun777 yujun777 commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Introduced by #56423, the field baseViewsOneLevel may be null for MTMVs created before that PR. This causes NPE when querying information_schema.view_dependency:

Cannot invoke java.util.Set.stream() because the return value of org.apache.doris.mtmv.MTMVRelation.getBaseViewsOneLevel() is null

baseViewsOneLevel is currently only used for tablefunction/MetadataGenerator, so this change is safe.

…atibility

getBaseViewsOneLevel() could return null when an MTMV was created before the
baseViewsOneLevel field was introduced, causing NPE when querying
information_schema.view_dependency via Stream.concat().

Key changes:
- getBaseViewsOneLevel() now falls back to baseViews when baseViewsOneLevel is null, consistent with getBaseTablesOneLevel()

Unit Test:
- MTMVRelationTest.testMTMVRelation
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@yujun777

Copy link
Copy Markdown
Contributor Author

run buildall

public Set<BaseTableInfo> getBaseViewsOneLevel() {
return baseViewsOneLevel;
// For compatibility, previously created MTMV may not have baseViewsOneLevel
return baseViewsOneLevel == null ? baseViews : baseViewsOneLevel;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add a gsonpostprocessor assign empty set to baseViewsOneLevel if it is null?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个文件中前面两个函数都是用一样的写法的。 保持一致的风格 ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 29273 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 85d7f56e906a3b76400ca6d2b127cb83d95bcbcd, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17797	4238	4245	4238
q2	q3	10771	1481	801	801
q4	4682	470	337	337
q5	7522	874	577	577
q6	185	172	134	134
q7	800	834	636	636
q8	9433	1448	1486	1448
q9	5793	4500	4497	4497
q10	6761	1814	1540	1540
q11	447	272	255	255
q12	630	411	292	292
q13	18116	3416	2816	2816
q14	269	256	243	243
q15	q16	814	776	711	711
q17	999	971	1019	971
q18	6953	5699	5615	5615
q19	1347	1280	1113	1113
q20	525	403	262	262
q21	6078	2613	2490	2490
q22	438	352	297	297
Total cold run time: 100360 ms
Total hot run time: 29273 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4450	4314	4342	4314
q2	q3	4556	4964	4351	4351
q4	2089	2179	1394	1394
q5	4463	4364	4331	4331
q6	251	172	125	125
q7	1733	1720	1800	1720
q8	2702	2200	2234	2200
q9	8157	8366	8096	8096
q10	4832	4810	4310	4310
q11	608	427	396	396
q12	755	759	539	539
q13	3318	3653	2973	2973
q14	309	291	284	284
q15	q16	722	731	672	672
q17	1365	1378	1310	1310
q18	8011	7341	7162	7162
q19	1161	1130	1066	1066
q20	2222	2226	1956	1956
q21	5274	4575	4413	4413
q22	543	444	423	423
Total cold run time: 57521 ms
Total hot run time: 52035 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 168946 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 85d7f56e906a3b76400ca6d2b127cb83d95bcbcd, data reload: false

query5	4328	638	474	474
query6	475	192	177	177
query7	4809	582	301	301
query8	353	214	197	197
query9	8800	4035	4060	4035
query10	430	322	262	262
query11	5847	2367	2148	2148
query12	158	102	101	101
query13	1260	636	414	414
query14	6322	5377	4967	4967
query14_1	4385	4398	4386	4386
query15	206	196	171	171
query16	1031	468	447	447
query17	1100	698	549	549
query18	2425	473	331	331
query19	198	178	139	139
query20	116	108	109	108
query21	220	146	126	126
query22	13743	13644	13382	13382
query23	17444	16464	16314	16314
query23_1	16217	16216	16350	16216
query24	7571	1816	1301	1301
query24_1	1346	1317	1343	1317
query25	588	472	392	392
query26	1303	337	176	176
query27	2684	584	327	327
query28	4497	2032	2033	2032
query29	1108	610	502	502
query30	311	246	205	205
query31	1140	1082	960	960
query32	110	74	62	62
query33	533	326	266	266
query34	1188	1191	653	653
query35	747	774	694	694
query36	1364	1411	1219	1219
query37	174	108	94	94
query38	3221	3138	3041	3041
query39	925	907	906	906
query39_1	893	895	876	876
query40	223	127	107	107
query41	70	66	67	66
query42	98	98	96	96
query43	342	330	289	289
query44	
query45	203	187	180	180
query46	1066	1233	708	708
query47	2374	2331	2228	2228
query48	415	427	310	310
query49	642	490	365	365
query50	1058	373	273	273
query51	4361	4326	4275	4275
query52	90	94	80	80
query53	245	284	195	195
query54	291	243	215	215
query55	81	81	73	73
query56	251	238	228	228
query57	1426	1401	1300	1300
query58	263	216	223	216
query59	1658	1685	1482	1482
query60	306	258	278	258
query61	156	150	157	150
query62	690	644	589	589
query63	234	190	187	187
query64	2509	756	608	608
query65	
query66	1834	447	335	335
query67	29695	29651	29547	29547
query68	
query69	436	302	264	264
query70	1004	1022	919	919
query71	302	225	214	214
query72	2954	2696	2384	2384
query73	830	780	435	435
query74	5129	4994	4771	4771
query75	2637	2577	2255	2255
query76	2288	1168	824	824
query77	364	378	294	294
query78	12434	12418	11800	11800
query79	1480	1084	768	768
query80	590	465	399	399
query81	453	282	244	244
query82	593	153	118	118
query83	356	273	248	248
query84	
query85	848	493	421	421
query86	373	312	287	287
query87	3379	3428	3157	3157
query88	3669	2766	2771	2766
query89	428	384	327	327
query90	1945	186	185	185
query91	174	160	134	134
query92	62	62	53	53
query93	1425	1393	922	922
query94	522	356	332	332
query95	679	372	353	353
query96	1034	816	362	362
query97	2700	2684	2532	2532
query98	214	206	199	199
query99	1158	1163	1013	1013
Total cold run time: 251070 ms
Total hot run time: 168946 ms

yujun777 and others added 2 commits June 11, 2026 18:15
Move the null-safety fallback from getBaseViewsOneLevel() to gsonPostProcess(), so the compatibility fix is applied once at deserialization time rather than on every getter call.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@yujun777

Copy link
Copy Markdown
Contributor Author

run buildall

yujun777 and others added 2 commits June 11, 2026 18:24
Avoid shared reference between baseViews and baseViewsOneLevel by creating a new HashSet copy.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fall back to empty set instead of null when baseViews is also null.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@yujun777

Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 29165 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4103dd33e30b970c9ed2659838aae5dd16d155f2, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17622	4045	4092	4045
q2	q3	10795	1359	793	793
q4	4684	469	342	342
q5	7549	887	587	587
q6	184	175	136	136
q7	784	836	625	625
q8	9333	1548	1587	1548
q9	5877	4491	4459	4459
q10	6789	1793	1538	1538
q11	430	271	263	263
q12	627	426	299	299
q13	18165	3353	2783	2783
q14	262	259	236	236
q15	q16	819	771	699	699
q17	947	924	937	924
q18	6978	5577	5644	5577
q19	1300	1241	1103	1103
q20	492	402	258	258
q21	6272	2824	2616	2616
q22	461	373	334	334
Total cold run time: 100370 ms
Total hot run time: 29165 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	5062	4739	4721	4721
q2	q3	4862	5208	4698	4698
q4	2088	2184	1400	1400
q5	4928	4625	4577	4577
q6	232	184	125	125
q7	1951	1727	1595	1595
q8	2444	2059	2081	2059
q9	7773	7331	7354	7331
q10	4721	4698	4216	4216
q11	519	379	349	349
q12	729	735	530	530
q13	3063	3425	2749	2749
q14	269	288	244	244
q15	q16	682	694	613	613
q17	1279	1245	1245	1245
q18	7190	6625	6780	6625
q19	1092	1101	1084	1084
q20	2229	2192	1947	1947
q21	5233	4539	4414	4414
q22	523	441	419	419
Total cold run time: 56869 ms
Total hot run time: 50941 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 28985 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4103dd33e30b970c9ed2659838aae5dd16d155f2, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17759	4063	4018	4018
q2	q3	10832	1421	794	794
q4	4678	476	339	339
q5	7503	867	611	611
q6	176	171	136	136
q7	778	836	611	611
q8	9353	1566	1540	1540
q9	5759	4506	4495	4495
q10	6753	1783	1523	1523
q11	427	271	259	259
q12	634	430	287	287
q13	18102	3339	2773	2773
q14	263	260	245	245
q15	q16	820	778	703	703
q17	984	975	934	934
q18	7000	5823	5630	5630
q19	1334	1337	1083	1083
q20	494	406	262	262
q21	5793	2575	2438	2438
q22	433	362	304	304
Total cold run time: 99875 ms
Total hot run time: 28985 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4329	4265	4265	4265
q2	q3	4506	4986	4335	4335
q4	2073	2177	1358	1358
q5	4486	4287	4322	4287
q6	229	176	130	130
q7	1724	1597	1481	1481
q8	3026	2211	2166	2166
q9	8173	8140	7881	7881
q10	4797	4792	4256	4256
q11	591	416	382	382
q12	742	752	567	567
q13	3306	3639	3057	3057
q14	306	304	284	284
q15	q16	723	721	625	625
q17	1357	1307	1313	1307
q18	8327	7318	7381	7318
q19	1176	1135	1090	1090
q20	2193	2222	1953	1953
q21	5243	4495	4420	4420
q22	505	457	409	409
Total cold run time: 57812 ms
Total hot run time: 51571 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 169321 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4103dd33e30b970c9ed2659838aae5dd16d155f2, data reload: false

query5	4316	616	451	451
query6	431	188	171	171
query7	4800	565	281	281
query8	360	209	201	201
query9	8754	4009	4049	4009
query10	462	323	254	254
query11	5932	2339	2114	2114
query12	154	101	94	94
query13	1247	607	421	421
query14	6404	5349	5011	5011
query14_1	4354	4330	4358	4330
query15	203	191	176	176
query16	982	463	451	451
query17	1090	675	564	564
query18	2458	456	334	334
query19	193	181	136	136
query20	109	104	100	100
query21	204	133	118	118
query22	13657	13540	13480	13480
query23	17316	16455	16119	16119
query23_1	16185	16240	16192	16192
query24	7487	1793	1319	1319
query24_1	1329	1313	1307	1307
query25	576	471	392	392
query26	1310	337	173	173
query27	2707	595	338	338
query28	4476	2066	2035	2035
query29	1109	613	496	496
query30	313	238	196	196
query31	1114	1078	950	950
query32	113	63	60	60
query33	522	329	261	261
query34	1180	1163	659	659
query35	767	795	679	679
query36	1408	1396	1199	1199
query37	156	107	92	92
query38	3202	3141	3040	3040
query39	921	930	929	929
query39_1	893	869	860	860
query40	224	129	106	106
query41	69	66	67	66
query42	95	95	94	94
query43	320	317	281	281
query44	
query45	203	186	187	186
query46	1096	1229	751	751
query47	2361	2347	2204	2204
query48	409	421	296	296
query49	633	473	397	397
query50	1005	347	249	249
query51	4295	4253	4305	4253
query52	88	89	79	79
query53	251	270	192	192
query54	287	228	223	223
query55	81	77	70	70
query56	263	239	225	225
query57	1448	1394	1347	1347
query58	258	228	222	222
query59	1559	1636	1429	1429
query60	292	253	237	237
query61	183	189	147	147
query62	711	639	584	584
query63	231	182	182	182
query64	2544	749	618	618
query65	
query66	1795	459	337	337
query67	29670	29612	29504	29504
query68	
query69	446	297	265	265
query70	1004	968	958	958
query71	300	222	213	213
query72	2962	2622	2437	2437
query73	869	769	454	454
query74	5160	4921	4751	4751
query75	2634	2551	2248	2248
query76	2334	1141	805	805
query77	344	390	274	274
query78	12491	12378	11799	11799
query79	1464	1066	788	788
query80	1267	454	397	397
query81	494	277	254	254
query82	583	155	123	123
query83	355	272	247	247
query84	
query85	904	507	418	418
query86	415	287	283	283
query87	3416	3351	3157	3157
query88	3680	2764	2775	2764
query89	428	393	332	332
query90	1895	184	186	184
query91	169	156	132	132
query92	60	64	52	52
query93	1459	1417	922	922
query94	696	353	311	311
query95	672	383	347	347
query96	1123	768	342	342
query97	2680	2692	2564	2564
query98	213	206	201	201
query99	1138	1166	1018	1018
Total cold run time: 251593 ms
Total hot run time: 169321 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 168542 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4103dd33e30b970c9ed2659838aae5dd16d155f2, data reload: false

query5	4295	647	476	476
query6	445	209	169	169
query7	4823	547	305	305
query8	361	221	194	194
query9	8720	3974	3986	3974
query10	439	302	278	278
query11	5915	2350	2118	2118
query12	161	101	99	99
query13	1252	598	424	424
query14	6395	5364	5036	5036
query14_1	4393	4341	4309	4309
query15	196	196	167	167
query16	955	444	408	408
query17	907	682	556	556
query18	2427	473	335	335
query19	194	174	138	138
query20	114	106	111	106
query21	213	135	115	115
query22	13744	13602	13449	13449
query23	17285	16356	16164	16164
query23_1	16223	16297	16192	16192
query24	7605	1757	1297	1297
query24_1	1330	1304	1280	1280
query25	532	423	366	366
query26	1311	313	165	165
query27	2691	541	330	330
query28	4471	2022	2023	2022
query29	1083	626	489	489
query30	316	244	204	204
query31	1104	1093	958	958
query32	109	63	60	60
query33	549	326	272	272
query34	1179	1112	661	661
query35	742	785	680	680
query36	1379	1380	1235	1235
query37	152	103	91	91
query38	3195	3137	3043	3043
query39	915	936	893	893
query39_1	885	865	867	865
query40	224	143	105	105
query41	74	71	77	71
query42	96	99	99	99
query43	316	318	279	279
query44	
query45	196	189	177	177
query46	1081	1231	723	723
query47	2425	2419	2243	2243
query48	409	421	315	315
query49	634	471	366	366
query50	1007	347	256	256
query51	4268	4345	4250	4250
query52	89	88	79	79
query53	252	276	197	197
query54	290	232	212	212
query55	81	81	71	71
query56	252	228	219	219
query57	1451	1441	1333	1333
query58	251	228	218	218
query59	1539	1612	1388	1388
query60	294	260	239	239
query61	183	179	174	174
query62	712	655	595	595
query63	232	189	186	186
query64	2584	761	591	591
query65	
query66	1817	459	339	339
query67	29653	29608	29484	29484
query68	
query69	445	304	259	259
query70	980	942	917	917
query71	314	217	215	215
query72	2964	2590	2430	2430
query73	823	799	424	424
query74	5100	4904	4743	4743
query75	2619	2546	2223	2223
query76	2351	1135	781	781
query77	347	370	273	273
query78	12318	12609	11905	11905
query79	1445	998	764	764
query80	605	504	386	386
query81	438	278	237	237
query82	568	156	119	119
query83	345	269	245	245
query84	
query85	861	522	413	413
query86	344	297	283	283
query87	3387	3310	3209	3209
query88	3592	2711	2704	2704
query89	421	372	328	328
query90	1963	182	172	172
query91	170	157	133	133
query92	60	61	57	57
query93	1462	1540	847	847
query94	547	357	304	304
query95	672	440	334	334
query96	1053	795	337	337
query97	2710	2720	2565	2565
query98	209	203	197	197
query99	1134	1205	995	995
Total cold run time: 250085 ms
Total hot run time: 168542 ms

@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 0.00% (0/3) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 33.33% (1/3) 🎉
Increment coverage report
Complete coverage report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants