Text chunkedForest benchmark testing

[brrichards]

Summary

Memory and time benchmark testing with plain and formatted text using objectForest and chuckedForest.

Memory Benchmarks (Initialization — Heap Used Avg, bytes)

Plain Text

Chars	ObjectForest	ChunkedForest
10	80,953	65,073
100	143,098	169,072
1,000	688,212	1,278,364

Formatted Text

Chars	ObjectForest	ChunkedForest
5	168,389	136,705
50	1,041,163	779,327
100	2,084,641	1,220,757

Summary Size (bytes)

Plain Text

Chars	OF baseline	CF baseline	CF incremental
100	4,666	4,666	4,796
1,000	15,466	15,466	15,596
10,000	123,466	123,466	123,596

Formatted Text

Chars	OF baseline	CF baseline	CF incremental
5	15,220	15,220	16,098
50	18,730	18,730	19,608
100	22,630	22,630	23,508

Summary Generation Time (ns/op)

Cold (first summary)

Plain Text

Chars	OF baseline	CF baseline	CF incremental
100	9,592,303	9,676,908	9,884,833
1,000	41,807,400	43,353,879	49,650,869
10,000	430,683,434	374,355,334	400,863,421

Formatted Text

Chars	OF baseline	CF baseline	CF incremental
5	15,775,944	14,381,956	15,250,266
50	70,376,890	49,045,693	54,111,759
100	132,279,570	85,964,392	96,286,260

After Small Edit (re-summarization)

Plain Text

Chars	OF baseline	CF baseline	CF incremental
100	861,734	384,306	355,924
1,000	6,301,416	1,447,611	1,389,360
10,000	59,590,615	11,307,145	10,612,077

Formatted Text

Chars	OF baseline	CF baseline	CF incremental
5	1,939,864	654,025	675,629
50	9,016,196	1,216,026	1,288,344
100	17,716,426	1,756,640	1,686,280

Edit Operation Time (ns/op)

Plain Text

Operation	Chars	ObjectForest	ChunkedForest
Insert 10	100	1,608,739	1,629,820
Insert 10	1,000	1,563,401	1,588,463
Insert 10	10,000	1,700,979	1,805,751
Remove 10	100	154,025	20,413
Remove 10	1,000	168,999	22,632
Remove 10	10,000	244,491	26,573
Read fullString	100	243,042	47,241
Read fullString	1,000	1,982,623	382,086
Read fullString	10,000	14,613,633	4,459,390

Formatted Text

Operation	Chars	ObjectForest	ChunkedForest
Insert 10	5	9,357,299	8,467,693
Insert 10	50	9,201,914	9,127,079
Insert 10	100	8,763,269	9,447,480
Remove 10	5	161,575	21,544
Remove 10	50	173,865	22,057
Remove 10	100	173,701	24,643
Read fullString	5	166,719	23,552
Read fullString	50	810,585	114,917
Read fullString	100	1,595,541	219,400

Formatted Text Read Operations — buildDeltaFromTree Hot Path (ns/op)

These benchmark the actual functions called by buildDeltaFromTree in the Quill integration
(getUniformRun + getString in a loop over the full document). Margins of error are <1%.

Operation	Chars	ObjectForest	ChunkedForest
getUniformRun	100	302,886	126,581
getUniformRun	500	1,516,689	668,629
getUniformRun	1,000	3,167,261	1,222,281
getUniformRun + getString	100	384,884	192,236
getUniformRun + getString	500	1,879,669	926,671
getUniformRun + getString	1,000	3,820,785	1,815,516

Summary Load Time (ns/op)

Plain Text

Chars	OF baseline	CF baseline	CF incremental
100	2,708,983	2,556,441	2,551,236
1,000	6,298,681	6,204,991	6,098,533
10,000	45,552,190	36,507,691	40,933,128

Formatted Text

Chars	OF baseline	CF baseline	CF incremental
5	3,330,018	3,049,986	3,143,361
50	5,416,129	4,918,183	4,387,451
100	7,685,150	6,504,543	6,248,753