Benchmark compiler

[rafaelbey]

Sample code to benchmark the compiler, and compare performance base on parallelism and PMCD size.

For the PMCD, the benchmark duplicates a sample PMCD. The assumption is that an average PMCD have similar distribution of elements as it gets bigger. There are scenarios this does not account or cover, and all this try to show that under some use cases, the parallelism will have a positive effect on the compiler.

Benchmark Results

• -1 is for the code before the refactoring. Given the number of synchronize blocks added, the performance impact is noticeable. We need to reduce these contention points.

Raw Data

(parallelism)	log(parallelism, 2)	(pmcdCount)	log(pmcdCount, 2)	Mode	Cnt	Score		Error	Units
-1	-1	1	0	avgt	25	0.202	±	0.066	s/op
-1	-1	2	1	avgt	25	0.2	±	0.016	s/op
-1	-1	4	2	avgt	25	0.214	±	0.002	s/op
-1	-1	8	3	avgt	25	0.251	±	0.002	s/op
-1	-1	16	4	avgt	25	0.385	±	0.098	s/op
-1	-1	32	5	avgt	25	0.466	±	0.022	s/op
-1	-1	64	6	avgt	25	0.773	±	0.017	s/op
-1	-1	128	7	avgt	25	1.356	±	0.011	s/op
1	0	1	0	avgt	10	0.517	±	0.023	s/op
1	0	2	1	avgt	10	0.765	±	0.064	s/op
1	0	4	2	avgt	10	0.855	±	0.006	s/op
1	0	8	3	avgt	10	1.093	±	0.006	s/op
1	0	16	4	avgt	10	1.533	±	0.014	s/op
1	0	32	5	avgt	10	2.449	±	0.148	s/op
1	0	64	6	avgt	10	4.147	±	0.028	s/op
1	0	128	7	avgt	10	7.605	±	0.054	s/op
2	1	1	0	avgt	10	0.529	±	0.031	s/op
2	1	2	1	avgt	10	0.523	±	0.009	s/op
2	1	4	2	avgt	10	0.71	±	0.006	s/op
2	1	8	3	avgt	10	0.913	±	0.015	s/op
2	1	16	4	avgt	10	1.223	±	0.051	s/op
2	1	32	5	avgt	10	2.083	±	0.109	s/op
2	1	64	6	avgt	10	3.78	±	0.065	s/op
2	1	128	7	avgt	10	7.207	±	0.097	s/op
4	2	1	0	avgt	10	0.506	±	0.005	s/op
4	2	2	1	avgt	10	0.573	±	0.034	s/op
4	2	4	2	avgt	10	0.541	±	0.004	s/op
4	2	8	3	avgt	10	0.744	±	0.005	s/op
4	2	16	4	avgt	10	1.126	±	0.05	s/op
4	2	32	5	avgt	10	1.793	±	0.116	s/op
4	2	64	6	avgt	10	3.451	±	0.401	s/op
4	2	128	7	avgt	10	6.441	±	0.713	s/op
8	3	1	0	avgt	10	0.514	±	0.005	s/op
8	3	2	1	avgt	10	0.542	±	0.028	s/op
8	3	4	2	avgt	10	0.541	±	0.004	s/op
8	3	8	3	avgt	10	0.717	±	0.034	s/op
8	3	16	4	avgt	10	1.008	±	0.019	s/op
8	3	32	5	avgt	10	1.585	±	0.079	s/op
8	3	64	6	avgt	10	2.919	±	0.07	s/op
8	3	128	7	avgt	10	5.586	±	0.221	s/op
16	4	1	0	avgt	10	0.535	±	0.034	s/op
16	4	2	1	avgt	10	0.543	±	0.006	s/op
16	4	4	2	avgt	10	0.556	±	0.027	s/op
16	4	8	3	avgt	10	0.619	±	0.005	s/op
16	4	16	4	avgt	10	0.863	±	0.037	s/op
16	4	32	5	avgt	10	1.306	±	0.038	s/op
16	4	64	6	avgt	10	2.237	±	0.1	s/op
16	4	128	7	avgt	10	4.267	±	0.115	s/op
32	5	1	0	avgt	10	0.518	±	0.019	s/op
32	5	2	1	avgt	10	0.519	±	0.009	s/op
32	5	4	2	avgt	10	0.546	±	0.008	s/op
32	5	8	3	avgt	10	0.622	±	0.005	s/op
32	5	16	4	avgt	10	0.752	±	0.011	s/op
32	5	32	5	avgt	10	1.196	±	0.077	s/op
32	5	64	6	avgt	10	1.963	±	0.105	s/op
32	5	128	7	avgt	10	3.655	±	0.193	s/op
64	6	1	0	avgt	10	0.548	±	0.035	s/op
64	6	2	1	avgt	10	0.54	±	0.027	s/op
64	6	4	2	avgt	10	0.556	±	0.013	s/op
64	6	8	3	avgt	10	0.631	±	0.007	s/op
64	6	16	4	avgt	10	0.759	±	0.007	s/op
64	6	32	5	avgt	10	1.172	±	0.055	s/op
64	6	64	6	avgt	10	1.992	±	0.109	s/op
64	6	128	7	avgt	10	3.601	±	0.21	s/op
128	7	1	0	avgt	10	0.502	±	0.013	s/op
128	7	2	1	avgt	10	0.545	±	0.006	s/op
128	7	4	2	avgt	10	0.551	±	0.023	s/op
128	7	8	3	avgt	10	0.608	±	0.013	s/op
128	7	16	4	avgt	10	0.743	±	0.014	s/op
128	7	32	5	avgt	10	1.097	±	0.042	s/op
128	7	64	6	avgt	10	1.988	±	0.092	s/op
128	7	128	7	avgt	10	3.657	±	0.185	s/op

Pivot

--	log(parallelism, 2)
log(pmcdCount, 2)	-1	0	1	2	3	4	5	6	7
0	0.202	0.517	0.529	0.506	0.514	0.535	0.518	0.548	0.502
1	0.2	0.765	0.523	0.573	0.542	0.543	0.519	0.54	0.545
2	0.214	0.855	0.71	0.541	0.541	0.556	0.546	0.556	0.551
3	0.251	1.093	0.913	0.744	0.717	0.619	0.622	0.631	0.608
4	0.385	1.533	1.223	1.126	1.008	0.863	0.752	0.759	0.743
5	0.466	2.449	2.083	1.793	1.585	1.306	1.196	1.172	1.097
6	0.773	4.147	3.78	3.451	2.919	2.237	1.963	1.992	1.988
7	1.356	7.605	7.207	6.441	5.586	4.267	3.655	3.601	3.657

Graph

System Information

• Laptop Legion 5 Pro 16IAH7H
• 12th Gen Intel(R) Core(TM) i7-12700H 2.30 GHz
• 32GB DDR5
• 64-bit operating system, x64-based processor
• Windows 11 Home - 22H2

[github-actions]

Test Results

0 files   -      798  0 suites   - 798   0s ⏱️ - 1h 12m 38s
0 tests  - 11 022  0 ✔️  - 10 947  0 💤  - 75  0 ❌ ±0 
0 runs   - 14 988  0 ✔️  - 14 913  0 💤  - 75  0 ❌ ±0 

Results for commit 5ef6cc7. ± Comparison against base commit c03e69a.

♻️ This comment has been updated with latest results.

[github-actions]

Workflow Telemetry - Build CI / Build

Workflow telemetry for commit 5ef6cc7a4fe4e202e9c9556f68dfef85dd7e022e
You can access workflow job details here

Step Trace

gantt
	title Build
	dateFormat x
	axisFormat %H:%M:%S
	Set up job : milestone, 1720367555000, 1720367557000
	Checkout repo : 1720367557000, 1720367561000
	Cache Maven dependencies : 1720367561000, 1720367662000
	Set up JDK : 1720367662000, 1720367675000
	Check Java version : 1720367675000, 1720367675000
	Configure git : 1720367675000, 1720367675000
	Download deps and plugins : 1720367675000, 1720367751000
	Collect Workflow Telemetry : 1720367751000, 1720367752000
	Build (PR) : crit, 1720367752000, 1720372235000
	Build (with Maven Deploy + Docker Snapshot) : done, 1720372235000, 1720372235000
	Test : done, 1720372235000, 1720372235000
	Upload Test Results : 1720372235000, 1720372235000

Loading

CPU Metrics

Memory Metrics

IO Metrics

	Read	Write
Network I/O
Disk I/O

Process Trace

Top 100 processes with highest duration

gantt
	title Build
	dateFormat x
	axisFormat %H:%M:%S

Loading