objectionary / benchmark

Java compilation and optimization benchmark that integrates EO, 𝜑-calculus, JEO, OPEO, and INEO optimizers

It's a benchmark that puts together:

• jeo-maven-plugin
• opeo-maven-plugin
• ineo-maven-plugin

The goal is to test the performance of Java code before optimization and after, then comparing the results. Intermediate artifacts are in this summary.html.

This is the summary of the tests performed with the TOTAL set to 10000000, at 2024-06-17 15:36, on Linux, with 4 CPUs:

	Before	After	Diff
Time, msec (with JIT, ×40 cycles)	4691	4345	-7%
Time, msec (no JIT)	5699	2638	-53%
Total .class files	3	4
Bytes in all .class files	8010	7300
JAR file size, bytes	49724094	43102077

This table is updated on every successful run of the make job of GitHub Actions. The following JDK is used:

java 21.0.3 2024-04-16 LTS
Java(TM) SE Runtime Environment (build 21.0.3+7-LTS-152)
Java HotSpot(TM) 64-Bit Server VM (build 21.0.3+7-LTS-152, mixed mode, sharing)

How to Contribute

Fork repository, make changes, then send us a pull request. We will review your changes and apply them to the master branch shortly, provided they don't violate our quality standards. To avoid frustration, before sending us your pull request please run full build:

$ make

You will need Maven 3.3+ and Java 11+ installed.

benchmark's Issues

Make removes the repo from FS

What I do:

1. Git clone (my exact copy of benchmark repository):

git clone [email protected]:volodya-lombrozo/benchmark.git

2. cd benchmark

3. Run make:

make

4. I get the following logs:

set -e
base=before
mvn --activate-profiles "${base}" --update-snapshots clean package "-DfinalName=${base}" "-Ddirectory=${base}" \
		-Djeo.version=0.2.18 \
		-Deo.version=0.34.4 \
		-Dopeo.version=0.1.2 \
		-Dineo.version=0.1.6
[INFO] Scanning for projects...
[INFO] 
[INFO] ------------------------< org.eolang:benchmark >------------------------
[INFO] Building benchmark 0.0.0
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- clean:3.2.0:clean (default-clean) @ benchmark ---
[INFO] Deleting /Users/lombrozo/Workspace/EOlang/benchmark
[INFO] 
[INFO] --- resources:3.3.1:resources (default-resources) @ benchmark ---
[INFO] skip non existing resourceDirectory /Users/lombrozo/Workspace/EOlang/benchmark/src/main/resources
[INFO] 
[INFO] --- compiler:3.11.0:compile (default-compile) @ benchmark ---
[INFO] No sources to compile
[INFO] 
[INFO] --- resources:3.3.1:testResources (default-testResources) @ benchmark ---
[INFO] skip non existing resourceDirectory /Users/lombrozo/Workspace/EOlang/benchmark/src/test/resources
[INFO] 
[INFO] --- compiler:3.11.0:testCompile (default-testCompile) @ benchmark ---
[INFO] No sources to compile
[INFO] 
[INFO] --- surefire:3.1.2:test (default-test) @ benchmark ---
[INFO] No tests to run.
[INFO] 
[INFO] --- jar:3.3.0:jar (default-jar) @ benchmark ---
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.900 s
[INFO] Finished at: 2024-01-23T11:05:56+03:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-jar-plugin:3.3.0:jar (default-jar) on project benchmark: Error assembling JAR: /Users/lombrozo/Workspace/EOlang/benchmark/pom.xml isn't a file. -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
make: *** [before.jar] Error 1

5. benchmark folder is removed and I have to repeat all the steps again

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Repository problems

These problems occurred while renovating this repository. View logs.

• WARN: File contents are invalid JSON but parse using JSON5. Support for this will be removed in a future release so please change to a support .json5 file name or ensure correct JSON syntax.

This repository currently has no open or pending branches.

Detected dependencies

github-actions

.github/workflows/make.yml

• actions/checkout v4
• actions/setup-java v4
• actions/cache v4
• peter-evans/create-pull-request v6
• JamesIves/github-pages-deploy-action v4.6.3
• ubuntu 22.04

.github/workflows/mvn.yml

• actions/checkout v4
• actions/setup-java v4
• actions/cache v4
• ubuntu 22.04

---

• Check this box to trigger a request for Renovate to run again on this repository

Do you use "fuse" optimization?

To complete the optimization pipeline I adapted opeo-maven-plugin to make fuse optimization possible. I even added the integration test for the fuse optimization.
But as I see, you don't use fuse optimization anymore. Is it correct?

publish intermediate results at gh-pages

start

default Maven lifecycle is not rich enough for our pipeline

We need to run a few plugins, one after another, all in one process-classes phase. We can't simply put them sequentially into the <plugins/> section, because they all have "start" and "finish" goals:

• jeo-maven-plugin:bytecode-to-eo
• opeo-maven-plugin:decompile
• normalizer-maven-plugin:normalize
• ineo-maven-plugin:optimize
• opeo-maven-plugin:compile
• jeo-maven-plugin:eo-to-bytecode

Putting them into different phases (taking 5+ phases) would create a total mess in the lifecycle -- it will be hard to use our plugins in existing Java projects.

Thus, I believe, we should define our own Maven lifecycle phases, which should be added to the default set of them. This is what I've found so far:

• https://stackoverflow.com/questions/36617759/adding-phases-to-the-maven-lifecycle
• https://github.com/raydac/mvn-finisher/

It seems that we should create our own "extension" that will inject new phases into the lifecycle, in runtime.

An alternative approach would involve creating a new umbrella plugin, which will call all other plugins, thus implementing a sub-lifecycle inside an existing Maven project.

Yet another alternative is ... Gradle.