Coder Social home page Coder Social logo

egen's Introduction

EGEN - Tool for compile-time optimization of Java classes serialization

Download

It's aimed to provide efficient custom serialization strategy for dedicated classes without compromising compatibility with default java serialization approach and writing serialization code by hand.

Java serialization API provides following interface for custom serialization:

    void writeObject(ObjectOutputStream out) throws IOException;

    void readObject(ObjectInputStream in) throws IOException, ClassNotFoundException;

The tool is implemented as annotation processor invoked by javac during compilation. It analyzes classes marked with @AutoSerializable annotation and generates methods on the fly.

EGEN stands for Externalizable GENerator, but actually it generates only writeObject()/readObject() methods. We have refused of implementing java.io.Externalizable interface due to inheritance issues.

Usage

To activate tool the egen-<version>.jar provided by package shall be placed in compilation class path with its dependencies. If you use maven for build just add EGEN as dependency.

Note: Bundled variant with all needed classes provided as egen-<version>-bundled.jar for convenience.

The tool will process all classes marked with com.devexperts.egen.processor.annotations.AutoSerializable annotation and generate appropriate code:

    import com.devexperts.egen.processor.annotations.Autoserializable;

    @AutoSerializable
    class VeryLargeClass {
        private int count;
        private String text;

        protected double[] prices;
        protected Object cachedDataBaseObject;
        ...
    }

Important: generated code will use class IOUtil from com.devexperts.io package of dxlib library which can be found on dxFeed distribution repository:

Manual download:

http://maven.dxfeed.com/release/com/devexperts/qd/dxlib/3.135/dxlib-3.135.jar

Maven:

    <repositories>
        <repository>
            <id>maven-dxfeed-com</id>
            <url>http://maven.dxfeed.com/release</url>
        </repository>
    </repositories>

    <dependencies>
        <dependency>
            <groupId>com.devexperts.qd</groupId>
            <artifactId>dxlib</artifactId>
            <version>3.135</version>
        </dependency>
    </dependencies>

Serialization directives

EGEN supports various directives implemented as annotations for adjusting serialization algorithm:

@Compact

Compact representation of primitives, arrays, collections and maps.

@Compact int a; — field serialized by IOUtils.writeCompactInt(a)

@Compact long[] barr; — array length serialized as compact int and elements by writeCompactLong(v[i])

@Compact V[];
@Compact ArrayList<V>;
@Compact HashMap<K, V>; — compact int for size, then elements one by one. Compaction applied recursively: if types K,V are compactible then compaction will be applied.

Supported types: int, long, Integer, Long, String, int[], long[], array of objects, non-abstract implementations [1] of Collection<V> and Map<K, V>.

@PresenceBit

Marks field (field group) that usually has default value (all fields at once for group).

    @PresenceBit(value = "6") int a;
    @PresenceBit(value = "7e3", groupId = 1) double d1;
    @PresenceBit(value = "5", groupId = 1) double d2;

If group (single field is a group of one) is in default state it's coded as 1 bit in mask or all fields in group serialized using @Compact method. Group state mask serialized by writeCompactLong (so maximum 64 groups per class supported).

Supported types: String, all primitives and box-classes

@Inline

Skip object type descriptor for @AutoSerializable field.

Important! Inlined class shall has public default constructor and all its ancestors shall be @AutoSerializable also (except Object). The field shall be exactly of declared type (using subclasses is not allowed).

Supported types: any class marked by @AutoSerializable [2]

@Ordinal

Support for nonstandard ordinal types (like enum surrogates appeared before native java support of enum).

@Ordinal class A shell provide following methods:

  • Instance method int code() shall return code of an object.
  • Class method static A findByCode(Class<A>, int) shell return instance by type token and code.

@Delta

Coding numeric fields and arrays with usually close values.

    int a1;
    @Delta("a1") int a2; // a2 encoded as writeCompactInt(a2 - a1)
    @Delta("42") int a3; // a3 encoded as writeCompactInt(a3 - 42)

    @Delta long[] v; // length and v[0] encoded usual way, following elements as writeCompactLong(v[i]-v[i-1])

Important! Generated code requires classes from egen.jar.

Supported types: int, long, int[], long[]

@AutoSerializationStrategy

Meta annotation (annotation for annotations) to specify new serialization strategies. Has following parameters:

  • targetStrategy - IOUtil method of encoding
  • toTarget - method to convert original value to presentation suitable for targetStrategy
  • fromTarget - method to convert strategy-value to original

Encoding will be performed by template com.devexperts.io.IOUtil.write<targetStrategy>(out,<toTarget>(field));, decoding - simmetrically - fromTarget(...)

Example:

Given Decimal class with methods int compose(double) and double toDouble(int) for doubles with limited precision

    @AutoSerializationStrategy(targetStrategy=”CompactInt”,
        toTarget=”util.Decimal.compose”,
        fromTarget=”util.Decimal.toDouble”)
    @Retention(RetentionPolicy.RUNTIME)
    @Target(ElementType.FIELD)
    public @interface Decimal {}

Now we can use it following way:

    @AutoSerializable 
    class C {
        @Decimal double d;
        ...
    }

Footnotes:

[1]: By default recursive compaction provided for standard containers ArrayList, LinkedList, HashSet, TreeSet, HashMap and TreeMap. To activate compaction for some other classes they shall be specified by following options of javac: -Aordinals, -Amaps and -Acollections.

Example: javac -Acollections=java.util.concurrent.LinkedBlockingDeque ...

[2]: If class A contains @Inline-field of class B, then A and B shall be compiled by single invocation of javac (otherwise it won't compile).

egen's People

Contributors

tsitelov avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

egen's Issues

writeInline method breaks compilation if both classes have the same name

public interface A extends Serializable {

    @AutoSerializable
    abstract class Base implements A {
        @Compact protected int a1;

        public Base(int a1) {
            this.a1 = a1;
        }
    }
}

public interface B extends A {

    @AutoSerializable
    abstract class Base extends A.Base implements B {
        @Compact protected int b1;

        public Base(int a1, int b1) {
            super(a1);
            this.b1 = b1;
        }
    }
}

The dependency provided does not contain the Autoserializable annotation

Hello,

It seems that your project is a perfect fit for quickly moving on from Serialization to Externalization. I'm following the documentation to add the repository and dependecy provided but I cannot import the corresponding annotation, should I add more dependencies

import com.devexperts.egen.processor.annotations.Autoserializable;

Thanks

Support for java 10

While trying to compile our project that uses egen using java 9/10, the following warning message was observed-
warning: Supported source version 'RELEASE_8' from annotation processor 'com.devexperts.egen.processor.AutoSerializableProcessor' less than -source '1.9'
warning: Supported source version 'RELEASE_8' from annotation processor 'com.devexperts.uilocalizer.LocalizableProcessor' less than -source '1.9'

This could also be a contributing factor to causing javac itself to fail (See exception below). We can see that AutoSerializableProcessor (which is currently at SourceVersion 8) also uses com.sun.tools.javac.tree.JCTree which could be causing the exception.-

An exception has occurred in the compiler (9.0.4). Please file a bug against the Java compiler via the Java bug reporting page (http://bugreport.java.com) after checking the Bug Database (http://bugs.java.com) for duplicates. Include your program and the following diagnostic in your report. Thank you.
java.lang.NullPointerException
at jdk.compiler/com.sun.tools.javac.comp.Flow$FlowAnalyzer.visitApply(Flow.java:1233)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCMethodInvocation.accept(JCTree.java:1628)
at jdk.compiler/com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:49)
at jdk.compiler/com.sun.tools.javac.comp.Flow$BaseAnalyzer.scan(Flow.java:393)
at jdk.compiler/com.sun.tools.javac.tree.TreeScanner.visitSelect(TreeScanner.java:302)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCFieldAccess.accept(JCTree.java:2104)
at jdk.compiler/com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:49)
at jdk.compiler/com.sun.tools.javac.comp.Flow$BaseAnalyzer.scan(Flow.java:393)
at jdk.compiler/com.sun.tools.javac.comp.Flow$FlowAnalyzer.visitApply(Flow.java:1231)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCMethodInvocation.accept(JCTree.java:1628)
at jdk.compiler/com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:49)
at jdk.compiler/com.sun.tools.javac.comp.Flow$BaseAnalyzer.scan(Flow.java:393)
at jdk.compiler/com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:57)
at jdk.compiler/com.sun.tools.javac.comp.Flow$FlowAnalyzer.visitNewClass(Flow.java:1239)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCNewClass.accept(JCTree.java:1683)
at jdk.compiler/com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:49)
at jdk.compiler/com.sun.tools.javac.comp.Flow$BaseAnalyzer.scan(Flow.java:393)
at jdk.compiler/com.sun.tools.javac.comp.Flow$FlowAnalyzer.visitReturn(Flow.java:1208)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCReturn.accept(JCTree.java:1540)
at jdk.compiler/com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:49)
at jdk.compiler/com.sun.tools.javac.comp.Flow$BaseAnalyzer.scan(Flow.java:393)
at jdk.compiler/com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:57)
at jdk.compiler/com.sun.tools.javac.comp.Flow$FlowAnalyzer.visitBlock(Flow.java:995)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCBlock.accept(JCTree.java:1014)
at jdk.compiler/com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:49)
at jdk.compiler/com.sun.tools.javac.comp.Flow$BaseAnalyzer.scan(Flow.java:393)
at jdk.compiler/com.sun.tools.javac.comp.Flow$FlowAnalyzer.visitTry(Flow.java:1106)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCTry.accept(JCTree.java:1308)
at jdk.compiler/com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:49)
at jdk.compiler/com.sun.tools.javac.comp.Flow$BaseAnalyzer.scan(Flow.java:393)
at jdk.compiler/com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:57)
at jdk.compiler/com.sun.tools.javac.comp.Flow$FlowAnalyzer.visitBlock(Flow.java:995)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCBlock.accept(JCTree.java:1014)
at jdk.compiler/com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:49)
at jdk.compiler/com.sun.tools.javac.comp.Flow$BaseAnalyzer.scan(Flow.java:393)
at jdk.compiler/com.sun.tools.javac.comp.Flow$FlowAnalyzer.visitMethodDef(Flow.java:962)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCMethodDecl.accept(JCTree.java:866)
at jdk.compiler/com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:49)
at jdk.compiler/com.sun.tools.javac.comp.Flow$BaseAnalyzer.scan(Flow.java:393)
at jdk.compiler/com.sun.tools.javac.comp.Flow$FlowAnalyzer.visitClassDef(Flow.java:925)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:774)
at jdk.compiler/com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:49)
at jdk.compiler/com.sun.tools.javac.comp.Flow$BaseAnalyzer.scan(Flow.java:393)
at jdk.compiler/com.sun.tools.javac.comp.Flow$FlowAnalyzer.visitClassDef(Flow.java:870)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:774)
at jdk.compiler/com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:49)
at jdk.compiler/com.sun.tools.javac.comp.Flow$BaseAnalyzer.scan(Flow.java:393)
at jdk.compiler/com.sun.tools.javac.comp.Flow$FlowAnalyzer.analyzeTree(Flow.java:1325)
at jdk.compiler/com.sun.tools.javac.comp.Flow$FlowAnalyzer.analyzeTree(Flow.java:1315)
at jdk.compiler/com.sun.tools.javac.comp.Flow.analyzeTree(Flow.java:213)
at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1389)
at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1363)
at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:959)
at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.lambda$doCall$0(JavacTaskImpl.java:100)
at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.handleExceptions(JavacTaskImpl.java:142)
at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:96)
at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:90)
at org.gradle.api.internal.tasks.compile.JdkJavaCompiler.execute(JdkJavaCompiler.java:50)
at org.gradle.api.internal.tasks.compile.JdkJavaCompiler.execute(JdkJavaCompiler.java:36)
at org.gradle.api.internal.tasks.compile.NormalizingJavaCompiler.delegateAndHandleErrors(NormalizingJavaCompiler.java:98)
at org.gradle.api.internal.tasks.compile.NormalizingJavaCompiler.execute(NormalizingJavaCompiler.java:51)
at org.gradle.api.internal.tasks.compile.NormalizingJavaCompiler.execute(NormalizingJavaCompiler.java:37)
at org.gradle.api.internal.tasks.compile.CleaningJavaCompilerSupport.execute(CleaningJavaCompilerSupport.java:35)
at org.gradle.api.internal.tasks.compile.CleaningJavaCompilerSupport.execute(CleaningJavaCompilerSupport.java:25)
at org.gradle.api.tasks.compile.JavaCompile.performCompilation(JavaCompile.java:156)
at org.gradle.api.tasks.compile.JavaCompile.compile(JavaCompile.java:141)
at org.gradle.api.tasks.compile.JavaCompile.compile(JavaCompile.java:114)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:73)
at org.gradle.api.internal.project.taskfactory.IncrementalTaskAction.doExecute(IncrementalTaskAction.java:50)
at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:39)
at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:26)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$1.run(ExecuteActionsTaskExecuter.java:124)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:317)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:309)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:185)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:97)
at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:113)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:95)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:73)
at org.gradle.api.internal.tasks.execution.OutputDirectoryCreatingTaskExecuter.execute(OutputDirectoryCreatingTaskExecuter.java:51)
at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:59)
at org.gradle.api.internal.tasks.execution.ResolveTaskOutputCachingStateExecuter.execute(ResolveTaskOutputCachingStateExecuter.java:54)
at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:59)
at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:101)
at org.gradle.api.internal.tasks.execution.FinalizeInputFilePropertiesTaskExecuter.execute(FinalizeInputFilePropertiesTaskExecuter.java:44)
at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:91)
at org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter.execute(ResolveTaskArtifactStateTaskExecuter.java:62)
at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:59)
at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:54)
at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:34)
at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker$1.run(DefaultTaskGraphExecuter.java:256)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:317)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:309)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:185)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:97)
at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:249)
at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:238)
at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:104)
at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:98)
at org.gradle.execution.taskgraph.DefaultTaskExecutionPlan.execute(DefaultTaskExecutionPlan.java:663)
at org.gradle.execution.taskgraph.DefaultTaskExecutionPlan.executeWithTask(DefaultTaskExecutionPlan.java:596)
at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker.run(DefaultTaskPlanExecutor.java:98)
at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
at java.base/java.lang.Thread.run(Thread.java:844)

FAILURE: Build failed with an exception.

  • What went wrong:
    Execution failed for task ':common:compileJava'.

Compilation failed; see the compiler error output for details.

  • Try:
    Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

  • Get more help at https://help.gradle.org

BUILD FAILED in 2s

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.