aldica / aldica

Alternative Distributed Caching (aldica) module for Alfresco Community Edition

Name

The portmanteau "aldica" can be taken to stand for either

• Alternative Distributed Caching, or
• Alfresco Distributed Caching

The primary / canon long name is Alternative Distributed Caching.

About

This module project provides a distributed caching and data grid module for Alfresco Content Services and Share (Community Edition). It currently uses the Apache Ignite library as the underlying cache and data grid framework, and enables multiple Alfresco Content Services / Share instances to share cache state and exchange messages about data invalidations. With this module, Alfresco Community Edition may be horizontally scaled by adding multiple instances on different hosts or in multiple containers (e.g. as part of a Kubernetes cluster). Even if horizontal scaling is not required, this module can still be used to improve the overall scalability / performance of a single server installation of Alfresco Content Services, by providing improved implementations for its caching layer, e.g. using off-heap memory and even disk-based swap space to reduce pressure on the Java Garbage Collector while allowing significantly larger amounts of data to be cached, reducing load and latency of database accesses.

Compatibility

The module of this project is built to be compatible with Alfresco Content Services 6.0.7 GA / Alfresco Share 6.0.c and above. Since the Alfresco core APIs on which this module relies have remained stable in the 6.x release lines, the module should also be compatible with Alfresco Content Services 5.2.g / Alfresco Share 5.2.f.

Note: Alfresco Enterprise Edition already ships with custom cache implementations that are tied directly to its Enterprise clustering feature. Though we have managed to install aldica and use its cache provider on a trial version of Alfresco Enterprise Edition with only a minor bit of tinkering, we cannot generally support Alfresco Enterprise Edition. Its code is not open to us and may change without notice / disclosure in any service pack or hotfix release, so we cannot safely adapt aldica to work without risk to Enterprise features in the range of versions of Alfresco Content Services that we would like to support. Furthermore, Enterprise customers choosing to install aldica would likely be left in a system state that Alfresco Support could rightfully refuse to support when issues are filed for problems remotely related to caching. As an open source project, we cannot take on the warranty / liability of such systems either.

Known Issue(s) / Limitation(s) with Alfresco 5.2

Due to an issue with transactional resource and cleanup handling (e.g. SPR-15194) fixed only with an upgraded Spring library in Alfresco 6.x, parts of aldica's serialisation improvements to reduce the memory footprint of cached values cannot be used on Alfresco 5.2. These must be disabled by setting the following Repository-tier global properties:

aldica.core.binary.optimisation.useIdsWhenReasonable=false
aldica.core.binary.optimisation.useIdsWhenPossible=false

Published Release / SNAPSHOT Artifacts

All artifacts of released versions of this module will be published on Maven Central, using the group ID org.aldica . SNAPSHOT versions of artifacts will be published on the Sonatype Open Source Software Repository Hosting service. In order to reference SNAPSHOT versions in Maven-based projects, an appropriate repository section has to be added to the project's POM file:

    <repositories>
        <repository>
            <id>ossrh</id>
            <url>https://oss.sonatype.org/content/repositories/snapshots</url>
            <snapshots>
                <enabled>true</enabled>
            </snapshots>
        </repository>
    </repositories>

Dependencies

This module depends on the following projects / libraries:

• Apache Ignite (Apache License, Version 2.0)
• IntelliJ IDEA Annotations (Apache License, Version 2.0)
• Acosix Alfresco Utility (Apache License, Version 2.0)

The module AMPs produced by the build of this project will include all the necessary components / libraries provided by the Apache Ignite project and IntelliJ IDEA Annotations. The Acosix Alfresco Utility project itself provides its own module AMPs, which need to be installed in Alfresco Content Services / Share before the corresponding AMP of this project can be installed.

When the installable JARs produced by the build of this project are used for installation, the developer / user is responsible to either manually install all the required components / libraries provided by the listed projects, or use a build system to collect all relevant direct / transitive dependencies. Note: The Acosix Alfresco Utility project is also built using templates from the Acosix Alfresco Maven project, and as such produces similar artifacts. Automatic resolution and collection of (transitive) dependencies using Maven / Gradle will resolve the Java classes JAR as a dependency, and not the installable (Simple Alfresco Module) variant. It is recommended to exclude Acosix Alfresco Utility from transitive resolution and instead include it directly / explicitly.

Documentation

The documentation is maintained as part of the source code of the alternative/Alfresco distributed caching (aldica) module. Each Markdown document represents an individual, cohesive section / topic of documentation, though may reference other sections which provide either more detailed information in a sub-topic or aggregate partial information bits from multiple sections (e.g. configuration reference sections).

• Build
• Getting Started
  • ACS configuration scenarios
  • Share configuration scenarios
  • Repository companion application
  • Tomcat (to be reviewed / updated)
• Installation / Configuration
  • Install via Alfresco SDK
  • Install via Docker Build
  • Install via Kubernetes
  • Java Virtual Machine (JVM) Properties
  • Repository Configuration Reference
  • Share Configuration Reference
• Concepts / High-Level
  • Alfresco/Ignite and On-Heap vs Off-Heap Caches
  • Grid Member Discovery
  • Binary Serialisation Optimisations
• Verification / Tests
  • K6 Benchmark
  • Memory Benchmark
  • JMeter Benchmark
  • Manual Verification

aldica's Issues

Using JAR files instead of AMPs when building Docker images

Currently, we are building Dockerfiles using AMPs. This causes a small overhead, so instead we should copy the aldica/utils JAR files into the exploded WAR files in Tomcat, i.e. into WEB-INF/lib

Binary Serialiser: specialised raw serial form for content URLs

The binary serialisation of ContentData / ContentDataWithId could be optimised further by using a specialised raw serial form for well known content URL patterns. A typical content URL in (default) Alfresco will be of the form store://YYYY/M/d/H/m/uuid.bin without leading zero digits. Instead of always writing the whole URL as a string, it could be fragmented and written using primitives with a leading flag-byte.

• flag byte: contains flag bits for
  • is default store protocol being used or will a custom protocol be written as a string fragment
  • is the default URL path pattern being used - if not, full URL path will be written as a regular string
  • does URL path contain an extra volume fragment based on the VolumeAwareContentUrlProvider URL pattern variant
  • does URL path contain an extra bucket fragment based on the bucketsPerMinute config parameter in TimeBasedFileContentUrlProvider
  • is file name a proper UUID - if not, file name will be written as a regular string
  • is file name suffix the default `.bin' - if not, suffix will be written as a regular string
  • does URL even contain a path
  • does URL even include a suffix - if not, no additional string will be written / expected when reading
• optional String with variable length integer for custom protocol
• optional String with variable length integer for volume fragment
• 4 byte integer for year + month bit mask (12 bit year, 4 bit month) and day of month + hour + minute bit mask (5 bit day of month, 5 bit hour of day, 6 bit minute)
• optional byte for bucket fragment
• 16 bytes for uuid or String with variable length integer for file name
• optional String with variable length integer for suffix

In a default Alfresco installation, a content URL would be 21 bytes long compared to the default string length of 63-67 bytes (including the length meta value) when writing the entire content URL as a string.

Resource bundles missing for teh default locale

Changing the POM to:

<parent>
        <groupId>de.acosix.alfresco.maven</groupId>
        <artifactId>de.acosix.alfresco.maven.project.parent-6.2.0</artifactId>
        <!-- artifactId>de.acosix.alfresco.maven.project.parent-6.1.2</artifactId -->
        <version>1.3.0-SNAPSHOT</version>
</parent>

and running the command

$ mvn install -B -T2C -DskipTests -Dquality.findBugs.skip

results in the following error:

[INFO] --- i18n-resources-plugin:1.0.4:validateI18nResources (default) @ aldica-repo-ignite ---
[WARNING] webscripts/de/acosix/acosix-ignite-repo/admin/ignite/ignite-caches.get is missing a localised bundle for the default locale
[WARNING] webscripts/de/acosix/acosix-ignite-repo/admin/ignite/ignite-data-regions.get is missing a localised bundle for the default locale
[WARNING] webscripts/de/acosix/acosix-ignite-repo/admin/ignite/ignite-grids.get is missing a localised bundle for the default locale
[ERROR] 3 resource bundle(s) missing for the default locale
[WARNING] webscripts/de/acosix/acosix-ignite-repo/admin/ignite/ignite-caches.get is missing a localised bundle for de
[WARNING] webscripts/de/acosix/acosix-ignite-repo/admin/ignite/ignite-data-regions.get is missing a localised bundle for de
[WARNING] webscripts/de/acosix/acosix-ignite-repo/admin/ignite/ignite-grids.get is missing a localised bundle for de
[ERROR] 3 resource bundle(s) missing for de
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Alternative/Alfresco Distributed Cache - Parent 1.0.0.0-SNAPSHOT:
[INFO] 
[INFO] Alternative/Alfresco Distributed Cache - Parent .... SUCCESS [  0.409 s]
[INFO] Alternative/Alfresco Distributed Cache - Common Ignite Library SUCCESS [  9.581 s]
[INFO] Alternative/Alfresco Distributed Cache - Repository Ignite Module FAILURE [  4.635 s]
[INFO] Alternative/Alfresco Distributed Cache - Share Ignite Module SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  19.630 s (Wall Clock)
[INFO] Finished at: 2020-04-15T12:37:02+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal de.acosix.maven:i18n-resources-plugin:1.0.4:validateI18nResources (default) on project aldica-repo-ignite: Some of the required locales are missing specific resource bundles -> [Help 1]

Are the required files missing from a commit?

Investigate possibility of triggering pro-active data region memory cleanup

As discovered during the memory-bm investigations where it turned out the memory-bm webscript for clearing the caches did not clear the Ignite cache data regions.

Multiple nodes in Docker-compose

Work on the feature supporting multiple repository and Share nodes in the docker-compose.yml is ongoing in the branch/folder: https://github.com/aldica/aldica/tree/feature/multiple-nodes-docker-compose/docker. Everything seems to be working except for an issue with Share (probably related to the configuration of load balancing). The following has been done:

• The Dockerfiles for Alfresco and Share have been updated
• Two Alfresco repo containers have been configured in docker-compose.yml
• Two Share containers have been configured in docker-compose.yml
• A Dockerfile for an Nginx load balancer has been added and the load balancer has also been added to the docker-compose.yml

Things to note
There were some problems with the Share CSRF filter and these were addressed as follows:

• A version of share-config-custom.xml with the CSRF filter disabled was attempted to be bind mounted into the Share containers, but this was not possible due to the way the upstream Share Dockerfile entrypoint is constructed, i.e. the sed commands in this file are not working due to permission issues when share-config-custom.xml is bind mounted into the container.
• The CSRF policies were attempted to to be set using the environment variables CSRF_FILTER_REFERER and CSRF_FILTER_ORIGIN (leaving these empty), but this did not have the desired effect.
• Finally a (sub-optimal) solution was chosen, i.e. to copy the share-config-custom.xml into the Docker images itself. This works, but it require that we set the Share custom configurations directly in the Share Dockerfile.

With the CSRF configuations addressed as just described, everything seems to be working except for the load balancing for the Share containers. When starting the stack from the docker folder with

$ docker-compose up -d --build

and having set

127.0.0.1	localhost alfresco1 alfresco2 loadbalancer share1 share2

in the host /etc/hosts, it can be seen from the logs that the systems start without errors, and it can also be verified that the distributed nature of the caching mechanism for the repos is working (by doing the authentication ticket test). But when going to http://loadbalancer:8080/share/page and logging in, the login POST request is made (giving a HTTP status 302), but after the redirect, the user is just faced with the login page again...

@AFaust This is probably due to some Share configuration detail that I am missing - do you have an idea what could be wrong?

Investigate / correct regressions in Ignite 2.8.0 upgrade

The branch ignite-2.8.0-upgrade contains my current attempts to upgrade Ignite from 2.7.6 to 2.8.0. Most tests on the common and repository sub-modules run perfectly without or only little changes to adapt to Ignite 2.8.0. But two main integration test cases in the repository sub-module fail.

[ERROR] Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 10.91 s <<< FAILURE! - in org.aldica.repo.ignite.integration.CacheConsistency
[ERROR] updateCachedNode(org.aldica.repo.ignite.integration.CacheConsistency)  Time elapsed: 2.773 s  <<< FAILURE!
org.junit.ComparisonFailure: Modified date of Shared folder should be identical (after update) in cluster expected:<2020-04-10T20:4[5:26.80]0+0000> but was:<2020-04-10T20:4[3:11.97]0+0000>
        at org.aldica.repo.ignite.integration.CacheConsistency.updateCachedNode(CacheConsistency.java:214)

[ERROR] dynamicModelActivation(org.aldica.repo.ignite.integration.CacheConsistency)  Time elapsed: 2.662 s  <<< ERROR!
javax.ws.rs.BadRequestException: HTTP 400 Bad Request
        at org.aldica.repo.ignite.integration.CacheConsistency.dynamicModelActivation(CacheConsistency.java:280)

[INFO] Running org.aldica.repo.ignite.integration.LockConsistency
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.65 s - in org.aldica.repo.ignite.integration.LockConsistency
[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]   CacheConsistency.updateCachedNode:214 Modified date of Shared folder should be identical (after update) in cluster expected:<2020-04-10T20:4[5:26.80]0+0000> but was:<2020-04-10T20:4[3:11.97]0+0000>
[ERROR] Errors:
[ERROR]   CacheConsistency.dynamicModelActivation:280  BadRequest HTTP 400 Bad Request
[INFO]
[ERROR] Tests run: 7, Failures: 1, Errors: 1, Skipped: 0

According to logging, the CacheConsistency.dynamicModelActivation test case fails because the grid message about the activated dictionary model is not received / processed on the second server, so the model is effectively never activated there and the subsequent request to create a node using that model fails (as would be expected in that case).
The reason why CacheConsistency.updateCachedNode fails is currently unclear. Obviously the update of the modified timestamp has not been propagated to the second server. The test fails on the modified timestamp assertion before the properties that were explicitly changed could be compared, but it is safe to assume that these would also not have been propagated since the relevant node.nodePropertiesCache entry encompasses all properties and is atomically updated at the end of a transaction.

Investigate possibilities to turn core nodesSharedCache into Ignite-backed cache

Currently, the nodesSharedCache is the most important, massive cache which has not yet been switched over to being backed by Ignite caches in our module. This is due to the fact that the Alfresco value entities stored in the cache include mutable state and potentially rely on object identity to work correctly in specific use cases. Since this cache manages twofold mappings of nodes by database ID and NodeRef, and is used for every node actively accessed in the last 5 minutes (default configuration), the amount of entries can grow quite large.

We should investigate if we can switch this cache over to using Ignite caches. Since the use of mutable state by Alfresco is a major design flaw and would normally prevent us from this kind of switch, we need to investigate if we could use a combination of secondary caches (for the mutable state) and value transformations to make this kind of switch. We also need to identify and document those specific use cases where the mutable state is essential, in order to create test scenarios to validate our low-level "hacks" that correct Alfresco's mistake.

Asteric value of property cors.allowed.origins causes Ignite error

BUG

Setting the Alfresco global property cors.allowed.origins to the value * like this:

cors.enabled=true
cors.allowed.origins=*

causes an Ignite error during start up of Alfresco. Setting the property to another value, e.g. cors.allowed.origins=http://xyz works fine.

Expected behavior

The aldica-enabled Alfresco repository is expected to start up without errors.

Actual behavior

The aldica-enabled Alfresco repository throws the following error during start up when cors.allowed.origins=*:

alfresco_1           |  2020-10-07 12:15:16,869  INFO  [extensions.webscripts.DeclarativeRegistry] [asynchronouslyRefreshedCacheThreadPool1] Registered 0 Schema Description Documents (+0 failed) 
alfresco_1           |  2020-10-07 12:15:17,172  WARN  [scripts.servlet.X509ServletFilterBase] [localhost-startStop-1] clientAuth does not appear to be set for Tomcat. clientAuth must be set to 'want' for X509 Authentication
alfresco_1           |  2020-10-07 12:15:17,172  WARN  [scripts.servlet.X509ServletFilterBase] [localhost-startStop-1] Attempting to set clientAuth=want through JMX...
alfresco_1           |  2020-10-07 12:15:17,194  WARN  [scripts.servlet.X509ServletFilterBase] [localhost-startStop-1] Unable to set clientAuth=want through JMX.
alfresco_1           | 07-Oct-2020 12:15:17.195 SEVERE [localhost-startStop-1] org.apache.catalina.core.StandardContext.startInternal One or more Filters failed to start. Full details will be found in the appropriate container log file
alfresco_1           | 07-Oct-2020 12:15:17.195 SEVERE [localhost-startStop-1] org.apache.catalina.core.StandardContext.startInternal Context [/alfresco] startup failed due to previous errors
alfresco_1           |  2020-10-07 12:15:17,243  INFO  [ignite.lifecycle.SpringIgniteLifecycleBean] [localhost-startStop-1] Closing Ignite isntance knrGrid
alfresco_1           |  2020-10-07 12:15:17,319  ERROR [ignite.internal.IgniteKernal%knrGrid] [localhost-startStop-1] Failed to stop component (ignoring): GridProcessorAdapter []
alfresco_1           |  java.lang.UnsupportedOperationException
alfresco_1           | 	at org.jsr166.ConcurrentLinkedHashMap.clear(ConcurrentLinkedHashMap.java:1542)
alfresco_1           | 	at org.apache.ignite.internal.processors.job.GridJobProcessor.stop(GridJobProcessor.java:375)
alfresco_1           | 	at org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:2627)
alfresco_1           | 	at org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:2499)
alfresco_1           | 	at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2650)
alfresco_1           | 	at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2613)
alfresco_1           | 	at org.apache.ignite.internal.IgnitionEx.stop(IgnitionEx.java:339)
alfresco_1           | 	at org.apache.ignite.Ignition.stop(Ignition.java:223)
alfresco_1           | 	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.lambda$shutdownGrid$5(SpringIgniteLifecycleBean.java:160)
alfresco_1           | 	at org.aldica.common.ignite.context.ExternalContext.withExternalContext(ExternalContext.java:112)
alfresco_1           | 	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.shutdownGrid(SpringIgniteLifecycleBean.java:154)
alfresco_1           | 	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.onApplicationEvent(SpringIgniteLifecycleBean.java:77)
alfresco_1           | 	at org.alfresco.repo.management.SafeApplicationEventMulticaster.multicastEventInternal(SafeApplicationEventMulticaster.java:221)
alfresco_1           | 	at org.alfresco.repo.management.SafeApplicationEventMulticaster.multicastEvent(SafeApplicationEventMulticaster.java:191)
alfresco_1           | 	at org.alfresco.repo.management.SafeApplicationEventMulticaster.multicastEvent(SafeApplicationEventMulticaster.java:206)
alfresco_1           | 	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:402)
alfresco_1           | 	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:359)
alfresco_1           | 	at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:1009)
alfresco_1           | 	at org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:975)
alfresco_1           | 	at org.springframework.web.context.ContextLoader.closeWebApplicationContext(ContextLoader.java:517)
alfresco_1           | 	at org.springframework.web.context.ContextLoaderListener.contextDestroyed(ContextLoaderListener.java:112)
alfresco_1           | 	at org.apache.catalina.core.StandardContext.listenerStop(StandardContext.java:4746)
alfresco_1           | 	at org.apache.catalina.core.StandardContext.stopInternal(StandardContext.java:5403)
alfresco_1           | 	at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:226)
alfresco_1           | 	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:154)
alfresco_1           | 	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:743)
alfresco_1           | 	at org.apache.catalina.core.ContainerBase.access$000(ContainerBase.java:129)
alfresco_1           | 	at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:150)
alfresco_1           | 	at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:140)
alfresco_1           | 	at java.base/java.security.AccessController.doPrivileged(Native Method)
alfresco_1           | 	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:717)
alfresco_1           | 	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:714)
alfresco_1           | 	at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1125)
alfresco_1           | 	at org.apache.catalina.startup.HostConfig$DeployDirectory.run(HostConfig.java:1859)
alfresco_1           | 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
alfresco_1           | 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
alfresco_1           | 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
alfresco_1           | 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
alfresco_1           | 	at java.base/java.lang.Thread.run(Thread.java:834)
alfresco_1           | [12:15:17] Ignite node stopped wih ERRORS [name=knrGrid, uptime=00:00:10.406]
alfresco_1           | 2020-10-07 12:15:17,388  INFO  [ignite.lifecycle.SpringIgniteLifecycleBean] [localhost-startStop-1] Closed Ignite instance knrGrid
alfresco_1           |  2020-10-07 12:15:17,427  INFO  [management.subsystems.ChildApplicationContextFactory] [localhost-startStop-1] Stopping 'Replication' subsystem, ID: [Replication, default]
alfresco_1           |  2020-10-07 12:15:17,428  INFO  [management.subsystems.ChildApplicationContextFactory] [localhost-startStop-1] Stopped 'Replication' subsystem, ID: [Replication, default]

Steps to reproduce the behavior

Configure a single aldica-enabled Alfresco 6.2.1-A8 repo with the following aldica/Ignite configuration:

    --add-exports=java.base/jdk.internal.misc=ALL-UNNAMED\
    --add-exports=java.base/sun.nio.ch=ALL-UNNAMED\
    --add-exports=java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED\
    --add-exports=jdk.internal.jvmstat/sun.jvmstat.monitor=ALL-UNNAMED\
    --add-exports=java.base/sun.reflect.generics.reflectiveObjects=ALL-UNNAMED\
    --illegal-access=permit\
    -XX:+UseG1GC\
    -XX:+ParallelRefProcEnabled\
    -XX:+UseStringDeduplication\
    -XX:+ScavengeBeforeFullGC\
    -XX:+DisableExplicitGC\
    -XX:+AlwaysPreTouch\
    -Dfile.encoding=UTF-8\
    -Djava.net.preferIPv4Stack=true\
    -DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true\
    -DIGNITE_QUIET=true\
    -DIGNITE_NO_ASCII=true\
    -DIGNITE_UPDATE_NOTIFIER=false\
    -DIGNITE_JVM_PAUSE_DETECTOR_DISABLED=true\
    -DIGNITE_SKIP_CONFIGURATION_CONSISTENCY_CHECK=true\

    -Daldica.core.name=knrGrid\
    -Daldica.caches.remoteSupport.enabled=true\

Tell us about your environment

The stack is started locally with the following Docker-compose configuration:

version: '3.4'
services:
  alfresco:
    image: magentaaps/knr-repo:development
    build:
      dockerfile: docker/Dockerfile.repo
      context: .
    environment:
      CATALINA_OPTS: "-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=0.0.0.0:8888"
    ports:
      - 8080:8080
      - 8888:8888
    volumes:
      - type: volume
        source: repo-volume
        target: /usr/local/tomcat/alf_data
      - type: bind
        source: ./dev-environment/alfresco-global.properties
        target: /usr/local/tomcat/shared/classes/alfresco-global.properties
      - type: bind
        source: ./dev-environment/dev-log4j.properties
        target: /usr/local/tomcat/shared/classes/alfresco/extension/dev-log4j.properties
      - type: bind
        source: ./dev-environment/disable-webscript-caching-context.xml
        target: /usr/local/tomcat/shared/classes/alfresco/extension/disable-webscript-caching-context.xml
    depends_on:
      - postgres

  postgres:
    image: postgres:11.4
    environment:
      POSTGRES_DB: alfresco
      POSTGRES_USER: alfresco
      POSTGRES_PASSWORD: alfresco
    command: postgres -c max_connections=300 -c log_min_messages=LOG
    ports:
      - 5432:5432
    volumes:
      - type: volume
        source: postgres-volume
        target: /var/lib/postgresql/data

  postgres-keycloak:
    image: postgres
    ports:
      - 5431:5432
    environment:
      POSTGRES_DB: keycloak
      POSTGRES_USER: keycloak
      POSTGRES_PASSWORD: keycloak
    volumes:
      - type: volume
        source: knr-postgres-keycloak-volume
        target: /var/lib/postgresql/data

  keycloak:
    depends_on:
      - postgres-keycloak
    image: quay.io/keycloak/keycloak:11.0.2
    command: [ "-Djboss.socket.binding.port-offset=200", "-Dkeycloak.migration.strategy=IGNORE_EXISTING" ]
    ports:
      - 8280:8280
    environment:
      KEYCLOAK_USER: admin
      KEYCLOAK_PASSWORD: admin
      KEYCLOAK_IMPORT: /tmp/realm-develop.json
      DB_USER: keycloak
      DB_PASSWORD: keycloak
      DB_ADDR: postgres-keycloak
    volumes:
      - type: volume
        source: knr-keycloak-volume
        target: /opt/jboss/keycloak/standalone/data
      - type: bind
        source: ./dev-environment/realm-develop.json
        target: /tmp/realm-develop.json

volumes:
  repo-volume:
  postgres-volume:
  knr-postgres-keycloak-volume:
  knr-keycloak-volume:

Question regarding Java versions (and POMs)

According to the Alfresco documentation the supported versions of Java is both Java 8 and Java 11 for Alfresco 6.0.1 and Java 11 for Alfresco 6.1.

Which versions of Java is required/used to build the module? As I understand it, this should be handled via the POM(s) and the toolchain, but I am not sure where this info is available in the POM(s)?

The reason I ask is that I build the module on my local machine (which has Java 8 available but not Java 11) and then deploy it in a LXC having only Java 11 available. When I started Tomcat there were errors from the module in the log, but then I installed Java 8 in the LXC and then everything worked fine.

Is it necessary to tweek some Java settings (in the POM(s) or elsewhere) to get the module working with Java 11?

Alfresco transactional caches not stabilising in concurrent access patterns

Analysing the benchmark results updated with the serialisation changes in PR 34, an odd heap memory cost profile was observed with regards to node aspects comparing Alfresco default vs. aldica Ignite-backed caches. The Alfresco default caches should have been able to reuse the same / identical QName instance for the custom benchmark model aspect, but instead there were many instances of the same aspect QName - spot checks have found that a single instance could be reused only in around 150 cache entries. In contrast to this, the QName for a common model aspect was properly reused for each of the 1,000,000 nodes.

The major difference between those two aspect QName instances is that the common model aspect will have been loaded during the initial system start, while the benchmark model aspect will only be loaded as part of the node loading, concurrent batch processing. This particular difference indicates that there might be an issue in Alfresco's transactional cache handling, causing the immutableEntityCache / immutableEntitySharedCache, which handles - amongst other entities - QName resolution, to not stabilise and load / evict the same QName cache entry over and over again.

Code analysis of the TransactionalCache class and transactional cache configurations verified that assumption. The post-commit handling of new cache entries specifically contains an execution branch that will remove ANY cache entry for a specific cache key, if that entry was put by a concurrent transaction and none of two special short-cut conditions apply. These conditions are:

• the cache's isMutable flag is not set, indicating that cache values will never be replaced once set
• the cache's allowEqualsCheck flag is set and the value put by the concurrent transaction equals the new value in the transaction currently in post-commit processing

For the immutableEntityCache is configured with isMutable as false, because the EntityLookupCache cache layer above it uses sentinel values for not found / null values which can be replaced once a real value is created in the DB, and with allowEqualsCheck left to its default value of false. This means that any concurrent process which may directly or indirectly load QName or other immutable entity instances runs the risk of both not actually putting its loaded value into the shared cache and also removing any value put there by another transaction.
This is a serious flaw. This issue should be reported to Alfresco and potentially be "fixed" by our module via some smart bean post processing, setting the proper configuration flags on applicable transactional caches (not just immutableEntityCache).

The aldica Ignite backed caches should not be significantly affected by this issue, as those are not able to reuse objects on the heap in the first place. The only side effect those caches may experience is a slightly more fragmented, slightly large data region backing the immutableEntitySharedCache, as the same value will be written and removed over and over again. Ideally, Ignite will just refill the empty memory space left by a removed entry, and not need to grow the region for this too much.

Upon re-execution of the benchmark with a special provision to pre-load the QName instances before the concurrent node load operation, it has been determined that the overall results of the benchmark with default Alfresco caches are not skewed by this behaviour. The sizes of caches as well as the entire heap in general are virtually identical. It is extremely likely that the application of G1 GC String deduplication has significantly reduced the impact of this issue, leaving only the basic QName object pointers as additional overhead cost.

Rework Tomcat Getting Started documentation

The Getting Started documentation for Tomcat is out of date and incomplate. It does not yet reflect the change in the Acosix Alfresco Utility dependency to only require the core module, and is lacking any Share-tier aspects. It thus needs to be reworked.

Cache clear on invalidating cache does not clear cache on remote nodes

BUG

Expected behavior

When a cache clear operation is invoked on an invalidating cache it is expected that the clear operation is replicated as an invalidation to any other member of the cluster.

Actual behavior

Caches on other members are only partially cleared - specifically only the entries for cache keys loaded into the cache on the member on which the clear operation was invoked are cleared.

Steps to reproduce the behavior

• start cluster with at least two members
• work with both members, loading different sets of data (e.g. by accessing/viewing alfresco-access audit data for different nodes)
• use OOTBee Support Tools to trigger a cache clear operation on an invalidating cache (e.g. cache.propertyValueCache)
• use OOTBee Support Tools to check cache size on other cluster member and verify it is empty

Additional details (analysis so far, log statements, references, etc.)

Tell us about your environment

Customer / user also has installed an early state of the https://github.com/Acosix/alfresco-audit module which performs incremental cleanup of alf_prop_xx database tables. This process has unveiled issues with foreign key constraint violations, as a result of caches not being properly synchronised. This is partly due to the customer not having the fix made in Acosix/alfresco-audit@3c98ab4 - but the subsequent attempt to manually clear the cache via OOTBee Support Tools then showed the issue with clear not being properly synchronised with invalidating aldica caches.

Error: Onheap cache must be enabled

The latest module was deployed and I get the following error when repo1 starts up:

2019-06-21 11:56:46,516  INFO  [ignite.lifecycle.SpringIgniteLifecycleBean] [localhost-startStop-1] Starting Ignite instance repositoryGrid
 [11:56:46] OS: Linux 4.15.0-1040-aws amd64
[11:56:46] VM information: OpenJDK Runtime Environment 1.8.0_212-8u212-b03-0ubuntu1.18.04.1-b03 Oracle Corporation OpenJDK 64-Bit Server VM 25.212-b03
2019-06-21 11:56:46,645  WARN  [ignite.internal.IgniteKernal%repositoryGrid] [localhost-startStop-1] Please set system property '-Djava.net.preferIPv4Stack=true' to avoid possible problems in mixed environments.
 [11:56:46] Please set system property '-Djava.net.preferIPv4Stack=true' to avoid possible problems in mixed environments.
[11:56:46] Configured plugins:
[11:56:46]   ^-- SimpleSecurityPlugin (org.aldica:aldica-common-ignite) 1.0.0.0
[11:56:46]   ^-- Copyright 2019 Acosix GmbH, Copyright 2019 MAGENTA ApS
[11:56:46] 
[11:56:46] Configured failure handler: [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]]]
2019-06-21 11:56:46,746  WARN  [communication.tcp.TcpCommunicationSpi] [localhost-startStop-1] Failure detection timeout will be ignored (one of SPI parameters has been set explicitly)
 [11:56:46] Security status [authentication=on, tls/ssl=off]
2019-06-21 11:56:47,102  WARN  [cache.persistence.IgniteCacheDatabaseSharedManager] [localhost-startStop-1] DataRegionConfiguration.maxWalArchiveSize instead DataRegionConfiguration.walHistorySize would be used for removing old archive wal files
 2019-06-21 11:56:47,104  WARN  [ignite.discovery.CredentialsAwareTcpDiscoverySpi] [localhost-startStop-1] Failure detection timeout will be ignored (one of SPI parameters has been set explicitly)
 2019-06-21 11:56:47,140  WARN  [dht.topology.PartitionsEvictManager] [localhost-startStop-1] Logging at INFO level without checking if INFO level is enabled: Evict partition permits=2
 2019-06-21 11:56:47,325  INFO  [ignite.discovery.CredentialsAwareTcpDiscoverySpi] [localhost-startStop-1] Connection check threshold is calculated: 2000
 2019-06-21 11:56:47,328  INFO  [ignite.discovery.CredentialsAwareTcpDiscoverySpi] [localhost-startStop-1] Successfully bound to TCP port [port=47110, localHost=repo1/172.31.44.230, locNodeId=c39b6c7e-ba8f-4036-bac5-cbd4ea2015f3]
 2019-06-21 11:56:47,451  WARN  [ignite.internal.IgniteKernal%repositoryGrid] [localhost-startStop-1] Nodes started on local machine require more than 80% of physical RAM what can lead to significant slowdown due to swapping (please decrease JVM heap size, data region size or checkpoint buffer size) [required=20520MB, available=7681MB]
 [11:56:47] Nodes started on local machine require more than 80% of physical RAM what can lead to significant slowdown due to swapping (please decrease JVM heap size, data region size or checkpoint buffer size) [required=20520MB, available=7681MB]
[11:56:47] To start Console Management & Monitoring run ignitevisorcmd.{sh|bat}
[11:56:47] Data Regions Configured:
[11:56:47]   ^-- repositoryGrid.defaultDataRegion [initSize=1,0 GiB, maxSize=16,0 GiB, persistence=false]
[11:56:47] 
[11:56:47] Ignite node started OK (id=c39b6c7e, instance name=repositoryGrid)
[11:56:47] Topology snapshot [ver=1, locNode=c39b6c7e, servers=1, clients=0, state=ACTIVE, CPUs=2, offheap=16.0GB, heap=4.0GB]
2019-06-21 11:56:47,957  INFO  [managers.discovery.GridDiscoveryManager] [localhost-startStop-1] Topology snapshot [ver=1, locNode=c39b6c7e, servers=1, clients=0, state=ACTIVE, CPUs=2, offheap=16.0GB, heap=4.0GB]
 2019-06-21 11:56:47,958  INFO  [ignite.lifecycle.SpringIgniteLifecycleBean] [localhost-startStop-1] Ignite instance repositoryGrid currently has 1 active nodes on addresses [172.31.44.230]
 2019-06-21 11:56:48,741  WARN  [internal.binary.BinaryContext] [localhost-startStop-1] Class "org.alfresco.service.cmr.repository.MLText" cannot be serialized using BinaryMarshaller because it either implements Externalizable interface or have writeObject/readObject methods. OptimizedMarshaller will be used instead and class instances will be deserialized on the server. Please ensure that all nodes have this class in classpath. To enable binary serialization either implement Binarylizable interface or set explicit serializer using BinaryTypeConfiguration.setSerializer() method.
 2019-06-21 11:56:49,194  ERROR [processors.cache.CacheAffinitySharedManager] [exchange-worker-#36%repositoryGrid%] Failed to initialize cache. Will try to rollback cache start routine. [cacheName=lockStore]
 class org.apache.ignite.IgniteCheckedException: Onheap cache must be enabled if eviction policy is configured [cacheName=lockStore]
	at org.apache.ignite.internal.processors.cache.GridCacheProcessor.validate(GridCacheProcessor.java:578)
	at org.apache.ignite.internal.processors.cache.GridCacheProcessor.createCache(GridCacheProcessor.java:1560)
	at org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:2146)
	at org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.processCacheStartRequests(CacheAffinitySharedManager.java:898)
	at org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onCacheChangeRequest(CacheAffinitySharedManager.java:798)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onCacheChangeRequest(GridDhtPartitionsExchangeFuture.java:1234)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:741)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2668)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2540)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
	at java.lang.Thread.run(Thread.java:748)
2019-06-21 11:56:49,198  INFO  [ignite.discovery.MemberTcpDiscoveryIpFinder] [localhost-startStop-1] Cleaning up after Ignite operation error
 javax.cache.CacheException: class org.apache.ignite.IgniteCheckedException: Onheap cache must be enabled if eviction policy is configured [cacheName=lockStore]
	at org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1337)
	at org.apache.ignite.internal.IgniteKernal.getOrCreateCache0(IgniteKernal.java:3023)
	at org.apache.ignite.internal.IgniteKernal.getOrCreateCache(IgniteKernal.java:2992)
	at org.aldica.repo.ignite.lock.LockStoreFactoryImpl.createIgniteLockStore(LockStoreFactoryImpl.java:188)
	at org.aldica.repo.ignite.lock.LockStoreFactoryImpl.createLockStore(LockStoreFactoryImpl.java:153)
	at org.aldica.repo.ignite.lock.LockStoreFactoryImpl$LockStoreInvokerWithLazySwapSupport.invoke(LockStoreFactoryImpl.java:240)
	at com.sun.proxy.$Proxy33.afterInstanceStartup(Unknown Source)
	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.lambda$null$1(SpringIgniteLifecycleBean.java:115)
	at java.util.LinkedHashMap$LinkedValues.forEach(LinkedHashMap.java:608)
	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.lambda$initializeGrid$2(SpringIgniteLifecycleBean.java:114)
	at org.aldica.common.ignite.context.ExternalContext.withExternalContext(ExternalContext.java:112)
	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.initializeGrid(SpringIgniteLifecycleBean.java:100)
	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.onBootstrap(SpringIgniteLifecycleBean.java:59)
	at org.springframework.extensions.surf.util.AbstractLifecycleBean.onApplicationEvent(AbstractLifecycleBean.java:56)
	at org.alfresco.repo.management.SafeApplicationEventMulticaster.multicastEventInternal(SafeApplicationEventMulticaster.java:221)
	at org.alfresco.repo.management.SafeApplicationEventMulticaster.multicastEvent(SafeApplicationEventMulticaster.java:186)
	at org.alfresco.repo.management.SafeApplicationEventMulticaster.multicastEvent(SafeApplicationEventMulticaster.java:206)
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:398)
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:355)
	at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:882)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549)
	at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:400)
	at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:291)
	at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:103)
	at org.alfresco.web.app.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:70)
	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4792)
	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5256)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:754)
	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:730)
	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:734)
	at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:985)
	at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1857)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: class org.apache.ignite.IgniteCheckedException: Onheap cache must be enabled if eviction policy is configured [cacheName=lockStore]
	at org.apache.ignite.internal.processors.cache.GridCacheProcessor.validate(GridCacheProcessor.java:578)
	at org.apache.ignite.internal.processors.cache.GridCacheProcessor.createCache(GridCacheProcessor.java:1560)
	at org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:2146)
	at org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.processCacheStartRequests(CacheAffinitySharedManager.java:898)
	at org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onCacheChangeRequest(CacheAffinitySharedManager.java:798)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onCacheChangeRequest(GridDhtPartitionsExchangeFuture.java:1234)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:741)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2668)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2540)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
	... 1 more
2019-06-21 11:56:49,228  ERROR [web.context.ContextLoader] [localhost-startStop-1] Context initialization failed
 javax.cache.CacheException: class org.apache.ignite.IgniteCheckedException: Onheap cache must be enabled if eviction policy is configured [cacheName=lockStore]
	at org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1337)
	at org.apache.ignite.internal.IgniteKernal.getOrCreateCache0(IgniteKernal.java:3023)
	at org.apache.ignite.internal.IgniteKernal.getOrCreateCache(IgniteKernal.java:2992)
	at org.aldica.repo.ignite.lock.LockStoreFactoryImpl.createIgniteLockStore(LockStoreFactoryImpl.java:188)
	at org.aldica.repo.ignite.lock.LockStoreFactoryImpl.createLockStore(LockStoreFactoryImpl.java:153)
	at org.aldica.repo.ignite.lock.LockStoreFactoryImpl$LockStoreInvokerWithLazySwapSupport.invoke(LockStoreFactoryImpl.java:240)
	at com.sun.proxy.$Proxy33.afterInstanceStartup(Unknown Source)
	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.lambda$null$1(SpringIgniteLifecycleBean.java:115)
	at java.util.LinkedHashMap$LinkedValues.forEach(LinkedHashMap.java:608)
	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.lambda$initializeGrid$2(SpringIgniteLifecycleBean.java:114)
	at org.aldica.common.ignite.context.ExternalContext.withExternalContext(ExternalContext.java:112)
	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.initializeGrid(SpringIgniteLifecycleBean.java:100)
	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.onBootstrap(SpringIgniteLifecycleBean.java:59)
	at org.springframework.extensions.surf.util.AbstractLifecycleBean.onApplicationEvent(AbstractLifecycleBean.java:56)
	at org.alfresco.repo.management.SafeApplicationEventMulticaster.multicastEventInternal(SafeApplicationEventMulticaster.java:221)
	at org.alfresco.repo.management.SafeApplicationEventMulticaster.multicastEvent(SafeApplicationEventMulticaster.java:186)
	at org.alfresco.repo.management.SafeApplicationEventMulticaster.multicastEvent(SafeApplicationEventMulticaster.java:206)
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:398)
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:355)
	at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:882)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549)
	at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:400)
	at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:291)
	at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:103)
	at org.alfresco.web.app.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:70)
	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4792)
	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5256)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:754)
	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:730)
	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:734)
	at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:985)
	at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1857)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: class org.apache.ignite.IgniteCheckedException: Onheap cache must be enabled if eviction policy is configured [cacheName=lockStore]
	at org.apache.ignite.internal.processors.cache.GridCacheProcessor.validate(GridCacheProcessor.java:578)
	at org.apache.ignite.internal.processors.cache.GridCacheProcessor.createCache(GridCacheProcessor.java:1560)
	at org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:2146)
	at org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.processCacheStartRequests(CacheAffinitySharedManager.java:898)
	at org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onCacheChangeRequest(CacheAffinitySharedManager.java:798)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onCacheChangeRequest(GridDhtPartitionsExchangeFuture.java:1234)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:741)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2668)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2540)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
	... 1 more
jun. 21, 2019 11:56:49 AM org.apache.catalina.core.StandardContext listenerStart
SEVERE: Exception sending context initialized event to listener instance of class [org.alfresco.web.app.ContextLoaderListener]
javax.cache.CacheException: class org.apache.ignite.IgniteCheckedException: Onheap cache must be enabled if eviction policy is configured [cacheName=lockStore]
	at org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1337)
	at org.apache.ignite.internal.IgniteKernal.getOrCreateCache0(IgniteKernal.java:3023)
	at org.apache.ignite.internal.IgniteKernal.getOrCreateCache(IgniteKernal.java:2992)
	at org.aldica.repo.ignite.lock.LockStoreFactoryImpl.createIgniteLockStore(LockStoreFactoryImpl.java:188)
	at org.aldica.repo.ignite.lock.LockStoreFactoryImpl.createLockStore(LockStoreFactoryImpl.java:153)
	at org.aldica.repo.ignite.lock.LockStoreFactoryImpl$LockStoreInvokerWithLazySwapSupport.invoke(LockStoreFactoryImpl.java:240)
	at com.sun.proxy.$Proxy33.afterInstanceStartup(Unknown Source)
	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.lambda$null$1(SpringIgniteLifecycleBean.java:115)
	at java.util.LinkedHashMap$LinkedValues.forEach(LinkedHashMap.java:608)
	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.lambda$initializeGrid$2(SpringIgniteLifecycleBean.java:114)
	at org.aldica.common.ignite.context.ExternalContext.withExternalContext(ExternalContext.java:112)
	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.initializeGrid(SpringIgniteLifecycleBean.java:100)
	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.onBootstrap(SpringIgniteLifecycleBean.java:59)
	at org.springframework.extensions.surf.util.AbstractLifecycleBean.onApplicationEvent(AbstractLifecycleBean.java:56)
	at org.alfresco.repo.management.SafeApplicationEventMulticaster.multicastEventInternal(SafeApplicationEventMulticaster.java:221)
	at org.alfresco.repo.management.SafeApplicationEventMulticaster.multicastEvent(SafeApplicationEventMulticaster.java:186)
	at org.alfresco.repo.management.SafeApplicationEventMulticaster.multicastEvent(SafeApplicationEventMulticaster.java:206)
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:398)
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:355)
	at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:882)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549)
	at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:400)
	at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:291)
	at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:103)
	at org.alfresco.web.app.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:70)
	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4792)
	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5256)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:754)
	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:730)
	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:734)
	at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:985)
	at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1857)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: class org.apache.ignite.IgniteCheckedException: Onheap cache must be enabled if eviction policy is configured [cacheName=lockStore]
	at org.apache.ignite.internal.processors.cache.GridCacheProcessor.validate(GridCacheProcessor.java:578)
	at org.apache.ignite.internal.processors.cache.GridCacheProcessor.createCache(GridCacheProcessor.java:1560)
	at org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:2146)
	at org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.processCacheStartRequests(CacheAffinitySharedManager.java:898)
	at org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onCacheChangeRequest(CacheAffinitySharedManager.java:798)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onCacheChangeRequest(GridDhtPartitionsExchangeFuture.java:1234)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:741)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2668)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2540)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
	... 1 more

jun. 21, 2019 11:56:49 AM org.apache.catalina.core.StandardContext listenerStart
SEVERE: Exception sending context initialized event to listener instance of class [org.alfresco.repo.webdav.WebDAVSessionListener]
javax.cache.CacheException: class org.apache.ignite.IgniteCheckedException: Onheap cache must be enabled if eviction policy is configured [cacheName=lockStore]
	at org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1337)
	at org.apache.ignite.internal.IgniteKernal.getOrCreateCache0(IgniteKernal.java:3023)
	at org.apache.ignite.internal.IgniteKernal.getOrCreateCache(IgniteKernal.java:2992)
	at org.aldica.repo.ignite.lock.LockStoreFactoryImpl.createIgniteLockStore(LockStoreFactoryImpl.java:188)
	at org.aldica.repo.ignite.lock.LockStoreFactoryImpl.createLockStore(LockStoreFactoryImpl.java:153)
	at org.aldica.repo.ignite.lock.LockStoreFactoryImpl$LockStoreInvokerWithLazySwapSupport.invoke(LockStoreFactoryImpl.java:240)
	at com.sun.proxy.$Proxy33.afterInstanceStartup(Unknown Source)
	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.lambda$null$1(SpringIgniteLifecycleBean.java:115)
	at java.util.LinkedHashMap$LinkedValues.forEach(LinkedHashMap.java:608)
	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.lambda$initializeGrid$2(SpringIgniteLifecycleBean.java:114)
	at org.aldica.common.ignite.context.ExternalContext.withExternalContext(ExternalContext.java:112)
	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.initializeGrid(SpringIgniteLifecycleBean.java:100)
	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.onBootstrap(SpringIgniteLifecycleBean.java:59)
	at org.springframework.extensions.surf.util.AbstractLifecycleBean.onApplicationEvent(AbstractLifecycleBean.java:56)
	at org.alfresco.repo.management.SafeApplicationEventMulticaster.multicastEventInternal(SafeApplicationEventMulticaster.java:221)
	at org.alfresco.repo.management.SafeApplicationEventMulticaster.multicastEvent(SafeApplicationEventMulticaster.java:186)
	at org.alfresco.repo.management.SafeApplicationEventMulticaster.multicastEvent(SafeApplicationEventMulticaster.java:206)
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:398)
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:355)
	at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:882)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549)
	at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:400)
	at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:291)
	at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:103)
	at org.alfresco.web.app.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:70)
	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4792)
	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5256)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:754)
	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:730)
	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:734)
	at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:985)
	at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1857)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: class org.apache.ignite.IgniteCheckedException: Onheap cache must be enabled if eviction policy is configured [cacheName=lockStore]
	at org.apache.ignite.internal.processors.cache.GridCacheProcessor.validate(GridCacheProcessor.java:578)
	at org.apache.ignite.internal.processors.cache.GridCacheProcessor.createCache(GridCacheProcessor.java:1560)
	at org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:2146)
	at org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.processCacheStartRequests(CacheAffinitySharedManager.java:898)
	at org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onCacheChangeRequest(CacheAffinitySharedManager.java:798)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onCacheChangeRequest(GridDhtPartitionsExchangeFuture.java:1234)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:741)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2668)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2540)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
	... 1 more

21-Jun-2019 11:56:49.233 SEVERE [localhost-startStop-1] org.apache.catalina.core.StandardContext.startInternal One or more listeners failed to start. Full details will be found in the appropriate container log file
21-Jun-2019 11:56:49.241 SEVERE [localhost-startStop-1] org.apache.catalina.core.StandardContext.startInternal Context [/alfresco] startup failed due to previous errors
jun. 21, 2019 11:56:49 AM org.apache.catalina.core.ApplicationContext log
INFO: Closing Spring root WebApplicationContext
2019-06-21 11:56:49,274  INFO  [ignite.lifecycle.SpringIgniteLifecycleBean] [localhost-startStop-1] Closing Ignite isntance repositoryGrid
 2019-06-21 11:56:49,275  INFO  [ignite.discovery.MemberTcpDiscoveryIpFinder] [localhost-startStop-1] Ignite instance repositoryGrid is about to be shut down - removing local address(es) from database
 2019-06-21 11:56:49,304  ERROR [ignite.internal.IgniteKernal%repositoryGrid] [localhost-startStop-1] Failed to stop component (ignoring): GridProcessorAdapter []
 java.lang.UnsupportedOperationException
	at org.jsr166.ConcurrentLinkedHashMap.clear(ConcurrentLinkedHashMap.java:1551)
	at org.apache.ignite.internal.processors.job.GridJobProcessor.stop(GridJobProcessor.java:264)
	at org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:2356)
	at org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:2228)
	at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2612)
	at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2575)
	at org.apache.ignite.internal.IgnitionEx.stop(IgnitionEx.java:379)
	at org.apache.ignite.Ignition.stop(Ignition.java:225)
	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.lambda$shutdownGrid$5(SpringIgniteLifecycleBean.java:140)
	at org.aldica.common.ignite.context.ExternalContext.withExternalContext(ExternalContext.java:112)
	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.shutdownGrid(SpringIgniteLifecycleBean.java:134)
	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.onShutdown(SpringIgniteLifecycleBean.java:87)
	at org.springframework.extensions.surf.util.AbstractLifecycleBean.onApplicationEvent(AbstractLifecycleBean.java:67)
	at org.alfresco.repo.management.SafeApplicationEventMulticaster.multicastEventInternal(SafeApplicationEventMulticaster.java:221)
	at org.alfresco.repo.management.SafeApplicationEventMulticaster.multicastEvent(SafeApplicationEventMulticaster.java:191)
	at org.alfresco.repo.management.SafeApplicationEventMulticaster.multicastEvent(SafeApplicationEventMulticaster.java:206)
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:398)
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:355)
	at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:994)
	at org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:961)
	at org.springframework.web.context.ContextLoader.closeWebApplicationContext(ContextLoader.java:516)
	at org.springframework.web.context.ContextLoaderListener.contextDestroyed(ContextLoaderListener.java:112)
	at org.apache.catalina.core.StandardContext.listenerStop(StandardContext.java:4839)
	at org.apache.catalina.core.StandardContext.stopInternal(StandardContext.java:5478)
	at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:226)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:154)
	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:754)
	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:730)
	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:734)
	at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:985)
	at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1857)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
[11:56:49] Ignite node stopped wih ERRORS [name=repositoryGrid, uptime=00:00:01.434]
2019-06-21 11:56:49,392  INFO  [ignite.lifecycle.SpringIgniteLifecycleBean] [localhost-startStop-1] Closed Ignite instance repositoryGrid
 2019-06-21 11:56:49,392  WARN  [context.support.XmlWebApplicationContext] [localhost-startStop-1] Exception thrown from ApplicationListener handling ContextClosedEvent
 java.lang.NullPointerException
	at org.alfresco.repo.workflow.activiti.ActivitiEngineInitializer.onShutdown(ActivitiEngineInitializer.java:65)
	at org.springframework.extensions.surf.util.AbstractLifecycleBean.onApplicationEvent(AbstractLifecycleBean.java:67)
	at org.alfresco.repo.management.SafeApplicationEventMulticaster.multicastEventInternal(SafeApplicationEventMulticaster.java:221)
	at org.alfresco.repo.management.SafeApplicationEventMulticaster.multicastEvent(SafeApplicationEventMulticaster.java:191)
	at org.alfresco.repo.management.SafeApplicationEventMulticaster.multicastEvent(SafeApplicationEventMulticaster.java:206)
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:398)
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:355)
	at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:994)
	at org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:961)
	at org.springframework.web.context.ContextLoader.closeWebApplicationContext(ContextLoader.java:516)
	at org.springframework.web.context.ContextLoaderListener.contextDestroyed(ContextLoaderListener.java:112)
	at org.apache.catalina.core.StandardContext.listenerStop(StandardContext.java:4839)
	at org.apache.catalina.core.StandardContext.stopInternal(StandardContext.java:5478)
	at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:226)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:154)
	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:754)
	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:730)
	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:734)
	at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:985)
	at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1857)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
2019-06-21 11:56:49,523  INFO  [management.subsystems.ChildApplicationContextFactory] [localhost-startStop-1] Stopping 'OOoJodconverter' subsystem, ID: [OOoJodconverter, default]
 2019-06-21 11:56:49,524  INFO  [management.subsystems.ChildApplicationContextFactory] [localhost-startStop-1] Stopped 'OOoJodconverter' subsystem, ID: [OOoJodconverter, default]
 2019-06-21 11:56:49,524  INFO  [management.subsystems.ChildApplicationContextFactory] [localhost-startStop-1] Stopping 'ContentStore' subsystem, ID: [ContentStore, managed, unencrypted]
 2019-06-21 11:56:49,525  INFO  [management.subsystems.ChildApplicationContextFactory] [localhost-startStop-1] Stopped 'ContentStore' subsystem, ID: [ContentStore, managed, unencrypted]
 2019-06-21 11:56:49,849  INFO  [management.subsystems.ChildApplicationContextFactory] [localhost-startStop-1] Stopping 'Search' subsystem, ID: [Search, managed, solr6]
 2019-06-21 11:56:50,115  INFO  [management.subsystems.ChildApplicationContextFactory] [localhost-startStop-1] Stopped 'Search' subsystem, ID: [Search, managed, solr6]
 2019-06-21 11:56:50,200  INFO  [management.subsystems.ChildApplicationContextFactory] [localhost-startStop-1] Stopping 'thirdparty' subsystem, ID: [thirdparty, default]
 2019-06-21 11:56:50,200  INFO  [management.subsystems.ChildApplicationContextFactory] [localhost-startStop-1] Stopped 'thirdparty' subsystem, ID: [thirdparty, default]
 2019-06-21 11:56:50,202  INFO  [management.subsystems.ChildApplicationContextFactory] [localhost-startStop-1] Stopping 'Authentication' subsystem, ID: [Authentication, managed, alfrescoNtlm1]
 2019-06-21 11:56:50,202  INFO  [management.subsystems.ChildApplicationContextFactory] [localhost-startStop-1] Stopped 'Authentication' subsystem, ID: [Authentication, managed, alfrescoNtlm1]
 2019-06-21 11:56:50,202  INFO  [management.subsystems.ChildApplicationContextFactory] [localhost-startStop-1] Stopping 'sysAdmin' subsystem, ID: [sysAdmin, default]
 2019-06-21 11:56:50,202  INFO  [management.subsystems.ChildApplicationContextFactory] [localhost-startStop-1] Stopped 'sysAdmin' subsystem, ID: [sysAdmin, default]
 21-Jun-2019 11:56:50.215 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [alfresco] appears to have started a thread named [QuartzScheduler_Worker-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
21-Jun-2019 11:56:50.216 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [alfresco] appears to have started a thread named [QuartzScheduler_Worker-2] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
21-Jun-2019 11:56:50.216 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [alfresco] appears to have started a thread named [Timer-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 java.lang.Object.wait(Object.java:502)
 java.util.TimerThread.mainLoop(Timer.java:526)
 java.util.TimerThread.run(Timer.java:505)
21-Jun-2019 11:56:50.216 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [alfresco] appears to have started a thread named [Timer-2] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 java.lang.Object.wait(Object.java:502)
 java.util.TimerThread.mainLoop(Timer.java:526)
 java.util.TimerThread.run(Timer.java:505)
21-Jun-2019 11:56:50.217 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [alfresco] appears to have started a thread named [Timer-3] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 java.lang.Object.wait(Object.java:502)
 java.util.TimerThread.mainLoop(Timer.java:526)
 java.util.TimerThread.run(Timer.java:505)
21-Jun-2019 11:56:50.218 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [alfresco] appears to have started a thread named [Timer-4] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 java.lang.Object.wait(Object.java:502)
 java.util.TimerThread.mainLoop(Timer.java:526)
 java.util.TimerThread.run(Timer.java:505)
21-Jun-2019 11:56:50.218 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [alfresco] appears to have started a thread named [Timer-5] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 java.lang.Object.wait(Object.java:502)
 java.util.TimerThread.mainLoop(Timer.java:526)
 java.util.TimerThread.run(Timer.java:505)
21-Jun-2019 11:56:50.218 WARNING [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [alfresco] appears to have started a thread named [QuartzScheduler_QuartzSchedulerThread] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 org.quartz.core.QuartzSchedulerThread.run(QuartzSchedulerThread.java:427)
21-Jun-2019 11:56:50.219 SEVERE [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [alfresco] created a ThreadLocal with key of type [java.lang.ThreadLocal.SuppliedThreadLocal] (value [java.lang.ThreadLocal$SuppliedThreadLocal@4fcec43b]) and a value of type [org.alfresco.util.transaction.TransactionSupportUtil.ResourcesHolder] (value [org.alfresco.util.transaction.TransactionSupportUtil$ResourcesHolder@8e0a827]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
21-Jun-2019 11:56:50.219 SEVERE [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [alfresco] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@7667843a]) and a value of type [org.apache.ignite.internal.marshaller.optimized.OptimizedObjectStreamRegistry.StreamHolder] (value [org.apache.ignite.internal.marshaller.optimized.OptimizedObjectStreamRegistry$StreamHolder@6b58d40e]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
21-Jun-2019 11:56:50.220 SEVERE [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [alfresco] created a ThreadLocal with key of type [org.apache.ignite.internal.util.tostring.GridToStringBuilder$1] (value [org.apache.ignite.internal.util.tostring.GridToStringBuilder$1@77dca2c7]) and a value of type [java.util.LinkedList] (value [[org.apache.ignite.internal.util.tostring.GridToStringThreadLocal@39936a28]]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
21-Jun-2019 11:56:50.220 SEVERE [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [alfresco] created a ThreadLocal with key of type [org.alfresco.repo.template.QNameAwareObjectWrapper$1] (value [org.alfresco.repo.template.QNameAwareObjectWrapper$1@3ee18dd3]) and a value of type [org.alfresco.repo.template.QNameAwareObjectWrapper$1$1] (value [org.alfresco.repo.template.QNameAwareObjectWrapper$1$1@364f11ed]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
21-Jun-2019 11:56:50.220 SEVERE [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [alfresco] created a ThreadLocal with key of type [org.apache.ignite.internal.binary.BinaryThreadLocalContext$1] (value [org.apache.ignite.internal.binary.BinaryThreadLocalContext$1@4e6a160]) and a value of type [org.apache.ignite.internal.binary.BinaryThreadLocalContext] (value [org.apache.ignite.internal.binary.BinaryThreadLocalContext@4a680868]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
21-Jun-2019 11:56:50.220 SEVERE [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [alfresco] created a ThreadLocal with key of type [org.apache.ignite.internal.util.tostring.GridToStringBuilder$2] (value [org.apache.ignite.internal.util.tostring.GridToStringBuilder$2@1801fd34]) and a value of type [org.apache.ignite.internal.util.tostring.SBLengthLimit] (value [org.apache.ignite.internal.util.tostring.SBLengthLimit@295db2f5]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
21-Jun-2019 11:56:50.221 SEVERE [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [alfresco] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@7367f031]) and a value of type [org.alfresco.util.CachingDateFormat] (value [yyyy-MM-dd]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
21-Jun-2019 11:56:50.221 SEVERE [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [alfresco] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@14c090c5]) and a value of type [org.apache.ignite.internal.binary.streams.BinaryMemoryAllocatorChunk] (value [org.apache.ignite.internal.binary.streams.BinaryMemoryAllocatorChunk@278f5f89]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
21-Jun-2019 11:56:50.221 SEVERE [localhost-startStop-1] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [alfresco] created a ThreadLocal with key of type [org.alfresco.repo.template.QNameAwareObjectWrapper$1] (value [org.alfresco.repo.template.QNameAwareObjectWrapper$1@28f5ebf1]) and a value of type [org.alfresco.repo.template.QNameAwareObjectWrapper$1$1] (value [org.alfresco.repo.template.QNameAwareObjectWrapper$1$1@38a31092]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
21-Jun-2019 11:56:50.231 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive [/opt/apache-tomcat-8.5.34/webapps/alfresco.war] has finished in [71,653] ms
21-Jun-2019 11:56:50.233 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deploying web application archive [/opt/apache-tomcat-8.5.34/webapps/ROOT.war]
21-Jun-2019 11:56:50.235 WARNING [localhost-startStop-1] org.apache.catalina.startup.SetContextPropertiesRule.begin [SetContextPropertiesRule]{Context} Setting property 'debug' to '100' did not find a matching property.
21-Jun-2019 11:56:50.302 INFO [localhost-startStop-1] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
21-Jun-2019 11:56:50.315 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive [/opt/apache-tomcat-8.5.34/webapps/ROOT.war] has finished in [81] ms
21-Jun-2019 11:56:50.315 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deploying web application archive [/opt/apache-tomcat-8.5.34/webapps/_vti_bin.war]
21-Jun-2019 11:56:50.415 INFO [localhost-startStop-1] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
21-Jun-2019 11:56:50.418 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive [/opt/apache-tomcat-8.5.34/webapps/_vti_bin.war] has finished in [103] ms
21-Jun-2019 11:56:50.425 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8080"]
21-Jun-2019 11:56:50.440 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["ajp-nio-8009"]
21-Jun-2019 11:56:50.448 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 71919 ms

Is this a big or am I missing a configurations?

Reconsider unit test classification default vs. full

With the recent addition of serialisation optimisations and the associated unit tests checking both correctness and efficiency, the previous classification of what expensive unit tests are no longer really feels right. Running the full build and test suite can now take several minutes as the efficiency tests for various optimisations cumulatively take a big chunk of time.
We should revisit the unit test classification and aim to distinguish between actual functional tests and additional performance / specialised tests that may not have to run on every build.

Missing Maven dependency (acosix utility 1.1.0.0)

We (@Skeen and I) are current working with the Dockerfiles related to PR #13, and one of these contain the line:

mvn dependency:copy -Dartifact=de.acosix.alfresco.utility:de.acosix.alfresco.utility.core.repo:1.1.0.0:amp -DoutputDirectory=target -B -T2C

which fails with the message

[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  4.057 s (Wall Clock)
[INFO] Finished at: 2020-02-06T15:18:50+01:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:3.1.1:copy (default-cli) on project aldica-parent: Unable to find/resolve artifact. Could not find artifact de.acosix.alfresco.utility:de.acosix.alfresco.utility.core.repo:amp:1.1.0.0 in ossrh (https://oss.sonatype.org/content/repositories/snapshots) -> [Help 1]

The corresponding command for version 1.0.8.0 of the artifact works fine, so it seems as if version 1.1.0.0 of the artifact is missing upstream?

Kubernetes deployment: update Helm charts

Currently, the Kubernetes deployment via Helm charts only works for Kubernetes version 1.15 due to issues with the upstream Helm charts (see this issue). If Alfresco chooses to fix this (e.g. by merging in the PR that addresses the problem) that would be great, but otherwise we probably will need to deal with it ourselves.

In general, the upstream Helm charts have some issues with name prefixing, i.e. it is currently not possible to run more than one Alfresco cluster in the same Kubernetes cluster. We should make a PR to the upstream repo addressing these issues.

Prepare for potential removal of Ignite local caches

As indicated in one exchange on the Ignite mailing list, local caches are not really treated as first grade citizens, e.g. not tested as extensively, and are being considered for a potential removal in future versions. We currently use local caches for all caches which Alfresco has configured as non-distributed by default. Using local Ignite caches instead of the Alfresco default does hold advantages for our use case, as this allows us to use off-heap caching, reducing our memory footprint. This would be hard / difficult to give up if these caches were removed in future Ignite versions.

We should investigate and prepare for this potential future, by:

• investigating, which Alfresco default caches could be safely turned from local-only to being distributed (i.e. where Alfresco simply choose a too defensive configuration by default)
• investigate which local caches barely ever hold significant amounts of data and could safely be switched to using default Alfresco on-heap caches (i.e. where the benifit of Ignite caches does not warrant using a cache that is not as extensively tested)
• implement any conversions / compatibility bridges required to enable local-only caches to be turned into distributed ones

Build error when using project parent 6.2.0 in aldica POM

I have tried to change the parent artifactId in the aldica project POM like this:

<parent>
        <groupId>de.acosix.alfresco.maven</groupId>
        <artifactId>de.acosix.alfresco.maven.project.parent-6.2.0</artifactId>
        <!-- artifactId>de.acosix.alfresco.maven.project.parent-6.1.2</artifactId -->
        <version>1.3.0-SNAPSHOT</version>
</parent>

However, this leads to an error when building the project with

$ mvn clean install -B -T2C -DskipTests

(the command used in docker/Dockerfile.repo in PR #13 )

The error is:

[INFO] Building jar: /home/andreas/alfresco/alfresco-community-open-grid/common/target/aldica-common-ignite-1.0.0.0-SNAPSHOT-javadoc.jar
[INFO] 
[INFO] >>> findbugs-maven-plugin:3.0.5:check (default) > :findbugs @ aldica-common-ignite >>>
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.5:findbugs (findbugs) @ aldica-common-ignite ---
[INFO] Downloading from ossrh-snapshots: http://oss.sonatype.org/content/repositories/snapshots/de/acosix/alfresco/maven/de.acosix.alfresco.maven.definitions/1.3.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloaded from ossrh-snapshots: http://oss.sonatype.org/content/repositories/snapshots/de/acosix/alfresco/maven/de.acosix.alfresco.maven.definitions/1.3.0-SNAPSHOT/maven-metadata.xml (1.0 kB at 1.9 kB/s)
[INFO] Downloading from ossrh-snapshots: http://oss.sonatype.org/content/repositories/snapshots/de/acosix/alfresco/maven/de.acosix.alfresco.maven.parent/1.3.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloaded from ossrh-snapshots: http://oss.sonatype.org/content/repositories/snapshots/de/acosix/alfresco/maven/de.acosix.alfresco.maven.parent/1.3.0-SNAPSHOT/maven-metadata.xml (632 B at 1.5 kB/s)
[INFO] Fork Value is true
     [java] The following errors occurred during analysis:
     [java]   Error scanning org/aldica/common/ignite/lifecycle/IgniteInstanceLifecycleAware for referenced classes
     [java]     java.lang.IllegalArgumentException
     [java]       At org.objectweb.asm.ClassReader.<init>(ClassReader.java:170)
     [java]       At org.objectweb.asm.ClassReader.<init>(ClassReader.java:153)
     ...
     [java]   Unable to get XClass for org/alfresco/util/ParameterCheck
     [java]     java.lang.IllegalArgumentException
     [java]       At org.objectweb.asm.ClassReader.<init>(ClassReader.java:170)
     [java]       At org.objectweb.asm.ClassReader.<init>(ClassReader.java:153)
     [java]       At edu.umd.cs.findbugs.asm.FBClassReader.<init>(FBClassReader.java:35)
     [java]       At edu.umd.cs.findbugs.classfile.engine.asm.ClassReaderAnalysisEngine.analyze(ClassReaderAnalysisEngine.java:48)
     [java]       At edu.umd.cs.findbugs.classfile.engine.asm.ClassReaderAnalysisEngine.analyze(ClassReaderAnalysisEngine.java:34)
     [java]       At edu.umd.cs.findbugs.classfile.impl.AnalysisCache.getClassAnalysis(AnalysisCache.java:262)
     [java]       At edu.umd.cs.findbugs.classfile.engine.ClassInfoAnalysisEngine.analyze(ClassInfoAnalysisEngine.java:75)
     [java]       At edu.umd.cs.findbugs.classfile.engine.ClassInfoAnalysisEngine.analyze(ClassInfoAnalysisEngine.java:38)
     [java]       At edu.umd.cs.findbugs.classfile.impl.AnalysisCache.getClassAnalysis(AnalysisCache.java:262)
     [java]       At edu.umd.cs.findbugs.ba.XFactory.getXClass(XFactory.java:673)
     [java]       At edu.umd.cs.findbugs.ba.AnalysisContext.setAppClassList(AnalysisContext.java:966)
     [java]       At edu.umd.cs.findbugs.FindBugs2.setAppClassList(FindBugs2.java:877)
     [java]       At edu.umd.cs.findbugs.FindBugs2.execute(FindBugs2.java:225)
     [java]       At edu.umd.cs.findbugs.FindBugs.runMain(FindBugs.java:402)
     [java]       At edu.umd.cs.findbugs.FindBugs2.main(FindBugs2.java:1200)
     [java] Exception in thread "main" edu.umd.cs.findbugs.NoClassesFoundToAnalyzeException: No classes found to analyze in /home/andreas/alfresco/alfresco-community-open-grid/common/target/classes */home/andreas/.m2/repository/org/apache/ignite/ignite-core/2.7.6/ignite-core-2.7.6.jar */home/andreas/.m2/repository/javax/cache/cache-api/1.0.0/cache-api-1.0.0.jar */home/andreas/.m2/repository/org/jetbrains/annotations/13.0/annotations-13.0.jar */home/andreas/.m2/repository/org/slf4j/slf4j-api/1.7.26/slf4j-api-1.7.26.jar */home/andreas/.m2/repository/org/apache/ignite/ignite-web/2.7.6/ignite-web-2.7.6.jar */home/andreas/.m2/repository/org/apache/

Are we missing a configuration parameter of some kind or is this a bug?

Binary Serialiser: optimisation of secondary key / entity types

In the state of binary serialisers after PR 34, only the primary key and entity / value types used to handle the core immutable entity, node and content data caches, as well as infrastructure types from Alfresco's cache framework have been optimised. Optimised serialiser variants should also be created for other key / entity types that may be used in Ignite-backed caches, e.g. CachePucKey, PropertyUniqueContextEntity, AclEntity etc...

Topology info not displayed correctly

When going to (e.g.) http://repo1:8080/alfresco and inspecting the info shown in the "Grid" section, the topology info is sometimes displayed incorrectly. Steps to reproduce (assuming our repos are named repo1 and repo2):

• Enable Aldica module and start repo1.
• Go to the "Grid" section (on repo1) and verity that there is exactly one node in the grid.
• Enable Aldica module and start repo2
• Go to the "Grid" section (on repo1) and verity that there are exactly two nodes in the grid.
• Stop repo2.
• When inspecting the info in the "Grid" section (on repo1) it now says Topology=3 and Modes=1 (is this correct?)
• Start repo2 again.
• When inspecting the info in the "Grid" section (on repo1) it still says Topology=3 and Modes=1
• When inspecting the info in the "Grid" section (on repo2) it says Topology=1 and Modes=1

So there seems to be a problem here with the grid.

Remove global cache statistics disablement-flag when Ignite 2.8 is integrated

In #8 we added a global flag to disable all cache statistics in the module to work around a bug in Ignite that affects any grids with more than 2 nodes. When Ignite 2.8 is released (which is slated to include a patch for CacheMetricsSnapshot serialisation) and integrated in the module, this global flag can/should be removed.

Forbiddenapis build problem

Changing the parent in the aldica project POM to

<parent>
    <groupId>de.acosix.alfresco.maven</groupId>
    <artifactId>de.acosix.alfresco.maven.project.parent-6.2.0</artifactId>
    <!--artifactId>de.acosix.alfresco.maven.project.parent-6.1.2</artifactId-->
    <version>1.3.0-SNAPSHOT</version>
</parent>

and running Maven with the quality.findBugs.skip flag fixes the issue #19, but I am experiencing another build problem related to the forbiddenapis artifact when using OpenJDK Java 11:

$ mvn clean install -B -T2C -DskipTests -Dquality.findBugs.skip

gives

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Alternative/Alfresco Distributed Cache - Parent 1.0.0.0-SNAPSHOT:
[INFO] 
[INFO] Alternative/Alfresco Distributed Cache - Parent .... SUCCESS [  0.612 s]
[INFO] Alternative/Alfresco Distributed Cache - Common Ignite Library SUCCESS [  8.245 s]
[INFO] Alternative/Alfresco Distributed Cache - Repository Ignite Module FAILURE [ 15.740 s]
[INFO] Alternative/Alfresco Distributed Cache - Share Ignite Module SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  24.878 s (Wall Clock)
[INFO] Finished at: 2020-02-07T14:06:55+01:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal de.thetaphi:forbiddenapis:2.7:check (default) on 
project aldica-repo-ignite: Parsing signatures failed: Invalid bundled signature 
reference (JDK version is invalid): jdk-unsafe-1.11 -> [Help 1]

I have tried to comment out the <build> section containing the forbiddenapis plugin in the project POM, but this does not work. I have been searching for (active) references to the forbiddenapis in the repository POM, but I have not been able to find any. @AFaust Do you know how to fix this?

(I will probably be off-the-grid for the next week, so I may not see your response until after next week)

Dynamic model activation not propagated to cluster members

The CacheConsistency integration test currently fails when validating that a dynamically deployed model has been correctly activated on other grid members. Since the CompiledModelsCache is an AsynchronouslyRefreshedCache, the proper way to ensure its refresh events are propagated to other grid members was supposed to be the inclusion of an event handler registered with the global registry of such caches. Whenever a refresh is triggered on such a cache, it sends a global trigger event via the registry, which the handler than distributes to other grid members to execute the refresh on all members.

During the analysis of the test failure it was discovered that the way these caches are handled by Alfresco is inherently flawed in a particular scenario. The default base class for cache instances contains an operation to force a refresh within the current transaction. This operation is not defined as part of the general interface, and thus only part of the public API of actual implementation classes. Nonetheless this operation is used in the DictionaryDAOImpl to init the compiled models cache from within the init() method. In this scenario, there is no global refresh event to notify listeners that a refresh has been requested, and the cache instance itself only notifies its direct listeners when the cache refresh has completed.

This is currently considered a design flaw of Alfresco that we cannot really work around at this time due to the following aspects:

• we cannot inject ourselves between DictionaryDAOImpl and CompiledModelsCache to intercept the in-transaction forced refresh and somehow notify other grid members
• we cannot generically register to all potential asynchronously refreshed cache instances - at most we can attempt to lookup all instances in the global Spring context via the application context
• we cannot / should not distribute the event triggered by the cache itself to its direct listeners about the refresh being complete - since cache instances do not distinguish between a "request" and "completion" event in their (listener method implementation)[https://github.com/Alfresco/alfresco-core/blob/master/src/main/java/org/alfresco/util/cache/AbstractAsynchronouslyRefreshedCache.java#L239], we would inadvertantly trigger a ping-pong effect of continuous cache refreshes when propagating the event across the grid
• even if we could intercept / react to the specific in-transaction refresh call to the cache, we cannot instantiate a proper refresh "request" event as its constructor is package-private, and thus not accessible to our code

From the analysis of this issue it has become clear that Alfresco Enterprise must also suffer from the same issue, and should not be able to properly notify other cluster members to refresh their compiled models cache when a refresh is forced in-transaction, which is unavoidable, since the relevant policy explicitely triggers the DictionaryDAOImpl#init() operation containing the forced refresh.

Update Docker / Kubernetes documentation and examples / chart patch to include companion application

Though the PR 29 already includes all necessary documentation to work with the Repository companion application on its own, we should update our Docker / Kubernetes documentation and examples / chart patches to include this app in a basic Alfresco setup. As this exceeded the time budget for the PR, this documentation update has been extracted into a separate issue / task.

Errors in log when 3rd repo connects to cluster

I have set up 4 repos on AWS with Ansible. The repos were started up one at the time starting with repo1, but when I got to repo3 i got a lot of errors in the log:

18-Jun-2019 15:04:41.408 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 82906 ms
2019-06-18 15:05:36,316  INFO  [ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-srvr-#3%repositoryGrid%] TCP discovery accepted incoming connection [rmtAddr=/172.31.42.224, rmtPort=54261]
 2019-06-18 15:05:36,329  INFO  [ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-srvr-#3%repositoryGrid%] TCP discovery spawning a new thread for connection [rmtAddr=/172.31.42.224, rmtPort=54261]
 2019-06-18 15:05:36,329  INFO  [ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-sock-reader-#5%repositoryGrid%] Started serving remote node connection [rmtAddr=/172.31.42.224:54261, rmtPort=54261]
 2019-06-18 15:05:36,356  INFO  [ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-sock-reader-#5%repositoryGrid%] Finished serving remote node connection [rmtAddr=/172.31.42.224:54261, rmtPort=54261
 [15:05:36] Joining node doesn't have encryption data [node=12b7775d-a278-4c31-9237-1576e904ed9e]
2019-06-18 15:05:36,396  INFO  [ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-msg-worker-#2%repositoryGrid%] New next node [newNext=TcpDiscoveryNode [id=12b7775d-a278-4c31-9237-1576e904ed9e, addrs=[172.31.42.224], sockAddrs=[repo2/172.31.42.224:47110], discPort=47110, order=0, intOrder=2, lastExchangeTime=1560863136340, loc=false, ver=2.7.5#20190603-sha1:be4f2a15, isClient=false]]
 2019-06-18 15:05:36,528  INFO  [ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-srvr-#3%repositoryGrid%] TCP discovery accepted incoming connection [rmtAddr=/172.31.42.224, rmtPort=41605]
 2019-06-18 15:05:36,528  INFO  [ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-srvr-#3%repositoryGrid%] TCP discovery spawning a new thread for connection [rmtAddr=/172.31.42.224, rmtPort=41605]
 2019-06-18 15:05:36,528  INFO  [ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-sock-reader-#6%repositoryGrid%] Started serving remote node connection [rmtAddr=/172.31.42.224:41605, rmtPort=41605]
 2019-06-18 15:05:36,534  INFO  [managers.discovery.GridDiscoveryManager] [disco-event-worker-#35%repositoryGrid%] Added new node to topology: TcpDiscoveryNode [id=12b7775d-a278-4c31-9237-1576e904ed9e, addrs=[172.31.42.224], sockAddrs=[repo2/172.31.42.224:47110], discPort=47110, order=2, intOrder=2, lastExchangeTime=1560863136340, loc=false, ver=2.7.5#20190603-sha1:be4f2a15, isClient=false]
 [15:05:36] Topology snapshot [ver=2, locNode=ca02fd41, servers=2, clients=0, state=ACTIVE, CPUs=4, offheap=32.0GB, heap=8.0GB]
2019-06-18 15:05:36,538  INFO  [managers.discovery.GridDiscoveryManager] [disco-event-worker-#35%repositoryGrid%] Topology snapshot [ver=2, locNode=ca02fd41, servers=2, clients=0, state=ACTIVE, CPUs=4, offheap=32.0GB, heap=8.0GB]
 2019-06-18 15:05:36,539  INFO  [ignite.lifecycle.SpringIgniteLifecycleBean] [disco-event-worker-#35%repositoryGrid%] Node 12b7775d-a278-4c31-9237-1576e904ed9e (on [172.31.42.224]) joined the Ignite instance repositoryGrid
 2019-06-18 15:05:36,539  INFO  [ignite.lifecycle.SpringIgniteLifecycleBean] [disco-event-worker-#35%repositoryGrid%] Ignite instance repositoryGrid currently has 2 active nodes on addresses [172.31.44.230, 172.31.42.224]
 2019-06-18 15:05:48,038  INFO  [extensions.webscripts.DeclarativeRegistry] [asynchronouslyRefreshedCacheThreadPool1] Registered 428 Web Scripts (+0 failed), 584 URLs
 2019-06-18 15:05:48,038  INFO  [extensions.webscripts.DeclarativeRegistry] [asynchronouslyRefreshedCacheThreadPool1] Registered 1 Package Description Documents (+0 failed) 
 2019-06-18 15:05:48,038  INFO  [extensions.webscripts.DeclarativeRegistry] [asynchronouslyRefreshedCacheThreadPool1] Registered 0 Schema Description Documents (+0 failed) 
 2019-06-18 15:05:48,968  INFO  [extensions.webscripts.DeclarativeRegistry] [asynchronouslyRefreshedCacheThreadPool1] Registered 428 Web Scripts (+0 failed), 584 URLs
 2019-06-18 15:05:48,968  INFO  [extensions.webscripts.DeclarativeRegistry] [asynchronouslyRefreshedCacheThreadPool1] Registered 1 Package Description Documents (+0 failed) 
 2019-06-18 15:05:48,968  INFO  [extensions.webscripts.DeclarativeRegistry] [asynchronouslyRefreshedCacheThreadPool1] Registered 0 Schema Description Documents (+0 failed) 
 2019-06-18 15:07:40,221  INFO  [ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-srvr-#3%repositoryGrid%] TCP discovery accepted incoming connection [rmtAddr=/172.31.44.64, rmtPort=50259]
 2019-06-18 15:07:40,221  INFO  [ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-srvr-#3%repositoryGrid%] TCP discovery spawning a new thread for connection [rmtAddr=/172.31.44.64, rmtPort=50259]
 2019-06-18 15:07:40,222  INFO  [ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-sock-reader-#7%repositoryGrid%] Started serving remote node connection [rmtAddr=/172.31.44.64:50259, rmtPort=50259]
 2019-06-18 15:07:40,246  INFO  [ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-sock-reader-#7%repositoryGrid%] Finished serving remote node connection [rmtAddr=/172.31.44.64:50259, rmtPort=50259
 [15:07:40] Joining node doesn't have encryption data [node=d2e10fe6-c017-4174-83ef-4d450f3a40a2]
2019-06-18 15:07:40,275  INFO  [ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-sock-reader-#6%repositoryGrid%] Finished serving remote node connection [rmtAddr=/172.31.42.224:41605, rmtPort=41605
 2019-06-18 15:07:40,429  INFO  [ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-srvr-#3%repositoryGrid%] TCP discovery accepted incoming connection [rmtAddr=/172.31.44.64, rmtPort=45243]
 2019-06-18 15:07:40,429  INFO  [ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-srvr-#3%repositoryGrid%] TCP discovery spawning a new thread for connection [rmtAddr=/172.31.44.64, rmtPort=45243]
 2019-06-18 15:07:40,429  INFO  [ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-sock-reader-#8%repositoryGrid%] Started serving remote node connection [rmtAddr=/172.31.44.64:45243, rmtPort=45243]
 2019-06-18 15:07:40,433  INFO  [managers.discovery.GridDiscoveryManager] [disco-event-worker-#35%repositoryGrid%] Added new node to topology: TcpDiscoveryNode [id=d2e10fe6-c017-4174-83ef-4d450f3a40a2, addrs=[172.31.44.64], sockAddrs=[repo3/172.31.44.64:47110], discPort=47110, order=3, intOrder=3, lastExchangeTime=1560863260246, loc=false, ver=2.7.5#20190603-sha1:be4f2a15, isClient=false]
 [15:07:40] Topology snapshot [ver=3, locNode=ca02fd41, servers=3, clients=0, state=ACTIVE, CPUs=6, offheap=48.0GB, heap=12.0GB]
2019-06-18 15:07:40,434  INFO  [managers.discovery.GridDiscoveryManager] [disco-event-worker-#35%repositoryGrid%] Topology snapshot [ver=3, locNode=ca02fd41, servers=3, clients=0, state=ACTIVE, CPUs=6, offheap=48.0GB, heap=12.0GB]
 2019-06-18 15:07:40,434  INFO  [ignite.lifecycle.SpringIgniteLifecycleBean] [disco-event-worker-#35%repositoryGrid%] Node d2e10fe6-c017-4174-83ef-4d450f3a40a2 (on [172.31.44.64]) joined the Ignite instance repositoryGrid
 2019-06-18 15:07:40,434  INFO  [ignite.lifecycle.SpringIgniteLifecycleBean] [disco-event-worker-#35%repositoryGrid%] Ignite instance repositoryGrid currently has 3 active nodes on addresses [172.31.44.230, 172.31.42.224, 172.31.44.64]
 2019-06-18 15:07:40,588  ERROR [internal.binary.BinaryContext] [sys-#57%repositoryGrid%] Failed to deserialize object [typeName=o.a.i.util.deque.FastSizeDeque]
 class org.apache.ignite.binary.BinaryObjectException: Failed to read field [name=deque]
	at org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:192)
	at org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:875)
	at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1764)
	at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
	at org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
	at org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
	at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
	at org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10141)
	at org.apache.ignite.internal.processors.continuous.GridContinuousProcessor$7.onMessage(GridContinuousProcessor.java:269)
	at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
	at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
	at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
	at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to unmarshal object with optimized marshaller
	at org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
	at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
	at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
	at org.apache.ignite.internal.binary.BinaryReaderExImpl.readField(BinaryReaderExImpl.java:1984)
	at org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.read0(BinaryFieldAccessor.java:703)
	at org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:188)
	... 15 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to deserialize object with given class loader: [clsLdr=ParallelWebappClassLoader
  context: alfresco
  delegate: false
----------> Parent Classloader:
java.net.URLClassLoader@3da9affb
, err=Failed to deserialize object [typeName=java.util.concurrent.ConcurrentLinkedDeque]]
	at org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
	at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
	at org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
	... 20 more
Caused by: java.io.IOException: Failed to deserialize object [typeName=java.util.concurrent.ConcurrentLinkedDeque]
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
	... 22 more
Caused by: java.io.IOException: java.lang.reflect.InvocationTargetException
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:607)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
	... 25 more
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:604)
	... 27 more
Caused by: java.io.IOException: Failed to deserialize object [typeName=org.apache.ignite.internal.GridEventConsumeHandler$EventWrapper]
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
	at java.util.concurrent.ConcurrentLinkedDeque.readObject(ConcurrentLinkedDeque.java:1543)
	... 32 more
Caused by: java.io.IOException: Failed to deserialize object [typeName=org.apache.ignite.events.DiscoveryEvent]
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
	at org.apache.ignite.internal.GridEventConsumeHandler$EventWrapper.readExternal(GridEventConsumeHandler.java:558)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
	... 35 more
Caused by: java.io.IOException: Failed to deserialize field [name=node]
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
	... 41 more
Caused by: java.io.IOException: Failed to deserialize object [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
	... 44 more
Caused by: java.io.IOException: Unexpected error occurred during unmarshalling of an instance of the class: org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot. Check that all nodes are running the same version of Ignite and that all nodes have GridOptimizedMarshaller configured with identical optimized classes lists, if any (see setClassNames and setClassNamesPath methods). If your serialized classes implement java.io.Externalizable interface, verify that serialization logic is correct.
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
	at org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
	... 47 more
2019-06-18 15:07:40,592  ERROR [processors.continuous.GridContinuousProcessor] [sys-#57%repositoryGrid%] Failed to process message (ignoring): GridContinuousMessage [type=MSG_EVT_NOTIFICATION, routineId=398c5446-951f-4c88-b31e-20bf2b3aea67, data=null, futId=null]
 class org.apache.ignite.IgniteCheckedException: Failed to deserialize object [typeName=org.apache.ignite.util.deque.FastSizeDeque]
	at org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10147)
	at org.apache.ignite.internal.processors.continuous.GridContinuousProcessor$7.onMessage(GridContinuousProcessor.java:269)
	at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
	at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
	at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
	at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to deserialize object [typeName=org.apache.ignite.util.deque.FastSizeDeque]
	at org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:914)
	at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1764)
	at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
	at org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
	at org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
	at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
	at org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10141)
	... 8 more
Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to read field [name=deque]
	at org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:192)
	at org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:875)
	... 14 more
Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to unmarshal object with optimized marshaller
	at org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
	at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
	at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
	at org.apache.ignite.internal.binary.BinaryReaderExImpl.readField(BinaryReaderExImpl.java:1984)
	at org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.read0(BinaryFieldAccessor.java:703)
	at org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:188)
	... 15 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to deserialize object with given class loader: [clsLdr=ParallelWebappClassLoader
  context: alfresco
  delegate: false
----------> Parent Classloader:
java.net.URLClassLoader@3da9affb
, err=Failed to deserialize object [typeName=java.util.concurrent.ConcurrentLinkedDeque]]
	at org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:237)
	at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
	at org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1762)
	... 20 more
Caused by: java.io.IOException: Failed to deserialize object [typeName=java.util.concurrent.ConcurrentLinkedDeque]
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:228)
	... 22 more
Caused by: java.io.IOException: java.lang.reflect.InvocationTargetException
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:607)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
	... 25 more
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:604)
	... 27 more
Caused by: java.io.IOException: Failed to deserialize object [typeName=org.apache.ignite.internal.GridEventConsumeHandler$EventWrapper]
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
	at java.util.concurrent.ConcurrentLinkedDeque.readObject(ConcurrentLinkedDeque.java:1543)
	... 32 more
Caused by: java.io.IOException: Failed to deserialize object [typeName=org.apache.ignite.events.DiscoveryEvent]
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
	at org.apache.ignite.internal.GridEventConsumeHandler$EventWrapper.readExternal(GridEventConsumeHandler.java:558)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
	... 35 more
Caused by: java.io.IOException: Failed to deserialize field [name=node]
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:526)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readSerializable(OptimizedObjectInputStream.java:611)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:954)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
	... 41 more
Caused by: java.io.IOException: Failed to deserialize object [typeName=org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode]
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:350)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readFields(OptimizedObjectInputStream.java:519)
	... 44 more
Caused by: java.io.IOException: Unexpected error occurred during unmarshalling of an instance of the class: org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot. Check that all nodes are running the same version of Ignite and that all nodes have GridOptimizedMarshaller configured with identical optimized classes lists, if any (see setClassNames and setClassNamesPath methods). If your serialized classes implement java.io.Externalizable interface, verify that serialization logic is correct.
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:364)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:425)
	at org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.readExternal(TcpDiscoveryNode.java:616)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:949)
	at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
	... 47 more
2019-06-18 15:07:51,694  INFO  [extensions.webscripts.DeclarativeRegistry] [asynchronouslyRefreshedCacheThreadPool1] Registered 428 Web Scripts (+0 failed), 584 URLs
 2019-06-18 15:07:51,694  INFO  [extensions.webscripts.DeclarativeRegistry] [asynchronouslyRefreshedCacheThreadPool1] Registered 1 Package Description Documents (+0 failed) 
 2019-06-18 15:07:51,694  INFO  [extensions.webscripts.DeclarativeRegistry] [asynchronouslyRefreshedCacheThreadPool1] Registered 0 Schema Description Documents (+0 failed) 
 2019-06-18 15:07:52,333  INFO  [extensions.webscripts.DeclarativeRegistry] [asynchronouslyRefreshedCacheThreadPool1] Registered 428 Web Scripts (+0 failed), 584 URLs
 2019-06-18 15:07:52,333  INFO  [extensions.webscripts.DeclarativeRegistry] [asynchronouslyRefreshedCacheThreadPool1] Registered 1 Package Description Documents (+0 failed) 
 2019-06-18 15:07:52,333  INFO  [extensions.webscripts.DeclarativeRegistry] [asynchronouslyRefreshedCacheThreadPool1] Registered 0 Schema Description Documents (+0 failed) 

Looking in the "Grids" section on the Ignite admin page it actually says that Topology=3 and Modes=3

Have you seen these errors before?

Binary Serialiser: variable length primitives in raw serial form

The binary serialisation could be optimised further, especially with regards to the raw serial form, by using variable length primitive values in specific use cases.

Examples:

• When writing non-null strings or arrays, the length value is always a non-negative integer. We could choose to only write the significant bytes of the value, and use the leading bit of all but the eigth byte to indicate if an additional byte follows, reappropriating the sign bit for one of those leading bits. The reduced value space of 30 bits would still be more than enough to handle Strings with a GiB of character data, which should never be created in an Alfresco application let alone be stored in a cache. Most Strings, e.g. in NodeRef / StoreRef / QName instance are shorter than 128 byte and few values should ever be longer than 16383 bytes, so an average reduction of 50% or better on the length value can be reasonably expected.
• When writing non-null long DB ID values, we could rely on the fact that Alfresco DB IDs are always non-negative, and will (almost guaranteed) never use the full value range of the 64 bit integer space. We could choose to only write the significant bytes of the value, and use the leading bit of all but the eigth byte to indicate if an additional byte follows. The reduced value space of 57 bits would still be more than enough for even the largest of Alfresco installations - filling an integer ID value space of 57 bits in 10 years would require a constant object allocation rate of 457 thousand per millisecond. The IDs for the most common QName or other immutableEntitySharedCache entry instances could be written as single bytes (ID < 128) and most would never need more than two bytes (ID < 16384).
• When writing the file size of a ContentData or ContentUrlEntity, the value is always a non-negative long, and could be similarly optimised as DB ID values albeit not written as single bytes but single shorts, as file sizes can regularly be expected to be at least a few hundred to thousands of bytes (single 15bit short < 32 KiB, two 15 bit shorts < 1 GiB). The reduced value space of 61 bits would still be sufficient to handle 2 EiB content files in extreme use cases.

Share Companion Application

FEATURE / ENHANCEMENT

We could add a (grid manager) companien application for Share to keep track of grid member addresses to simplify grid setup, recovery and failover. This we especially useful in a Kubernetes usecase.

Alfresco uses AclEntity as cache key

This issue does not denote a bug in our module, but serves to document a design issue found in default Alfresco. Specifically, as part of work on #38, it was found that Alfresco uses instances of the complex AclEntity class as cache keys in both the AclDAOImpl and PermissionServiceImpl. This type of use is highly irregular, and is made even worse by the fact that the use case relies on a deliberately partial implementation of hashCode and equals within the AclEntity class.

Since this kind of irregular use causes issues with Ignite-backed caches, which use the entire serialiesd state for hashCode / equals on serialised objects to avoid the overhead of temporarily deserialising them and using the actual implementation, a workaround had to be implemented in commit 3c5a6b7
Nevertheless, this should be fed back to Alfresco either as a simple issue or a PR with an appropriate correction.

Startup error on Alfresco 5.2.g with enabled serial optimisations

BUG

With aldica 1.0.1 deployed on an Alfresco 5.2.g instance, there is a failure during startup relating to a Hibernate exception about a connection proxy being unusable after transaction completion. This occurs during the initialisation of the file servers subsystem and/or web script registry initialisation, and the error disappears when aldica.core.binary.optimisation.enabled=false is set.
Though Alfresco 5.2.g is not officially supported / tested, we do claim it should be generally compatible...

Expected behavior

Startup of Alfresco 5.2.g succeeds in the default configuration of Aldica, or a documented 5.2.g specific configuration.

Actual behavior

Startup of Alfresco 5.2.g fails with an error that is caused by a org.hibernate.HibernateException, specifically with the error message connnection proxy not usable after transaction completion

Steps to reproduce the behavior

Install aldica 1.0.1 on Alfresco 5.2.g and start the system

Additional details (analysis so far, log statements, references, etc.)

[ALFRESCO] 2020-09-21 15:04:06,236  INFO  [management.subsystems.ChildApplicationContextFactory] [localhost-startStop-1] Starting 'fileServers' subsystem, ID: [fileServers, default]
 [ALFRESCO] 2020-09-21 15:04:06,457  INFO  [management.subsystems.ChildApplicationContextFactory] [localhost-startStop-1] Starting 'sysAdmin' subsystem, ID: [sysAdmin, default]
 [ALFRESCO] 2020-09-21 15:04:06,472  INFO  [management.subsystems.ChildApplicationContextFactory] [localhost-startStop-1] Startup of 'sysAdmin' subsystem, ID: [sysAdmin, default] complete
 [ALFRESCO] 2020-09-21 15:04:06,483  ERROR [hibernate.jdbc.BorrowedConnectionProxy] [localhost-startStop-1] Unusable connection proxy encountered: org.hibernate.jdbc.BorrowedConnectionProxy@2b8f820f
 [ALFRESCO] 2020-09-21 15:04:06,484  ERROR [hibernate.jdbc.BorrowedConnectionProxy] [localhost-startStop-1] Logging of call stack is enforced and will affect performance.
 [ALFRESCO] 2020-09-21 15:04:06,493  WARN  [management.subsystems.ChildApplicationContextFactory] [localhost-startStop-1] Startup of 'fileServers' subsystem, ID: [fileServers, default] failed
 org.alfresco.error.AlfrescoRuntimeException: 08210018 File server configuration error, nested exception is org.apache.ibatis.executor.ExecutorException: Error preparing statement.  Cause: org.hibernate.HibernateException: connnection proxy not usable after transaction completion
	at org.alfresco.filesys.AbstractServerConfigurationBean.init(AbstractServerConfigurationBean.java:449)
	at org.alfresco.filesys.AbstractServerConfigurationBean.onApplicationEvent(AbstractServerConfigurationBean.java:790)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:96)
	at org.alfresco.repo.management.subsystems.ChildApplicationContextFactory$ChildApplicationContext.publishEvent(ChildApplicationContextFactory.java:569)
	at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:954)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
	at org.alfresco.repo.management.subsystems.ChildApplicationContextFactory$ApplicationContextState.start(ChildApplicationContextFactory.java:824)
	at org.alfresco.repo.management.subsystems.AbstractPropertyBackedBean.start(AbstractPropertyBackedBean.java:1098)
	at org.alfresco.repo.management.subsystems.AbstractPropertyBackedBean.onApplicationEvent(AbstractPropertyBackedBean.java:637)
	at org.alfresco.repo.management.SafeApplicationEventMulticaster.multicastEventInternal(SafeApplicationEventMulticaster.java:214)
	at org.alfresco.repo.management.SafeApplicationEventMulticaster.multicastEvent(SafeApplicationEventMulticaster.java:185)
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:334)
	at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:954)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
	at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:410)
	at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:306)
	at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:112)
	at org.alfresco.web.app.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:70)
	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5136)
	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5662)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:1007)
	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:983)
	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:639)
	at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1296)
	at org.apache.catalina.startup.HostConfig$DeployDirectory.run(HostConfig.java:2038)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.mybatis.spring.MyBatisSystemException: nested exception is org.apache.ibatis.executor.ExecutorException: Error preparing statement.  Cause: org.hibernate.HibernateException: connnection proxy not usable after transaction completion
	at org.mybatis.spring.MyBatisExceptionTranslator.translateExceptionIfPossible(MyBatisExceptionTranslator.java:76)
	at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:421)
	at com.sun.proxy.$Proxy16.selectList(Unknown Source)
	at org.mybatis.spring.SqlSessionTemplate.selectList(SqlSessionTemplate.java:214)
	at org.alfresco.repo.domain.node.ibatis.NodeDAOImpl.selectChildAssocs(NodeDAOImpl.java:1219)
	at org.alfresco.repo.domain.node.AbstractNodeDAOImpl.getChildAssocs(AbstractNodeDAOImpl.java:3571)
	at org.alfresco.repo.node.db.DbNodeServiceImpl.getChildAssocs_aroundBody76(DbNodeServiceImpl.java:1931)
	at org.alfresco.repo.node.db.DbNodeServiceImpl$AjcClosure77.run(DbNodeServiceImpl.java:1)
	at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
	at org.alfresco.traitextender.RouteExtensions.intercept(RouteExtensions.java:100)
	at org.alfresco.repo.node.db.DbNodeServiceImpl.getChildAssocs(DbNodeServiceImpl.java:1881)
	at org.alfresco.repo.node.db.DbNodeServiceImpl.getChildAssocs_aroundBody74(DbNodeServiceImpl.java:1874)
	at org.alfresco.repo.node.db.DbNodeServiceImpl$AjcClosure75.run(DbNodeServiceImpl.java:1)
	at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
	at org.alfresco.traitextender.RouteExtensions.intercept(RouteExtensions.java:100)
	at org.alfresco.repo.node.db.DbNodeServiceImpl.getChildAssocs(DbNodeServiceImpl.java:1868)
	at org.alfresco.repo.node.db.DbNodeServiceImpl.getChildAssocs_aroundBody72(DbNodeServiceImpl.java:1861)
	at org.alfresco.repo.node.db.DbNodeServiceImpl$AjcClosure73.run(DbNodeServiceImpl.java:1)
	at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
	at org.alfresco.traitextender.RouteExtensions.intercept(RouteExtensions.java:100)
	at org.alfresco.repo.node.db.DbNodeServiceImpl.getChildAssocs(DbNodeServiceImpl.java:1859)
	at sun.reflect.GeneratedMethodAccessor188.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
	at org.alfresco.repo.lock.mem.LockableAspectInterceptor.invoke(LockableAspectInterceptor.java:244)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
	at com.sun.proxy.$Proxy27.getChildAssocs(Unknown Source)
	at sun.reflect.GeneratedMethodAccessor188.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
	at org.alfresco.repo.tenant.MultiTNodeServiceInterceptor.invoke(MultiTNodeServiceInterceptor.java:111)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
	at com.sun.proxy.$Proxy27.getChildAssocs(Unknown Source)
	at sun.reflect.GeneratedMethodAccessor188.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.alfresco.repo.service.StoreRedirectorProxyFactory$RedirectorInvocationHandler.invoke(StoreRedirectorProxyFactory.java:231)
	at com.sun.proxy.$Proxy50.getChildAssocs(Unknown Source)
	at org.alfresco.repo.search.DocumentNavigator.getChildAxisIterator(DocumentNavigator.java:388)
	at org.jaxen.expr.iter.IterableChildAxis.namedAccessIterator(IterableChildAxis.java:98)
	at org.jaxen.expr.DefaultNameStep.evaluate(DefaultNameStep.java:180)
	at org.jaxen.expr.DefaultLocationPath.evaluate(DefaultLocationPath.java:140)
	at org.jaxen.expr.DefaultAbsoluteLocationPath.evaluate(DefaultAbsoluteLocationPath.java:113)
	at org.jaxen.expr.DefaultXPathExpr.asList(DefaultXPathExpr.java:102)
	at org.jaxen.BaseXPath.selectNodesForContext(BaseXPath.java:677)
	at org.jaxen.BaseXPath.selectNodes(BaseXPath.java:216)
	at org.alfresco.repo.search.NodeServiceXPath.selectNodes(NodeServiceXPath.java:193)
	at org.alfresco.repo.search.impl.NodeSearcher.selectNodes(NodeSearcher.java:96)
	at org.alfresco.repo.search.impl.SearchServiceSubSystemDelegator.selectNodes(SearchServiceSubSystemDelegator.java:167)
	at org.alfresco.repo.search.impl.SearchServiceSubSystemDelegator.selectNodes(SearchServiceSubSystemDelegator.java:149)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
	at net.sf.acegisecurity.intercept.method.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:80)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
	at org.alfresco.repo.security.permissions.impl.ExceptionTranslatorMethodInterceptor.invoke(ExceptionTranslatorMethodInterceptor.java:53)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
	at org.alfresco.repo.audit.AuditMethodInterceptor.invoke(AuditMethodInterceptor.java:166)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
	at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:96)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:260)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:94)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
	at com.sun.proxy.$Proxy30.selectNodes(Unknown Source)
	at org.alfresco.filesys.repo.ContentDiskDriver2$1.doWork(ContentDiskDriver2.java:484)
	at org.alfresco.filesys.repo.ContentDiskDriver2$1.doWork(ContentDiskDriver2.java:1)
	at org.alfresco.repo.security.authentication.AuthenticationUtil.runAs(AuthenticationUtil.java:555)
	at org.alfresco.filesys.repo.ContentDiskDriver2.registerContext(ContentDiskDriver2.java:556)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:198)
	at com.sun.proxy.$Proxy305.registerContext(Unknown Source)
	at org.alfresco.filesys.repo.NonTransactionalRuleContentDiskDriver.registerContext(NonTransactionalRuleContentDiskDriver.java:609)
	at org.alfresco.filesys.repo.LegacyFileStateDriver.registerContext(LegacyFileStateDriver.java:541)
	at org.alfresco.filesys.repo.BufferedContentDiskDriver.registerContext(BufferedContentDiskDriver.java:571)
	at org.alfresco.filesys.config.ServerConfigurationBean.processFilesystemsConfig(ServerConfigurationBean.java:1638)
	at org.alfresco.filesys.AbstractServerConfigurationBean.init(AbstractServerConfigurationBean.java:444)
	... 30 more
Caused by: org.apache.ibatis.executor.ExecutorException: Error preparing statement.  Cause: org.hibernate.HibernateException: connnection proxy not usable after transaction completion
	at org.apache.ibatis.executor.statement.BaseStatementHandler.prepare(BaseStatementHandler.java:97)
	at org.apache.ibatis.executor.statement.RoutingStatementHandler.prepare(RoutingStatementHandler.java:58)
	at org.apache.ibatis.executor.SimpleExecutor.prepareStatement(SimpleExecutor.java:76)
	at org.apache.ibatis.executor.SimpleExecutor.doQuery(SimpleExecutor.java:61)
	at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:303)
	at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:154)
	at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:102)
	at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:82)
	at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:120)
	at sun.reflect.GeneratedMethodAccessor159.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:408)
	... 121 more
Caused by: org.hibernate.HibernateException: connnection proxy not usable after transaction completion
	at org.hibernate.jdbc.BorrowedConnectionProxy.invoke(BorrowedConnectionProxy.java:142)
	at com.sun.proxy.$Proxy14.prepareStatement(Unknown Source)
	at org.apache.ibatis.executor.statement.PreparedStatementHandler.instantiateStatement(PreparedStatementHandler.java:79)
	at org.apache.ibatis.executor.statement.BaseStatementHandler.prepare(BaseStatementHandler.java:88)
	... 133 more

Tell us about your environment

• OpenJDK 1.8 / Tomcat 8.5
• Alfresco 5.2.g
• Order of the Bee Support Tools 1.1.0.0
• JavaScript Console 0.6
• Acosix Alfresco Utility Core 1.2.2
• aldica 1.0.1

Investigate configuration option to enable read-through / no-op caches

In some situations / constellations, administrators / users may want to set up a system in which data is not cached or a subset of caches should be set to read-through. We should investigate the various optinos to enable something like this and come up with an approach that would be able to cover most of the requirements.

Requirements / variants to consider:

• administrator wants to set all data caches to read-through with a global flag - note: not all caches can safely be set to "no-op" because Alfresco includes some "improper" caches for service lookup / i18n bundles
• developer wants to set specific caches / all data caches to read-through in a specific code context (need to investigate what Alfresco already uses, e.g. via NodeDAO transaction-level flags)
• administrator wants to set individual cache to a no-op cache via regular cache properties (may support a "null" cache option to use Alfresco's default class NullCache class, and replace XML pre-defined NullCache instances with actual instantiation calls via cacheFactory and default properties set to "null")

Initialisation hang during Share startup in private/public address mapping test

BUG

Using a general "Cluster Test" setup based on Docker Compose and using separate Docker networks for each vertical node (e.g. node1 = ACS-01 + Share-01, node2 = ACS02 + Share-02) and configuration to run Ignite communication via the Docker host using public-private address mapping, Share startup is stuck in a seemingly infinite loop for initialising the Ignite grid.

Startup works without issues when Share instances are in the same Docker network and directly communicate with each other without private/public address mapping. The same test previously worked without issues, so may be a potential regression due to some underlying changes in Docker on the specific platform (see section about environment).

Expected behavior

Share instances start up and are able to connect to each other within reasonable amount of time, regardless of order of startup.

Actual behavior

Share instances never fully start up, being stuck in the Ignite grid initialisation lifecyle method.

Steps to reproduce the behavior

Set up Docker Compose with 2 Share services using the following pattern:

    share0X:
        build:
            context: ./share
        image: acosix/cluster-test-share:latest
        hostname: share0X
        networks:
          - nodeX
        environment:
          - REPO_HOST=repository0X
          - REPO_PORT=8080
# loaded from .env
          - CATALINA_OPTS=>-
                -Xms1g -Xmx1g -XX:+UseG1GC -XX:+ParallelRefProcEnabled -XX:+UseStringDeduplication -XX:+ScavengeBeforeFullGC -XX:+DisableExplicitGC -XX:+AlwaysPreTouch
                -DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true -DIGNITE_QUIET=true -DIGNITE_NO_ASCII=false -DIGNITE_UPDATE_NOTIFIER=false -DIGNITE_JVM_PAUSE_DETECTOR_DISABLED=true -DIGNITE_DUMP_THREADS_ON_FAILURE=false -Djava.net.preferIPv4Stack=true
                --add-exports=java.base/jdk.internal.misc=ALL-UNNAMED --add-exports=java.base/sun.nio.ch=ALL-UNNAMED --add-exports=java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED --add-exports=jdk.internal.jvmstat/sun.jvmstat.monitor=ALL-UNNAMED --add-exports=java.base/sun.reflect.generics.reflectiveObjects=ALL-UNNAMED --illegal-access=permit
                -Daldica.webSessionCache.enabled=false -Daldica.core.local.comm.filterReachableAddresses=true
                -Daldica.core.local.host=share0X -Daldica.core.public.host=host.docker.internal -Daldica.core.local.id=share0X -Daldica.core.public.comm.port=4X130 -Daldica.core.public.disco.port=47X40 -Daldica.core.public.time.port=4X150 -Daldica.core.initialMembers=host.docker.internal:4Y140
        ports:
          - 8281:8080
          - 4X130:47130
          - 4X140:47140
          - 4X150:47150
        restart: unless-stopped

with X = vertical stack index, and Y = (X == 0 ? 1 : X - 1)

and Dockerfile

FROM alfresco/alfresco-share:6.1.0

COPY *.amp /usr/local/tomcat/amps/

RUN java -jar /usr/local/tomcat/alfresco-mmt/alfresco-mmt*.jar install /usr/local/tomcat/amps/de.acosix.alfresco.utility.core.share-*.amp /usr/local/tomcat/webapps/share -nobackup -force \
   && java -jar /usr/local/tomcat/alfresco-mmt/alfresco-mmt*.jar install /usr/local/tomcat/amps/aldica-share-ignite-*.amp /usr/local/tomcat/webapps/share -nobackup -force \
   && rm -rf /usr/local/tomcat/amps/*

with the relevant module AMPs stored in the same folder.

Setup ACS instances + shared DB to support Share.

Startup

Additional details (analysis so far, log statements, references, etc.)

The test setup previously worked fine for validating aldica 1.0.0 and 1.0.1, as well as the dervied, commercialised Acosix variant with full Alfresco Enterprise cluster-replacement support. The only real, major infrastructure that has since changed is the switch from Hyper-V to WSL 2-based Docker engine, as well as various general Docker engine version updates.

Logging shows:

2021-05-03 15:30:17,204 DEBUG [org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi] [nio-acceptor-tcp-comm-#18%shareGrid%] Balancing data [min0=0, minIdx=0, max0=-1, maxIdx=-1]
2021-05-03 15:30:17,724 INFO  [org.aldica.common.ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-sock-reader-[34f0d629 192.168.0.1:34666]-#269%shareGrid%] Finished serving remote node connection [rmtAddr=/192.168.0.1:34666, rmtPort=34666
2021-05-03 15:30:17,724 INFO  [org.aldica.common.ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-sock-reader-[34f0d629 192.168.0.1:34658]-#268%shareGrid%] Finished serving remote node connection [rmtAddr=/192.168.0.1:34658, rmtPort=34658
2021-05-03 15:30:19,732 INFO  [org.aldica.common.ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-srvr-[:47140]-#3%shareGrid%] TCP discovery accepted incoming connection [rmtAddr=/192.168.0.1, rmtPort=34674]
2021-05-03 15:30:19,732 INFO  [org.aldica.common.ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-srvr-[:47140]-#3%shareGrid%] TCP discovery spawning a new thread for connection [rmtAddr=/192.168.0.1, rmtPort=34674]
2021-05-03 15:30:19,733 INFO  [org.aldica.common.ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-sock-reader-[]-#270%shareGrid%] Started serving remote node connection [rmtAddr=/192.168.0.1:34674, rmtPort=34674]
2021-05-03 15:30:19,734 INFO  [org.aldica.common.ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-sock-reader-[34f0d629 192.168.0.1:34674]-#270%shareGrid%] Initialized connection with remote server node [nodeId=34f0d629-c192-4877-b6e7-caa89a386c7b, rmtAddr=/192.168.0.1:34674]
2021-05-03 15:30:19,742 INFO  [org.aldica.common.ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-sock-reader-[34f0d629 192.168.0.1:34674]-#270%shareGrid%] Finished serving remote node connection [rmtAddr=/192.168.0.1:34674, rmtPort=34674
2021-05-03 15:30:21,751 INFO  [org.aldica.common.ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-srvr-[:47140]-#3%shareGrid%] TCP discovery accepted incoming connection [rmtAddr=/192.168.0.1, rmtPort=34682]
2021-05-03 15:30:21,752 INFO  [org.aldica.common.ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-srvr-[:47140]-#3%shareGrid%] TCP discovery spawning a new thread for connection [rmtAddr=/192.168.0.1, rmtPort=34682]
2021-05-03 15:30:21,753 INFO  [org.aldica.common.ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-sock-reader-[]-#271%shareGrid%] Started serving remote node connection [rmtAddr=/192.168.0.1:34682, rmtPort=34682]
2021-05-03 15:30:21,755 INFO  [org.aldica.common.ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-sock-reader-[34f0d629 192.168.0.1:34682]-#271%shareGrid%] Initialized connection with remote server node [nodeId=34f0d629-c192-4877-b6e7-caa89a386c7b, rmtAddr=/192.168.0.1:34682]
2021-05-03 15:30:21,769 INFO  [org.aldica.common.ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-sock-reader-[34f0d629 192.168.0.1:34682]-#271%shareGrid%] Finished serving remote node connection [rmtAddr=/192.168.0.1:34682, rmtPort=34682
2021-05-03 15:30:23,208 DEBUG [org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi] [nio-acceptor-tcp-comm-#18%shareGrid%] Balancing data [min0=0, minIdx=0, max0=-1, maxIdx=-1]
2021-05-03 15:30:23,778 INFO  [org.aldica.common.ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-srvr-[:47140]-#3%shareGrid%] TCP discovery accepted incoming connection [rmtAddr=/192.168.0.1, rmtPort=34690]
2021-05-03 15:30:23,778 INFO  [org.aldica.common.ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-srvr-[:47140]-#3%shareGrid%] TCP discovery spawning a new thread for connection [rmtAddr=/192.168.0.1, rmtPort=34690]
2021-05-03 15:30:23,779 INFO  [org.aldica.common.ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-sock-reader-[]-#272%shareGrid%] Started serving remote node connection [rmtAddr=/192.168.0.1:34690, rmtPort=34690]
2021-05-03 15:30:23,780 INFO  [org.aldica.common.ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-sock-reader-[34f0d629 192.168.0.1:34690]-#272%shareGrid%] Initialized connection with remote server node [nodeId=34f0d629-c192-4877-b6e7-caa89a386c7b, rmtAddr=/192.168.0.1:34690]
2021-05-03 15:30:23,787 INFO  [org.aldica.common.ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-sock-reader-[34f0d629 192.168.0.1:34690]-#272%shareGrid%] Finished serving remote node connection [rmtAddr=/192.168.0.1:34690, rmtPort=34690
2021-05-03 15:30:25,793 INFO  [org.aldica.common.ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-srvr-[:47140]-#3%shareGrid%] TCP discovery accepted incoming connection [rmtAddr=/192.168.0.1, rmtPort=34698]
2021-05-03 15:30:25,794 INFO  [org.aldica.common.ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-srvr-[:47140]-#3%shareGrid%] TCP discovery spawning a new thread for connection [rmtAddr=/192.168.0.1, rmtPort=34698]
2021-05-03 15:30:25,794 INFO  [org.aldica.common.ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-sock-reader-[]-#273%shareGrid%] Started serving remote node connection [rmtAddr=/192.168.0.1:34698, rmtPort=34698]
2021-05-03 15:30:25,796 INFO  [org.aldica.common.ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-sock-reader-[34f0d629 192.168.0.1:34698]-#273%shareGrid%] Initialized connection with remote server node [nodeId=34f0d629-c192-4877-b6e7-caa89a386c7b, rmtAddr=/192.168.0.1:34698]
2021-05-03 15:30:25,802 INFO  [org.aldica.common.ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-sock-reader-[34f0d629 192.168.0.1:34698]-#273%shareGrid%] Finished serving remote node connection [rmtAddr=/192.168.0.1:34698, rmtPort=34698
2021-05-03 15:30:27,808 INFO  [org.aldica.common.ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-srvr-[:47140]-#3%shareGrid%] TCP discovery accepted incoming connection [rmtAddr=/192.168.0.1, rmtPort=34706]
2021-05-03 15:30:27,809 INFO  [org.aldica.common.ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-srvr-[:47140]-#3%shareGrid%] TCP discovery spawning a new thread for connection [rmtAddr=/192.168.0.1, rmtPort=34706]
2021-05-03 15:30:27,810 INFO  [org.aldica.common.ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-sock-reader-[]-#274%shareGrid%] Started serving remote node connection [rmtAddr=/192.168.0.1:34706, rmtPort=34706]
2021-05-03 15:30:27,811 INFO  [org.aldica.common.ignite.discovery.CredentialsAwareTcpDiscoverySpi] [tcp-disco-sock-reader-[34f0d629 192.168.0.1:34706]-#274%shareGrid%] Initialized connection with remote server node [nodeId=34f0d629-c192-4877-b6e7-caa89a386c7b, rmtAddr=/192.168.0.1:34706]

(extremely similar log output on both nodes - only difference being the remote IP address for the incomming connection)

Tell us about your environment

Alfresco ACS 6,2.0 Share base image
Acosix Alfresco Utility Core 1.3.0-SNAPSHOT
aldica 1.0.1
(+ unrelated addons JS Console / OOTBee Support Tools)
WIndows 10
Docker Desktop 3.3.1, Engine 20.10.5, Compose 1.19.0
WSL 2 based engine enabled

Alfresco 7 support

FEATURE / ENHANCEMENT

We'd like to use aldica in Alfresco Community Edition 7.*. But currently, 1.0.1 amp cannot be installed into the Alresco 7 war.

Do you plan to support this version of Alfresco?

Thanks,
Artem

Toolchain dependency

Hi Axel.

I just cloned the (main) project from GitHub and tried to run mvn (clean) package and got the following error:

[INFO] --- maven-toolchains-plugin:1.1:toolchain (default) @ de.acosix.alfresco.ignite.common ---
[INFO] Required toolchain: jdk [ version='1.8' ]
[ERROR] No toolchain found for type jdk
[ERROR] Cannot find matching toolchain definitions for the following toolchain types:
jdk [ version='1.8' ]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Alfresco Community Open Grid - Parent .............. SUCCESS [  0.077 s]
[INFO] Alfresco Ignite - Common Library ................... FAILURE [  0.354 s]
[INFO] Alfresco Ignite - Repository Module ................ SKIPPED
[INFO] Alfresco Ignite - Share Module ..................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.609 s
[INFO] Finished at: 2019-04-15T12:58:34+02:00
[INFO] Final Memory: 20M/607M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-toolchains-plugin:1.1:toolchain (default) on project de.acosix.alfresco.ignite.common: Cannot find matching toolchain definitions for the following toolchain types:
[ERROR] jdk [ version='1.8' ]
[ERROR] Please make sure you define the required toolchains in your ~/.m2/toolchains.xml file.

Maybe the POM(s) should be changed so the project can be build without depending on toolchains in the .m2 folder?

(or am I missing a step in the build process?)

GridAffinityAssignmentCache warning

I'm getting repository warnings like the ones shown below. The warnings seem to be thrown when the number of members in the Ignite grid changes (and at startup). The warnings are thrown from an upstream Apache Ignite class, but is this something we should worry about?

Maybe it can be avoid be making configuration changes to the aldica global properties settings?

Additional details (analysis so far, log statements, references, etc.)

2020-11-27 07:15:09,946  WARN  [processors.affinity.GridAffinityAssignmentCache] [exchange-worker-#34%alf-test-alfresco-cs-alfrescoGrid%] Logging at INFO level without checking if INFO level is enabled: Local node affinity assignment distribution is not ideal [cache=aclEntitySharedCache, expectedPrimary=32.00, actualPrimary=32, expectedBackups=32.00, actualBackups=0, warningThreshold=50.00%]
2020-11-27 07:15:09,949  WARN  [processors.affinity.GridAffinityAssignmentCache] [sys-#8484%alf-test-alfresco-cs-alfrescoGrid%] Logging at INFO level without checking if INFO level is enabled: Local node affinity assignment distribution is not ideal [cache=authenticationSharedCache, expectedPrimary=32.00, actualPrimary=32, expectedBackups=32.00, actualBackups=0, warningThreshold=50.00%]
2020-11-27 07:15:09,952  WARN  [processors.affinity.GridAffinityAssignmentCache] [sys-#8479%alf-test-alfresco-cs-alfrescoGrid%] Logging at INFO level without checking if INFO level is enabled: Local node affinity assignment distribution is not ideal [cache=usernameToTicketIdCache, expectedPrimary=32.00, actualPrimary=32, expectedBackups=32.00, actualBackups=0, warningThreshold=50.00%]
2020-11-27 07:15:09,959  WARN  [processors.affinity.GridAffinityAssignmentCache] [exchange-worker-#34%alf-test-alfresco-cs-alfrescoGrid%] Logging at INFO level without checking if INFO level is enabled: Local node affinity assignment distribution is not ideal [cache=readersSharedCache, expectedPrimary=32.00, actualPrimary=32, expectedBackups=32.00, actualBackups=0, warningThreshold=50.00%]
...

Tell us about your environment

The following aldica properties are set (repositories are running in an Kubernetes environment):

-Daldica.caches.remoteSupport.enabled=true
-Daldica.core.binary.optimisation.enabled=false
-Daldica.core.login=<login>
-Daldica.core.password=<password>
-Daldica.core.local.id=alf-test-alfresco-cs-repository-866bfff597-zzsj5
-Daldica.core.public.host=10.180.1.160

Binary serialization error

BUG

Binary serialization errors are thrown in the logs when opening document in Share. aldica is only enabled on the repository side and only a single repo node is running. The entire stack is running in Kubernetes (see details below).

Expected behavior

No errors should be thrown in the logs.

Actual behavior

Binary serialization errors are found in the log (see below).

Steps to reproduce the behavior

1. Fire up a local Kubernetes cluster version 1.17.6 in Minikube.
2. Install the latest version of Helm (3.4.0).
3. Clone the repository https://github.com/magenta-aps/acs-deployment.git
4. Check out the branch feature/aldica.
5. Set aldica.core.binary.optimisation.enabled=true (or remove the line) in the file helm/alfresco-content-services/templates/config-repository.yaml.
6. Point cluster.local to the external Minikube IP-address to in your local hosts file.
7. cd helm/alfresco-content-services
8. Build the Helm dependencies with helm dependency build
9. Install the Helm chart with helm install -f community_values.yaml --set externalProtocol="http" --set externalHost="cluster.local" --set externalPort="80" my-alf .
10. Log in to Share as admin.
11. Search for budget.
12. Open the budget.xls file located in the swsdp sample site.

Additional details (analysis so far, log statements, references, etc.)

Setting -Daldica.core.binary.optimisation.useIdsWhenPossible=false does not resolve the issue, but the error does not seem to occur when the binary optimizations are disabled, i.e when setting aldica.core.binary.optimisation.enabled=false. The following errors are found in the repository log:

2020-11-12 07:28:56,650  ERROR [internal.binary.BinaryContext] [http-nio-8080-exec-27] Failed to serialize object [typeName=org.aldica.repo.ignite.cache.NodeAspectsCacheSet]
org.alfresco.error.AlfrescoRuntimeException: 10120231 Cannot resolve {http://www.alfresco.org/model/system/1.0}referenceable to DB ID
	at org.aldica.repo.ignite.binary.NodeAspectsBinarySerializer.writeAspectsRawSerialForm(NodeAspectsBinarySerializer.java:142)
	at org.aldica.repo.ignite.binary.NodeAspectsBinarySerializer.writeBinary(NodeAspectsBinarySerializer.java:94)
	at org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:765)
	at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:232)
	at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:165)
	at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:152)
	at org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteObject(BinaryWriterExImpl.java:524)
	at org.apache.ignite.internal.binary.BinaryWriterExImpl.writeObjectField(BinaryWriterExImpl.java:1194)
	at org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.write0(BinaryFieldAccessor.java:687)
	at org.apache.ignite.internal.binary.BinaryFieldAccessor.write(BinaryFieldAccessor.java:157)
	at org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:822)
	at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:232)
	at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:165)
	at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:152)
	at org.apache.ignite.internal.binary.GridBinaryMarshaller.marshal(GridBinaryMarshaller.java:251)
	at org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.marshalToBinary(CacheObjectBinaryProcessorImpl.java:493)
	at org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toBinary(CacheObjectBinaryProcessorImpl.java:1248)
	at org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toCacheObject(CacheObjectBinaryProcessorImpl.java:1085)
	at org.apache.ignite.internal.processors.cache.GridCacheContext.toCacheObject(GridCacheContext.java:1846)
	at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.mapSingleUpdate(GridNearAtomicSingleUpdateFuture.java:552)
	at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.map(GridNearAtomicSingleUpdateFuture.java:457)
	at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.mapOnTopology(GridNearAtomicSingleUpdateFuture.java:446)
	at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.map(GridNearAtomicAbstractUpdateFuture.java:249)
	at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update0(GridDhtAtomicCache.java:1164)
	at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.getAndPut0(GridDhtAtomicCache.java:612)
	at org.apache.ignite.internal.processors.cache.GridCacheAdapter.getAndPut(GridCacheAdapter.java:2471)
	at org.apache.ignite.internal.processors.cache.GridCacheAdapter.getAndPut(GridCacheAdapter.java:2450)
	at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.getAndPut(IgniteCacheProxyImpl.java:1348)
	at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.getAndPut(GatewayProtectedCacheProxy.java:880)
	at org.aldica.repo.ignite.cache.SimpleIgniteBackedCache.getAndPutImpl(SimpleIgniteBackedCache.java:414)
	at org.aldica.repo.ignite.cache.SimpleIgniteBackedCache.put(SimpleIgniteBackedCache.java:309)
	at jdk.internal.reflect.GeneratedMethodAccessor167.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.aldica.repo.ignite.cache.SimpleCacheInvoker.invoke(SimpleCacheInvoker.java:49)
	at org.aldica.repo.ignite.cache.CacheFactoryImpl$SimpleLazySwapCacheInvoker.invoke(CacheFactoryImpl.java:726)
	at com.sun.proxy.$Proxy21.put(Unknown Source)
	at org.alfresco.repo.cache.TransactionalCache.putSharedCacheValue(TransactionalCache.java:502)
	at org.alfresco.repo.cache.TransactionalCache$NewCacheBucket.doPostCommit(TransactionalCache.java:1253)
	at org.alfresco.repo.cache.TransactionalCache.afterCommit(TransactionalCache.java:1078)
	at org.alfresco.util.transaction.TransactionSupportUtil$TransactionSynchronizationImpl.afterCompletion(TransactionSupportUtil.java:598)
	at org.springframework.transaction.support.TransactionSynchronizationUtils.invokeAfterCompletion(TransactionSynchronizationUtils.java:171)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.invokeAfterCompletion(AbstractPlatformTransactionManager.java:992)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerAfterCompletion(AbstractPlatformTransactionManager.java:967)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:788)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:714)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:534)
	at org.alfresco.util.transaction.SpringAwareUserTransaction.commit(SpringAwareUserTransaction.java:479)
	at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:472)
	at org.alfresco.repo.security.person.PersonServiceImpl.makeHomeFolderIfRequired(PersonServiceImpl.java:923)
	at org.alfresco.repo.security.person.PersonServiceImpl.getPersonImpl(PersonServiceImpl.java:527)
	at org.alfresco.repo.security.person.PersonServiceImpl.getPerson(PersonServiceImpl.java:484)
	at org.alfresco.repo.security.person.PersonServiceImpl.getPerson(PersonServiceImpl.java:441)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at net.sf.acegisecurity.intercept.method.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:80)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.alfresco.repo.security.permissions.impl.ExceptionTranslatorMethodInterceptor.invoke(ExceptionTranslatorMethodInterceptor.java:53)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.alfresco.repo.audit.AuditMethodInterceptor.invoke(AuditMethodInterceptor.java:166)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:295)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
	at com.sun.proxy.$Proxy122.getPerson(Unknown Source)
	at org.alfresco.repo.quickshare.QuickShareServiceImpl.getMetaData(QuickShareServiceImpl.java:552)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.alfresco.repo.security.permissions.impl.AlwaysProceedMethodInterceptor.invoke(AlwaysProceedMethodInterceptor.java:41)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.alfresco.repo.security.permissions.impl.ExceptionTranslatorMethodInterceptor.invoke(ExceptionTranslatorMethodInterceptor.java:53)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.alfresco.repo.audit.AuditMethodInterceptor.invoke(AuditMethodInterceptor.java:166)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.alfresco.repo.transaction.CheckTransactionAdvice.invoke(CheckTransactionAdvice.java:54)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.alfresco.repo.transaction.RetryingTransactionAdvice$1.execute(RetryingTransactionAdvice.java:71)
	at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:450)
	at org.alfresco.repo.transaction.RetryingTransactionAdvice.invoke(RetryingTransactionAdvice.java:74)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
	at com.sun.proxy.$Proxy186.getMetaData(Unknown Source)
	at org.alfresco.repo.web.scripts.quickshare.MetaDataGet.executeImpl(MetaDataGet.java:70)
	at org.springframework.extensions.webscripts.DeclarativeWebScript.execute(DeclarativeWebScript.java:64)
	at org.alfresco.repo.web.scripts.RepositoryContainer$3.execute(RepositoryContainer.java:519)
	at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:450)
	at org.alfresco.repo.web.scripts.RepositoryContainer.transactionedExecute(RepositoryContainer.java:595)
	at org.alfresco.repo.web.scripts.RepositoryContainer.transactionedExecuteAs(RepositoryContainer.java:664)
	at org.alfresco.repo.web.scripts.RepositoryContainer.executeScriptInternal(RepositoryContainer.java:435)
	at org.alfresco.repo.web.scripts.RepositoryContainer.executeScript(RepositoryContainer.java:315)
	at org.springframework.extensions.webscripts.AbstractRuntime.executeScript(AbstractRuntime.java:399)
	at org.springframework.extensions.webscripts.AbstractRuntime.executeScript(AbstractRuntime.java:210)
	at org.springframework.extensions.webscripts.servlet.WebScriptServlet.service(WebScriptServlet.java:132)
	at org.alfresco.repo.web.scripts.AlfrescoWebScriptServlet.service(AlfrescoWebScriptServlet.java:43)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
	at jdk.internal.reflect.GeneratedMethodAccessor627.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/javax.security.auth.Subject.doAsPrivileged(Subject.java:550)
	at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314)
	at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:170)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:225)
	at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:47)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:149)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:145)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at jdk.internal.reflect.GeneratedMethodAccessor620.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/javax.security.auth.Subject.doAsPrivileged(Subject.java:550)
	at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314)
	at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:191)
	at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:47)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:149)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:145)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
	at org.alfresco.module.aosmodule.service.ContextRootFilter.doFilter(ContextRootFilter.java:93)
	at jdk.internal.reflect.GeneratedMethodAccessor620.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/javax.security.auth.Subject.doAsPrivileged(Subject.java:550)
	at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314)
	at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:191)
	at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:47)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:149)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:145)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
	at org.alfresco.web.app.servlet.GlobalLocalizationFilter.doFilter(GlobalLocalizationFilter.java:68)
	at jdk.internal.reflect.GeneratedMethodAccessor620.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/javax.security.auth.Subject.doAsPrivileged(Subject.java:550)
	at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314)
	at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:191)
	at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:47)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:149)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:145)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
	at org.alfresco.web.app.servlet.ClearSecurityContextFilter.doFilter(ClearSecurityContextFilter.java:53)
	at jdk.internal.reflect.GeneratedMethodAccessor620.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/javax.security.auth.Subject.doAsPrivileged(Subject.java:550)
	at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314)
	at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:191)
	at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:47)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:149)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:145)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:493)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:660)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:798)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:808)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.base/java.lang.Thread.run(Thread.java:834)
2020-11-12 07:28:56,653  ERROR [internal.binary.BinaryContext] [http-nio-8080-exec-27] Failed to serialize object [typeName=org.alfresco.repo.cache.TransactionalCache$ValueHolder]
class org.apache.ignite.binary.BinaryObjectException: Failed to write field [name=value]
	at org.apache.ignite.internal.binary.BinaryFieldAccessor.write(BinaryFieldAccessor.java:164)
	at org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:822)
	at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:232)
	at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:165)
	at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:152)
	at org.apache.ignite.internal.binary.GridBinaryMarshaller.marshal(GridBinaryMarshaller.java:251)
	at org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.marshalToBinary(CacheObjectBinaryProcessorImpl.java:493)
	at org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toBinary(CacheObjectBinaryProcessorImpl.java:1248)
	at org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toCacheObject(CacheObjectBinaryProcessorImpl.java:1085)
	at org.apache.ignite.internal.processors.cache.GridCacheContext.toCacheObject(GridCacheContext.java:1846)
	at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.mapSingleUpdate(GridNearAtomicSingleUpdateFuture.java:552)
	at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.map(GridNearAtomicSingleUpdateFuture.java:457)
	at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.mapOnTopology(GridNearAtomicSingleUpdateFuture.java:446)
	at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.map(GridNearAtomicAbstractUpdateFuture.java:249)
	at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update0(GridDhtAtomicCache.java:1164)
	at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.getAndPut0(GridDhtAtomicCache.java:612)
	at org.apache.ignite.internal.processors.cache.GridCacheAdapter.getAndPut(GridCacheAdapter.java:2471)
	at org.apache.ignite.internal.processors.cache.GridCacheAdapter.getAndPut(GridCacheAdapter.java:2450)
	at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.getAndPut(IgniteCacheProxyImpl.java:1348)
	at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.getAndPut(GatewayProtectedCacheProxy.java:880)
	at org.aldica.repo.ignite.cache.SimpleIgniteBackedCache.getAndPutImpl(SimpleIgniteBackedCache.java:414)
	at org.aldica.repo.ignite.cache.SimpleIgniteBackedCache.put(SimpleIgniteBackedCache.java:309)
	at jdk.internal.reflect.GeneratedMethodAccessor167.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.aldica.repo.ignite.cache.SimpleCacheInvoker.invoke(SimpleCacheInvoker.java:49)
	at org.aldica.repo.ignite.cache.CacheFactoryImpl$SimpleLazySwapCacheInvoker.invoke(CacheFactoryImpl.java:726)
	at com.sun.proxy.$Proxy21.put(Unknown Source)
	at org.alfresco.repo.cache.TransactionalCache.putSharedCacheValue(TransactionalCache.java:502)
	at org.alfresco.repo.cache.TransactionalCache$NewCacheBucket.doPostCommit(TransactionalCache.java:1253)
	at org.alfresco.repo.cache.TransactionalCache.afterCommit(TransactionalCache.java:1078)
	at org.alfresco.util.transaction.TransactionSupportUtil$TransactionSynchronizationImpl.afterCompletion(TransactionSupportUtil.java:598)
	at org.springframework.transaction.support.TransactionSynchronizationUtils.invokeAfterCompletion(TransactionSynchronizationUtils.java:171)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.invokeAfterCompletion(AbstractPlatformTransactionManager.java:992)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerAfterCompletion(AbstractPlatformTransactionManager.java:967)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:788)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:714)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:534)
	at org.alfresco.util.transaction.SpringAwareUserTransaction.commit(SpringAwareUserTransaction.java:479)
	at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:472)
	at org.alfresco.repo.security.person.PersonServiceImpl.makeHomeFolderIfRequired(PersonServiceImpl.java:923)
	at org.alfresco.repo.security.person.PersonServiceImpl.getPersonImpl(PersonServiceImpl.java:527)
	at org.alfresco.repo.security.person.PersonServiceImpl.getPerson(PersonServiceImpl.java:484)
	at org.alfresco.repo.security.person.PersonServiceImpl.getPerson(PersonServiceImpl.java:441)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at net.sf.acegisecurity.intercept.method.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:80)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.alfresco.repo.security.permissions.impl.ExceptionTranslatorMethodInterceptor.invoke(ExceptionTranslatorMethodInterceptor.java:53)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.alfresco.repo.audit.AuditMethodInterceptor.invoke(AuditMethodInterceptor.java:166)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:295)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
	at com.sun.proxy.$Proxy122.getPerson(Unknown Source)
	at org.alfresco.repo.quickshare.QuickShareServiceImpl.getMetaData(QuickShareServiceImpl.java:552)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.alfresco.repo.security.permissions.impl.AlwaysProceedMethodInterceptor.invoke(AlwaysProceedMethodInterceptor.java:41)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.alfresco.repo.security.permissions.impl.ExceptionTranslatorMethodInterceptor.invoke(ExceptionTranslatorMethodInterceptor.java:53)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.alfresco.repo.audit.AuditMethodInterceptor.invoke(AuditMethodInterceptor.java:166)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.alfresco.repo.transaction.CheckTransactionAdvice.invoke(CheckTransactionAdvice.java:54)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.alfresco.repo.transaction.RetryingTransactionAdvice$1.execute(RetryingTransactionAdvice.java:71)
	at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:450)
	at org.alfresco.repo.transaction.RetryingTransactionAdvice.invoke(RetryingTransactionAdvice.java:74)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
	at com.sun.proxy.$Proxy186.getMetaData(Unknown Source)
	at org.alfresco.repo.web.scripts.quickshare.MetaDataGet.executeImpl(MetaDataGet.java:70)
	at org.springframework.extensions.webscripts.DeclarativeWebScript.execute(DeclarativeWebScript.java:64)
	at org.alfresco.repo.web.scripts.RepositoryContainer$3.execute(RepositoryContainer.java:519)
	at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:450)
	at org.alfresco.repo.web.scripts.RepositoryContainer.transactionedExecute(RepositoryContainer.java:595)
	at org.alfresco.repo.web.scripts.RepositoryContainer.transactionedExecuteAs(RepositoryContainer.java:664)
	at org.alfresco.repo.web.scripts.RepositoryContainer.executeScriptInternal(RepositoryContainer.java:435)
	at org.alfresco.repo.web.scripts.RepositoryContainer.executeScript(RepositoryContainer.java:315)
	at org.springframework.extensions.webscripts.AbstractRuntime.executeScript(AbstractRuntime.java:399)
	at org.springframework.extensions.webscripts.AbstractRuntime.executeScript(AbstractRuntime.java:210)
	at org.springframework.extensions.webscripts.servlet.WebScriptServlet.service(WebScriptServlet.java:132)
	at org.alfresco.repo.web.scripts.AlfrescoWebScriptServlet.service(AlfrescoWebScriptServlet.java:43)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
	at jdk.internal.reflect.GeneratedMethodAccessor627.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/javax.security.auth.Subject.doAsPrivileged(Subject.java:550)
	at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314)
	at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:170)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:225)
	at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:47)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:149)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:145)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at jdk.internal.reflect.GeneratedMethodAccessor620.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/javax.security.auth.Subject.doAsPrivileged(Subject.java:550)
	at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314)
	at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:191)
	at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:47)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:149)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:145)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
	at org.alfresco.module.aosmodule.service.ContextRootFilter.doFilter(ContextRootFilter.java:93)
	at jdk.internal.reflect.GeneratedMethodAccessor620.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/javax.security.auth.Subject.doAsPrivileged(Subject.java:550)
	at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314)
	at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:191)
	at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:47)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:149)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:145)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
	at org.alfresco.web.app.servlet.GlobalLocalizationFilter.doFilter(GlobalLocalizationFilter.java:68)
	at jdk.internal.reflect.GeneratedMethodAccessor620.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/javax.security.auth.Subject.doAsPrivileged(Subject.java:550)
	at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314)
	at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:191)
	at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:47)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:149)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:145)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
	at org.alfresco.web.app.servlet.ClearSecurityContextFilter.doFilter(ClearSecurityContextFilter.java:53)
	at jdk.internal.reflect.GeneratedMethodAccessor620.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/javax.security.auth.Subject.doAsPrivileged(Subject.java:550)
	at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314)
	at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:191)
	at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:47)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:149)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:145)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:493)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:660)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:798)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:808)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to serialize object [typeName=org.aldica.repo.ignite.cache.NodeAspectsCacheSet]
	at org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:853)
	at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:232)
	at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:165)
	at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:152)
	at org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteObject(BinaryWriterExImpl.java:524)
	at org.apache.ignite.internal.binary.BinaryWriterExImpl.writeObjectField(BinaryWriterExImpl.java:1194)
	at org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.write0(BinaryFieldAccessor.java:687)
	at org.apache.ignite.internal.binary.BinaryFieldAccessor.write(BinaryFieldAccessor.java:157)
	... 192 more
Caused by: org.alfresco.error.AlfrescoRuntimeException: 10120231 Cannot resolve {http://www.alfresco.org/model/system/1.0}referenceable to DB ID
	at org.aldica.repo.ignite.binary.NodeAspectsBinarySerializer.writeAspectsRawSerialForm(NodeAspectsBinarySerializer.java:142)
	at org.aldica.repo.ignite.binary.NodeAspectsBinarySerializer.writeBinary(NodeAspectsBinarySerializer.java:94)
	at org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:765)
	... 199 more
2020-11-12 07:28:56,660  ERROR [util.transaction.TransactionSupportUtil] [http-nio-8080-exec-27] After completion (committed) exception
org.alfresco.error.AlfrescoRuntimeException: 10120233 Failed to transfer updates to shared cache
	at org.alfresco.repo.cache.TransactionalCache.afterCommit(TransactionalCache.java:1104)
	at org.alfresco.util.transaction.TransactionSupportUtil$TransactionSynchronizationImpl.afterCompletion(TransactionSupportUtil.java:598)
	at org.springframework.transaction.support.TransactionSynchronizationUtils.invokeAfterCompletion(TransactionSynchronizationUtils.java:171)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.invokeAfterCompletion(AbstractPlatformTransactionManager.java:992)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerAfterCompletion(AbstractPlatformTransactionManager.java:967)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:788)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:714)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:534)
	at org.alfresco.util.transaction.SpringAwareUserTransaction.commit(SpringAwareUserTransaction.java:479)
	at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:472)
	at org.alfresco.repo.security.person.PersonServiceImpl.makeHomeFolderIfRequired(PersonServiceImpl.java:923)
	at org.alfresco.repo.security.person.PersonServiceImpl.getPersonImpl(PersonServiceImpl.java:527)
	at org.alfresco.repo.security.person.PersonServiceImpl.getPerson(PersonServiceImpl.java:484)
	at org.alfresco.repo.security.person.PersonServiceImpl.getPerson(PersonServiceImpl.java:441)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at net.sf.acegisecurity.intercept.method.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:80)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.alfresco.repo.security.permissions.impl.ExceptionTranslatorMethodInterceptor.invoke(ExceptionTranslatorMethodInterceptor.java:53)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.alfresco.repo.audit.AuditMethodInterceptor.invoke(AuditMethodInterceptor.java:166)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:295)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
	at com.sun.proxy.$Proxy122.getPerson(Unknown Source)
	at org.alfresco.repo.quickshare.QuickShareServiceImpl.getMetaData(QuickShareServiceImpl.java:552)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.alfresco.repo.security.permissions.impl.AlwaysProceedMethodInterceptor.invoke(AlwaysProceedMethodInterceptor.java:41)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.alfresco.repo.security.permissions.impl.ExceptionTranslatorMethodInterceptor.invoke(ExceptionTranslatorMethodInterceptor.java:53)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.alfresco.repo.audit.AuditMethodInterceptor.invoke(AuditMethodInterceptor.java:166)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.alfresco.repo.transaction.CheckTransactionAdvice.invoke(CheckTransactionAdvice.java:54)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.alfresco.repo.transaction.RetryingTransactionAdvice$1.execute(RetryingTransactionAdvice.java:71)
	at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:450)
	at org.alfresco.repo.transaction.RetryingTransactionAdvice.invoke(RetryingTransactionAdvice.java:74)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
	at com.sun.proxy.$Proxy186.getMetaData(Unknown Source)
	at org.alfresco.repo.web.scripts.quickshare.MetaDataGet.executeImpl(MetaDataGet.java:70)
	at org.springframework.extensions.webscripts.DeclarativeWebScript.execute(DeclarativeWebScript.java:64)
	at org.alfresco.repo.web.scripts.RepositoryContainer$3.execute(RepositoryContainer.java:519)
	at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:450)
	at org.alfresco.repo.web.scripts.RepositoryContainer.transactionedExecute(RepositoryContainer.java:595)
	at org.alfresco.repo.web.scripts.RepositoryContainer.transactionedExecuteAs(RepositoryContainer.java:664)
	at org.alfresco.repo.web.scripts.RepositoryContainer.executeScriptInternal(RepositoryContainer.java:435)
	at org.alfresco.repo.web.scripts.RepositoryContainer.executeScript(RepositoryContainer.java:315)
	at org.springframework.extensions.webscripts.AbstractRuntime.executeScript(AbstractRuntime.java:399)
	at org.springframework.extensions.webscripts.AbstractRuntime.executeScript(AbstractRuntime.java:210)
	at org.springframework.extensions.webscripts.servlet.WebScriptServlet.service(WebScriptServlet.java:132)
	at org.alfresco.repo.web.scripts.AlfrescoWebScriptServlet.service(AlfrescoWebScriptServlet.java:43)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
	at jdk.internal.reflect.GeneratedMethodAccessor627.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/javax.security.auth.Subject.doAsPrivileged(Subject.java:550)
	at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314)
	at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:170)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:225)
	at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:47)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:149)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:145)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at jdk.internal.reflect.GeneratedMethodAccessor620.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/javax.security.auth.Subject.doAsPrivileged(Subject.java:550)
	at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314)
	at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:191)
	at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:47)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:149)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:145)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
	at org.alfresco.module.aosmodule.service.ContextRootFilter.doFilter(ContextRootFilter.java:93)
	at jdk.internal.reflect.GeneratedMethodAccessor620.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/javax.security.auth.Subject.doAsPrivileged(Subject.java:550)
	at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314)
	at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:191)
	at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:47)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:149)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:145)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
	at org.alfresco.web.app.servlet.GlobalLocalizationFilter.doFilter(GlobalLocalizationFilter.java:68)
	at jdk.internal.reflect.GeneratedMethodAccessor620.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/javax.security.auth.Subject.doAsPrivileged(Subject.java:550)
	at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314)
	at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:191)
	at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:47)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:149)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:145)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
	at org.alfresco.web.app.servlet.ClearSecurityContextFilter.doFilter(ClearSecurityContextFilter.java:53)
	at jdk.internal.reflect.GeneratedMethodAccessor620.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282)
	at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/javax.security.auth.Subject.doAsPrivileged(Subject.java:550)
	at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314)
	at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:191)
	at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:47)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:149)
	at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:145)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:493)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:660)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:798)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:808)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.alfresco.error.AlfrescoRuntimeException: 10120232 CacheBucket postCommit transfer to shared cache failed: 
   Cache:      org.aldica.repo.ignite.cache.SimpleIgniteBackedCache@4b1270b6
   Key:        N.A.NodeVersionKey [nodeId=596, version=3]
   New Value:  [{http://www.alfresco.org/model/content/1.0}personDisabled, {http://www.alfresco.org/model/content/1.0}ownable, {http://www.alfresco.org/model/system/1.0}cascadeUpdate, {http://www.alfresco.org/model/system/1.0}referenceable, {http://www.alfresco.org/model/system/1.0}localized, {http://www.alfresco.org/model/content/1.0}preferences]
   Cache Value:null
	at org.alfresco.repo.cache.TransactionalCache.afterCommit(TransactionalCache.java:1088)
	... 162 more
Caused by: javax.cache.CacheException: class org.apache.ignite.IgniteCheckedException: Failed to serialize object [typeName=org.alfresco.repo.cache.TransactionalCache$ValueHolder]
	at org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1317)
	at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:2066)
	at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.getAndPut(IgniteCacheProxyImpl.java:1351)
	at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.getAndPut(GatewayProtectedCacheProxy.java:880)
	at org.aldica.repo.ignite.cache.SimpleIgniteBackedCache.getAndPutImpl(SimpleIgniteBackedCache.java:414)
	at org.aldica.repo.ignite.cache.SimpleIgniteBackedCache.put(SimpleIgniteBackedCache.java:309)
	at jdk.internal.reflect.GeneratedMethodAccessor167.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.aldica.repo.ignite.cache.SimpleCacheInvoker.invoke(SimpleCacheInvoker.java:49)
	at org.aldica.repo.ignite.cache.CacheFactoryImpl$SimpleLazySwapCacheInvoker.invoke(CacheFactoryImpl.java:726)
	at com.sun.proxy.$Proxy21.put(Unknown Source)
	at org.alfresco.repo.cache.TransactionalCache.putSharedCacheValue(TransactionalCache.java:502)
	at org.alfresco.repo.cache.TransactionalCache$NewCacheBucket.doPostCommit(TransactionalCache.java:1253)
	at org.alfresco.repo.cache.TransactionalCache.afterCommit(TransactionalCache.java:1078)
	... 162 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to serialize object [typeName=org.alfresco.repo.cache.TransactionalCache$ValueHolder]
	at org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7507)
	at org.apache.ignite.internal.util.future.GridFutureAdapter.resolve(GridFutureAdapter.java:260)
	at org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:172)
	at org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:141)
	at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.getAndPut0(GridDhtAtomicCache.java:619)
	at org.apache.ignite.internal.processors.cache.GridCacheAdapter.getAndPut(GridCacheAdapter.java:2471)
	at org.apache.ignite.internal.processors.cache.GridCacheAdapter.getAndPut(GridCacheAdapter.java:2450)
	at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.getAndPut(IgniteCacheProxyImpl.java:1348)
	... 174 more
Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to serialize object [typeName=org.alfresco.repo.cache.TransactionalCache$ValueHolder]
	at org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:853)
	at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:232)
	at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:165)
	at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:152)
	at org.apache.ignite.internal.binary.GridBinaryMarshaller.marshal(GridBinaryMarshaller.java:251)
	at org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.marshalToBinary(CacheObjectBinaryProcessorImpl.java:493)
	at org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toBinary(CacheObjectBinaryProcessorImpl.java:1248)
	at org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toCacheObject(CacheObjectBinaryProcessorImpl.java:1085)
	at org.apache.ignite.internal.processors.cache.GridCacheContext.toCacheObject(GridCacheContext.java:1846)
	at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.mapSingleUpdate(GridNearAtomicSingleUpdateFuture.java:552)
	at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.map(GridNearAtomicSingleUpdateFuture.java:457)
	at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.mapOnTopology(GridNearAtomicSingleUpdateFuture.java:446)
	at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.map(GridNearAtomicAbstractUpdateFuture.java:249)
	at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update0(GridDhtAtomicCache.java:1164)
	at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.getAndPut0(GridDhtAtomicCache.java:612)
	... 177 more
Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to write field [name=value]
	at org.apache.ignite.internal.binary.BinaryFieldAccessor.write(BinaryFieldAccessor.java:164)
	at org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:822)
	... 191 more
Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to serialize object [typeName=org.aldica.repo.ignite.cache.NodeAspectsCacheSet]
	at org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:853)
	at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:232)
	at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:165)
	at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:152)
	at org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteObject(BinaryWriterExImpl.java:524)
	at org.apache.ignite.internal.binary.BinaryWriterExImpl.writeObjectField(BinaryWriterExImpl.java:1194)
	at org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.write0(BinaryFieldAccessor.java:687)
	at org.apache.ignite.internal.binary.BinaryFieldAccessor.write(BinaryFieldAccessor.java:157)
	... 192 more
Caused by: org.alfresco.error.AlfrescoRuntimeException: 10120231 Cannot resolve {http://www.alfresco.org/model/system/1.0}referenceable to DB ID
	at org.aldica.repo.ignite.binary.NodeAspectsBinarySerializer.writeAspectsRawSerialForm(NodeAspectsBinarySerializer.java:142)
	at org.aldica.repo.ignite.binary.NodeAspectsBinarySerializer.writeBinary(NodeAspectsBinarySerializer.java:94)
	at org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:765)
	... 199 more

Tell us about your environment

Kubernetes as described above (running an Alfresco repository (Docker image) version 6.2.2-RC1.

Ignite binary configuration validation throws false-positive errors due to map handling bug

When more than one custom binary type configuration is used, the binary configuration validation of Ignite may run into a false-positive valdiation issue. This is due to a simple bug in handling the type configuration map as part of the node's attributes, specifically the fact that instead of the type name being used as the key for a particular binary type configuration, a boolean value indicating the non-null-ness of the type name is used. Since all type names in all binary type configurations will be non-null, this means that effectively only one binary type configuration will be put in the type configuration map, and which one that is depends on the inherhent ordering of an internal HashMap, so may differ from node to node.

Buggy code:

                BinaryConfiguration bCfg = ctx.config().getBinaryConfiguration();

                if (bCfg != null) {
                    Map<String, Object> map = new HashMap<>();

                    map.put("globIdMapper", bCfg.getIdMapper() != null ? bCfg.getIdMapper().getClass().getName() : null);
                    map.put("globSerializer", bCfg.getSerializer() != null ? bCfg.getSerializer().getClass() : null);
                    map.put("compactFooter", bCfg.isCompactFooter());

                    if (bCfg.getTypeConfigurations() != null) {
                        Map<Object, Object> typeCfgsMap = new HashMap<>();

                        for (BinaryTypeConfiguration c : bCfg.getTypeConfigurations()) {
                            typeCfgsMap.put(
                                c.getTypeName() != null, // <<< THIS is wrong
                                Arrays.asList(
                                    c.getIdMapper() != null ? c.getIdMapper().getClass() : null,
                                    c.getSerializer() != null ? c.getSerializer().getClass() : null,
                                    c.isEnum()
                                )
                            );

                            if (c.isEnum())
                                BinaryUtils.validateEnumValues(c.getTypeName(), c.getEnumValues());
                        }

                        map.put("typeCfgs", typeCfgsMap);
                    }

                    ctx.addNodeAttribute(IgniteNodeAttributes.ATTR_BINARY_CONFIGURATION, map);
                }

This is an upstream issue with Ignite, and so far I have not been able to find an issue for this in Ignite's JIRA.

For the time being, we are forced to essentially disable configuration consistency checks to deal with this. These types of false-positive validation errors can affect any situation where there are more than one grid instances on the Repository-tier, so either more than one repository or one repository and at least one companion app. This issue was found while adaption PR 29 after PR 34 adding various custom serialisers was merged.

Build failure

I have checked out the latest master branch and I get a build failure, when I try to build the project with Maven:

$ mvn clean package -DskipTests
[INFO] 2 errors 
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Alternative/Alfresco Distributed Cache - Parent 1.0.0.0-SNAPSHOT:
[INFO] 
[INFO] Alternative/Alfresco Distributed Cache - Parent .... SUCCESS [  0.558 s]
[INFO] Alternative/Alfresco Distributed Cache - Common Ignite Library SUCCESS [  5.160 s]
[INFO] Alternative/Alfresco Distributed Cache - Repository Ignite Module FAILURE [  8.398 s]
[INFO] Alternative/Alfresco Distributed Cache - Share Ignite Module SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  14.407 s
[INFO] Finished at: 2019-05-23T09:47:03+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile (default-testCompile) on project aldica-repo-ignite: Compilation failure: Compilation failure: 
[ERROR] /home/andreas/alfresco/alfresco-community-open-grid/repository/src/test/java/org/aldica/repo/ignite/integration/CacheConsistency.java:[290,16] error: cannot find symbol
[ERROR]   symbol:   method setContent(String,ByteArrayInputStream,String)
[ERROR]   location: variable nodesAPI of type NodesV1
[ERROR] /home/andreas/alfresco/alfresco-community-open-grid/repository/src/test/java/org/aldica/repo/ignite/integration/CacheConsistency.java:[294,15] error: cannot find symbol
[ERROR]   symbol:   method setProperty(String,Boolean)
[ERROR]   location: variable updates of type CommonNodeEntity<PermissionsInfo>
[ERROR] -> [Help 1]

Copy dependencies from the .m2 folder

Currently, we are using the command mvn dependency:copy to copy the Acosix Utility into the required folder when building our Docker images. This should be changed so the dependencies are taken from the .m2 folder instead

Use of SNAPSHOT versions in the POM(s)

The use of SNAPSHOT versions in the POMs causes a little trouble when building the Docker images using the Dockerfiles here. The reason is that Maven renames e.g. the repo AMP to

[INFO] Copying de.acosix.alfresco.utility.core.repo-1.1.0-SNAPSHOT.amp to 
/aldica/target/de.acosix.alfresco.utility.core.repo-1.1.0-20200221.175559-23.amp

I.e. the resulting AMP contains a timestamp/tag in the file name. When the AMP is copied in the Dockerfile, the following error occurs:

[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  28.722 s (Wall Clock)
[INFO] Finished at: 2020-04-16T08:54:44Z
[INFO] ------------------------------------------------------------------------
Removing intermediate container 906ab4b62704
 ---> 8839bf237447
Step 13/19 : FROM alfresco/alfresco-content-repository-community:6.2.0-ga
 ---> 5644477d8b35
Step 14/19 : COPY --from=0 --chown=root:Alfresco /aldica/repository/target/aldica-repo-ignite-1.0.0.0-SNAPSHOT.amp .
 ---> 8762dd85fd34
Step 15/19 : COPY --from=0 --chown=root:Alfresco /aldica/target/de.acosix.alfresco.utility.core.repo-1.1.0-SNAPSHOT.amp .
COPY failed: stat /mnt/nvme0n1p6/docker/overlay2/b6427d78fac88bbf2036efbe4466fbdb7af0816d4db031785e4b12868efe8bf2/merged/aldica/target/de.acosix.alfresco.utility.core.repo-1.1.0-SNAPSHOT.amp: no such file or directory

The issue does not happen when a non-SNAPSHOT version the Acosix utility module is used. This leads to two questions:

• It it possible that non-SNAPSHOT versions of the dependencies can be released, so we can make use of these?
• The aldica AMPs are also SNAPSHOT versions, so should we change this also? In general, we could use the common versioning system <major version>.<minor version>.<hotfix/bugfix> (where the minor version is bumped when new features are added and the major version is bumped when backwards breaking changes are introduced)?

Share cannot start if initial members not set

BUG

An aldica-enabled Share container cannot start without errors if aldica.core.initialMembers is not set.

Expected behavior

Share should be able to start / join grid even if aldica.core.initialMembers is not set.

Actual behavior

When the Share container(s) start up, the error below is thrown in the log.

Steps to reproduce the behavior

Remove the two lines containing the aldica.core.initialMembers setting from the Share containers in our docker-compose.yml file and start the stack with docker-compose up -d --build.

Additional details (analysis so far, log statements, references, etc.)

The following error is thrown in the log:

share1_1                 |  [14:39:42]    __________  ________________ 
share1_1                 | [14:39:42]   /  _/ ___/ |/ /  _/_  __/ __/ 
share1_1                 | [14:39:42]  _/ // (7 7    // /  / / / _/   
share1_1                 | [14:39:42] /___/\___/_/|_/___/ /_/ /___/  
share1_1                 | [14:39:42] 
share1_1                 | [14:39:42] ver. 2.8.1#20200521-sha1:86422096
share1_1                 | [14:39:42] 2020 Copyright(C) Apache Software Foundation
share1_1                 | [14:39:42] 
share1_1                 | [14:39:42] Ignite documentation: http://ignite.apache.org
share1_1                 | [14:39:42] 
share1_1                 | [14:39:42] Quiet mode.
share1_1                 | [14:39:42]   ^-- Logging by 'Slf4jLogger [impl=org.slf4j.impl.Log4jLoggerAdapter(o.a.i.i.IgniteKernal%shareGrid), quiet=true]'
share1_1                 | [14:39:42]   ^-- To see **FULL** console log here add -DIGNITE_QUIET=false or "-v" to ignite.{sh|bat}
share1_1                 | [14:39:42] 
share1_1                 | [14:39:42] OS: Linux 4.15.0-122-generic amd64
share1_1                 | [14:39:42] VM information: OpenJDK Runtime Environment 11.0.7+10 Oracle Corporation OpenJDK 64-Bit Server VM 11.0.7+10
share1_1                 | 2020-11-13 14:39:42,051  WARN  [ignite.internal.IgniteKernal%shareGrid] [localhost-startStop-1] Please set system property '-Djava.net.preferIPv4Stack=true' to avoid possible problems in mixed environments.
share1_1                 |  [14:39:42] Please set system property '-Djava.net.preferIPv4Stack=true' to avoid possible problems in mixed environments.
share1_1                 | 2020-11-13 14:39:42,144  WARN  [ignite.internal.GridDiagnostic] [pub-#12%shareGrid%] Initial heap size is 500MB (should be no less than 512MB, use -Xms512m -Xmx512m).
share1_1                 |  [14:39:42] Initial heap size is 500MB (should be no less than 512MB, use -Xms512m -Xmx512m).
share1_1                 | [14:39:42] Configured plugins:
share1_1                 | [14:39:42]   ^-- SimpleSecurityPlugin (org.aldica:aldica-common-ignite) 1.0.0
share1_1                 | [14:39:42]   ^-- Copyright 2020 Acosix GmbH, Copyright 2020 MAGENTA ApS
share1_1                 | [14:39:42] 
share1_1                 | [14:39:42] Configured failure handler: [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]]]
share1_1                 | 2020-11-13 14:39:42,373  INFO  [ignite.plugin.SimpleSecurityProcessor] [localhost-startStop-1] No allowed client credentials have been configured - no client will be allowed to connect
share1_1                 |  2020-11-13 14:39:42,764  WARN  [communication.tcp.TcpCommunicationSpi] [localhost-startStop-1] Failure detection timeout will be ignored (one of SPI parameters has been set explicitly)
share1_1                 |  [14:39:43] Security status [authentication=on, tls/ssl=off]
share1_1                 | 2020-11-13 14:39:43,356  WARN  [ignite.discovery.CredentialsAwareTcpDiscoverySpi] [localhost-startStop-1] Failure detection timeout will be ignored (one of SPI parameters has been set explicitly)
share1_1                 |  2020-11-13 14:39:43,416  WARN  [cache.persistence.IgniteCacheDatabaseSharedManager] [localhost-startStop-1] DataRegionConfiguration.maxWalArchiveSize instead DataRegionConfiguration.walHistorySize would be used for removing old archive wal files
share1_1                 |  2020-11-13 14:39:43,453  WARN  [dht.topology.PartitionsEvictManager] [localhost-startStop-1] Logging at INFO level without checking if INFO level is enabled: Evict partition permits=1
share1_1                 |  2020-11-13 14:39:43,987  INFO  [ignite.discovery.CredentialsAwareTcpDiscoverySpi] [localhost-startStop-1] Connection check threshold is calculated: 2000
share1_1                 |  2020-11-13 14:39:44,014  INFO  [ignite.discovery.CredentialsAwareTcpDiscoverySpi] [localhost-startStop-1] Successfully bound to TCP port [port=47140, localHost=share1/172.24.0.8, locNodeId=25af04d7-bdbf-4529-ad41-b2ccf1188750]
share1_1                 |  2020-11-13 14:39:44,028  ERROR [ignite.internal.IgniteKernal%shareGrid] [localhost-startStop-1] Failed to start manager: GridManagerAdapter [enabled=true, name=o.a.i.i.managers.discovery.GridDiscoveryManager]
share1_1                 |  class org.apache.ignite.IgniteCheckedException: Failed to start SPI: TcpDiscoverySpi [addrRslvr=org.aldica.common.ignite.discovery.GridAddressResolutionManager@45d59dcc, sockTimeout=5000, ackTimeout=5000, marsh=JdkMarshaller [clsFilter=org.apache.ignite.marshaller.MarshallerUtils$1@6674f891], reconCnt=10, reconDelay=2000, maxAckTimeout=600000, soLinger=5, forceSrvMode=false, clientReconnectDisabled=false, internalLsnr=null, skipAddrsRandomization=false]
share1_1                 | 	at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:302)
share1_1                 | 	at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:943)
share1_1                 | 	at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1960)
share1_1                 | 	at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1276)
share1_1                 | 	at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2045)
share1_1                 | 	at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1703)
share1_1                 | 	at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1117)
share1_1                 | 	at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:637)
share1_1                 | 	at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:563)
share1_1                 | 	at org.apache.ignite.Ignition.start(Ignition.java:321)
share1_1                 | 	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.lambda$initializeGrid$2(SpringIgniteLifecycleBean.java:130)
share1_1                 | 	at org.aldica.common.ignite.context.ExternalContext.withExternalContext(ExternalContext.java:112)
share1_1                 | 	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.initializeGrid(SpringIgniteLifecycleBean.java:120)
share1_1                 | 	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.onApplicationEvent(SpringIgniteLifecycleBean.java:68)
share1_1                 | 	at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
share1_1                 | 	at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
share1_1                 | 	at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
share1_1                 | 	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:402)
share1_1                 | 	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:359)
share1_1                 | 	at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:896)
share1_1                 | 	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:552)
share1_1                 | 	at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:401)
share1_1                 | 	at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:292)
share1_1                 | 	at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:103)
share1_1                 | 	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4699)
share1_1                 | 	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5165)
share1_1                 | 	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
share1_1                 | 	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:743)
share1_1                 | 	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:719)
share1_1                 | 	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:714)
share1_1                 | 	at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1125)
share1_1                 | 	at org.apache.catalina.startup.HostConfig$DeployDirectory.run(HostConfig.java:1859)
share1_1                 | 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
share1_1                 | 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
share1_1                 | 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
share1_1                 | 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
share1_1                 | 	at java.base/java.lang.Thread.run(Thread.java:834)
share1_1                 | Caused by: class org.apache.ignite.spi.IgniteSpiException: Non-shared IP finder must have IP addresses specified in TcpDiscoveryIpFinder.getRegisteredAddresses() configuration property (specify list of IP addresses in configuration).
share1_1                 | 	at org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:413)
share1_1                 | 	at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2099)
share1_1                 | 	at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:299)
share1_1                 | 	... 36 more
share1_1                 | 2020-11-13 14:39:44,044  ERROR [ignite.internal.IgniteKernal%shareGrid] [localhost-startStop-1] Got exception while starting (will rollback startup routine).
share1_1                 |  class org.apache.ignite.IgniteCheckedException: Failed to start manager: GridManagerAdapter [enabled=true, name=org.apache.ignite.internal.managers.discovery.GridDiscoveryManager]
share1_1                 | 	at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1965)
share1_1                 | 	at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1276)
share1_1                 | 	at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2045)
share1_1                 | 	at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1703)
share1_1                 | 	at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1117)
share1_1                 | 	at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:637)
share1_1                 | 	at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:563)
share1_1                 | 	at org.apache.ignite.Ignition.start(Ignition.java:321)
share1_1                 | 	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.lambda$initializeGrid$2(SpringIgniteLifecycleBean.java:130)
share1_1                 | 	at org.aldica.common.ignite.context.ExternalContext.withExternalContext(ExternalContext.java:112)
share1_1                 | 	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.initializeGrid(SpringIgniteLifecycleBean.java:120)
share1_1                 | 	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.onApplicationEvent(SpringIgniteLifecycleBean.java:68)
share1_1                 | 	at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
share1_1                 | 	at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
share1_1                 | 	at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
share1_1                 | 	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:402)
share1_1                 | 	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:359)
share1_1                 | 	at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:896)
share1_1                 | 	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:552)
share1_1                 | 	at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:401)
share1_1                 | 	at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:292)
share1_1                 | 	at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:103)
share1_1                 | 	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4699)
share1_1                 | 	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5165)
share1_1                 | 	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
share1_1                 | 	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:743)
share1_1                 | 	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:719)
share1_1                 | 	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:714)
share1_1                 | 	at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1125)
share1_1                 | 	at org.apache.catalina.startup.HostConfig$DeployDirectory.run(HostConfig.java:1859)
share1_1                 | 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
share1_1                 | 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
share1_1                 | 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
share1_1                 | 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
share1_1                 | 	at java.base/java.lang.Thread.run(Thread.java:834)
share1_1                 | Caused by: class org.apache.ignite.IgniteCheckedException: Failed to start SPI: TcpDiscoverySpi [addrRslvr=org.aldica.common.ignite.discovery.GridAddressResolutionManager@45d59dcc, sockTimeout=5000, ackTimeout=5000, marsh=JdkMarshaller [clsFilter=org.apache.ignite.marshaller.MarshallerUtils$1@6674f891], reconCnt=10, reconDelay=2000, maxAckTimeout=600000, soLinger=5, forceSrvMode=false, clientReconnectDisabled=false, internalLsnr=null, skipAddrsRandomization=false]
share1_1                 | 	at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:302)
share1_1                 | 	at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:943)
share1_1                 | 	at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1960)
share1_1                 | 	... 34 more
share1_1                 | Caused by: class org.apache.ignite.spi.IgniteSpiException: Non-shared IP finder must have IP addresses specified in TcpDiscoveryIpFinder.getRegisteredAddresses() configuration property (specify list of IP addresses in configuration).
share1_1                 | 	at org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:413)
share1_1                 | 	at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2099)
share1_1                 | 	at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:299)
share1_1                 | 	... 36 more
share1_1                 | 2020-11-13 14:39:59,097  INFO  [ignite.discovery.CredentialsAwareTcpDiscoverySpi] [localhost-startStop-1] No verification for local node leave has been received from coordinator (will stop node anyway).
share1_1                 |  2020-11-13 14:39:59,100  ERROR [ignite.internal.IgniteKernal%shareGrid] [localhost-startStop-1] Failed to stop component (ignoring): GridProcessorAdapter []
share1_1                 |  java.lang.UnsupportedOperationException
share1_1                 | 	at org.jsr166.ConcurrentLinkedHashMap.clear(ConcurrentLinkedHashMap.java:1542)
share1_1                 | 	at org.apache.ignite.internal.processors.job.GridJobProcessor.stop(GridJobProcessor.java:375)
share1_1                 | 	at org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:2627)
share1_1                 | 	at org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:2499)
share1_1                 | 	at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1395)
share1_1                 | 	at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2045)
share1_1                 | 	at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1703)
share1_1                 | 	at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1117)
share1_1                 | 	at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:637)
share1_1                 | 	at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:563)
share1_1                 | 	at org.apache.ignite.Ignition.start(Ignition.java:321)
share1_1                 | 	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.lambda$initializeGrid$2(SpringIgniteLifecycleBean.java:130)
share1_1                 | 	at org.aldica.common.ignite.context.ExternalContext.withExternalContext(ExternalContext.java:112)
share1_1                 | 	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.initializeGrid(SpringIgniteLifecycleBean.java:120)
share1_1                 | 	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.onApplicationEvent(SpringIgniteLifecycleBean.java:68)
share1_1                 | 	at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
share1_1                 | 	at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
share1_1                 | 	at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
share1_1                 | 	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:402)
share1_1                 | 	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:359)
share1_1                 | 	at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:896)
share1_1                 | 	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:552)
share1_1                 | 	at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:401)
share1_1                 | 	at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:292)
share1_1                 | 	at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:103)
share1_1                 | 	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4699)
share1_1                 | 	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5165)
share1_1                 | 	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
share1_1                 | 	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:743)
share1_1                 | 	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:719)
share1_1                 | 	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:714)
share1_1                 | 	at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1125)
share1_1                 | 	at org.apache.catalina.startup.HostConfig$DeployDirectory.run(HostConfig.java:1859)
share1_1                 | 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
share1_1                 | 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
share1_1                 | 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
share1_1                 | 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
share1_1                 | 	at java.base/java.lang.Thread.run(Thread.java:834)
share1_1                 | [14:39:59] Ignite node stopped wih ERRORS [name=shareGrid, uptime=00:00:24.224]
share1_1                 | 2020-11-13 14:39:59,126  ERROR [web.context.ContextLoader] [localhost-startStop-1] Context initialization failed
share1_1                 |  class org.apache.ignite.IgniteException: Failed to start manager: GridManagerAdapter [enabled=true, name=org.apache.ignite.internal.managers.discovery.GridDiscoveryManager]
share1_1                 | 	at org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:1067)
share1_1                 | 	at org.apache.ignite.Ignition.start(Ignition.java:324)
share1_1                 | 	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.lambda$initializeGrid$2(SpringIgniteLifecycleBean.java:130)
share1_1                 | 	at org.aldica.common.ignite.context.ExternalContext.withExternalContext(ExternalContext.java:112)
share1_1                 | 	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.initializeGrid(SpringIgniteLifecycleBean.java:120)
share1_1                 | 	at org.aldica.common.ignite.lifecycle.SpringIgniteLifecycleBean.onApplicationEvent(SpringIgniteLifecycleBean.java:68)
share1_1                 | 	at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
share1_1                 | 	at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
share1_1                 | 	at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
share1_1                 | 	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:402)
share1_1                 | 	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:359)
share1_1                 | 	at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:896)
share1_1                 | 	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:552)
share1_1                 | 	at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:401)
share1_1                 | 	at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:292)
share1_1                 | 	at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:103)
share1_1                 | 	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4699)
share1_1                 | 	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5165)
share1_1                 | 	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
share1_1                 | 	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:743)
share1_1                 | 	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:719)
share1_1                 | 	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:714)
share1_1                 | 	at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1125)
share1_1                 | 	at org.apache.catalina.startup.HostConfig$DeployDirectory.run(HostConfig.java:1859)
share1_1                 | 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
share1_1                 | 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
share1_1                 | 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
share1_1                 | 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
share1_1                 | 	at java.base/java.lang.Thread.run(Thread.java:834)
share1_1                 | Caused by: class org.apache.ignite.IgniteCheckedException: Failed to start manager: GridManagerAdapter [enabled=true, name=org.apache.ignite.internal.managers.discovery.GridDiscoveryManager]
share1_1                 | 	at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1965)
share1_1                 | 	at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1276)
share1_1                 | 	at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2045)
share1_1                 | 	at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1703)
share1_1                 | 	at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1117)
share1_1                 | 	at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:637)
share1_1                 | 	at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:563)
share1_1                 | 	at org.apache.ignite.Ignition.start(Ignition.java:321)
share1_1                 | 	... 27 more
share1_1                 | Caused by: class org.apache.ignite.IgniteCheckedException: Failed to start SPI: TcpDiscoverySpi [addrRslvr=org.aldica.common.ignite.discovery.GridAddressResolutionManager@45d59dcc, sockTimeout=5000, ackTimeout=5000, marsh=JdkMarshaller [clsFilter=org.apache.ignite.marshaller.MarshallerUtils$1@6674f891], reconCnt=10, reconDelay=2000, maxAckTimeout=600000, soLinger=5, forceSrvMode=false, clientReconnectDisabled=false, internalLsnr=null, skipAddrsRandomization=false]
share1_1                 | 	at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:302)
share1_1                 | 	at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:943)
share1_1                 | 	at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1960)
share1_1                 | 	... 34 more
share1_1                 | Caused by: class org.apache.ignite.spi.IgniteSpiException: Non-shared IP finder must have IP addresses specified in TcpDiscoveryIpFinder.getRegisteredAddresses() configuration property (specify list of IP addresses in configuration).
share1_1                 | 	at org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:413)
share1_1                 | 	at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2099)
share1_1                 | 	at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:299)
share1_1                 | 	... 36 more
share1_1                 | 13-Nov-2020 14:39:59.378 SEVERE [localhost-startStop-1] org.apache.catalina.core.StandardContext.startInternal One or more listeners failed to start. Full details will be found in the appropriate container log file

Tell us about your environment

Docker Compose file (modified as described above) provided by the aldica project.

Ignite-backed caches may exceed configured limits

In my "1 million cached nodes" memory benchmark I noticed that after explicitly loading 1 million nodes into caches, the caches for properties and aspects contained slightly more than the configured limit of 1 million entries (presumably standard nodes loaded by Alfresco, e.g. company home, people / group nodes etc). This is slightly different to the default Alfresco caches, which never exceed the configured limit, and in fact, never reach it as they evict nodes immediately upon reaching the limit (it appears to be an exclusive upper bound rather than an inclusive one). We should at least make sure that we never exceed the configured limit, though I would prefer to treat it as an inclusive upper bound rather than an exclusive one for a more predictable behaviour.