oracle / coherence

Oracle Coherence Community Edition

---

Oracle Coherence Community Edition

Contents

1. Introduction
2. Downloading Coherence Community Edition
3. Coherence Overview
4. Getting Started
   1. Install the Coherence CLI
   2. Create a Cluster
   3. CohQL Console
   4. Coherence Console
   5. Code Example
5. Building
6. Documentation
7. Examples
8. Contributing
9. License

Introduction

Coherence is a scalable, fault-tolerant, cloud-ready, distributed platform for building grid-based applications and reliably storing data. The product is used at scale, for both compute and raw storage, in a vast array of industries such as critical financial trading systems, high performance telecommunication products and eCommerce applications.

Typically these deployments do not tolerate any downtime and Coherence is chosen due to its novel features in death detection, application data evolvability, and the robust, battle-hardened core of the product that enables it to be seamlessly deployed and adapted within any ecosystem.

At a high level, Coherence provides an implementation of the familiar Map<K,V> interface but rather than storing the associated data in the local process it is partitioned (or sharded) across a number of designated remote nodes. This partitioning enables applications to not only distribute (and therefore scale) their storage across multiple processes, machines, racks, and data centers but also to perform grid-based processing to truly harness the CPU resources of the machines.

The Coherence interface NamedMap<K,V> (an extension of Map<K,V>) provides methods to query, aggregate (map/reduce style) and compute (send functions to storage nodes for locally executed mutations) the data set. These capabilities, in addition to numerous other features, enable Coherence to be used as a framework for writing robust, distributed applications.

Downloading Coherence Community Edition

As Coherence is generally embedded into an application by using Coherence APIs, the natural place to consume this dependency is from Maven:

<dependencies>
    <dependency>
        <groupId>com.oracle.coherence.ce</groupId>
        <artifactId>coherence</artifactId>
        <version>22.06.7</version>
    </dependency>
</dependencies>

You can also get Coherence container images from the official GitHub Packages site. For other language clients, follow the links to C++, .NET, JavaScript, Go and Python. For commercial offerings, go to Oracle Technology Network.

Overview

First and foremost, Coherence provides a fundamental service that is responsible for all facets of clustering and is a common denominator / building block for all other Coherence services. This service, referred to as 'service 0' internally, ensures that the mesh of members is maintained and responsive, taking action to collaboratively evict, shun, or in some cases, voluntarily depart the cluster when deemed necessary. As members join and leave the cluster, other Coherence services are notified, thus enabling those services to react accordingly.

Note: This part of the Coherence product has been in production for more that 10 years, being the subject of some extensive and imaginative testing. While this feature has been discussed here, it certainly is not something that customers, generally, interact with directly, but is important to be aware of.

Coherence services build on top of the cluster service. The key implementations to be aware of are PartitionedService, InvocationService, and ProxyService.

In the majority of cases, customers deal with maps. A map is represented by an implementation of NamedMap<K,V>. A NamedMap is hosted by a service, generally the PartitionedService, and is the entry point to store, retrieve, aggregate, query, and stream data.

Coherence Maps provide a number of features:

• Fundamental key-based access: get/put getAll/putAll.
• Client-side and storage-side events:
  • MapListeners to asynchronously notify clients of changes to data.
  • EventInterceptors (either sync or async) to be notified storage level events, including mutations, partition transfer, failover, and so on.
• NearCaches - Locally cached data based on previous requests with local content invalidated upon changes in the storage tier.
• ViewCaches - Locally stored view of remote data that can be a subset based on a predicate and is kept in sync, real time.
• Queries - Distributed, parallel query evaluation to return matching key, values, or entries with potential to optimize performance with indices.
• Aggregations - A map/reduce style aggregation where data is aggregated in parallel on all storage nodes, and results streamed back to the client for aggregation of those results to produce a final result.
• Data local processing - Ability to send a function to the relevant storage node to execute processing logic for the appropriate entries with exclusive access.
• Partition local transactions - Ability to perform scalable transactions by associating data (thus being on the same partition) and manipulating other entries on the same partition, potentially across different maps.
• Non-blocking / async NamedMap API
• C++ and .NET clients - Access the same NamedMap API from either C++ or .NET.
• Portable Object Format - Optimized serialization format, with the ability to navigate the serialized form for optimized queries, aggregations, or data processing.
• Integration with Databases - Database and third party data integration with CacheStores, including both synchronous or asynchronous writes.
• CohQL - Ansi-style query language with a console for adhoc queries.
• Topics - Distributed topics implementation that offers pub/sub messaging with the storage capacity, the cluster, and parallelizable subscribers.

Coherence also provides a number of non-functional features:

• Rock solid clustering - Highly tuned and robust clustering stack that enables Coherence to scale to thousands of members in a cluster with thousands of partitions and terabytes of data being accessed, mutated, queried, and aggregated concurrently.
• Safety first - Resilient data management that ensures backup copies are on distinct machines, racks, or sites, and the ability to maintain multiple backups.
• 24/7 Availability - Zero downtime with rolling redeployment of cluster members to upgrade application or product versions.
  • Backward and forward compatibility of product upgrades, including major versions.
• Persistent Maps - Ability to use local file system persistence (thus avoid extra network hops) and leverage Coherence consensus protocols to perform distributed disk recovery when appropriate.
• Distributed State Snapshot - Ability to perform distributed point-in-time snapshot of cluster state, and recover snapshot in this or a different cluster (leverages persistence feature).
• Lossy redundancy - Ability to reduce the redundancy guarantee by making backups and/or persistence asynchronous from a client perspective.
• Single Mangement View - Provides insight into the cluster with a single JMX server that provides a view of all members of the cluster.
• Management over REST - All JMX data and operations can be performed over REST, including cluster wide thread dumps and heapdumps.
• Non-cluster Access - Provides access to the cluster from the outside via proxies, for distant (high latency) clients and for non-java languages such as C++ and .NET.
• Kubernetes friendly - Enables seamless and safe deployment of applications to k8s with our own operator.

Getting Started

Prerequisites

You must have the following installed and available on your PATH.

1. Java - JDK 17 or higher
2. Maven - 3.8.5 or higher
3. Cohrence CLI Installed (see below)

The following example shows you how to quickly get started with Coherence using the Coherence CLI to create a 3 node Coherence cluster scoped to you local machine. You will then access data using the CohQL and Coherence consoles.

Install the Coherence CLI

For macOS or Linux platforms, use the following to install the latest version of the CLI:

curl -sL https://raw.githubusercontent.com/oracle/coherence-cli/main/scripts/install.sh | bash

When you install the CLI, administrative privileges are required as the cohctl executable is moved to the /usr/local/bin directory.

For Windows, see here for installation guide.

Create and start a Cluster

Use the following command to create a 3 node Coherence cluster called my-cluster, scoped to your local machine using the default of Coherence CE 22.06.7.

$ cohctl create cluster my-cluster

Cluster name:         my-cluster
Cluster version:      22.06.7
Cluster port:         7574
Management port:      30000
Replica count:        3
Initial memory:       128m
Persistence mode:     on-demand
Group ID:             com.oracle.coherence.ce
Additional artifacts:
Startup Profile:      
Dependency Tool:      mvn
Are you sure you want to create the cluster with the above details? (y/n) y

Checking 3 Maven dependencies...
- com.oracle.coherence.ce:coherence:22.06.7
- com.oracle.coherence.ce:coherence-json:22.06.7
- org.jline:jline:3.20.0
Starting 3 cluster members for cluster my-cluster
Starting cluster member storage-0...
Starting cluster member storage-1...
Starting cluster member storage-2...
Current context is now my-cluster
Cluster added and started

Note: If you do not have the Maven artefacts locally, it may take a short while to download them from Maven central.

Once the cluster is created, wait it a couple of seconds, and use the following command to see the members.

$ cohctl get members

Using cluster connection 'my-cluster' from current context.

Total cluster members: 3
Cluster Heap - Total: 384 MB Used: 114 MB Available: 270 MB (70.3%)
Storage Heap - Total: 128 MB Used: 16 MB Available: 112 MB (87.5%)

NODE ID  ADDRESS     PORT   PROCESS  MEMBER     ROLE             STORAGE  MAX HEAP  USED HEAP  AVAIL HEAP
      1  /127.0.0.1  55654    58270  storage-1  CoherenceServer  true       128 MB      16 MB      112 MB
      2  /127.0.0.1  55655    58271  storage-2  CoherenceServer  true       128 MB      74 MB       54 MB
      3  /127.0.0.1  55656    58269  storage-0  CoherenceServer  true       128 MB      24 MB      104 MB

Note: If you do not see the above, then ensure the java executable is on your PATH, you are using JDK17, and then issue cohctl start cluster my-cluster to start the cluster.

CohQL Console

Start the CohQL Console using the CLI, and run the statements at the CohQL> prompt to insert data into your cache.

$ cohctl start cohql

CohQL> select * from welcomes

CohQL> insert into welcomes key 'english' value 'Hello'

CohQL> insert into welcomes key 'spanish' value 'Hola'

CohQL> insert into welcomes key 'french' value 'Bonjour'

CohQL> select key(), value() from welcomes
Results
["french", "Bonjour"]
["english", "Hello"]
["spanish", "Hola"]

CohQL> bye

# Restart to CohQL to show that the data is still present in the Coherence cluster.

$ cohctl start cohql

CohQL> select key(), value() from welcomes
Results
["french", "Bonjour"]
["english", "Hello"]
["spanish", "Hola"]

CohQL> bye

Coherence Console

Use the following command to start the Coherence console, which is a different way to interact with the data in a Cache.

$ cohctl start console

Map (?): cache welcomes

Map (welcomes): get english
Hello

Map (welcomes): list
french = Bonjour
spanish = Hola
english = Hello

Map (welcomes): put australian Gudday
null

Map (welcomes): list
spanish = Hola
english = Hello
australian = Gudday
french = Bonjour

Map (welcomes): bye

Shutdown your Cluster

Note: Ensure you shutdown your Coherence cluster using the following:

cohctl stop cluster my-cluster

Programmatic Hello Coherence Example

The following example illustrates starting a storage enabled Coherence server, followed by running the HelloCoherence application. The HelloCoherence application inserts and retrieves data from the Coherence server.

Build HelloCoherence

1. Create a maven project either manually or by using an archetype such as maven-archetype-quickstart
2. Add a dependency to the pom file:

    <dependency>
        <groupId>com.oracle.coherence.ce</groupId>
        <artifactId>coherence</artifactId>
        <version>22.06.7</version>
    </dependency>

3. Copy and paste the following source to a file named src/main/java/HelloCoherence.java:

    import com.tangosol.net.CacheFactory;
    import com.tangosol.net.NamedMap;

    public class HelloCoherence
        {
        // ----- static methods -------------------------------------------------

        public static void main(String[] asArgs)
            {
            NamedMap<String, String> map = CacheFactory.getCache("welcomes");

            System.out.printf("Accessing map \"%s\" containing %d entries\n",
                    map.getName(),
                    map.size());

            map.put("english", "Hello");
            map.put("spanish", "Hola");
            map.put("french" , "Bonjour");

            // list
            map.entrySet().forEach(System.out::println);
            }
        }

4. Compile the maven project:

    mvn package

5. Start a Storage Server

    mvn exec:java -Dexec.mainClass="com.tangosol.net.DefaultCacheServer" &

6. Run HelloCoherence

    mvn exec:java -Dexec.mainClass="HelloCoherence"

7. Confirm you see output including the following:

    Accessing map "welcomes" containing 3 entries
    ConverterEntry{Key="french", Value="Bonjour"}
    ConverterEntry{Key="spanish", Value="Hola"}
    ConverterEntry{Key="english", Value="Hello"}

8. Kill the storage server started previously:

    kill %1

Building

$> git clone [email protected]:oracle/coherence.git
$> cd coherence/prj

# build Coherence module
$> mvn clean install

# build Coherence module skipping tests
$> mvn clean install -DskipTests

# build all other modules skipping tests
$> mvn -Pmodules clean install -DskipTests

# build specific module, including all dependent modules and run tests
$> mvn -Pmodules -am -pl test/functional/persistence clean verify

# only build coherence.jar without running tests
$> mvn -am -pl coherence clean install -DskipTests

# only build coherence.jar and skip compilation of CDBs and tests
$> mvn -am -pl coherence clean install -DskipTests -Dtde.compile.not.required

Documentation

Oracle Coherence Documentation

Oracle Coherence product documentation is available here.

Features Not Included in Coherence Community Edition

The following Oracle Coherence features are not included in Coherence Community Edition:

• Management of Coherence via the Oracle WebLogic Management Framework
• Deployment of Grid Archives (GARs)
• HTTP Session Management for Application Servers (Coherence*Web)
• GoldenGate HotCache
• TopLink-based CacheLoaders and CacheStores
• Elastic Data
• Federation and WAN (wide area network) Support
• Transaction Framework
• CommonJ Work Manager

Below is an overview of features supported in each Coherence edition for comparison purposes:

Please refer to Oracle Fusion Middleware Licensing Documentation for official documentation of Oracle Coherence commercial editions and licensing details.

Examples

Examples related to Coherence features are located under examples directory of this repository.

Contributing

This project welcomes contributions from the community. Before submitting a pull request, please review our contribution guide

Security

Please consult the security guide for our responsible security vulnerability disclosure process

License

Copyright (c) 2000, 2024 Oracle and/or its affiliates.

Released under the Universal Permissive License v1.0 as shown at https://oss.oracle.com/licenses/upl/.

coherence's Issues

Backups in Flash

in particular in cloud environment (with typically 3 AZs that one at a time may experience large scale events or failures) or other situations with few large racks that one need to handle as a failure mode without data loss the memory consumption for "backup count" is significant and it would be a good option to be able to store this data on a flash based (or for instance off-heap) structure (both options for instance provided by Chronicle Map so perhaps a usable component ro base an implementation on)

POF config not loaded

Version: CE 20.06.1
I created a POF config file "cache-pof-config.xml". And add below to tangosol-coherence-override.xml. When startup, it don't load cache-pof-config.xml. I also tried setting coherence.pof.config system property, didn't work.

<serializers> <serializer id="java"> <class-name>com.tangosol.io.DefaultSerializer</class-name> </serializer> <serializer id="pof"> <class-name>com.tangosol.io.pof.ConfigurablePofContext</class-name> <init-params> <init-param> <param-type>String</param-type> <param-value system-property="coherence.pof.config">cache-pof-config.xml</param-value> </init-param> </init-params> </serializer> </serializers>

In cache config file, it works by setting default serializer.
<defaults> <serializer> <instance> <class-name>com.tangosol.io.pof.ConfigurablePofContext</class-name> <init-params> <init-param> <param-type>String</param-type> <param-value>cache-pof-config.xml</param-value> </init-param> </init-params> </instance> </serializer> </defaults>

Multicast error message on MacOS

After a long period of using a Windows I just switched back to using Mac (this time ARM M3) and ran into a problem I have never experienced before starting Coherence locally :-(

This is using Coherence CE 24.03, Java Open JDK 21 and MacOS Sonoma 14.3.1

When I run "mvn exec:java -Dexec.mainClass="com.tangosol.net.DefaultCacheServer" I get an error message saying "multicast networking appears to be inoperable on interface 192.168.1.19 as this process isn't receiving even its own transmissions"...

Just for fun I tested using the coherence console app and to my surprise it can without any problem start a 3 node cluster, I could insert data etc. so there is nothing wrong with the MacOS setup it seems so the question is why I get this error when I try to do it myself using mvn rather than with the command line client?!

I also tried the suggested work-around of setting "set JAVA_TOOL_OPTS="-Djava.net.preferIPv4Stack=true" but it did not make any difference so the problem seem to be something else...

I also tried copying and running the exact same project on one of my Windows boxes and there it ran as it always have without any errors....

Probably some really silly Mac beginners misstake but right now I do not see it... suggestions appreciated!

Small typo in dev-guidelines and the example source code

There is a small error in the example and the referenced source code in Import Declarations section.

dev-guidelines says,

All import statements should be sorted alphabetically.

though the example shown and the respective source file has wrong ordering

import com.oracle.coherence.common.base.Exceptions;
import com.oracle.coherence.common.base.Hasher;
import com.oracle.coherence.common.base.HashHelper;
import com.oracle.coherence.common.base.Formatting;
import com.oracle.coherence.common.base.Objects;

Consider W-TinyLFU eviction policy

Enhancement Request

Is your feature request related to a problem? Please describe.
Currently, the cache supports the policies LRU, LFU, and a Hybrid (default). Unfortunately in many workloads these algorithms have low hit rates.

Describe the solution you'd like
Incorporate a modern algorithms, borrowing from Caffeine. Perhaps ideally there would be a direct dependency, e.g. to take advantage of it being concurrent (whereas LocalCache is fully synchronized). However that is a unreasonly significant change, so instead a port of the algorithm is provided into your APIs (commit) to assist in the initial bootstrapping.

The bootstrap commit leaves a few gaps such as

• Loop during eviction to discard multiple entries when required by custom unit sizes
• Unit / Integration testing
• Style, formatting, etc
• External configuration

Additional context
Window TinyLFU (W-TinyLFU) is a modern eviction policy that outperforms the currently implemented algorithms: LRU, LFU, and Hybrid (default).

W-TinyLFU utilizes a popularity sketch, CountMin, to compactly estimate the frequency over a time period. This sketch is used as a frequency filter by comparing the entry being added to the cache to eviction policy's victim. If the victim is more popular then the candidate is discarded, which avoids polluting the cache with "one-hit wonders". The sketch is aged by halving all of its counters periodically. Because it only needs to detect heavy hitters, a matrix of 4-bit counters is used.

In recency-biased workloads, aggressively filtering by the frequency can reduce the hit rate. Therefore the cache is split into two LRUs, one before and one after the filter, and uses TinyLFU to conditionally promote. Thus the admission window's victim and the main space's compete. Since the main region is frequency-biased, it uses SLRU to choose a better victim.

The optimal relative size of these two regions is workload dependent. It is configured dynamically by using hill climbing. The hit rate is sampled and the admission window's size is increased or decreased depending on whether the last guess was beneficial. Eventually the climber oscillates around the best configuration, so the step size decays gradually to allow for convergence. An adaption will restart if the hit rate changes dramatically, indicating different workload.

The hit rate across a variety of workloads shows a significant improvement.

Trace	Cache Size	W-TinyLFU	Hybrid	Lru	Optimal	Description
glimpse	512	32.65%	2.64%	0.98%	35.13%	an analytical loop
multi3	512	44.16%	31.41%	32.78%	49.10%	four concurrent workloads
corda	512	32.29%	12.14%	33.33%	33.33%	blockchain mining
loop/corda/loop	512	36.44%	14.95%	21.65%	39.38%	chaining of mru/lru/mru traces
oltp	1024	40.21%	15.39%	33.22%	53.86%	a CODASYL database
s3	400K	42.52%	31.32%	12.04%	59.96%	a web search engine
ds1	4M	44.76%	36.95%	20.24%	48.09%	a SQL database

Restrictive license text found in one of the versions of Coherence

Hi,

We have been trying to use the version 21.12.3 of Oracle Coherence in one of our open source projects. We downloaded the version from https://repo.maven.apache.org/maven2/com/oracle/coherence/ce/coherence/21.12.3/coherence-21.12.3-sources.jar. The sources contain the following files with a very restrictive license text:

• com/tangosol/dev/tools/copyright.cmd.txt
• com/tangosol/dev/tools/copyright.sh.txt
• com/tangosol/dev/tools/copyright.txt
• com/tangosol/dev/tools/copyright.xml.txt

The license text is the following:

@Rem Oracle is a registered trademarks of Oracle Corporation and/or its
@Rem affiliates.
@Rem
@Rem This software is the confidential and proprietary information of Oracle
@Rem Corporation. You shall not disclose such confidential and proprietary
@Rem information and shall use it only in accordance with the terms of the
@Rem license agreement you entered into with Oracle.
@Rem
@Rem This notice may not be removed or altered.

This looks strange to us and the license texts seem to be there for historical reasons, and do not apply to any source file.

Does anyone know why the files are there in that version of Coherence, and if it has any relevance or not.

Dynamic backups

As described already in discussion section it may for various reasons (work around to allow scale down while maintaining HA, mirror typical HA setup of cloud providers with 3 AZs, speed up get&getAll requests in 3 AZ cloud setups) be interesting to have backup count of 2 but in the event of an AZ failure it would be desirable to abandon the second backup to conserve memory.

Cache frequently used data from Informix db

Hi,

We are using weblogic 12.2.1.4 app server to host our java apps. Also using Informix 14 as database. Would it be possible to coherence community edition in this landscape to cache frequently queried data from db to improve performance?

Regards
Jacob

Does Coherence optimize request routing to stay within racks where possible?

I am looking for general advice related to how Coherence routes requests - if a "non-storage enabled" node requests an object from a partitioned cache in the same cluster will the request be routed to the most optimal of the primary or backups (for instance if one of them is in the same rack as the requesting node - determined using the rack id) or will it always go to the node that is the "primary" owner" of the key no matter if that will result in a higher latency?
If this optimization exists would it be possible to configure if it also should take asynchronously updated backups (if used) into consideration (i.e. if it is ok to route requests to a possible slighly stale copy in the same rack rather than to one of the synchronously updated ones in another rack)?
If only two racks are used (or if having say three racks and using one asynchronoulsy updated backup) it would be quite sweet if this optimization existed as one then always would be routed within the same rack (unless during rebalancing).
In fact this optimization could even make "stretch clusters" spaning two data centres possible (in low update frequency use-cases) without using cluster replication (that is not available in CE)...

Bad performance pruning large caches

I have previously reported (#95) problems with very poor performance pruning large caches due to a code line where the first element in an array list is removed in every iteration of the list resulting in a O(n ** 2) algorithm. I got the response the coherence CE team would look into the problem.

This problem is present in the class OldCache method pruneIncremental at the line "iterEvict.remove(); // COH-23644".

A better solution, in my mind, would be to just ignore removing the element as the whole structure is anyway released as garbage once the iteration ios completed. As an alternative one could iterate the list from the end as deleting the last element of an array list seem to be an optimized case (at least in later SDKs).

Due to the very slow performance of the algorithm as it is written now I doubt memory will actually be released faster with the current implementation (according to discussion on my previous report this seemed to be the motivation of introducing this line / COH-23644) compared to much faster getting through the whole structure and releasing it at the end...

I have not tested the new v14.1.1.0.13 release or verified if the iterated structure somewhere else in the code has been changed to no longer be an ArrayList (a LinkedList, is generally slower but would in this specific algorithm be better as removal of any elements is a O(1) operation) but a quick check of the source show this line to still be present and the problem probably not fixed :-(

To work around this problem we have ourselves removed the offending line and rebuilt Coherence CE for our use but would like to not having to perform this step for every release...

PagedTopic MBean attributes documentation

The content of the columns in the PagedTopic MBean attributes table at https://coherence.community/23.03/docs/#/docs/core/02_topics#mbeans appears to be incorrect in the rows after the PublishedMeanRate row.

To reproduce, look at the PagedTopic MBean attributes table at https://coherence.community/23.03/docs/#/docs/core/02_topics#mbeans.

I think what is expected is that the attribute name is in the first column, its type is in the second column, etc.

CaffeineCache doesn't track entry and cache size correctly when unit factor is used

Describe the bug

CaffeineCache calculates the "weight" of individual entries at insertion time, using weigh function:

    private int weigh(Object oKey, Object oValue)
        {
        int cUnits = m_unitCalculator.calculateUnits(oKey, oValue);
        if (cUnits < 0)
            {
            throw new IllegalStateException(String.format(
                    "Negative unit (%s) for %s=%s", cUnits, oKey, oValue));
            }
        return (cUnits / m_nUnitFactor);
        }

This is problematic, because the method above returns 0 if a unit factor is larger than the number of units for a give entry, which is often the case when unit factor is used (typical values for unit factor are 2^10 and 2^20, in order to track and limit cache size in KBs or MBs). Even if the factor is smaller than the entry size, the loss of precision due to integer arithmetic at the single entry level is significant.

To Reproduce

1. Configure unit factor that is larger than the average entry size
2. Configure BINARY unit calculator
3. Add some entries to the cache
4. Observe that the getUnits method returns zero, even though there are entries in the cache

Expected behavior
getUnits methods should return the actual number of bytes consumed by all entries, divided by the unit factor.

In other words, unit factor should only be applied during final calculation for the backing map as a whole, not at the individual entry level.

Environment (please complete the following information):

• Coherence CE 22.06.1
• Java 11
• OS: any
• OS Version: n/a

Is there a function to obtain hotkeys implemented

Is there a function to obtain hotkeys implemented

Allow auto scaling without making cluster vulnerable

Today there is (according to answer to a previous question) no way for a storage enabled cluster member to signal that it intends to leave the cluster and stigger re-balancing of its partitions in an safe and ordered way.

As a consequence it is a risky operation to "scale in" a coherence cluster (with the standard backup count of one we are vulnerable to data loss if a storage enabled node would fail while a node has terminated due to scale in). Having to use backup count of two is a possible work-around but results in a significant increase in memory use and reduced performance of updates...

A way to address this would be to provide a way to programatically indicate that a storage enabled node would like to leave and that this would initiate an orderly re-balancing of the partitions WHILE THE NODE IS STILL AVAILABLE to avoid data loss in the case of another node failing during the re-balancing.

Once this re-balancing is completed the node would be able to terminate and the scale-in operation completing.

Having this kind of functionality would be a big step taking coherence into the "cloud era" allowing cluters to not only scale out but also scale in to track needed capacity.

Question - does Coherence now work on JDK17?

I read somewhere that Coherence only worked with JDK8 - does this still apply or can it now work with more recent ones?

Is "-Dtangosol.coherence.override" not used in Coherence CE? Still talking about the old v14 sorry for that.... :-(

According to the documentation shared between Oracle Coherence and Coherence CE one should be able to specify an "override file" with -Dtangosol.coherence.override but as far as we can tell (we just noticed it and is still investigating/finding work arounds), at least in V14 that we still use, this parameter is ignored by CE and only the file "jar:file:/opt/oas/cache/lib/cache-coherence-2.7941.0.jar!/tangosol-coherence-override-prod.xml" seem to be read?!
Not sure if this is the intended behaviour in CE and if so is it documented somewhere?

COH-23644?

I am working in a project where we due to limitations of using JDK 8 and other components used are "stuck" with Coherence CE 14 and is having a challenge where evicting entries in the local cache (part of a near cache) takes a LONG time.

I realize this is not a supported release so we are looking into solving the problem ourselves but have a specific question that could help us in this...

We are profiling and seeing that the line that seem to take a lot of time is " iterEvict.remove()" in OldCache and is marked with an issue "COH-23644" that we assume introduced the line.

Not sure if this is a reference to an issue in Coherence CE or an internal Oracle issue to the enterprise edition but either way we have not been able to find any info about it and are very curious to what the issue was and why the line was introduced...

Naively when looking at the code it seems to remove keys from an ArrayList of elements to be evicted as it is iterated - it is not clear to us WHY there is a need to do this and in general removing elements from ArrayLists is expensive (at least in the past required copying the array) so we are suspecting this could be at least one reason we see slow eviction (our caches are big and so could the list of keys to evict be)....

Our first work-around that we are testing now is to increase the low-units to be about 99% of the high-units to ensure the size of the array list will be small (then the ~O(n ** 2) effect of the repeated array.remove operation is minimized) - lets see if it works...

We are also considering, as a further work around, try using Caffeine (2.X version ) as a local cache with this Coherence CE release (using the old lower level mechanism that has been available to provide your owen cache implementation) but not sure how well it would work - are experiences of this would be welcome!

I have noticed that the above mentioned line is present also in the latest Coherence CE release so if this is the cause of the problem we are seeing it may be present also with the latest...

Provide a Graddle alternative for Coherence POF Maven plugin

Followed description of using POF annotations to leverage generation of Portable Object code for application in ce22.06 guide for key association.

Excerpt from Key Associator guide:

Bothof the model classes are annotated with the @PortableType. This annotation is used by the Coherence POF Maven plugin to generate Portable Object code for the classes. Using the Coherence POF generator in this way avoids having to manually write serialization code and ensures that the serialization code generated is supports evolvability betweenversions.

I am using Graddle and would like to have an equivalent for the POF Maven plugin to generate Portable Object code for the classes.

For now, I am just falling back to specifying user pof-config file and specifying PofSerializers.

initiating connection migration with tmb?

We are getting quite frequent warnings in our Coherence log on storage enabled nodes mentioning "initiating connection migration with tmb" and I would like some info about what it means on a more technical level and perhaps suggestions on most common causes?

I am guessing some problem on the physical network as well as long GC pauses could result in more or less any network related warning in Coherence but are there also other possible reasons and are there any tips on how to further debug the problem?

2023-03-13 20:17:56.310/95085.948 Oracle Coherence CE 14.1.1.0.12 (thread=SelectionService(channels=112, selector=MultiplexedSelector(sun.nio.ch.EPollSelectorImpl@3676ac27), id=1660451908), member=17): tmb://138.106.96.41:9001.49982 initiating connection migration with tmb://138.106.96.25:33316.40573 after 15s ack timeout health(read=false, write=true), receiptWait=Message "PartialValueResponse"
{
FromMember=Member(Id=17, Timestamp=2023-03-12 17:53:14.974, Address=138.106.96.41:9001, MachineId=46694, Location=site:sss.se.xxxxyyyy.com,machine:l4041p.sss.se.xxxxyyyy.com,process:391,member:l4041p-2, Role=storagenode)
FromMessageId=0
MessagePartCount=0
PendingCount=0
BufferCounter=1
MessageType=70
ToPollId=19827300
Poll=null
Packets
{
}
Service=PartitionedCache{Name=DistributedCache, State=(SERVICE_STARTED), Id=3, OldestMemberId=1, LocalStorage=enabled, PartitionCount=601, BackupCount=1, AssignedPartitions=18, BackupPartitions=19, CoordinatorId=1}
ToMemberSet=MemberSet(Size=1
Member(Id=179, Timestamp=2023-03-12 18:43:36.847, Address=138.106.96.25:33316, MachineId=48549, Location=site:sss.se.xxxxyyyy.com,machine:l4025p,process:3473,member:l4025p_11990, Role=scex)
)
NotifyDelivery=false
}: peer=tmb://138.106.96.25:33316.40573, state=ACTIVE, socket=MultiplexedSocket{Socket[addr=/138.106.96.25,port=38114,localport=9001]}, migrations=17, bytes(in=104371345, out=101784244), flushlock false, bufferedOut=0B, unflushed=0B, delivered(in=203177, out=197772), timeout(ack=0ns), interestOps=1, unflushed receipt=0, receiptReturn 0, isReceiptFlushRequired false, bufferedIn(), msgs(in=95922, out=99203/99206)
2023-03-13 20:17:56.310/95085.948 Oracle Coherence CE 14.1.1.0.12 (thread=SelectionService(channels=112, selector=MultiplexedSelector(sun.nio.ch.EPollSelectorImpl@3676ac27), id=1660451908), member=17): tmb://138.106.96.41:9001.49982 initiating connection migration with tmb://138.106.96.32:41070.40752 after 15s ack timeout health(read=true, write=false), receiptWait=null: peer=tmb://138.106.96.32:41070.40752, state=ACTIVE, socket=MultiplexedSocket{Socket[addr=/138.106.96.32,port=41070,localport=36388]}, migrations=5, bytes(in=95752773, out=99458811), flushlock false, bufferedOut=1.54KB, unflushed=0B, delivered(in=192506, out=187239), timeout(ack=0ns), interestOps=1, unflushed receipt=0, receiptReturn 0, isReceiptFlushRequired false, bufferedIn(), msgs(in=90667, out=93950/93953)

Need some advice about Coherent indexes

A need some general advice on how Indexes work from a consistency and backup point of view? Lets say I update some data spread over multiple partitions using an entry processor - when are the indexes updated with those changes?
Also are Coherence indexes backed up over several nodes (how many, asynch or synch...)?

TDE and CDB files

Type of question

I would like to know more about the background of the TDE toolchain and the CDB files which seem to be used as basis for generating large parts of the Coherence code.

Question

What did you do?

I wanted to do some experiments by tweaking the method public QueryResult query(Filter filter, int nQueryType, PartitionSet partMask) in com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache$Storage but struggled to find the Java source code for this.

It seems that the code is generated out of tde/core-net/3.0/cdb/Component/Util/Daemon/QueueProcessor/Service/Grid/PartitionedService/PartitionedCache.cdb (and a lot of other code too analogously)?

I found the tde utility (Tangsol Desktop) under tools/bin/tde which allows to open those cdb files and look at them/modify them. With a bit of try and error I managed to modify the method I was interested in and by re-running the toolchain get the result I wanted to in the first place, but to be honest, half of the time I did not understand what I was doing.

So my questions are:

1. What are those cdb files and the tde tool and what is the general idea behind it? Is this something tangosol-properitary or part of/based on some other tooling?
2. Is there some documentation on how it all works in more details (e. g. what do all the options mean in tde, how as a developer you are supposed to use the tool, where would you do changes to code if you want to introduce new functionality, etc.)?
3. Are there plans to move away from this toolchain to something, let's say, more intuitive to the regular developer or is the current situation that tde/cdb is just too tightly integrated into the Coherence DNA to make it worth the effort?

Thanks, regards, Leo

What did you expect to see?
I would like to see documentation about TDE and the CDB format.

What did you see instead? Under which circumstances?
I couldn't find any documentation.

Environment
N/a

Additional context
N/a

Coherence CE transaction support?

I need some general guidance on what support is available in Coherence CE for transactional update of multiple objects at the same time (in addition to what can be done with entry processors).

At one hand the documentation states that "Transaction Framework" is not included in CE on the other hand I can see that the CacheAdapter class is present in the source code?

Is the intention that you need to write your own equivalent of the old coherence-tx.rar (that is not provided with CE) or what is the intended use of CacheAdpater?

Does the community edition allow adding your own map implementation?

Type of question

Question about availability of a feature

Question

What did you do?
A clear and concise description of the steps you took (or insert a code snippet).

I would like to be able to write my own cache map implementation with Coherent community edition - is this possible?

OldCache violates the Map contract

When testing for #16 (comment), I used Guava's testlib to validate conformance to the Collections Framework contracts. Running these tests against the existing OldCache uncovered a bug in the iterator.

CollectionIteratorTester.testIterator_unknownOrderRemoveSupported

junit.framework.AssertionFailedError: failed with stimuli [hasNext, hasNext, next, hasNext, remove]

	at com.google.common.collect.testing.Helpers.fail(Helpers.java:248)
	at com.google.common.collect.testing.AbstractIteratorTester.compareResultsForThisListOfStimuli(AbstractIteratorTester.java:370)
	at com.google.common.collect.testing.AbstractIteratorTester.recurse(AbstractIteratorTester.java:344)
	at com.google.common.collect.testing.AbstractIteratorTester.recurse(AbstractIteratorTester.java:349)
	at com.google.common.collect.testing.AbstractIteratorTester.recurse(AbstractIteratorTester.java:349)
	at com.google.common.collect.testing.AbstractIteratorTester.recurse(AbstractIteratorTester.java:349)
	at com.google.common.collect.testing.AbstractIteratorTester.recurse(AbstractIteratorTester.java:349)
	at com.google.common.collect.testing.AbstractIteratorTester.recurse(AbstractIteratorTester.java:349)
	at com.google.common.collect.testing.AbstractIteratorTester.test(AbstractIteratorTester.java:317)
	at com.google.common.collect.testing.testers.CollectionIteratorTester.runIteratorTest(CollectionIteratorTester.java:133)
	at com.google.common.collect.testing.testers.CollectionIteratorTester.testIterator_unknownOrderRemoveSupported(CollectionIteratorTester.java:111)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at junit.framework.TestCase.runTest(TestCase.java:177)
	at junit.framework.TestCase.runBare(TestCase.java:142)
	at junit.framework.TestResult$1.protect(TestResult.java:122)
	at junit.framework.TestResult.runProtected(TestResult.java:142)
	at junit.framework.TestResult.run(TestResult.java:125)
	at junit.framework.TestCase.run(TestCase.java:130)
	at junit.framework.TestSuite.runTest(TestSuite.java:241)
	at junit.framework.TestSuite.run(TestSuite.java:236)
	at junit.framework.TestSuite.runTest(TestSuite.java:241)
	at junit.framework.TestSuite.run(TestSuite.java:236)
	at junit.framework.TestSuite.runTest(TestSuite.java:241)
	at junit.framework.TestSuite.run(TestSuite.java:236)
	at junit.framework.TestSuite.runTest(TestSuite.java:241)
	at junit.framework.TestSuite.run(TestSuite.java:236)
	at junit.framework.TestSuite.runTest(TestSuite.java:241)
	at junit.framework.TestSuite.run(TestSuite.java:236)
	at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:90)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
	at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:235)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)
Caused by: junit.framework.AssertionFailedError: Target threw exception when reference did not
	at com.google.common.collect.testing.Helpers.fail(Helpers.java:248)
	at com.google.common.collect.testing.AbstractIteratorTester.internalExecuteAndCompare(AbstractIteratorTester.java:442)
	at com.google.common.collect.testing.AbstractIteratorTester.access$600(AbstractIteratorTester.java:44)
	at com.google.common.collect.testing.AbstractIteratorTester$8.executeAndCompare(AbstractIteratorTester.java:553)
	at com.google.common.collect.testing.AbstractIteratorTester.compareResultsForThisListOfStimuli(AbstractIteratorTester.java:367)
	... 37 more
Caused by: java.lang.IllegalStateException
	at com.tangosol.util.FilterEnumerator.remove(FilterEnumerator.java:157)
	at com.google.common.collect.testing.AbstractIteratorTester$1.execute(AbstractIteratorTester.java:469)
	at com.google.common.collect.testing.AbstractIteratorTester.internalExecuteAndCompare(AbstractIteratorTester.java:398)
	... 40 more

There were also failures due to the custom string representation. Unfortunately there is not a setting to ignore that, as it does seem reasonable for your use-case when debugging. Source code below.

OldCacheMapTests

import com.google.common.collect.testing.MapTestSuiteBuilder;
import com.google.common.collect.testing.TestStringMapGenerator;
import com.google.common.collect.testing.features.CollectionFeature;
import com.google.common.collect.testing.features.CollectionSize;
import com.google.common.collect.testing.features.MapFeature;
import com.tangosol.net.cache.OldCache;
import java.util.Map;
import java.util.Map.Entry;
import junit.framework.TestCase;
import junit.framework.TestSuite;

/** Guava testlib map tests. */
public final class OldCacheMapTests extends TestCase {

  public static TestSuite suite() {
    return MapTestSuiteBuilder
        .using(generator())
        .named("OldCache")
        .withFeatures(
            CollectionSize.ANY,
            MapFeature.GENERAL_PURPOSE,
            MapFeature.ALLOWS_NULL_KEYS,
            MapFeature.ALLOWS_NULL_VALUES,
            CollectionFeature.SUPPORTS_ITERATOR_REMOVE)
        .createTestSuite();
  }

  private static TestStringMapGenerator generator() {
    return new TestStringMapGenerator() {
      @Override protected Map<String, String> create(Entry<String, String>[] entries) {
        var cache = new OldCache();
        for (var entry : entries) {
          cache.put(entry.getKey(), entry.getValue());
        }
        return cache;
      }
    };
  }
}

How to safely scale in storage enabled nodes in cloud environment?

I am seeking general advice on how safely "scale in" storage enabled nodes in a cloud environment (in my case AWS)? I would like to use an auto scaling group (ASG) to maintain a (typically slowly varying) number of nodes to accomodate varying need of vCPUs over the day/week to preocess requests to a partitioned cache.
One way would be to only allow the ASG to only scale in ONE node at a time and set a long delay before it is allowed to scale in one more. I would however prefer to instead of a fixed delay use the "lifecycle hook" support of the ASG where I can let a scaled in instance communicate to the ASG when it is ok to kill it (after it has performed an orgerly shutdown).
My question is if there is a simple way for a storage enabled node to tell Coherence that it intends to leave the cluster that will also work if rebalancing is already in progress (for instance due to a node failure) and if so would this mechanism work even if allowing the ASG to scale in more than one node at the same time i.e. would the leaving nodes be allowed to do so in sequence?

ConfigPropertyResolver.java does not permit config customization by the application in Helidon

Describe the bug
My application modifies the config sources for Helidon. I found that the Coherence ConfigPropertyResolver creates its own config sources and caches it locally in the class, disregarding the config sources that are later setup by my application startup code. The problem is that Coherence cannot find the system properties that I have configured, such as the cluster name and role.

To Reproduce
final var config = ... some custom config with coherence.cluster specified ...
Server.Builder serverBuilder = Server.builder().config(config);

Observe the logs, some default cluster name is used instead of what is specified in coherence.cluster

Expected behaviour
I should be able to define the coherence config using my own config sources and have it get used.

Environment (please complete the following information):

• Coherence CE version 20.06

Does Coherence work on ARM architecture?

Type of question

Coherence CPU architecture support

Question

Does Coherence work on ARM architecture (like the AWS Graviton2) or only with X86?

Relationship between future releases of Coherence Enterprise/Grid and Coherence Community

First and foremost a big thank you to Oracle for open-sourcing this great product.

I have a couple of questions for the Oraclers here around:

1 - What will be the future relationship between Community and Coherence Enterprise/Grid in terms of source code?
Will CE be the base for future releases of the commercial product as well?

2 - Version numbering: will at some point Coherence Grid and Enterprise adopt the same version numbering of CE?

TY,

R

Support to change the percentage used for calculating default low-units

If you don't specify a value for "low-units" (or specify 0) the value used is today calculated as 0.75 * high-units.

It would be very nice to be able to set this percentage to another value than 0.75 and in this way not having to specify an explicit calculated value for "low-units" for every cache as soon as you would like another percentage.

Perhaps there is already a parameter or method for overriding this but in that case we have failed to find it....