For some reason, on Java 8 in Ubuntu 18.04 running in WSL, the DS 3.x tests are failing for testGetFreePort
. It looks like the JRE or Windows are allowing Java to bind two different server sockets to the same port (port sharing), which violates the test's setup expectations.
The build should succeed with no test failures.
Full failure log
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running TestSuite
Configuring TestNG with: TestNG652Configurator
Test environment:
Java version: 1.8.0_191
Java vendor: Oracle Corporation
JVM name: OpenJDK 64-Bit Server VM
JVM version: 25.191-b12
JVM vendor: Oracle Corporation
JVM info: mixed mode
Java home: /usr/lib/jvm/java-8-openjdk-amd64/jre
OS: Linux 4.4.0-17763-Microsoft amd64
Processors: 4
Max memory: 2837446656
Total memory: 192937984
How to read the progressive status info:
Test duration status: {Total min:sec. Since last status sec.}
Test count status: {# test classes # test methods # test method invocations # test failures}.
TestClass (the class that just completed)
{ 0:00 ( 0s)} { 0c 0m 0i 0f} : starting
{ 0:00 ( 0s)} { 1c 6m 6i 0f} : CertificateTestCase
{ 0:00 ( 0s)} { 2c 9m 9i 0f} : ProductInformationTest
{ 0:00 ( 0s)} { 3c 13m 21i 0f} : SetupCliTestCase
{ 0:00 ( 0s)} { 4c 16m 24i 0f} : DataConfigurationTestCase
T E S T F A I L U R E ! ! !
Failed Test: org.forgerock.opendj.server.setup.model.ListenerSettingsTestCase#testGetFreePort
Failure Cause: java.lang.AssertionError: actual value:<61728> should not be equal to:<61728>
org.fest.assertions.Fail.failure(Fail.java:228)
org.fest.assertions.Fail.fail(Fail.java:218)
org.fest.assertions.Fail.failIfEqual(Fail.java:53)
org.fest.assertions.GenericAssert.isNotEqualTo(GenericAssert.java:228)
org.fest.assertions.IntAssert.isNotEqualTo(IntAssert.java:71)
org.forgerock.opendj.server.setup.model.ListenerSettingsTestCase.testGetFreePort(ListenerSettingsTestCase.java:80)
{ 0:00 ( 0s)} { 5c 20m 28i 1f} : ListenerSettingsTestCase
{ 0:00 ( 0s)} { 6c 38m 46i 1f} : ModelTestCase
{ 0:00 ( 0s)} { 7c 43m 51i 1f} : RuntimeOptionsTestCase
The following unit tests failed:
org.forgerock.opendj.server.setup.model.ListenerSettingsTestCase#testGetFreePort
Include the ant option '-Dtest.failures=true' to rerun only the failed tests.
Wrote full test report to:
/mnt/c/Users/guypa/Documents/WrenProjects/wrends/opendj-server/target/surefire-reports/results.txt
Test classes run interleaved: 0
Final amount of memory in use: 3.0 MB
Final number of threads: 1
Tests run: 51, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.897 sec <<< FAILURE! - in TestSuite
testGetFreePort(org.forgerock.opendj.server.setup.model.ListenerSettingsTestCase) Time elapsed: 0.06 sec <<< FAILURE!
java.lang.AssertionError: actual value:<61728> should not be equal to:<61728>
at org.fest.assertions.Fail.failure(Fail.java:228)
at org.fest.assertions.Fail.fail(Fail.java:218)
at org.fest.assertions.Fail.failIfEqual(Fail.java:53)
at org.fest.assertions.GenericAssert.isNotEqualTo(GenericAssert.java:228)
at org.fest.assertions.IntAssert.isNotEqualTo(IntAssert.java:71)
at org.forgerock.opendj.server.setup.model.ListenerSettingsTestCase.testGetFreePort(ListenerSettingsTestCase.java:80)
Results :
Failed tests:
ListenerSettingsTestCase.testGetFreePort:80 actual value:<61728> should not be equal to:<61728>
Tests run: 51, Failures: 1, Errors: 0, Skipped: 0
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 6.884 s
[INFO] Finished at: 2019-04-18T14:51:48Z
[INFO] Final Memory: 27M/252M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (default-test) on project opendj-server: There are test failures.
[ERROR]
[ERROR] Please refer to /mnt/c/Users/guypa/Documents/WrenProjects/wrends/opendj-server/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
Environment
- Product version (include GIT commit ID if using a dev build): 3.0.0 @ 3a60e66
- Operating system:
Ubuntu 18.04.1 LTS
Linux 4.4.0-17763-Microsoft #379-Microsoft Wed Mar 06 19:16:00 PST 2019 x86_64 x86_64 x86_64 GNU/Linux
- JRE version:
openjdk version "1.8.0_191"
OpenJDK Runtime Environment (build 1.8.0_191-8u191-b12-2ubuntu0.18.04.1-b12)
OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)
- Was instance upgraded from a previous version? If so, what version were you using before? No
Additional Notes
Looking into this issue revealed a separate issue with the way that the DS 3.x SDK looks for a free port. A separate issue for that is pending.
Wren:DS has a plugable storage backend. Currently it primarily uses Berkeley DB Java edition as storage backend. At some point it also supported PDE however that didn't workout well so support was marked deprecated. PDE support was added due to ForgeRock's concerns about the licensing of Berkeley DB. However soon after PDE support was released the licensing of Berkeley DB was changed to a license which no longer posed a problem for ForgeRock.
While Berkeley DB was long the go to embedded storage backend new challengers have emerged. One of these is RocksDB. RocksDB was / is developed by Facebook and is based on research by Google (LevelDB).
RocksDB is supposedly, very, very fast. I think it would be interesting to do some research to see if A) RocksDB could be used as a backend for Wren:DS and B) If so, how much performance gain would that give us.
Summary
HTTP endpoints do not return full JSON response.
$ curl -u admin:password http://localhost:8080/admin
{"_rev":"0000000054c141ca"
$ curl -u admin:password http://localhost:8080/api
{"_rev":"0000000054c141ca"
Steps To Reproduce ("Repro Steps")
- Enable HTTP Connector Handler
- Send GET requests to HTTP endpoints
Expected Result (Behavior You Expected to See)
Full readable JSON response.
Actual Result (Behavior You Saw)
Incomplete JSON response.
Environment
- Product version: 4.0.0-RC2
- JRE version: openjdk 17.0.6
From gitter chat:
The only difference between sustaining/3.5.x and the current master are the following commits:
ad669ec93a Upgrade Wren:DS to `wrensec-parent` (2.2.0)
7765a3cb9a Remove Unused Wren Deploy Values
7dd5225a57 Fix Acct. Change Notification Package Name
b210895b0e Fix file names to conform to class names in them.
47b06874c8 Fix path to Wren:DS JAR file for Doc Generation
bef165bab8 Add note about OPENDJ-3377 as OpenIG Workaround
8bef7b5dcb OPENDJ-3377 Switch OpenDJ dependency from openig-toolkit to chf-oauth2
55da63bff3 Update 3.5.1 for WrenSec repos & info
91c3b0ba77 Add Official OpenDJ 3.5.1 Sources from Backstage
I think we can go through those and apply them if necessary.
The following will be evaluated (the main idea behind this is that if there is a critical patch, it has unit tests as well):
git show 91c3b0ba77 -I'^ *([/\*]|@Override)' -- **/src/test/** > changes.diff
When building Wren:DS in a network without public DNS available, maven fails while trying to verify the artifact signatures:
[DEBUG] (f) project = MavenProject: org.forgerock.opendj:opendj-server-parent:3.0.0 @ /home/olbohlen/git/wren/wrends/pom.xml
[DEBUG] (f) scope = test
[DEBUG] (f) session = org.apache.maven.execution.MavenSession@704b2127
[DEBUG] (f) verifyPomFiles = true
[DEBUG] -- end configuration --
[DEBUG] The resource 'http://wrensecurity.org/trustedkeys.properties' was not found with resourceLoader org.codehaus.plexus.resource.loader.JarResourceLoader.
[DEBUG] The resource 'http://wrensecurity.org/trustedkeys.properties' was not found with resourceLoader org.codehaus.plexus.resource.loader.FileResourceLoader.
[DEBUG] The resource 'http://wrensecurity.org/trustedkeys.properties' was not found with resourceLoader org.codehaus.plexus.resource.loader.ThreadContextClasspathResourceLoader.
[DEBUG] URLResourceLoader: Exception when looking for 'http://wrensecurity.org/trustedkeys.properties
java.net.UnknownHostException: wrensecurity.org
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at java.net.Socket.connect(Socket.java:538)
at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
as a work-around, mvn -Dignore-artifact-sigs clean install
works.
This one comes from an internal feature request we're working on for Rest2LDAP.
Summary
As a developer querying a deeply nested LDAP directory of users, I would like it if nested entries could be retrieved and filtered through the Rest2LDAP interface like flat collections so that I don't have to pull the full list of entries just to get information about a single entry.
Component Affected
Rest2LDAP (OpenDJ 3.5+)
Conditions of Acceptance
The Rest2LDAP module within OpenDJ is modified as follows:
- A new option is added to the Rest2LDAP configuration file to allow collection sub-resources to support sub-tree flattening, such that nested LDAP entries are included as though they were all at the same flat level in the directory.
- A new option is added to the Rest2LDAP configuration file to allow collection sub-resources to support an optional "base" search filter that can be used to restrict what entries get returned through the sub-resource. This search filter supplements any filters provided in the request.
- Validation is added to ensure that the new sub-tree flattening option is only being used if the sub-resource is marked as read-only in the config (without this, there is no way to unambiguously map attempts to create new entries to a specific DN). Any attempt to configure a writable sub-resource with sub-tree flattening turned on must result in an error.
- Sub-resources must be modified to behave as follows:
- When the sub-resource has the new subtree flattening option toggled on: it should return all LDAP entries and sub-entries under the base DN that match both search filters provided on the query string as well as the search filter specified in the configuration for the sub-resource (i.e. it expands the search scope to
SearchScope.SUBORDINATES
).
- When the sub-resource has the new subtree flattening option toggled off or has not specified whether the option is toggled on: it should return only entries that exist directly under the base DN, excluding sub-entries (i.e. the search scope is
SearchScope.SINGLE_LEVEL
), and then apply both search filters provided on the query string as well as the search filter specified in the configuration for the sub-resource (if any).
Workarounds / Other Approaches
- Set-up a singleton resource that's mapped to a virtual static group that queries the whole sub-tree and selects the desired users from everywhere under the desired base DN. By rendering the members of the virtual static group, you can access all of the users everywhere in the subtree instead of just at the base level. Major drawback: Huge performance hit, since trying to access a single record requires pulling down the whole list of users from the server.
- Don't use subtrees -- put all users under a single flat portion of the directory (not ideal for most use cases).
- Don't use Rest2LDAP -- query LDAP directly from the application (not ideal because it requires all consumers to understand LDAP).
Summary
The GeneralizedTime
component inconsistently formats / parses the date when the year of the date is greater than 9999. During serialization, the year is serialized as is (e.g. 292278994
), but the parsing process expects the year to be in YYYY
format.
Steps To Reproduce ("Repro Steps")
@Test
public void testGeneralizedTimeMaxYear() {
String serialized = GeneralizedTime.valueOf(Long.MAX_VALUE).toString();
GeneralizedTime parsed = GeneralizedTime.valueOf(serialized);
}
The preceding test ends with the following exception:
org.forgerock.i18n.LocalizedIllegalArgumentException: The provided value "2922789940817071255.807Z" is not a valid generalized time value because "78" is not a valid month specification
at org.forgerock.opendj.ldap.GeneralizedTime.valueOf(GeneralizedTime.java:221)
Expected Result (Behavior You Expected to See)
The behaviour of the GeneralizedTime
component will be consistent if the year is greater than 9999.
Summary
After upgrading Wren:DS to use wrensec-parent-3.0.0-SNAPSHOT
, the build fails to complete successfully when building opendj-server-legacy
.
Affects Version
4.0.0-SNAPSHOT (at 513505e
)
Environment
- Ubuntu 16.04.5 LTS
- Apache Maven 3.3.9
- Java 1.8.0_181-b13
Steps
Within the wrends
folder checked out to master
, run the following command:
mvn clean install -DskipTests -Denforcer.skip=true -DignoreArtifactSigs -DskipTests
Expected
The build succeeds without error.
Actual
The build fails during the maven-jar-plugin
execution of the opendj-server-legacy
module:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-jar-plugin:3.1.0:jar (build-bootstrap-client-jar) on project opendj-server-legacy: You have to use a classifier to attach supplemental artifacts to the project instead of replacing them. -> [Help 1]
Root Cause
The project builds many JARs with different names, but without any classifiers. In the 3.x version of the maven-jar-plugin
, classifiers are now mandatory. But, adding them will cause the JARs to be renamed.
Summary
When the control panel's Manage Entries is first opened it always shows Unsaved Changes warning when a focus is lost on a selected entry. This warning stops showing after clicking on a non-OU entry.
Steps To Reproduce ("Repro Steps")
- Open Control Panel
- Click on any OU
Expected Result (Behavior You Expected to See)
Do not show "Unsaved Changes" warning when nothing has been touched.
Actual Result (Behavior You Saw)
"Unsaved Changes" warning is always displayed when any entry is selected.
Screenshots
![image](https://private-user-images.githubusercontent.com/2012726/355347083-ace3c338-8614-4931-9dbb-22951594d593.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQyNjEyMjEsIm5iZiI6MTcyNDI2MDkyMSwicGF0aCI6Ii8yMDEyNzI2LzM1NTM0NzA4My1hY2UzYzMzOC04NjE0LTQ5MzEtOWRiYi0yMjk1MTU5NGQ1OTMucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDgyMSUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA4MjFUMTcyMjAxWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9YWI4MDUwNmNjMjQ3YTBlMzVjZjc0YzQ3MmRjYTY1YTBmZjA4ZGRmOTY3MTUxZGEwM2EwNjU0ODc3ODA4M2VhZSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.Vw06ILnhBj3JVtsgGdP5NTuI8XQSwiTGxh580vXAolQ)
Environment
- Wren:DS 5.0.0
- Kubuntu 24.04 LTS
- Java 17.0.11
Summary
Connection handlers have too long and descriptive name with Connection Handler
suffix. Connection handler names are used when referencing the handler so this makes all commands unnecessarily more complex.
Connection Handler : Type : enabled : listen-port : use-ssl
-------------------------:------:---------:-------------:--------
HTTP Connection Handler : http : false : 8080 : false
JMX Connection Handler : jmx : false : 1689 : false
LDAP Connection Handler : ldap : true : 1389 : false
LDAPS Connection Handler : ldap : true : 1636 : true
LDIF Connection Handler : ldif : false : - : -
Solution You'd Like to See
I would like to remove Connection Handler
suffix and leave only the protocol name as the handler name.
Workarounds/Alternatives
None.
Additional Notes
None.
Summary
Wren:DS SDK defines a PORT_INCREMENT
value of 1000
, which means that if a port it wants to use is in-use, it skips 1,000 ports at a time until it can find the desired port. This can lead to failures during test runs if the port number the OS hands to the test is close to the maximum port number (e.g. max is 65535
and starting port number is 64536
).
At run-time, presumably this issue is less severe because Wren:DS has no preference for starting port number, so it has a larger pool of addresses to search 1,000 at a time. This defect only appears when the starting port # is within a few thousand port numbers of the RFC-defined max port #.
Steps To Reproduce ("Repro Steps")
NOTE: These steps do not reliably reproduce the issue because the issue depends on what ports are available on the host machine.
- Checkout Wren:DS @ 9e5a6fa.
- Build the project with
mvn clean install
.
Expected Result (Behavior You Expected to See)
The build should succeed with no test failures.
Actual Result (Behavior You Saw)
Sporadically, tests may fail with an IllegalArgumentException
and message of "Invalid port."
.
Environment
- Product version (include GIT commit ID if using a dev build): 3.0.0 @ 9e5a6fa
- Operating system:
Ubuntu 18.04.1 LTS
Linux 4.4.0-17763-Microsoft #379-Microsoft Wed Mar 06 19:16:00 PST 2019 x86_64 x86_64 x86_64 GNU/Linux
- JRE version:
openjdk version "1.8.0_191"
OpenJDK Runtime Environment (build 1.8.0_191-8u191-b12-2ubuntu0.18.04.1-b12)
OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)
- Was instance upgraded from a previous version? If so, what version were you using before?
No.
Getting the following exception when building the codebase:
Caused by: java.lang.IllegalArgumentException: TransformerFactory does not recognise attribute 'jdk.xml.xpathExprGrpLimit'.
at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl.setAttribute (TransformerFactoryImpl.java:541)
at org.forgerock.opendj.maven.GenerateConfigMojo.initializeStylesheets (GenerateConfigMojo.java:400)
at org.forgerock.opendj.maven.GenerateConfigMojo.execute (GenerateConfigMojo.java:193)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:77)
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:568)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
$ mvn -version
Apache Maven 3.8.1 (05c21c65bdfed0f71a2f2ada8b84da59348c4c5d)
Maven home: N:\apps\apache-maven-3.8.1
Java version: 17.0.1, vendor: Eclipse Adoptium, runtime: C:\Program Files\Eclipse Adoptium\jdk-17.0.1.12-hotspot
Default locale: en_US, platform encoding: Cp1250
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
This is connected with #41. Nut sure why and how that setting does not work. Might need someone else to confirm this is a real issue and not just some inconsistency in my Windows environment.
Running recommended command docker run --rm --name wrends-test -p 1389:1389 -p 1636:1636 wrensecurity/wrends:latest
causes docker to crash.
opendj-setup-5163692219923348077.log
On the other hand, the docker image runs smoothly when i build it first and then run it.
docker build . -t wrends-arm:latest
docker run --rm --name wrends-test -p 1389:1389 -p 1636:1636 wrends-arm:latest
Summary
I would like Wren:DS distributions not to contain any reference to ForgeRock's commercial license.
Solution You'd Like to See
- The TXT files that contain the terms of the FR commercial license are no longer included in JAR packages for Wren:DS.
- The FR commercial license is no longer displayed on the first screen of the interactive installer. In its place, we should be displaying the CDDL instead.
Workarounds/Alternatives
Users currently have to ignore the commercial license during install, since it doesn't apply to them. This causes confusion.
It's not possible to build all of OpenDJ successfully in one pass. the opendj-dsml-servlet
project will fail because it can't find packages, but re-building just that package succeeds.
something is causing the classpath to be stale during the build.
Steps
- checkout Wren:DS from
sustaining/3.0.0
.
- run
mvn clean install
.
Expected
Currently
- the build fails because it cannot find several packages of generated source code:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.3:compile (default-compile) on project opendj-dsml-servlet: Compilation failure: Compilation failure:
[ERROR] opendj-dsml-servlet/src/main/java/org/opends/dsml/protocol/DSMLAddOperation.java:[37,40] package org.opends.server.protocols.ldap does not exist
- afterwards, building with
mvn clean install -pl opendj-dsml-servlet
succeeds.
Summary
Wren:DS in Docker (release 5.0.0) attempts to initialize every time, even when instance dir is already populated, which results in error on subsequent starts.
Steps To Reproduce ("Repro Steps")
- Run Wren:DS with empty volume for instance data:
docker run --rm -v wrends-data:/opt/wrends/instance wrensecurity/wrends:5.0.0
, wait for it to initialize.
- Stop Wren:DS (Ctrl+C)
- Run the same command again (re-using the now populated volume)
Alternatively, instead of using volumes, create a container, start it, stop it and start it again.
Expected Result (Behavior You Expected to See)
Wren:DS start with existing instance data which has been created during the first start.
Actual Result (Behavior You Saw)
After following repro steps above, Wren:DS fails to start with the following output:
$ docker run --rm -v wrends-data:/opt/wrends/instance wrensecurity/wrends:5.0.0
First start...
Server Already Configured
setup command-line can only be used with servers that have not yet been
configured. The current server:
- Contains data
- Has already been configured
Screenshots
N/A
Environment
- Product version (include GIT commit ID if using a dev build): Image from Docker Hub, version 5.0.0.
- Operating system: N/A (Docker)
- JRE version: N/A (Docker)
- Was instance upgraded from a previous version? If so, what version were you using before? N/A
Additional Notes
The problem lies in unset variable INSTANCE_DIR
here:
|
if [ ! -d $INSTANCE_DIR/config ]; then |
.
Summary
If a collection sub-resource defined in the Rest2LDAP endpoints JSON file is marked "isReadOnly": true
, that endpoint does not appear at all in the API descriptor returned by CREST.
Affected Version
master
(4.0.0-SNAPSHOT
)
Steps to Reproduce
- Alter the
example-v1.json
file, and add the following under resourceTypes/example-v1/subResources
(ensure you don't remove the existing users
and groups
sub-resource definitions):
"all-users": {
"type": "collection",
"dnTemplate": "ou=people,dc=example,dc=com",
"resource": "frapi:opendj:rest2ldap:user:1.0",
"namingStrategy": {
"type": "clientDnNaming",
"dnAttribute": "uid"
},
"isReadOnly": true
2, Alter the Rest2LdapJsonConfiguratorTest
, adding the line System.out.println(prettyPrint(api));
right after the parseJson(prettyPrint(api));
line.
3. Run the test.
4. Observe the console output from the test.
Expected results
The output includes the new all-users
sub-resource, even though it's read-only. Something like the attached expected.json
.
Actual results
The output omits all read-only collection sub-resources. See the attached actual.json
.
Attachments
actual vs. expected JSON.zip
Summary
import-ldif
crashes with OutOfMemoryError on Java 17.
Steps To Reproduce ("Repro Steps")
-
Run setup with sample data creation:
./wrends/setup \
--cli \
--backendType je \
--baseDN dc=example,dc=com \
--ldapPort 1389 \
--adminConnectorPort 4444 \
--rootUserDN cn=Directory\ Manager \
--rootUserPassword password \
--no-prompt \
--noPropertiesFile \
--doNotStart \
--sampleData 200
Expected Result (Behavior You Expected to See)
import-ldif
successfully imports data without errors.
Actual Result (Behavior You Saw)
The import stucks on message:
Importing Automatically-Generated Data (200 Entries) ........
The log in /tmp/opendj-setup-*.log
contains following error:
[24/11/2022:11:56:07 +0100] category=QUICKSETUP seq=72 severity=INFO msg=import-ldif arg list: [/tmp/wrends/bin/import-ldif, --offline, -n, userRoot, -A, /tmp/opendj-install17722320096080648957.template, -s, 0, -F, --noPropertiesFile]
[24/11/2022:11:56:11 +0100] category=QUICKSETUP seq=73 severity=INFO msg=import-ldif out log: [24/11/2022:11:56:11 +0100] category=JVM seq=0 severity=INFO msg=Installation Directory: /tmp/wrends
[24/11/2022:11:56:11 +0100] category=QUICKSETUP seq=74 severity=INFO msg=import-ldif out log: [24/11/2022:11:56:11 +0100] category=JVM seq=1 severity=INFO msg=Instance Directory: /tmp/wrends
[24/11/2022:11:56:11 +0100] category=QUICKSETUP seq=75 severity=INFO msg=import-ldif out log: [24/11/2022:11:56:11 +0100] category=JVM seq=2 severity=INFO msg=JVM Information: 17.0.5+8-Ubuntu-2ubuntu1 by Private Build, 64-bit architecture, 3856662528 bytes heap size
[24/11/2022:11:56:11 +0100] category=QUICKSETUP seq=76 severity=INFO msg=import-ldif out log: [24/11/2022:11:56:11 +0100] category=JVM seq=3 severity=INFO msg=JVM Host: foobar, running Linux 5.19.0-23-generic amd64, 15426191360 bytes physical memory size, number of processors available 16
[24/11/2022:11:56:11 +0100] category=QUICKSETUP seq=77 severity=INFO msg=import-ldif out log: [24/11/2022:11:56:11 +0100] category=JVM seq=4 severity=INFO msg=JVM Arguments: "--add-exports=java.base/sun.security.x509=ALL-UNNAMED", "--add-exports=java.base/sun.security.tools.keytool=ALL-UNNAMED", "-Dorg.opends.server.scriptName=import-ldif"
[24/11/2022:11:56:11 +0100] category=QUICKSETUP seq=78 severity=INFO msg=import-ldif out log: [24/11/2022:11:56:11 +0100] category=BACKEND seq=5 severity=FINE msg=JE backend 'userRoot' does not specify the number of cleaner threads: defaulting to 16 threads
[24/11/2022:11:56:11 +0100] category=QUICKSETUP seq=79 severity=INFO msg=import-ldif out log: [24/11/2022:11:56:11 +0100] category=BACKEND seq=6 severity=FINE msg=JE backend 'userRoot' does not specify the number of lock tables: defaulting to 37
[24/11/2022:11:56:11 +0100] category=QUICKSETUP seq=80 severity=INFO msg=import-ldif out log: [24/11/2022:11:56:11 +0100] category=BACKEND seq=24 severity=INFO msg=Wren:DS Server 4.0.0-M4-SNAPSHOT starting import (build 20221124115033, Rd7998ffe96597b13df5e2d4af125455f05fbf0da)
[24/11/2022:11:56:12 +0100] category=QUICKSETUP seq=81 severity=INFO msg=import-ldif out log: [24/11/2022:11:56:12 +0100] category=BACKEND seq=25 severity=INFO msg=The amount of free memory available to the import task is 3470996275 bytes. The number of phase one buffers required is 640 buffers
[24/11/2022:11:56:12 +0100] category=QUICKSETUP seq=82 severity=INFO msg=import-ldif out log: [24/11/2022:11:56:12 +0100] category=BACKEND seq=26 severity=INFO msg=Setting DB cache size to 33554432 bytes and phase one buffer size to 4194304 bytes
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=84 severity=WARNING msg=import-ldif error log: Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=85 severity=WARNING msg=import-ldif error log: at java.base/java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:64)
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=86 severity=WARNING msg=import-ldif error log: at java.base/java.nio.ByteBuffer.allocate(ByteBuffer.java:363)
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=87 severity=WARNING msg=import-ldif error log: at org.opends.server.backends.pluggable.OnDiskMergeImporter$BufferPool.<init>(OnDiskMergeImporter.java:2890)
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=88 severity=WARNING msg=import-ldif error log: at org.opends.server.backends.pluggable.OnDiskMergeImporter$StrategyImpl.newHeapBufferPool(OnDiskMergeImporter.java:466)
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=89 severity=WARNING msg=import-ldif error log: at org.opends.server.backends.pluggable.OnDiskMergeImporter$StrategyImpl.newBufferPool(OnDiskMergeImporter.java:401)
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=90 severity=WARNING msg=import-ldif error log: at org.opends.server.backends.pluggable.OnDiskMergeImporter$StrategyImpl.importLDIF(OnDiskMergeImporter.java:210)
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=91 severity=WARNING msg=import-ldif error log: at org.opends.server.backends.pluggable.BackendImpl.importLDIF(BackendImpl.java:660)
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=92 severity=WARNING msg=import-ldif error log: at org.opends.server.tools.ImportLDIF.processLocal(ImportLDIF.java:853)
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=93 severity=WARNING msg=import-ldif error log: at org.opends.server.tools.tasks.TaskTool.process(TaskTool.java:246)
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=94 severity=WARNING msg=import-ldif error log: at org.opends.server.tools.ImportLDIF.process(ImportLDIF.java:243)
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=95 severity=WARNING msg=import-ldif error log: at org.opends.server.tools.ImportLDIF.mainImportLDIF(ImportLDIF.java:113)
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=96 severity=WARNING msg=import-ldif error log: at org.opends.server.tools.ImportLDIF.main(ImportLDIF.java:88)
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=83 severity=INFO msg=import-ldif out log: [24/11/2022:11:56:14 +0100] category=BACKEND seq=27 severity=INFO msg=Import LDIF environment close took 0 seconds
[24/11/2022:11:56:14 +0100] category=QUICKSETUP seq=97 severity=INFO msg=import-ldif out log: [24/11/2022:11:56:14 +0100] category=BACKEND seq=28 severity=INFO msg=Flushing data to disk
Environment
- Product version (include GIT commit ID if using a dev build): master (d7998ff)
- Operating system: Kubuntu 22.10, Linux 5.19.0-23-generic
- JRE version: openjdk 17.0.5 2022-10-18
- Was instance upgraded from a previous version? If so, what version were you using before? No
- CPU: AMD Ryzen 7 PRO 4750U (16 threads)
- Memory: 14GB
Additional Notes
- I did not notice this problem on Java 8 neither 11.
- Manual invocation of
import-ldif
with -Xmx256m
works.
- It works on some other computers with different CPU and memory size.
When doing persistent search via LDAPConnectionHandler2
the result entry does not contain EntryChangeNotification
control. This causes issues in Wren:AM that uses that information in SMS component.
I have already debugged what is happening and the issue is that LDAPClientConnection2
is mapping search result entry to response using method that leads to calling incorrect overloaded constructor.
Quite obvious fix is to add overloaded method to Responses
class. However I would like to verify that it is the correct way and also possibly investigate if there is a place where we can add unit test to test this behaviour.
we have a CDDL copy of OpenDJ 3.5.x.
we should:
- create a
sustaining/3.5
branch off of the forgerock/3.0.0
tag
- check-in our copy of 3.5.x on that branch.
- update the branch with build changes from
sustaining/3.0
so that it can build.
ForgeRock introduced the Persistit Data Backend (PDB) in OpenDJ 3.0.0 and deprecated it in DS 5 (i.e. OpenDJ 4.0.0).
There are multiple reasons why they abandoned it:
- ForgeRock had to fork the project because Akiban was acquired and no longer maintains it.
- Even after patches from FR, it has remaining issues that can cause deadlock in production,
- Unit tests for Persistit are painfully slow and fail if you actually run them.
- PDB is no longer needed because JE licensing is now compatible with OEM deployments of OpenDJ / Wren:DS. We still need to upgrade the JE backend – see #1.
Summary
When in multi-master replication mode a single server is unreachable (socket connection has to timeout) it will cause replication server to not be able to accept any connection due to data server's own socket timeout. This happens during the first DS-RS handshake phase and is accompanied with the following error in the log:
category=SYNC severity=ERROR msgID=178 msg=Directory server 15265 was attempting to connect to replication server 25602 but has disconnected in handshake phase
Steps To Reproduce ("Repro Steps")
- Download attached docker-compose.yml
- Start both server instances with
docker-compose up -d
- Configure replication:
docker exec -it wrends-test1 \
dsreplication enable --adminUID admin --adminPassword password --trustAll --no-prompt --baseDN dc=example,dc=com \
--host1 wrends-test1 --port1 4444 --bindDN1 "cn=Directory Manager" \
--bindPassword1 password --replicationPort1 8989 \
--host2 wrends-test2 --port2 4444 --bindDN2 "cn=Directory Manager" \
--bindPassword2 password --replicationPort2 8989
docker exec -it wrends-test1 \
dsreplication initialize-all --adminUID admin --adminPassword password --trustAll --no-prompt \
--baseDN dc=example,dc=com --hostname wrends-test1 --port 4444
- Shutdown both servers with
docker-compose stop
- Start only the first server with
docker-compose up -d wrends-test1
- Try to perform modification with
docker exec -it wrends-test1 ldapdelete -h localhost -p 1389
docker exec -it wrends-test1 ldapdelete -h localhost -p 1389 "uid=user.1,ou=People,dc=example,dc=com"
Expected Result (Behavior You Expected to See)
Server deletes the requested LDAP entry.
Actual Result (Behavior You Saw)
The following error is returned:
Processing DELETE request for uid=user.1,ou=People,dc=example,dc=com
The LDAP delete request failed: 53 (Unwilling to Perform)
Additional Information: The Replication is configured for suffix
dc=example,dc=com but was not able to connect to any Replication Server
Additional Notes
I have spent several hours trying to debug this. The underlying issue is that replication server's connection listener is trying to contact all other replication servers when accepting new connection. As the other server does not exist, this attempt timeouts with the same timeout value as is the data server's timeout for the connection handshake.
I am not sure why we need to wait for the replication domain to actually contact all servers. Simply increasing handshake timeout wouldn't work if we have multiple servers in the domain.
Creating this issue to actually track potential discussion regarding solving this problem.
It is possible to move all repositories from WrenArchiver here?
And you can convert the WrenArchiver personal account to organization too.
When calling for status of DS i'm getting an exit code 6 as a return.
./status -ns
Server Run Status: Started
Open Connections: 0
Host Name: <hostname>
Administrative Users: cn=Directory Manager
Installation Path:
<installation path>/wrends/opendj-server-legacy/target/package/wrends
Version: Wren:DS Server 4.0.0-RC2-SNAPSHOT
Java Version: <not available> (*)
Administration Connector: Port 4444 (LDAPS)
Address:Port: --
Protocol: LDIF
State: Disabled
Address:Port: 0.0.0.0:636
Protocol: LDAPS
State: Disabled
Address:Port: 0.0.0.0:1389
Protocol: LDAP
State: Enabled
Address:Port: 0.0.0.0:1689
Protocol: JMX
State: Disabled
Address:Port: 0.0.0.0:8080
Protocol: HTTP
State: Disabled
Base DN: dc=example,dc=com
Backend ID: userRoot
Entries: <not available> (*)
Replication:
Get exit code in bash:
echo $?
6
Affected Versions
Summary
If you build the Wren:DS project on Windows, the artifacts that get produced for the "OpenIDM Account Change Notification Handler" do not contain the correct files. This leads to duplicate JAR files inside the opendj-openidm-account-change-notification-handler-3.5.1.zip
file, and a missing extensions manifest file inside the opendj-openidm-account-change-notification-handler-3.5.1.jar
file.
Environment
- Windows 10 - Version 1709 (OS Build 16299.309)
- Maven 3.5.3 (3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T14:49:05-05:00)
Steps to Reproduce
- Build Wren:DS 3.5.1 from source on a Windows 10 machine.
- Using the ZIP file under
opendj-openidm-account-change-notification-handler/target
, attempt to follow plug-in installation instructions from ForgeRock.
Note that step 5 in FR's documentation is incorrect; the command should be:
cd /path/to/opendj/bin
./ldapmodify \
--port 389 \
--bindDN "cn=Directory Manager" \
--bindPassword "password" \
--defaultAdd \
--filename \
../config/openidm-accountchange-plugin-sample-config
Expected Results
- The plug-in can be installed successfully (even with the default, sample config).
Actual Results
- The ZIP file contains two JAR files under
lib/extensions
(opendj-openidm-account-change-notification-handler-3.5.1.jar
and opendj-openidm-account-change-notification-handler-3.5.1-shaded.jar
), when it should only contain one.
- The JAR inside the ZIP file is missing the appropriate manifest file (
org.forgerock.opendj.config.AbstractManagedObjectDefinition
).
- Attempting to use the JAR file without that manifest leads to the following error message when trying to install the plug-in:
The Directory Server is unwilling to add configuration entry cn=OpenIDM Notification Handler,cn=Account Status Notification Handlers,cn=config because one of the add listeners registered with the parent entry cn=Account Status Notification Handlers,cn=config rejected this change with the message: An error occurred while trying to initialize an instance of class org.forgerock.openidm.accountchange.OpenidmAccountStatusNotificationHandler as an account status notification handler as defined in configuration entry cn=OpenIDM Notification Handler,cn=Account Status Notification Handlers,cn=config: ClassCastException: org.forgerock.opendj.server.config.meta.AccountStatusNotificationHandlerCfgDefn$AccountStatusNotificationHandlerCfgServerImpl cannot be cast to org.forgerock.openidm.accountchange.server.OpenidmAccountStatusNotificationHandlerCfg (OpenidmAccountStatusNotificationHandler.java:221 AccountStatusNotificationHandlerConfigManager.java:360 AccountStatusNotificationHandlerConfigManager.java:225 AccountStatusNotificationHandlerConfigManager.java:48 ServerManagedObjectAddListenerAdaptor.java:66 ConfigAddListenerAdaptor.java:221 ConfigurationHandler.java:463 ConfigurationBackend.java:404 LocalBackendAddOperation.java:460 LocalBackendAddOperation.java:161 LocalBackendWorkflowElement.java:736 LocalBackendWorkflowElement.java:1051 LocalBackendWorkflowElement.java:894 AddOperationBasis.java:506 TraditionalWorkerThread.java:148)
The error results from Wren:DS not bootstrapping the configuration classes while loading the extension, which in turn means that the classes are not loaded by the time it needs to provide them to the extension. The extension tries to typecast the configuration to match the expected configuration type, but fails because Wren:DS is using only the generic configuration definition instead of the one for the plug-in.
Additional Information
The issue appears to be platform-specific. Packaging this on Ubuntu Linux 16.04 with Apache Maven 3.3.9 worked fine.
Recommend Projects
-
-
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
An Open Source Machine Learning Framework for Everyone
-
The Web framework for perfectionists with deadlines.
-
A PHP framework for web artisans
-
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
Some thing interesting about web. New door for the world.
-
A server is a program made to process requests and deliver data to clients.
-
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Some thing interesting about visualization, use data art
-
Some thing interesting about game, make everyone happy.
-
Recommend Org
-
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Open source projects and samples from Microsoft.
-
Google ❤️ Open Source for everyone.
-
Alibaba Open Source for everyone
-
Data-Driven Documents codes.
-
China tencent open source team.
-