Extensions for OpenJDK8 for Eclipse OpenJ9
License: GNU General Public License v2.0
For details of how to build Eclipse OpenJ9 see https://github.com/eclipse-openj9/openj9/tree/master/doc/build-instructions.
#62 added Eclipse licenses to OpenJDK code. The license of the code cannot be changed and these licenses must be removed (although an IBM copyright can be added).
I also made a couple of other review comments in ibmruntimes/openj9-openjdk-jdk9#138
Failures started on the 6th Feb build. Other platforms appear ok
../makelib/targets.mk:301: recipe for target 'jswalk.o' failed
makefile:491: recipe for target 'j9codert_vm' failed
OpenJ9.gmk:291: recipe for target 'build-j9' failed
/mnt/home/jenkins/workspace/openjdk8_openj9_build_x86-64_linux/openjdk/closed/make/Main.gmk:31: recipe for target 'j9vm-build' failed: integer expression expected
sun/awt/motif/X11GB2312$Decoder.class sun/awt/motif/X11GB2312$Encoder.class sun/awt/motif/X11GBK$Encoder.class sun/awt/motif/X11KSC5601$Decoder.class sun/awt/motif/X11KSC5601$Encoder.class are duplicated between rt.jar, charsets.jar, and localedata.jar
The problem is due to the CreateJars.gmk $$$$ escaping and the extensions implementation of ListPathsSafely which results in $$ in the class name output in the build exclude file, rather than just $.
The original implementation works for this.
libjncrypto.so library has dependency on OpenSSL crypto library libcrypto.so.1.1. OpenJ9-openjdk build has configure option --enable-openssl-bundling to facilitate bundling of OpenSSL crypto library with JDK. However, libjncrypto.so load fails even when libcrypto.so.1.1 is available alongside.
bhamaram@jcllnx01:/home/share/JDKs/OPENJ9/80/openssl/j2sdk-image/jre/lib/amd64$ ldd libjncrypto.so
linux-vdso.so.1 => (0x00007ffef10c7000)
libcrypto.so.1.1 => not found
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f5cf4bbf000)
/lib64/ld-linux-x86-64.so.2 (0x00007f5cf518d000)
bhamaram@jcllnx01:/home/share/JDKs/OPENJ9/80/openssl/j2sdk-image/jre/lib/amd64$ ls libcrypto.so.1.1
libcrypto.so.1.1
The java implementation of crypto is not performing very well with openJ9-OpenJDK build. To improve the crypto performance, native implementation can be used.
OpenSSL is an open source crypto TLS/SSL implementation [https://www.openssl.org/] and it is well established and stable native implementation used with enterprise applications. The source code is available at https://github.com/openssl/openssl.
The proposal is to use the OpenSSL crypto algorithms with OpenJ9-OpenJDK for better performance.
The design approach is to provide a switch in the existing security providers to switch to the native crypto implementation instead of creating a new security provider.
In the first phase, three crypto algorithms from OpenSSL are enabled for the native crypto. They are Digest [only SHA algorithms], Cipher Block Chaining [CBC] and Galois Counter Mode [GCM]. We can consider enabling more algorithms from OpenSSL in future.
The native crypto implementation is disabled by default and can be enabled using one or more of the following properties.
-Djdk.nativeCrypto=true
-Djdk.nativeDigest=true
-Djdk.nativeCBC=true
-Djdk.nativeGCM=true
If all three crypto algorithms from openssl to be enabled, use jdk.nativeCrypto. If any specific crypto algorithms only or its combinations to be enabled, use the corresponding property [jdk.nativeDigest, jdk.nativeCBC or jdk. nativeGCM]. Once the native implementation is stable, we can consider enabling them by default.
From the driver programs of the Digest [sun/security/provider/SunEntries], CBC and GCM [com/sun/crypto/provider/CipherCore] algorithms, check for the new property and switch to the native implementation.
A java interface [jdk.crypto.jniprovider.NativeCrypto] is introduced to call the JNI program [native/jdk/crypto /jniprovider/NativeCrypto.c] which act as a native wrapper program to make openssl calls for the Digest, CBC and GCM algorithms.
The high level design approach for implementing the openssl support is added as an attachment.
OpenJDK_NativeCrypto_DesignApproach.pptx
Specific version of openssl source can be cloned from the github based on the version specified with get_source.sh using --openssl-version=1.1.xx. Currently, only the latest version [1.1.xx] is supported. We can consider supporting other versions later. If openssl-version is not specified, openssl source does not get downloaded. If the openssl-version is specified, it will check if the same version is already available in standard download location SRC_DIR/openssl. If same version exists, it will skip downloading the source. Otherwise, it cleanup the openssl directory and download the source.
While configuring, user can select the openssl directory which is used for building opendk and whether the openssl crypto library [libcrypto.so] needs to be included in jre/lib/arch path.
The configure option --with-openssl can be used to select the openssl directory. The user can specify three options:
fetched - to use the downloaded openssl source.
system - to use the package installed openssl library
path to openssl library - to use a custom openssl library already built.
By default, the openssl crypto library[libcrypto.so] is not bundled with JDK build. If user wants to include the crypto library while building openj9-openjdk, they can use the option --enable-openssl-bundling. The openj9-openjdk8 build systems (AdoptOpenJDK and Eclipse OpenJ9 CI) can use the property --enable-openssl-bundling to include the crypto library with JDK which makes it easier for the Java users to exploit openssl crypto as otherwise, they have to build/install the openssl library in the target system.
A new library libjncrypto [where jn stands for java-native] is introduced to include the NativeCrypto object module. The libcrypto is dynamically linked to libjncrypto so that user has the flexibility to decide whether the crypto libraries need to be bundled with the JDK or not.
With the openssl support, the user is able to build OpenJDK_8-Eclipse_OpenJ9 with and without openssl.
For developers who want to continue without openssl, they can continue to do so without making any changes.
For developers who want to generate JDK with openssl can use the option '--with-openssl=path_to_crypto_library | fetched | system'
For developers who want to generate JDK with openssl and bundle the crypto library with JDK build can use the options -with-openssl= and --enable-openssl-bundling
In order to merge the openssl support change-set, there is no build environment dependency for AdoptOpenJDK and Eclipse OpenJ9. When ever, we install the pre-built openssl binaries in the build system and enable the property, it started building JDK with openssl support. Till then it continue to build the JDK as it was before.
#120 changed whitespace to be inconsistent with the rest of the file. The whitespace should be restored for consistency.
(sorry,because I speak a poor English,so I have to speak Chinese)
如果我使用git命令下载源码，编译时会提示不支持aarch64架构，而下载源码包（zip）的话，不会提示这个问题，但编译时会因没有hotspot目录而失败
Google translate:
If I use the git command to download the source code, the compiler will prompt that the aarch64 architecture is not supported. If the source package (zip) is downloaded, the problem will not be prompted, but the compilation will fail because there is no hotspot directory.
This is related to the following commit: 6eea72c#diff-fd91f51d4e7b6cdf50c66e28e4a5d7b1
Issue was introduced in the following builds:
*** GOOD ***
root@photon-machine [ /usr/java ]# jre18/bin/java -version
openjdk version "1.8.0_192"
OpenJDK Runtime Environment (build 1.8.0_192-201810180419-b12)
Eclipse OpenJ9 VM (build master-0d6badbd, JRE 1.8.0 Linux amd64-64-Bit Compressed References 20181018_56 (JIT enabled, AOT enabled)
OpenJ9 - 0d6badbd
OMR - 2bc0d2ac
JCL - 3f58849 based on jdk8u192-b12)
*** BAD ***
root@photon-machine [ /usr/java ]# jre19/bin/java -version
openjdk version "1.8.0_192"
OpenJDK Runtime Environment (build 1.8.0_192-201810191117-b12)
Eclipse OpenJ9 VM (build openj9-0.11.0-rc1, JRE 1.8.0 Linux amd64-64-Bit Compressed References 20181019_63 (JIT enabled, AOT enabled)
OpenJ9 - 72f731f8
OMR - ea548a66
JCL - fa63a62 based on jdk8u192-b12)
There are a number of extra shared libraries in the SDK and JRE images. Only the required libraries should be included in these images.
libjclse9_29.so is only for Java 9 and should just be excluded
libgdb_j929.so is for development/debugging and should be put elsewhere
the following are for testing, and should be put into another location so they can be used by the tests: libSharedClassesNativeAgent.so, libballoon29.so, libbcuwhite.so, libbcvwhite.so, libgptest.so, libhooktests.so, libj9ben.so, libj9lazyClassLoad.so, libj9thrnumanatives29.so, libj9thrtestnatives29.so, libj9unresolved.so, libj9vmtest.so, libjcoregen29.so, libjlmagent29.so, libjniargtests.so, libmemorywatcher29.so, libmigration.so, libosmemory29.so, libsoftmxtest.so, libtestjvmtiA.so, libtestjvmtiB.so, libtestlibA.so, libtestlibB.so, libvmruntimestateagent29.so
Moving the test libraries needs to be coordinated with the test team since the tests will need to find the libraries in the new location. eclipse-openj9/openj9#805
https://ci.eclipse.org/openj9/job/Test-sanity.system-JDK8-aix_ppc-64_cmprssptrs/107/console
https://ci.eclipse.org/openj9/job/Test-extended.system-JDK8-aix_ppc-64_cmprssptrs/58/console
03:47:33 clone_stf:
03:47:33 [exec] Cloning into 'stf'...
03:47:33 [exec] exec(): 0509-036 Cannot load program /opt/freeware/libexec/git-core/git-remote-https because of the following errors:
03:47:33 [exec] 0509-022 Cannot load module /usr/lib/libcurl.a(libcurl.so.4).
03:47:33 [exec] 0509-150 Dependent module /home/u0020236/workspace/Test-sanity.system-JDK8-aix_ppc-64_cmprssptrs/openjdkbinary/j2sdk-image/jre/lib/ppc64/libcrypto.a(libcrypto.so) could not be loaded.
03:47:33 [exec] 0509-152 Member libcrypto.so is not found in archive
03:47:33 [exec] 0509-022 Cannot load module git-remote-https.
03:47:33 [exec] 0509-150 Dependent module /usr/lib/libcurl.a(libcurl.so.4) could not be loaded.
03:47:33 [exec] 0509-022 Cannot load module .
03:47:33 [exec] Result: 128
The Java 9 extensions remove the unsupported executables ibmruntimes/openj9-openjdk-jdk9#52 and they also should be removed from Java 8.
jcmd, jinfo, jmap, jps, jstack, jstat, jstatd
and jhat
When running a make all after a make clean the build fils with the error below.
openj9_version.c:27:33: fatal error: openj9_version_info.h: No such file or directory
#include "openj9_version_info.h"
^
compilation terminated.
../makelib/targets.mk:296: recipe for target 'openj9_version.o' failed
In order to get this to work the bash configure script needs to be run again.
This should not be required and a make all should be able to be run after a make clean
Currently when the latest updates from openjdk are merged into the repository they are applied directly to the openj9 branch. This can cause issues for the eclipse/openj9 development process if the changes introduce new incompatibilities by causing the eclipse/openj9 build pipelines to fail until the issues are resolved.
This issue proposes a 'staging' branch which the eclipse/openj9 project can use to test the changes prior to them being incorporated into the main openj9 branch.
The Java 8 build doesn't build the dtfj, dtfjview or traceformat java code. Once the code is available, there should be executables created to invoke the code (dtfjview, traceformat, jextract).
While the IBM SDK creates three separate jar files (dtfj.jar, dtfjview.jar, traceformat.jar) and puts them into the jre/lib/ext directory, I don't think this is the proper place for them. These are diagnostic tools which don't need to be part of the runtime. I suggest they be put into the tools.jar instead.
The java source code is created by the Java Preprocessor ($(JPP_JAR)), using the following preprocessor configs: SIDECAR18-DTFJ, SIDECAR18-DTFJVIEW, SIDECAR18-TRACEFORMAT
Windows:
bin\jsig.dll
Linux/AIX:
lib/libjsig.so
The jsig file does not need to be in the compressedrefs/default directory.
As copied from the OpenJ9 Slack, from @sxa555 .
GNUmakefile:210: recipe for target 'mainbuild' failed
GNUmakefile:32: recipe for target 'phase_omr' failed
OpenJ9.gmk:306: recipe for target 'build-j9' failed
/ramdisk0/build/workspace/openjdk8_openj9_build_ppc64_aix/openjdk/closed/make/Main.gmk:29: recipe for target 'j9vm-build' failed: integer expression expected
Built the JDK!
�
/ramdisk0/build/workspace/openjdk8_openj9_build_ppc64_aix/openjdk/build/*/images/j2sdk-image not found
'' does not exist, build might have not been successful or not produced the expected JDK image at this location.�
Archiving artifacts
ERROR: No artifacts found that match the file pattern "OpenJDK*.tar.gz, *.zip". Configuration error?
ERROR: ?OpenJDK*.tar.gz? doesn?t match anything
#180 added jextract to the jre/bin directory, but I notice it's also in the bin directory. Since it's not a development tool, it doesn't need to be in the bin directory.
Java 8 RI has following fontconfig property files within jre/lib:
fontconfig.Turbo.properties.src
fontconfig.RedHat.5.bfc
fontconfig.RedHat.6.bfc
fontconfig.RedHat.5.properties.src
fontconfig.SuSE.10.bfc
fontconfig.RedHat.6.properties.src
fontconfig.SuSE.11.bfc
fontconfig.SuSE.10.properties.src
fontconfig.Turbo.bfc
fontconfig.bfc
fontconfig.SuSE.11.properties.src
fontconfig.properties.src
But OpenJ9 Java 8 doesn't have these files, is this intentional?
related to: eclipse-openj9/openj9#2659
Note: Java 11 RI ea doesn't have these files so far.
FYI: @DanHeidinga
Share classes specific code patched to OpenJDK usually have comments such as //IBM-shared_classes_misc which is going to be changed to //OpenJ9-shared_classes_misc as per ibmruntimes/openj9-openjdk-jdk11#113.
Unlike https://github.com/ibmruntimes/openj9-openjdk-jdk, https://github.com/ibmruntimes/openj9-openjdk-jdk12, and https://github.com/ibmruntimes/openj9-openjdk-jdk11, those share classes specific code within this repo doesn't have such comments which makes identifying those share classes code less straightforward.
https://ci.eclipse.org/openj9/view/Build/job/Build-JDK8-aix_ppc-64_cmprssptrs/1182/consoleFull
02:26:54.529 ## Starting images
02:27:07.480 Creating images/lib/charsets.jar
02:27:07.480 Creating images/lib/ext/cldrdata.jar
02:27:07.480 Creating images/lib/ext/dnsns.jar
02:27:10.716 Creating images/lib/ext/localedata.jar
02:27:10.716 Creating images/lib/jsse.jar
02:27:47.462 BuildJdk.gmk:101: recipe for target 'images' failed
02:27:47.462 gmake[1]: *** [images] Segmentation fault (core dumped)
02:27:47.462 gmake[1]: INTERNAL: Exiting with 3 jobserver tokens available; should be 8!
02:27:47.462 /home/jenkins/workspace/Build-JDK8-aix_ppc-64_cmprssptrs//make/Main.gmk:140: recipe for target 'images-only' failed
02:27:47.462 make: *** [images-only] Error 2
Machine is setup using Adopt playbooks and a change to the version of perl (5.24 -> 5.28).
I think this is using the bootjdk for generating these jars. If that is the case its failing with both openjdk7 and Adopt OpenJ9 JDK8
$ /usr/java7_org/bin/java -version
openjdk version "1.7.0-internal"
OpenJDK Runtime Environment (build 1.7.0-internal-jvmtests_2017_05_20_22_30-b00)
OpenJDK 64-Bit Server VM (build 24.95-b00, mixed mode)
$ /usr/java8_64/bin/java -version
openjdk version "1.8.0_202"
OpenJDK Runtime Environment (build 1.8.0_202-b08)
Eclipse OpenJ9 VM (build openj9-0.12.1, JRE 1.8.0 AIX ppc64-64-Bit Compressed References 20190205_216 (JIT enabled, AOT enabled)
OpenJ9 - 90dd8cb
OMR - d2f4534
JCL - d002501 based on jdk8u202-b08)
#65 moved the dtfj, dtfjview & traceformat code out of tools.jar and into individual jar files which are added into the lib/ext directory.
I'm not convinced this is the correct way to go. Although having the jars in the ext directory is how it works for IBM Java 8, its not the correct way to do things. Putting these jars in the ext directory incurs some costs. Both in the disk size of a JRE (which shouldn't contain these development tools), and also in class loading performance at runtime. Any class lookup based on the system classloader will be a little slower because of these additional jar files are in the ext directory.
The code should be put back into tools.jar. jdmpview and anything else required should be modified to find it there.
Initially TM (transactional memory) support was added to CLQ (ConcurrenLinkedQueue) and CHM (ConcurrentHashMap). However, further testing didn't show significant improvements with CHM, and changes for CHM were not added to IBM java 8, however CLQ is manintained in IBM Java 8.
This was discussed under the following Slack archive -
https://ibm-cloud.slack.com/archives/C401VJBD2/p1539375869000100
Summary of discussion - ::
It's ~100 lines of fairly straightforward code in the JIT and probably <= 20 lines of code in the JCL. In other words the "cost" of maintaining CLQ is not that much.
This issue is to add TM support for CLQ in the ibmruntimes extension repo.
JDK8 details:
The JDK9 build takes 12 minutes
The build log is as follows:
Creating images/lib/charsets.jar
Creating images/lib/ext/cldrdata.jar
Creating images/lib/ext/dnsns.jar
Creating images/lib/ext/localedata.jar
Creating images/lib/jsse.jar
Creating images/lib/jconsole.jar
Creating images/lib/dt.jar
Creating images/lib/tools.jar
Updating images/sec-bin.zip
Creating images/lib/ext/jaccess.jar
Updating images/src.zip
WARNING: Path does not exist as file or directory: java/lang/invoke/DirectMethodHandle.class
WARNING: Path does not exist as file or directory: java/lang/invoke/MethodHandleNatives.class
WARNING: Path does not exist as file or directory: java/lang/ref/Reference$Lock.class
WARNING: Path does not exist as file or directory: java/lang/ref/Reference$ReferenceHandler.class
WARNING: Path does not exist as file or directory: java/lang/ref/ReferenceQueue$Null.class
WARNING: Path does not exist as file or directory: java/lang/ref/ReferenceQueue$Lock.class
WARNING: Path does not exist as file or directory: java/lang/ref/Finalizer$FinalizerThread.class
WARNING: Path does not exist as file or directory: java/lang/Class$ReflectionData.class
WARNING: Path does not exist as file or directory: java/lang/Class$Atomic.class
WARNING: Path does not exist as file or directory: java/lang/Class$AnnotationData.class
WARNING: Path does not exist as file or directory: java/lang/ClassLoader$3.class
WARNING: Path does not exist as file or directory: java/lang/ClassLoader$NativeLibrary.class
WARNING: Path does not exist as file or directory: java/lang/System$2.class
WARNING: Path does not exist as file or directory: java/lang/Compiler$1.class
WARNING: Path does not exist as file or directory: java/lang/ClassLoader$ParallelLoaders.class
WARNING: Path does not exist as file or directory: java/net/URLClassLoader$7.class
WARNING: Path does not exist as file or directory: java/lang/SystemClassLoaderAction.class
WARNING: Path does not exist as file or directory: sun/misc/PostVMInitHook.class
WARNING: Path does not exist as file or directory: sun/usagetracker/UsageTrackerClient.class
WARNING: Path does not exist as file or directory: sun/usagetracker/UsageTrackerClient$1.class
WARNING: Path does not exist as file or directory: sun/usagetracker/UsageTrackerClient$4.class
WARNING: Path does not exist as file or directory: sun/usagetracker/UsageTrackerClient$3.class
WARNING: Path does not exist as file or directory: java/lang/Class$MethodArray.class
WARNING: Path does not exist as file or directory: java/lang/ClassLoader$2.class
WARNING: Path does not exist as file or directory: java/net/URLClassLoader$3$1.class
WARNING: Path does not exist as file or directory: java/lang/invoke/MethodType$ConcurrentWeakInternSet.class
WARNING: Path does not exist as file or directory: java/lang/invoke/Invokers.class
WARNING: Path does not exist as file or directory: java/lang/invoke/MethodType$ConcurrentWeakInternSet$WeakEntry.class
WARNING: Path does not exist as file or directory: java/lang/invoke/BoundMethodHandle$Factory.class
WARNING: Path does not exist as file or directory: java/lang/invoke/BoundMethodHandle$Species_L.class
WARNING: Path does not exist as file or directory: java/lang/invoke/DirectMethodHandle$Lazy.class
WARNING: Path does not exist as file or directory: java/lang/invoke/InvokerBytecodeGenerator$2.class
WARNING: Path does not exist as file or directory: java/lang/invoke/InvokerBytecodeGenerator$CpPatch.class
WARNING: Path does not exist as file or directory: java/lang/invoke/DirectMethodHandle$Accessor.class
WARNING: Path does not exist as file or directory: java/lang/invoke/LambdaFormEditor$Transform$Kind.class
WARNING: Path does not exist as file or directory: java/lang/invoke/LambdaFormEditor$Transform.class
WARNING: Path does not exist as file or directory: java/lang/invoke/LambdaFormBuffer.class
WARNING: Path does not exist as file or directory: java/security/AccessController$1.class
WARNING: Path does not exist as file or directory: java/lang/invoke/InfoFromMemberName.class
WARNING: Path does not exist as file or directory: sun/dc/DuctusRenderingEngine.class
WARNING: Path does not exist as file or directory: java/lang/invoke/DirectMethodHandle$Special.class
WARNING: Path does not exist as file or directory: sun/font/T2KFontScaler.class
WARNING: Path does not exist as file or directory: sun/font/T2KFontScaler$1.class
WARNING: Path does not exist as file or directory: sun/security/provider/DSA$LegacyDSA.class
WARNING: Path does not exist as file or directory: java/lang/Throwable$WrappedPrintStream.class
WARNING: Path does not exist as file or directory: java/lang/Throwable$PrintStreamOrWriter.class
WARNING: Path does not exist as file or directory: com/sun/crypto/provider
WARNING: Path does not exist as file or directory: javax/crypto
WARNING: Path does not exist as file or directory: sun/security/ec
WARNING: Path does not exist as file or directory: sun/security/internal
WARNING: Path does not exist as file or directory: sun/security/mscapi
WARNING: Path does not exist as file or directory: sun/security/pkcs11
WARNING: Path does not exist as file or directory: com/oracle/jrockit/jfr
WARNING: Path does not exist as file or directory: oracle/jrockit/jfr
WARNING: Path does not exist as file or directory: jdk/jfr
WARNING: Path does not exist as file or directory: com/ibm/jvm/traceformat
Creating rt.jar Compressed=false
Creating resources.jar
warning: package javax.crypto does not exist
warning: package javax.crypto.interfaces does not exist
warning: package javax.crypto.spec does not exist
warning: package javax.tools.annotation does not exist
warning: package jdk.management.cmm does not exist
warning: package com.oracle.nio does not exist
Using boot class path = [/apps/was/jenkins/workspace/Compile_OpenJ9/openj9-openjdk-jdk8/build/linux-x86_64-normal-server-release/images/lib/rt.jar, /apps/was/jenkins/workspace/Compile_OpenJ9/openj9-openjdk-jdk8/build/linux-x86_64-normal-server-release/langtools/dist/bootstrap/lib/javac.jar, /apps/was/java/oracle-jdk-8.162/jre/lib/resources.jar, /apps/was/java/oracle-jdk-8.162/jre/lib/sunrsasign.jar, /apps/was/java/oracle-jdk-8.162/jre/lib/jsse.jar, /apps/was/java/oracle-jdk-8.162/jre/lib/jce.jar, /apps/was/java/oracle-jdk-8.162/jre/lib/charsets.jar, /apps/was/java/oracle-jdk-8.162/jre/lib/jfr.jar, /apps/was/java/oracle-jdk-8.162/jre/classes, /apps/was/java/oracle-jdk-8.162/jre/lib/ext/dnsns.jar, /apps/was/java/oracle-jdk-8.162/jre/lib/ext/zipfs.jar, /apps/was/java/oracle-jdk-8.162/jre/lib/ext/nashorn.jar, /apps/was/java/oracle-jdk-8.162/jre/lib/ext/jaccess.jar, /apps/was/java/oracle-jdk-8.162/jre/lib/ext/cldrdata.jar, /apps/was/java/oracle-jdk-8.162/jre/lib/ext/jfxrt.jar, /apps/was/java/oracle-jdk-8.162/jre/lib/ext/sunec.jar, /apps/was/java/oracle-jdk-8.162/jre/lib/ext/sunjce_provider.jar, /apps/was/java/oracle-jdk-8.162/jre/lib/ext/localedata.jar, /apps/was/java/oracle-jdk-8.162/jre/lib/ext/sunpkcs11.jar]
Ignoring (other) com.sun.management.package-info :
ClassSymbol
Ignoring (other) com.sun.net.httpserver.package-info :
ClassSymbol
Ignoring (other) com.sun.net.httpserver.spi.package-info :
ClassSymbol
Ignoring (other) com.sun.nio.sctp.package-info :
ClassSymbol
Ignoring (other) com.sun.security.auth.package-info :
ClassSymbol
Ignoring (other) com.sun.security.auth.callback.package-info :
ClassSymbol
Ignoring (other) com.sun.security.auth.login.package-info :
ClassSymbol
Ignoring (other) com.sun.security.auth.module.package-info :
ClassSymbol
Ignoring (other) com.sun.security.jgss.package-info :
ClassSymbol
Ignoring (other) javax.xml.ws.wsaddressing.package-info :
ClassSymbol
Ignoring (other) jdk.net.package-info :
ClassSymbol
6 warnings
Creating images/lib/ct.sym
The content of https://github.com/ibmruntimes/openj9-openjdk-jdk8/blob/openj9/openj9-notices.html is out of date wrt OpenJ9 https://github.com/eclipse/openj9/blob/master/longabout.html
The file is copied into the jdk and jre images when builds are created. Rather than update the copy, a better solution is to delete openj9-notices.html, and copy the OpenJ9 file to the images instead, which can be renamed to openj9-notices.html during the copy.
I am attempting to compile on a rhel7.2 ppcle machine and getting this error.
OpenJ9 compile complete
Creating support/modules_libs/java.base/server/libjvm.so from J9 sources
Creating support/modules_libs/java.base/libjsig.so from J9 sources
(cd /tmp/bld_371745/openjdk/closed && /usr/bin/gmake -s VERBOSE="-s" LOG_LEVEL="warn" -I /tmp/bld_371745/openjdk/make/common -f OpenJ9.gmk SPEC=/tmp/bld_371745/openjdk/build/linux-ppc64-normal-server-release/spec.gmk stage_openj9_build_jdk)
Setting /tmp/bld_371745/openjdk/build/linux-ppc64-normal-server-release/jdk/lib/classlib.properties
## Starting corba
Compiling 6 files for BUILD_LOGUTIL
/tmp/bld_371745/openjdk/corba/src/share/classes/com/sun/tools/corba/se/logutil/IndentingPrintWriter.java:35: error: cannot access Object
public class IndentingPrintWriter extends PrintWriter {
^
class file for java.lang.Object not found
/tmp/bld_371745/openjdk/corba/src/share/classes/com/sun/tools/corba/se/logutil/IndentingPrintWriter.java:38: error: cannot find symbol
private String indentString = "" ;
^
symbol: class String
location: class IndentingPrintWriter
The SDK I am using is:
Java(TM) SE Runtime Environment (build pxl6470_27sr5-20171115_03(SR5))
IBM J9 VM (build 2.7, JRE 1.7.0 Linux ppc64le-64 Compressed References 20171111_369890 (JIT enabled, AOT enabled)
J9VM - R27_jvm.27_20171111_1528_B369890
JIT - tr.r13.java_20171111_369890
GC - R27_jvm.27_20171111_1528_B369890_CMPRSS
J9CL - 20171111_369890)
JCL - 20171109_01 based on Oracle jdk7u161-b13
It appears that OpenJ9 compilation (native only??) is complete; I don't see any Object or String class within the $openjdk/build/ directory. When I use openjdk7 (b95) I don't have this issue, but 'make images' hangs at 'Creating images/lib/jsse.jar'. Performing top shows that gmake is consuming 100% cpu, I've been killing it after an hour.
Currently I am attempting with openjdk b151 to see if I get any further.
As a result of bundling openssl libraries with the jdk the openssl license needs to be added to the third party license notices file included in the built jdk.
Create job configuration on Eclipse openj9 CI (ci.eclipse.org/openj9) Jenkins to run "mirror" script - https://github.ibm.com/open-class-libraries/openjdk-build/blob/master/git-hg/update8.sh
(Note : this is bash script)
It requires the jenkins job to provide 2 variables
$GITHUB_IBM (eg https://github.ibm.com/open-class-libraries)
$J9REPOSITORY (eg openJDK8u152-b16-update-test)
Currently the OpenSSL support is only for the 1.1.x version. Since most OSes don't provide this atm, to work around it the OpenJ9 0.11.0 release builds bundle the openssl 1.1 library. This is not ideal since the JVM will not pick up new openssl libraries when they are available. It may be desired to use updated openssl libraries to get bug fixes or security updates. If the openssl library is found on the OS, the user can install and update the library using standard package managers. This requires adding openssl support for 1.0
I presume this is somewhere around the redirector.c code within openj9
if you download the s390x AO8/J9 builds from www.AdoptOpenJDK.net and try to run it you get this error:
jcktest@csz25074:/dev/shm/jdk8u152-b16/bin$ ./java -version
Error: missing `server' JVM at `/dev/shm/jdk8u152-b16/jre/lib/s390x/server/libjvm.so'.
Please install or use the JRE or JDK that contains these missing components.
jcktest@csz25074:/dev/shm/jdk8u152-b16/bin$
Creating a symbolic link from jre/lib/s390x/server to jre/lib/s390x/j9vm allows the JVM to start up, so somewhere along the lines in the build it's still looking for libjvm.so in HotSpot's location.
$ make profiles
[...]
Creating rt.jar profile_1 Compressed=false
Creating resources.jar
Updating rt.jar profile_1
Analyzing /home/keithc/space/jdk08/build/linux-s390x-normal-server-release/images/j2re-compact1-image/lib/resources.jar
Analyzing /home/keithc/space/jdk08/build/linux-s390x-normal-server-release/images/j2re-compact1-image/lib/rt.jar
Analyzing /home/keithc/space/jdk08/build/linux-s390x-normal-server-release/images/j2re-compact1-image/lib/jce.jar
Analyzing /home/keithc/space/jdk08/build/linux-s390x-normal-server-release/images/j2re-compact1-image/lib/jsse.jar
com.ibm.lang.management.internal.ExtendedMemoryMXBeanImpl -> java.lang.management.MemoryType (unknown type)
com.ibm.java.lang.management.internal.MemoryMXBeanImpl -> java.lang.management.MemoryType (unknown type)
com.ibm.java.lang.management.internal.ManagementUtils -> java.lang.management.MemoryType (unknown type)
com.ibm.lang.management.internal.MemoryPoolMXBeanImpl -> java.lang.management.MemoryType (unknown type)
com.ibm.java.lang.management.internal.MemoryPoolMXBeanImpl -> java.lang.management.MemoryType (unknown type)
com.ibm.java.lang.management.internal.ManagementUtils -> javax.management.openmbean.TabularType (unknown type)
com.ibm.java.lang.management.internal.MemoryUsageUtil -> javax.management.openmbean.TabularType (unknown type)
com.ibm.java.lang.management.internal.ManagementUtils -> com.sun.management.OperatingSystemMXBean (unknown type)
com.ibm.lang.management.OperatingSystemMXBean -> com.sun.management.OperatingSystemMXBean (unknown type)
com.ibm.java.lang.management.internal.ManagementUtils -> java.lang.management.BufferPoolMXBean (unknown type)
[...]
It would seem J9 JCL source is not included in the generated src.zip, only the base OpenJDK src...
Re #112
On aix using
get_source.sh --openssl-version=1.1.1
configure with --with-openssl=fetched --enable-openssl-bundling
15:36:02 ar: creating an archive file apps/libapps_a.a
15:36:19 147 1500-010: (W) WARNING in async_start_func: Infinite loop. Program may not stop.
15:37:03 "crypto/ec/ecp_nistz256.c", line 227.31: 1506-1298 (W) The subscript 4 is out of range. The valid range is 0 to 3.
15:37:03 "crypto/ec/ecp_nistz256.c", line 228.31: 1506-1298 (W) The subscript 5 is out of range. The valid range is 0 to 3.
15:37:03 "crypto/ec/ecp_nistz256.c", line 229.31: 1506-1298 (W) The subscript 6 is out of range. The valid range is 0 to 3.
15:37:03 "crypto/ec/ecp_nistz256.c", line 1251.37: 1506-1298 (W) The subscript 4 is out of range. The valid range is 0 to 3.
15:37:03 "crypto/ec/ecp_nistz256.c", line 1251.48: 1506-1298 (W) The subscript 5 is out of range. The valid range is 0 to 3.
15:37:03 "crypto/ec/ecp_nistz256.c", line 1251.59: 1506-1298 (W) The subscript 6 is out of range. The valid range is 0 to 3.
15:37:03 "crypto/ec/ecp_nistz256.c", line 1251.70: 1506-1298 (W) The subscript 7 is out of range. The valid range is 0 to 3.
15:37:03 "crypto/ec/ecp_nistz256.c", line 1252.37: 1506-1298 (W) The subscript 4 is out of range. The valid range is 0 to 3.
15:37:03 "crypto/ec/ecp_nistz256.c", line 1252.48: 1506-1298 (W) The subscript 5 is out of range. The valid range is 0 to 3.
15:37:03 "crypto/ec/ecp_nistz256.c", line 1252.59: 1506-1298 (W) The subscript 6 is out of range. The valid range is 0 to 3.
15:37:03 "crypto/ec/ecp_nistz256.c", line 1252.70: 1506-1298 (W) The subscript 7 is out of range. The valid range is 0 to 3.
15:37:03 "crypto/ec/ecp_nistz256.c", line 1262.36: 1506-1298 (W) The subscript 4 is out of range. The valid range is 0 to 3.
15:37:03 "crypto/ec/ecp_nistz256.c", line 1262.27: 1506-1298 (W) The subscript 4 is out of range. The valid range is 0 to 3.
15:37:03 "crypto/ec/ecp_nistz256.c", line 1263.36: 1506-1298 (W) The subscript 5 is out of range. The valid range is 0 to 3.
15:37:03 "crypto/ec/ecp_nistz256.c", line 1263.27: 1506-1298 (W) The subscript 5 is out of range. The valid range is 0 to 3.
15:37:03 "crypto/ec/ecp_nistz256.c", line 1264.36: 1506-1298 (W) The subscript 6 is out of range. The valid range is 0 to 3.
15:37:03 "crypto/ec/ecp_nistz256.c", line 1264.27: 1506-1298 (W) The subscript 6 is out of range. The valid range is 0 to 3.
15:37:03 "crypto/ec/ecp_nistz256.c", line 1265.36: 1506-1298 (W) The subscript 7 is out of range. The valid range is 0 to 3.
15:37:03 "crypto/ec/ecp_nistz256.c", line 1265.27: 1506-1298 (W) The subscript 7 is out of range. The valid range is 0 to 3.
15:37:13 1500-030: (I) INFORMATION: openssl_add_all_ciphers_int: Additional optimization may be attained by recompiling and specifying MAXMEM option with a value greater than 16384.
15:38:36 ar: creating an archive file libcrypto_a.a
15:38:57 ar: creating an archive file libssl_a.a
15:38:59 ar: creating an archive file test/libtestutil_a.a
15:39:03 ar: creating an archive file libcrypto.a
15:39:06 ar: creating an archive file libssl.a
...
16:08:21 Compiling 226 files for BUILD_SECURITY
16:08:28 Note: Some input files use or override a deprecated API.
16:08:28 Note: Recompile with -Xlint:deprecation for details.
16:08:33 gmake[2]: *** No rule to make target '/home/u0020236/workspace/PullRequest-Compile-JDK8-aix_ppc-64_cmprssptrs-OpenJ9/openssl/libcrypto.so.1.1', needed by '/home/u0020236/workspace/PullRequest-Compile-JDK8-aix_ppc-64_cmprssptrs-OpenJ9/build/aix-ppc64-normal-server-release/jdk/lib/ppc64/libcrypto.so.1.1'. Stop.
16:08:33 gmake[2]: *** Waiting for unfinished jobs....
16:08:33 BuildJdk.gmk:74: recipe for target 'libs-only' failed
16:08:33 gmake[1]: *** [libs-only] Error 2
16:08:33 /home/u0020236/workspace/PullRequest-Compile-JDK8-aix_ppc-64_cmprssptrs-OpenJ9//make/Main.gmk:119: recipe for target 'jdk-only' failed
16:08:33 make: *** [jdk-only] Error 2
https://ci.eclipse.org/openj9/job/PullRequest-Compile-JDK8-aix_ppc-64_cmprssptrs-OpenJ9/36/console
If smt is enabled the correct number of processors is not correct on configure.
There are a number of ways that can be used to determine actual number of processors, I found this works for me on the systems I have access to, smt enabled or not at time of configure
--with-jobs='pmcycles -m | wc -l | awk '{print $1}''
#36 added OPENJ9_TAG, which is intended to be the TAG associated with the current SHA being built for the OpenJ9 repo. However OPENJ9_TAG is the last tag available.
Java version output of openJ9 builds doesn't contain proper OPENJDK_TAG.
openjdk version "1.8.0-internal"
OpenJDK Runtime Environment (build 1.8.0-internal-jenkins_2018_01_09_13_02-b00)
Eclipse OpenJ9 VM (build 2.9, JRE 1.8.0 Linux amd64-64 Compressed References 20180109_42 (JIT enabled, AOT enabled)
OpenJ9 - b116fb3
OMR - 7a770d6
OpenJDK - 00e69f9 based on )
Proper output
openjdk version "1.8.0-internal"
OpenJDK Runtime Environment (build 1.8.0-internal-bhamaram_2018_01_10_00_29-b00)
Eclipse OpenJ9 VM (build 2.9, JRE 1.8.0 Linux amd64-64 Compressed References 20180110_000000 (JIT enabled, AOT enabled)
OpenJ9 - b116fb3
OMR - 7a770d6
OpenJDK - 00e69f9 based on jdk8u152-b16)
The following tests fail for OpenJ9 on zLinux due to the test specified timeouts, between 1 and 5 seconds, most likely being too short for the environment:
If the native crypto library is not available in the library path, then System.loadLibrary fails with an UnsatisfiedLinkError. In this situation, there is a fall back mechanism to the java crypto implementation by catching the exception. However, there is no warning messages issued while falling back and hence the user do not know the native crypto implementation is not used.
https://ci.eclipse.org/openj9/job/PullRequest-Compile-JDK8-osx_x86-64_cmprssptrs-OpenJ9/7
Using --with-openssl=fetched --enable-openssl-bundling
16:56:01 checking for OPENSSL... yes
16:56:01 checking if we should bundle openssl... yes
...
16:56:05 /usr/bin/make -s VERBOSE="-s" LOG_LEVEL="warn" -I /Users/jenkins/workspace/PullRequest-Compile-JDK8-osx_x86-64_cmprssptrs-OpenJ9/make/common -f /Users/jenkins/workspace/PullRequest-Compile-JDK8-osx_x86-64_cmprssptrs-OpenJ9/closed/openssl.gmk SPEC=/Users/jenkins/workspace/PullRequest-Compile-JDK8-osx_x86-64_cmprssptrs-OpenJ9/build/macosx-x86_64-normal-server-release/spec.gmk
16:56:05 Compiling OpenSSL in /Users/jenkins/workspace/PullRequest-Compile-JDK8-osx_x86-64_cmprssptrs-OpenJ9/openssl
...
17:09:15 make[2]: *** No rule to make target `/Users/jenkins/workspace/PullRequest-Compile-JDK8-osx_x86-64_cmprssptrs-OpenJ9/openssl/libcrypto.so.1.1', needed by `/Users/jenkins/workspace/PullRequest-Compile-JDK8-osx_x86-64_cmprssptrs-OpenJ9/build/macosx-x86_64-normal-server-release/jdk/lib/libcrypto.so.1.1'. Stop.
The script closed/get_j9_source.sh checks that the bash version in use it at least 4.0. The relevant line (
) checks that the bash version string (first line ofbash --version) contains 'version 4.'.Re #112
On zlinux using
get_source.sh --openssl-version=1.1.1
configure with --with-openssl=fetched --enable-openssl-bundling
15:34:21 + make all
15:34:27 /usr/bin/make -s VERBOSE="-s" LOG_LEVEL="warn" -I /home/jenkins/jenkins-agent/workspace/PullRequest-Compile-JDK8-linux_390-64_cmprssptrs-OpenJ9/make/common -f /home/jenkins/jenkins-agent/workspace/PullRequest-Compile-JDK8-linux_390-64_cmprssptrs-OpenJ9/closed/openssl.gmk SPEC=/home/jenkins/jenkins-agent/workspace/PullRequest-Compile-JDK8-linux_390-64_cmprssptrs-OpenJ9/build/linux-s390x-normal-server-release/spec.gmk
15:34:27 Compiling OpenSSL in /home/jenkins/jenkins-agent/workspace/PullRequest-Compile-JDK8-linux_390-64_cmprssptrs-OpenJ9/openssl
15:34:28 Configuring OpenSSL version 1.1.1 (0x1010100fL) for linux64-s390x
15:34:28 Using os-specific seed configuration
15:34:29 Creating configdata.pm
15:34:29 Creating Makefile
15:34:29
15:34:29 **********************************************************************
15:34:29 *** ***
15:34:29 *** If you want to report a building issue, please include the ***
15:34:29 *** output from this command: ***
15:34:29 *** ***
15:34:29 *** perl configdata.pm --dump ***
15:34:29 *** ***
15:34:29 **********************************************************************
15:34:33 ar: creating apps/libapps.a
15:35:43 crypto/sha/keccak1600-s390x.S: Assembler messages:
15:35:43 crypto/sha/keccak1600-s390x.S:356: Error: operand out of range (0xfffffffffffffff8 is not between 0x0000000000000000 and 0x0000000000000fff)
15:35:43 Makefile:4903: recipe for target 'crypto/sha/keccak1600-s390x.o' failed
15:35:43 make[3]: *** [crypto/sha/keccak1600-s390x.o] Error 1
15:35:43 Makefile:171: recipe for target 'all' failed
15:35:43 make[2]: *** [all] Error 2
15:35:43 /home/jenkins/jenkins-agent/workspace/PullRequest-Compile-JDK8-linux_390-64_cmprssptrs-OpenJ9/closed/openssl.gmk:37: recipe for target 'build_openssl' failed
15:35:43 make[1]: *** [build_openssl] Error 2
15:35:43 /home/jenkins/jenkins-agent/workspace/PullRequest-Compile-JDK8-linux_390-64_cmprssptrs-OpenJ9/closed/make/Main.gmk:33: recipe for target 'openssl-build' failed
15:35:43 make: *** [openssl-build] Error 2
https://ci.eclipse.org/openj9/job/PullRequest-Compile-JDK8-linux_390-64_cmprssptrs-OpenJ9/44/console
The OpenJ9 0.9 release is imminent, expected to have a rc1 candidate tag by 20180614. An appropriate branch needs to be created, and versions numbers for the release updated.
The jsig.lib file should be included in the lib directory.
In OpenJ9 PPC LE builds, the vm directory path uses ppc64, while it should be ppc64le. The OpenJDK builds use ppc64le.
There is a recent change for Windows that is blocking us from adding Windows JDK8 builds at https://ci.eclipse.org/openj9/
Please update the openj9 branch with the new new changes
Link to the change: http://hg.openjdk.java.net/jdk8u/jdk8u/rev/a05ed58d5ae0
author | kevinw
-- | --
Sat, 07 Apr 2018 03:28:39 -0700 (3 weeks ago)
cddffb220808
2209644bcac4
common/autoconf/basics_windows.m4 common/autoconf/generated-configure.sh
2 files changed, 5 insertions(+), 5 deletions(-) [+]
- # The cmd output ends with Windows line endings (CR/LF), the grep command will strip that away
- cygwin_winpath_root=`cd / ; cmd /c cd | grep ".*"`
+ # The cmd output ends with Windows line endings (CR/LF)
+ cygwin_winpath_root=`cd / ; cmd /c cd | $TR -d '\r\n'`
Blocking:
The javadoc created as part of the OpenJ9 build, found in build/<platform>/docs, does not contain the javadoc for OpenJ9 extensions:
See also eclipse-openj9/openj9#897
This issue is opened to fix the Crash issue reported in eclipse-openj9/openj9#3413
----------- Stack Backtrace -----------
(0x00007F91B731FB20 [libj9gc29.so+0x65b20])
Java_jdk_crypto_jniprovider_NativeCrypto_DigestUpdate+0x5c (0x00007F910961D73C [libjncrypto.so+0x173c])
(0x00007F91216A2B6C [<unknown>+0x0])
---------------------------------------
Third argument to ReleasePrimitiveArrayCritical() should be the return value of GetPrimitiveArrayCritical()
Build Log: https://gitlab.com/snippets/1851154
I've successfully compiled the 'normal' openjdk8 on this machine, but openj9 fails for some reason.
I'm happy to provide any additional info you need.
The following test cases are failed with openssl:
com/sun/crypto/provider/Cipher/AES/TestAESCiphers/TestAESWithProviderChange.java.TestAESWithProviderChange
com/sun/crypto/provider/Cipher/AES/TestAESCiphers/TestAESWithRemoveAddProvider.java.TestAESWithRemoveAddProvider
Caused by: java.security.AccessControlException: Access denied ("java.lang.RuntimePermission" "accessClassInPackage.sun.security.action")
at java.security.AccessController.throwACE(AccessController.java:176)
at java.security.AccessController.checkPermissionHelper(AccessController.java:237)
at java.security.AccessController.checkPermission(AccessController.java:373)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
at java.lang.SecurityManager.checkPackageAccess(SecurityManager.java:1564)
at java.lang.J9VMInternals$2.run(J9VMInternals.java:268)
at java.security.AccessController.doPrivileged(AccessController.java:673)
at java.lang.J9VMInternals.checkPackageAccess(J9VMInternals.java:263)
at com.sun.crypto.provider.CipherCore.(CipherCore.java:1231)
The issue is that from com.sun.crypto.provider.CipherCore, we are trying to access the class sun.security.action.GetPropertyAction. CipherCore is loaded with extension classloader and GetPropertyAction is loaded with boot class loader. This will result in a security exception as package.access restriction is defined in java.security. We can workaround by providing the RuntimePermission in the policy file. However, that is not a good approach as it is against the definition in java.security.. We need to read the property differently to avoid the issue..
Also, it makes more sense to have the value of jdk.nativeCrypto as 'yes' or 'no' instead of 'true' or 'false' as we need to change the default to enable native crypto when code stabilizes and then jdk.nativeCrypto=no instead of jdk.nativeCrypto=false to disable native crypto makes more sense.
The javadoc created as part of the OpenJ9 build, found in build/<platform>/docs, contains extensions which are not supported on OpenJ9.
See also eclipse-openj9/openj9#898
when --with-openssl=/path/to/pre-built/openssl --enable-openssl-bundling configure options are provided, build will copy libcrypto.so.1.1 to JDK directory. However, when directory structure of the openssl directory is different, configure couldn't check for openssl and thereby it fails. The same is fixed with #134. With this configure step is successful and fails when copying OpenSSL crypto library to JDK folder.
It is also required to fix the path from where OpenSSL crypto library is copied in the same way as #134.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.