jakartaee / platform-tck

Jakartaee-tck

JakartaEE TCK Jenkins Jobs

The Jenkins jobs required for certifying Eclipse GlassFish nightly builds using the latest Jakarta EE TCK bundles are hosted in the Eclipse CloudBees Infrastructure and are available under https://jenkins.eclipse.org/jakartaee-tck/

For information regarding the various JakartaEE TCK related jobs, Please refer to the wiki page below https://github.com/eclipse-ee4j/jakartaee-tck/wiki/Jakarta-EE-TCK-Jenkins-jobs

Steps required to run CTS against Standalone RI changes

1. Build the individual project and release it to Eclipse Maven repositories.

2. Integrate the individual project to GlassFish.

   Sample steps done for JTA:

   git clone https://github.com/eclipse-ee4j/glassfish.git
   git checkout -b EE4J_8
   find . -name \pom.xml -exec sed -i.bak "s/javax.transaction</jakarta.transaction</g" {} ;
   find . -name \pom.xml -exec sed -i.bak "s/javax.transaction-api</jakarta.transaction-api</g" {} ;
   mvn clean install
   

3. Upload the glassfish bundle built in the previous step as attachment to the corresponding pull request.

4. Request CTS team to run the CTS suites by tagging @anajosep and @bhatpmk

5. Provide the names of the jars (under glassfish6/glassfish/modules) that were modified in this bundle.

6. If known, specify the test areas that needs to be run, if not leave it empty.

7. Wait for the confirmation and the results uploaded from the runs against the pull request.

8. If the results are clean, commit the changes to the individual project and the changes required for integration to GlassFish.

Jakarta EE TCK Build and Run Instructions

Instructions for building and running JakartaEE TCK bundle from scratch is available in the following wiki page: JakartaEE TCK - Build and Run Instructions

CTS User Guide

Introduction

CTS Overview

A Java EE 8 CTS is a set of tools and tests used to verify that a licensee's implementation of Java EE 8 technology conforms to the applicable specification.

All tests in the CTS are based on the written specifications for the Java platform. The CTS tests compatibility of a licensee's implementation of a technology to the applicable specification of the technology. Compatibility testing is a means of ensuring correctness, completeness, and consistency across all implementations developed by technology licensees. The set of tests included with the Java EE 8 CTS is called the test suite. All tests in the CTS test suite are self-checking, but some tests may require tester interaction. Most tests return either a Pass or Fail status. For a given platform to be certified, all of the required tests must pass. The definition of required tests may change from platform to platform.

The definition of required tests will change over time. Before your final certification test pass, be sure to download the latest Exclude List for the Java EE 8 CTS.

About JavaEE8 CTS

Java EE 8 CTS is a portable, configurable, automated test suite for verifying the compliance of a licensee's implementation of the Java EE 8 technologies. Java EE 8 CTS includes version 5.0 of the JT harness.

For documentation on the test harness used for running the Java EE 8 CTS test suite, see https://wiki.openjdk.java.net/display/CodeTools/Documentation.

Hardware Requirements

The following section lists the hardware requirements for the Java EE 8 CTS software, using the Java EE 8 RI or Java EE 8 Web Profile RI. Hardware requirements for other reference implementations will vary.

All systems should meet the following recommended hardware requirements:

• CPU running at 2.0 GHz or higher

• 4 GB of RAM or more

• 2 GB of swap space , if required

• 6 GB of free disk space for writing data to log files, the Java EE 8 repository, and the database

• Network access to the Internet

Software Requirements

You can run the Java EE 8 CTS software on platforms running the Solaris, Linux, Windows, and Mac OS software that meet the following software requirements:

• Operating Systems:

  • Solaris 10 and newer

  • MAC OS X Mountain Lion (10.8.1+)

  • Windows XP SP3, Windows 2008 R2

  • Oracle Linux 6.4

  • Fedora 18

  • Ubuntu Linux 12.10

  • Suse Enterprise Linux 12.2

• Java SE 8 SDK

• Java EE 8 RI or Java EE 8 Web Profile RI

• Mail server that supports the IMAP and SMTP protocols

• One of the following databases:

  • Oracle

  • Sybase

  • DB2

  • Microsoft SQL Server

  • Postgres SQL

  • MySQL

  • Java DB

Additional Java EE 8 CTS Requirements

In addition to the instructions and requirements described in this document, all Java EE 8 and Java EE 8 Web Profile implementations must also pass the standalone TCKs for the following technologies:

• Contexts and Dependency Injection for Java 2.0 (JSR 365)

• Dependency Injection for Java 1.0 (JSR 330)

• Bean Validation 2.0 (JSR 380)

Installation

Install CTS bundle

Complete the following procedure to install the Java EE 8 CTS on a system running the Solaris, Linux, or Windows operating system.

1. Copy or download the CTS 8 software.

2. Change to the directory in which you want to install the Java EE 8 CTS software and use the unzip command to extract the bundle:

   cd install_directory
   unzip jakartaeetck-nnn.zip
   

This creates the jakartaeetck directory. The install_directory/jakartaeetck directory will be TS_HOME.

3. Set the TS_HOME environment variable to point to the jakartaeetck directory.

Install Apache Ant

1. Download the latest version of Apache Ant from the below link https://archive.apache.org/dist/ant/binaries/apache-ant--bin.zip

2. Change to the directory in which you want to install Apache Ant and extract the bundle

   unzip apache-ant-<version>-bin.zip
   

or tar zxvf apache-ant-<version>-bin.tar.gz 3. Set ANT_HOME environment variable to point to the apache-ant- directory.

4. Set PATH environment variable to use the installed ant.

Setup and Configuration

Allowed Modifications

You can modify the following test suite components only:

• Your implementation of the porting package

• ts.jte environment file

• The vendor-specific SQL files in <TS_HOME>/sql

• Any files in <TS_HOME>/bin and <TS_HOME>/bin/xml (except for ts.* files)

Configuring the Java EE 8 RI as the VI

To configure the Java EE 8 RI as the server under test (that is, to use the Java EE 8 RI as the VI) follow the steps listed below.

In this scenario, the goal is simply to test the Java EE 8 RI against the CTS for the purposes of familiarizing yourself with CTS test procedures. You may also want to refer to the Quick Start guides included with the Java EE 8 CTS for similar instructions.

1. Set server properties in your <TS_HOME>/bin/ts.jte file to suit your test environment. Be sure to set the following properties: a. Set the webServerHost property to the name of the host on which your Web server is running that is configured with the RI. The default setting is localhost.

   b. Set the webServerPort property to the port number of the host on which the Web server is running and configured with the RI. The default setting is 8001.

   c. Set the wsgen.ant.classname property to the Vendor's implementation class that mimics the RI Ant task that in turn calls the wsgen Java-to-WSDL tool. The default setting is com.sun.tools.ws.ant.WsGen.

   d. Set the wsimport.ant.classname property to the Vendor's implementation class that mimics the RI Ant task that in turn calls the wsimport WSDL-to-Java tool. The default setting is com.sun.tools.ws.ant.WsImport.

   e. Set the porting.ts.url.class property to your porting implementation class that is used for obtaining URLs. The default setting for the RI porting implementation is com.sun.ts.lib.implementation.sun.common.SunRIURL.

   f. Set the database-related properties in the <TS_HOME>/bin/ts.jte file.

   g. Add the following JVM option to the command.testExecuteAppClient property to enable the Security Manager in the application client container: -Djava.security.manager

Add this option to the list of other -D JVM options for this property. As mentioned previously, these settings can vary, but must match whatever you used when setting up the Java EE 8 RI server.

2. Install the Java EE 8 RI and configure basic settings, as described in

3. Start the Java EE 8 RI application server. Refer to the application server documentation for complete instructions.

4. Enable the Security Manager. If you are using the Java EE 8 RI, execute the following command from the command line:

   asadmin create-jvm-options -Djava.security.manager
   

5. Stop and restart your application server so it is running with the Security Manager enabled.

6. Change to the <TS_HOME>/bin directory.

7. Start your backend database.

If you are using Derby as your backend database, execute the start.javadb Ant target: ant -f xml/impl/glassfish/s1as.xml start.javadb Otherwise, refer to your backend database administration documentation for information about starting your database server.

8. Initialize your backend database. If you are using Derby as your backend database, execute the init.derby Ant target:

   ant -f xml/init.xml init.derby
   

9. Run the configuration Ant target.

   ant config.vi
   

10. Build the special web services clients.

The special webservices tests under the webservices12/specialcases directory have prebuilt endpoints, but the clients are not prebuilt. The clients will be built after the endpoints are first predeployed to the application server under test. During the build, the clients import the WSDLs (by means of the Java EE wsimport and wsgen tools) from the predeployed webservices endpoints. This process verifies that importing a WSDL from a predeployed webservice endpoint works properly. To build the special webservices clients, the following command must be executed: ant build.special.webservices.clients

Executing tests

Running tests in CLI mode

1. Set the TS_HOME environment variable to the directory in which Java EE 8 CTS was installed.

2. Set the JAVA_HOME environment variable to the latest version of JDK 8

3. Set the ANT_HOME environment variable to the latest version of Apache Ant installed.

4. Set the PATH environment to use the latest binaries.

   export PATH=$ANT_HOME/bin:$JAVA_HOME/bin:$PATH
   

5. Change to any subdirectory under <TS_HOME>/src/com/sun/ts/tests.

6. Ensure that the ts.jte file contains information relevant to your setup.

7. Execute the runclient Ant target to start the JavaTest:

   ant runclient
   

This runs all tests in the current directory and any subdirectories.

8. To run the Java EE 8 CTS signature tests, enter the following commands:

   cd <TS_HOME>/src/com/sun/ts/tests/signaturetest/javaee
   ant runclient
   

9. To run a single test directory in the forward direction, enter the following commands:

   cd <TS_HOME>/src/com/sun/ts/tests/jaxws/api/jakarta_xml_ws/Dispatch
   ant -Dkeywords=forward runclient
   

10. To run a subset of test directories in the reverse direction, enter the following commands:

    cd <TS_HOME>/src/com/sun/ts/tests/jaxws/api
    ant -Dkeywords=reverse runclient
    

platform-tck's Issues

ejb30/timer/schedule/lifecycle/Client.java#timerEquals failed when timer already expired

This test (com/sun/ts/tests/ejb30/timer/schedule/lifecycle/Client.java#timerEquals_from_ejbliteservlet) may fail when the timer already expired when the test tries to cancel it.

It failed because the test tries to cancel a timer that has already expired in this test run, which results in javax.ejb.NoSuchObjectLocalException. This is correct on the ejb container side.

The test should be updated to handle this exception by ignoring it since this is part of cleanup after the main test logic for this test.

The error when running with WildFly:

[javatest.batch] [TEST FAILED] testName=timerEquals 
[javatest.batch] 
[javatest.batch] Compare timer to itself. Got the expected result:[id=3c8fc544-fb7f-45f0-a435-dc8d128dcfd2 timedObjectId=ejb30_timer_schedule_lifecycle_ejbliteservlet_vehicle_web.ejb30_timer_schedule_lifecycle_ejbliteservlet_vehicle_web.ScheduleBean auto-timer?:false persistent?:true timerService=org.jboss.as.ejb3.timerservice.TimerServiceImpl@75e81342 initialExpiration=null intervalDuration(in milli sec)=0 nextExpiration=Wed Sep 11 23:31:45 EDT 2019 timerState=ACTIVE info=TimerInfo testName=timerEqualst1 testNumber=0 status=false doubleVar=0.0 longVar=0 charVar=  booleanVar=false intVar=0] ScheduleExpression [second=45;minute=31;hour=23;dayOfMonth=11;month=9;dayOfWeek=*;year=2019;timezoneID=null;start=null;end=null]	
[javatest.batch] 
[javatest.batch] Compare timer to t1Found. Got the expected result:[id=3c8fc544-fb7f-45f0-a435-dc8d128dcfd2 timedObjectId=ejb30_timer_schedule_lifecycle_ejbliteservlet_vehicle_web.ejb30_timer_schedule_lifecycle_ejbliteservlet_vehicle_web.ScheduleBean auto-timer?:false persistent?:true timerService=org.jboss.as.ejb3.timerservice.TimerServiceImpl@75e81342 initialExpiration=null intervalDuration(in milli sec)=0 nextExpiration=Wed Sep 11 23:31:45 EDT 2019 timerState=ACTIVE info=TimerInfo testName=timerEqualst1 testNumber=0 status=false doubleVar=0.0 longVar=0 charVar=  booleanVar=false intVar=0] ScheduleExpression [second=45;minute=31;hour=23;dayOfMonth=11;month=9;dayOfWeek=*;year=2019;timezoneID=null;start=null;end=null]	
[javatest.batch] Compare timer to null. Got the expected NotEquals result. compareTo:[id=3c8fc544-fb7f-45f0-a435-dc8d128dcfd2 timedObjectId=ejb30_timer_schedule_lifecycle_ejbliteservlet_vehicle_web.ejb30_timer_schedule_lifecycle_ejbliteservlet_vehicle_web.ScheduleBean auto-timer?:false persistent?:true timerService=org.jboss.as.ejb3.timerservice.TimerServiceImpl@75e81342 initialExpiration=null intervalDuration(in milli sec)=0 nextExpiration=Wed Sep 11 23:31:45 EDT 2019 timerState=ACTIVE info=TimerInfo testName=timerEqualst1 testNumber=0 status=false doubleVar=0.0 longVar=0 charVar=  booleanVar=false intVar=0] ScheduleExpression [second=45;minute=31;hour=23;dayOfMonth=11;month=9;dayOfWeek=*;year=2019;timezoneID=null;start=null;end=null], actual:null	
[javatest.batch] Compare timer to 1. Got the expected NotEquals result. compareTo:[id=3c8fc544-fb7f-45f0-a435-dc8d128dcfd2 timedObjectId=ejb30_timer_schedule_lifecycle_ejbliteservlet_vehicle_web.ejb30_timer_schedule_lifecycle_ejbliteservlet_vehicle_web.ScheduleBean auto-timer?:false persistent?:true timerService=org.jboss.as.ejb3.timerservice.TimerServiceImpl@75e81342 initialExpiration=null intervalDuration(in milli sec)=0 nextExpiration=Wed Sep 11 23:31:45 EDT 2019 timerState=ACTIVE info=TimerInfo testName=timerEqualst1 testNumber=0 status=false doubleVar=0.0 longVar=0 charVar=  booleanVar=false intVar=0] ScheduleExpression [second=45;minute=31;hour=23;dayOfMonth=11;month=9;dayOfWeek=*;year=2019;timezoneID=null;start=null;end=null], actual:1	
[javatest.batch] Compare timer to true. Got the expected NotEquals result. compareTo:[id=3c8fc544-fb7f-45f0-a435-dc8d128dcfd2 timedObjectId=ejb30_timer_schedule_lifecycle_ejbliteservlet_vehicle_web.ejb30_timer_schedule_lifecycle_ejbliteservlet_vehicle_web.ScheduleBean auto-timer?:false persistent?:true timerService=org.jboss.as.ejb3.timerservice.TimerServiceImpl@75e81342 initialExpiration=null intervalDuration(in milli sec)=0 nextExpiration=Wed Sep 11 23:31:45 EDT 2019 timerState=ACTIVE info=TimerInfo testName=timerEqualst1 testNumber=0 status=false doubleVar=0.0 longVar=0 charVar=  booleanVar=false intVar=0] ScheduleExpression [second=45;minute=31;hour=23;dayOfMonth=11;month=9;dayOfWeek=*;year=2019;timezoneID=null;start=null;end=null], actual:true	
[javatest.batch] Compare timer to TimerHandle. Got the expected NotEquals result. compareTo:[id=3c8fc544-fb7f-45f0-a435-dc8d128dcfd2 timedObjectId=ejb30_timer_schedule_lifecycle_ejbliteservlet_vehicle_web.ejb30_timer_schedule_lifecycle_ejbliteservlet_vehicle_web.ScheduleBean auto-timer?:false persistent?:true timerService=org.jboss.as.ejb3.timerservice.TimerServiceImpl@75e81342 initialExpiration=null intervalDuration(in milli sec)=0 nextExpiration=Wed Sep 11 23:31:45 EDT 2019 timerState=ACTIVE info=TimerInfo testName=timerEqualst1 testNumber=0 status=false doubleVar=0.0 longVar=0 charVar=  booleanVar=false intVar=0] ScheduleExpression [second=45;minute=31;hour=23;dayOfMonth=11;month=9;dayOfWeek=*;year=2019;timezoneID=null;start=null;end=null], actual:org.jboss.as.ejb3.timerservice.TimerHandleImpl@8dee6604	
[javatest.batch] Compare TimerHandle to timer. Got the expected NotEquals result. compareTo:org.jboss.as.ejb3.timerservice.TimerHandleImpl@8dee6604, actual:[id=3c8fc544-fb7f-45f0-a435-dc8d128dcfd2 timedObjectId=ejb30_timer_schedule_lifecycle_ejbliteservlet_vehicle_web.ejb30_timer_schedule_lifecycle_ejbliteservlet_vehicle_web.ScheduleBean auto-timer?:false persistent?:true timerService=org.jboss.as.ejb3.timerservice.TimerServiceImpl@75e81342 initialExpiration=null intervalDuration(in milli sec)=0 nextExpiration=Wed Sep 11 23:31:45 EDT 2019 timerState=ACTIVE info=TimerInfo testName=timerEqualst1 testNumber=0 status=false doubleVar=0.0 longVar=0 charVar=  booleanVar=false intVar=0] ScheduleExpression [second=45;minute=31;hour=23;dayOfMonth=11;month=9;dayOfWeek=*;year=2019;timezoneID=null;start=null;end=null]	
[javatest.batch] Compare 2 timers. Got the expected NotEquals result. compareTo:[id=3c8fc544-fb7f-45f0-a435-dc8d128dcfd2 timedObjectId=ejb30_timer_schedule_lifecycle_ejbliteservlet_vehicle_web.ejb30_timer_schedule_lifecycle_ejbliteservlet_vehicle_web.ScheduleBean auto-timer?:false persistent?:true timerService=org.jboss.as.ejb3.timerservice.TimerServiceImpl@75e81342 initialExpiration=null intervalDuration(in milli sec)=0 nextExpiration=Wed Sep 11 23:31:45 EDT 2019 timerState=ACTIVE info=TimerInfo testName=timerEqualst1 testNumber=0 status=false doubleVar=0.0 longVar=0 charVar=  booleanVar=false intVar=0] ScheduleExpression [second=45;minute=31;hour=23;dayOfMonth=11;month=9;dayOfWeek=*;year=2019;timezoneID=null;start=null;end=null], actual:[id=2a61a3bd-a8f7-432b-aa85-1b3f98d25b76 timedObjectId=ejb30_timer_schedule_lifecycle_ejbliteservlet_vehicle_web.ejb30_timer_schedule_lifecycle_ejbliteservlet_vehicle_web.ScheduleBean auto-timer?:false persistent?:true timerService=org.jboss.as.ejb3.timerservice.TimerServiceImpl@75e81342 initialExpiration=null intervalDuration(in milli sec)=0 nextExpiration=Sat Sep 11 23:31:44 EDT 2021 timerState=ACTIVE info=TimerInfo testName=timerEquals testNumber=0 status=false doubleVar=0.0 longVar=0 charVar=  booleanVar=false intVar=0] ScheduleExpression [second=44;minute=31;hour=23;dayOfMonth=11;month=9;dayOfWeek=*;year=2021;timezoneID=null;start=null;end=null]	
[javatest.batch] Compare timer 3 to timer 4. Got the expected NotEquals result. compareTo:[id=eb1e9bde-3bfe-4944-9e53-1ec9ca1305ed timedObjectId=ejb30_timer_schedule_lifecycle_ejbliteservlet_vehicle_web.ejb30_timer_schedule_lifecycle_ejbliteservlet_vehicle_web.ScheduleBean auto-timer?:false persistent?:true timerService=org.jboss.as.ejb3.timerservice.TimerServiceImpl@75e81342 initialExpiration=null intervalDuration(in milli sec)=0 nextExpiration=Thu Sep 12 00:00:00 EDT 2019 timerState=ACTIVE info=TimerInfo testName=timerEquals testNumber=0 status=false doubleVar=0.0 longVar=0 charVar=  booleanVar=false intVar=0] ScheduleExpression [second=0;minute=0;hour=0;dayOfMonth=*;month=*;dayOfWeek=*;year=*;timezoneID=null;start=null;end=null], actual:[id=739aae2c-8305-47fa-b66e-d24edeccf675 timedObjectId=ejb30_timer_schedule_lifecycle_ejbliteservlet_vehicle_web.ejb30_timer_schedule_lifecycle_ejbliteservlet_vehicle_web.ScheduleBean auto-timer?:false persistent?:true timerService=org.jboss.as.ejb3.timerservice.TimerServiceImpl@75e81342 initialExpiration=null intervalDuration(in milli sec)=0 nextExpiration=Thu Sep 12 00:00:00 EDT 2019 timerState=ACTIVE info=TimerInfo testName=timerEquals testNumber=0 status=false doubleVar=0.0 longVar=0 charVar=  booleanVar=false intVar=0] ScheduleExpression [second=0;minute=0;hour=0;dayOfMonth=*;month=*;dayOfWeek=*;year=*;timezoneID=null;start=null;end=null]	Failed with exception 
[javatest.batch] javax.ejb.NoSuchObjectLocalException: WFLYEJB0330: Timer has expired
[javatest.batch] 	at org.jboss.as.ejb3.timerservice.TimerImpl.assertTimerState(TimerImpl.java:476)
[javatest.batch] 	at org.jboss.as.ejb3.timerservice.TimerServiceImpl.cancelTimer(TimerServiceImpl.java:636)
[javatest.batch] 	at org.jboss.as.ejb3.timerservice.TimerImpl.cancel(TimerImpl.java:164)
[javatest.batch] 	at com.sun.ts.tests.ejb30.timer.common.TimerBeanBaseWithoutTimeOutMethod.cancelTimer(TimerBeanBaseWithoutTimeOutMethod.java:95)

WebSocket TCK async batchingAllowedOnServerTest fails if conainer supports batching

The full test name is:
com/sun/ts/tests/websocket/ee/javax/websocket/remoteendpoint/async/WSClient.java#batchingAllowedOnServerTest

Batching should be disabled by default (as per the Javadoc for RemoteEndpoint#getBatchingAllowed().

The test (on the server side) inverts the current batching allowed setting, sends a message, flushes batched messages and then sends 2 more messages. The client waits for three messages.

The problem is that, if batching is supported, the second and third messages are batched by the server, not received by the client and therefore the test fails.

I have a fix in mind for this and I'll provide a PR shortly.

EDP violation: Sign-Off missing for initial commit

According to the EDP all commits MUST contain a Sign-Off.

ejb32/lite/timer/schedule/expression/annotated/Client.java#dayOfMonthAndDayOfWeek failed when running the day before daylight time switch

The following 4 tests failed when running the day before daylight time switch (e.g., 2019-11-02):

com/sun/ts/tests/ejb32/lite/timer/schedule/expression/annotated/Client.java\#dayOfMonthAndDayOfWeek_from_ejblitejsf
com/sun/ts/tests/ejb32/lite/timer/schedule/expression/annotated/Client.java\#dayOfMonthAndDayOfWeek_from_ejblitejsp 
com/sun/ts/tests/ejb32/lite/timer/schedule/expression/annotated/Client.java\#dayOfMonthAndDayOfWeek_from_ejbliteservlet 
com/sun/ts/tests/ejb32/lite/timer/schedule/expression/annotated/Client.java\#dayOfMonthAndDayOfWeek_from_ejbliteservlet2 

The test creates a timer that expires this DAY_OF_WEEK and next DAY_OF_MONTH. When running this test the day before daylight time switch, the next day of month is +/- 1 hour, instead of the exact 24 hours. The test manually computes an expected remaining time as long expectedNext = 24 * 60 * 60 * 1000;, which is off by 1 hour when running on the day before daylight time switch.

See test class com.sun.ts.tests.ejb32.lite.timer.schedule.expression.annotated.ScheduleBean

The error messages:

Nov 03, 2019 12:01:26 AM com.sun.ts.tests.common.vehicle.ejbliteshare.EJBLiteWebVehicleRunner run
INFO: Connecting http://localhost:8080/ejb32_lite_timer_schedule_expression_annotated_ejblitejsf_vehicle_web/faces/ejblitejsf_vehicle.jsp?testName=dayOfMonthAndDayOfWeek
<html>
    <head><title>dayOfMonthAndDayOfWeek_from_ejblitejsf</title></head>
    <body>
        [TEST FAILED] testName=dayOfMonthAndDayOfWeek Created timer: TimerInfo testName=dayOfMonthAndDayOfWeek testNumber=0 status=false doubleVar=0.0 longVar=0 charVar=  booleanVar=false intVar=0	schedule:ScheduleExpression [second=27;minute=1;hour=0;dayOfMonth=4;month=*;dayOfWeek=7;year=*;timezoneID=null;start=null;end=null]	nextTimeout:Sun Nov 03 00:01:27 EDT 2019	timeRemaining:551	isPersistent:false
[Invoking Timeout method for timer with schedule: year=* month=* dayOfMonth=4 dayOfWeek=7 hour=0 minute=1 second=27 start=null end=null timezone=null, info: TimerInfo testName=dayOfMonthAndDayOfWeek testNumber=0 status=false doubleVar=0.0 longVar=0 charVar=  booleanVar=false intVar=0, and isPersistent: false, Expecting approximate timeRemaining 86400000, actual timeRemaining 89999998]


Check timeout status Expecting true, but actual falseFailed with exception 
java.lang.RuntimeException: Created timer: TimerInfo testName=dayOfMonthAndDayOfWeek testNumber=0 status=false doubleVar=0.0 longVar=0 charVar=  booleanVar=false intVar=0	schedule:ScheduleExpression [second=27;minute=1;hour=0;dayOfMonth=4;month=*;dayOfWeek=7;year=*;timezoneID=null;start=null;end=null]	nextTimeout:Sun Nov 03 00:01:27 EDT 2019	timeRemaining:551	isPersistent:false
[Invoking Timeout method for timer with schedule: year=* month=* dayOfMonth=4 dayOfWeek=7 hour=0 minute=1 second=27 start=null end=null timezone=null, info: TimerInfo testName=dayOfMonthAndDayOfWeek testNumber=0 status=false doubleVar=0.0 longVar=0 charVar=  booleanVar=false intVar=0, and isPersistent: false, Expecting approximate timeRemaining 86400000, actual timeRemaining 89999998]

Remove jwsdp dependency on jaxr test runs.

We use jwsdp-1.3 as a UDDI Registry Server which requires JavaSE1.4 to work.

We should be able to use any registry server. Need to experiment with jUDDI (http://juddi.apache.org/) or anything that works.

Skip JDK setup + use of cts-trunk-almgit.properties file if they do not exist

I tried using the docker/build_cts8.sh script locally and hacked past assumptions of the JDK location and file cts-trunk-almgit.properties. My changes are in https://github.com/scottmarlow/jakartaee-tck/tree/skipsome but aren't ready for a pull request as I don't know how to handle the absence of cts-trunk-almgit.properties. I'm creating this issue as a tracker, so others can see my branch, in case they try to do the same as I have.

I understand that all of this is still being worked on. After addressing the above issues, I then hit missing jars issue.

Servlet TCK java.endorsed.dirs not configured

A number of tests in com/sun/ts/tests/servlet/spec/serverpush depend on the flow.jar which is designed to make some Java 9 capability available when using Java 8. However, the configuration necessary to make this JAR available to the test harness is missing causes most of the classes in the above package to fail.

PR to address to to follow shortly.

ejb30/timer/schedule/expire/Client.java#dayOfWeekSunday2 failed when running near daylight saving time

When running the test (ejb30/timer/schedule/expire/Client.java#dayOfWeekSunday2_from_ejbliteservlet) one week before switching to or from daylight saving time, this test failed with the following error:

Oct 21, 2019 1:40:01 AM com.sun.ts.tests.common.vehicle.ejbliteshare.EJBLiteWebVehicleRunner run
INFO: Connecting http://localhost:8080/ejb30_timer_schedule_expire_ejbliteservlet_vehicle_web/ejbliteservlet_vehicle.jsp?testName=dayOfWeekSunday2
[TEST FAILED] testName=dayOfWeekSunday2 
Created a timer with expression year=* month=* dayOfMonth=* dayOfWeek=7 hour=1 minute=40 second=1 start=Mon Oct 28 01:40:01 EDT 2019 end=null timezone=null
Compare expected nextTimeout Sun Nov 03 01:40:01 EDT 2019, and actual nextTimeout Sun Nov 03 01:40:01 EST 2019

Expecting true, but actual falseRounded dates are not equal; next check if they are close.
Failed with exception 
java.lang.RuntimeException: The time diff 3600000 > 60000
	at com.sun.ts.tests.ejb30.timer.schedule.expire.Client.verifyNextTimeout(Client.java:160)
	at com.sun.ts.tests.ejb30.timer.schedule.expire.Client.dayOfWeek0(Client.java:103)
	at com.sun.ts.tests.ejb30.timer.schedule.expire.Client.dayOfWeekSunday2(Client.java:587)

The test tries to verify the next timeout for a scheduled timer that fires every Sunday, by comparing the result of calling timer.getNextTimeout(), to a computed expected date. When computing the expected date, the test does not take into consideration the daylight saving time switch, hence the actual timeout date is +/- 1 hour compared to the expected date.

Revert, back out, or properly fix this expiry date Servlet TCK commit

Per the discussion on the TCK mailing list, there were some errant changes made to the Servlet TCK. These changes are preventing some app servers from certifying to Jakarta EE 8. These changes need to be reverted, backed out, or properly fixed.

5d63739#diff-fe27b4530bdef6baa5447c990d9f786f

Regenerate expiring certificate

The certificates in jakartaee-tck are set to expire soon
(Jan 2, 2020). The tests using this certificate will fail if not upgraded before that.

Below are the locations for the cts alias certificate in three files
clientcert.jks, clientcert.pks & cts_cert . (all are the same cert)
./install/jaspic/bin/certificates/
./install/jakartaee/bin/certificates/
./install/websocket/bin/certificates/
./install/securityapi/bin/certificates/
./install/jacc/bin/certificates/
./install/connector/bin/certificates/

Commands for deleting, generating and exporting the certificate:

keytool -delete -alias cts -keystore clientcert.jks -storepass changeit -keypass changeit
keytool -genkey -keyalg RSA -alias cts -keystore clientcert.jks -storepass changeit -keypass changeit
keytool -export -rfc -alias cts -file cts_cert -keystore clientcert.jks -storepass changeit -keypass changeit

Correct JAXRS references in SAAJ TCK userguide

config.inc + rebuild.inc + _config.yml refer to jaxrs.

I'm not sure of what the content should be exactly, but jaxrs settings probably don't belong in the SAAJ userguide. For example:

Set the jaxrs_impl_name property to the name of the
{TechnologyShortName}

Intermittent failure for jaxrs test sseStageCheckTest_from_standalone

The jaxrs test sseStageCheckTest_from_standalone fails intermittently. The issue need to be analysed further.

01-07-2019 22:22:38: ERROR: InboundEvent{name='null', id='null', comment=[no comments], data=CompletionStage has been done} does not contain expected some_ServiceUnavailableEndpoint_message
01-07-2019 22:22:38: ERROR: Test case throws exception: InboundEvent{name='null', id='null', comment=[no comments], data=CompletionStage has been done} does not contain expected some_ServiceUnavailableEndpoint_message
01-07-2019 22:22:38: ERROR: Exception at:
01-07-2019 22:22:38: ERROR: com.sun.ts.lib.harness.EETest$Fault: InboundEvent{name='null', id='null', comment=[no comments], data=CompletionStage has been done} does not contain expected some_ServiceUnavailableEndpoint_message
at com.sun.ts.tests.jaxrs.common.JAXRSCommonClient.fault(JAXRSCommonClient.java:767)
at com.sun.ts.tests.jaxrs.common.JAXRSCommonClient.assertTrue(JAXRSCommonClient.java:662)
at com.sun.ts.tests.jaxrs.jaxrs21.ee.sse.sseeventsink.JAXRSClient.sseStageCheckTest(JAXRSClient.java:318)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.ts.lib.harness.EETest.run(EETest.java:596)
at com.sun.ts.lib.harness.ServiceEETest.run(ServiceEETest.java:115)
at com.sun.ts.tests.common.vehicle.EmptyVehicleRunner.run(EmptyVehicleRunner.java:40)
at com.sun.ts.lib.harness.ServiceEETest.run(ServiceEETest.java:105)
at com.sun.ts.lib.harness.EETest.getPropsReady(EETest.java:486)
at com.sun.ts.lib.harness.ServiceEETest.run(ServiceEETest.java:209)
at com.sun.ts.lib.harness.EETest.run(EETest.java:285)
at com.sun.ts.tests.common.vehicle.VehicleClient.main(VehicleClient.java:38)

01-07-2019 22:22:38: [JAXRSCommonClient] Test cleanup OK
STATUS:Failed.Test case throws exception: InboundEvent{name='null', id='null', comment=[no comments], data=CompletionStage has been done} does not contain expected some_ServiceUnavailableEndpoint_message
result: Failed. Test case throws exception: InboundEvent{name='null', id='null', comment=[no comments], data=CompletionStage has been done} does not contain expected some_ServiceUnavailableEndpoint_message

test result: Failed. Test case throws exception: InboundEvent{name='null', id='null', comment=[no comments], data=CompletionStage has been done} does not contain expected some_ServiceUnavailableEndpoint_message

Change the references javaee, j2ee in bundles and other files generated as part of the build

Following items needs to be addressed as part of this bug.

• Some build scripts and targets still refer to javaee, j2ee. This needs to be changed to jakartaee.
• Some temporary directories created as part of the build procedure, uses javaee or j2ee. This needs to be modified as jakartaee.
• Generated bundles and the directories inside them needs to be changed from javaee/j2ee to jakartee.

No reason com.sun.ts.tests.jsonb.defaultmapping.collections.CollectionsMappingTest#testEnumMap fails deserialization

com.sun.ts.tests.jsonb.defaultmapping.collections.CollectionsMappingTest#testEnumMap expects EnumMap can't be unmarshalled, there is no technical reason for that, was even in the spec: https://github.com/javaee/jsonb-spec/blob/dc81f781f490e26ca545f98c50a7319551f724e2/examples/runtime/src/main/java/examples/mapping/DefaultMapping.java#L460

edit: same for EnumSet test

Remove timestamp from zip file names

The timestamp or "latest" should be removed from the zip file names.
The version number should remain, and should include a micro version number
to allow for future micro version release to (e.g.) update the exclude list.
The major and minor version numbers need to match the corresponding spec.

So, the name should be of the form tck-.zip, e.g.,
connectortck-1.7.0.zip.

Several JTA tests do not check the status of the transaction before tryng to roll it back creating lots of stack traces that are noisy

It seems that a lot of the cleanup phase in the JTA tests throw an exception which doesn't fail the TCK but is noisy in the log. For instance from https://jenkins.eclipse.org/jta/job/TCK_run_pipeline/22/artifact/jtatck/bin/run.log

This is part of the output of test https://github.com/eclipse-ee4j/jakartaee-tck/blob/master/src/com/sun/ts/tests/jta/ee/usertransaction/commit/UserCommitClient.java#L103
[javatest.batch] 07-04-2019 11:20:00: SVR: IllegalStateException was caught as Expected !!
[javatest.batch] 07-04-2019 11:20:00: SVR-ERROR: Cleanup Failed
[javatest.batch] 07-04-2019 11:20:00: SVR-ERROR: com.sun.ts.lib.harness.EETest$Fault
[javatest.batch] at com.sun.ts.tests.jta.ee.usertransaction.commit.UserCommitClient.cleanup(UserCommitClient.java:387)
///
[javatest.batch] 07-04-2019 11:20:00: SVR-TRACE: Could not clean the environment
[javatest.batch] 07-04-2019 11:20:00: SVR: Test running in jsp vehicle passed

Here is the line doing the rollback which I would expect to fail as there is no transaction: https://github.com/eclipse-ee4j/jakartaee-tck/blob/master/src/com/sun/ts/tests/jta/ee/usertransaction/commit/UserCommitClient.java#L387 - it could be guarded with something like an "if transaction is active" check.

JAXRPC tests in standalone TCKS fail with Stub not found error.

Many JAX-RPC related tests fail in standalone TCK. The build process for JAXRPC needs to be validated to check if the stubs were generated properly.

More test failure Logs can be found here:
https://jenkins.eclipse.org/jakartaee-tck/job/jakartaee-tck/job/master/262/#showFailuresLink

01-05-2019 19:44:09: ERROR: [failed to localize] service.implementation.not.found(com.sun.ts.tests.jaxrpc.ee.j2w.marshalltest.MarshallTestService_Impl)
at com.sun.xml.rpc.client.ServiceFactoryImpl.createService(ServiceFactoryImpl.java:124)
at com.sun.xml.rpc.client.ServiceFactoryImpl.loadService(ServiceFactoryImpl.java:145)
at com.sun.ts.tests.jaxrpc.common.JAXRPC_Util.getStub(JAXRPC_Util.java:164)
at com.sun.ts.tests.jaxrpc.ee.j2w.marshalltest.Client.getStubStandalone(Client.java:104)
at com.sun.ts.tests.jaxrpc.ee.j2w.marshalltest.Client.setup(Client.java:163)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.ts.lib.harness.EETest.run(EETest.java:569)
at com.sun.ts.lib.harness.ServiceEETest.run(ServiceEETest.java:115)
at com.sun.ts.tests.common.vehicle.EmptyVehicleRunner.run(EmptyVehicleRunner.java:40)
at com.sun.ts.lib.harness.ServiceEETest.run(ServiceEETest.java:105)
at com.sun.ts.lib.harness.EETest.getPropsReady(EETest.java:486)
at com.sun.ts.lib.harness.ServiceEETest.run(ServiceEETest.java:209)
at com.sun.ts.lib.harness.EETest.run(EETest.java:285)
at com.sun.ts.tests.common.vehicle.VehicleClient.main(VehicleClient.java:38)

01-05-2019 19:44:09: ERROR: Test case throws exception: com.sun.ts.lib.harness.EETest$Fault: setup failed:
01-05-2019 19:44:09: ERROR: Exception at:
01-05-2019 19:44:09: ERROR: [failed to localize] service.implementation.not.found(com.sun.ts.tests.jaxrpc.ee.j2w.marshalltest.MarshallTestService_Impl)
at com.sun.xml.rpc.client.ServiceFactoryImpl.createService(ServiceFactoryImpl.java:124)
at com.sun.xml.rpc.client.ServiceFactoryImpl.loadService(ServiceFactoryImpl.java:145)
at com.sun.ts.tests.jaxrpc.common.JAXRPC_Util.getStub(JAXRPC_Util.java:164)
at com.sun.ts.tests.jaxrpc.ee.j2w.marshalltest.Client.getStubStandalone(Client.java:104)
at com.sun.ts.tests.jaxrpc.ee.j2w.marshalltest.Client.setup(Client.java:163)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.ts.lib.harness.EETest.run(EETest.java:569)
at com.sun.ts.lib.harness.ServiceEETest.run(ServiceEETest.java:115)
at com.sun.ts.tests.common.vehicle.EmptyVehicleRunner.run(EmptyVehicleRunner.java:40)
at com.sun.ts.lib.harness.ServiceEETest.run(ServiceEETest.java:105)
at com.sun.ts.lib.harness.EETest.getPropsReady(EETest.java:486)
at com.sun.ts.lib.harness.ServiceEETest.run(ServiceEETest.java:209)
at com.sun.ts.lib.harness.EETest.run(EETest.java:285)
at com.sun.ts.tests.common.vehicle.VehicleClient.main(VehicleClient.java:38)

cts-base Docker image port 8080 collides with GlassFish

Following the instructions on the Wiki, the cts-base Docker image opens a server on port 8080. That port collides with GlassFish when it's being tested.

The UUID registry should ideally expose a different port.

The ejb30/lite tests failing at the first time

"Some of the ejb30 tests in jakarataee-tck, especially the ones in ejb30/lite fail with different count at the first time. The failures occur when they are run in a single shot due to deployment failures (possibly caused by OOM) and hence the failed ones are rerun once.

Even in multiple attempts we might not be able to get them all pass in one shot. We had also tried splitting the suite to the maximum possible & increased the jvm memory for glassfish (at this line : https://github.com/eclipse-ee4j/jakartaee-tck/blob/c12222e3acd5b4c3e7981045698174c579592f16/docker/run_jakartaeetck.sh#L240 ). "

rerun logic is here : https://github.com/eclipse-ee4j/jakartaee-tck/blob/4687f0a79e3ce4f25b8505a7281b603de6455ae9/docker/run_jakartaeetck.sh#L457

WebSocket TCK basic batchingAllowed* tests fail if container supports batching

Full path of failing tests:

• com/sun/ts/tests/websocket/ee/javax/websocket/remoteendpoint/basic/WSClient.java#batchingAllowedOnClientTest
• com/sun/ts/tests/websocket/ee/javax/websocket/remoteendpoint/basic/WSClient.java#batchingAllowedOnServerTest

The tests fail because after enabling batching, the batched messages are not flushed so the client never receives them causing the test to fail.

PR to follow.

Change to run jaxr signature tests and functional tests together

jaxr signature test is now run by a separate ant task unlike other TCKs line : https://github.com/eclipse-ee4j/jakartaee-tck/blob/4687f0a79e3ce4f25b8505a7281b603de6455ae9/docker/jaxrtck.sh#L87 .

We need to make changes to jaxr run.all target to include signature test too.

servlet_sec_secform_web.xml contains a wrong url-pattern

This is preventing starting the webapp.
https://github.com/eclipse-ee4j/jakartaee-tck/blob/master/src/com/sun/ts/tests/servlet/spec/security/secform/servlet_sec_secform_web.xml#L348

  <servlet-mapping>
    <servlet-name>OneTestServletLogicalName</servlet-name>
    <url-pattern>/On*</url-pattern>
  </servlet-mapping>

/On* is not correct regarding to specs.

platform mode value javaEE to be replaced by jakartaEE

platform mode value "javaEE" needs to be replaced by "jakartaEE". The userguides/docs has to be updated too.

ejb32/lite/timer/schedule/expression/annotated/Client.java\#dayOfWeekOverDayOfMonth failed at midnight

This issue affects the following tests:

com/sun/ts/tests/ejb32/lite/timer/schedule/expression/annotated/Client.java\#dayOfWeekOverDayOfMonth_from_ejbliteservlet2

com/sun/ts/tests/ejb32/lite/timer/schedule/expression/annotated/Client.java\#dayOfMonthOverDayOfWeek_from_ejbliteservlet2

com/sun/ts/tests/ejb32/lite/timer/schedule/expression/annotated/Client.java\#dayOfMonthOverDayOfWeek_from_ejbliteservlet 

com/sun/ts/tests/ejb32/lite/timer/schedule/expression/annotated/Client.java\#dayOfMonthOverDayOfWeek_from_ejblitejsf

com/sun/ts/tests/ejb32/lite/timer/schedule/expression/annotated/Client.java\#dayOfMonthOverDayOfWeek_from_ejblitejsp

com/sun/ts/tests/ejb32/lite/timer/schedule/expression/annotated/Client.java\#dayOfWeekOverDayOfMonth_from_ejblitejsf 

com/sun/ts/tests/ejb32/lite/timer/schedule/expression/annotated/Client.java\#dayOfWeekOverDayOfMonth_from_ejblitejsp 

similar tests under com/sun/ts/tests/ejb30/timer/...

It failed because the test ran at mid-night (11:59:59 PM). The test created a timer to fire the next day, and expect no timer is fired. When the timer is created at the end of day 1, but it was already day 2 when checking test results.

Snippet from test log:

Sep 03, 2019 11:59:59 PM com.sun.ts.tests.common.vehicle.ejbliteshare.EJBLiteWebVehicleRunner run

[TEST FAILED] testName=dayOfWeekOverDayOfMonth Created timer: TimerInfo
testName=dayOfWeekOverDayOfMonth testNumber=0 status=false doubleVar=0.0 longVar=0 charVar=  booleanVar=false intVar=0 schedule:ScheduleExpression [second=*;minute=*;hour=*;dayOfMonth=*;month=*;dayOfWeek=3;year=*;timezoneID=null;start=null;end=null] nextTimeout:Wed Sep 04 00:00:00 EDT 2019 timeRemaining:130 isPersistent:false

Mar 12, 2020 11:59:45 PM com.sun.ts.tests.common.vehicle.ejbliteshare.EJBLiteWebVehicleRunner run
INFO: Connecting http://localhost:8080/ejb32_lite_timer_schedule_expression_annotated_ejblitejsf_vehicle_web/faces/ejblitejsf_vehicle.jsp?testName=dayOfWeekOverDayOfMonth
<html>
    <head><title>dayOfWeekOverDayOfMonth_from_ejblitejsf</title></head>
    <body>
        [TEST FAILED] testName=dayOfWeekOverDayOfMonth Created timer: TimerInfo testName=dayOfWeekOverDayOfMonth testNumber=0 status=false doubleVar=0.0 longVar=0 charVar=  booleanVar=false intVar=0  schedule:ScheduleExpression [second=*;minute=*;hour=*;dayOfMonth=*;month=*;dayOfWeek=5;year=*;timezoneID=null;start=null;end=null]      nextTimeout:Fri Mar 13 00:00:00 EDT 2020        timeRemaining:14743     isPersistent:false

Timeout status must not be set. Expecting null, but actual trueFailed with exception
java.lang.RuntimeException: Created timer: TimerInfo testName=dayOfWeekOverDayOfMonth testNumber=0 status=false doubleVar=0.0 longVar=0 charVar=  booleanVar=false intVar=0     schedule:ScheduleExpression [second=*;minute=*;hour=*;dayOfMonth=*;month=*;dayOfWeek=5;year=*;timezoneID=null;start=null;end=null]      nextTimeout:Fri Mar 13 00:00:00 EDT 2020        timeRemaining:14743     isPersistent:false

Timeout status must not be set. Expecting null, but actual true

These tests currently adds 1 to the day field to ensure this timer does not fire during the test run. They should be updated to add more buffer time (e.g., by adding 2 days).

Servlet TCK Cookie setMaxAgePositiveTests assumes server uses en_US Locale

The full name of the tests in question are:

• com/sun/ts/tests/servlet/api/javax_servlet_http/cookie/URLClient.java#setMaxAgePositiveTest
• com/sun/ts/tests/servlet/pluggability/api/javax_servlet_http/cookie/URLClient.java#setMaxAgePositiveTest

These test use a custom header to pass a date from the server to the client. The TCK client (at least the GUI anyway) is hard-coded to use the en_US Locale. This causes the above tests to fail if the server is running using a Local that formats dates differently - such as en_GB. The switching of the day and month triggers a test failure.

I have a proposed fix for this. PR to follow shortly.

Jakarta EE 8 challenge, permissions.xml must also be added to EAR level

To correctly resolve a failure in com.sun.ts.tests.securityapi.ham.sam.obtainbean.Client#testSAMObtainBean, the securityapi/ham/sam/obtainbean/build.xml should be changed to include the permissions.xml in the EAR (should also keep permissions.xml at the WAR level).

This is wrong in Jakarta platform TCK versions 8.0 - 8.0.1, this can be seen with a change that we are testing for (Red Hat) WildFly, where we need this test to be corrected (e.g. likely via a minor build.xml change).

The securityapi/idstore/idstorepermission/build.xml should also be changed to include the permissions.xml in the EAR (should also keep permissions.xml at the WAR level).

As per Jakarta EE 8 platform specification ApplicationProgrammingInterface (as well as section EE.6.2.2.6 of Java EE 8 platform spec), permissions must be declared at EAR level:

For applications packaged in an .ear file, the declaration of permissions must be at .ear file level. This permission set is applied to all modules and libraries packaged within the .ear file or within its contained modules. Any permissions.xml files within such packaged modules are ignored, regardless of whether a permissions.xml file has been supplied for the .ear file itself.

An example of the EAR correctly including the permissions.xml can be found in securityapi/idstore/customhandler/build.xml

Starting the JWSDP 1.3 server fails for jaxr standalone tck run

This issue is duplicated to track the bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=541542 .

Using the same information from the bug:

[Anand Francis Joseph] 2018-11-25 23:19:02 EST

As part of JAXR suite, we start the JWSDP 1.3 server, It was working fine earlier in the Jakarta EE TCK CJE environment. Now I suddenly see this error

    '[' jaxr == jaxr ']'
    '[' -f /opt/jwsdp-1.3/bin/startup.sh ']'
    /opt/jwsdp-1.3/bin/startup.sh
    Error: no client' JVM at /opt/j2sdk1.4.2_41/jre/lib/i386/client/libjvm.so'.

Was there any change in the CJE instance or its configuration recently, that is causing this error ?

The same Docker image runs fine in my environment, So I don't think its an issue with the docker image. If its due to some restrictions in the OpenShift instance, please suggest the required changes to be done in the Docker image.

The jvm.cfg (at /opt/j2sdk1.4.2_41/jre/lib/i386/jvm.cfg) contains the following entries.

-client KNOWN
-server KNOWN
-hotspot ALIASED_TO -client
-classic WARN
-native ERROR
-green ERROR

[Anand Francis Joseph] 2018-11-26 05:26:36 EST

The issue does not happen always. I reran the job and there this issue was not seen and the JWSDP server was started without any issue.
https://jenkins.eclipse.org/jakartaee-tck/blue/organizations/jenkins/jakartaee-tck/detail/master/116/pipeline/36/

Solution suggested by Frederic Gurr:
[Frederic Gurr] 2018-11-26 12:32:23 EST

This issue seems to be related:
=> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=623419
Possible fix is provided here:
=> https://www.mail-archive.com/[email protected]/msg29226.html
Please try the described fix and report back if it worked.

Create new branch for Jakartaee8

The existing EE4J_8 branch is currently stale and is not used for any developments or job execution from jenkins CI.

The latest changes for jakartaee8 are all in master branch. We need to create a separate branch for jakartaee8 release or use the current EE4J_8 by merging all the new changes from master.

Run ant gui

Hello,

When I execute ant gui in the bin folder, it opens the UI but freeze. In the console, it stops at

filter.if.vi.is.glassfish:
[javatest.gui] ************************************************************
[javatest.gui] * props file set to "/var/folders/xm/slj52q7x6ms98kqysv835g080000gn/T/dev-cts-props.txt"
[javatest.gui] ************************************************************
[javatest.gui] 01-17-2019 09:32:49:  Harness - Use BinaryTestFinder...

Any help is appreciated
Regards.
Gurkan

New Project Names

Revise project names, per https://docs.google.com/spreadsheets/d/18SraPxRBCOyaS6w-UV6TR-UA1bWy1--sV0ky6msAjWY/edit#gid=0
Also apply simple substitutions as noticed.

JSP TCK Signature tests fail in standalone mode

There appear to be two issues.

1. sig-test.map defines javax.servlet.jsp=2.3_se6 whereas it should define javax.servlet.jsp=2.3_se8
2. src/com/sun/ts/tests/signaturetest/siganture-repository does not exist. It should exist and contain javax.servlet.jsp.sig_2.3_se8

The first issue I can fix and can provide a PR. The second I could do with a pointer on where to look / how to get started.

Servlet TCK denyUncovered tests depend on container specific behaviour

The tests in question are:

• src/com/sun/ts/tests/servlet/spec/security/denyUncovered/* (5 tests)

These tests are packaged in servlet_sec_denyUncovered_web.war which usually means a context path of /servlet_sec_denyUncovered_web.
However, these tests expect a contenxt path of /servlet_sec_denyUncovered. The TCK uses a container specific configuration file (https://github.com/eclipse-ee4j/jakartaee-tck/blob/master/src/com/sun/ts/tests/servlet/spec/security/denyUncovered/servlet_sec_denyUncovered_web.war.sun-web.xml) to ensure that these tests pass with Glassfish. This work-around does not work with other containers.

Possible solutions include:

• update the tests to use the actual context path
• align the name of the WAR file with the expected context path

I've no strong preference.
I'm happy to try and produce a PR for this once there is consensus on the desired approach.

Create revised CTS bundle to pick up changes in JSON B test challenge

Please build new version of Jakarta EE 8 CTS release that includes changes required by probable accepted test challenge jsonb-api#180. As a working presumption, this will be Jakarta EE CTS 8.0.1.

build of JAXRPC TCK fails due to ant-props.jar reference

build_standalone-tcks.sh assumes that $BASEDIR/lib/ant-props.jar exists but it doesn't in my build, which results in the below error:

'[' jaxrpc == jaxrpc ']'
echo 'Generating JAXRPC specific classes using wscompile'
Generating JAXRPC specific classes using wscompile
cp /mnt/hudson_workspace/workspace/jakartaee-ee8/buildtck/lib/ant-props.jar /mnt/hudson_workspace/workspace/jakartaee-ee8/buildtck/apache-ant-1.10.1/lib
cp: cannot stat ‘/mnt/hudson_workspace/workspace/jakartaee-ee8/buildtck/lib/ant-props.jar’: No such file or directory

Should it be a requirement that $BASEDIR/lib/ant-props.jar exists before starting 'build_standalone-tcks.sh jaxrpc' or should ant-props be downloaded by the build_standalone-tcks.sh script?

Servlet TCK serverpushInitTest uses case sensitive test for HTTP headers

HTTP header names are case insensitive. This test (full name below) performs case sensitive tests which will fail if the container uses, for example, all lower case HTTP header names.

com/sun/ts/tests/servlet/spec/serverpush/Client.java#serverPushInitTest

PR to follow shortly.

Package of CDI TCK?

There's no issue tracker for https://github.com/eclipse-ee4j/cdi-tck, not sure, if by accident or on purpose, but there were other issues raised here about specific TCKs.

The CDI TCK currently uses the package org.jboss, will this eventually change to org.eclipse, at least after Jakarta EE 8?

Jsonb assertions for gregorian calendar does not check for value

Issue is detailled at jakartaee/jsonb-api#177 (comment)

A jsonb tck uses equals between gregorian calendars making the value assertions not validated properly - it validates the way/method the calendar is built which is not required by the spec since it is opaque to the user.

Remove jtreportparser.jar from the source docker/JTReportParser

We are currently including the JTReportParser jar in the repo and use that to generate the reports. Avoid this and as Anand&Bill suggested it should be downloaded from a central repo.

From the email -

Anand: The source code for JTReportParser.jar is available at https://github.com/eclipse-ee4j/jakartaee-tck-tools/tree/master/tools/jtreport-parser. Currently its just a set of java source files with a basic pom.xml file. There can be a build job which builds this and publishes it to maven repository. Should it be placed in a separate github project of its own or can we live with the existing structure inside jakarataee-tck-tools repository?

Bill: It's fine in the tools repository. I just don't want the binary to be duplicated in lots of other repositories for TCK jobs. The TCK jobs that need to use it should download the binary (and dependencies) from (e.g.) Maven.

Analyze failures in ejb30/lite/tx, ejb30/lite/packaging in Jenkins Pipeline

Test failures in webservices12/specialcases

The JakartaEE TCK run has 9 failures in from the suite webservices12/specialcases/clients/j2w/doclit, as the test fails to find the archives to deploy.

These tests have prebuilt endpoints, where the clients are built after deploying the endpoints to the serve. The test validates that importing a WSDL from a predeployed webservice endpoint works properly.

Correct the date time data printed in bundles info file from CI certification jobs

The below jobs get the datetime of the promoted bundles to be output into a info file.
-https://ci.eclipse.org/jakartaee-tck/job/eftl-standalonetck-certification/
-https://ci.eclipse.org/jakartaee-tck/job/eftl-standalonetck-certification-web/
-https://ci.eclipse.org/jakartaee-tck/job/standalonetck-certification/
-https://ci.eclipse.org/jakartaee-tck/job/standalonetck-certification-web/

The command used is echo '*** Date: 'ssh [email protected] stat -c \"%y\" ${NAME} /home/data/httpd/${BUNDLE_URL}" ***">> $WORKSPACE/tckinfo.txt

This command does not seem to be reliable and need to be corrected where it is output into *info.txt files. For eg: https://ci.eclipse.org/jakartaee-tck/job/standalonetck-certification/lastSuccessfulBuild/artifact/tckinfo.txt has xml-rpc-tck-1.1.0.zip bundle dated 2019-08-08 but in real it was generated at a later time.

This issue is to record that this date time is not to be depended on now but the checksum values are correctly displayed.

Other option is to download the bundle and do 'stat' on that.

Servlet TCK secbasic#test7[_anno] depends on container specific behaviour

The two tests in question are:

• src/com/sun/ts/tests/servlet/spec/security/secbasic/Client.java#test7
• src/com/sun/ts/tests/servlet/spec/security/secbasic/Client.java#test7_anno

The tests are identical apart from the second tests uses annotations rather than web.xml for configuration.
The tests use a servlet mapped to /ServletSecTest. Note this is an exact mapping, not a path mapping.
There is a security constraint also mapped to /ServletSecTest
The client requests /ServletSecTest/j_security_check without any credentials
The tests expects a 401.
The reasoning is explained at https://github.com/eclipse-ee4j/jakartaee-tck/blob/master/src/com/sun/ts/tests/common/jspservletsec/SecBasicClient.java#L340

I do not think that the specification supports this behaviour.

I think the specification requires the following:
Since exact mapping is used, the request does not match the servlet mapping.
This means the request is passed to default servlet since it has a mapping of /*.
The servletPath will be the empty string and the pathInfo will be /ServletSecTest/j_security_check.
This doesn't match any security constraints so the default servlet will try to server this resource.
The resource does not exist so a 404 is returned.

The test could be 'fixed' by making the servlet mapping /ServletSecTest/* but that does not address the issue of the underlying premise of the test being incorrect. Changing the expected output (and the explanatory comment) to expect 404 is one possible solution. Simply removing/disabling the tests is another.

I'm happy to try and produce a PR for this once there is consensus on how it should be handled.

Missing libraries

There are some missing JAR libraries that need to be on the /lib folder. From what I see on https://github.com/eclipse-ee4j/jakartaee-tck/blob/febd0176d513a2b4d9b13354bf69b6d6c03d79a3/release/tools/common.xml#L78 they are:

• ant_sun.jar
• ant-contrib-1.0b3.jar
• javatest.jar
• tsharness.jar
• apiCheck.jar
• sigtest.jar
• tspackager.jar
• commons-httpclient.jar (3.1 implied from NOTICE)
• commons-logging.jar (1.1.1 implied from NOTICE)
• jdom-1.1.3.jar
• jaxen-core-1.0.jar
• jaxen-1.1.6.jar
• saxpath.jar (1.0 implied from NOTICE)

Some of them are easy to find but others don't give much clues. Particularly the ant_sun.jar, that should have a class com.sun.ant.taskdefs.common.PathTranslator that I can't find on Google.

The code doesn't have any Ant task or so to download the needed dependencies.

Second-later calendar-based timer can be too short for some tests

Some cts tests create a second-later calendar-based timer, and expect it to fire shortly. For example, in com.sun.ts.tests.ejb30.timer.common.TimerBeanBaseWithoutTimeOutMethod class:

    public Timer createSecondLaterTimer(String name) {
        return TimerUtil.createSecondLaterTimer(timerService, name);
    }

    public Timer createSecondLaterTimer(TimerConfig timerConfig) {
        return TimerUtil.createSecondLaterTimer(timerService, timerConfig);
    }

And in TimerUtil class:

    public static Timer createSecondLaterTimer(TimerService timerService,
            TimerInfo info) {
        Calendar cal = Calendar.getInstance();
        cal.add(Calendar.SECOND, 1);  //next second
        return timerService.createCalendarTimer(getPreciseScheduleExpression(cal),
                new TimerConfig(info, true));
    }

Timers created this way only have precision up to second. If such a ScheduleExpression is created towards the end of a second, then the timer is set to expire in just a few milliseconds. When the ejb container creates the new timer, the ScheduleExpression passed to the timer create method may already represent a time in the past. So the test timer will not fire and the test will not get the expected result.

For example,

calling createSecondLaterTimer(...) at: 11:06:37,990
creating timer to expire at: 11:06:38,000
ejb container create the timer at: 11:06:38,200

This has occassionally caused test failures when running with WildFly:

11:06:38,936 INFO [org.jboss.as.ejb3.timer] (default task-1) WFLYEJB0017: Next expiration is null. No tasks will be scheduled for timer [ID=880F2CEC-333D-4D08-B152-376EE69A0428 TIMEDOBJECTID=EJB32_LITE_TIMER_SCHEDULE_TX_EJBLITESERVLET_VEHICLE_WEB.EJB32_LITE_TIMER_SCHEDULE_TX_EJBLITESERVLET_VEHICLE_WEB.SCHEDULEBMTBEAN AUTO-TIMER?:FALSE PERSISTENT?:FALSE TIMERSERVICE=ORG.JBOSS.AS.EJB3.TIMERSERVICE.TIMERSERVICEIMPL@340D51E0 INITIALEXPIRATION=NULL INTERVALDURATION(IN MILLI SEC)=0 NEXTEXPIRATION=NULL TIMERSTATE=ACTIVE INFO=TIMERINFO TESTNAME=TIMEOUTSYSTEMEXCEPTIONBMT TESTNUMBER=0 STATUS=FALSE DOUBLEVAR=0.0 LONGVAR=0 CHARVAR= BOOLEANVAR=FALSE INTVAR=0] SCHEDULEEXPRESSION [SECOND=38;MINUTE=6;HOUR=11;DAYOFMONTH=13;MONTH=11;DAYOFWEEK=*;YEAR=2019;TIMEZONEID=NULL;START=NULL;END=NULL]

Any tests that use the above testing mechanism suffer from this problem. We've so far noticed failures in the following tests:

com/sun/ts/tests/ejb30/timer/interceptor/aroundtimeout/singleton/annotated/Client.java#aroundTimeoutMethod_from_ejbliteservlet

com/sun/ts/tests/ejb30/timer/interceptor/aroundtimeout/singleton/annotated/Client.java\#aroundTimeoutMethod2_from_ejbliteservlet 

com/sun/ts/tests/ejb30/timer/schedule/txnonpersistent/Client.java\#createRollbackTxPropagationBMT_from_ejbliteservlet

com/sun/ts/tests/ejb30/timer/schedule/descriptor/stateless/Client.java\#programmatic_from_ejbliteservlet 

com/sun/ts/tests/ejb32/lite/timer/schedule/txnonpersistent/Client.java\#createRollbackTxPropagationBMT_from_ejbliteservlet 

com/sun/ts/tests/ejb32/lite/timer/schedule/tx/Client.java\#timeoutSystemExceptionBMT_from_ejbliteservlet

com/sun/ts/tests/ejb32/lite/timer/schedule/tx/Client.java\#timeoutSystemException_from_ejblitejsf 

com/sun/ts/tests/ejb32/lite/timer/schedule/txnonpersistent/Client.java\#timeoutSystemExceptionBMT_from_ejblitejsf 

com/sun/ts/tests/ejb32/lite/timer/schedule/txnonpersistent/Client.java\#timeoutSystemException_from_ejbliteservlet 

com/sun/ts/tests/ejb32/lite/timer/interceptor/aroundtimeout/singleton/annotated/Client.java\#allInterceptors_from_ejblitejsf

com/sun/ts/tests/ejb32/lite/timer/interceptor/aroundtimeout/singleton/annotated/Client.java\#allInterceptorsOverride_from_ejbliteservlet 

com/sun/ts/tests/ejb32/lite/timer/interceptor/aroundtimeout/singleton/annotated/Client.java\#allInterceptorsOverride_from_ejblitejsp 

com/sun/ts/tests/ejb32/lite/timer/interceptor/aroundtimeout/singleton/annotated/Client.java\#allInterceptorsComplement_from_ejblitejsp

com/sun/ts/tests/ejb32/lite/timer/interceptor/aroundtimeout/singleton/annotated/Client.java\#allInterceptorsComplement_from_ejblitejsf

com/sun/ts/tests/ejb32/lite/timer/schedule/expire/Client.java\#timerAccessInTimeoutMethod_from_ejblitejsf 

com/sun/ts/tests/ejb32/lite/timer/schedule/descriptor/stateless/Client.java\#programmatic_from_ejbliteservlet 

The test client side errors may vary, but the root cause is that the test client did not get the expected result because the test timer did not expire.

The test error message for timeoutSystemExceptionBMT test:

[javatest.batch] INFO: Connecting http://localhost:8080/ejb32_lite_timer_schedule_tx_ejbliteservlet_vehicle_web/ejbliteservlet_vehicle.jsp?testName=timeoutSystemExceptionBMT
[javatest.batch] [TEST FAILED] testName=timeoutSystemExceptionBMT If the transaction rolls back in timeout method, must retry at least once.
[javatest.batch] timeout callback result: 
[javatest.batch] []
[javatest.batch] 
[javatest.batch] Expecting 0>1, but failedFailed with exception 
[javatest.batch] java.lang.RuntimeException: If the transaction rolls back in timeout method, must retry at least once.
[javatest.batch] timeout callback result: 
[javatest.batch] []
[javatest.batch] 
[javatest.batch] Expecting 0>1, but failed
[javatest.batch] 	at com.sun.ts.tests.ejb30.common.helper.Helper.assertGreaterThan(Helper.java:155)
[javatest.batch] 	at com.sun.ts.tests.ejb30.common.lite.EJBLiteClientBase.assertGreaterThan(EJBLiteClientBase.java:394)
[javatest.batch] 	at com.sun.ts.tests.ejb32.lite.timer.schedule.tx.ClientBase.timeoutRollback(ClientBase.java:239)
[javatest.batch] 	at com.sun.ts.tests.ejb32.lite.timer.schedule.tx.ClientBase.timeoutSystemExceptionBMT(ClientBase.java:192)

The test error message for createRollbackTxPropagationBMT test:

[javatest.batch] INFO: Connecting http://localhost:8080/ejb30_timer_schedule_txnonpersistent_ejbliteservlet_vehicle_web/ejbliteservlet_vehicle.jsp?testName=createRollbackTxPropagationBMT
[javatest.batch] [TEST FAILED] testName=createRollbackTxPropagationBMT []
[javatest.batch] 
[javatest.batch] 
[javatest.batch] Check timeout status Expecting true, but actual nullFailed with exception 
[javatest.batch] java.lang.RuntimeException: []
[javatest.batch] 
[javatest.batch] 
[javatest.batch] Check timeout status Expecting true, but actual null
[javatest.batch] 	at com.sun.ts.tests.ejb30.common.helper.Helper.assertEquals(Helper.java:89)
[javatest.batch] 	at com.sun.ts.tests.ejb30.common.lite.EJBLiteClientBase.assertEquals(EJBLiteClientBase.java:384)
[javatest.batch] 	at com.sun.ts.tests.ejb30.timer.common.ClientBase.passIfTimeout(ClientBase.java:91)
[javatest.batch] 	at com.sun.ts.tests.ejb30.timer.common.ClientBase.passIfTimeout(ClientBase.java:79)
[javatest.batch] 	at com.sun.ts.tests.ejb30.timer.schedule.tx.ClientBase.createRollbackTxPropagationBMT(ClientBase.java:101)

The test error for allInterceptorsOverride test:

Feb 05, 2020 12:38:33 AM com.sun.ts.tests.common.vehicle.ejbliteshare.EJBLiteWebVehicleRunner run
INFO: Connecting http://localhost:8080/ejb32_lite_timer_interceptor_aroundtimeout_singleton_annotated_ejbliteservlet_vehicle_web/ejbliteservlet_vehicle.jsp?testName=allInterceptorsOverride
[TEST FAILED] testName=allInterceptorsOverride []


Check timeout status Expecting true, but actual nullFailed with exception 
java.lang.RuntimeException: []


Check timeout status Expecting true, but actual null
	at com.sun.ts.tests.ejb30.common.helper.Helper.assertEquals(Helper.java:72)
	at com.sun.ts.tests.ejb30.common.lite.EJBLiteClientBase.assertEquals(EJBLiteClientBase.java:355)
	at com.sun.ts.tests.ejb30.timer.common.ClientBase.passIfTimeout(ClientBase.java:73)
	at com.sun.ts.tests.ejb30.timer.common.ClientBase.passIfTimeout(ClientBase.java:61)
	at com.sun.ts.tests.ejb30.timer.interceptor.aroundtimeout.common.ClientBase.checkAndClearAroundTimeoutRecords(ClientBase.java:186)
	at com.sun.ts.tests.ejb30.timer.interceptor.aroundtimeout.common.ClientBase.aroundTimeoutTest(ClientBase.java:180)
	at com.sun.ts.tests.ejb30.timer.interceptor.aroundtimeout.common.ClientBase.allInterceptorsOverride(ClientBase.java:80)

These tests need to be improved to have longer delay, or make it configurable.

Change the groupId and artifactId for that JTA API jar

As per the information on https://github.com/eclipse-ee4j/ee4j/wiki/New-Maven-Coordinates Jakarta EE implementation will be a different Jar file name

Javamail CTS tests fails due to mail server start up issue in Eclipse CJE infrastructure

Javamail server is started as part of the pod instance used for testing. However, when the javamail server is used by testing, Authentication exception is being thrown.
The issue does not happen always. If Javmail test suite alone is re-run, it passes. So this might be infrastructure related as well.

javax.mail.AuthenticationFailedException: AUTHENTICATE failed. Authentication failed.
at com.sun.mail.imap.IMAPStore.protocolConnect(IMAPStore.java:708)
at javax.mail.Service.connect(Service.java:364)
at com.sun.ts.tests.javamail.ee.common.MailTestUtil.connect2host(MailTestUtil.java:319)
at com.sun.ts.tests.javamail.ee.fetchprofile.fetchprofile_Test.setup(fetchprofile_Test.java:77)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.ts.lib.harness.EETest.run(EETest.java:569)
at com.sun.ts.lib.harness.ServiceEETest.run(ServiceEETest.java:115)
at com.sun.ts.tests.common.vehicle.EmptyVehicleRunner.run(EmptyVehicleRunner.java:40)
at com.sun.ts.lib.harness.ServiceEETest.run(ServiceEETest.java:105)
at com.sun.ts.lib.harness.EETest.getPropsReady(EETest.java:486)
at com.sun.ts.lib.harness.ServiceEETest.run(ServiceEETest.java:209)
at com.sun.ts.lib.harness.EETest.run(EETest.java:285)
at com.sun.ts.tests.common.vehicle.VehicleClient.main(VehicleClient.java:38)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.glassfish.appclient.client.acc.AppClientContainer.launch(AppClientContainer.java:422)
at org.glassfish.appclient.client.AppClientFacade.launch(AppClientFacade.java:159)
at org.glassfish.appclient.client.AppClientGroupFacade.main(AppClientGroupFacade.java:41)
java.lang.IllegalStateException: Not connected
at com.sun.mail.imap.IMAPStore.checkConnected(IMAPStore.java:1959)
at com.sun.mail.imap.IMAPStore.getDefaultFolder(IMAPStore.java:1726)
at com.sun.ts.tests.javamail.ee.fetchprofile.fetchprofile_Test.getRootFolder(fetchprofile_Test.java:328)
at com.sun.ts.tests.javamail.ee.fetchprofile.fetchprofile_Test.setup(fetchprofile_Test.java:81)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.ts.lib.harness.EETest.run(EETest.java:569)
at com.sun.ts.lib.harness.ServiceEETest.run(ServiceEETest.java:115)
at com.sun.ts.tests.common.vehicle.EmptyVehicleRunner.run(EmptyVehicleRunner.java:40)
at com.sun.ts.lib.harness.ServiceEETest.run(ServiceEETest.java:105)
at com.sun.ts.lib.harness.EETest.getPropsReady(EETest.java:486)
at com.sun.ts.lib.harness.ServiceEETest.run(ServiceEETest.java:209)
at com.sun.ts.lib.harness.EETest.run(EETest.java:285)
at com.sun.ts.tests.common.vehicle.VehicleClient.main(VehicleClient.java:38)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.glassfish.appclient.client.acc.AppClientContainer.launch(AppClientContainer.java:422)
at org.glassfish.appclient.client.AppClientFacade.launch(AppClientFacade.java:159)
at org.glassfish.appclient.client.AppClientGroupFacade.main(AppClientGroupFacade.java:41)
01-05-2019 10:48:29: ERROR: Exception : null
01-05-2019 10:48:29: ERROR: Setup Failed!
01-05-2019 10:48:29: ERROR: java.lang.NullPointerException
at com.sun.ts.tests.javamail.ee.fetchprofile.fetchprofile_Test.setup(fetchprofile_Test.java:82)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.ts.lib.harness.EETest.run(EETest.java:569)
at com.sun.ts.lib.harness.ServiceEETest.run(ServiceEETest.java:115)
at com.sun.ts.tests.common.vehicle.EmptyVehicleRunner.run(EmptyVehicleRunner.java:40)
at com.sun.ts.lib.harness.ServiceEETest.run(ServiceEETest.java:105)
at com.sun.ts.lib.harness.EETest.getPropsReady(EETest.java:486)
at com.sun.ts.lib.harness.ServiceEETest.run(ServiceEETest.java:209)
at com.sun.ts.lib.harness.EETest.run(EETest.java:285)
at com.sun.ts.tests.common.vehicle.VehicleClient.main(VehicleClient.java:38)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.glassfish.appclient.client.acc.AppClientContainer.launch(AppClientContainer.java:422)
at org.glassfish.appclient.client.AppClientFacade.launch(AppClientFacade.java:159)
at org.glassfish.appclient.client.AppClientGroupFacade.main(AppClientGroupFacade.java:41)

01-05-2019 10:48:29: ERROR: Unexpected Exception null
01-05-2019 10:48:29: ERROR: java.lang.NullPointerException
at com.sun.ts.tests.javamail.ee.fetchprofile.fetchprofile_Test.test2(fetchprofile_Test.java:154)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.ts.lib.harness.EETest.run(EETest.java:596)
at com.sun.ts.lib.harness.ServiceEETest.run(ServiceEETest.java:115)
at com.sun.ts.tests.common.vehicle.EmptyVehicleRunner.run(EmptyVehicleRunner.java:40)
at com.sun.ts.lib.harness.ServiceEETest.run(ServiceEETest.java:105)
at com.sun.ts.lib.harness.EETest.getPropsReady(EETest.java:486)
at com.sun.ts.lib.harness.ServiceEETest.run(ServiceEETest.java:209)
at com.sun.ts.lib.harness.EETest.run(EETest.java:285)
at com.sun.ts.tests.common.vehicle.VehicleClient.main(VehicleClient.java:38)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.glassfish.appclient.client.acc.AppClientContainer.launch(AppClientContainer.java:422)
at org.glassfish.appclient.client.AppClientFacade.launch(AppClientFacade.java:159)
at org.glassfish.appclient.client.AppClientGroupFacade.main(AppClientGroupFacade.java:41)

01-05-2019 10:48:29: ERROR: test2 Failed
01-05-2019 10:48:29: ERROR: Test case throws exception: test2 Failed
01-05-2019 10:48:29: ERROR: Exception at:
01-05-2019 10:48:29: ERROR: com.sun.ts.lib.harness.EETest$Fault: test2 Failed
at com.sun.ts.tests.javamail.ee.fetchprofile.fetchprofile_Test.test2(fetchprofile_Test.java:242)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.ts.lib.harness.EETest.run(EETest.java:596)
at com.sun.ts.lib.harness.ServiceEETest.run(ServiceEETest.java:115)
at com.sun.ts.tests.common.vehicle.EmptyVehicleRunner.run(EmptyVehicleRunner.java:40)
at com.sun.ts.lib.harness.ServiceEETest.run(ServiceEETest.java:105)
at com.sun.ts.lib.harness.EETest.getPropsReady(EETest.java:486)
at com.sun.ts.lib.harness.ServiceEETest.run(ServiceEETest.java:209)
at com.sun.ts.lib.harness.EETest.run(EETest.java:285)
at com.sun.ts.tests.common.vehicle.VehicleClient.main(VehicleClient.java:38)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.glassfish.appclient.client.acc.AppClientContainer.launch(AppClientContainer.java:422)
at org.glassfish.appclient.client.AppClientFacade.launch(AppClientFacade.java:159)
at org.glassfish.appclient.client.AppClientGroupFacade.main(AppClientGroupFacade.java:41)

Clean-up .adoc files under user_guides

The source files for CTS / TCK User's Guides have some references to Java Partner Engineering, Licensee, etc. Although many of the sentences which are not valid now are removed, there are still some more clean-up required for the .adoc files under user_guides.

The documentation is in the process of being revised to reflect the new Jakarta EE branding. Additional changes will be made as requirements and procedures evolve for Jakarta EE.

This issue will track the remaining changes required for files under user_guides.

Turning TCK into a multi-dependency maven project

To make contributing and using the TCK easier for those working only on a small subset of APIs without making it overly hard for those who work on Jakarta EE as a full set of APIs, I'd like to propose to turn the TCK project into a Maven multi-dependency assembly project:

• Split up the monolith into TCK-per-API and move the TCKs into the repositories of the particular API projects.
• The API projects compile and release the TCKs together with their APIs to ensure that API and TCK always fit together, so the TCKs are found on Maven Central at the same time the APIs appear there. This will be used by single-API-vendors (like EclipseLink, Jersey, etc.).
• The original (ex-monolith) TCK project provides a POM which pulls all those single TCKs from Maven Central as dependencies and packs them together in a single-shot umbrella assembly. This will be used by profile vendors (like GlassFish, Payara, etc.).

JPA TCK - Start derby with a security policy

The JPA TCK installs java procedures using task, which is failing after the recent upgrade of Derby in GlassFish.

install_java_procedures:
[sql] Executing commands
[sql] Failed to execute: CALL sqlj.remove_jar('APP.dbprocedures', 0)
[sql] java.sql.SQLTransactionRollbackException: Jar file 'DBPROCEDURES' does not exist in schema 'APP'.
[sql] Failed to execute: CALL sqlj.install_jar('/scratch/pabhat/jpatck/bin/xml/../../lib/dbprocedures.jar', 'APP.dbprocedures', 0)
[sql] java.sql.SQLTransactionRollbackException: The exception 'java.security.AccessControlException: access denied ("java.io.FilePermission" "/scratch/pabhat/jpatck/bin/xml/../../lib/dbprocedures.jar" "read")' was thrown while evaluating an expression.
[sql] Failed to execute: CALL SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY('derby.database.classpath', 'APP.dbprocedures')
[sql] java.sql.SQLSyntaxErrorException: The database class path contains an unknown jar file '"APP"."DBPROCEDURES"'.

The TCK run should start Derby by supplying a security policy, which will give required permissions to dbprocedures.jar

WebSocket TCK some async sendObject() tests don't check Future

While investigating some TCK failures with Apache Tomcat's WebSocket 1.1 implementation I discovered that 8 of the tests under com/sun/ts/tests/websocket/ee/javax/websocket/remoteendpoint/async start sending a message using the asynchronous interface but do not explicitly wait for those messages to complete before trying to send a second message using the basic (synchronous) interface.
The Javadoc for RemoteEndpoint.Basic states that

If the websocket connection underlying this RemoteEndpoint is busy sending a message when a call is made to send another one, for example if two threads attempt to call a send method concurrently, or if a developer attempts to send a new message while in the middle of sending an existing one, the send method called while the connection is already busy may throw an java.lang.IllegalStateException.

Note that "may throw" in the quoted sentence. The TCK tests will pass or fail depending on whether the implementation opts to throw the Exception in these circumstances. It is my view that this is not correct.

Note also that other tests in the same class explicitly wait for the Future returned by sendObject() and therefore never attempt to send concurrent messages and do not exhibit this bug.

I have a patch for this. PR to follow shortly.