JBERET000601: Failed to get job xml file for job about jsr352

[chengfang]

Are you running the test with Arq embedded? How did you install your module to the WildFly? Can you try Arq remote config, or package the test as a standard war and deploy to WildFly, just to eliminate one variable?

[chengfang]

I"m a little confused. Your last paragraph says the arq deployment works with jboss-deployment-structure.xml. So what is it that is not working? Can you elaborate under what circumstances you had the errors, and your expected behavior wrt job loading in your module?

[mincong-h]

I’m still a student and lack experience in this field, but I’ll try my best to describe the situation.

Structure of the project

There’re several maven modules in my project and 3 of them are concerned in this issue.

• hsearch-jsr352-core uses JSR 352 to create its batch job, described in META-INF/batch-jobs/mass-index.xml
• hsearch-jsr352-modules aims to package the core Maven module hsearch-jsr352-core as a module in a container managed WildFly server.
• hsearch-jsr352-integrationtest-wildfly tests the core Maven module installed in WildFly. The test WAR is deployed to the same WF as configured above. The deployment of WAR works with Arquillian and jboss deployment structure.

Expected behaviors

1. hsearch-jsr352-modules packages the core with its meta inf files and install the core into WF as a WF module
2. hsearch-jsr352-integrationtest-wildfly deploys the WAR file for integration test
3. Thanks to the jboss-deployment-structure.xml, the test WAR file can detect the core module installed in WF, including its META-INF folder
4. The @Test of the deployed WAR starts
5. The @Test asks batch runtime to start the job mass-index
6. JBeret resolves job mass-index and find its correspondent XML file from the installed module and then reads the target job mass-index.xml. Please notice that the job XML is not located in the WAR, but in the installed module (core). ❌
7. Job starts.

The job failed at step 6.

[chengfang]

Thanks for the detailed explanation, now I have better picture what you're trying to do. Can you step thru in a debugger to see what job loader gets used (your custom job loader, or the default job loader), and their associated class loader, which is used to load job xml as resource. You can start from, say, org.jberet.operations.JobOperatorImpl#start(java.lang.String, java.util.Properties)

[mincong-h]

Hi @chengfang, the module is packaged by Maven plugin maven-assembly-plugin and installed by Maven plugin maven-dependency-plugin with the below script:

            <plugin>
                <artifactId>maven-dependency-plugin</artifactId>
                <executions>
                    <execution>
                        <id>unpack</id>
                        <phase>pre-integration-test</phase>
                        <goals>
                            <goal>unpack</goal>
                        </goals>
                        <configuration>
                            <artifactItems>
                                <artifactItem>
                                    <groupId>org.wildfly</groupId>
                                    <artifactId>wildfly-dist</artifactId>
                                    <version>${org.wildfly}</version>
                                    <type>zip</type>
                                    <overWrite>true</overWrite>
                                    <outputDirectory>${project.build.directory}/node1</outputDirectory>
                                </artifactItem>
                                <!-- Provide the current project to WildFly as a new module -->
                                <artifactItem>
                                    <groupId>org.hibernate</groupId>
                                    <artifactId>hsearch-jsr352-modules</artifactId>
                                    <version>${project.version}</version>
                                    <classifier>wildfly-10-dist</classifier>
                                    <type>zip</type>
                                    <overWrite>true</overWrite>
                                    <outputDirectory>${project.build.directory}/node1/wildfly-${org.wildfly}/modules</outputDirectory>
                                </artifactItem>
                            </artifactItems>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

I'm running the test with Arquillian managed. Thanks to @gunnarmorling, I've activated the remote debugger from Eclipse. So I can now debug the managed WF server. However, I can only give a breakpoint to

javax.batch.operations.JobOperator#start(String, Properties)

But I cannot give a breakpoint to

org.jberet.operations.JobOperatorImpl#start(String, Properties)

Any idea how can I achieve the second one?

[gunnarmorling]

You need to add the add the right version of JBeret (the Java Batch impl) as a provided dependency to your project.

[mincong-h]

Thx, @gunnarmorling. Yes, you've mentioned in your email, but for some reasons I didn't achieve. Now it works! So @chengfang, the break point is made at the line 102 of JobOperatorImpl and here're some variables shown in Variables windows from Eclipse.

• XmlLoader = ArchiveXmlLoader
• batchEnvironment = DelegatingBatchEnvironment
• delegate = BatchEnvironmentServices$WildFlyBatchEnvironment
  - jobXmlResolver = WildFlyJobXmlResolver
  • classLoader = null

[chengfang]

Great progress! in your test webapp WAR structure, can you add:

WEB-INF/classes/META-INF/batch-jobs

I know yoru job.xml is inside the module, so it's just a dummy dir, or you can even add a dummy-job.xml there. WildFly might have identified your test webapp as non-batch app.

In your debug output, what's concerning is the null classLoader, and emptySet jobXmlResolvers.

Once you added the dummy job xml files, can you compare the debug output? If you continue to step into the loadJobXml method, you may find more.

[mincong-h]

It works by following your suggestion! 💯 😺 I don't know how to add an empty folder to Arquillian, so I added a dummy xml as you said:

WEB-INF/classes/META-INF/batch-jobs/should-not-be-used.xml

The job mass-index started and finished correctly and all the assertions are OK. I also compared the debug output in the method ArchiveXmlLoader#loadJobXml(String, ClassLoader, List<Job>, JobXmlResolver) and I found that the variables are exactly the same with or without the batch folder META-INF/batch-jobs in the deployment. However, without having the batch jobs folder leads to a failure and having the batch jobs folder makes the test run correctly.

• jobXmlName = "mass-index"
• classLoader = ModuleClassLoader
• jobXmlResolver = WildFlyJobXmlResolver

I think we did a great job for arriving here and I'm really happy about it!! Thanks everybody 👏 👏 Now, the new question is why the batch folder should be added? According the WildFly 10 document Implicit module dependencies for deployments, the Batch Subsystem, javax.batch.api, should be always added as a dependency without additional trigger. And more importantly, is there any way to avoid adding this folder? If Hibernate Search users need to add explicitly this folder META-INF/batch-jobs to benefit from the new impl of mass indexing, then we're in a very complicated situation.

BTW, I can only develop this project during the night time, so I'll try to continue tomorrow night. Thx again.

[jamezp]

It looks like WildFly was setup to ignore non-batch applications https://github.com/wildfly/wildfly/blob/10.0.0.Final/batch/extension-jberet/src/main/java/org/wildfly/extension/batch/jberet/deployment/WildFlyJobXmlResolver.java#L87-L108. I'd consider this a bug since there is an intention to allow using the JobXmlResolver interface. Would you mind filing an issue here https:// issues.jboss.org/browse/WFLY? If you don't have a user just let me know and I'll file one.

[jamezp]

In fact this should fix it wildfly/wildfly@6e869ac

[mincong-h]

Issue filled: https://issues.jboss.org/browse/WFLY-7000
I didn't mean to assign @chengfang in that issue, but it seems that system did it automatically. BTW, if someone can enhance my description in WFLY-7000, that will be very appreciated.

In fact this should fix it wildfly/wildfly@6e869ac

How can I download this specific version of WF? Should I add some repo setup in Maven, so that it can download the WF from GitHub instead of Maven center?

[gunnarmorling]

How can I download this specific version of WF?

You can check out and build the WF sources as described here:
https://developer.jboss.org/wiki/HackingOnWildFly

To check out that specific commit, you'd run

git checkout 6e869ac

Then build it (check how to skip the tests likely to save some time) and
use that snapshot version for your Arquillian test.

Great to see the progress, thanks everyone!

[mincong-h]

In fact this should fix it wildfly/wildfly@6e869ac

I confirmed that the above commit solved this issue. See mincong-h/gsoc-hsearch@0640913 too.

[jamezp]

Excellent! Well done @mincong-h. I've got a PR open for it already :) wildfly/wildfly#9154

[mincong-h]

Great @jamezp. Thanks everybody!