spring-guides / gs-batch-processing Goto Github PK

I am getting this error while running the jar, even as admin

2018-02-28 13:27:22.371 ERROR 7460 --- [ main] o.s.boot.SpringApplication : Application startup failed java.lang.IllegalStateException: Failed to execute CommandLineRunner at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:735) [spring-boot-1.5.10.RELEASE.jar!/:1.5.10.RELEASE] at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:716) [spring-boot-1.5.10.RELEASE.jar!/:1.5.10.RELEASE] at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:703) [spring-boot-1.5.10.RELEASE.jar!/:1.5.10.RELEASE] at org.springframework.boot.SpringApplication.run(SpringApplication.java:304) [spring-boot-1.5.10.RELEASE.jar!/:1.5.10.RELEASE] at org.springframework.boot.SpringApplication.run(SpringApplication.java:1118) [spring-boot-1.5.10.RELEASE.jar!/:1.5.10.RELEASE] at org.springframework.boot.SpringApplication.run(SpringApplication.java:1107) [spring-boot-1.5.10.RELEASE.jar!/:1.5.10.RELEASE] at hello.Application.main(Application.java:10) [classes!/:0.1.0] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:na] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:na] at java.base/java.lang.reflect.Method.invoke(Unknown Source) ~[na:na] at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [gs-batch-processing-0.1.0.jar:0.1.0] at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [gs-batch-processing-0.1.0.jar:0.1.0] at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [gs-batch-processing-0.1.0.jar:0.1.0] at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51) [gs-batch-processing-0.1.0.jar:0.1.0] Caused by: org.springframework.jdbc.BadSqlGrammarException: PreparedStatementCallback; bad SQL grammar [SELECT JOB_INSTANCE_ID, JOB_NAME from BATCH_JOB_INSTANCE where JOB_NAME = ? order by JOB_INSTANCE_ID desc]; nested exception is java.sql.SQLSyntaxErrorException: user lacks privilege or object not found: BATCH_JOB_INSTANCE in statement [SELECT JOB_INSTANCE_ID, JOB_NAME from BATCH_JOB_INSTANCE where JOB_NAME = ? order by JOB_INSTANCE_ID desc] at org.springframework.jdbc.support.SQLExceptionSubclassTranslator.doTranslate(SQLExceptionSubclassTranslator.java:91) ~[spring-jdbc-4.3.14.RELEASE.jar!/:4.3.14.RELEASE] at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:73) ~[spring-jdbc-4.3.14.RELEASE.jar!/:4.3.14.RELEASE] at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:82) ~[spring-jdbc-4.3.14.RELEASE.jar!/:4.3.14.RELEASE] at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:655) ~[spring-jdbc-4.3.14.RELEASE.jar!/:4.3.14.RELEASE] at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:690) ~[spring-jdbc-4.3.14.RELEASE.jar!/:4.3.14.RELEASE] at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:722) ~[spring-jdbc-4.3.14.RELEASE.jar!/:4.3.14.RELEASE] at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:732) ~[spring-jdbc-4.3.14.RELEASE.jar!/:4.3.14.RELEASE] at org.springframework.batch.core.repository.dao.JdbcJobInstanceDao.getJobInstances(JdbcJobInstanceDao.java:230) ~[spring-batch-core-3.0.8.RELEASE.jar!/:3.0.8.RELEASE] at org.springframework.batch.core.explore.support.SimpleJobExplorer.getJobInstances(SimpleJobExplorer.java:173) ~[spring-batch-core-3.0.8.RELEASE.jar!/:3.0.8.RELEASE] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:na] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:na] at java.base/java.lang.reflect.Method.invoke(Unknown Source) ~[na:na] at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333) ~[spring-aop-4.3.14.RELEASE.jar!/:4.3.14.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) ~[spring-aop-4.3.14.RELEASE.jar!/:4.3.14.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.3.14.RELEASE.jar!/:4.3.14.RELEASE] at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) ~[spring-batch-core-3.0.8.RELEASE.jar!/:3.0.8.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.3.14.RELEASE.jar!/:4.3.14.RELEASE] at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) ~[spring-aop-4.3.14.RELEASE.jar!/:4.3.14.RELEASE] at com.sun.proxy.$Proxy46.getJobInstances(Unknown Source) ~[na:na] at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.getNextJobParameters(JobLauncherCommandLineRunner.java:131) ~[spring-boot-autoconfigure-1.5.10.RELEASE.jar!/:1.5.10.RELEASE] at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.execute(JobLauncherCommandLineRunner.java:210) ~[spring-boot-autoconfigure-1.5.10.RELEASE.jar!/:1.5.10.RELEASE] at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.executeLocalJobs(JobLauncherCommandLineRunner.java:227) ~[spring-boot-autoconfigure-1.5.10.RELEASE.jar!/:1.5.10.RELEASE] at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.launchJobFromProperties(JobLauncherCommandLineRunner.java:123) ~[spring-boot-autoconfigure-1.5.10.RELEASE.jar!/:1.5.10.RELEASE] at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.run(JobLauncherCommandLineRunner.java:117) ~[spring-boot-autoconfigure-1.5.10.RELEASE.jar!/:1.5.10.RELEASE] at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:732) [spring-boot-1.5.10.RELEASE.jar!/:1.5.10.RELEASE] ... 14 common frames omitted Caused by: java.sql.SQLSyntaxErrorException: user lacks privilege or object not found: BATCH_JOB_INSTANCE in statement [SELECT JOB_INSTANCE_ID, JOB_NAME from BATCH_JOB_INSTANCE where JOB_NAME = ? order by JOB_INSTANCE_ID desc] at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source) ~[hsqldb-2.3.5.jar!/:2.3.5] at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source) ~[hsqldb-2.3.5.jar!/:2.3.5] at org.hsqldb.jdbc.JDBCPreparedStatement.<init>(Unknown Source) ~[hsqldb-2.3.5.jar!/:2.3.5] at org.hsqldb.jdbc.JDBCConnection.prepareStatement(Unknown Source) ~[hsqldb-2.3.5.jar!/:2.3.5] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:na] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:na] at java.base/java.lang.reflect.Method.invoke(Unknown Source) ~[na:na] at org.apache.tomcat.jdbc.pool.ProxyConnection.invoke(ProxyConnection.java:126) ~[tomcat-jdbc-8.5.27.jar!/:na] at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:108) ~[tomcat-jdbc-8.5.27.jar!/:na] at org.apache.tomcat.jdbc.pool.interceptor.AbstractCreateStatementInterceptor.invoke(AbstractCreateStatementInterceptor.java:75) ~[tomcat-jdbc-8.5.27.jar!/:na] at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:108) ~[tomcat-jdbc-8.5.27.jar!/:na] at org.apache.tomcat.jdbc.pool.DisposableConnectionFacade.invoke(DisposableConnectionFacade.java:81) ~[tomcat-jdbc-8.5.27.jar!/:na] at com.sun.proxy.$Proxy53.prepareStatement(Unknown Source) ~[na:na] at org.springframework.jdbc.core.JdbcTemplate$SimplePreparedStatementCreator.createPreparedStatement(JdbcTemplate.java:1530) ~[spring-jdbc-4.3.14.RELEASE.jar!/:4.3.14.RELEASE] at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:633) ~[spring-jdbc-4.3.14.RELEASE.jar!/:4.3.14.RELEASE] ... 36 common frames omitted Caused by: org.hsqldb.HsqlException: user lacks privilege or object not found: BATCH_JOB_INSTANCE at org.hsqldb.error.Error.error(Unknown Source) ~[hsqldb-2.3.5.jar!/:2.3.5] at org.hsqldb.error.Error.error(Unknown Source) ~[hsqldb-2.3.5.jar!/:2.3.5] at org.hsqldb.ParserDQL.readTableName(Unknown Source) ~[hsqldb-2.3.5.jar!/:2.3.5] at org.hsqldb.ParserDQL.readTableOrSubquery(Unknown Source) ~[hsqldb-2.3.5.jar!/:2.3.5] at org.hsqldb.ParserDQL.XreadTableReference(Unknown Source) ~[hsqldb-2.3.5.jar!/:2.3.5] at org.hsqldb.ParserDQL.XreadFromClause(Unknown Source) ~[hsqldb-2.3.5.jar!/:2.3.5] at org.hsqldb.ParserDQL.XreadTableExpression(Unknown Source) ~[hsqldb-2.3.5.jar!/:2.3.5] at org.hsqldb.ParserDQL.XreadQuerySpecification(Unknown Source) ~[hsqldb-2.3.5.jar!/:2.3.5] at org.hsqldb.ParserDQL.XreadSimpleTable(Unknown Source) ~[hsqldb-2.3.5.jar!/:2.3.5] at org.hsqldb.ParserDQL.XreadQueryPrimary(Unknown Source) ~[hsqldb-2.3.5.jar!/:2.3.5] at org.hsqldb.ParserDQL.XreadQueryTerm(Unknown Source) ~[hsqldb-2.3.5.jar!/:2.3.5] at org.hsqldb.ParserDQL.XreadQueryExpressionBody(Unknown Source) ~[hsqldb-2.3.5.jar!/:2.3.5] at org.hsqldb.ParserDQL.XreadQueryExpression(Unknown Source) ~[hsqldb-2.3.5.jar!/:2.3.5] at org.hsqldb.ParserDQL.compileCursorSpecification(Unknown Source) ~[hsqldb-2.3.5.jar!/:2.3.5] at org.hsqldb.ParserCommand.compilePart(Unknown Source) ~[hsqldb-2.3.5.jar!/:2.3.5] at org.hsqldb.ParserCommand.compileStatement(Unknown Source) ~[hsqldb-2.3.5.jar!/:2.3.5] at org.hsqldb.Session.compileStatement(Unknown Source) ~[hsqldb-2.3.5.jar!/:2.3.5] at org.hsqldb.StatementManager.compile(Unknown Source) ~[hsqldb-2.3.5.jar!/:2.3.5] at org.hsqldb.Session.execute(Unknown Source) ~[hsqldb-2.3.5.jar!/:2.3.5] ... 50 common frames omitted

Every detail here:
https://stackoverflow.com/questions/51738718/flatfileitemreader-tab-delimiter-not-working

esay to reproduce.

The tutorial on the spring site (https://spring.io/guides/gs/batch-processing/) is missing the sample-data.csv file and (more importantly) the schema-all.sql file. Please update the tutorial if you have access.

A typo in the gradle.build prevents the project from compiling.
mvn install fails with the following error:

Plugin [id: 'org.springframework.boot', version: '3,0.0'] was not found in any of the following sources:

The coma in 3,0.0 needs to be replaced by a dot and become 3.0.0 (or 3.0.2 at this time).

I was wondering what these "tags" in your code samples are about:

Took me a while to find out these are artifacts of your AsciiDoc markup. Since in the tutorial you include the source code file as a whole, the tags are also included. That way they are useless, but visible. And funnily, your tags have been copied all over the Internet :-D

Maybe you can remove these AsciiDoc tags from the source?

Hi,
I'm new to Spring batch but your great tutorial is allowing me to explore the JobExplorer APIs for finished Job instances (excuse the pun).

My question is, how can I add my own JobParameters to the JobLauncher.run(...) invocation of the "importUserJob" Job bean?

I'm not entirely sure how to override the job's execution, your 'jobLauncher.run(...)' call is a little bit 'hidden' within your Spring-derived dependency library, at 'JobLauncherCommandLineRunner.execute()', within 'spring-boot-autoconfigure-1.5.2.RELEASE.jar'.

Am I right in assuming that, just having the bean references to the Job/JobLauncher, in BatchConfiguration.java, isn't enough for me to add my own parameters to that job? Do I need to just, explicitly, execute my own Job instances via JobLauncher.run(...), where I can also supply my own 'JobParameters' instance?

The 'JobExecutionListenerSupport.beforeJob(JobExecution)' method is actually called too 'late', as, a Job instanced has already been started, and passed an immutable JobParameter object, at that time.

This is just a suggestion to improve the documentation, partly related to spring-guides/getting-started-guides#9.

It would be great to explain what happens in this code:

@SpringBootApplication
public class Application {
    public static void main(String[] args) throws Exception {
        SpringApplication.run(Application.class, args);
    }
}

I understand the main reason to have Getting Started guides mostly based around Spring Boot is to avoid having to re-explain data sources and so on, but I'm talking about what's specific to Spring Batch: more specifically, how to make the jobs actually run.

Thankfully, I've found this external article: http://malsolo.com/blog4java/?p=260

I've been able to adapt it to my examples like so:

@Component
public class HelloApplication {
    @Autowired
    private JobLauncher jobLauncher;

    @Autowired
    private Job importUserJob;

    @Autowired
    public static void main(String[] args) throws Exception {
        try (AnnotationConfigApplicationContext springContext = new AnnotationConfigApplicationContext(
                ConfigBean.class)) {
            HelloApplication main = springContext
                    .getBean(HelloApplication.class);

            main.jobLauncher.run(main.importUserJob,
                    new JobParameters());
        }
    }
}

It would be great to elaborate a bit more in the guides about where the JobLauncher.run(...) is implied, and under which circumstances certain things will be configured or called.

What can often be tricky is to distinguish between what's automatically picked up by non-Boot Spring (e.g. regular @ComponentScan) and may need to be explicitly enabled or disabled, and what's automatically done because of Spring Boot.

Hi there,

I'm brand new to Spring Batch, so I was following the tutorial at https://spring.io/guides/gs/batch-processing/ and it made sense for the most part. However, when I got to the point of actually running the code, I noticed that I was only seeing the output for when the data was processed, e.g.

Converting (firstName: Jill, lastName: Doe) into (firstName: JILL, lastName: DOE)

but not for when the data was read from the internal database, e.g.

Found <firstName: JILL, lastName: DOE> in the database.

This occurred regardless of whether I used "gradlew bootRun" or "gradlew build" followed by "java -jar build/libs/gs-batch-processing-0.1.0.jar." After adding in some debug statements, it appeared to me that the SELECT statement in Application.run() was getting called before the INSERT statements of BatchConfiguration.writer().

Do you have any suggestions for why this is happening?

In case it matters, I'm using Java version 1.8.0_20 on Windows 7 64-bit.

Thank you for any help you can provide!
David

JobExecutionListenerSupport, used in JobCompletionNotificationListener is now deprecated and can be replaced by implements JobExecutionListener instead.

Looking at the example, I can't figure out how the importUserJob can just declare an argument of type Step and know to pull in the value of the step1 bean. I can only assume it knows to do that because there's only one Step-typed bean defined, which leads me to wonder what I'd need to do if I wanted two steps defined for a job, or if I had two jobs defined each with one step.

Do you think it would be valuable to add information about how to ensure steps are going into the right job? I know it's not directly applicable to this example, but it may merit either making the example slightly more complicated to include another step. Alternatively, and a potential better approach, may be to use a way of defining steps for jobs in a less opaque way (directly calling the method, for example). Or maybe keep the example the same but explain what's going on?

I'm far from an expert on this, so there might be something obvious that I'm missing.

I'm evaluating spring boot for a batch project and I'd like to integrate also batch admin ui. I used this project as proof of concept to develop basic functionalities but I'm not able to integrate the admin ui. Is it possible? Any hint on how to do that?

Hi,

When using maven to build the complete/ sub-project, the jar produced cannot be used in a standalone way. The gradlew build path does create a suitable jar.

Steps to reproduce:

• git clone https://github.com/spring-guides/gs-batch-processing.git
• cd gs-batch-processing/complete
• mvn clean package && java -jar target/gs-batch-processing-0.1.0.jar

Expected output:

Similar to the gradlew output, an ascii art "SPRING" banner, some logging and then, intermixed with the logging:

Converting (firstName: Jill, lastName: Doe) into (firstName: JILL, lastName: DOE)
Converting (firstName: Joe, lastName: Doe) into (firstName: JOE, lastName: DOE)
Converting (firstName: Justin, lastName: Doe) into (firstName: JUSTIN, lastName: DOE)
Converting (firstName: Jane, lastName: Doe) into (firstName: JANE, lastName: DOE)
Converting (firstName: John, lastName: Doe) into (firstName: JOHN, lastName: DOE)
Found <firstName: JILL, lastName: DOE> in the database.
Found <firstName: JOE, lastName: DOE> in the database.
Found <firstName: JUSTIN, lastName: DOE> in the database.
Found <firstName: JANE, lastName: DOE> in the database.
Found <firstName: JOHN, lastName: DOE> in the database.

Actual Output:

Exception in thread "main" java.lang.NoClassDefFoundError: org/springframework/context/ApplicationContext
        at java.lang.Class.getDeclaredMethods0(Native Method)
        at java.lang.Class.privateGetDeclaredMethods(Class.java:2482)
        at java.lang.Class.getDeclaredMethod(Class.java:1982)
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
        at java.lang.Thread.run(Thread.java:679)
Caused by: java.lang.ClassNotFoundException: org.springframework.context.ApplicationContext
        at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
        at org.springframework.boot.loader.LaunchedURLClassLoader.findClass(LaunchedURLClassLoader.java:110)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
        at org.springframework.boot.loader.LaunchedURLClassLoader.doLoadClass(LaunchedURLClassLoader.java:93)
        at org.springframework.boot.loader.LaunchedURLClassLoader.loadClass(LaunchedURLClassLoader.java:65)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
        ... 5 more

Branch:

master, commit hash: 6915a41

Versions:

• Java:

java version "1.6.0_27"
OpenJDK Runtime Environment (IcedTea6 1.12.6) (6b27-1.12.6-1~deb6u1)
OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)

• Maven:

Apache Maven 2.2.1 (rdebian-4)
Java version: 1.6.0_26
Java home: /usr/lib/jvm/java-6-sun-1.6.0.26/jre
Default locale: en_GB, platform encoding: UTF-8
OS name: "linux" version: "2.6.32-5-amd64" arch: "amd64" Family: "unix"

Existing work-arounds:

Just use gradlew for the moment.

Other notes:

I've had a quick play around with gs-spring-boot today as well, and that project also displayed similar behaviour, though I did not test that the gradlew workaround works.

Update the guide to use the most recent Spring Boot version.

Files that require changes are:

initial/build.gradle
initial/pom.xml
complete/build.gradle
complete/pom.xml

Please see spring-projects/spring-boot#22092 for some background. The guide should encourage the use of Spring Boot's Application Exit support.

Update the guide to use the most recent Spring Boot version.

Files that require changes are:
initial/build.gradle
initial/pom.xml
complete/build.gradle
complete/pom.xml

I used this example as a reference for a quick (and successful) implementation of Spring batch in a Spring boot application. The doc/reference mainly shows skip policy configuration in XML, where would be the proper placement for skipping records within the sample? Thanks for your time.

Hi,

I tried to little bit enhance your nice example with the possibility to set the CSV file location from outside (instead of using the classpath).

If I modify the code from the ItemReader to

@Bean
  @Scope("step")
  public ItemReader<Person> reader() {

and then running the commandline (without any further changes!) I get following error:

    ... 37 more
Caused by: java.lang.IllegalStateException: No context holder available for step scope
        at org.springframework.batch.core.scope.StepScope.getContext(StepScope.java:160)
        at org.springframework.batch.core.scope.StepScope.get(StepScope.java:99)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:338)
        ... 43 more

From the Spring documentation, the scope should be available (because of the Batch annotation).

Do you have any hint/idea whats the problem?

Two days ago I completed this guide with Java 11 version.
But I retried it today but It didn't work.
There are compile error:
[class file has wrong version 61.0, should be 55.0 ]

Is it only can tested by Java 17?

AS-IS my pom.xml file:

org.springframework.boot
spring-boot-starter-parent
2.7.6

TOBE :

org.springframework.boot
spring-boot-starter-parent
3.0.0

if I change version
JobBuilder class and StepBuilder class are having more constructor.

return new JobBuilder("importUserJob", jobRepository)
.incrementer(new RunIdIncrementer())
.listener(listener)
.flow(step1)
.end()
.build();

So this code cannot be resolved!
because when I use 2.7.6 version JobBuilder class has only one constructor

public JobBuilder(String name) {
super(name);
}

How can I resolve this problem?