j-easy / easy-jobs Goto Github PK

YAML is easier for humans to write and less error prone than JSON.

The goal is to write job descriptors in YAML instead of JSON. For example:

---
id: 1
name: my first job
class: org.mycompany.jobs.MyFirstJob
method: doWork
---
id: 2
name: my second job
class: org.mycompany.jobs.MySecondJob
method: doWork

Hi,

When I try to run
java -cp "drivers/h2/*;lib/*" ^ -Deasy.jobs.database.config.file=%cd%\conf\database.properties ^ -Dserver.port=9000 ^ org.jeasy.jobs.admin.Application
I got
java.lang.IllegalArgumentException: Could not resolve placeholder 'application.version' in value "${application.version}"
The server works fine but it's impossible to run the admin web interface.
I didn't make any modifications. I'm just trying to run the "demo" that you provide.
Did I miss something ?
BTW, java version is openjdk 1.8.0_171 on a debian 9.

Thanks

As of v0.1, the classpath of user's jobs is the same as the server's classpath.

This may result in classes clash at runtime.

Easy Jobs should use a separate classloader for user's jobs.

In the requests table page (and request details page), there should be a link to the corresponding execution details for each processed request.

Issue #23 should be implemented before this feature.

In order to make it easy to get started with Easy Jobs, it would be interesting to add a "hello world" job in the distribution file.

This will allow the user to start playing with the job server right away after downloading the distribution instead of creating/compiling/packaging the job himself.

Many thanks to @nabilov for this suggestion!

Jult like #12 but for postman rest client:

The postman workspace can be downloaded here:

Easy-Jobs-v0.2_postman_workspace.zip

When the JobServer is launched without specifying the mandatory jobs directory parameter, a NPE is thrown

A job execution may complete successfully or fail.

• If it succeeds, the return type of the execution method can be serialized and shown in the GUI (it can be a calculation result or a job report as in Easy Batch).
• If it fails, it would be great to be able to access the exception through the admin interface.

Is it possible to schedule repeated job?

It should be possible to submit job request with higher priorities to be executed first.

The goal of this issue is to add the possibility to specify a request priority.

Easy Jobs should execute high priority requests first.

Just like issue #9 , the goal is to add templates files for job descriptor and database properties that can be directly modified by the user.

This will avoid guessing database properties keys or job descriptor attributes.

Another credit goes to @nabilov for this suggestion 👍

What is the easy-jobs runtime dependency? It seems that the server has its own main(), therefore, there is no dependency of, for example, netty http lib？

Let's consider the following server configuration:

• worker pool size: 10
• polling interval: 1h
• number of requests received since last polling run: 1000

As of version 0.1, the server will query for pending requests. It will get all the 1000 requests and submit them to the executor service. With only 10 worker threads, this may exhaust resources on the server.

It should be possible to limit the number of pending requests to take in each polling run.
This should provide throttling/back pressure on the server side.

The idea is to help lazy people (like me!) to get started quickly playing around with Easy Jobs server through its REST API.

This issue provides a pre-configured workspace with all endpoints for insomnia rest client:

Download the workspace here:

Easy-Jobs-v0.2_insomnia_workspace.zip

You can then import it through insomnia's wizard.

As of v0.1, when a job execution is fast enough, the job server cannot find a job execution for the executed job. Here is the exception:

org.springframework.dao.EmptyResultDataAccessException: No entity found for query; nested exception is javax.persistence.NoResultException: No entity found for query
	at org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:392) ~[spring-orm-4.3.9.RELEASE.jar:4.3.9.RELEASE]
	at org.springframework.orm.hibernate5.HibernateExceptionTranslator.translateExceptionIfPossible(HibernateExceptionTranslator.java:55) ~[spring-orm-4.3.9.RELEASE.jar:4.3.9.RELEASE]
	at org.springframework.dao.support.ChainedPersistenceExceptionTranslator.translateExceptionIfPossible(ChainedPersistenceExceptionTranslator.java:59) ~[spring-tx-4.3.9.RELEASE.jar:4.3.9.RELEASE]
	at org.springframework.dao.support.DataAccessUtils.translateIfNecessary(DataAccessUtils.java:213) ~[spring-tx-4.3.9.RELEASE.jar:4.3.9.RELEASE]
	at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:147) ~[spring-tx-4.3.9.RELEASE.jar:4.3.9.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.3.9.RELEASE.jar:4.3.9.RELEASE]
	at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99) ~[spring-tx-4.3.9.RELEASE.jar:4.3.9.RELEASE]
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282) ~[spring-tx-4.3.9.RELEASE.jar:4.3.9.RELEASE]
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) ~[spring-tx-4.3.9.RELEASE.jar:4.3.9.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.3.9.RELEASE.jar:4.3.9.RELEASE]
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673) ~[spring-aop-4.3.9.RELEASE.jar:4.3.9.RELEASE]
	at org.jeasy.jobs.execution.JobExecutionRepository$$EnhancerBySpringCGLIB$$529c1f6b.findByJobRequestId(<generated>) ~[classes/:na]
	at org.jeasy.jobs.job.JobService.updateJobExecutionAndItsCorrespondingRequest(JobService.java:62) ~[classes/:na]
	at org.jeasy.jobs.job.DefaultJob.call(DefaultJob.java:30) [classes/:na]
	at org.jeasy.jobs.job.DefaultJob.call(DefaultJob.java:9) [classes/:na]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_73]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_73]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_73]
	at java.lang.Thread.run(Thread.java:745) [na:1.8.0_73]
Caused by: javax.persistence.NoResultException: No entity found for query
	at org.hibernate.query.internal.AbstractProducedQuery.getSingleResult(AbstractProducedQuery.java:1465) ~[hibernate-core-5.2.10.Final.jar:5.2.10.Final]
	at org.jeasy.jobs.execution.JobExecutionRepository.findByJobRequestId(JobExecutionRepository.java:25) ~[classes/:na]
	at org.jeasy.jobs.execution.JobExecutionRepository$$FastClassBySpringCGLIB$$2068fec7.invoke(<generated>) ~[classes/:na]
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) ~[spring-core-4.3.9.RELEASE.jar:4.3.9.RELEASE]
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738) ~[spring-aop-4.3.9.RELEASE.jar:4.3.9.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.3.9.RELEASE.jar:4.3.9.RELEASE]
	at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:136) ~[spring-tx-4.3.9.RELEASE.jar:4.3.9.RELEASE]
	... 14 common frames omitted

The job descriptor should support describing which job parameters (and their types) are expected.

Then, the GUI can introspect the descriptor (or better the job description in the DB) to assist the user in creating job execution requests.

These details can be shown in a Job details page.

Table names may interfere with existing tables.

The idea is to add a prefix specific to easy jobs:

• job -> ej_job
• job_execution -> ej_ job_execution
• job_execution_request -> ej_ job_execution_request

There should be a shell/batch script to start the server/admin apps for unix/windows systems.

A work in progress is in branch startup-script

As of v0.3, when a request is rejected, the reason is not shown in the UI. Hence there is a need to check the server's log in order to understand the issue. It should be possible to check the reason why the request was rejected in the UI (new column with the exception stack trace in the Job execution requests page)

When a job execution request is not correct (malformed json), it still in the 'PENDING' status and the job server will continuously try to parse it and submit it. This has a useless overhead in execution, may pollute the logs with repetitive information and may mislead the user.

A new status REJECTED is needed to prevent this overhead.

Sometimes, two instances of the same job should not be executed concurrently.

The goal is to add a allowConcurrent parameter in the job definition to allow/disallow concurrent job executions.

Whenever a job request comes in and there is already an execution running for the requested job, then the request should be postponed (or may be ignored, this should be configurable).

It should be possible to cancel a job request from the admin interface.

This may requires to add a new status CANCELED to JobExecutionRequestStatus.

It would be great to add swagger to the rest API.

It is easier for documenting and testing API endpoints

"Job execution request" is a better and more explicit name than "Job request"

Jobs are submitted to a pool of threads.
Stopping a thread in Java is not the easiest thing to do 😄

It should be possible through the admin interface to stop an execution.
The server should keep track of the execution's Future objects and call cancel on the requested one.

This will set the interrupted flag on the execution thread which should be detected by the user's job using Thread.currentThread().isInterrupted(). This is possible out of the box if the job is an Easy Batch job.