opendistro-for-elasticsearch / job-scheduler Goto Github PK

Hi,

do you have plans to publish Artifacts to Maven Central. As an enduser I would expect to find them there.

Regards
Andreas

ScheduledJob executions are not a singleton runner and if a job execution takes longer than the interval/cron time then there can be two or more execution running at a time.

For these kinds of jobs we need some ways to ensure the previous execution is finished before starting the new execution.

Proposed solution is to create job locking such that each job execution needs to ensure the previous job execution is complete by acquiring lock. If the lock cannot be acquired that job execution is skipped.

When we delete a job, how lock is handled. If we delete the job without releasing or deleting it from lock table we might leave an entry in lock index right?

Also when we lock the job will it lock and write threads until the time mentioned or lock is released?

Make the plugin compatible with Elasticsearch 7.2.1

LockServiceIT has some tests fails randomly using ESIntegTestCase. We should refactor the test into sample-extension-plugin integ test and use ESRestTestCase to use external Elasticsearch cluster.

Based on this discussion https://discuss.elastic.co/t/use-junit-with-esintegtestcase/132061

One feature request we got is to allow more dynamic job scheduling. The idea is to allow job runner to return a hint to scheduler, indicate the when to execute the job runner next time. Then scheduler can evaluate the hint time and override the scheduled next execution if possible.

We have use case where we need to have jobs run at predefined time as these are Aggregation/Rollup jobs that need to be executed for say at the end of every hour, every 1 day and so on.

For now, it looks like when Elasticsearch is restarted, Jobs are scheduled based on time Elasticsearch is restarted thus causing delays and sometimes even some jobs also missed depending on time Elastic search is restarted. Instead of arbitrary scheduling can we have option where user can set start Time for Job or atleast have Job after restart of Elasticsearch run based on last run time.

Hi,
The maven endpoint for ODFE has changed from https://oss.sonatype.org/service/local/staging/deploy/maven2 to https://aws.oss.sonatype.org/service/local/staging/deploy/maven2

The login credentials are the same as before. However, the user token has been updated. Infra team will updated the user token credentials (used in maven workflows). However, the URLs in the codebase needs to be updated by the plugin teams.

Currently, the old URL and user token is active. Infra team will update the token on 6/22. Make sure to update the URLs by then

More details: https://issues.sonatype.org/browse/OSSRH-47276

Thanks!

Enable support for Elasticsearch 7.0.1

JobScheduler plugin maybe extended by multiple extension plugins. Extension plugins may support the configuration to allow users to disable the plugin at runtime, which requires all scheduled jobs of that plugin to be descheduled.

Let extension plugin to disable all jobs by updating all job documents by setting enabled flag to false is an option, but it is not ideal, as the enabled field is meant for extension plugins to control at job level to support their use cases.

Thus we may want to implement a new mechanism for extension plugins to disable all their jobs at one shot.

Right now for some jobs that have larger intervals the very first execution has to wait for an entire interval first.
It would be nice if there was a choice given to the plugin on whether or not the very first scheduling executes it immediately + reschedules for next interval vs schedules for next interval.

Make the plugin compatible with Elasticsearch 7.7.0

Currently it's quite hard to debug something wrong in job scheduler - such as a job not running when everything pointing to it should be running.

I'd like to see an API that returns all information about the current in-memory jobs along with the time they'll be executed next/how much time is left.

Perhaps even an API to forcefully reschedule a job? I had to allocate shards to a new node to force this behavior. Ideally this shouldn't be needed but having it available as an admin tool would be nice.

Hi, it would be great to have a little bit more documentation on this plugin or even add it to the official OpenDistro docs.

To make job scheduler plugin work in cluster, do i need to install plugin in single node of cluster or all nodes of cluster, please reply

Some jobs have no need to run exactly at the scheduled time. If lots of jobs start at the same time point, it will bring burst and may overload cluster. We should build some mechanism to allow job run with some jitter delay to spread out load.

• Any way to schedule one time job to run
• Is it possible to start the job immediately after scheduling, rather running it as delayed job?

If these options are already available , please update the documentation with relevant examples

Job Scheduler should use the same four-digit version numbers as the other plugins. Currently it uses three.

$ curl -XGET --insecure https://localhost:9200/_cat/plugins -u admin:admin
a44d0821e28e opendistro-job-scheduler        1.1.0
a44d0821e28e opendistro_alerting             1.1.0.0
a44d0821e28e opendistro_performance_analyzer 1.1.0.0
a44d0821e28e opendistro_security             1.1.0.0
a44d0821e28e opendistro_sql                  1.1.0.0

#37

We saw some failures when running the build on github actions workflow on ubuntu-latest.
Interestingly it my fork with the same exact setup passed. Builds are passing on macos-latest, and given time constraints we'll move forward with this for now and look into the ubuntu issue later.

Found from this forum discussion in ISM: https://discuss.opendistrocommunity.dev/t/ism-attempting-to-interact-with-an-obsolete-index/3224

Job scheduler has an in-memory map that contains the scheduled jobs that are scheduled to run. When a job document is created, updated, or deleted this map is updated with the appropriate action. In this specific case the delete somehow failed which left a job that was still executing every 2 hours even though it didn't exist anymore. Ideally the sweeper would catch this and resolve the failure, but the sweeper has a bug where it doesn't remove jobs that were deleted.

For reference:

The sweeper is a background process that sweeps the job indices for job documents to schedule, re-schedule, and de-schedule documents. It does this on an interval defined by the sweep period. Every execution it will sweep all indices that were registered by plugins extending job scheduler which in turn will sweep the shards for each index. This sweepShard function is the one with the bug that is not handling job documents that were deleted from the index.

Make the plugin compatible with Elasticsearch 7.2.0

Version 1.6.0.0
Make the plugin compatible with Elasticsearch version 7.6.

Need to have the capability to report the job scheduling metrics to extension plugins. the minimal metrics we need to share is whether job executions are on schedule. We may consider adding other metrics.