bart6114 / scheduler

An interface to schedule R scripts

Mostly dad of three. Often founder of Dataroots. Sometimes a trail runner, now and then a cyclist. Trying to survive creative & outdoor splurges. Enjoys fantasy writing & literate programming.

💬 Ask me about: open source, sports & outdoors, data, creative ideas.
📫 How to reach me:
     💌 professional mail
     🧳 LinkedIn
     💿 dataroots
     🥷 mastodon
😄 Pronouns: he/him
🧀 Cheese: YAS.

scheduler's Issues

R scripts not being saved

New to scheduleR here but excited at it's usefulness. When I try to upload a R script to the web interface, it doesn't save. Has anyone else run into this issue?

Notify on success and Notify on failure not working

I am scheduleR using git pull. I am able to run the tasks without any issues.
I've set my email address on "Notify on success" and "Notify on failure" option.
But scheduleR is not sending any notifications.

I've also configured with proper mail configurations in user.config.json

Installation on Mac

Hi, I've tried to install on Mac and got the following errors:

Tell the author that this fails on your system:
npm ERR! node-pre-gyp install --fallback-to-build

I have Mongodb and Node js installed and tried both npm install and also the sudo npm update -g npm.

Would be great to see how we could try this on Mac.

Thank you,
Eliano

Running report Error: Rscript ENONENT

When trying to run uploaded R scripts I'm getting Error: spawn Rscript Enoent.

syscall:'spawn Rscript'

My upload path is fine something to do with the RscriptExecutable in the user.config?

Attaching Excel or any other file

Hey Bart,

Thanks a lot for the schedulR. It's been most useful, saves us a lot of time and allows us to do a lot of new things.

One thing we were wondering about was, how do we attach excel files to the mailer?

I'm guessing I could use sendmailr in the tasks and schedule that, but is there any other built in way?

Thanks,

Robin

Please add feature for task running every x minutes

First this is not a bug or any issue, i'd like if there is a feature to run Rscripts lets say every 10 minutes, so i run it 6 times in an hour. It can be triggered using the start time of a particular task, and then it keeps running every 10 minutes, it is pretty simple using cron, but dont; know how to specify it in scheduleR cron tab

rmarkdown set up problems

Hi, I'm currently running R scripts in scheduleR on a linux server, and writing the output to sql tables. I'm trying to get .Rmd scripts to run as well, but for unclear reasons, they are failing. My test .Rmd script works fine when running locally in Rstudio. Both html and pdf outputs have the same error. Does a server version of Rstudio need to be installed, or some additional packages required for rendering? I'm also wondering how to best define the output path. Am thinking initially, just direct the output doc to another folder in the repo. Does the rmarkdown library need to be called somewhere in the script when running in scheduleR ?

rmarkdown version 2.25

script intro

title: "Test of rmarkdown in scheduleR"
author: "Will Gilks"
date: "2023-10-04"
output: html_document
---

` ``{r setup1, include=FALSE}
# output: html_document
# RMARKDOWN_WRITE_PATH="~/Desktop/"
# knitr::opts_knit$set(root.dir=RMARKDOWN_WRITE_PATH)
knitr::opts_chunk$set(echo=FALSE)
renv::use(lockfile=Sys.getenv("R_LOCK"))
Sys.setenv(TZ="UTC")

library(tidyverse)

error message below

Quitting from lines 9-38 (test_rmarkdown.Rmd)
Error in if (source %in% c("git2r", "xgit")) source <- "git" :
  argument is of length zero
Calls: <Anonymous> ... enumerate -> lapply -> FUN -> f -> renv_record_source_normalize
In addition: Warning message:
In do_once((if (is_R_CMD_check()) stop else warning)("The function xfun::isFALSE() will be deprecated in the future. Please ",  :
  The function xfun::isFALSE() will be deprecated in the future. Please consider using base::isFALSE(x) or identical(x, FALSE) instead.
Traceback (most recent calls last):
27: rmarkdown::render("code/scripts/data-science/daily/test_rmarkdown.Rmd",
        "html_document")
26: knitr::knit(knit_input, knit_output, envir = envir, quiet = quiet)
25: process_file(text, output)
24: withCallingHandlers(if (tangle) process_tangle(group) else process_group(group),
        error = function(e) {
            setwd(wd)
            cat(res, sep = "\n", file = output %n% "")
            message("Quitting from lines ", paste(current_lines(i),
                collapse = "-"), " (", knit_concord$get("infile"),
                ") ")
        })
23: process_group(group)
22: process_group.block(group)
21: call_block(x)
20: block_exec(params)
19: eng_r(options)
18: in_dir(input_dir(), evaluate(code, envir = env, new_device = FALSE,
        keep_warning = !isFALSE(options$warning), keep_message = !isFALSE(options$message),
        stop_on_error = if (is.numeric(options$error)) options$error else {
            if (options$error && options$include)
                0L
            else 2L
        }, output_handler = knit_handlers(options$render, options)))
17: evaluate(code, envir = env, new_device = FALSE, keep_warning = !isFALSE(options$warning),
        keep_message = !isFALSE(options$message), stop_on_error = if (is.numeric(options$error)) options$error else {
            if (options$error && options$include)
                0L
            else 2L
        }, output_handler = knit_handlers(options$render, options))
16: evaluate::evaluate(...)
15: evaluate_call(expr, parsed$src[[i]], envir = envir, enclos = enclos,
        debug = debug, last = i == length(out), use_try = stop_on_error !=
            2L, keep_warning = keep_warning, keep_message = keep_message,
        output_handler = output_handler, include_timing = include_timing)
14: timing_fn(handle(ev <- withCallingHandlers(withVisible(eval(expr,
        envir, enclos)), warning = wHandler, error = eHandler, message = mHandler)))
13: handle(ev <- withCallingHandlers(withVisible(eval(expr, envir,
        enclos)), warning = wHandler, error = eHandler, message = mHandler))
12: withCallingHandlers(withVisible(eval(expr, envir, enclos)), warning = wHandler,
        error = eHandler, message = mHandler)
11: withVisible(eval(expr, envir, enclos))
10: eval(expr, envir, enclos)
 9: eval(expr, envir, enclos)
 8: renv::use(lockfile = Sys.getenv("R_LOCK"))
 7: restore(lockfile = lockfile, clean = FALSE)
 6: renv_lockfile_repair(lockfile)
 5: enumerate(records, function(package, record) {
        source <- renv_record_source_normalize(record, record$Source)
        if (identical(source, "Repository") && is.null(record$Version)) {
            entry <- renv_available_packages_latest(package)
            record$Version <- entry$Version
        }
        record
    })
 4: lapply(idx, callback)
 3: FUN(X[[i]], ...)
 2: f(n[[i]], x[[i]], ...)
 1: renv_record_source_normalize(record, record$Source)
Execution halted

Can't start on CentOS

I can't seems to start the software. This is the log that I get from the failure.

0 info it worked if it ends with ok
1 verbose cli [ '/usr/bin/node', '/usr/bin/npm', 'start' ]
2 info using [email protected]
3 info using [email protected]
4 verbose run-script [ 'prestart', 'start', 'poststart' ]
5 info prestart [email protected]
6 info start [email protected]
7 verbose unsafe-perm in lifecycle true
8 info [email protected] Failed to exec start script
9 error [email protected] start: node bin/scheduler.bin.js
9 error Exit status 8
10 error Failed at the [email protected] start script.
10 error This is most likely a problem with the scheduler-for-r package,
10 error not with npm itself.
10 error Tell the author that this fails on your system:
10 error node bin/scheduler.bin.js
10 error You can get their info via:
10 error npm owner ls scheduler-for-r
10 error There is likely additional logging output above.
11 error System Linux 2.6.32-431.23.3.el6.x86_64
12 error command "/usr/bin/node" "/usr/bin/npm" "start"
13 error cwd /home/thunt/scheduleR
14 error node -v v0.10.33
15 error npm -v 1.4.28
16 error code ELIFECYCLE
17 verbose exit [ 1, true ]

Failed at the [email protected] install script

Error when "npm install"
System: Ubuntu Server 15.10, node v6.2.2, npm v3.10.5

Dear scheduleR owner. On my local machine the scheduleR runs perfect but I get the following errors when installing on my remote server. Can you help?

The relevant parts in the nam-debug.log are:
...
1355 warn optional Skipping failed optional dependency /chokidar/fsevents:
1356 warn notsup Not compatible with your operating system or architecture: [email protected]
1357 warn [email protected] requires a peer of grunt@~0.4.0 but none was installed.
1358 warn [email protected] requires a peer of grunt@~0.4.0 but none was installed.
1359 warn [email protected] requires a peer of grunt@~0.4.1 but none was installed.
1360 warn [email protected] requires a peer of grunt@~0.4.0 but none was installed.
1361 warn [email protected] requires a peer of grunt@~0.4.0 but none was installed.
1362 warn [email protected] requires a peer of grunt@~0.4.0 but none was installed.
1363 warn [email protected] requires a peer of [email protected] but none was installed.
1364 warn [email protected] requires a peer of mocha@>=1.20.0 but none was installed.
1365 warn [email protected] requires a peer of grunt@~0.4.1 but none was installed.
1366 warn [email protected] requires a peer of grunt@~0.4.1 but none was installed.
1367 warn [email protected] requires a peer of grunt@~0.4.2 but none was installed.
1368 warn [email protected] No license field.
1369 verbose stack Error: [email protected] install: node-pre-gyp install --fallback-to-build
1369 verbose stack Exit status 1
1369 verbose stack at EventEmitter. (/usr/lib/node_modules/npm/lib/utils/lifecycle.js:242:16)
1369 verbose stack at emitTwo (events.js:106:13)
1369 verbose stack at EventEmitter.emit (events.js:191:7)
1369 verbose stack at ChildProcess. (/usr/lib/node_modules/npm/lib/utils/spawn.js:40:14)
1369 verbose stack at emitTwo (events.js:106:13)
1369 verbose stack at ChildProcess.emit (events.js:191:7)
1369 verbose stack at maybeClose (internal/child_process.js:852:16)
1369 verbose stack at Process.ChildProcess._handle.onexit (internal/child_process.js:215:5)
1370 verbose pkgid [email protected]
1371 verbose cwd /home/node/scheduleR
1372 error Linux 4.2.0-36-generic
1373 error argv "/usr/bin/nodejs" "/usr/bin/npm" "install"
1374 error node v6.2.2
1375 error npm v3.10.5
1376 error code ELIFECYCLE
1377 error [email protected] install: node-pre-gyp install --fallback-to-build
1377 error Exit status 1
1378 error Failed at the [email protected] install script 'node-pre-gyp install --fallback-to-build'.
1378 error Make sure you have the latest version of node.js and npm installed.
1378 error If you do, this is most likely a problem with the v8-profiler package,
1378 error not with npm itself.
1378 error Tell the author that this fails on your system:
1378 error node-pre-gyp install --fallback-to-build
1378 error You can get information on how to open an issue for this project with:
1378 error npm bugs v8-profiler
1378 error Or if that isn't available, you can get their info via:
1378 error npm owner ls v8-profiler
1378 error There is likely additional logging output above.
1379 verbose exit [ 1, true ]

throw new TypeError('callback provided to sync glob')

I`m facing a problem when trying to start

$ NODE_ENV=production
$ node server.js

/usr/bin/node_modules/glob/glob.js:70
throw new TypeError('callback provided to sync glob')
^
TypeError: callback provided to sync glob
at glob (/usr/bin/node_modules/glob/glob.js:70:13)
at module.exports (/home/rodrigo/scheduleR/config/init.js:16:2)
at Object. (/home/rodrigo/scheduleR/server.js:8:36)
at Module._compile (module.js:456:26)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Function.Module.runMain (module.js:497:10)
at startup (node.js:119:16)

at node.js:935:3

$ node -v
v0.10.42
$ npm -v
#3.8.7

I`m able to start any other app using node.

Any help?

fail to load shiny

This is most likely an issue on my end, but not sure why this is happening:
Running my shiny app using the same command as below from command line works, but starting from the ui fails at loading shiny when I start the app, see below the error:

running
'C:\PROGRA1\R\R-321.3\bin\x64\Rterm.exe --slave --no-restore --no-restore --file=R/run_shiny_app.R --args c:\scheduleR\scripts\15756-1h1yg2o.zip 9018'

Loading required package: shiny
Warning message:
In library(package, lib.loc = lib.loc, character.only = TRUE, logical.return = TRUE, :
there is no package called 'shiny'

An error was detected.
<simpleError in loadNamespace(name): there is no package called 'shiny'>

sometimes a report is generated / send twice at same time

haven't found a way to reproduce

error with scheduler.bin.js

0 info it worked if it ends with ok
1 verbose cli [ 'node',
1 verbose cli 'c:\SCHEDULE_R\scheduleR-master\node_modules\npm\bin\npm-cli.js',
1 verbose cli 'start' ]
2 info using [email protected]
3 info using [email protected]
4 verbose run-script [ 'prestart', 'start', 'poststart' ]
5 info prestart [email protected]
6 info start [email protected]
7 verbose unsafe-perm in lifecycle true
8 info [email protected] Failed to exec start script
9 error [email protected] start: node bin/scheduler.bin.js
9 error Exit status 8
10 error Failed at the [email protected] start script.
10 error This is most likely a problem with the scheduler-for-r package,
10 error not with npm itself.
10 error Tell the author that this fails on your system:
10 error node bin/scheduler.bin.js
10 error You can get their info via:
10 error npm owner ls scheduler-for-r
10 error There is likely additional logging output above.
11 error System Windows_NT 6.2.9200
12 error command "node" "c:\SCHEDULE_R\scheduleR-master\node_modules\npm\bin\npm-cli.js" "start"
13 error cwd c:\SCHEDULE_R\scheduleR-master

Task locking feature

Hi,
Any built-in support to detect file lock and not run the same script while it is still running?
My current cron task (non via scheduleR) is:

flock -w 1 /tmp/scriptname.lockfile Rscript scriptname.R 2>> error_log.Rout 1>> cat_log.Rout

I wonder if you can support/improve/simplify it somehow? as it is quite critical feature.

Run script within project directory

R tasks and reports are sometimes bigger than a single script or rmarkdown file. They often source scripts and data from elsewhere, say from a project directory or common assets folder. Could this project be expanded to include the scheduling of scripts already uploaded onto one's scheduleR server, and to run these scripts from within their project directory?

How to recover lost username

How to recover lost username, I'm out of clues for my previous user name, can I create new?

'enabled reports' should link to report list

now links to task list...

mail msg body not send

Information entered into report task not showing up, along with no such file or directory.

So I have just installed scheduleR and now want to create a task to send a report to myself. I have gone to the reports section and selected create a new report. I enter all the required information, including the .rmd file, emails to send the report and notifications to, and the time to schedule it. After pushing "save" I am taken back out to the reports overview page where I can see the details of the report. The issue is that not much of the information I entered seems to be persisted, and what I end up seeing looks like this:

Description: test report 1
Filename:
Arguments:
Cron: 49 14 * * *
Activated: enabled
Next runtime: Aug 6, 2015 2:49:00 PM
Notify on success
Notify on error
Mail report to

In addition to this, when the process runs I get the following error.

An error was detected.
<simpleError in tools::file_path_as_absolute(input): file '/path/to/scripts/' does not exist>
Warning message:
In normalizePath(input, winslash = "/") :
path[1]="/path/to/scripts/": No such file or directory

My guess is that the file path was not persisted and so when trying to run the report it doesnt find it. Let me know if you need more information.

mail to adresses list from detailed report view

overlooked when separating tasks and reports

Can't start scheduleR [Mac OSX]

Great tool and idea. Sadly, scheduleR won't start on my Mac.
Imho all components have been correctly installed.
Error logs are attached.

npm-debug.log:

0 info it worked if it ends with ok
1 verbose cli [ 'node', '/usr/local/bin/npm', 'start' ]
2 info using [email protected]
3 info using [email protected]
4 verbose run-script [ 'prestart', 'start', 'poststart' ]
5 info prestart [email protected]
6 info start [email protected]
7 verbose unsafe-perm in lifecycle true
8 info [email protected] Failed to exec start script
9 verbose stack Error: [email protected] start: `node bin/scheduler.bin.js`
9 verbose stack Exit status 8
9 verbose stack     at EventEmitter.<anonymous> (/usr/local/lib/node_modules/npm/lib/utils/lifecycle.js:213:16)
9 verbose stack     at EventEmitter.emit (events.js:98:17)
9 verbose stack     at ChildProcess.<anonymous> (/usr/local/lib/node_modules/npm/lib/utils/spawn.js:14:12)
9 verbose stack     at ChildProcess.emit (events.js:98:17)
9 verbose stack     at maybeClose (child_process.js:766:16)
9 verbose stack     at Process.ChildProcess._handle.onexit (child_process.js:833:5)
10 verbose pkgid [email protected]
11 verbose cwd /Users/mhn/scheduleR
12 error Darwin 14.1.0
13 error argv "node" "/usr/local/bin/npm" "start"
14 error node v0.10.35
15 error npm  v2.6.1
16 error code ELIFECYCLE
17 error [email protected] start: `node bin/scheduler.bin.js`
17 error Exit status 8
18 error Failed at the [email protected] start script 'node bin/scheduler.bin.js'.
18 error This is most likely a problem with the scheduler-for-r package,
18 error not with npm itself.
18 error Tell the author that this fails on your system:
18 error     node bin/scheduler.bin.js
18 error You can get their info via:
18 error     npm owner ls scheduler-for-r
18 error There is likely additional logging output above.
19 verbose exit [ 1, true ]

Hardware:

    Hardware Overview:

      Model Name: MacBook Air
      Model Identifier: MacBookAir4,2
      Processor Name: Intel Core i5
      Processor Speed: 1,7 GHz
      Number of Processors: 1
      Total Number of Cores: 2
      L2 Cache (per Core): 256 KB
      L3 Cache: 3 MB
      Memory: 4 GB

Software:

    System Software Overview:

      System Version: OS X 10.10.2 (14C109)
      Kernel Version: Darwin 14.1.0
      Boot Volume: Macintosh HD
(...)

Files are only copied to specified location when email adresses are specified

Should be independent...

html report aren't renamed to original filename

No email capability so no new users

I have ScheduleR in an environment where it cannot access the internet (but it can be accessed from the intranet). I don't want to setup a mail server (in fact I don't know if I could) so I currently have no way of adding a user account. Is there a manual process I can go through? I'm happy to use MongoDB.

It's installed on Windows Server 2012.

consistent url shiny app

shiny url switches to portnumber instead of URL suffix

dist version cannot find glyph resources

report isn't run when months are contained in cron string

noted by @OliverBLMS

list view status doesn't correctly show status of last log