BatchJobs config file about biocparallel

[DarwinAwardWinner]

BatchJobsParam is mustly just telling BiocParallel to use the BatchJobs configuration, which is specified in the config file. I believe bpworkers doesn't return anything meaningful for BatchJobsParam because in the "normal" use case of submitting jobs to a cluster across multiple machines and shared by multiple users, BatchJobs doesn't control the level of parallelism, the cluster's job scheduler does.

[mllg]

Hi,

Both of you are right, the bpworkers setting has no purpose for BatchJobs. I'm unsure about consistency, I guess I would propose NA_integer_ as return value.

Was it intended that the user would set most parameters through the config file and not the BatchJobsParam() constructor interface?

Actually, no. We opted for configuration files to allow sysadmins to set global, system-wide options. You can always overrule these settings in your own config file or from R via setConfig(). Furthermore, configuration settings depend primarily on the system, and not on the user -- therefore, to keep the R scripts portable, configuration options should not be set in script files of the user. Ideally you should not have to touch most options once you've figured out a working configuration for your system.

Maybe we were careless in the past and some conf options should have been function arguments. These can still be exposed as arguments in BiocParallel. Do you have some specific options in mind?

[vobencha]

Hi,

In general I was hoping to lower the overhead involved in setting up the config and template files. Thinking of the less-experienced user, it would be great if they could do this completely within R.

BatchJobsParam fields:

• I'll modify the show method for BatchJobsParam so 'workers' isn't shown (and not documented).
• getConfig() is not mentioned on ?BatchJobsParam so I think it's difficult to know what config options are available. I'm not sure if the best approach is to (a) expose all of these as args or (b) document that they can be set with setConfig() or passed to BatchJobsParam() in the '...'. I'm leaning towards (b).

config and template files:

Interactive, Local and Multicore only need config file info (not template) so we're set for these.

I believe LSF, SLURM, Torque, SGE and SSH need both a template and config. Template files aren't included in BatchJobs and must be downloaded from github. For these cluster types the 'cluster.functions' arg only gets us part way there. To enable a complete R approach maybe we need a function that generates cluster-specific template files? Does something like this exist?

[vobencha]

Closing this issue.