Note that generation of QC objects runs to completion, need to use use_cache option to finish with normalization. Laptop has low memory so consider reducing number of cores and increase docker memory. Should be able to work because got over half way through QC process. May need to make normalization more memory efficient. Almost there!!!
$plot
Check QC report and select number of PCs. Will add option in future to adjust thresholds.
/usr/local/lib/python3.6/dist-packages/rpy2/robjects/pandas2ri.py:191: FutureWarning: from_items is deprecated. Please use DataFrame.from_dict(dict(items), ...) instead. DataFrame.from_dict(OrderedDict(items)) may be used to preserve the key order.
res = PandasDataFrame.from_items(items)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: Error in FUN(X[[i]], ...) :
The operating system has decided that some forks of mclapply are using too much memory.
Try reducing the max.bytes parameter or the R option 'mc.cores'.
warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: In addition:
warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: There were 29 warnings (use warnings() to see them)
warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning:
warnings.warn(x, RRuntimeWarning)
Traceback (most recent call last):
File "/usr/local/bin/pymethyl-preprocess", line 10, in <module>
sys.exit(preprocess())
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/pymethylprocess/preprocess.py", line 280, in preprocess_pipeline
preprocesser.preprocessMeffil(n_cores=n_cores,n_pcs=n_pcs,qc_report_fname=os.path.join(output_dir,'qc.report.html'), normalization_report_fname=os.path.join(output_dir,'norm.report.html'), pc_plot_fname=os.path.join(output_dir,'pc.plot.pdf'), useCache=use_cache, qc_only=qc_only, qc_parameters=qc_parameters)
File "/usr/local/lib/python3.6/dist-packages/pymethylprocess/PreProcessDataTypes.py", line 453, in preprocessMeffil
pval_beadnum=load_detection_p_values_beadnum(qc_list,n_cores)
File "/usr/local/lib/python3.6/dist-packages/pymethylprocess/meffil_functions.py", line 64, in load_detection_p_values_beadnum
}""")(qc_list, n_cores)
File "/usr/local/lib/python3.6/dist-packages/rpy2/robjects/functions.py", line 178, in __call__
return super(SignatureTranslatedFunction, self).__call__(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/rpy2/robjects/functions.py", line 106, in __call__
res = super(Function, self).__call__(*new_args, **new_kwargs)
rpy2.rinterface.RRuntimeError: Error in FUN(X[[i]], ...) :
The operating system has decided that some forks of mclapply are using too much memory.
Try reducing the max.bytes parameter or the R option 'mc.cores'.