pierrepo / autoclassweb

AutoClassWeb: a simple web interface for Bayesian clustering of omics data :package: :rocket:

Pierre Poulain, associate professor

I am associate professor at Université Paris Cité, France. I currently perform my research at the Laboratory of Theoretical Biochemistry. I study the sharing and reuse of molecular dynamics simulation data and code. For this, I develop new methods and software, mostly in Python, using traditional machine learning technics or deep learning. More generally, I have a strong interest in data analysis and data management in biology and bioinformatics.

Projects

Bayesian clustering

Jupyter-based platform for e-learning

Have a look to the Plasma project website.

Structural bioinformatics

Misc

autoclassweb's Issues

Automaticaly detect max number of jobs

Ideally, should be number of core - 1

Ask user of a job name

Find more usable/informative job name

Maybe something like 180930ABCDEF ?

Use logging module

Some ressources:

• https://gist.github.com/ibeex/3257877
• https://stackoverflow.com/questions/14037975/how-do-i-write-flasks-excellent-debug-log-message-to-a-file-in-production
• https://fangpenlin.com/posts/2012/08/26/good-logging-practice-in-python/

Autoremove old results

Add example how to control autoclassweb with curl

Rename output files from job name

Send mail with download link to results instead of results as attachement

Need a generated key to forge url to download results

Input log could contain log from previous run (in addition to actual one)

When 2 clusterings are run almost at the same time. The input log file of one run may contain input log from the other run.
For instance:

2020-07-22 15:43:39 :: INFO     :: Reading data file 'Proteome_random_ARATH_AA_data_Frq_Prot.txt' as 'real location' with error 0.01
2020-07-22 15:43:40 :: INFO     :: Detected encoding: ascii
2020-07-22 15:43:40 :: INFO     :: Found 39380 rows and 21 columns
2020-07-22 15:43:40 :: INFO     :: Checking data format
2020-07-22 15:43:40 :: INFO     :: Column 'Freq_G_Proteome'
2020-07-22 15:43:40 :: INFO     :: count    39380.000000
2020-07-22 15:43:40 :: INFO     :: mean         0.000326
2020-07-22 15:43:40 :: INFO     :: std          0.244267
[...]
2020-07-22 15:43:40 :: INFO     :: min         -1.000000
2020-07-22 15:43:40 :: INFO     :: 50%         -0.038956
2020-07-22 15:43:40 :: INFO     :: max         15.176349
2020-07-22 15:43:40 :: INFO     :: ---
2020-07-22 15:43:40 :: INFO     :: Final dataframe has 39380 lines and 21 columns
2020-07-22 15:43:40 :: INFO     :: Searching for missing values
2020-07-22 15:43:40 :: INFO     :: No missing values found
2020-07-22 15:43:40 :: INFO     :: Writing autoclass.db2 file
2020-07-22 15:43:40 :: INFO     :: If any, missing values will be encoded as '?'
2020-07-22 15:43:42 :: INFO     :: Writing .hd2 file
2020-07-22 15:43:42 :: INFO     :: Writing .model file
2020-07-22 15:43:42 :: INFO     :: Writing .s-params file
2020-07-22 15:43:42 :: INFO     :: Writing .r-params file
2020-07-22 15:43:42 :: INFO     :: Writing run file
2020-07-22 15:43:42 :: INFO     :: AutoClass C executable found in /app/autoclass-c/autoclass
2020-07-22 15:43:42 :: INFO     :: AutoClass C version: AUTOCLASS C (version 3.3.6unx)
2020-07-22 15:43:42 :: INFO     :: Running clustering...
2020-07-22 16:01:06 :: INFO     :: Reading data file 'Proteome_random_HUMAN_AA_data_Frq_Prot.txt' as 'real location' with error 0.01
2020-07-22 16:01:06 :: INFO     :: Detected encoding: ascii
2020-07-22 16:01:07 :: INFO     :: Found 73928 rows and 21 columns
2020-07-22 16:01:07 :: INFO     :: Checking data format
2020-07-22 16:01:07 :: INFO     :: Column 'Freq_G_Proteome'
2020-07-22 16:01:07 :: INFO     :: count    73928.000000
2020-07-22 16:01:07 :: INFO     :: mean         0.001658
2020-07-22 16:01:07 :: INFO     :: std          0.337749
[...]

This is not a desirable feature.

Add timout for autoclass job

An autoclass job should not run forever...

Add working download results link

See:

• https://stackoverflow.com/questions/40697585/flask-download-file-with-href
• https://stackoverflow.com/questions/24577349/flask-download-a-file

Implement job autocleaning

Jobs older than xx days should be deleted.

Send status error upon failed request

https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#Unofficial_codes

Rename output files sent to user (.cdt) with input filenames

Proposal :

• store input filenames in summary file
• no space in input filename
• rename output files before zip creation

Show job name in status with monospace font

font-family: monospace;

https://www.w3schools.com/Css/css_font.asp

Sort job status

Use datatables

Add input log in exported zip file

Ajust max size of input files

See https://stackoverflow.com/questions/48936687/flask-gunicron-not-letting-me-upload-large-files for Flask.

Find also config for gunicorn / Nginx.

Block job creation when max number of jobs is reached