ricomnl / bioinformatics-pipeline-tutorial

A tutorial on how to create bioinformatics pipelines as bash scripts, Makefiles and using tools like Nextflow.

Hi there 👋

Currently @ retro.bio. I'm interested in gene regulatory networks, bioelectricity and longevity.

Scientific Software

• nf-encyclopedia - A NextFlow pipeline for chromatogram library DIA proteomics workflows

Contributions:

• scvi-tools
• redun
• kb_python
• During the bitsinbio hackathon in March '22, I contributed to graphein
• EncyclopeDIA - a library search engine comprised of several algorithms for data independent acquisition (DIA) mass spectrometry data analysis: encyclopedia
• Gopher - Gene ontology enrichment analysis using protein expression: gopher

Some other stuff I've worked on:

• Semantic Podcast Search (using Lex Fridman's AI podcast)
• Accompanying code to my blog post Bioinformatics pipeline example from the bottom up: bioinformatics-pipeline-tutorial
• Instacart2vec: My bachelor thesis on using word2vec for online grocery recommendations

While participating in the How To Grow (Almost) Anything course on Synthetic Bio:

• Opentrons protocol to draw 8x8 digits in 96-well plates: opentron_digits
• Protein structure analysis of KLF4 using nglview: protein_structure_analysis | online docs

Fun projects

• Tensor Puzzle exercises put together by @srush_nlp
• WIP: Nand2Tetris

Twitter: @ricomnl

Website: https://ricomnl.com

bioinformatics-pipeline-tutorial's Issues

Redun pipeline is broken with orjson ~3.8

Hey Rico,

When downloading, installing, and running the main redun pipeline, one gets this error message:

Error message

``` [redun] Execution duration: 0.84 seconds [redun] *** Execution failed. Traceback (most recent task last): [redun] Job eea2a311: File "/Users/falexwolf/repos/bioinformatics-pipeline-tutorial/wf/workflow.py", line 27, in main [redun] def main( [redun] amino_acid = 'C' [redun] enzyme_regex = '[KR]' [redun] executor = [redun] input_dir = 'fasta/' [redun] max_length = 75 [redun] min_length = 4 [redun] missed_cleavages = 0 [redun] Job 299f076a: File "/Users/falexwolf/repos/bioinformatics-pipeline-tutorial/bioinformatics_pipeline_tutorial/lib.py", line 269, in plot_count_task [redun] def plot_count_task(input_count: File) -> File: [redun] input_count = File(path=data/KLF4.count.tsv, hash=be612cb0) [redun] File "/Users/falexwolf/repos/bioinformatics-pipeline-tutorial/bioinformatics_pipeline_tutorial/lib.py", line 278, in plot_count_task [redun] counts_plot = plot_counts(output_path, counts) [redun] File "/Users/falexwolf/repos/bioinformatics-pipeline-tutorial/bioinformatics_pipeline_tutorial/lib.py", line 108, in plot_counts [redun] fig.write_image(tmp_file.path) [redun] File "/opt/anaconda3/envs/base1/lib/python3.9/site-packages/plotly/basedatatypes.py", line 3829, in write_image [redun] return pio.write_image(self, *args, **kwargs) [redun] File "/opt/anaconda3/envs/base1/lib/python3.9/site-packages/plotly/io/_kaleido.py", line 267, in write_image [redun] img_data = to_image( [redun] File "/opt/anaconda3/envs/base1/lib/python3.9/site-packages/plotly/io/_kaleido.py", line 144, in to_image [redun] img_bytes = scope.transform( [redun] File "/opt/anaconda3/envs/base1/lib/python3.9/site-packages/kaleido/scopes/plotly.py", line 153, in transform [redun] response = self._perform_transform( [redun] File "/opt/anaconda3/envs/base1/lib/python3.9/site-packages/kaleido/scopes/base.py", line 296, in _perform_transform [redun] export_spec = self._json_dumps(dict(kwargs, data=data)).encode('utf-8') [redun] File "/opt/anaconda3/envs/base1/lib/python3.9/site-packages/kaleido/scopes/plotly.py", line 76, in _json_dumps [redun] return pio.to_json(val, validate=False, remove_uids=False) [redun] File "/opt/anaconda3/envs/base1/lib/python3.9/site-packages/plotly/io/_json.py", line 199, in to_json [redun] return to_json_plotly(fig_dict, pretty=pretty, engine=engine) [redun] File "/opt/anaconda3/envs/base1/lib/python3.9/site-packages/plotly/io/_json.py", line 126, in to_json_plotly [redun] opts = orjson.OPT_NON_STR_KEYS | orjson.OPT_SERIALIZE_NUMPY [redun] AttributeError: partially initialized module 'orjson' has no attribute 'OPT_NON_STR_KEYS' (most likely due to a circular import) ```

I tried downgrading plotly from 5.11 to 5.8, and it didn't help.

I then downgraded orjson from 3.8.1 to 3.5.4, and it fixed the issue.

If other people want to try out your code, to keep your example running, I'd suggest to pin the plotly and orjson versions for the time being - even if it's not great to pin such fundamental dependencies it's much better than an erroring workflow. ☺️