error running Aeron software about aeron

[maickrau]

Hi Sandeep,

There was an issue with the snakemake file. Could you please try again with the newest version from this repo?

[sandybioteck]

Hi,

I am getting following error now while running the software
(I am using following version of dependency software: vg version 1.22, snakemake version 5.2.2, python version 3.6.6)
Please help.

sandeep@Aeron: snakemake --cores=1
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1
Rules claiming more threads will be scaled down.
Job counts:
count jobs
2 align
1 all
1 assign_reads_to_transcripts
1 convert_json
1 find_best_assignments
1 find_unambiguous_assignments
1 generateCountMatrix
2 output_alignment_statistics
1 output_assignment_statistics
2 postprocess
13

[Fri Apr 3 12:53:33 2020]
rule align:
input: input/hg38.gfa, input/SampleDataset.fq
output: output/aln_SampleDataset_hg38_all.gam
log: tmp/aligner_stdout_SampleDataset_hg38.txt, tmp/aligner_stderr_SampleDataset_hg38.txt
jobid: 6
benchmark: benchmark/aln_SampleDataset_hg38_all.txt
wildcards: reads=SampleDataset, graph=hg38

[Fri Apr 3 12:53:49 2020]
Finished job 6.
1 of 13 steps (8%) done

[Fri Apr 3 12:53:49 2020]
rule postprocess:
input: output/aln_SampleDataset_hg38_all.gam, input/SampleDataset.fq
output: output/aln_SampleDataset_hg38_selected.gam, output/aln_SampleDataset_hg38_full_length.gam, tmp/run_SampleDataset_hg38_summary.txt
jobid: 2
benchmark: benchmark/postprocess_SampleDataset_hg38.txt
wildcards: reads=SampleDataset, graph=hg38

[Fri Apr 3 12:53:51 2020]
Finished job 2.
2 of 13 steps (15%) done

[Fri Apr 3 12:53:51 2020]
rule align:
input: input/hg38.gfa, input/ensembl89HomosapiensGRCh38cdnaall.fasta
output: output/aln_ensembl89HomosapiensGRCh38cdnaall_hg38_all.gam
log: tmp/aligner_stdout_ensembl89HomosapiensGRCh38cdnaall_hg38.txt, tmp/aligner_stderr_ensembl89HomosapiensGRCh38cdnaall_hg38.txt
jobid: 12
benchmark: benchmark/aln_ensembl89HomosapiensGRCh38cdnaall_hg38_all.txt
wildcards: reads=ensembl89HomosapiensGRCh38cdnaall, graph=hg38

[Fri Apr 3 12:54:17 2020]
Finished job 12.
3 of 13 steps (23%) done

[Fri Apr 3 12:54:17 2020]
rule postprocess:
input: output/aln_ensembl89HomosapiensGRCh38cdnaall_hg38_all.gam, input/ensembl89HomosapiensGRCh38cdnaall.fasta
output: output/aln_ensembl89HomosapiensGRCh38cdnaall_hg38_selected.gam, output/aln_ensembl89HomosapiensGRCh38cdnaall_hg38_full_length.gam, tmp/run_ensembl89HomosapiensGRCh38cdnaall_hg38_summary.txt
jobid: 5
benchmark: benchmark/postprocess_ensembl89HomosapiensGRCh38cdnaall_hg38.txt
wildcards: reads=ensembl89HomosapiensGRCh38cdnaall, graph=hg38

[Fri Apr 3 12:54:20 2020]
Finished job 5.
4 of 13 steps (31%) done

[Fri Apr 3 12:54:20 2020]
rule assign_reads_to_transcripts:
input: output/aln_SampleDataset_hg38_selected.gam, output/aln_ensembl89HomosapiensGRCh38cdnaall_hg38_full_length.gam, input/SampleDataset.fq
output: output/matrix_SampleDataset_ensembl89HomosapiensGRCh38cdnaall_hg38_all.txt
jobid: 11
benchmark: benchmark/assign_reads_to_transcript_SampleDataset_ensembl89HomosapiensGRCh38cdnaall_hg38.txt
wildcards: reads=SampleDataset, transcripts=ensembl89HomosapiensGRCh38cdnaall, graph=hg38

[Fri Apr 3 12:54:21 2020]
Finished job 11.
5 of 13 steps (38%) done

[Fri Apr 3 12:54:21 2020]
rule output_alignment_statistics:
input: tmp/run_ensembl89HomosapiensGRCh38cdnaall_hg38_summary.txt, tmp/aligner_stdout_ensembl89HomosapiensGRCh38cdnaall_hg38.txt, tmp/aligner_stderr_ensembl89HomosapiensGRCh38cdnaall_hg38.txt
output: output/alignmentstats_ensembl89HomosapiensGRCh38cdnaall_hg38.txt
jobid: 3
wildcards: reads=ensembl89HomosapiensGRCh38cdnaall, graph=hg38

Job counts:
count jobs
1 output_alignment_statistics
1
[Fri Apr 3 12:54:22 2020]
Finished job 3.
6 of 13 steps (46%) done

[Fri Apr 3 12:54:22 2020]
rule output_alignment_statistics:
input: tmp/run_SampleDataset_hg38_summary.txt, tmp/aligner_stdout_SampleDataset_hg38.txt, tmp/aligner_stderr_SampleDataset_hg38.txt
output: output/alignmentstats_SampleDataset_hg38.txt
jobid: 9
wildcards: reads=SampleDataset, graph=hg38

Job counts:
count jobs
1 output_alignment_statistics
1
[Fri Apr 3 12:54:23 2020]
Finished job 9.
7 of 13 steps (54%) done

[Fri Apr 3 12:54:23 2020]
rule find_unambiguous_assignments:
input: output/matrix_SampleDataset_ensembl89HomosapiensGRCh38cdnaall_hg38_all.txt
output: output/matrix_SampleDataset_ensembl89HomosapiensGRCh38cdnaall_hg38_unambiguous.txt
jobid: 1
benchmark: benchmark/matrix_unambigous_SampleDataset_ensembl89HomosapiensGRCh38cdnaall_hg38.txt
wildcards: runid=SampleDataset_ensembl89HomosapiensGRCh38cdnaall_hg38

[Fri Apr 3 12:54:23 2020]
Finished job 1.
8 of 13 steps (62%) done

[Fri Apr 3 12:54:23 2020]
**rule convert_json:
input: output/aln_SampleDataset_hg38_selected.gam
output: output/aln_SampleDataset_hg38_selected.json
jobid: 7
benchmark: benchmark/convertjson_SampleDataset_hg38.txt
wildcards: reads=SampleDataset, graph=hg38

vg: variation graph tool, version v1.22.0 "Rotella"

usage: vg [options]

main mapping and calling pipeline:
-- construct graph construction
-- index index graphs or alignments for random access or mapping
-- map MEM-based read alignment
-- augment augment a graph from an alignment
-- pack convert alignments to a compact coverage index
-- call call or genotype VCF variants
-- help show all subcommands

For more commands, type vg help.
For technical support, please visit: https://www.biostars.org/t/vg/

**[Fri Apr  3 12:54:24 2020]
Error in rule convert_json:
    jobid: 7
    output: output/aln_SampleDataset_hg38_selected.json

RuleException:
CalledProcessError in line 89 of /sfs/lustre/bahamut/scratch/ss7mh/softwares/AERON/Aeron/Snakefile:
Command ' set -euo pipefail; ~/bin/vg view -a output/aln_SampleDataset_hg38_selected.gam > output/aln_SampleDataset_hg38_selected.json ' returned non-zero exit status 1.
File "/sfs/lustre/bahamut/scratch/ss7mh/softwares/AERON/Aeron/Snakefile", line 89, in __rule_convert_json
File "/apps/software/standard/core/anaconda/5.2.0-py3.6/lib/python3.6/concurrent/futures/thread.py", line 56, in run
Removing output files of failed job convert_json since they might be corrupted:
output/aln_SampleDataset_hg38_selected.json
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message****

[sandybioteck]

the above problem was solved. However now getting this error (possibly because of python indentation problem, however, I can't find it).

[Fri Apr 3 14:27:01 2020]
rule generateCountMatrix:
input: output/matrix_SampleDataset_ensembl89HomosapiensGRCh38cdnaall_hg38_bestmatch.txt, input/HomosapiensGRCh3889.gtf, output/aln_SampleDataset_hg38_selected.json
output: output/CountMatrix_SampleDataset_ensembl89HomosapiensGRCh38cdnaall_hg38.txt
jobid: 3
benchmark: benchmark/generateCount_SampleDataset_ensembl89HomosapiensGRCh38cdnaall_hg38.txt
wildcards: reads=SampleDataset, transcripts=ensembl89HomosapiensGRCh38cdnaall, graph=hg38

Traceback (most recent call last):
File "AeronScripts/ThreePrime.py", line 3, in
from ParseGTF import *
File "/sfs/lustre/bahamut/scratch/ss7mh/softwares/AERON/Aeron/AeronScripts/ParseGTF.py", line 69
def getTranscriptPosition(self, key):
^
TabError: inconsistent use of tabs and spaces in indentation
[Fri Apr 3 14:27:01 2020]
Error in rule generateCountMatrix:
jobid: 3
output: output/CountMatrix_SampleDataset_ensembl89HomosapiensGRCh38cdnaall_hg38.txt

RuleException:
CalledProcessError in line 201 of /sfs/lustre/bahamut/scratch/ss7mh/softwares/AERON/Aeron/Snakefile:
Command ' set -euo pipefail; python AeronScripts/ThreePrime.py -g input/HomosapiensGRCh3889.gtf -m output/matrix_SampleDataset_ensembl89HomosapiensGRCh38cdnaall_hg38_bestmatch.txt -j output/aln_SampleDataset_hg38_selected.json >> output/CountMatrix_SampleDataset_ensembl89HomosapiensGRCh38cdnaall_hg38.txt ' returned non-zero exit status 1.
File "/sfs/lustre/bahamut/scratch/ss7mh/softwares/AERON/Aeron/Snakefile", line 201, in __rule_generateCountMatrix
File "/apps/software/standard/core/anaconda/5.2.0-py3.6/lib/python3.6/concurrent/futures/thread.py", line 56, in run
Removing output files of failed job generateCountMatrix since they might be corrupted:
output/CountMatrix_SampleDataset_ensembl89HomosapiensGRCh38cdnaall_hg38.txt
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: /sfs/lustre/bahamut/scratch/ss7mh/softwares/AERON/Aeron/.snakemake/log/2020-04-03T142607.895574.snakemake.log

[sandybioteck]

After correcting the indentation problem in one of the python scripts, the processes successfully completed, however many of the output files are empty

ls -lrt
total 7
-rw-r--r-- 1 ss7mh users 21 Apr 3 15:38 aln_both_hg38_all.gam
-rw-r--r-- 1 ss7mh users 0 Apr 3 15:38 aln_both_hg38_selected.gam
-rw-r--r-- 1 ss7mh users 0 Apr 3 15:38 aln_both_hg38_full_length.gam
-rw-r--r-- 1 ss7mh users 21 Apr 3 15:39 aln_ensembl89HomosapiensGRCh38cdnaall_hg38_all.gam
-rw-r--r-- 1 ss7mh users 0 Apr 3 15:39 aln_ensembl89HomosapiensGRCh38cdnaall_hg38_selected.gam
-rw-r--r-- 1 ss7mh users 0 Apr 3 15:39 aln_ensembl89HomosapiensGRCh38cdnaall_hg38_full_length.gam
-rw-r--r-- 1 ss7mh users 0 Apr 3 15:39 matrix_both_ensembl89HomosapiensGRCh38cdnaall_hg38_all.txt
-rw-r--r-- 1 ss7mh users 1002 Apr 3 15:39 alignmentstats_ensembl89HomosapiensGRCh38cdnaall_hg38.txt
-rw-r--r-- 1 ss7mh users 0 Apr 3 15:39 matrix_both_ensembl89HomosapiensGRCh38cdnaall_hg38_bestmatch.txt
-rw-r--r-- 1 ss7mh users 947 Apr 3 15:39 alignmentstats_both_hg38.txt
-rw-r--r-- 1 ss7mh users 0 Apr 3 15:39 aln_both_hg38_selected.json
-rw-r--r-- 1 ss7mh users 294 Apr 3 15:39 matrixstats_both_ensembl89HomosapiensGRCh38cdnaall_hg38.txt
-rw-r--r-- 1 ss7mh users 0 Apr 3 15:39 matrix_both_ensembl89HomosapiensGRCh38cdnaall_hg38_unambiguous.txt
-rw-r--r-- 1 ss7mh users 17 Apr 3 15:39 CountMatrix_both_ensembl89HomosapiensGRCh38cdnaall_hg38.txt