ltrdigest_complete.fas_DfamAnnotation.out' does not exist about ltrpred

[a7032018]

Hi HajkD,

Yes. I'd like to run all 300K elements against Dfam for the annotation purpose.
You indicate a good idea to take one representative rather than go through many sequences sharing homology. I will give a try.

Thanks!

[HajkD]

Fatal exception (source file esl_hmm.c, line 198):
malloc of size -307968 failed
Aborted (core dumped)
Error running command:

It seems like you don't have enough memory to perform the search. I would recommend to use a computer with more RAM.

I hope this helps.

[sadikmu]

Thanks for that pointing out. I run it on 500G RAM machine with 32 cpu still getting the same error. I am not aware of how to allocate memory size to LTRpred in the parameterization options listed in the documentation.

Another challenge, is there a tweak to set up LTRpred to pick dfamscan from local installation or conda version which could help to test it in a server machine where one doesn't have admin privilege to install dfamscan in /usr/local/bin/?

[sadikmu]

Any suggestions on this, please?

[HajkD]

Hi Sadik,

Since this is an issue coming from the dfamscan script provided by the Dfam community, I would suggest to contact them.

Alternatively, have you tried running the failed command? :

nhmmscan --noali -E 0.001 --dfamtblout /tmp/nXeK2iJYcP --cpu=16 dfam/Dfam.hmm /home/ltrpred/epo_ltrdigest/epo-ltrdigest_complete.fas

Maybe this yields more comprehensive error messages?

For me to understand, does the yeast example from the documentation work for you on this machine or does it also fail? It clearly seems to be a memory assignment issue and now you would need to troubleshoot where this comes from in detail.

Regarding your question:

Another challenge, is there a tweak to set up LTRpred to pick dfamscan from local installation or conda version which could help to test it in a server machine where one doesn't have admin privilege to install dfamscan in /usr/local/bin/?

You can run the dfamscan.pl directly without sudo rights in any folder by typing:

perl dfamscan.pl -fastafile [[seq_file]] -hmmfile path/to/Dfam.hmm -dfam_outfile DfamAnnotation.out -E 1E-5 -cpu 16 --log_file logfile.txt --masking_thresh

I hope this helps.

Cheers,
Hajk

[a7032018]

I am facing the same issue when feeding 300K sequences to LTRpred Dfam scan with 1TB RAM, 112 threads machine.

---

Fatal exception (source file esl_hmm.c, line 198):
malloc of size -148920 failed

---

If RAM is the limitation causing the fault by feeding so many sequences, is it possible to do Dfam nhmmscan by batch? (eg. scan 1-1000 sequences -> store the result in tmp -> scan 1001-2000 sequences ->store the result in tmp -> ....->combine the chunks to the final output.

[HajkD]

Hi @a7032018

This is an excellent idea.

Do I understand correctly that LTRpred annotated 300k elements and you would like to run all 300k elements against Dfam?

By any chance, did you enable the TE family clustering option in LTRpred to check whether some elements generate huge clusters and thus only a cluster representative (family member) needs to be hmmered against the Dfam? This could be an alternative option.

Regarding the batch Dfam scans I noted it down as feature request and will work on it when time permits.