overlapper thoughts/questions about htstream

[dstreett]

Hey, @msettles . :) I hope all is going well for you! Thanks for the comments and questions. I will get to them all today.

Welcome to the new overlapper, we are still tweaking the parameters, but it should be quite a bit faster than the old flash. It is a new algorithm compared to FLASH as well.

Take the longest of the two reads and create a dictionary/map with kmers with a certain offset (parameters –k and –r). We now use this as quick lookup table for the shorter of the two reads.

The shorter of the two reads checks perfect overlapping kmers on both sides of the reads going in a certain amount (-c, --check-length) against the dict/map. We can make this assumption because the overlap must contain the ends of the shorter read on the left or right side. Once a hit happens, a base by base comparison happens just like flash.

Hopefully that made sense, let me know if there are any questions. To answer specific question.

1 – Yep, histograms aren’t outputted automatically. This argument takes a string for the location you want it outputted to.

2 – Yup, new algorithm, I think I hit those two in the explanation above, but let me know if it is still unclear.

3 – Again, check above, and let me know if it doesn’t make sense.

4 – Yes, that is much clearer, I will change that on the help

5 – So, my logic for this one which might be totally off. If adapter trimming is on, R1 might get trimmed to the point it doesn’t meet minimum length, thus discarded. R2 is long enough after adapter trimming, so if stranded is selected, it will be RC and outputted.

Let me know if I answered your questions for overlapper. If anything is still unclear, please, let me know. Right now validation and documentation is happening so I appreciate all the input.

[msettles]

So just to expand on 5 – So, my logic for this one which might be totally off. If adapter trimming is on, R1 might get trimmed to the point it doesn’t meet minimum length, thus discarded. R2 is long enough after adapter trimming, so if stranded is selected, it will be RC and outputted. If there is an adapter, then by definition R1 and R2 are the same length, except if base quality trimming (or 5’ primer trimming) were done before overlapping, in that case R2 could be the longer of the 2 and then the “reference” and would need to be RC???? Is that more correct to say? I don’t see any way in which one read would get discarded and the other read kept in the overlapping algorithm? In your example above this would be a case of R1 wholly within R2 (R1 was radically pretrimmed), in which you would still produce a consensus sequence? And then RC, but that would also imply that R2 also has adapter that won’t be trimmed. This got complicated quick. See attached figure of the 3 possibilities, ASSUMING NO trimming before overlapping len(R1) == len(R2), have to consider the product when len(R1) != len(R2) Matt From: David Streett <[email protected]> Reply-To: ibest/HTStream <[email protected]> Date: Tuesday, February 21, 2017 at 10:59 AM To: ibest/HTStream <[email protected]> Cc: Matt Settles <[email protected]>, Mention <[email protected]> Subject: Re: [ibest/HTStream] overlapper thoughts/questions (#64) 5 – So, my logic for this one which might be totally off. If adapter trimming is on, R1 might get trimmed to the point it doesn’t meet minimum length, thus discarded. R2 is long enough after adapter trimming, so if stranded is selected, it will be RC and outputted.

[samhunter]

Hey Matt,

So yes, obviously this gets kind of complicated. I think the rule we were trying to operate under went something like this:
If stranded=True: all SE reads -- both due to overlap and due to discard from length -- should be in the orientation of R1.
So, if R1 is quality trimmed until it is the shorter read, it should still be used as the read which orients R2 in overlap so that the SE overlapped output is always in R1 orientation regardless of whether R1 or R2 was the longer read.
AND
If R1 is discarded then R2 is reported as RC in order to ensure that all outputs in the SE file are in R1 orientation.

Hopefully that seems reasonable :)

Sam

[msettles]

Agreed, it was the R1 discard statement that got me, I couldn't think of how you would discard a read in the overlapper algorithm. For overlapper, I would say the default should be R1 orientation and then you can remove the --strand flag On Feb 23, 2017 10:55 AM, "Sam Hunter" <[email protected]> wrote: Hey Matt, So yes, obviously this gets kind of complicated. I think the rule we were trying to operate under went something like this: If stranded=True: all SE reads -- both due to overlap and due to discard from length -- should be in the orientation of R1. So, if R1 is quality trimmed until it is the shorter read, it should still be used as the read which orients R2 in overlap so that the SE overlapped output is always in R1 orientation regardless of whether R1 or R2 was the longer read. AND If R1 is discarded then R2 is reported as RC in order to ensure that all outputs in the SE file are in R1 orientation. Hopefully that seems reasonable :) Sam — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/ibest/HTStream/issues/64#issuecomment-282085252>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAno5qep2nOyJ6YIFAzcEqQQpL6maliMks5rfdYJgaJpZM4MGbpN> .

[msettles]

Think this is all covered, although there is one proposed change I'll detail in a commit