Asan: heap-buffer-overflow in `combineFilterAndIndices` (`enable_vertical_final`) about clickhouse

[canhld94]

I will fix this asap

[canhld94]

@tavplubix can we get some information about the query that triggered the issue?

[canhld94]

We're running some tests to find if we can reproduce the issue. Nevertheless, it's likely that even without this bug, we will remove combineFilterAndIndices in FilterTransform.

Remind that for vertical FINAL, there're two major steps:

• (1) in merging final step, we don't copy selected row but remember the position of selected row and generate a filter (in the form of list of indices).
• (2) if query has WHERE, we will merge this filter with the filter generated by WHERE expression in FilterTransform, otherwise we have a filter step right after merging final step.

The feature has been tested thoroughly in our environment. During testing, we hardly found any issue with (1), but we found two issues with (2), including 1 bug and 1 potential performance issue:

• The bug is if there's an additional steps between merging final and filter transform, those steps may change number of rows in the chunk and (2) will not work correctly anymore. At that time, the case we found is when query has ARRAY JOIN before filter. We fixed the bug by checking if query has any ARRAY JOIN before WHERE before applying (2). We thought that the fix is sufficient because ARRAY JOIN is the only possible expression that will change number of rows in chunk.
• The performance issue is when WHERE expression is heavy to compute, (2) may be worse than normal because we need to compute WHERE on more rows.

Nevertheless, we decided to keep (2). But now with this bug, I think it's rational to remove (2). The bug appears to me that the number of rows in chunk produced by merging final step has been changed before it reaches FilterTransform. And to be honest, at this point I'm not 100% sure that ARRAY JOIN is the only possible expression that can change number of rows in a chunk, or how we can guarantee that the FilterTransform right after merging final step is produced by WHERE.

So in stead of doing (2), there is a more solid and less error-prone optimization: to leverage the PREWHERE filter in the chunk (if any) to skip row in merging final step.

@tavplubix @nickitat @KochetovNicolai would like to hear your opinion.
cc @jorisgio