Superfluous query cancelled exception about clickhouse

[cangyin]

This causes the metric FailedQuery to surge.

[Algunenano]

It might be possible to improve this, but it's expected in order to not waste resources.

[filimonov]

We also observed the same problem recently. After the upgrade of one of the clusters from 22.3 to 23.8 the number of QUERY_WAS_CANCELLED for non-initial queries jumped high (the same query always).

The real reason was not the LIMIT, but the INNER JOIN where the right side returned the empty set. (after that the left subquery was cancelled, which is good, but it was not obvious - required deeper look to debug / understand).

Initiator-side message was also not transparent enough

StorageDistributed (...): (10.166.2.140:9000) Cancelling query because enough data has been read