tuor713 / trino-kdb Goto Github PK

During query processing table and column names appear to get converted to lower case. However, KDB expects symbols in proper case.

ConnectorSession appears to have the ability to retrieve properties from the session.

The use case would for example be to disable push down aggregation selectively for a query where it yields an incorrect or inefficient query plan.

So something like SET SESSION push.down.aggregation = false; would disable aggregation in the user's session.

Count, sum etc queries are much more efficient to run on KDB and only source aggregated results.

KDBConnectFactory#create eagerly loads the KDB connection, which gets invoked during Trino startup as part of catalog creation.

the project of javakdb seems not to be maintained.
however, a project named pykx is rising and I tested it is far more faster than javakdb, because the kdb setup a new data gateway to serve pykx request. (I don't know if my test is correct)

I found it is possible to invoke python runtime in java. and I wonder if using the pykx could lead to a better performance in latency and scale.

thanks.

presto seems to be more stable and has a more energetic community.

I have encountered a performance issue which causes the bottleneck at client side, however when I migrate to presto, the bottleneck dismissed.

I think the main structure of the plugin about trino and presto is similar, is it possible to support them both and add a compiler flag to allow user to choose the artifact.

great thanks.

Tables can have columns with empty type - i.e. no data is loaded and the column is not yet typed. This should ideally not crash the Trino process or at least give a good error message.

During push down columns get converted to lower case, which leads to an error in KDB.

Hi, might have cause to use this project. Seems like no license is indicated. Safe to assume a permissive license or did you have something else in mind?

PS: Couldn’t seem to find a way to reach out through email or anything, but would appreciate picking your brain about how well you feel kdb and trino work together sometime

Usually tables in KDB are in the top namespace .. However, in some more advanced installations tables can reside in namespaces. These could be exposed using the schema concept in Trino.