gglanzani / hmsclient

HMSClient

This project aims to be an up to date Python client to interact with the Hive metastore using the Thrift protocol.

Installation

Install it with pip install hmsclient or directly from source

python setup.py install

Usage

Using it from Python is simple:

from hmsclient import hmsclient
client = hmsclient.HMSClient(host='localhost', port=9083)
with client as c:
    c.check_for_named_partition('db', 'table', 'date=20180101')

Regenerate the Python thrift library

The hmsclient.py is just a thin wrapper around the generated Python code to interact with the metastore through the Thrift protocol.

To regenerate the code using a newer version of the .thrift files, you can use generate.py (note: you need to have thrift installed, see here)

python generate.py --help

Usage: generate.py [OPTIONS]

Options:
  --fb303_url TEXT      The URL where the fb303.thrift file can be downloaded
  --metastore_url TEXT  The URL where the hive_metastore.thrift file can be
                        downloaded
  --package TEXT        The package where the client should be placed
  --subpackage TEXT     The subpackage where the client should be placed
  --help                Show this message and exit.

Otherwise the defaults will be used.

hmsclient's Issues

Missing tests

Hi,

I'm confused. From the setup.py I see that it will install pytest for dev/test, but I can't find any tests. Any idea?

https://github.com/gglanzani/hmsclient/blob/master/setup.py#L44-L45

Cheers, Fokko

Can we have a flexibility to pass s3 location as a parameter rather than creating it using partitionKey

https://github.com/gglanzani/hmsclient/blob/master/hmsclient/hmsclient.py#L135

get_partitions_by_filter spec catch error when partition like '$date=20220315'

error message like
hmsclient.genthrift.hive_metastore.ttypes.MetaException: MetaException(message="Error parsing partition filter : line 1:0 no viable alternative at character '$'")

hmsclient doesn't set ugi on connection open with hms

Our hms users through hmsclient couldn't pass their user names to the server, which makes it difficult to debug when hms server side has some perf issue due to user queries.
We want to expose user and group parameters from hmsclient so there is a way from the client side to pass in the user info without having to add a wrapper on top of hmsclient.