Liquibase-impala is a Liquibase extension, which adds support for Impala and Hive.
As of version 1.1.x the plugin was tested and should work with the following versions of external dependencies:
| Dependency | Versions |
|---|---|
| Liquibase | 3.5.2, 3.5.3 |
| Impala JDBC driver | Cloudera Impala JDBC 2.5.32 |
| Hive JDBC driver | Cloudera Impala Hive 2.5.18 |
Version 1.2.x
| Dependency | Versions |
|---|---|
| Liquibase | 3.5.2 |
| Impala JDBC driver | Cloudera Impala JDBC 2.6.4 |
| Hive JDBC driver | Cloudera Impala Hive 2.6.2 |
Other configurations are likely to work too so you are encouraged to test with your versions. Let us know when you do!
As of version 1.1.x liquibase-impala depends on proprietary Cloudera connectors for Impala and Hive. These are not present in any public Maven repositories. Therefore, to build and install the plugin, you must do the following:
- Download Impala JDBC driver and its dependencies from http://www.cloudera.com/downloads/connectors/impala/jdbc/2-5-32.html
- Download Hive JDBC driver from http://www.cloudera.com/downloads/connectors/hive/jdbc/2-5-18.html
- Unpack and install the following dependencies in your local Maven repository, using standard Maven command:
mvn install:install-file -Dfile=${file} -DgroupId=${groupId} -DartifactId=${artifactId} -Dversion=${version} -Dpackaging=jar
| file | groupId | artifactId | version |
|---|---|---|---|
| ql.jar | com.cloudera.impala.jdbc | ql | 2.5.32 |
| hive_metastore.jar | com.cloudera.impala.jdbc | hive_metastore | 2.5.32 |
| hive_service.jar | com.cloudera.impala.jdbc | hive_metastore | 2.5.32 |
| ImpalaJDBC41.jar | com.cloudera.impala.jdbc | ImpalaJDBC41.jar | 2.5.32 |
| TCLIServiceClient.jar | com.cloudera.impala.jdbc | TCLIServiceClient.jar | 2.5.32 |
| HiveJDBC41.jar | com.cloudera.hive.jdbc | HiveJDBC41.jar | 2.5.18 |
- (optional, but recommended) Deploy the above artifacts to an internal, private Maven repository such as Nexus or Artifactory, for subsequent use.
- Build liquibase-impala by executing
mvn clean install. This will install liquibase-impala in your local Maven repo and create a liquibase-impala.jar fat-jar in the target/ directory. - (optional, but recommended) Deploy liquibase-impala to your internal, private Maven repository.
- Download Impala JDBC driver and its dependencies from https://www.cloudera.com/downloads/connectors/impala/jdbc/2-6-4.html
- Download Hive JDBC driver from https://www.cloudera.com/downloads/connectors/hive/jdbc/2-6-2.html
- Unpack and install the following dependencies in your local Maven repository, using standard Maven command:
mvn install:install-file -Dfile=${file} -DgroupId=${groupId} -DartifactId=${artifactId} -Dversion=${version} -Dpackaging=jar
| file | groupId | artifactId | version |
|---|---|---|---|
| ImpalaJDBC41.jar | com.cloudera.impala.jdbc | ImpalaJDBC41 | 2.6.4 |
| HiveJDBC41.jar | com.cloudera.hive.jdbc | HiveJDBC41 | 2.6.2 |
- (optional, but recommended) Deploy the above artifacts to an internal, private Maven repository such as Nexus or Artifactory, for subsequent use.
- Build liquibase-impala by executing
mvn clean install. This will install liquibase-impala in your local Maven repo and create a liquibase-impala.jar fat-jar in the target/ directory. - (optional, but recommended) Deploy liquibase-impala to your internal, private Maven repository.
There are two distinct ways liquibase-impala can be used to manage your Impala or Hive database.
To use liquibase-impala in concert with liquibase-maven-plugin:
- Make sure liquibase-impala is present in your local or remote (internal) Maven repo.
- Add the following to your
pom.xmlfile:
<build>
<plugins>
<!-- (...) -->
<plugin>
<groupId>org.liquibase</groupId>
<artifactId>liquibase-maven-plugin</artifactId>
<version>${liquibase.version}</version>
<dependencies>
<!-- (...) -->
<dependency>
<groupId>org.liquibase.ext.impala</groupId>
<artifactId>liquibase-impala</artifactId>
<version>${liquibase.impala.version}</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>- Run Liquibase as you normally would using Maven plugin, for example:
mvn liquibase:update \
-Dliquibase.changeLogFile=changelog/changelog.xml \
-Dliquibase.driver=com.cloudera.hive.jdbc41.HS2Driver \
-Dliquibase.username=<user>
-Dliquibase.password=<password>
-Dliquibase.url=jdbc:hive2://<host>:<port>/<database>;UID=<user>;UseNativeQuery=1
- Make sure that
liquibaseis on your$PATH - Modify
liquibase.propertiesaccording to your Impala/Hive endpoint - Put liquibase-impala fat-jar on your classpath, f.e. under the
${LIQUIBASE_HOME}/lib - Start migration, f.e.:
liquibase update
Liquibase-impala provides a number of additional configuration parameters that can be used to influence its behaviour:
| parameter | values | description |
|---|---|---|
| liquibase.lock | true (default), false | enables/disables locking facility for a given job |
| liquibase.syncDDL | true (default), false | wraps every statement with SYNC_DDL |
Script examples/run.sh performs basic integration testing of Impala and Hive, which includes:
- update execution
- tag execution
- rollback execution
The script can be executed with the command ./run.sh <both|hive|impala> PATH_TO_LIQUIBASE_HOME
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent