Coder Social home page Coder Social logo

solr-dih-redis's Introduction

solr-dih-redis

A Simple Solr Data Import Handler (DIH) plugin to read from Redis.

Setup

Sample solrconfig.xml
  <!-- 
        Note: Copy all your DIH Jars to the second path. 
        To pull data from Redis, you will NEED to copy  the following: 
            1) commons-pool2-x.x.x.jar
            2) jedis-x.x.x.jar 
            3) solr-dih-redis-x.x.x.jar
   -->
  <lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-dataimporthandler-.*\.jar"/>
  <lib dir="${solr.install.dir:../../../..}/contrib/dataimporthandler/lib" regex=".*\.jar"/>
Sample data-config.xml
<?xml version="1.0" encoding="UTF-8" ?>
 <dataConfig>
   <dataSource
       type="JdbcDataSource"
       driver="com.mysql.jdbc.Driver"
       url="jdbc:mysql://some_mysql_server:3306/ptn?zeroDateTimeBehavior=convertToNull"
       user="user"
       password="password"
       name="mysql"/>
       
  <dataSource
      type="RedisDataSource"
      name="redis"
      host="localhost"
      port="6379"
      ssl="false"/>
 
   <document>
     <!-- Root Table -->
     <entity
         dataSource="mysql"
         processor="SqlEntityProcessor"
         name="foo"
         pk="id"
         query="SELECT * FROM foo t"
         deltaImportQuery="SELECT * FROM foo t WHERE t.id = '${dataimporter.delta.id}'"
         deltaQuery="SELECT t.id FROM foo t WHERE t.updated_at > '${dataimporter.last_index_time}'">
       <field column="id" name="id"/>
       <field column="bar" name="bar_s"/>
 
       <!-- Dependent Key/Value Pairs -->
       <entity
           dataSource="redis"
           processor="RedisEntityProcessor"
           name="baz"
           key="some:really:long:key:namespace:${foo.id}">
         <field column="value" name="value_ss"/>
         <field column="key" name="key_ss"/>
       </entity>
   </entity>
 </document>
</dataConfig>

TODOs: 1. Maybe add some way to support deltas from Redis. 2. Add unit tests.. Sorry TDD. 3. Connection pooling and other performance stuff.

solr-dih-redis's People

Stargazers

 avatar  avatar

Watchers

 avatar

Forkers

jasonhao123

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.