[API] Add convenience wrappers for Python about jgit-spark-connector

[bzz]

Not sure if implicits are supported by py4j call - in that case we need .scala wrapper.

[erizocosmico]

• Python wrapper
  • Add jar automagically
  • configure DataSource
  • select/join
  • wrapper for the proposed API
• Scala side
  • check our api works the way it was described in the document

[erizocosmico]

I've been playing a little bit with our spark api in pyspark and it seems that we might not need the scala wrapper. Even though implicits don't work on pyspark we can always import the Implicits class and manually use it in the python wrapper.

from py4j.java_gateway import java_import

jvm = sparkSession.sparkContext._gateway.jvm

java_import(jvm, 'tech.sourced.api.Implicits')

implicits = jvm.tech.sourced.api.Implicits

sparkSession.conf.set('tech.sourced.api.repositories.path', '/siva/files/path')

reposDf = implicits.getDataSource('repositories', sparkSession._jsparkSession)
refsDf = implicits.ApiDataFrame(reposDf).getReferences()
commitsDf = implicits.ApiDataFrame(refsDf).getCommits()
filesDf = implicits.ApiDataFrame(commitsDf).getFiles()

NOTE: ApiDataFrame can only be instantiated with a Java DataFrame, so if the dataframe comes from python it needs to be implicits.ApiDataFrame(pyDf._jdf) instead.

Of course, more experimenting is needed to see if this is feasible in a real api, but it seems promising.

[erizocosmico]

PR: #39