hammerlab / path-utils

Scala convenience-wrapper for java.nio.file.Path

paths

Serializable wrapper for java.nio.file.Path

Filesystem-path convenience methods and bug-fixes:

import hammerlab.path._
val path = Path("")    // current directory; URIs and Strings accepted
path.exists            // true
path / 'src list       // list children of src dir: Iterator(src/main,src/test)

Examples

Existence / Child-resolution

val src = path / 'src  // "src" subdir
src.exists             // true
path / 'foo exists     // false

Directory-traversal

(path / 'src / 'test walk) filter (_.isFile)
// Iterator(
//   src/test/resources/log4j.properties,
//   src/test/resources/META-INF/services/java.nio.file.spi.FileSystemProvider,
//   src/test/scala/org/hammerlab/paths/PathTest.scala
// )

IO

Self-explanatory methods:

• def inputStream: InputStream
• def outputStream: OutputStream
• def read: String
• def readBytes: Array[Byte]
• def write(String)
• def writeLines(Iterable[String])
• def lines: Iterator[String]
• def size: Long

Serializability

Paths are Serializable, delegating to the SerializablePath proxy, which serializes them as URIs.

JSR203 FileSystemProvider improvements

Interacting with Java NIO FileSystemProviders from Scala is broken; see scala/bug#10247.

Paths from this repo have hooks to lazily initialize and tweak the FileSystemProvider initialization code to resolve these problems; see FileSystems.scala.

There is also fix for a missing newByteChannel implementation in the JRE's default implementation of JarFileSystemProvider; use of Paths replaces it with a custom JarFileSystemProvider with this issue resolved.

As an example, try downloading the Google Cloud Storage NIO connector JAR, and querying for the existence and size of a public file:

wget -O lib/gcs-nio.jar http://search.maven.org/remotecontent?filepath=com/google/cloud/google-cloud-nio/0.28.0-alpha/google-cloud-nio-0.28.0-alpha-shaded.jar
sbt console

import hammerlab.path._
val index = Path("gs://gcp-public-data-landsat/index.csv.gz")

index.exists  // true
index.size    // 476118237

In contrast, trying the equivalent using vanilla Java NIO paths fails:

import java._, nio.file._, net._
val uri = new URI("gs://gcp-public-data-landsat/index.csv.gz")

Paths.get(uri)
// java.nio.file.FileSystemNotFoundException: Provider "gs" not installed
//   at java.nio.file.Paths.get(Paths.java:147)
//   ... 42 elided

(test this in a fresh sbt console, as initializing a hammerlab Path fixes the providers in that JVM)