calvinmetcalf / syncthrough

Transform your data as it pass by, synchronously.

syncthrough    

Transform your data as it pass by, synchronously.

syncthrough is a synchronous transform stream, similar to Transform stream and through2, but with a synchronous processing function. syncthrough enforces backpressure, but it maintain no internal buffering, allowing much greater throughput. In fact, it delivers 10x performance over a standard Transform.

Because of the caveats, it is best used in combination of pipe() or pump().

Install

npm i syncthrough --save

Example

'use strict'

var fs = require('fs')
var syncthrough = require('syncthrough')

fs.createReadStream(__filename)
  .pipe(syncthrough(function (chunk) {
    // there is no callback here
    // you can return null to end the stream
    // returning undefined will let you skip this chunk
    return chunk.toString().toUpperCase()
  }))
  .pipe(process.stdout, { end: false })

API

syncthrough([transform(chunk)], [flush()])

Returns a new instance of syncthrough, where transform(chunk) is the transformation that will be applied to all incoming chunks.

The default transform function is:

function (chunk) {
  return chunk
}

If it returns null, the stream will be closed. If it returns undefined, the chunk will be skipped.

There is currently no way to split an incoming chunk into multiple chunks.

The flush() function will be called before the transform sends end() on the destination.

syncthrough([transform(object)], [flush()])

Returns a new instance of syncthrough, where transform(object) is the transformation that will be applied to all incoming objects.

Syncthrough is compatible with Streams in Object Mode, the API is exactly the same, simply expect objects instead of buffer chunks.

instance.push(chunk)

Push a chunk to the destination.

Caveats

The API is the same of a streams 3 Transform, with some major differences:

1. backpressure is enforced, and the instance performs no buffering, e.g. when write() cannot be called after it returns false or it will throw (you need to wait for a 'drain' event).
2. It does not inherits from any of the Streams classes, and it does not have _readableState nor _writableState.
3. it does not have a read(n) method, nor it emits the 'readable' event, the data is pushed whenever ready.

Acknowledgements

This project was kindly sponsored by nearForm.

License

MIT