vitaly-t / spex

:bulb: Specialized Promise Extensions

Specialized Promise Extensions

batch, page, sequence - promise methods for the following patterns:

• Data Throttling & Load Balancing
• Linked and Detached Sequencing
• Streaming and Paging
• Batch Processing

Installing

$ npm i spex

Usage

• For any Promises/A+ library: Promise, Bluebird, When, Q, RSVP, etc.

const promise = require('bluebird');
const spex = require('spex')(promise);

• For ES6 Promise:

const spex = require('spex')(Promise);

See also: client-side usage.

API

• Module
• Methods
  • batch
  • page
  • sequence
  • stream
    • read

Testing

• Clone the repository (or download, if you prefer):

$ git clone https://github.com/vitaly-t/spex

• Install the library's DEV dependencies:

$ npm install

• To run all tests:

$ npm test

• To run all tests with coverage:

$ npm run coverage

License

Copyright © 2020 Vitaly Tomilov; Released under the MIT license.

spex's Issues

Returning Errors From sequence()

The sequence() function has a track parameter that allows the function to return resolved data. I would like to propose that one adds functionality to track/capture rejected data too.

My particular scenario would be in file streaming. I use sequence() to process the data in large files into a database. Some rows fail from being inserted due to a variety of reasons. When an error is thrown/rejected, the whole run ends. In some cases, I would rather have an option that the sequence continues (add a skip boolean param) and/or that the problem rows be collected and reported (add a trackErrors param).

Looking at the code, my first idea would be to implement via checking in each catch() in sequence() for those params. The skip == true would result in no rejections and the trackErrors == true would, much like track, push data into the results.

Any suggestions or alternatives? I'd be willing to put in a pull request if so desired.

errors namespace is not available before library initialization

I'm not sure if this is as-intended, but right now, in order to access the errors namespace, spex needs to be initialized with a promise library.
To access the error namespace one needs to:

var promise = require('bluebird');
var spex = require('spex')(promise);

// spex.errors is defined

It would be convenient to be able to access the errors namespace:

var spex = require('spex');

// spex.errors is undefined

spex.squence() on an array of promises errors

I am building a list of typical functions that return promise with different data:

var getPostPromise = function(userData)
                {
                    return new Promise(
                        function(resolve, reject) {
                           // goes and does operations on userData then
                            resolve()
                            })
                        }
                    )
                }

These are pushed into an array. I am calling these in a chain for promises

firstPromise.then(function(){
        return spex.sequence( postsPromise )
    }).then(function(){ // finish})

I have used spex.batch but the order of the array of postsPromise changes. From reading about sequence it should resolve like promise.all but in sequence.

Thanks for your help

How can I use Transform streams with stream.read?

Spex stream.read consumes and processes data from a Readable stream. How can I use transformed Readable stream?

Eg. I've large JSON file and want to parse it with JSONStream and then stream it into database with spex and pg-promise. How can I implement this?

stream.read cannot handle a stream that doesn't support readable events

Specific case: pg-query-stream 1.1.0 and 1.1.1

brianc/node-pg-query-stream#37

Synchronously serialize api requests in react browser environment?

Hello. This is not really an issue, rather a question.

Can this library be used to synchronously serialize api requests?

The situation is: Given an array of API URL's to fetch (or axios), with the constraint of only one outstanding api request at a time. How to schedule api requests such that request #2 doesn't get sent until response #1 has been received (or timed out).

Turns out this is easy in the node environment, but I haven't been able to figure out how to do it in the browser environment. (Using create-react-app's built in dev server.)

Please provide example if possible. Or pointer to alternate library if more applicable. Thanks in advance.

Positional -> Named Parameters

I'm using spex.sequence on a project and it's working well, but the API is awkward to use IMO. Why not make the additional arguments to sequence (and other spex methods) an options object rather than trying to shoehorn everything into positional args e.g.:

Instead of:

spex.sequence(source, null, null, true);

How about:

spex.sequence(source, { track: true })

?