tc39 / proposal-joint-iteration

a TC39 proposal to synchronise the advancement of multiple iterators

Joint Iteration

A TC39 proposal to synchronise the advancement of multiple iterators.

Stage: 2

Demo: https://tc39.es/proposal-joint-iteration/demo/

Specification: https://tc39.es/proposal-joint-iteration/

motivation

Often you have 2 or more iterators that are positionally aligned (the first value yielded by the first iterator corresponds to the first value yielded by the other iterators, and so on), and you would like to operate on the corresponding values together. A common solution to this is zip, which produces an iterator of the combined values. zipWith allows combination of values in some way other than tupling. Some languages express zipWith as a variadic map.

presentations to committee

• September 2023
• November 2023
• January 2024

considered design space

1. do we support just 2 iterators or something else? 2+? 1+? 0+?
   1. if 0 is allowed, is that considered never-ending or already completed?
   2. should the iterators be passed positionally (combining to arrays) or named (combining to objects)?
   3. do we take the iterators as varargs or as an iterable/object?
      1. varargs eliminates design space for potentially passing an options bag or a combining function
2. do we support iterators and iterables like Iterator.from and flatMap?
   1. if so, which string handling do we match? Iterator.from iterates strings; flatMap rejects strings
3. if an iterator completes, do we still advance the other iterators?
   1. do we return them?
4. if an iterator fails to advance, do we still advance the other iterators?
   1. if so, do we return an AggregateError? Only if 2+ failures?
5. do we want -With variants for combining the values in other ways than tupling?
   1. what about always requiring the combiner?
6. do we want a zipLongest/zipFilled/zipAll?
   1. if so, do we want a filler element or to call a function to provide the filler?
   2. what about a variant that matches the length of a privileged iterator (this)?
7. do we want a zipEqual/zipStrict that throws if they do not complete after the same number of yields?

prior art

other languages

language	shortest	longest	privileged	strict	-With	3+ sources	1 source	0 sources
C++	std::ranges::views::zip				::zip_transform	yes	yes
Clojure	variadic map				yes	yes	yes
Elm					List.map2	yes	yes
Haskell	zip				zipWith	yes
OCaml	zip			combine	map2		yes
Python	zip	itertools.zip_longest		zip(..., strict=True)		yes	yes	yes, empty
Ruby			Enumerable#zip		zip	yes	yes
Rust	Iterator::zip
Scala	zip	it.zipAll(jt, x, y)
Swift	zip

JS libraries

library	shortest	longest	privileged	strict	-With	3+ sources	1 source	0 sources
@iterable-iterator/zip	zip	zipLongest				yes	yes
@softwareventures/iterator	zipOnce
extra-iterable	zip	zip			zip	yes	yes	yes, empty
immutable.js	Seq::zip				zipWith	yes	yes
iter-ops	zip					yes	yes	yes, empty
iter-tools	zip	zipAll				yes	yes	yes, empty
iterablefu	zip	zipAll				yes	yes	yes, empty
iterare	zip
itertools-ts	zip	zipFilled, zipLongest		zipEqual		yes	yes	yes, empty
ixjs	zip					yes	yes	yes, empty
lodash		zip			zipWith	yes	yes	yes, empty
ramda	zip				zipWith
sequency	zip
wu	zip	zipLongest			zipWith	yes	yes
zipiterators		zipiterators

proposal-joint-iteration's Issues

Iterator.zip reads properties of the "bag of iterables" overload twice

Iterator.zip step 14.c.i does iterables.[[GetOwnProperty]](key), and then 14.d.i does Get(iterables, key). That's silly: we already have the property descriptor in the earlier step and can just use the [[Value]] of it directly, instead of doing another observable lookup. The loops in c and d should be collapsed.

Why just on iterators and not on arrays?

I have this need on Arrays just as often as on non-array iterators; any reason not to add it to both?

`fillers` option naming

I'm thinking that padding might be a better name than fillers? I think I've seen the term used more often. On the other hand, it doesn't scream "plural" as strongly as fillers. Any opinions? This may set precedent for other std lib APIs that need padding/filler elements such as iterator chunking.

Positional and named variants should be separate methods

I don't think it's a good idea to overload the zip method based on whether the first argument is iterable or not. This would make it a refactoring hazard to add a [Symbol.iterator] method to objects. The only precedence for such overloading is Array.from, but I think it is generally discouraged. I understand that most zip invocations would use an inline array literal or object literal where this would not be an issue, but it's still a concern for other cases, and the language should avoid establishing this as a pattern.

stage 2.7 tracking

• complete spec text
• updated polyfill
• assigned reviewer sign-off
  • @nicolo-ribaudo
  • @ljharb
• editor sign-off
  • @syg
  • @bakkot
  • @michaelficarra

API name choices (#9) can be finalised at plenary during advancement IMO.

Consider async iteration

This should also add AsyncIterator.zip (or whatever methods are being added to synchronous Iterator).

IteratorCloseAll should be in reverse order

There's a TODO asking this question, so here's the answer: resources should be closed in LIFO order, as if you'd nested try-finally blocks. See also the dispose method of the DisposableStack, which does disposing "in reverse list order".

`fillers` does not need to support providing an iterator

The first argument to Iterator.zip is either an iterable of iterators or an object whose values are expected to be iterators.

In the former case, fillers (if passed) should be expected to be an iterable of values. Right now step 13.g.ii.1 uses GetIteratorFlattenable, which is for iterator-or-iterables, not iterables. There's no reason to support bare iterators here. It should match the first argument in being an iterable.

exceptions thrown while reading from an iterator should not close that iterator

For example, Iterator.zip step 13.b puts inputIter in openIters, and then 13.d.ii closes everything in openIters (including inputIter) if the reading the value property of the iterator result object from inputIter throws.

But a throwy getter for the value property from the iterator result object from inputIter is a violation of the iterator protocol, and so should not cause inputIter to be closed.

Similarly for step 13.d.vii/viii.

inputIter should probably not get put in openIters at the start of the loop. Instead, only add it after the Let iter be Completion(GetIteratorFlattenable(value, ITERATE-STRINGS)) step in the case that this step is an abrupt completion: something like

1. If _iter_ is an abrupt completion, then
  1. Set _openIters_ to the list-concatenation of « _inputIter_ » and _openIters_.
  1. Return ? IteratorCloseAll(_openIters_, _iter_).