Improve Iterator Helper Type Signatures

[nikolaybotev]

⚙ Compilation target

ESNext

⚙ Library

esnext.iterator.d.ts

Missing / Incorrect Definition

• Iterator.from(value: Iterator) - PR Improve generic type signature of Iterator.from() #59927
  • value iterator argument with TNext other than undefined should be accepted; this is a bad idea; my mistake
  • Since IteratorObject instances (objects whose prototype is Iterator.prototype) are not wrapped by Iterator.from, we could add an Iterator.from overload to represent those cases and pass through all generic type parameters: from<T, TReturn = any, TNext = any>(value: IteratorObject<T, TReturn, TNext>): IteratorObject<T, TReturn, TNext>;
  • TReturn type should be passed through from the argument to the return type
• Iterator helper intermediate operators (filter, map etc) do not propagate the return value of their source iterator and should have TReturn set to undefined
  • This is already the case in the current declarations
• Iterator objects returned from Iterator.from and the other built-in iterators returned from Array.values etc always have a return method (built-in iterators returned from Array.values etc also always have a throw method) thanks to @bakkot for the clarification
  • There is no way that I am aware of to express this behavior in TypeScript with the current definition of IteratorObject without also forcing user-defined classes extending from the javascript Iterator class to also implement the return method, which would be wrong. The impact on client code is minimal as the Iterator object's return method can always be called safely using the return!() notation.

Sample Code

https://github.com/nikolaybotev/iteratorhelpersdemo

Documentation Link

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Iterator

and

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Iteration_protocols

[bakkot]

Iterator objects returned from Iterator.from and the other built-in iterators returned from Array.values etc always have a return method

Not all built-in iterators. Your two specific examples in fact do not: e.g. Iterator.from([].values()).return or [].values().return.

But many built-in iterators do have a return method, including those returned by the iterator helper methods. I'd actually originally intended for the types to reflect this (in #58222) but I didn't implement it initially and then I think that desiderata got lost during the merge.

There is no way that I am aware of to express this behavior in TypeScript

I haven't actually tested it, but I was originally thinking of just having the return types of such methods be an intersection type, along the lines of the following (probably with some tweaks for the various type parameters):

type ClosableIteratorObject<T> = IteratorObject<T> & { return(): { done: true, value: undefined } };

interface IteratorObject<T, TReturn, TNext> {
    map<U>(callbackfn: (value: T, index: number) => U): ClosableIteratorObject<U>;
}

(instead of having map return IteratorObject<U, undefined, unknown> as it currently does).

[nikolaybotev]

Thanks for the clarification. and thanks for the prompt response.

I also realized that the following statement is wrong:

value iterator argument with TNext other than undefined should be accepted;

I have updated the issue description to reflect that.

I do believe there is value in adding the TReturn type parameter to Iterator.from.

I was able to confirm that all major implementation - Node.js (and by extension Chrome, Edge etc browsers sharing the same engine, which I have not tested), Firefox, Opera and core-js all exhibit the behavior of passing through the wrapped iterator return value:

core-js

❯ node -p 'console.log(process.version, typeof Iterator); require("core-js/stage/3"); Iterator.from(function* () { return 42; }()).next();'
v20.17.0 undefined
{ value: 42, done: true }

node.js 22

❯ node -p 'console.log(process.version, typeof Iterator); Iterator.from(function* () { return 42; }()).next();'         
v22.0.0 function
{ value: 42, done: true }

Firefox

navigator.userAgent
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:132.0) Gecko/20100101 Firefox/132.0"
Iterator.from(function* () { return 42; }()).next()
Object { value: 42, done: true }

Opera

navigator.userAgent
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36 OPR/113.0.0.0'
Iterator.from(function* () { return 42; }()).next()
{value: 42, done: true}

[nikolaybotev]

The PR that implements the change proposed here is ready for review - #59927

[nikolaybotev]

I haven't actually tested it, but I was originally thinking of just having the return types of such methods be an intersection type, along the lines of the following (probably with some tweaks for the various type parameters):

Just confirmed that the behavior of adding a return method to the Iterator object is consistent across Firefox, Opera, Node, and core-js:

Iterator.from({ next: () => ({ value: 42 }) }).return

Do you want me to open a PR adding the return method to the return type of Iterator.from using an intersection type?

[bakkot]

Just confirmed that the behavior of adding a return method to the Iterator object is consistent across Firefox, Opera, Node, and core-js:

Iterator.from will wrap the underlying thing only when the underlying thing does not inherit from Iterator.prototype, and the wrapper does have return. So

Iterator.from({ next: () => ({ value: 42 }) }).return

is present, but

Iterator.from({ __proto__: Iterator.prototype, next: () => ({ value: 42 }) }).return

is not.

I wouldn't worry about trying to reflect this behavior in the TypeScript types. It's OK for them to be a little imprecise.

Do you want me to open a PR adding the return method to the return type of Iterator.from using an intersection type?

I'm not on the TS team. I personally am unlikely to open such a PR in the near future, but if you want to see if the TS team would accept such a PR you are welcome to do so.

[nikolaybotev]

Just confirmed that the behavior of adding a return method to the Iterator object is consistent across Firefox, Opera, Node, and core-js:

Iterator.from will wrap the underlying thing only when the underlying thing does not inherit from Iterator.prototype, and the wrapper does have return. So

Iterator.from({ next: () => ({ value: 42 }) }).return

is present, but

Iterator.from({ __proto__: Iterator.prototype, next: () => ({ value: 42 }) }).return

is not.

Amazing. Thanks for the clarification.

I wouldn't worry about trying to reflect this behavior in the TypeScript types. It's OK for them to be a little imprecise.

👍

Do you want me to open a PR adding the return method to the return type of Iterator.from using an intersection type?

I'm not on the TS team. I personally am unlikely to open such a PR in the near future, but if you want to see if the TS team would accept such a PR you are welcome to do so.

Given the above behavior you brought up, which I was unaware of, I am not convinced it is any good to try and reflect the addition of the return method when wrapping an iterator in the type signature.

[nikolaybotev]

Since IteratorObject instances (objects whose prototype is Iterator.prototype) are not wrapped by Iterator.from, we could add an Iterator.from overload to represent those cases and pass through all generic type parameters:

from<T, TReturn = any, TNext = any>(value: IteratorObject<T, TReturn, TNext>): IteratorObject<T, TReturn, TNext>;

This would make the following two examples compile:

declare const g1: Generator<string, number, boolean>;
const iter1 = Iterator.from(g1);

and

class A extends Iterator<string, void, string> { next = (value: string): IteratorResult<string, void> => ({ value }); [Symbol.iterator] = () => this }
const g3 = new A();
const iter5 = Iterator.from(g3);

while still failing when a custom iterator would get wrapped by Iterator.from and lose its ability to receive next arguments:

const g2 = { next: (value: string) => ({ value }) };
const iter4 = Iterator.from(g2);

[nikolaybotev]

@bakkot do you have plans or are you aware of anyone else's plans to add async iterator helper and iterator helper polyfill type definitions to @type/core-js, now that TypeScript has proper type definitions for async iterator and iterator helpers (via IteratorObject and AsyncIteratorObject) that allows polyfills for older javascript runtimes to also be successfully typed (without polluting the [Async]Iterable/Iterator protocol space)?

If not, I would like to make an effort to write these type definitions.

[bakkot]

I don't know anything about how types for core-js work; if you do and you want to contribute, go for it.

[nikolaybotev]

Thanks. There are types in DefinitelyTyped where one could add type definitions for AsyncIterator helpers, which core-js implements today.