Add BuiltinIteratorReturn to IterableIterator and AsyncIterableIterator return types #1713

rbuckton · 2024-05-07T14:46:27Z

This is a follow-on PR to microsoft/TypeScript#58243 to add BuiltinIteratorReturn to all IterableIterator and AsyncIterableIterator return types.

TODO:

Update the "typescript" dependency in package.json to a shipping version that supports BuiltinIteratorReturn and the TReturn type parameter for IterableIterator/AsyncIterableIterator
~~Update README.md to indicate the minimum TypeScript version~~

…or return types

saschanaz · 2024-05-08T07:32:48Z

baselines/audioworklet.asynciterable.generated.d.ts

@@ -3,6 +3,6 @@
 /////////////////////////////

 interface ReadableStream<R = any> {
-    [Symbol.asyncIterator](options?: ReadableStreamIteratorOptions): AsyncIterableIterator<R>;
-    values(options?: ReadableStreamIteratorOptions): AsyncIterableIterator<R>;
+    [Symbol.asyncIterator](options?: ReadableStreamIteratorOptions): AsyncIterableIterator<R, BuiltinIteratorReturn>;


So BuiltinIteratorReturn can't be the default value to reduce verbosity?

It can't be the default value because it's not the default behavior of any iterator, just the native ones provided by the host. User-defined iterables might not match that behavior, and we don't intend to break those cases unnecessarily.

rbuckton · 2024-07-19T20:44:00Z

@jakebailey could you review? Also, what is the correct approach to handling the typescript reference in package.json here?

rbuckton · 2024-07-19T20:53:40Z

I've reverted the package.json/package-lock.json changes, but tests will fail until we at least have a nightly to point to

jakebailey · 2024-07-19T20:55:53Z

Yeah, I would just wait for a nightly.

If we really, really, really need it, we can run pack this on a PR and then reference the "pr-deploys" one on npm. (The artifact is ephemeral so shouldn't be used.)

jakebailey

Seems correct; feel free to use a PR build from pr-deploys if you want to be sure before tomorrow.

saschanaz · 2024-07-20T18:36:18Z

You might want to update TS dependency to nightly to fix the test. (Edit: #1713 (comment) already mentioned it)

jakebailey · 2024-07-20T20:16:35Z

Updated it to help things along.

jakebailey · 2024-07-20T20:28:42Z

Yay, green!

petamoriken · 2024-07-21T22:59:36Z

IMO, BuiltinAsyncIterator is better name (microsoft/TypeScript#58222 (comment)).

rbuckton · 2024-07-21T23:08:55Z

IMO, BuiltinAsyncIterator is better name (microsoft/TypeScript#58222 (comment)).

Honestly, I agree. I'll change it.

rbuckton · 2024-07-22T04:09:26Z

The last commit aligns the generator with microsoft/TypeScript#59388

rbuckton · 2024-07-22T04:47:04Z

I'll update this to the nightly tomorrow before merging

rbuckton · 2024-07-23T15:06:34Z

How will this change affect @types/web publishing given that it has a dependency on TS 5.6+ for the BuiltinIterator and BuiltinAsyncIterator types? Would it just be published as @types/[email protected] (or similar) with an entry in the README.md, or do we have some other mechanism of ensuring the correct TS version, such as adding an entry to peerDependencies?

jakebailey · 2024-07-23T17:35:34Z

I'm not 100% certain; the readme for the @types/web package claims compatibility with even TS 4.5, and in general support for old versions. This PR would change that to only support 5.6, so we would need to wait to publish this package for a while, and then it would break compat.

Does @types/web publishing / usage support typesVersions, maybe?

rbuckton · 2024-07-23T18:13:29Z

It probably could, by tweaking deploy\createTypesPackages.js slightly to add a typesVersions entry, though we'd have to generate two sets of types: one using IterableIterator for older TS, and one using BuiltinIterator for newer TS.

saschanaz · 2024-07-24T09:47:22Z

Maybe either bump the minor version or add type AsyncIterableIterator<R> = BuiltinAsyncIterator<R, BuiltinIteratorReturn> in the TS side for better compat? (Because this would also become headache for other definitelytyped packages)

jakebailey · 2024-08-05T19:39:59Z

This LGTM but do we need to come up with some typesVersions something to make things not break for old TS versions? Or just remove the language about TS version support?

rbuckton · 2024-08-06T17:44:29Z

To bump semver here I think someone with npm permission will have to manually deploy a new set of packages as 0.1.0 and then the deploy script should automatically make future versions as 0.1.x.

TypeScript-DOM-lib-generator/deploy/createTypesPackages.js

Lines 125 to 131 in 2939d4f

/** @type {*} */

const npmPackage = await npmResponse.json();

const semverMarkers = npmPackage["dist-tags"].latest.split(".");

const bumpedVersion = `${semverMarkers[0]}.${semverMarkers[1]}.${

Number(semverMarkers[2]) + 1

}`;

Is a semver bump necessary if we add "typesVersions" to disambiguate?

jakebailey · 2024-08-06T18:08:54Z

I don't think we need to bump semver if we are just adding a typesVersions; adding that is not a backwards incompatible change (otherwise we'd be totally dead on DT)

jakebailey · 2024-08-06T18:13:20Z

Do the baselines also need the 5.5 variants? (I think so?)

saschanaz · 2024-08-06T18:15:50Z

Oh, TIL that and sounds nice. I guess it's fine in that case.

rbuckton · 2024-08-06T18:17:26Z

Do the baselines also need the 5.5 variants? (I think so?)

They did, yes. I just pushed up a commit that checks them.

I think this should be ready for one more review pass.

jakebailey · 2024-08-06T18:35:40Z

Actually, do typesVersions work with lib replacement? I think so, reading pathForLibFile, given it seems to still go through regular module resolution? Would be good to quick double check the PR against 5.5 and 4.9. Wish we tested the list of supported versions here too.

Otherwise, things look correct to me.

rbuckton · 2024-08-06T18:36:25Z

Actually, do typesVersions work with lib replacement? I think so, reading pathForLibFile, given it seems to still go through regular module resolution? Would be good to quick double check the PR against 5.5 and 4.9. Wish we tested the list of supported versions here too.

Otherwise, things look correct to me.

I'll test that locally and report back shortly.

saschanaz · 2024-08-06T19:22:30Z

It would be nice to have some CI for that here, but I guess that can be done separately.

rbuckton · 2024-08-06T19:30:23Z

"typesVersions" works, but the autoImports behavior in createTypesPackages.js is causing problems. I'm surprised autoImports is a thing as it seems like it would conflict with --target ES5 since dom.iterable.generated.d.ts tries to load Symbol but it wouldn't exist.

rbuckton · 2024-08-06T20:17:13Z

The last commit should fix the autoImport reference path added to the index of each @types package, and I've verified "typesVersions" is working correctly for both 5.5.4 and 5.6.0-dev.

saschanaz · 2024-08-06T20:18:32Z

The question would be that whether it works in 4.4 or otherwise it will still be a breaking change 🤔

rbuckton · 2024-08-06T20:20:54Z

I've tested it and it works fine with 4.4 as well. I mentioned 5.5 since that's where the split occurs as a result of the "typesVersions" entry, so 5.5.* and earlier (incl 4.4) will use the version in the ts5.5 directory, while 5.6.0-dev and later will use the version in the root.

jakebailey · 2024-08-06T20:28:20Z

TS 4.4 is well out of support on DT (and that's what most tools use as their window), so I wouldn't worry about that too much. But good that it works!

saschanaz · 2024-08-06T20:38:44Z

(Maybe this should also use DT window)

saschanaz · 2024-08-06T20:50:37Z

src/test.ts

+  try {
+    outputFiles = fs.readdirSync(outputFolder);
+  } catch {
+    // do nothing


Are we actually expecting an error here? This looks like it could potentially silence a real failure 🤔

the failures that matter are mismatched files. If either the input or output dir don't exist, then the file mismatch will be reported when we traverse whichever side did exist and find the file on the other side is missing.

This actually catches more failures as we previously only checked what was in baseline and ignored new things in generated. This now enumerates all distinct entries from both so we will report on excess baselines or excess outputs.

Yeah, but we call stat before entering directory, so not sure this one matters.

We call stat on both, but we still traverse the directory if the stat only succeeded on one side or the other, so the readddir can still potentially fail.

saschanaz · 2024-08-06T20:58:42Z

baselines/dom.asynciterable.generated.d.ts

@@ -2,14 +2,22 @@
 /// Window Async Iterable APIs
 /////////////////////////////

+interface FileSystemDirectoryHandleAsyncIterator<T> extends AsyncIteratorObject<T, BuiltinIteratorReturn, unknown> {


I don't understand this, is FooAsyncIterator basically same as BarAsyncIterator, because I don't see a difference between this and ReadableStreamAsyncIterator below?

They have independent prototypes on the web, so you could theoretically augment ReadbleStreamAsyncIterator independently via its prototype:

var rs = new ReadableStream(); var proto = Object.getPrototypeOf(rs[Symbol.asyncIterator]()); proto; // [object ReadableStream AsyncIterator]

As with ArrayIterator, MapIterator, SetIterator, etc., we've opted to use interface names that more closely align with the underlying prototype.

They have independent prototypes because the underlying iterator algorithms are different, but I wouldn't expect any independent members on them that would require separate declarations, at least not in the foreseeable future, because Web IDL simply doesn't have syntax for that and I don't see why IDL would want that.

@MattiasBuelens may have some opinion.

Yes, but the general principle we've agreed on in microsoft/TypeScript#58243 and in design meeting is to try to give these things a relevant name. BuiltinIterator was too broad and confusing, and IteratorObject<T, BuiltinIteratorReturn, unknown> was too unwieldy and verbose. Since we must give these names, we've opted to give them names that reflect the actual runtime reality.

Hmmmm. I guess it would be fine as we don't have too many iterable interfaces here, and also matching with TS principle makes sense. I still worry that this would encourage people do weird things that doesn't fit the spec intention though.

rbuckton · 2024-08-07T22:23:44Z

@saschanaz: any other concerns or feedback before this is merged?

saschanaz · 2024-08-07T22:43:00Z

Besides what I already said, otherwise I'm good.

Add BuiltinIteratorReturn to IterableIterator and AsyncIterableIterat…

b7df2ac

…or return types

rbuckton requested a review from jakebailey May 7, 2024 14:46

github-actions bot added the maintainers label May 7, 2024

github-actions bot assigned sandersn May 7, 2024

rbuckton mentioned this pull request May 7, 2024

Add TReturn/TNext to Iterable et al microsoft/TypeScript#58243

Merged

saschanaz reviewed May 8, 2024

View reviewed changes

Update to use BuiltinIterator

f4cb461

Revert package.json

13ee015

Merge branch 'main' into builtiniteratorreturn

b3feaf7

rbuckton mentioned this pull request Jul 19, 2024

Manual update of generated DOM and Worker libs microsoft/TypeScript#59366

Merged

jakebailey approved these changes Jul 19, 2024

View reviewed changes

rbuckton marked this pull request as ready for review July 19, 2024 21:59

Update TS to nightly

faa9be8

Use 'BuiltinAsyncIterator' spelling

8245db1

Temporarily use pr build for testing

fe02922

Bump nightly

5b83895

rbuckton added 2 commits August 6, 2024 13:15

Bump typescript

b6a6718

Support typesVersions in @types/web et al

1864a71

Update baseline tests to support legacy folder

b010afb

rbuckton added 4 commits August 6, 2024 14:19

Merge branch 'main' into builtiniteratorreturn

0c19a29

Accept baselines

101623e

Accept baselines

ee46158

Accept baselines

f66fea8

Fix autoImport

91e30ab

jakebailey approved these changes Aug 6, 2024

View reviewed changes

saschanaz reviewed Aug 6, 2024

View reviewed changes

rbuckton merged commit 907f5dc into main Aug 8, 2024
6 of 16 checks passed

rbuckton deleted the builtiniteratorreturn branch August 8, 2024 08:34

Add BuiltinIteratorReturn to IterableIterator and AsyncIterableIterator return types #1713

Add BuiltinIteratorReturn to IterableIterator and AsyncIterableIterator return types #1713

Conversation

rbuckton commented May 7, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rbuckton commented Jul 19, 2024

rbuckton commented Jul 19, 2024

jakebailey commented Jul 19, 2024 • edited Loading

jakebailey left a comment

Choose a reason for hiding this comment

saschanaz commented Jul 20, 2024 • edited Loading

jakebailey commented Jul 20, 2024

jakebailey commented Jul 20, 2024

petamoriken commented Jul 21, 2024

rbuckton commented Jul 21, 2024

rbuckton commented Jul 22, 2024

rbuckton commented Jul 22, 2024

rbuckton commented Jul 23, 2024

jakebailey commented Jul 23, 2024

rbuckton commented Jul 23, 2024

saschanaz commented Jul 24, 2024 • edited Loading

jakebailey commented Aug 5, 2024

rbuckton commented Aug 6, 2024

jakebailey commented Aug 6, 2024

jakebailey commented Aug 6, 2024

saschanaz commented Aug 6, 2024

rbuckton commented Aug 6, 2024

jakebailey commented Aug 6, 2024

rbuckton commented Aug 6, 2024

saschanaz commented Aug 6, 2024

rbuckton commented Aug 6, 2024

rbuckton commented Aug 6, 2024

saschanaz commented Aug 6, 2024

rbuckton commented Aug 6, 2024

jakebailey commented Aug 6, 2024

saschanaz commented Aug 6, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rbuckton Aug 6, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

saschanaz Aug 6, 2024 • edited Loading

Choose a reason for hiding this comment

rbuckton commented Aug 7, 2024

saschanaz commented Aug 7, 2024

rbuckton commented May 7, 2024 •

edited

Loading

jakebailey commented Jul 19, 2024 •

edited

Loading

saschanaz commented Jul 20, 2024 •

edited

Loading

saschanaz commented Jul 24, 2024 •

edited

Loading

rbuckton Aug 6, 2024 •

edited

Loading

saschanaz Aug 6, 2024 •

edited

Loading