[How To] Use with yarn2

[elby22]

When installing ts-patch with Yarn 2 (pnp) I get errors regarding undeclared dependencies for glob and resolve:
Error: ts-patch tried to access glob, but it isn't declared in its dependencies; this makes the require call ambiguous and unsound.

When I resolve these dependencies using my .yarnrc.yaml file in this way:

packageExtensions:
  "ts-patch@*":
    "dependencies":
      "glob": "*"
      "resolve": "*"
      

I get the following error:
[!] [BackupError]: Error backing up tsc.js - Couldn't create backup directory. EROFS: read-only filesystem, mkdir '/node_modules/typescript/lib-backup'

Is there a way to support Yarn 2 with ts-patch?

[nonara]

I investigated this a bit today and ended up fighting for awhile with odd issues. It seems that yarn2 patching is a little temperamental right now, so if you have any problems, let me know. This should work.

These instructions are for yarn v3.

package.json

{
  "dependencies": {
    "ts-patch": "^1.4.4", // You need to use >=1.4.4. I fixed the missing deps issue + another relevant issue
    "typescript": "4.4.2"
  }
}

Install steps

# Install — Strangely, we have to call it twice to make the next cmd work
$ yarn install && yarn install

# Generate patchable dir
$ yarn patch typescript@npm:4.4.2
➤ YN0000: Package typescript@npm:4.4.2 got extracted with success!
➤ YN0000: You can now edit the following folder: /tmp/xfs-ce018a91/user
➤ YN0000: Once you are done run yarn patch-commit /tmp/xfs-ce018a91/user and Yarn will store a patchfile based on your changes.
➤ YN0000: Done in 1s 447ms

# Patch with ts-patch — Note: your directory will be different. Copy and paste what the last command output
$ yarn run ts-patch -d /tmp/xfs-ce018a91/user install
[+] ts-patch installed!

# Commit our patch to yarn
$ yarn patch-commit -s /tmp/xfs-ce018a91/user

# Cleanup
$ rm -rf /tmp/xfs-ce018a91/user

# Start using our patched version
$ yarn install

# Confirm it worked
$ yarn run ts-patch c
Checking TypeScript v4.4.2 installation in /your-module/.yarn/cache/typescript-patch-dc146214d1-9745be4574.zip/node_modules/typescript

[+] tsc.js is patched with ts-patch version 1.4.4
[-] tsserverlibrary.js is not patched.
[+] typescript.js is patched with ts-patch version 1.4.4
[-] typescriptServices.js is not patched.
[-] tsserver.js is not patched.

Native Support via CLI

I don't have the bandwidth right now to add this into v1, but if someone wants to do a PR, it should be pretty straight-forward to implement.

1. Add --yarn2 flag to ts-patch to enable it
2. Use yarn's patch API
3. Be sure to clean up temp dir after

[elby22]

Thanks @nonara. I was also able to accomplish this by using typescript as an unplugged dependency. I might take a crack at a PR if we decide to use that approach.

[jasonkuhrt]

For others: I am using Yarn 2 in Yarn 1 compat mode (meaning node_modules node linker).

I had to use postinstall hook in addition to prepare script.

[jekozyra]

@jasonkuhrt Could you explain more about the postinstall hook?

[finnmerlett]

@nonara @elby22 I was super excited to read this, I've been really wanting to use the custom transformer typescript-is with my yarn3 create-react-app project. I followed your steps exactly, using the exact same versions of ts-patch and typescript under dependancies, and it all appeared to work. The only indication that something might not be right was the following message on yarn install following the patch-commit, where it stated Cannot apply hunk #1:

➤ YN0000: ┌ Fetch step
➤ YN0013: │ typescript@patch:typescript@npm%3A4.4.2#.yarn/patches/typescript-npm-4.4.2-8c784d6e94::version=4.4.
➤ YN0013: │ typescript@patch:typescript@patch%3Atypescript@npm%253A4.4.2%23.yarn/patches/typescript-npm-4.4.2-8
➤ YN0013: │ typescript@patch:typescript@npm%3A4.4.2#.yarn/patches/typescript-npm-4.4.2-8c784d6e94::version=4.4.
➤ YN0066: │ typescript@patch:typescript@patch%3Atypescript@npm%253A4.4.2%23.yarn/patches/typescript-npm-4.4.2-8c784d6e94%3A%3Aversion=4.4.2&hash=e4c573&locator=main-pwa%2540workspace%253A.#~builtin<compat/typescript>::version=4.4.2&hash=ddd1e8: Cannot apply hunk #1

The test with yarn run ts-patch c gave the correct output so I was hopeful. Yet when I started the dev server react-scripts start I was met with an endless stream of errors reported on the console (literally it would not stop scrolling at lightning speed, even as I was mashing ctrl-c), seemingly reporting an error on every single line of my code! I think it was that typescript (as patched) was unable to find any modules at all.

I tried unplugging the typescript module before patching, but this had no effect. I tried the latest version of ts-patch and typescript and that failed too, in exactly the same way.

I did notice that when running yarn why typescript that the typescript was patched already - I think it was mentioned on another thread here, that it was due to modifying for pnp support? If so, my guess would be that the built-in patch was applied second and failed. This checks out, as the Cannot apply hunk #1 warning appeared after the only patch report line that contained builtin<compat/typescript>, the signifier for the the internal pnp patch. This would explain why typescript wasn't able to find any of the modules - without the pnp patch it would probably be looking for them in the non-existent node_modules.

Anyway, if you have any advice on how to fix this, I would love to hear it. Getting those sweet sweet run-time type-checks up and running would be an absolute dream.

[nonara]

@finnmerlett if you can put together a quick reproduction repo, I'll have a look and see if I can help you get it going!

[finnmerlett]

@nonara Thank you so much for such a speedy reply, and sorry I took a while to get back to you - I've been quite busy with work. Anyway, I've set it up here https://github.com/finnmerlett/ts-patch-cra-test. Running yarn start works just fine, but following your steps above to the letter produces that Cannot apply hunk #1 warning, and then yarn start gives many compile errors. VSCode also highlights the errors, it appears that it cannot find any of the modules.

You can check through the commits on the repo to see how I set it up, but essentially I've just done a create-react-app using their latest alpha build, then set up Yarn 3 with P'n'P. Many thanks in advance!

[finnmerlett]

@nonara Did you find any leads with this?

[nonara]

@finnmerlett Sorry. Was swamped and this fell through the cracks. I'll take a look tomorrow and let you know!

[nonara]

@finnmerlett Ok. I cloned it and had a look. No errors applying hunk for me, but I do see the issue of it not resolving modules.

From what I understand, yarn 2 actually modifies common libraries in order to add support. If I recall correctly, it does do this for typescript automatically, as well.

As an aside, this seemed, to me, like a terrible route to attempt to solve the issue. I think a better route would be to use a loader to hook filesystem bindings (like we do with mock-fs) or better yet, to just submit a PR for node which provides zip support (allowing ability to treat a zip like a directory).

Anyway, that's more info than you need. My best guess on this is that some sort of compatibility modification made somewhere is broken. Maybe React is doing a CRC check on TS to add yarn support and it fails because it's modified (??) Hard to say. It's odd that this works outside of React.

In my opinion, yarn2's core ideas are excellent and necessary advancements for the node ecosystem. That said, not to be unkind to the developers, but their approach to implementing the ideas have been poorly thought out.

In any event, I'm sorry that it didn't work for you. I would recommend trying what other users have reported working for them using node_modules linker / unplugged dependency.

Best of luck to you!

[AlCalzone]

I would recommend trying what other users have reported working for them using node_modules linker / unplugged dependency

Since it might interest some readers here... After suffering through the ecosystem implications by yarn PnP, I switched to the new pnpm compatible linker they added in v3.1 and haven't looked back. Roughly the installation speed of PnP, but without any of these hacks necessary.

[finnmerlett]

Thanks @nonara for looking into it (and no worries about the delay - I'm getting your time for free so I'm not complaining!). And ooo @AlCalzone that sounds super interesting. Will investigate further

[finnmerlett]

Oh boy did I come close. Maybe. I switched to using the pnpm compatible linker, and all the modules unresolved errors vanished. I checked the yarn install after applying the patch though, and was still getting the hunk failed error. Yarn was still patching typescript before ts-patch was, despite the pnpm linker, and after a bit of research I found that yarn always patches pnp support into typescript (and some other key modules) in order to maintain compatibility with the other linkers (and avoid changing the .cache when swapping linkers). I found this comment yarnpkg/berry#2384 (comment) that described how to replace the yarn plugin-compat (the plugin that I believe patches in pnp support for key modules) with a dummy script. After doing that, and following the ts-patch process (described at the top of this thread) again, I could see that the pnp-compat patch was successfully skipped and the ts-patch applied with no errors! Running yarn run ts-patch c gave a positive confirmation, everything built and loaded successfully and I was super excited to start using the typescript plugins...

Until I tried them out and they weren't working. Their errors suggested that the transformer plugins weren't being applied at all. I can't think what I could be missing, but it feels like something obvious. I have updated the test repo to include the latest changes I made, and the test scripts in App.tsx (they console log their success or errors). The last commit to the repo is the (at first glance apparently successful) applying of ts-patch, so if you want to re-do that yourself just pull the repo and undo the last commit.

@nonara is this something you might have time to consider looking further into? It feels like it should work now, we have modules in node_modules as standard, no extra patches applied to typescript and the ts-patch test coming back positive. I'm at a dead end as to what I could be missing! Thanks again for your help.

PS. I am still using typescript 4.4.2 and ts-patch 1.x.x in the test repo since those were what you specified in the original instructions, but I want to note that I've also tried it with ts 4.4.4 and ts-patch 2, with the same unsuccessful outcome.

PPS. I just had a thought that perhaps react-scripts is somehow finding and using a non-patched typescript package. I don't know where or how though, since running yarn why typescript shows only one version being used, and that is the version I thought I successfully patched. Is there any way to check for the successful application of the ts-patch using the ts object? If so a testing step could be to console log inside react-scripts' files on the ts object it fetches.

PPPS. I console logged the ts path that react-scripts is using and it matches the ts path that ts-patch confirms it has successfully applied to. This makes me think it might be an issue with ts-patch not applying correctly?

[nonara]

I would debug and step through, but if your source files are in zips, it's going to be more difficult. Still doable, though. You can insert debugger statements into files.

First thing I'd do is create a local empty plugin and inspect the arguments that it passes. Use tsp v2. If it passes a ts instance, check that ts.tspVersion is set.

Example plugin:

export default function () {
  console.log(arguments);
  debugger;
}

Add the local file to your tsconfig.json plugins and run with debugger. To make sure it works, try the setup in yarn1 first, in another dir. It should pause on that breakpoint. Once you get it with y1, try it with y2. If it hits the breakpoint, you'll know that plugins work.

From what you've explained, I can think of a few possibilities

1. React is using a different typescript lib somewhere
   You'll need to find the code where it works with the compiler API. Set a breakpoint before and after the require. First, after the require, check if the instance has tspVersion. If not, you know it's a different version. If that's the case, run again and step into the require call to see where it's getting the file.
2. It's doing something odd like using swc to compile and only using the compiler API for typechecking
3. It's compiling without creating a Program instance (very unlikely)
4. Maybe it's using tsserverlibrary.js to compile (also very unlikely)

Let me know what you discover and I'll try to offer what ideas I can.

[finnmerlett]

Hum. So it turns out create-react-app (CRA, the framework I'm using) does what is closest to item no.2 on your list. As I found out (painfully slowly) after stumbling about in the react-scripts package and reading about this, this and finally this (which actually stated what I had started to suspect) it turns out CRA indeed uses babel-loader through webpack to do the compilation, and only uses typescript to carry out typechecking.

Rightio, this all means ts-patch is out for me. The plugins were applied, but only for typechecking - @nonara thanks for your pointers in helping me figure this out.

If I want to get the typescript plugins I was eyeing up working, I need to swap out babel-loader for ts-loader in CRA's webpack.config.js (which will involve some yarn patching), add the plugin's transformers to the getCustomTransformers option on ts-loader, and hope my compile-time refresh speed isn't unmanageably slow 😬

[nonara]

Ah. Good work tracking it down! Thanks for sharing the final details. I imagine that will be helpful for anyone facing the same issue. Good luck with it!

[CarsonF]

I was able to make this work with pnpm linker & postinstall script (yarn 3.2.0), thanks above for that.
EDIT: I was also able to get it to work with pnp linker with typescript unplugged.

I think a custom yarn plugin would work to apply this patch while still being compatible with yarn's TS patch.
That could work for any linker & without a custom user script.