Long-running tasks watch mode

[dstoyanoff]

Hey guys,
I've been using NX for a long time and I am pretty happy with the optimizations that it brings. I have one huge issue however.

I have a classic yarn monorepo with package.json files in every project and a central nx.json. I got a few apps using NextJS and a bunch of libraries that are used across the apps.

At the moment, I am importing the libraries using the non-buildable approach, where only tsconfig path aliases are enough to compile the libs together with the app and hot-reload also works fine. I would like to switch to buildable libraries however to introduce other dimensions of optimizations and not build libs that didn't change for a while.

What I can't figure out is how to get the best DX. The next apps reload automatically on change, but since the apps have different artifacts, they need a different process. I use rollup and it has a watch flag, but this doesn't work with NX as I can't have my dependencies run in watch mode.

What I think would be nice is if NX provides a way to run long-running tasks as a dependency of another task, which would automatically manage their lifetime and stop them when the main task stops. Happy to explore other options too, but so far I am pretty stuck on this.

I know there is nx watch, but it just executes a single command for all changed projects, which doesn't seem to achieve my goal.

Thanks

[unleashit]

I just got this working albeit with some memory leak warnings: https://github.com/unleashit/npm-library/tree/nx (make sure it's on the nx branch). Just posted a bug report about them so we'll see: #16788.

What would really be cool would be if NX could somehow cache the result of the watch processes...

[neodon]

I'm currently accomplishing something like this as follows:

In each project, I have a dev script that starts whatever dev mode / watcher tool that project needs via nx exec. For example, for vite, it's just nx exec -- vite.

For the api project using tsup, it's nx exec -- tsup --silent --watch --onSuccess 'node --enable-source-maps dist/start.js'.

Libraries do not have a dev mode, just a build script.

In the workspace root, the dev script is set to nodemon -w libs --ext cjs,js,json,mjs,ts --delay 3 --exec nx run-many --target=dev --nx-bail --skip-nx-cache. This allows individual projects to use their own watchers and hot reload, but if a library changes then everything restarts (with nx re-building the library dependencies first).

I set --nx-bail to make it easier to CRL+C to interrupt everything. For --skip-nx-cache I'm just being paranoid because I don't want it to cache anything dev related.

I just got all this working this evening, so it might be the completely wrong way to go. I'm open to wisdom anyone could share on this front. Let me know if an example repo would be helpful. I'd love to see a way in the future to do all this via nx watch or nx dev maybe.

[unleashit]

A great discovery I recently made is the publishConfig property in package.json. It allows for swapping certain values when exporting a package (like during p/npm/yarn publish or tar). This way you can set main and exports to your source files but have them automatically switched to, for example, a dist folder at publish time. Next include the lib source directories in your app(s) bundlers (only during development, unless you want to bundle them), and you get live reload for the app + deps with no watch processes needed.

Package.json looks something like this (some parts are pretty specific to my setup):

{
  "name": "@namespace/lib",
  "main": "src/index.ts",
  "types": "src/index.d.ts",
  "exports": {
    ".": "./src/index.ts",
    "./dist/*.css": "./src/scss/*.scss"
  },
  "publishConfig": {
    "main": "dist/index.js",
    "module": "dist/index.mjs",
    "types": "dist/index.d.ts",
    "exports": {
      ".": {
        "import": "./dist/index.mjs",
        "require": "./dist/index.js"
      },
      "./dist/*.css": "./dist/*.css"
    },
    "access": "public"
  },
  "sideEffects": [
    "*.css", "*.scss"
  ],
}

One thing I noticed, is as of now this is only reliable with pnpm and I believe yarn (according to someone one Stackoverflow). In my testing, plain npm does not overwrite main for some reason.

If you're not publishing and just need to build, you should be able to achieve the same thing by pointing main (and any others) to the src directory.