As a library author, how do I dual publish to both node and deno?

[brainkim]

Let’s say that I have a library which is published on npm. If that library

1. is written 100% in typescript
2. has no dependencies
3. does not use node.js built-in modules
4. does not use deno.js built-in modules

what is the best way to add support for deno? Is there a good example repository which does this?

Additionally, if I have library which depends only on modules which dual support both node and deno, does this story change?

Related: #2644

[kitsonk]

For Deno, you don't need npm at all. Having it located somewhere on a CDN which the .ts code is supported directly. An example of a rich library would be something like oak.

The biggest "problem" with dual support would be that modules have to end in the extension in Deno. Also Node.js doesn't support TypeScript directly, and doesn't support ESM (without enabling experimental modules). So you would have to have a build a version that would work frictionlessly in Node.js and have that be your distributable to npm. I'm personally not aware of anyone who has created a good build script for Deno .ts to CommonJS .js, but it should/could be possible fairly straight forward. The builder would need to provide a Compiler Host that resolved modules in line with Deno's module resolution and also strip out the .ts extensions on emit.

If you are using Deno namespace APIs, almost the reverse of #2644 will be needed, almost a "browserify" layer that translates Deno APIs to Node.js ones.

[brainkim]

Ahhh yeah the big issue is the requirement of file extensions for deno. I like your suggestion of moduleResolution: "literal" (preserve?) in the related typescript issue. If the typescript team is gonna stick with following the node module resolution algorithm, maybe deno can offer a competitive alternative to tsc which can compile to node compatible code?

Regardless of how successful deno becomes, I see dual-supporting both node and deno as pretty much necessary for most library authors.

Related: microsoft/TypeScript#27481

[kitsonk]

@brainkim regarding the modules resolution, there is also: microsoft/TypeScript#33437 that would be more of an ideal, but getting Node.js to support it... Though Node.js is moving towards extension required modules under the experimental ES support, there would still be a need to change the extensions on the emitted code, and if TypeScript (tsc) touches it, they will get it right for some and wrong for a lot of others.

[beshanoe]

I think for the purpose of creating "isomorphic" Deno libs and CLIs it makes sense to implement sort of Node.js polyfill for all the Deno's built-in APIs, then bundle your tool using deno bundle and after that prepend it with the require from the polyfill, like

const {Deno, TextDecoder, TextEncoder, ...} = require('deno-polyfill');

/// deno generated bundle

probably it also makes sense to convert AMD modules in the original bundle to commonjs

[hayd]

allegedly this is what @YounGoat's deno package on npm is supposed to do...
https://www.npmjs.com/package/deno

[brainkim]

Okay so deno bundle is a piece of the puzzle that I wasn’t aware of. According to the website, deno bundle is a tool which compiles your deno code to AMD (😬). Does this mean I can use typescript or webpack to compile to amd as well and have deno modules consume this?

it makes sense to implement sort of Node.js polyfill for all the Deno's built-in APIs,

This is interesting, but I’m also interested in a simpler use-case where I use 0 deno builtins and 0 node builtins (pretty much vanilla typescript).

[ry]

@brainkim Deno can consume a single JS file or ES modules if you need imports. It doesn’t know about AMD except as an output from the bundle command.

(The AMD bit is really throwing people off - we should modify deno bundle to use https://github.com/denoland/deno/blob/master/deno_typescript/amd_runtime.js to execute the main module.)

[kitsonk]

@ry we talked about it for Bundling v2 (#2475), but it does feel like embedding the runner directly in the bundle so it is a totally self contained script is warranted and knock a few features off the v2 list. That way we can just say "it generates a standalone script" and let people forget about the implementation.

I can start working on this.

[tunnckoCore]

The whole thing would be easier if deno bundle is outputting a better format. I'm curious why AMD is chosen instead of UMD? Or at least it would be pretty trivial to allow users to output different format, through flag?

[kitsonk]

@tunnckoCore again, people are obsessing about something they shouldn't obsess about. 😁

UMD specifically would be useless. That is, depending on the implementation, 2 different module formats and an ability to load as a global script, but when you only need one module format that is loaded into an environment that doesn't support CommonJS for good reason.

If you really want to know the reason though, it goes something like this:

• ES Modules are not concatenatable. All module bundlers that support consuming ESM have to manipulate the modules to some intermediary format, usually proprietary.
• CommonJS modules are not concatenatable. Again bundles rewrite them to some intermediary format.
• UMD modules are not typically concatenatable. UMD is great if you don't know if you are going to load individual modules under Node.js or if you are going to load individual modules in a browser. It isn't so great if you know your final target is a single .js file.
• AMD modules are always concatenatable without any manipulation. It was part of the module design, because it was designed specifically for the web, knowing that you might have a single file that defines lots of modules.
• TypeScript supports outputting both JavaScript and TypeScript (irrespective of source module format) as AMD JavaScript modules into a single file without any additional code.

So we were left with the scenarios of:

• Creating our own emit logic to some non-standard bundle format and dealing with all the edge cases around default exports, imports, dynamic imports and re-exports of ES modules.
• Integrating some 3rd party bundler into the Deno binary that uses a proprietary output format.
• Leverage something that "just works" but needs a minimal module loader added in.

We opted for the last. The only value in the first two is to be able to say we weren't using AMD, which seems like a very odd thing to do.

AMD came into being for some valid good and valid reasons, and ESM doesn't solve all of them. There are very good reasons why ESM isn't concatenatable, and I respect that, but it does mean in a lot of cases, we will be stuck with some form of bundlers in perpetuity (and likely AMD).

[brainkim]

@kitsonk I would consider System.js as a bundle output as well. I think typescript supports it as an output (though last I checked support is not great and System.js can be tricky to set up), it’s more closely aligned with ESmodules, and System.register calls can be robustly concatenated. It just feels more future-proof/standards-aligned IMO.

But most of this is unrelated to this issue, which is, given a pure typescript codebase without dependencies, how do I add support for deno consumers? The two approaches would be:

1. Write for deno, compile for node.
   This can’t be done because there is no stand-alone compiler which understands deno’s explicit module resolution at present. Maybe babel can help us here?
2. Write for node, compile for both node and deno.
   This is doable at present but non-ideal because deno supports typescript natively and having to compile to deno means we need another explicit d.ts pragma to find the typings. Also I need some guidance about what deno as a compilation target looks like. Do I need to compile to amd require modules? Can I use UMD or System.js? Should modules be concatenated/minified?

[tunnckoCore]

@brainkim , last week or two I'm thinking about the second too.

Write Nodejs (js/ts), so need for deno vscode/typescript plugins. Output esm and cjs formats - e.g. dist/index.js and dist/esm/index.js, and put the typings files on the dist/esm (e.g. dist/esm/index.d.ts), so later when Deno users want to use your module then will use some CDN (preferable UNPKG), like https://unpkg.com/@scope/pkg?module. Not sure if Deno will pick up the separate index.d.ts file?

import koko from 'https://unpkg.com/@tunnckocore/kokoko?module';

koko('sasas', 200);
// should error, because both arguments must be `number`

I have two dummy modules, that I'm playing with: @tunnckocore/kokoko and @tunnckocore/qwqwqw check them on unpkg.

The interesting thing is that UNPKG is helping us when using the ?module query param. The only problem, I think... (cc @mjackson) is that it strips the sourceMapping comment when using this query param. I'm not sure if this is a mistake or it should be that way, but anyway.

[tunnckoCore]

Ha... I already wrote to that issue, haha.

Just thought about another idea that spawned after looking around the other "compat" issues and discussions, mainly triggered to look on them today because just seen that denolib/node was archieved with saying it's merged into Deno. So, first, what's that, what that mean?

And second, the another possible idea:

/**
 * ## Deno <-> Node.js Compat (a CLI)
 *
 * - Deno source, two build targets: nodejs (esm & cjs) & browser (esm)
 * - For nodejs target:
 *   + ** rollup babel as very first plugin (with babelrc: false)
 *     to transform url (deno apis) to bare specifier (nodejs apis)
 *   + output cjs and esm rollup "format"s
 * - For browsers target:
 *   + detect if URL specifier ends with `.ts`
 *   + then rollup url import/resolve plugin (as very first),
 *   + then rollup babel (as very first) transform (with babelrc: false)
 *     the typescript to javascript, then allow user-land (allow babelrc)
 *
 */

** another variant of that step is to directly URL import/resolve from Deno and transform that TS to JS, BUT that way consumers of that nodejs code source may have problems if there are any significant differences or bugs or behaviors between what Deno gives, for example, for the path.join and what Node gives for path.join

Write mainly in Deno. And if you want to support both Browsers and Node (and Deno, hence the source) you can use a small smart CLI that can be created, nothing fancy, just a thin translation layer using @rollup so we can output whatever format we want. It's would probably be best to be written in Node.js and then bundled as single file binary.

[brainkim]

I think the best course of action is just to convince typescript peeps to allow explicit file names.

[dubiousjim]

Don't know if this is the best place to post this, but I wanted to share the pattern I came up with for running code on both node and deno. I append this to the end of a .mjs file, which I can run in Node 12+ (using --experimental-modules) or in Deno.

function main (script, args, exit, host, handler) {
  if (args.length !== 1) {
    console.log('Usage: '+script+' FILE');
    exit(1);
  }
  const filename = args[0];
  if (host === 'deno') {
    const decoder = new TextDecoder('utf-8');
    const source = decoder.decode(Deno.readFileSync(filename));
    return handler(source);
  } else {
    return (async () => {
      const fs = await import('fs');
      const path = await import('path');
      const source = fs.readFileSync(path.normalize(filename), 'utf8');
      return handler(source);
    })();
  }
};

if (typeof process === 'object' && process.argv[0].endsWith('/node') && import.meta.url == 'file://' + process.argv[1])
  main(process.argv[1], process.argv.slice(2), process.exit, 'node', MYFILEHANDLER);
else if (typeof Deno === 'object' && import.meta.main)
  main(location && location.pathname, Deno.args, Deno.exit, 'deno', MYFILEHANDLER);

Not up to the needs of library authors, I know, but it's useful to have such a recipe for quick scripts that might be run on either host.

[caspervonb]

Neat. Although Node's WASI support is still behind an experimental flag so I guess we need a couple of months/years until we can reasonably ask library users to use a WASI supporting Node version.

It's not completely unreasonable to ask that today depending on what the library does, e.g is it impractical to do the same thing in JavaScript. WebAssembly is a lot simpler than dealing with whatever incarnation of NAPI we're at now.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[shadowtime2000]

Not stale I guess - I think this could be converted to a GH discussion

[eemeli]

I'm still a bit boggled about how little documentation there's anywhere about writing libraries for Deno, and the existence of this issue keeps reminding me about it.

There's plenty about writing applications, but for libraries, the only thing I can find on deno.land is the "Add a module" button under "Third Party Modules". I just did that for yaml. And no, I don't actually know what a "module" is supposed to look like because there doesn't appear to be any documentation on that. Guess I'll find out when someone complains about it.

[rivy]

TLDR; design pattern for single-source to CJS/ESM/TypeScript + Deno.

Being somewhat frustrated by tooling in this space, and inspired by the yargs solution, I created a design pattern for module creation that is single-source and directly generates both JS and Deno compatible modules, without the use of any post-production conversion tools.

TypeScript is used as the fundamental source language for the module source code. Importantly, all internal module imports have fully specified relative paths (with extensions). All source mode files are written in TypeScript, with '.ts' extensions. But, all internal imports use the '.js' extension. TypeScript's resolution algorithm will find the source file correctly. Yes, this is definitely confusing, but it is the officially supported method available since at least TypeScript-v2.0.

The TypeScript source is transpiled into an ESM which, with some platform abstraction, can be utilized by both NodeJS and Deno. Importantly, for Deno operation, the target transpiled ESM code must be stored back into the repository (eg, in a 'dist' folder).

The platform abstraction (Platform.Adapter) contains all external dependencies and platform-specific built-in variations. Platform-dependent definitions of Platform.Adapter contain the necessary implementation code. For example, from 'os-paths':

// src/platform-adapters/_base.ts
export namespace Platform {
    export type Adapter = {
        readonly env: { readonly get: (_: string) => string | undefined };
        readonly os: { readonly homedir?: () => string; readonly tmpdir?: () => string };
        readonly path: {
            readonly join: (..._: readonly string[]) => string;
            readonly normalize: (_: string) => string;
        };
        readonly process: {
            readonly platform: string;
        };
    };
}

// src/platform-adapters/node.ts
import * as os from 'os';
import * as path from 'path';

import { Platform } from './_base.js';

export const adapter: Platform.Adapter = {
    env: {
        get: (s) => {
            return process.env[s];
        },
    },
    os,
    path,
    process,
};

From working TypeScript code, an Adapt(adapter: Platform.Adapter) factory function can be constructed, with relative ease, as a closure around all platform-dependent code.

Then, when the module is imported, the Adapt() factory function is used which injects an instance of the required platform-specific adapter into the module object, invisible to the consumer. For example, again from 'os-paths':

// src/mod.esm.ts
import { Adapt, OSPaths } from './lib/OSPaths.js';
import { adapter } from './platform-adapters/node.js';

const default_: OSPaths = Adapt(adapter).OSPaths;

export type { OSPaths };
export default default_;

After creating a module, coded in TypeScript, producing ESM-compatible code (which is not simple), the platform abstraction code and Deno implementation is a small, straight-forward changeset (from xdg-app-paths).

For detailed/working examples, I've just finished converting three small modules:

• 'os-paths' (Deno and NPMjs)
• 'xdg-portable' ('xdg' on Deno and NPMjs)
• 'xdg-app-paths' (Deno and NPMjs)

To be sure, I'm not overly happy with the necessary npm/yarn dev scripts and long list of dev dependencies, but such is the state of JS/TypeScript author tooling for cross-platform projects.

Ultimately, all of the modules are available for use by CJS/ESM/TypeScript (via NPMjs, github repo, or jsdelivr CDN) and Deno (via 'deno.land/x', github repo, or jsdelivr CDN). See the respective repositories for further implementation details and READMEs for further usage details.

Of important note, avoiding CJS/TypeScript, and coding the module just to ESM and Deno would still allow the same platform-adapter pattern but with a huge drop in dev complexity, scripting, and tooling.

[ebebbington]

@eemeli that isn’t necessarily related to this issue, but you can find all info in the manual, everything else is knowledge from Js and Ts

[eemeli]

@ebebbington Huh, maybe I just missed it then? I'm looking for answers to the following sorts of things, if they're in the Deno manual maybe you could give me a link?

• What is expected of a well-behaved Deno library module? Should I e.g. have a separate README for Deno users?
• Is there an example somewhere of such a module?
• If my library has some external dependency, can I provide an import map or something for Deno users?
• The "Add a module" flow asks for an optional "subdirectory in your repository that the module to be published is located in." If I want to change my choice for that later, what should I do?

[Soremwar]

@eemeli

What is expected of a well-behaved Deno library module?

There isn't a standard for creating a Deno module. JS libraries are a collection of methods, classes and objects that are made available through the user as they see fit. However well documented libraries tend to contain the following:

• An example folder that users can execute to test functionalities of the library (Example: https://deno.land/x/[email protected]/examples)
• JSDOC blocks and TypeScript typings that can provide examples of usability of each function or class and that can be available in module auto-generated documentation (see: https://deno.land/x/[email protected]/client.ts and https://doc.deno.land/https/deno.land/x/[email protected]/client.ts)
• A well documented README of course, examples shouldn't have to be that different if your library works on Node and Deno and the only differences should be the way of importing (CommonJS vs URL imports) (See https://github.com/Hopding/pdf-lib#deno-usage)

If my library has some external dependency, can I provide an import map or something for Deno users?

Import maps are not mean for libraries, but for end users. It's encouraged for library authors to use deps.ts convention or a similar approach

[zandaqo]

TL;DR: Another way to write and distribute your libraries for both Deno and NPM.

I'm porting my isomorphic libraries to Deno. I'm not using Deno in production myself or expect many of the users to do so, but Deno's built-in toolchain simplifies development/maintenance a lot and I like the opinionated approach Deno takes in it. Imports with extensions did cause some head-scratching, after reading this thread and trying various approaches I ended up using Deno.emit and post-processing to generate JavaScript for NPM. For example, in structurae I'm using this build script to first compile each TS module into ESM with declarations and sourcemaps, and then to remove extra .ts extension in various places as well as fixing file paths in sourcemaps to bring them in sync with what tsc would produce. The resulting .js/.d.ts/.js.map files are published as ESM NPM module and available as such in Node, while the Deno users can load them directly from the .ts source files on GitHub or deno.land.

[wojpawlik]

https://github.com/wojpawlik/deno2node

[KnorpelSenf]

An amazing tool. I started development on a web framework (tailored towards chat bots) which runs on both Deno and Node. It is written for Deno and I used to transpile the hacky way: with a bash script that inserts explicit file extensions and polyfills for fetch via node-fetch etc.

deno2node more or less emerged out of dissatisfaction with this solution, and the are countless plugins (middleware) for all sorts of things around the framework that are already using the 0.x versions of deno2node to obtain Node-compatible JS files from the pure Deno code. We will eventually migrate the core package, too. It works flawlessly and was a major step forward.

In case someone wants to take a look, the simplest plugin that we have is probably this: https://github.com/grammyjs/router

[dsherret]

We've started work on a Deno to npm package build tool. The hope is you add a single Deno script to your Deno-first repo and it will build an npm package from your Deno code (ESM and CommonJS), type check, and run your Deno tests in Node. From there you should just have to cd into the output directory and npm publish.

It's still early days, but I would appreciate if you try it out, throughly inspect its output, and see what problems or challenges you run into so we can fix them: https://github.com/denoland/dnt

[sgwilym]

I've been working on a decent-sized codebase with non-trivial stuff like platform-specific drivers (here) and dnt has covered all our needs. It's really what I've been hoping for!

[zandaqo]

Just checked it out, looks great, the only significant problem is missing source maps.

As a side note, while I appreciate the ambition and ergonomics behind dnt's production of an NPM package in a single command, I would be happy just to have an option for Deno.emit to produce tsc compatible output (i.e. sans extensions), from there I can arrange my package any way I like. For example, I prefer to treat package.json as a source and keep it in a repository, not as a production artifact. I also hate debugging modules with deeply nested folder structure and keep mine flat, and so on. These are all subjective preferences that others likely don't share and I'm afraid the result will be incessant requests for additional configuration options.

[wojpawlik]

You might prefer https://github.com/wojpawlik/deno2node ;)

[zandaqo]

These tools are great for libraries that use Node/Deno specific APIs and/or have extensive dependencies, but for isomorphic and self-contained ones like mine simple compilation is good enough so I'm sticking with Deno.emit, I just wouldn't mind avoiding silly string replacements in post-processing and shortening my build script a bit.

[dsherret]

@zandaqo sourcemaps are on my todo list and I believe trivial to implement at this point. Opened denoland/dnt#13

The package.json is essentially the package object provided to the build options. The outputted package.json will include other stuff discovered during the build, but anything in there can be overwritten.

I would be happy just to have an option for Deno.emit to produce tsc compatible output (i.e. sans extensions), from there I can arrange my package any way I like.

You may be interested in the lower level API that does this that’s shown at the bottom of the readme https://github.com/denoland/dnt#js-api-example (it can be used from JS or Rust)

[zandaqo]

You may be interested in the lower level API that does this that’s shown at the bottom of the readme https://github.com/denoland/dnt#js-api-example (it can be used from JS or Rust)

That looks interesting, any way to feed its output into Deno.emit? So far I'm trying and failing with the following:

  const outputResult = await transform({
    entryPoints: ["./index.ts"],
    shimPackageName: "deno.ns",
  });
  const sources = outputResult.main.files.reduce((a, i) => {
    // tranforming names to match imports in the source code
    a[`./${i.filePath.replace(/\.ts/, ".js")}`] = i.fileText; 
    return a;
  }, {} as Record<string, string>);

  const { files } = await Deno.emit("./index.js", {
    compilerOptions: {
      "target": "es2020",
      "module": "es2015",
      "sourceMap": true,
      "inlineSources": true,
      "declaration": true,
      "removeComments": false,
    },
    sources,
  });
// files are empty and no errors thrown

I'm guessing Deno.emit is thrown off by extensions: the imports in code end on .js while containing ts code and emit has no way of knowing that.

[dsherret]

@zandaqo Deno.emit won't understand it because the output is tsc compatible (it's not Deno compatible as the original source was). You can write the output to the file system and then use tsc or else you can put the output into a bundler to transform it to JS. Alternatively, you could feed the output to the TS compiler API and then emit it as JavaScript (which is what the build function does internally).

[cowboyd]

Since we started using dnt to maintain most of our npm packages, I wrote up a bit on how we do this use dnt + git tags + github actions to publish to both deno.land and npm. It might be of help to folks coming here https://frontside.com/blog/2023-04-27-deno-is-the-easiest-way-to-author-npm-packages/

[Conaclos]

Is there plan to ship dnt with the deno binary?

[cowboyd]

I think it might be more ergonomic as a library since you end up scripting it a bit, and if the primary entry point were the deno binary, then you'd be doing your scripting in bash instead of JavaScript.

[hckhanh]

@brainkim, I think you should try https://jsr.io

[jake-danton]

I would love to add JSR to our flow, but it doesn't integrate with DNT nor does it have true import map support, so it would be too much more work for an unproven new registry.

We use modern web tooling, meaning we are doing all dependency management using import maps pulling from esm.sh, etc. and DNT has made it so that we can easily keep to that flow and publish NPM packages that show up on esm.sh, etc.

To use JSR, we would have to have duplicate dependencies on jsr:* and npm:* that we manually keep in sync with our true import maps (since you can't use jsr:* dependencies in browsers. And it wouldn't give us anything new except putting our modules on JSR when they are already on esm.sh... so it just seems like unnecessary lock-in to using a JSR flow when we already have a more flexible and robust one.

That being said, I am keeping an eye on JSR updates to see if it ever makes more sense. If they integrate it with DNT or allow it to import from esm.sh or even support some https://jsr.io/@scope/package URL to specify a JSR dependency that was consistent with browsers then it would probably be a nicer flow than DNT.