-
Notifications
You must be signed in to change notification settings - Fork 524
Home
rules_nodejs 3.6.0 introduced exports_directories_only attributes on npm_install and yarn_install that reduce the number of files Bazel has to work with, making builds a lot faster. See https://docs.aspect.dev/bazelbuild/rules_nodejs/3.6.0/docs/builtins.html#npm_install-exports_directories_only
ts_library doesn't work with this install mode at all.
rules_sass doesn't really support third-party dependencies directly. Many people have worked around this by passing files from node_modules into a sass_library rule as discussed in https://github.com/bazelbuild/rules_sass/issues/98 However that approach won't work with directory artifacts.
We suggest switching to the vanilla sass rule instead, see this example or the following macros which should align with the rules_sass rules:
def sass_library(name, srcs, deps = [], **args):
native.filegroup(
name = name,
srcs = srcs + deps,
**args
)
def sass_binary(name, src, output_style = "compressed", include_paths = [], output_name = None, deps = [], **args):
output_name = output_name if output_name else src.replace(".scss", "").replace(":", "") + ".css"
sass(
name = name,
outs = [output_name, "%s.map" % output_name],
args = [
"$(execpath %s)" % src,
"$(execpath %s)" % output_name,
"--style=%s" % output_style,
"--no-error-css", # don't output any files on error
"--quiet", # don't output warnings
"--load-path=external/npm/node_modules/", # for node modules
"--load-path=$(BINDIR)", # for generated files
] + [
"--load-path=%s" % p # custom include_paths
for p in include_paths
] + [
"--load-path=$(BINDIR)/%s" % p # custom include_paths for generated files
for p in include_paths
],
data = [src] + deps,
**args
)
--bazel_patch_module_resolver now defaults to false (#2324)
This option to nodejs_binary causes us to monkey-patch the implementation of require() in NodeJS, so that it becomes "Bazel-aware" and
knows how to resolve between the source folder and the output folder. However it has always had a compatibility problem. Subprocesses and code that uses alternate resolvers like https://www.npmjs.com/package/resolve don't get the patches, and patching require() is a risky undertaking that can cause bugs.
If this breaks you, the quickest fix is to flip the flag back on a nodejs_binary/nodejs_test/npm_package_bin with templated_args = ["--bazel_patch_module_resolver"], see #2344 as an example.
One effect of the patched require() is to make the source tree and the bazel-out output tree appear to be joined by reading from Bazel's "runfiles" symlink forest. This can be achieved instead in programs you control (including tests) by using our Runfiles helper library. See #2341 as an example. Note that since the Runfiles helper library is distributed as part of our "built-in" package installed in your WORKSPACE file, you reference its path with an Environment variable like require(process.env['BAZEL_NODE_RUNFILES_HELPER']) unlike an npm-installed package.
Removal of the @bazel/karma package (#2313)
The @bazel/karma package has now been deprecated and will no longer be published. The existing karma_web_test and karma_web_test_suite rules have moved to @bazel/concatjs (see Introducing @bazel/concatjs for more information on this new package).
Remove references to npm_bazel_karma_dependencies in the WORKSPACE file:
load("@npm//@bazel/karma:package.bzl", "npm_bazel_karma_dependencies")
npm_bazel_karma_dependencies()
Buildozer can be used to update the load statements in existing BUILD[.bazel] files.
npx @bazel/buildozer 'replace_load @npm//@bazel/concatjs:index.bzl karma_web_test_suite' //...:__pkg__
npx @bazel/buildozer 'replace_load @npm//@bazel/concatjs:index.bzl karma_web_test' //...:__pkg__
Finally, remove the package.json dependency on @bazel/karma and replace it with @bazel/concatjs
In some cases, it's likely possible to use the generated karma_test rules directly by installing karma and using load("@npm//karma:index.bzl", "karma_test").
If you also had a peer dependency on jasmine-core for @bazel/karma this can also be removed as it is no longer required (see 2253)
Rename ts_devserver -> concatjs_devserver (#2239)
ts_devserver has been renamed to concactjs_devserver and moved into the @bazel/concatjs package (see Introducing @bazel/concatjs for more information on this new package).
Buildozer can be used to update the load statements in existing BUILD[.bazel] files.
npx @bazel/buildozer 'new_load @npm//@bazel/concatjs:index.bzl concatjs_devserver' //...:__pkg__
npx @bazel/buildozer 'set kind concatjs_devserver' //...:%ts_devserver
npx @bazel/buildozer 'fix unusedLoads' //...:__pkg__
3.0 introduces a new NPM package, @bazel/concatjs. This is now the home for all concatjs based rules, such as karma_web_test and ts_devserver. This package closely mirrors the internal toolchain used in Google. Users may decide to use this toolchain, but there are tradeoffs in complexity (for example, ts_devserver requires all sources as named AMD bundles), it's often recommended to use alternatives to this package where possible.
Removal of install_bazel_dependencies (#1877)
In a previous release, install_bazel_dependencies was deprecated but still needed in some situations (eg when loading from @angular/bazel). As @angular/bazel is now deprecated, this is no longer required, and has now been removed.
Remove the following from WORKSPACE if it is still present:
load("@npm//:install_bazel_dependencies.bzl", "install_bazel_dependencies")
install_bazel_dependencies(suppress_warning = True)
Removal of pkg_npm#replace_with_version (#2312)
replace_with_version on pkg_npm was deprecated and has now been removed. Users should use substitutions instead.
pkg_npm(
name = "publish",
...
replace_with_version = "my_version_placeholder",
substitutions = {
"replace_me": "replaced",
},
...
)
becomes:
pkg_npm(
name = "publish",
...
substitutions = {
"my_version_placeholder": "{BUILD_SCM_VERSION}",
"replace_me": "replaced",
},
...
)
Rollup config_file no longer substitutes bazel_stamp_file (#2312)
The config_file used for rollup_bundle no longer substitutes bazel_stamp_file, only bazel_version_file and bazel_info_file are now substituted. Rename bazel_stamp_file to bazel_version_file as they were both substituted with the same value.
pkg_web/pkg_web.bzl no longer exports move_files as public API (#2159)
The move_files function from pkg_web.bzl is no longer public API and its usages should be removed.
ts_project#extends no longer takes a list (#2266)
Previously, the extends attr on ts_project took a list, it now takes a single label from a ts_config rule, or a tsconfig.json file.
strict_visibility on yarn_install and npm_install now defaults True (#2199)
In 2.x, the npm_install and yarn_install rules introduced a strict_visibility flag which applied limited visibility to transitive dependencies from resolved from the package.json file. This flag has now been flipped to default to True. All dependencies that are now referenced from external npm / yarn install rules must now be present in the given package.json file.
To opt out of this behaviour, set strict_visibility to False on the npm_install or yarn_install repo rule:
npm_install(
name = "npm",
strict_visibility = False,
...
)
yarn_install(
name = "npm",
strict_visibility = False,
...
)
tsetse, our third-party strictness plugin for TypeScript under ts_library, now checks for more patterns, for example you might get an error like
error TS21231: [tsetse] type assert `JSON.parse() as SomeExplicitType` for type & optimization safety.
See http://tsetse.info/must-type-assert-json-parse.
you can either repair the code in the way it suggests, or disable the check. To do the latter, in your tsconfig.json do this:
{
"compilerOptions": {
"plugins": [
{"name": "@bazel/tsetse", "disabledRules": ["must-type-assert-json-parse"]}
]
}
}
We removed the node_modules attribute from nodejs_binary, nodejs_test, jasmine_node_test & ts_library.
If you are using the node_modules attribute, you can simply add the target specified there to the data or deps attribute of the rule instead.
nodejs_test(
name = "test",
data = [
"test.js",
"@npm//:node_modules",
],
entry_point = "test.js",
)
or
ts_library(
name = "lib",
srcs = glob(["*.ts"]),
tsconfig = ":tsconfig.json",
deps = ["@npm//:node_modules"],
)
We also dropped support for filegroup based node_modules target and removed node_modules_filegroup from index.bzl.
If you are using this feature for user-managed deps, you must now a js_library target with external_npm_package set to True instead.
js_library(
name = "node_modules",
srcs = glob(
include = [
"node_modules/**/*.js",
"node_modules/**/*.d.ts",
"node_modules/**/*.json",
"node_modules/.bin/*",
],
exclude = [
# Files under test & docs may contain file names that
# are not legal Bazel labels (e.g.,
# node_modules/ecstatic/test/public/中文/檔案.html)
"node_modules/**/test/**",
"node_modules/**/docs/**",
# Files with spaces in the name are not legal Bazel labels
"node_modules/**/* */**",
"node_modules/**/* *",
],
),
# Provide ExternalNpmPackageInfo which is used by downstream rules
# that use these npm dependencies
external_npm_package = True,
)
nodejs_test(
name = "test",
data = [
"test.js",
":node_modules",
],
entry_point = "test.js",
)
See examples/user_managed_deps for a working example of user-managed npm dependencies
Buildozer can be used to update the load statements in existing BUILD[.bazel] files. The following will migrate all load statements within the current workspace from the @npm_bazel_ format, to @npm//@bazel/ format
npx @bazel/buildozer 'substitute_load ^@npm_bazel_(.*?)//:index\.bzl$ @npm//@bazel/${1}:index.bzl' //...:__pkg__
The install_source_map_support attribute is removed from nodejs_binary. source-map-support is vendored in at /third_party/github.com/source-map-support so it can always be installed. Simply remove any usage of that attribute. The following buildozer command will remove this off all nodejs_binary rules in the workspace
npx @bazel/buildozer 'remove install_source_map_support' //...:%nodejs_binary
If your WORKSPACE file has workspace(name="foo") and your code expects to import("foo/thing"), you need to make this explicit. Alternately you can keep the old behavior with this flag: https://github.com/bazelbuild/rules_nodejs/pull/2175
(Note, ts_library converts relative imports to absolute, so even import("./thing") will become import("foo/thing") in the JS files in bazel-out)
In your root BUILD.bazel file, add a pkg_npm between the declaration site and the use site of these JS files. For example:
load("@build_bazel_rules_nodejs//:index.bzl", "pkg_npm")
pkg_npm(
name = "foo_pkg",
# Makes this code importable as "foo"
package_name = "foo",
deps = [":lib_that_declares_foo_thing"],
)
my_test(
data = ["foo_pkg"],
)If you have upgraded to Angular Ivy, we no longer recommend depending on the deprecated @angular/bazel package. The only rule needed from it was ng_module which you can replace with the faster equivalent ts_library(use_angular_plugin=True)
Buildozer can be used to edit multiple BUILD files at once, with multiple commands.
Create a text with the following commands below. The first line can be changed to provide more focused list of packages to add the ts_library rule too, however ones that aren't then used will be removed.
new_load @npm//@bazel/typescript:index.bzl ts_library|//...:__pkg__
set use_angular_plugin True|//...:%ng_module
rename assets angular_assets|//...:%ng_module
set kind ts_library|//...:%ng_module
fix unusedLoads|//...:__pkg__
Optional worker support with the following, added below the previous set ... line
set supports_workers True|//...:%ng_module
Then run the following to execute (assuming the file is named commands.txt)
npx @bazel/buildozer -f commands.txt
If you still use Angular's old compiler ("View Engine") then consult our examples/angular_view_engine directory. We consider View Engine plus Bazel to be deprecated, but are doing our best to keep it working.
- you'll still need to use the
@angular/bazelpackage, which requires you keep theinstall_bazel_dependenciescall in your/WORKSPACE - a patch is required, see the patches/ directory under the example
To use Bazel, it obviously needs to be downloaded to your computer. It also needs to be the right version to be compatible with the code you're building.
This is the same problem we have with Node, which you need to fetch and correctly version. The community-accepted solution for this is to use nvm and a .nvmrc file. .nvmrc says what version(s) of node work with the project, and nvm downloads a node.js version to your machine if needed.
Bazel has a nearly identical solution. The @bazel/bazelisk package is a Bazel wrapper that downloads the correct version of Bazel based on your .bazelversion file.
Now that this is available, we plan to deprecate our old mechanism for downloading and versioning Bazel in npm-based projects, the @bazel/bazel package.
The bazel binary is pretty big (compared with typical JS tooling) at 40MB. But it's a native binary, so we actually had three optional dependencies from the @bazel/bazel package to platform-specific ones. This is a nice solution, except that we found that in practice, both yarn and npm have behavior (bug?) where they download the tarballs for all three packages even though it was known beforehand that two of them are not compatible with the local platform. So you end up with 120MB download. We didn't succeed in escalating this issue with the package managers.
Second, by having Bazel versioned in the package.json file, we were not compatible with the rest of Bazel environment. For example, if you downloaded Bazel outside of the npm mirror, you wouldn't get any warning that you have the wrong version. Bazel itself knows to look for the .bazelversion file so we want that to work for JS developers too.
Next, we had a governance issue. rules_nodejs maintainers were mirroring Bazel to npm, but we didn't always mirror all the versions (like release candidates). The release process should be hosted entirely by the tool itself so every version is available and you can rely on the release engineer doing all the steps. Since Bazelisk now releases to npm (thanks @philwo!), we get out of the mirroring business and you can use any version of Bazel.
Finally, bazelisk has better support as a Bazel wrapper than the npm mirror had. It knows to execute the tools/bazel wrapper if it exists, and can be hooked up with command-line completion. It even has features to help you migrate past Bazel breaking changes (akin to the ng update command in Angular ecosystem)
- Remove your dependency on
@bazel/bazelwhich is deprecated - Add a devDependency on
@bazel/bazeliskinstead. The version doesn't matter much since it's just a wrapper. (you could even use a global install ofbazeliskif you'd rather have the tool on your$PATH) - Add a
.bazelversionfile in your workspace root containing the version you want to pin to. See the bazelisk readme - In your
package.json#scripts, callbazeliskrather thanbazel -
@bazel/bazelhad a transitive dependency on a workaround package,@bazel/hide-build-files, whose purpose was to prevent Bazel from seeing thenode_modules/**/BUILDfiles under external repositories. You can either- update Bazel to >= 2.1 and rules_nodejs >= 1.3.0 since the workaround is no longer needed, or
- add a devDependency on
@bazel/hide-build-filesto keep the workaround in place
If you had been relying on a
bazelcommand in your$PATH, you can still do this by aliasing it to thebazeliskcommand. You can also add a"bazel": "bazelisk"alias in thepackage.json#scripts.
Note, Bazelisk is adding more robust handling of network constraints. Check this issue if you need to download via a corporate proxy: https://github.com/bazelbuild/bazelisk/issues/115.
For consistency, rules_nodejs always loads from a file called index.bzl since this matches the node ecosystem, and Bazel itself doesn't have a single standard (defs.bzl, build_defs.bzl and others are common)
If you depend on another ruleset that still depends on defs.bzl, you must update. For rules_sass, the minimum is:
http_archive(
name = "io_bazel_rules_sass",
sha256 = "617e444f47a1f3e25eb1b6f8e88a2451d54a2afdc7c50518861d9f706fc8baaa",
urls = [
"https://github.com/bazelbuild/rules_sass/archive/1.23.7.zip",
"https://mirror.bazel.build/github.com/bazelbuild/rules_sass/archive/1.23.7.zip",
],
strip_prefix = "rules_sass-1.23.7",
)For rules_docker, the minimum release is https://github.com/bazelbuild/rules_docker/releases/tag/v0.13.0
If you have trouble updating to the latest rules_docker, you can also apply a simple local patch to the version you're currently on. For example, within rules_nodejs we added one "patches" attribute to our rules_docker fetch:
http_archive(
name = "io_bazel_rules_docker",
patches = ["//:rules_docker.patch"], # <-- add this to apply a patch
sha256 = "7d453450e1eb70e238eea6b31f4115607ec1200e91afea01c25f9804f37e39c8",
strip_prefix = "rules_docker-0.10.0",
urls = ["https://github.com/bazelbuild/rules_docker/archive/v0.10.0.tar.gz"],
)and add a file rules_docker.patch next to WORKSPACE like this:
diff --git nodejs/image.bzl nodejs/image.bzl
index a01ea3e..617aa06 100644
--- nodejs/image.bzl
+++ nodejs/image.bzl
@@ -17,7 +17,7 @@ The signature of this rule is compatible with nodejs_binary.
"""
load("@bazel_skylib//lib:dicts.bzl", "dicts")
-load("@build_bazel_rules_nodejs//:defs.bzl", "nodejs_binary")
+load("@build_bazel_rules_nodejs//:index.bzl", "nodejs_binary")
load(
"//container:container.bzl",
"container_pull",
The rollup_bundle rule from the //internal package has a bunch of problems, and has been replaced by a new implementation. The new implementation is much simpler, does just one thing, and is meant to run nearly identically to how you would use Rollup outside of Bazel. The //internal/rollup package will be removed before 1.0. Some work is required to move to the new rule.
High-level notes:
- Terser minification was run in the old rule when you requested a
.min.jsoutput.rollup_bundleandterser_minifiedare now separate rules. - New rules use a peerDependency, so you need to install
rollup/terserif you haven't already - We don't have a downleveling step built-in, so if you need es5 output you can use a rollup plugin to do it. Note that this means differential loading scenarios need two rollup builds, which is slow - we'll have a better way to downlevel rollup outputs before we remove the old rollup_bundle rule.
- Works with
ng_modulewith a minimum@angular/bazelversion of9.0.0-next.11.
Rough steps to follow:
- Replace
load("@build_bazel_rules_nodejs//:defs.bzl", "rollup_bundle")withload("@npm_bazel_rollup//:index.bzl", "rollup_bundle") - If you reference a
min.jsoutput, also addload("@npm_bazel_terser//:index.bzl", "terser_minified") - Generally, you can get the same
.min.jsoutput as before just by addingterser_minifiedafter the existingrollup_bundle:# this will output bundle.min.js, but not a predeclared label terser_minified( name = "bundle.min", # assuming your rollup_bundle rule has name="bundle" src = "bundle.js", ) - The new terser rule cannot provide a label for
bundle.min.js(because it doesn't know if the input is a directory until after the predeclared outputs are determined), so you can't reference that in your other targets. You'll have to usebundle.min. If a downstream target expects a label which produces a single file, you can disable thesourcemapfeature interser_minifiedso that the .js output is the only one. - If you used
enable_code_splitting = False, you can just remove it. The new rule only produces chunked output when given aoutput_dir = Trueattribute. -
additional_entry_pointsare now combined with the primary entry_point in a singleentry_pointsattribute. You can give TypeScript files as the entry point, likeindex.tsand the rule will resolve that to the.jsoutput file.
Thanks to @jbedard for the design doc that motivated the changes.
These are detailed migration notes for https://github.com/bazelbuild/rules_nodejs/releases/tag/0.30.0
In prior releases, Bazel installed a copy of the node_modules tree in its private external directory. This meant that you typically had to install dependencies twice, and if you wanted to do some println debugging in the node_modules directory, you had to know where this other copy is installed.
As of Bazel 0.26, there is a new feature called "managed directories" which allows Bazel to reference the same node_modules directory that your package manager creates and that the editor uses to resolve tools like TypeScript. This feature is now enabled by default.
You must update your Bazel dependency and/or your locally installed Bazel to 0.26 or greater before updating rules_nodejs to 0.30.
To use managed directories, make these changes to your app:
- if you are using the
@bazel/bazelnpm package, update it to"@bazel/bazel": "0.26.0-rc13"or newer - update
WORKSPACEto rules_nodejs 0.30.0 release - if you are using the
@bazel/typescriptnpm package this needs to be updated to 0.30.0 as well - tell Bazel to opt-in to this feature by adding to
.bazelrc:build --experimental_allow_incremental_repository_updates query --experimental_allow_incremental_repository_updates - make sure you don’t have multiple
yarn_install/npm_installrules referencing the same package.json - add to
WORKSPACEthemanaged_directoriesof all youryarn_install/npm_installrules. This typically looks like:workspace( name = "my_wksp", managed_directories = {"@npm": ["node_modules"]} )
- add the same
node_modulesdirectories as lines in.bazelignore- these contain BUILD files that only work when Bazel references them externally
You can turn managed directories off for individual yarn_install or npm_install rules with the symlink_node_modules = False attribute. If you you have multiple yarn_install and/or npm_install rules, any that have symlink_node_modules = False do not need their node_modules folders be added to managed_directories in your WORKSPACE or in .bazelignore. If you add them anyway it won't break anything.
If the managed_directories feature breaks you, please file an issue on this repo to let us know. Then you can proceed to 0.30 release without the feature by adding symlink_node_modules = False to any affected npm_install/yarn_install rules.
Due to an issue on Windows with managed directories and the node_modules/.bin folder, no BUILD file is generated under node_mdoules/.bin. If you were relying on targets in this BUILD file prior to 0.30 this will be an additional breaking change for you. You can still reference files in .bin with labels such as @npm//:node_modules/.bin/foobar or if you require a filegroup group with a multiple files in the .bin folder you can use manual_build_file_contents of your yarn_install or npm_install rule to add the appropriate filegroup to the root @npm build file .
For example,
yarn_install(
name = "npm",
package_json = "//:package.json",
yarn_lock = "//:yarn.lock",
manual_build_file_contents = """
filegroup(
name = "bin_files",
srcs = glob(["node_modules/.bin/*"]),
)"""
)
would create target @npm//:bin_files which would contain all files in the node_modules/.bin folder. If this doesn't work for you please file an issue and we'll help you find an appropriate fix.
We are making a breaking change for version 0.13, related to loading npm dependencies.
nodejs_binary and other rules that depend on it typically accept a node_modules attribute. This attribute was optional and defaulted to @//:node_modules, which means "find the node_modules target in the root of the workspace where the build is happening."
However, we now are moving to a model where npm dependencies can be expressed as deps (or data) of rules. That means that a nodejs_binary that requests specific npm modules as dependencies wants to get no additional modules through the node_modules attribute.
To make this possible, node_modules will now default to an empty filegroup. To unbreak your code, you can typically just set node_modules = "@//:node_modules" - that is, inline the previous default into an explicit attribute.
As of 0.13.2, this also applies to rollup_bundle which has a node_modules attribute that now defaults to an empty filegroup.
Bazel version 0.17.1 is required as of version 0.13.0 in order to support fined grained npm dependency labels. Bazel 0.17.1 adds support for @ in label names which is required for labels such as @npm//:@types/jasmine.
We used to take all transitive JS files and hand them to Jasmine as specs.
This causes a problem when some files have global side-effects (e.g. calling process.exit)
We now filter only files ending with spec.js or test.js and ask Jasmine to execute those.