Migrating from canon to Next.js

canon-core has many outdated dependencies, and Datawheel as a company have decided to start moving our front-end stack away from a proprietary front-end framework in favor of a more widely adopted, supported, and updated framework (such as Next.js). This guide walks through the process of porting an already existing canon-core site to Next.js, and any pitfalls and caveats we found along the way. We expect this guide to grow slowly over time, as more sites and features are ported to Next.js.

The initial impetus for this guide was to get around some critical package vulnerabilities in a consulting project where the client's DevOps team would not host the site until all critical issues were addressed.

Upgrade Node

canon-core is currently locked to Node v12, which reached end of life in April 2022 (no more security fixes). As of this writing the currently active LTS version is 16, and jumping to nextjs allows us to use this version.

Docker

If your project uses Docker for deploying, you will need to update the Dockerfile to Node v16:

# starting point: an image of node-12
FROM node:16

# create the app directory inside the image
WORKDIR /usr/src/app

# install app dependencies from the files package.json and package-lock.json
# installing before transfering the app files allows us to take advantage of cached Docker layers
COPY package*.json ./

RUN npm install

# If you are building your code for production
# RUN npm ci --only=production

# transfer all the app files to the working directory
COPY ./ ./

# build the app
RUN npm run build

# start the app on image startup
CMD ["npm", "run", "start"]

The build and start scripts remain unchanged, so most of the build/deployment process is left untouched. You will also need to remove index.js from your .dockerignore file.

Next.js Documentation: Self-Hosting

Local Machine

When running the site locally, whether testing a production build or using npm run dev to live develop, you must be on version 16 of Node. If you currently need Node v12 installed for use with other canon-core sites, we suggest using nvm to switch between versions as needed (Node Version Manager). Install it using Homebrew:

brew install nvm

Then, use it to install a version of Node 16:

nvm install 16

You can then use nvm use <version> to switch between versions, with system allowing you to return back to your globally installed version (nvm use system or nvm use 16).

Fix Dependencies

Goodbye Canon: npm uninstall @datawheel/canon-core
Hello Next: npm i next react@latest react-dom@latest

Find Nested `canon-core` Dependencies

As a best practice, all libraries being used in an import statement in a project should be listed in that projects package.json dependencies. As canon was rapidly developed and used, many bad habits were formed by quickly importing nested dependencies from canon-core because "we knew they were there". Now that we do not have canon-core installed, we need to identify all missing dependencies that were being imported this way. As you progress through this migration, you will continually find a few small dependencies to add along the way. As a starting point, some commone examples are:

@blueprintjs/core
react-redux
axios
classnames

React Table

react-table is not currently compatible with the latest version of React. You will have to upgrade react-table to the latest version, which is a substantial refactor (guide coming soon).

Setup Files

env vars

Replace all instances of CANON_CONST_ with NEXT_PUBLIC_
make sure all usages are inline (and not using object destructures)

🛑 Incorrect Import 🛑

const {CANON_CONST_TESSERACT} = process.env;

✅ Correct Import ✅

const NEXT_PUBLIC_TESSERACT = process.env.NEXT_PUBLIC_TESSERACT;

Next.js Documentation: Environment Variables

.gitignore

We need to add the new .next directory to .gitignore, and remove these old canon-core specific directories and files:

# these static files will be generated on build
**/static/assets
**/static/reports
**/*.bundle.js

A simple example of a .gitignore should look something like this:

npm-debug.log*
.DS_Store

# node modules should never be synced with git
node_modules

# nextjs build files
.next

# environment variable files for autoenv and direnv
.env
.envrc

# docker files
dockerfiles/nginx/certs/*
dockerfiles/nginx/conf.d/default.conf

# pm2 ecosystem config
ecosystem.config.js

package.json

We need to update our "scripts" in package.json to point to the new nextjs scripts:

  ...
  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start",
    "lint": "next lint"
  },
  ...

Next.js Documentation: CLI

.eslintrc

Next.js comes with some fairly strict/opinionated linting by default, and if you were previously using the linting provided by @datawheel/eslint-config, then you will need to combine the two by creating a small .eslintrc file the root directory:

{
  "extends": [
    "@datawheel",
    "next"
  ]
}

Next.js Documentation: ESLint

helmet.js

The default <head> values stored in app/helmet.js need to be migrated to JSX and use the Head component exported from next/head. Most commonly this means merging these defaults with an already existing HelmetWrapper type of component. Here is an example:

import React from "react";
import {useRouter} from "next/router";
import Head from "next/head";

const HelmetWrapper = ({info = {}}) => {
  const baseUrl = process.env.NEXT_PUBLIC_BASE_URL;

  const router = useRouter();
  const {pathname} = router;

  const defaults = {
    title: info?.title ? `${info?.title}` : "Unlocking Knowledge",
    desc: info?.desc ?? "A journey to the heart of knowledge.",
    img: info?.img ?? `${baseUrl}/images/share/unlocking-knowledge.png`,
    url: `${baseUrl}${pathname}`,
    locale: "en"
  };

  return (
    <Head>

      <title>{defaults.title}</title>

      <meta httpEquiv="X-UA-Compatible" content="IE=edge" />
      <meta name="viewport" content="width=device-width, initial-scale=1" />
      <meta name="mobile-web-app-capable" content="yes" />
      <meta name="apple-mobile-web-app-capable" content="yes" />
      <meta name="apple-mobile-web-app-status-bar-style" content="black" />
      <meta name="apple-mobile-web-app-title" content={defaults.title} />

      <meta name="title" content={defaults.title} />
      <meta name="description" content={defaults.desc} />

      <meta name="twitter:title" content={defaults.title} />
      <meta name="twitter:description" content={defaults.desc} />
      <meta name="twitter:image" content={defaults.img} />
      <meta name="twitter:card" content="summary_large_image" />

      <meta property="og:title" content={defaults.title} />
      <meta property="og:description" content={defaults.desc} />
      <meta property="og:locale" content={defaults.locale} />
      <meta property="og:url" content={defaults.url} />
      <meta property="og:image" content={defaults.img} />

    </Head>
  );
};

export default HelmetWrapper;

Next.js Documentation: next/head

Quick Changes

remove canon.js (unused, but you may want to keep it around while you develop if there is anything in there)
rename and move app/App.jsx to /pages/_app.js
rename and move the current Homepage component to /pages/index.jsx
move /app/components to root /components
delete /types folder (if exists)
rename /static to /public

Move Components

The React components and CSS currently located in the app/ directory of a canon-core package need to be split into 2 separate folders:

pages/ - components that are attached to routes in app/routes.jsx. Next.js does not use a "routes" file, and relies on very specific folder/file nesting and naming conventions (see docs here).
components/ - all components that are not directly attached to a route (even if they are only used on one page), cannot be inside of the pages/ directory. Common practices puts them all in one large nested components/ directory, but they can really be any where except pages/.

Next.js Documentation: Pages

Convert CSS

normalize

canon-core comes prebuilt with a few hardcoded CSS imports, mainly normalize.css to reset common styles across browsers. Install it like any other dependency:

npm i normalize.css

And then add the global import to the top of pages/_app.js (along with two blueprint imports, if blueprint is used in your project):

import "normalize.css";
import "@blueprintjs/core/lib/css/blueprint.css";
import "@blueprintjs/icons/lib/css/blueprint-icons.css";

Components

Out of the box, Next.js supports two types of style imports:

global styles imported into _app.js (to be included for all pages)
scoped CSS Modules that apply CSS in JS.

The benefits of CSS Modules include:

strict scoping of styles so that they do not interfere with other components
chunking production build so that each page only serves styles applicable to it

Since it can be a large effort to convert current styles to CSS Modules, the quick migration is to import all of your current styles at the top of _app.js like this:

import "$root/components/Nav.css";
import "$root/components/Footer.css";
import "$root/pages/About/index.css";

Next.js Documentation: Built-In CSS Support

style.yml

Convert the current app/style.yml file to JavaScript using an online converter (like this one), and copy/paste the result into a new file at root called (including module.exports):

./postcss.variables.js

You may be missing certain fallback values that we injected by canon, so if you see console warnings stating this, take a look at this CSS file that includes all of the canon fallbacks and add them to your new postcss.variables.js file. Here is an example of what this file should look like in terms of structure:

module.exports = {
  "navy": "#001D3A",
  "navy-light": "#193552",
  "bahama-blue": "#2B5681",
  "blood-orange": "#FD3001",
  "light-orange": "#FC8300",
  "light-blue": "#0074E3",
  "mint": "#0DD1AB",
  ...
};

PostCSS

canon-core supports about a dozen of postcss plugins to enable advanced CSS features that get transpiled down to browser code at runtime. If you have migrated your CSS to modules and are using Next.js build-in CSS, this step should not be necessary. Otherwise, create this file at root:

Install postcss plugins: ./postcss.config.js

And the copy/paste the following code into that file. This code injects your postcss.variables.js into the code, as well as enabling a custom list of postcss plugins (which all need to be installed one by one).

const variables = require("./postcss.variables");
for (const key in variables) {
  if ({}.hasOwnProperty.call(variables, key) && !key.includes(" ")) {
    const fallbackRegex = /var\((\-\-[^\)]+)\)/gm;
    let match;
    const testString = variables[key];
    do {
      match = fallbackRegex.exec(testString);
      if (match) variables[key] = variables[key].replace(match[0], variables[match[1]]);
    } while (match);
  }
}

module.exports = {
  plugins: [
    "postcss-import",
    "postcss-mixins",
    [
      "postcss-preset-env",
      {
        autoprefixer: {
          flexbox: "no-2009"
        },
        stage: 3,
        features: {
          "custom-properties": true,
          "focus-within-pseudo-class": false,
          "nesting-rules": true
        }
      }
    ],
    [
      "postcss-css-variables",
      {
        preserve: true,
        variables
      }
    ],
    "postcss-flexbugs-fixes"
  ]
};

For this basic example, you would need to install the following dependencies: npm i postcss-import postcss-mixins postcss-preset-env postcss-css-variables postcss-flexbugs-fixes

Depending on the project and it's usage of plugins, the following libraries are also included by default in canon-core, and may be necessary for your project:

lost
pixrem
postcss-each
postcss-for
postcss-map
postcss-conditionals

And finally, replace all front-end imports of style.yml with $root/postcss.variables, so something like this: import style from "style.yml";

Would become this: import style from "$root/postcss.variables";

Next.js Documentation: Customizing PostCSS Config

Update Routing

Change Link Components

replace all import {Link} from "react-router"; imports with import Link from "next/link";
replace all to props to href
if nesting DOM elements inside of the <Link> component (not just simple text links), put an <a> tag inside of the <Link> and move className to that new <a> tag.

As an example, you would change the following component:

<Link to="/about" className="link">
  <div className="link-icon">
    <SVG src="/images/icon-see-large.svg" width="50%" height="50%" />
  </div>
  <h2 className="link-title">About</h2>
</Link>

To this:

<Link href="/about">
  <a className="link">
    <div className="link-icon">
      <SVG src="/images/icon-see-large.svg" width="50%" height="50%" />
    </div>
    <h2 className="link-title">About</h2>
  </a>
</Link>

Next.js Documentation: next/link

Router

Don't use redux to access router. Remove the connect wrapper and use the useRouter hook:

import {useRouter} from "next/router";
...
const router = useRouter();
const {pathname} = router;

Next.js Documentation: useRouter

Fix D3plus

The latest version of d3plus-react uses the latest context provider/hook logic supported by React, so you will need to manually add the Provider and global config into your project, most likely in _app.js, here is an excerpt:

import {D3plusContext} from "d3plus-react";
...
const globalConfig = {
  shapeConfig: {
    fill: "red"
  }
};
...
function App(props) {
  const {Component, pageProps} = props;
  ...
  return (
    <D3plusContext.Provider value={globalConfig}>
      <Component {...pageProps} />
    </D3plusContext.Provider>
  );
}
...

Upgrade Redux

Dependencies

If using redux, you will need to manually install some new dependencies:

npm i react-redux redux-thunk @reduxjs/toolkit

Store Setup

Create a new store/index.js directory and file that looks like this:

import thunk from "redux-thunk";
import {configureStore} from "@reduxjs/toolkit";
import {setupListeners} from "@reduxjs/toolkit/query";
import {reducers} from "./reducers/index";

const middleware = [thunk];

export const store = configureStore({
  reducer: reducers,
  // Adding the api middleware enables caching, invalidation, polling,
  // and other useful features of `rtk-query`.
  middleware: (getDefaultMiddleware) =>
    getDefaultMiddleware().concat(middleware)
});

// configure listeners using the provided defaults
setupListeners(store.dispatch);

Connecting to the Store

Move all of your actions into store/actions and all of your reducers into store/reducers. When needing to use the store, use the new useSelector hook instead of the old connect wrapper. As an example, to grab an item from the redux state:

import {useSelector} from "react-redux";
...
const {searchVisible} = useSelector(state => ({
  searchVisible: state.searchVisible
}));

And then, when you need to dispatch an action, use the new useDispatch hook:

import {useDispatch, useSelector} from "react-redux";
import {searchToggle} from "$root/store/actions/search.js";
...
const dispatch = useDispatch();

const {searchVisible} = useSelector(state => ({
  searchVisible: state.searchVisible
}));

const toggleSearch => dispatch(searchToggle());

Provider

As the final step, we need to manually wrap our app in the redux <Provider> component in pages/_app.js:

import React from "react";
import {Provider} from "react-redux";
import {store} from "../store/index";

import "./_app.css";

import Nav from "../components/Nav";
import Footer from "../components/Footer";

const App = ({Component, pageProps}) =>
  <Provider store={store}>
    <Nav />
    <Component {...pageProps} />
    <Footer />
  </Provider>
  ;

export default App;

Migrate Needs

Any components that use the fetchData function exported by @datawheel/canon-core must be upgraded to use the Next.js build-in getStaticProps function. Here is an example of a getStaticProps that injects a static file and the results of an async fetch function that contains an axios call:

import {promises as fs} from "fs";
import path from "path";

import fetchLatestYear from "$root/cache/latestYear";

/** */
export async function getStaticProps() {

  const topoPath = path.join(process.cwd(), "public/topojson/world-50m.json");
  const topoFile = await fs.readFile(topoPath, "utf8");

  const latestYear = await fetchLatestYear();

  return {
    props: {
      latestYear,
      topojson: JSON.parse(topoFile)
    },
    revalidate: 60 * 60 // results will regenerate in 1 hour
  };
}

Next.js Documentation: getStaticProps

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrating from canon to Next.js

Migration Steps

Upgrade Node

Docker

Local Machine

Fix Dependencies

Find Nested `canon-core` Dependencies

React Table

Setup Files

env vars

.gitignore

package.json

.eslintrc

helmet.js

Quick Changes

Move Components

Convert CSS

normalize

Components

style.yml

PostCSS

Update Routing

Change Link Components

Router

Fix D3plus

Upgrade Redux

Dependencies

Store Setup

Connecting to the Store

Provider

Migrate Needs

Clone this wiki locally

Migrating from canon to Next.js

Migration Steps

Upgrade Node

Docker

Local Machine

Fix Dependencies

Find Nested canon-core Dependencies

React Table

Setup Files

env vars

.gitignore

package.json

.eslintrc

helmet.js

Quick Changes

Move Components

Convert CSS

normalize

Components

style.yml

PostCSS

Update Routing

Change Link Components

Router

Fix D3plus

Upgrade Redux

Dependencies

Store Setup

Connecting to the Store

Provider

Migrate Needs

Clone this wiki locally

Find Nested `canon-core` Dependencies