Release 2.22.0 (#2983)

## [2.22.0] - 2023-01-01

Thanks to: @angeldeejay, @buxxi, @dariom, @dWoolridge,
@KristjanESPERANTO, @MagMar94, @naveensrinivasan, @retroflex, @SkySails
and @tom.

Special thanks to @khassel, @rejas and @sdetweil for taking over most
(if not all) of the work on this release as project collaborators. This
version would not be there without their effort. Thank you!

### Added

- Added test for remoteFile option in compliments module
- Added hourlyWeather functionality to Weather.gov weather provider
- Removed weatherEndpoint definition from weathergov.js (not used)
- Added css class names "today" and "tomorrow" for default calendar
- Added Collaboration.md
- Added new github action for dependency review (#2862)
- Added a WeatherProvider for Open-Meteo
- Added Yr as a weather provider
- Added config options "ignoreXOriginHeader" and
"ignoreContentSecurityPolicy"

### Removed

- Removed usage of internal fetch function of node until it is more
stable

### Updated

- Cleaned up test directory (#2937) and jest config (#2959)
- Wait for all modules to start before declaring the system ready
(#2487)
- Updated e2e tests (moved `done()` in helper functions) and use es6
syntax in all tests
- Updated da translation
- Rework weather module
- Make sure smhi provider api only gets a maximum of 6 digits
coordinates (#2955)
  - Use fetch instead of XMLHttpRequest in weatherprovider (#2935)
  - Reworked how weatherproviders handle units (#2849)
  - Use unix() method for parsing times, fix suntimes on the way (#2950)
  - Refactor conversion functions into utils class (#2958)
- The `cors`-method in `server.js` now supports sending and recieving
HTTP headers
- Replace `…` by `…`
- Cleanup compliments module
- Updated dependencies including electron to v22 (#2903)

### Fixed

- Correctly show apparent temperature in SMHI weather provider
- Ensure updatenotification module isn't shown when local is _ahead_ of
remote
- Handle node_helper errors during startup (#2944)
- Possibility to change FontAwesome class in calendar, so icons like
`fab fa-facebook-square` works.
- Fix cors problems with newsfeed articles (as far as possible), allow
disabling cors per feed with option `useCorsProxy: false` (#2840)
- Tests not waiting for the application to start and stop before
starting the next test
- Fix electron tests failing sometimes in github workflow
- Fixed gap in clock module when displayed on the left side with
displayType=digital
- Fixed playwright issue by upgrading to v1.29.1 (#2969)

Signed-off-by: naveen <[email protected]>
Co-authored-by: Karsten Hassel <[email protected]>
Co-authored-by: Malte Hallström <[email protected]>
Co-authored-by: Veeck <[email protected]>
Co-authored-by: veeck <[email protected]>
Co-authored-by: dWoolridge <[email protected]>
Co-authored-by: Johan <[email protected]>
Co-authored-by: Dario Mratovich <[email protected]>
Co-authored-by: Dario Mratovich <[email protected]>
Co-authored-by: Magnus <[email protected]>
Co-authored-by: Naveen <[email protected]>
Co-authored-by: buxxi <[email protected]>
Co-authored-by: Thomas Hirschberger <[email protected]>
Co-authored-by: Kristjan ESPERANTO <[email protected]>
Co-authored-by: Andrés Vanegas Jiménez <[email protected]>

.github/workflows/depsreview.yaml

@@ -0,0 +1,14 @@
+name: "Dependency Review"
+on: [pull_request]
+
+permissions:
+  contents: read
+
+jobs:
+  dependency-review:
+    runs-on: ubuntu-latest
+    steps:
+      - name: "Checkout Repository"
+        uses: actions/checkout@v3
+      - name: "Dependency Review"
+        uses: actions/dependency-review-action@v2

.gitignore

@@ -16,7 +16,6 @@ vendor/node_modules/**/*
 jspm_modules
 .npm
 .node_repl_history
-.nyc_output/
 
 # Visual Studio Code ignoramuses.
 .vscode/

.prettierignore

@@ -1,4 +1,3 @@
 /config
 /coverage
-.nyc_output
 package-lock.json

CHANGELOG.md

@@ -5,30 +5,82 @@ This project adheres to [Semantic Versioning](https://semver.org/).
 
 ❤️ **Donate:** Enjoying MagicMirror²? [Please consider a donation!](https://magicmirror.builders/donate) With your help we can continue to improve the MagicMirror².
 
+## [2.22.0] - 2023-01-01
+
+Thanks to: @angeldeejay, @buxxi, @dariom, @dWoolridge, @KristjanESPERANTO, @MagMar94, @naveensrinivasan, @retroflex, @SkySails and @Tom.
+
+Special thanks to @khassel, @rejas and @sdetweil for taking over most (if not all) of the work on this release as project collaborators. This version would not be there without their effort. Thank you!
+
+### Added
+
+- Added test for remoteFile option in compliments module
+- Added hourlyWeather functionality to Weather.gov weather provider
+- Removed weatherEndpoint definition from weathergov.js (not used)
+- Added css class names "today" and "tomorrow" for default calendar
+- Added Collaboration.md
+- Added new github action for dependency review (#2862)
+- Added a WeatherProvider for Open-Meteo
+- Added Yr as a weather provider
+- Added config options "ignoreXOriginHeader" and "ignoreContentSecurityPolicy"
+
+### Removed
+
+- Removed usage of internal fetch function of node until it is more stable
+
+### Updated
+
+- Cleaned up test directory (#2937) and jest config (#2959)
+- Wait for all modules to start before declaring the system ready (#2487)
+- Updated e2e tests (moved `done()` in helper functions) and use es6 syntax in all tests
+- Updated da translation
+- Rework weather module
+  - Make sure smhi provider api only gets a maximum of 6 digits coordinates (#2955)
+  - Use fetch instead of XMLHttpRequest in weatherprovider (#2935)
+  - Reworked how weatherproviders handle units (#2849)
+  - Use unix() method for parsing times, fix suntimes on the way (#2950)
+  - Refactor conversion functions into utils class (#2958)
+- The `cors`-method in `server.js` now supports sending and recieving HTTP headers
+- Replace `…` by `…`
+- Cleanup compliments module
+- Updated dependencies including electron to v22 (#2903)
+
+### Fixed
+
+- Correctly show apparent temperature in SMHI weather provider
+- Ensure updatenotification module isn't shown when local is _ahead_ of remote
+- Handle node_helper errors during startup (#2944)
+- Possibility to change FontAwesome class in calendar, so icons like `fab fa-facebook-square` works.
+- Fix cors problems with newsfeed articles (as far as possible), allow disabling cors per feed with option `useCorsProxy: false` (#2840)
+- Tests not waiting for the application to start and stop before starting the next test
+- Fix electron tests failing sometimes in github workflow
+- Fixed gap in clock module when displayed on the left side with displayType=digital
+- Fixed playwright issue by upgrading to v1.29.1 (#2969)
+
 ## [2.21.0] - 2022-10-01
 
 Special thanks to: @BKeyport, @buxxi, @davide125, @khassel, @kolbyjack, @krukle, @MikeBishop, @rejas, @sdetweil, @SkySails and @veeck
 
-## Added
+### Added
 
-- Possibility to fetch calendars through socket notifications.
+- Added possibility to fetch calendars through socket notifications.
 - New scripts `install-mm` (and `install-mm:dev`) for simplifying mm installation (now: `npm run install-mm`) and adding params `--no-audit --no-fund --no-update-notifier` for less noise.
 - New `showTimeToday` option in calendar module shows time for current-day events even if `timeFormat` is `"relative"`.
-- Add hourly forecasts, apparent temperature & custom location name to SMHI weather provider.
+- Added hourly forecasts, apparent temperature & custom location name to SMHI weather provider.
+- Added new electron tests for calendar and moved some compliments tests from `e2e` to `electron` because of date mocking, removed mock stuff from compliments module.
 
-## Removed
+### Removed
 
-- Old weather deprecated modules `currentweather` and `weatherforecast`.
+- Removed old and deprecated weather modules `currentweather` and `weatherforecast`.
+- Removed `DAYAFTERTOMORROW` from English.
 
-## Updated
+### Updated
 
-- Removed `DAYAFTERTOMORROW` from English.
-- Update dependencies.
+- Updated dependencies.
 - Updated jsdoc.
-- Updated font tree to use variables consistantly.
+- Updated font tree to use variables consistently.
 - Removed deprecated Docker Repository from issue template.
 
-## Fixed
+### Fixed
 
 - Broadcast all calendar events while still honoring global and per-calendar maximumEntries.
 - Respect rss ttl provided by newsfeed (#2883).

Collaboration.md

@@ -0,0 +1,12 @@
+This document describes how collaborators of this repository should work together.
+
+## Pull Requests
+
+- never merge your own PR's
+- never merge without someone having approved (approving and merging from same person is allowed)
+- wait for all approvals requested (or the author decides something different in the comments)
+
+## Issues
+
+- "real" Issues are closed if the problem is solved and the fix is released
+- unrelated Issues (e.g. related to a foreign module) are closed immediately with a comment to open an issue in the module repository or to discuss this further in the forum or discord

jest.config.js

@@ -0,0 +1,32 @@
+module.exports = async () => {
+	return {
+		verbose: true,
+		testTimeout: 20000,
+		testSequencer: "<rootDir>/tests/configs/test_sequencer.js",
+		projects: [
+			{
+				displayName: "unit",
+				moduleNameMapper: {
+					logger: "<rootDir>/js/logger.js"
+				},
+				testMatch: ["**/tests/unit/**/*.[jt]s?(x)"],
+				testPathIgnorePatterns: ["<rootDir>/tests/unit/mocks"]
+			},
+			{
+				displayName: "electron",
+				testMatch: ["**/tests/electron/**/*.[jt]s?(x)"],
+				testPathIgnorePatterns: ["<rootDir>/tests/electron/helpers/"]
+			},
+			{
+				displayName: "e2e",
+				setupFilesAfterEnv: ["<rootDir>/tests/e2e/helpers/mock-console.js"],
+				testMatch: ["**/tests/e2e/**/*.[jt]s?(x)"],
+				modulePaths: ["<rootDir>/js/"],
+				testPathIgnorePatterns: ["<rootDir>/tests/e2e/helpers/", "<rootDir>/tests/e2e/mocks"]
+			}
+		],
+		collectCoverageFrom: ["./clientonly/**/*.js", "./js/**/*.js", "./modules/**/*.js", "./serveronly/**/*.js"],
+		coverageReporters: ["lcov", "text"],
+		coverageProvider: "v8"
+	};
+};

js/app.js

@@ -222,18 +222,33 @@ function App() {
 				}
 			}
 
-			loadModules(modules, function () {
-				httpServer = new Server(config, function (app, io) {
-					Log.log("Server started ...");
-
-					for (let nodeHelper of nodeHelpers) {
-						nodeHelper.setExpressApp(app);
-						nodeHelper.setSocketIO(io);
-						nodeHelper.start();
+			loadModules(modules, async function () {
+				httpServer = new Server(config);
+				const { app, io } = await httpServer.open();
+				Log.log("Server started ...");
+
+				const nodePromises = [];
+				for (let nodeHelper of nodeHelpers) {
+					nodeHelper.setExpressApp(app);
+					nodeHelper.setSocketIO(io);
+
+					try {
+						nodePromises.push(nodeHelper.start());
+					} catch (error) {
+						Log.error(`Error when starting node_helper for module ${nodeHelper.name}:`);
+						Log.error(error);
 					}
+				}
 
-					Log.log("Sockets connected & modules started ...");
+				Promise.allSettled(nodePromises).then((results) => {
+					// Log errors that happened during async node_helper startup
+					results.forEach((result) => {
+						if (result.status === "rejected") {
+							Log.error(result.reason);
+						}
+					});
 
+					Log.log("Sockets connected & modules started ...");
 					if (typeof callback === "function") {
 						callback(config);
 					}
@@ -247,14 +262,16 @@ function App() {
 	 * exists.
 	 *
 	 * Added to fix #1056
+	 *
+	 * @param {Function} callback Function to be called after the app has stopped
 	 */
-	this.stop = function () {
+	this.stop = function (callback) {
 		for (const nodeHelper of nodeHelpers) {
 			if (typeof nodeHelper.stop === "function") {
 				nodeHelper.stop();
 			}
 		}
-		httpServer.close();
+		httpServer.close().then(callback);
 	};
 
 	/**

js/electron.js

@@ -103,6 +103,20 @@ function createWindow() {
 			}, 1000);
 		});
 	}
+
+	//remove response headers that prevent sites of being embedded into iframes if configured
+	mainWindow.webContents.session.webRequest.onHeadersReceived((details, callback) => {
+		let curHeaders = details.responseHeaders;
+		if (config["ignoreXOriginHeader"] || false) {
+			curHeaders = Object.fromEntries(Object.entries(curHeaders).filter((header) => !/x-frame-options/i.test(header[0])));
+		}
+
+		if (config["ignoreContentSecurityPolicy"] || false) {
+			curHeaders = Object.fromEntries(Object.entries(curHeaders).filter((header) => !/content-security-policy/i.test(header[0])));
+		}
+
+		callback({ responseHeaders: curHeaders });
+	});
 }
 
 // This method will be called when Electron has finished

js/fetch.js

@@ -1,20 +1,28 @@
 /**
- * fetch
+ * Helper class to provide either third party fetch library or (if node >= 18)
+ * return internal node fetch implementation.
  *
+ * Attention: After some discussion we always return the third party
+ * implementation until the node implementation is stable and more tested
+ *
+ * @see https://github.com/MichMich/MagicMirror/pull/2952
+ * @see https://github.com/MichMich/MagicMirror/issues/2649
  * @param {string} url to be fetched
  * @param {object} options object e.g. for headers
  * @class
  */
-async function fetch(url, options) {
-	const nodeVersion = process.version.match(/^v(\d+)\.*/)[1];
-	if (nodeVersion >= 18) {
-		// node version >= 18
-		return global.fetch(url, options);
-	} else {
-		// node version < 18
-		const nodefetch = require("node-fetch");
-		return nodefetch(url, options);
-	}
+async function fetch(url, options = {}) {
+	// const nodeVersion = process.version.match(/^v(\d+)\.*/)[1];
+	// if (nodeVersion >= 18) {
+	// 	// node version >= 18
+	// 	return global.fetch(url, options);
+	// } else {
+	// 	// node version < 18
+	// 	const nodefetch = require("node-fetch");
+	// 	return nodefetch(url, options);
+	// }
+	const nodefetch = require("node-fetch");
+	return nodefetch(url, options);
 }
 
 module.exports = fetch;

js/main.js

@@ -70,7 +70,7 @@ const MM = (function () {
 	 * Select the wrapper dom object for a specific position.
 	 *
 	 * @param {string} position The name of the position.
-	 * @returns {HTMLElement} the wrapper element
+	 * @returns {HTMLElement | void} the wrapper element
 	 */
 	const selectWrapper = function (position) {
 		const classes = position.replace("_", " ");