This updates all dependencies to their latest versions - except two
packages:
- eslint: Some plugins we use here aren't compatible yet to ESLint v10.
- node-ical: The new version has revealed an issue in our calendar
logic. I would prefer to address this in a separate PR.
After updating the dependencies, eslint-plugin-package-json rules
complained about missing fields in the package.json. I added them in the
second commit.
## Loading `config.js`
### Previously
Loaded on server-side in `app.js` and in the browser by including
`config.js` in `index.html`. The web server has an endpoint `/config`
providing the content of server loaded `config.js`.
### Now
Loaded only on server-side in `app.js`. The browser loads the content
using the web server endpoint `/config`. So the server has control what
to provide to the clients.
Loading the `config.js` was moved to `Utils.js` so that
`check_config.js` can use the same functions.
## Using environment variables in `config.js`
### Previously
Environment variables were not allowed in `config.js`. The workaround
was to create a `config.js.template` with curly braced bash variables
allowed. While starting the app the `config.js.template` was converted
via `envsub` into a `config.js`.
### Now
Curly braced bash variables are allowed in `config.js`. Because only the
server loads `config.js` he can substitute the variables while loading.
## Secrets in MagicMirror²
To be honest, this is a mess.
### Previously
All content defined in the `config` directory was reachable from the
browser. Everyone with access to the site could see all stuff defined in
the configuration e.g. using the url http://ip:8080/config. This
included api keys and other secrets.
So sharing a MagicMirror² url to others or running MagicMirror² without
authentication as public website was not possible.
### Now
With this PR we add (beta) functionality to protect sensitive data. This
is only possible for modules running with a `node_helper`. For modules
running in the browser only (e.g. default `weather` module), there is no
way to hide data (per construction). This does not mean, that every
module with `node_helper` is safe, e.g. the default `calendar` module is
not safe because it uses the calendar url's as sort of id and sends them
to the client.
For adding more security you have to set `hideConfigSecrets: true` in
`config.js`. With this:
- `config/config.env` is not deliverd to the browser
- the contents of environment variables beginning with `SECRET_` are not
published to the clients
This is a first step to protect sensitive data and you can at least
protect some secrets.
- remove param `--enable-features=UseOzonePlatform` in start electron
tests (as we did already in `package.json`)
- update node versions in github workflows, remove `22.21.1`, add `25.x`
- fix formatting in tests
- update dependencies including electron to v40
This is still a draft PR because most calendar electron tests are not
running which is caused by the electron update from `v39.3.0` to
`v40.0.0`. Maybe @KristjanESPERANTO has an idea ...
---------
Co-authored-by: Kristjan ESPERANTO <35647502+KristjanESPERANTO@users.noreply.github.com>
## Summary
PR [#3976](https://github.com/MagicMirrorOrg/MagicMirror/pull/3976)
introduced smart HTTP error handling for the Calendar module. This PR
extracts that HTTP logic into a central `HTTPFetcher` class.
Calendar is the first module to use it. Follow-up PRs would migrate
Newsfeed and maybe even Weather.
**Before this change:**
- ❌ Each module had to implemented its own `fetch()` calls
- ❌ No centralized retry logic or backoff strategies
- ❌ No timeout handling for hanging requests
- ❌ Error detection relied on fragile string parsing
**What this PR adds:**
- ✅ Unified HTTPFetcher class with intelligent retry strategies
- ✅ Modern AbortController with configurable timeout (default 30s)
- ✅ Proper undici Agent for self-signed certificates
- ✅ Structured error objects with translation keys
- ✅ Calendar module migrated as first consumer
- ✅ Comprehensive unit tests with msw (Mock Service Worker)
## Architecture
**Before - Decentralized HTTP handling:**
```
Calendar Module Newsfeed Module Weather Module
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ fetch() own │ │ fetch() own │ │ fetch() own │
│ retry logic │ │ basic error │ │ no retry │
│ error parse │ │ handling │ │ client-side │
└─────────────┘ └─────────────┘ └─────────────┘
│ │ │
└───────────────────────┴───────────────────────┘
▼
External APIs
```
**After - Centralized with HTTPFetcher:**
```
┌─────────────────────────────────────────────────────┐
│ HTTPFetcher │
│ • Unified retry strategies (401/403, 429, 5xx) │
│ • AbortController timeout (30s) │
│ • Structured errors with translation keys │
│ • undici Agent for self-signed certs │
└────────────┬──────────────┬──────────────┬──────────┘
│ │ │
┌───────▼───────┐ ┌────▼─────┐ ┌──────▼──────┐
│ Calendar │ │ Newsfeed │ │ Weather │
│ ✅ This PR │ │ future │ │ future │
└───────────────┘ └──────────┘ └─────────────┘
│ │ │
└──────────────┴──────────────┘
▼
External APIs
```
## Complexity Considerations
**Does HTTPFetcher add complexity?**
Even if it may look more complex, it actually **reduces overall
complexity**:
- **Calendar already has this logic** (PR #3976) - we're extracting, not
adding
- **Alternative is worse:** Each module implementing own logic = 3× the
code
- **Better testability:** 443 lines of tests once vs. duplicating tests
for each module
- **Standards-based:** Retry-After is RFC 7231, not custom logic
## Future Benefits
**Weather migration (future PR):**
Moving Weather from client-side to server-side will enable:
- **Same robust error handling** - Weather gets 429 rate-limiting, 5xx
backoff for free
- **Simpler architecture** - No proxy layer needed
Moving the weather modules from client-side to server-side will be a big
undertaking, but I think it's a good strategy. Even if we only move the
calendar and newsfeed to the new HTTP fetcher and leave the weather as
it is, this PR still makes sense, I think.
## Breaking Changes
**None**
----
I am eager to hear your opinion on this 🙂
This PR makes three small changes to the ESLint setup:
1. Migrate from
[eslint-plugin-vitest](https://www.npmjs.com/package/eslint-plugin-vitest)
to it's successor
[@vitest/eslint-plugin](https://www.npmjs.com/package/@vitest/eslint-plugin).
2. Change the eslint config so that only the test files are checked with
the vitest rules. Previously, it was just unnecessary and inefficient to
check all js files with them.
3. We had defined some of the test rules as warnings, but that is not
really ideal. I changed them to errors.
Adapts calendar module to node-ical changes and fixes a bug with moved
full-day recurring events in eastern timezones.
## Changes
### 1. Update node-ical to 0.23.1
- Includes upstream fixes for UNTIL UTC validation errors from CalDAV
servers (reported by @rejas in PR #4010)
- Changes to `getDateKey()` behavior for VALUE=DATE events (now uses
local date components)
- Fixes issue with malformed DURATION values (reported by MagicMirror
user here: https://github.com/jens-maus/node-ical/issues/381)
### 2. Remove dead code
- Removed ineffective UNTIL modification code (rule.options is read-only
in rrule-temporal)
- The code attempted to extend UNTIL for all-day events but had no
effect
### 3. Fix recurrence lookup for full-day events
node-ical changed the behavior of `getDateKey()` - it now uses local
date components for VALUE=DATE events instead of UTC. This broke
recurrence override lookups for full-day events in eastern timezones.
**Why it broke:**
- **before node-ical update:** Both node-ical and MagicMirror used UTC →
keys matched ✅
- **after node-ical update:** node-ical uses local date (RFC 5545
conform), MagicMirror still used UTC → **mismatch** ❌
**Example:**
- Full-day recurring event on October 12 in Europe/Berlin (UTC+2)
- node-ical 0.23.1 stores override with key: `"2024-10-12"` (local date)
- MagicMirror looked for key: `"2024-10-11"` (from UTC: Oct 11 22:00)
- **Result:** Moved event not found, appears on wrong date
**Solution:** Adapt to node-ical's new behavior by using local date
components for full-day events, UTC for timed events.
**Note:** This is different from previous timezone fixes - those
addressed event generation, this fixes the lookup of recurrence
overrides.
## Background
node-ical 0.23.0 switched from `rrule` to `rrule-temporal`, introducing
breaking changes. Version 0.23.1 fixed the UNTIL validation issue and
formalized the `getDateKey()` behavior for DATE vs DATE-TIME values,
following RFC 5545 specification that DATE values represent local
calendar dates without timezone context.
Updating `node-ical` and adapt logic to new behaviour.
## Problem
node-ical 0.23.0 switched from `rrule.js` to `rrule-temporal`, changing
how recurring event dates are returned. Our code assumed the old
behavior where dates needed manual timezone conversion.
## Solution
Updated `getMomentsFromRecurringEvent()` in `calendarfetcherutils.js`:
- Removed `tzid = null` clearing (no longer needed)
- Simplified timed events: `moment.tz(date, eventTimezone)` instead of
`moment.tz(date, "UTC").tz(eventTimezone, true)`
- Kept UTC component extraction for full-day events to prevent date
shifts
### 1. Replace `XMLHttpRequest` with the modern `fetch` API for loading
translation files
#### Changes
- **translator.js**: Use `fetch` with `async/await` instead of XHR
callbacks
- **loader.js**: Align URL handling and add error handling (follow-up to
fetch migration)
- **Tests**: Update infrastructure for `fetch` compatibility
#### Benefits
- Modern standard API
- Cleaner, more readable code
- Better error handling and fallback mechanisms
### 2. Migrate e2e tests to Playwright
This wasn't originally planned for this PR, but is related. While
investigating suspicious log entries which surfaced after the fetch
migration I kept running into JSDOM’s limitations. That pushed me to
migrate the E2E suite to Playwright instead.
#### Changes
- switch e2e harness to Playwright (`tests/e2e/helpers/global-setup.js`)
- rewrite specs to use Playwright locators + shared `expectTextContent`
- install Chromium via `npx playwright install --with-deps` in CI
#### Benefits
- much closer to real browser behaviour
- and no more fighting JSDOM’s quirks
`node-libgpiod` uses deprecated NAN which is incompatible with Electron
v39+. `serialport` uses N-API ensuring compatibility with current and
future Electron versions.
`node-libgpiod` is only used by 1 of ~1300 3rd-party-modules
(MMM-PresenceScreenControl), while serialport is used by at least 4
modules (MMM-Serial-Notification, MMM-RadarPresence, MMM-LKY-TIC and
MMM-Gestures), making it a better test candidate.
Also updates Electron to v39.
Fixes#3933
This is a big change, but I think it's a good move, as `vitest` is much
more modern than `jest`.
I'm excited about the UI watch feature (run `npm run test:ui`), for
example - it's really helpful and saves time when debugging tests. I had
to adjust a few tests because they had time related issues, but
basically we are now testing the same things - even a bit better and
less flaky (I hope).
What do you think?
seems we dont need the parse5 pin as long as jsdom is fixed to v27.0.0.
not sure if there is anything else we can do to the deps?
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: veeck <gitkraken@veeck.de>
Normally, I wouldn't update the dependencies again so soon, but
`node-ical` underwent some major changes (see
https://github.com/jens-maus/node-ical/pull/404) with the last release,
and I'd like to use it here as early as possible to see if there are any
problems with it.
This fixes security issue
[CVE-2023-42282](https://github.com/advisories/GHSA-78xj-cgh5-2h22),
which is not very likely to be exploitable in MagicMirror² setups, but
still should be fixed.
The [express-ipfilter](https://www.npmjs.com/package/express-ipfilter)
package depends on the obviously unmaintained
[ip](https://github.com/indutny/node-ip) package, which has known
security vulnerabilities. Since no fix is available, this commit
replaces both dependencies with a custom middleware using the better
maintained [ipaddr.js](https://www.npmjs.com/package/ipaddr.js) library.
Changes:
- Add new `js/ip_access_control.js` with lightweight middleware
- Remove `express-ipfilter` dependency, add `ipaddr.js`
- Update `js/server.js` to use new middleware
- In addition, I have formulated the descriptions of the corresponding
tests a little more clearly.
- removes the external unmaintained `module-alias` dependency ->
reducing complexity and risk
- introduces a small internal alias mechanism for `logger` and
`node_helper`
- preserves backward compatibility for existing 3rd‑party modules
- should simplify a future ESM migration of MagicMirror
I'm confident that it shouldn't cause any problems, but we could also
consider including it in the release after next. What do you think?
This PR is inspired by PR #2934 - so thanks to @thesebas! 🙇😃
electron uses node v22.18 in its [current
releases](https://releases.electronjs.org/), so we should go hand in
hand and use that as the minimal node version
nothing fancy in these though
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: veeck <gitkraken@veeck.de>
- Remove `sinon` dependency in favor of Jest native mocking
- Unify test helper functions across translation test suites
- Rename `setupDOMEnvironment` to `createTranslationTestEnvironment` for
consistency
- Simplify DOM setup by removing unnecessary Promise/async patterns
- Avoid potential port conflicts by using port 3001 for translator unit
tests
- Improve test reliability and maintainability
Just some normal maintainance after the holidays
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: veeck <gitkraken@veeck.de>
e2e:
- needed window.close(), otherwise the objects are not destroyed
- add missing `await` in clock test
- set maxListeners for all tests
remaining todo (comes with another PR if I find the problem):
- calendar e2e is now the only test which still needs `--forceExit`
- animateCSS_spec test did throw errors at least with newest
dependencies (running locally or on gitlab)
- dependency updates: New jest v30 breaks our tests so we have to stay
with v29 until fixed (will take a look)
I was always unhappy when maintaining dependency updates to have 3
`package.json` files.
This PR moves all deps into the main `package.json` and removes the
folders `fonts` and `vendor`.
If accepted I will update the docs too.
---------
Co-authored-by: Kristjan ESPERANTO <35647502+KristjanESPERANTO@users.noreply.github.com>
