The bottom line of this PR is, that it fixes an issue and simplifies the
code by dealing with the TODOs in the code.
For review, I suggest looking at each commit individually. If there are
too many changes for a PR, let me know and I'll split it up
🙂
## 1. [fix(calendar): prevent excessive fetching with smart refresh
strategy](8892cd3d5a)
- Add lastFetch timestamp tracking to CalendarFetcher
- Add shouldRefetch() method with configurable minimum interval
(default: 3 minutes)
- When reusing existing fetcher: fetch if data is stale (>3 min),
otherwise broadcast cached events
- Prevents double broadcasts to consuming modules while maintaining
fresh data
- Balances rate limit prevention (Issue
https://github.com/MagicMirrorOrg/MagicMirror/issues/3971) with data
freshness on user reload
- Prevents excessive fetching during rapid reloads (e.g., Fully Kiosk
screensaver use case)
- Allows fresh calendar data when enough time has passed since last
fetch
## 2. [refactor(calendar): simplify event exclusion
logic](d507aba82d)
- Extract filtering logic from `shouldEventBeExcluded` into new helper
`checkEventAgainstFilter`
- Simplify the main loop in `shouldEventBeExcluded
It resolves a TODO from the comments in the code:
* `This seems like an overly complicated way to exclude events based on
the title.`
## 3. [refactor(calendar): extract recurring event expansion
logic](d510160bd2)
This change separates the expansion of recurring events from the main
filtering loop into a new helper function 'expandRecurringEvent'.
It resolves two TODOs from the comments in the code:
- `This should be a separate function`
- `This should create an event per moment so we can change anything we
want`
This improves code readability, reduces complexity in 'filterEvents',
and allows for cleaner handling of individual recurrence instances.
## 4. [refactor(calendar): simplify recurring event
handling](b04f716cc0)
- Simplify 'getMomentsFromRecurringEvent' using modern syntax
- Improve handling of full-day events across different timezones
## 5. [test(calendar): fix UNTIL date in fullday_until.ics
fixture](1d762b2ade)
The issue was with the UNTIL date being May 4th while DTSTART was May
5th. This created an invalid recurrence rule where the end date came
before the start date.
The fix only adjusts the UNTIL date from May 4th to May 5th, so it
matches the start date.
1. Convert CalendarFetcher from legacy constructor function pattern to
ES6 class (which simplifies future migration from CommonJS to ES
modules).
2. Implement targeted HTTP error handling with smart retry strategies
for common calendar feed issues:
- 401/403: Extended retry delay (5× interval, min 30 min)
- 429: Retry-After header parsing with 15 min fallback
- 5xx: Exponential backoff (2^count, max 3 retries)
- 4xx: Extended retry (2× interval, min 15 min)
- Add serverErrorCount tracking for exponential backoff
- Error messages now include specific HTTP status codes and calculated
retry delays for better debugging and user feedback
Previously, CalendarFetcher did not respond appropriately to HTTP
errors, continuing to hammer endpoints without backoff, potentially
overloading servers and triggering rate limits. This refactoring
implements respectful retry strategies that adapt to server responses
and reduce unnecessary load.
Maybe we could later centralize the HTTP error handling and use it for
weather and newsfeed as well.
The PR was inspired by having worked on the calendar fetcher for
MMM-CalendarExt2, where there was already better error handling.
When I saw PR #3951, I wondered why `cspell` didn't catch these typos
before. Unfortunately, the default modules were excluded from the check.
I have corrected this with these changes.
This even revealed a code error in
`modules/default/weather/providers/overrideWrapper.js`:
- before: `fetchEatherHourly`
- after: `fetchWeatherHourly`
This is just to reduce a little noise in the dev console. The following
line will not appear with this change:
```shell
module.js:480 Check MagicMirror² version for module 'calendar' - Minimum version: 2.1.0 - Current version: 2.34.0-
```
Since version 2.1.0 is so old, we can surely throw it out without
concern.
- removes the external unmaintained `module-alias` dependency ->
reducing complexity and risk
- introduces a small internal alias mechanism for `logger` and
`node_helper`
- preserves backward compatibility for existing 3rd‑party modules
- should simplify a future ESM migration of MagicMirror
I'm confident that it shouldn't cause any problems, but we could also
consider including it in the release after next. What do you think?
This PR is inspired by PR #2934 - so thanks to @thesebas! 🙇😃
Fixes#3253
Adds a configuration option to overwrite the default `User-Agent` header
that is send at least by the calendar and news module. Allows other
modules to use the individual user agent as well.
The configuration accepts either a string or a function:
```
var config =
{
...
userAgent: 'Mozilla/5.0 (My User Agent)',
...
}
```
or
```
var config =
{
...
userAgent: () => 'Mozilla/5.0 (My User Agent)',
...
}
```
---------
Co-authored-by: Veeck <github@veeck.de>
Co-authored-by: veeck <gitkraken@veeck.de>
Co-authored-by: Karsten Hassel <hassel@gmx.de>
Co-authored-by: Kristjan ESPERANTO <35647502+KristjanESPERANTO@users.noreply.github.com>
to avoid exception in eslint config.
Background: If someone has a copy of the `modules` folder in his setup
eslint fails because the file `debug.js` uses `console` statements. I
need the `modules` folder renamed in my docker setup so this test always
fails. I think this is a simple solution which has no impact.
- fix newsfeed and calendar e2e tests so they don't need `--forceExit`
anymore
- `--forceExit` should stay in test runs to avoid running until test
limit in case of errors
- remove mocking console, not needed anymore
- configFactory-stuff should not run in browser (otherwise we get errors
`[ReferenceError: exports is not defined]`)
Refactored calendarfetcherutils to remove as many of the date
conversions as possible and use moment tz when calculating recurring
events, this will make debugging a lot easier and fixes problems from
the past with offsets and DST not being handled properly. Also added
some tests to test the behavior of the refactored methodes to make sure
the correct event dates are returned.
Refactored calendar.js aswell to make sure the unix UTC start and end
date of events are properly converted to a local timezone and displayed
correctly for the user.
This PR relates to:
https://github.com/MagicMirrorOrg/MagicMirror/issues/3797
---------
Co-authored-by: Koen Konst <c.h.konst@avisi.nl>
Co-authored-by: Kristjan ESPERANTO <35647502+KristjanESPERANTO@users.noreply.github.com>
fixes#3267 AGAIN, correctly, add testcase
add testcase for #3679 , broadcast clipping incorrectly
I added a test module (tests/testNotification) to catch the notification
and check the count. (one way to configure)
i put this module in the tests folder, and added /tests to the server
paths.
(can't have a module in a nested folder, like tests/modules/xxx)
but I have a problem. i can run the test config (MM_CONFIG_FILE), and
the two modules work correctly,
but in the spec runner, the calendar module times out on the broadcast
test.. there is only local ics file access, no outside hosts
I forced my system date to 1/1/24 (same as runner) and again the manual
testcase works fine
I added two test config.js,(configs/calendar) one works great (symbols)
, one fails (broadcast test)
I added additional delay in the calendarspec runner to try to debug the
module, but it still not long enough.. no messages of trouble when I get
into the browser.. BUT, this may be because of the log being turned
off... (just thought of this)
I created a special ICS (in mocks) that has 12 events, 1 for each
month.. (so I can check clipping and broadcast) the US holidays one is
sensitive to the current date, and I couldn't get it to work on
1/1/2024..
also, in general, is there a mechanism to run test:just_one_runner?
waiting thru the electron test to get to one testcase.. ugh..
here is an updated test version of the fixes for all kinds of calendar
date problems.
NOTE: the changed branch name
NOTE: this used the node-cal@0.19.0 library UNCHANGED
best to make a new folder and git clone there
git clone https://github.com/sdetweil/MagicMirror
cd MagicMirror
git checkout fixcaldates2 // <------ note this is a changed branch name
npm run install-mm
copy your config.js and custom.css from the prior folder
and the non-default modules you have installed…
this ONLY changes the default calendar
but DOES ship an updated node-ical library too
if you need to fall back, just rename the folders around again so that
your original is called MagicMirror
all the testcases for node-ical and MagicMirror execute successfully.
the ‘BIG’ change here is to get the local NON-TZ dates for the
rrule.between()
all the checking and conversion code is commented out or not used
the node-ical fixes are for excluded dates (exdate) values being
adjusted for DST/STD time… waiting to submit that PR
one fix in calendar.js for checking if a past date was too far back,
but it never checked to see IF the event date was in the past… (before
today) so it chopped off too many
and one change in calendarfetcher.js to put out a better diagnostic
message of the parsed data… (exdate was excluded cause JSON stringify
couldn’t convert the complex structure)
I added the tests you all have documented
please re-pull and checkout the new branch (I deleted the old branch)
and npm run install-mm again
---------
Co-authored-by: Veeck <github@veeck.de>
sliceMultiDayEvents occasionally gets the number of events wrong and
produces too many rows
Math.ceil() rounds up over 1.04 so we get an abnormal count
then the calcs for the midnight loop control used different moment()
functions, producing different results
fixes#3542
Hey!
This PR should change the behaviour of starting fullDay events that last
several days. The goal was to change the behavior of the "Starting
today, ends T" (T=Tomorrow) event, so it should show how many days it
will occur from the first day on
Before situation:
a fullDay event that started 'today' and ends several days later showed
Today on the first day. The rest of the days it showed X days left.
Y => Yesterday
T => Tomorrow
Target situation with this commit:
a fullDay event that started 'today' shows 'X days left' from the first
day on and 'Today' on the last day.
---------
Co-authored-by: Veeck <github@veeck.de>
Hi, I had the case of some users who set a very small fetchinterval (10
sec for example)
in some cases, it may be that the request did not have time to complete
correctly and that the next one has already been sent (generally on
start of MM²)
I think that lock min fetchInterval to 60000 is a good idea
* Worked around several issues in the RRULE library that were causing
deleted calender events to still show, some initial and recurring events
to not show, and some event times to be off an hour. (#3291)
* Renamed variables in *calendarfetcherutils.js* to be more clear about
use of `moment` and js's `Date` class.
* Added calendar config option `forceUseCurrentTime` (default:`false`)
which will ignore overridden `Date.now` in the config in order to keep
some tests consistent.
* Added several unit tests for crossing DST in different timezones with
excluded events.
see #3358
used command: `find ./ -type f -exec perl -i -0pe
's/\/\*\s*magicmirror.*?\*\/\s*//si' {} \;`
This is a first draft, I think we should preserve some of the comments.
It is basically a cosmetic thing, but has the following advantages:
1. Consistency with the official node documentation. The prefix is used
there.
2. It is easier to recognize the build-in modules.
## [2.25.0] - 2023-10-01
Thanks to: @bugsounet, @dgoth, @dependabot, @kenzal, @Knapoc,
@KristjanESPERANTO, @martingron, @NolanKingdon, @Paranoid93,
@TeddyStarinvest and @Ybbet.
Special thanks to @khassel, @rejas and @sdetweil for taking over most
(if not all) of the work on this release as project collaborators. This
version would not be there without their effort. Thank you guys! You are
awesome!
> ⚠️ This release needs nodejs version >= `v18`, older releases have
reached end of life and will not work!
### Added
- Added UV Index support to OpenWeatherMap
- Added 'hideDuplicates' flag to the calendar module
- Added `allowOverrideNotification` to weather module to enable sending
current weather objects with the `CURRENT_WEATHER_OVERRIDE` notification
to supplement/replace the current weather displayed
- Added optional AnimateCSS animate for `hide()`, `show()`,
`updateDom()`
- Added AnimateIn and animateOut in module config definition
- Apply AnimateIn rules on the first start
- Added automatic client page reload when server was restarted by
setting `reloadAfterServerRestart: true` in `config.js`, per default
`false` (#3105)
- Added eventClass option for customEvents on the default calendar
- Added AnimateCSS integration in tests suite (#3206)
- Added npm dependabot [Reserved to developer] (#3210)
- Added improved logging for calendar (#3110)
### Removed
- **Breaking Change**: Removed `digest` authentication method from
calendar module (which was already broken since release `2.15.0`)
### Updated
- Update roboto fonts to version v5
- Update issue template
- Update dev/dependencies incl. electron to v26
- Replace pretty-quick by lint-staged
(<https://github.com/azz/pretty-quick/issues/164>)
- Update engine node >=18. v16 reached it's end of life. (#3170)
- Update typescript definition for modules
- Cleaned up nunjuck templates
- Replace `node-fetch` with internal fetch (#2649) and remove
`digest-fetch`
- Update the French translation according to the English file.
- Update dependabot incl. vendor/fonts (monthly check)
- Renew `package-lock.json` for release
### Fixed
- Fix engine check on npm install (#3135)
- Fix undefined formatTime method in clock module (#3143)
- Fix clientonly startup fails after async added (#3151)
- Fix electron width/heigth when using xrandr under bullseye
- Fix time issue with certain recurring events in calendar module
- Fix ipWhiteList test (#3179)
- Fix newsfeed: Convert HTML entities, codes and tag in description
(#3191)
- Respect width/height (no fullscreen) if set in electronOptions
(together with `fullscreen: false`) in `config.js` (#3174)
- Fix: AnimateCSS merge hide() and show() animated css class when we do
multiple call
- Fix `Uncaught SyntaxError: Identifier 'getCorsUrl' has already been
declared (at utils.js:1:1)` when using `clock` and `weather` module
(#3204)
- Fix overriding `config.js` when running tests (#3201)
- Fix issue in weathergov provider with probability of precipitation not
showing up on hourly or daily forecast
---------
Signed-off-by: naveen <172697+naveensrinivasan@users.noreply.github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Karsten Hassel <hassel@gmx.de>
Co-authored-by: Malte Hallström <46646495+SkySails@users.noreply.github.com>
Co-authored-by: Veeck <github@veeck.de>
Co-authored-by: veeck <michael@veeck.de>
Co-authored-by: dWoolridge <dwoolridge@charter.net>
Co-authored-by: Johan <jojjepersson@yahoo.se>
Co-authored-by: Dario Mratovich <dario_mratovich@hotmail.com>
Co-authored-by: Dario Mratovich <dario.mratovich@outlook.com>
Co-authored-by: Magnus <34011212+MagMar94@users.noreply.github.com>
Co-authored-by: Naveen <172697+naveensrinivasan@users.noreply.github.com>
Co-authored-by: buxxi <buxxi@omfilm.net>
Co-authored-by: Thomas Hirschberger <47733292+Tom-Hirschberger@users.noreply.github.com>
Co-authored-by: Kristjan ESPERANTO <35647502+KristjanESPERANTO@users.noreply.github.com>
Co-authored-by: Andrés Vanegas Jiménez <142350+angeldeejay@users.noreply.github.com>
Co-authored-by: Dave Child <dave@addedbytes.com>
Co-authored-by: grenagit <46225780+grenagit@users.noreply.github.com>
Co-authored-by: Grena <grena@grenabox.fr>
Co-authored-by: Magnus Marthinsen <magmar@online.no>
Co-authored-by: Patrick <psieg@users.noreply.github.com>
Co-authored-by: Piotr Rajnisz <56397164+rajniszp@users.noreply.github.com>
Co-authored-by: Suthep Yonphimai <tomzt@users.noreply.github.com>
Co-authored-by: CarJem Generations (Carter Wallace) <cwallacecs@gmail.com>
Co-authored-by: Nicholas Fogal <nfogal.misc@gmail.com>
Co-authored-by: JakeBinney <126349119+JakeBinney@users.noreply.github.com>
Co-authored-by: OWL4C <124401812+OWL4C@users.noreply.github.com>
Co-authored-by: Oscar Björkman <17575446+oscarb@users.noreply.github.com>
Co-authored-by: Ismar Slomic <ismar@slomic.no>
Co-authored-by: Jørgen Veum-Wahlberg <jorgen.wahlberg@amedia.no>
Co-authored-by: Eddie Hung <6740044+eddiehung@users.noreply.github.com>
Co-authored-by: Bugsounet - Cédric <github@bugsounet.fr>
Co-authored-by: bugsounet <bugsounet@bugsounet.fr>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Knapoc <Knapoc@users.noreply.github.com>
Co-authored-by: sam detweiler <sdetweil@gmail.com>
Co-authored-by: veeck <michael.veeck@nebenan.de>
Co-authored-by: Paranoid93 <6515818+Paranoid93@users.noreply.github.com>
Co-authored-by: NolanKingdon <27908974+NolanKingdon@users.noreply.github.com>
Co-authored-by: J. Kenzal Hunter <kenzal.hunter@gmail.com>
Co-authored-by: Teddy <teddy.payet@gmail.com>
Co-authored-by: TeddyStarinvest <teddy.payet@starinvest.com>
Co-authored-by: martingron <61826403+martingron@users.noreply.github.com>
Co-authored-by: dgoth <132394363+dgoth@users.noreply.github.com>
## [2.23.0] - 2023-04-04
Thanks to: @angeldeejay, @buxxi, @CarJem, @dariom, @DaveChild, @dWoolridge, @grenagit, @Hirschberger, @KristjanESPERANTO, @MagMar94, @naveensrinivasan, @nfogal, @psieg, @rajniszp, @retroflex, @SkySails and @tomzt.
Special thanks to @khassel, @rejas and @sdetweil for taking over most (if not all) of the work on this release as project collaborators. This version would not be there without their effort. Thank you guys! You are awesome!
### Added
- Added increments for hourly forecasts in weather module (#2996)
- Added tests for hourly weather forecast
- Added possibility to ignore MagicMirror repo in updatenotification module
- Added Pirate Weather as new weather provider (#3005)
- Added possibility to use your own templates in Alert module
- Added error message if `<modulename>.js` file is missing in module folder to get a hint in the logs (#2403)
- Added possibility to use environment variables in `config.js` (#1756)
- Added option `pastDaysCount` to default calendar module to control of how many days past events should be displayed
- Added thai language to alert module
- Added option `sendNotifications` in clock module (#3056)
### Removed
- Removed darksky weather provider
- Removed unneeded (and unwanted) '.' after the year in calendar repeatingCountTitle (#2896)
### Updated
- Use develop as target branch for dependabot
- Update issue template, contributing doc and sample config
- The weather modules clearly separates precipitation amount and probability (risk of rain/snow)
- This requires all providers that only supports probability to change the config from `showPrecipitationAmount` to `showPrecipitationProbability`.
- Update tests for weather and calendar module
- Changed updatenotification module for MagicMirror repo only: Send only notifications for `master` if there is a tag on a newer commit
- Update dates in Calendar widgets every minute
- Cleanup jest coverage for patches
- Update `stylelint` dependencies, switch to `stylelint-config-standard` and handle `stylelint` issues, update `main.css` matching new rules
- Update Eslint config, add new rule and handle issue
- Convert lots of callbacks to async/await
- Revise require imports (#3071 and #3072)
### Fixed
- Fix wrong day labels in envcanada forecast (#2987)
- Fix for missing default class name prefix for customEvents in calendar
- Fix electron flashing white screen on startup (#1919)
- Fix weathergov provider hourly forecast (#3008)
- Fix message display with HTML code into alert module (#2828)
- Fix typo in french translation
- Yr wind direction is no longer inverted
- Fix async node_helper stopping electron start (#2487)
- The wind direction arrow now points in the direction the wind is flowing, not into the wind (#3019)
- Fix precipitation css styles and rounding value
- Fix wrong vertical alignment of calendar title column when wrapEvents is true (#3053)
- Fix empty news feed stopping the reload forever
- Fix e2e tests (failed after async changes) by running calendar and newsfeed tests last
- Lint: Use template literals instead of string concatenation
- Fix default alert module to render HTML for title and message
- Fix Open-Meteo wind speed units
## [2.22.0] - 2023-01-01
Thanks to: @angeldeejay, @buxxi, @dariom, @dWoolridge,
@KristjanESPERANTO, @MagMar94, @naveensrinivasan, @retroflex, @SkySails
and @Tom.
Special thanks to @khassel, @rejas and @sdetweil for taking over most
(if not all) of the work on this release as project collaborators. This
version would not be there without their effort. Thank you!
### Added
- Added test for remoteFile option in compliments module
- Added hourlyWeather functionality to Weather.gov weather provider
- Removed weatherEndpoint definition from weathergov.js (not used)
- Added css class names "today" and "tomorrow" for default calendar
- Added Collaboration.md
- Added new github action for dependency review (#2862)
- Added a WeatherProvider for Open-Meteo
- Added Yr as a weather provider
- Added config options "ignoreXOriginHeader" and
"ignoreContentSecurityPolicy"
### Removed
- Removed usage of internal fetch function of node until it is more
stable
### Updated
- Cleaned up test directory (#2937) and jest config (#2959)
- Wait for all modules to start before declaring the system ready
(#2487)
- Updated e2e tests (moved `done()` in helper functions) and use es6
syntax in all tests
- Updated da translation
- Rework weather module
- Make sure smhi provider api only gets a maximum of 6 digits
coordinates (#2955)
- Use fetch instead of XMLHttpRequest in weatherprovider (#2935)
- Reworked how weatherproviders handle units (#2849)
- Use unix() method for parsing times, fix suntimes on the way (#2950)
- Refactor conversion functions into utils class (#2958)
- The `cors`-method in `server.js` now supports sending and recieving
HTTP headers
- Replace `…` by `…`
- Cleanup compliments module
- Updated dependencies including electron to v22 (#2903)
### Fixed
- Correctly show apparent temperature in SMHI weather provider
- Ensure updatenotification module isn't shown when local is _ahead_ of
remote
- Handle node_helper errors during startup (#2944)
- Possibility to change FontAwesome class in calendar, so icons like
`fab fa-facebook-square` works.
- Fix cors problems with newsfeed articles (as far as possible), allow
disabling cors per feed with option `useCorsProxy: false` (#2840)
- Tests not waiting for the application to start and stop before
starting the next test
- Fix electron tests failing sometimes in github workflow
- Fixed gap in clock module when displayed on the left side with
displayType=digital
- Fixed playwright issue by upgrading to v1.29.1 (#2969)
Signed-off-by: naveen <172697+naveensrinivasan@users.noreply.github.com>
Co-authored-by: Karsten Hassel <hassel@gmx.de>
Co-authored-by: Malte Hallström <46646495+SkySails@users.noreply.github.com>
Co-authored-by: Veeck <github@veeck.de>
Co-authored-by: veeck <michael@veeck.de>
Co-authored-by: dWoolridge <dwoolridge@charter.net>
Co-authored-by: Johan <jojjepersson@yahoo.se>
Co-authored-by: Dario Mratovich <dario_mratovich@hotmail.com>
Co-authored-by: Dario Mratovich <dario.mratovich@outlook.com>
Co-authored-by: Magnus <34011212+MagMar94@users.noreply.github.com>
Co-authored-by: Naveen <172697+naveensrinivasan@users.noreply.github.com>
Co-authored-by: buxxi <buxxi@omfilm.net>
Co-authored-by: Thomas Hirschberger <47733292+Tom-Hirschberger@users.noreply.github.com>
Co-authored-by: Kristjan ESPERANTO <35647502+KristjanESPERANTO@users.noreply.github.com>
Co-authored-by: Andrés Vanegas Jiménez <142350+angeldeejay@users.noreply.github.com>
