Enrich crash email with detail and hints where crash reason can be inferred

[Steve-Mcl]

closes #4934

Description

• passes crash detail from API to postoffice routine
• checks/infers the 2 suggested stop reasons from the data:
  • out of memory
  • unhandled exception
• Adds 2 email templates for these conditions with each one including specific extra info:
  • possible reasons
  • possible solutions
  • last (max) 20 log entries
• Updates the generic Crash template to also display last (max) 20 log entries (only displayed if the context data includes logs)

Screenshots

Related Issue(s)

owner: #4934
parent: #4653

Checklist

• I have read the contribution guidelines
• Suitable unit/system level tests have been added and they pass
• Documentation has been updated
  • Upgrade instructions
  • Configuration details
  • Concepts
• Changes flowforge.yml?
  • Issue/PR raised on FlowFuse/helm to update ConfigMap Template
  • Issue/PR raised on FlowFuse/CloudProject to update values for Staging/Production

Labels

• Includes a DB migration? -> add the area:migration label

[Steve-Mcl]

@joepavitt added as 2nd reviewer for the visual aspect

[codecov]

Codecov Report

Attention: Patch coverage is 92.72727% with 4 lines in your changes missing coverage. Please review.

Project coverage is 78.37%. Comparing base (ad8ffa7) to head (fde43a3).
Report is 19 commits behind head on main.

Files with missing lines	Patch %	Lines
forge/postoffice/index.js	89.28%	3 Missing ⚠️
forge/routes/logging/index.js	83.33%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4936      +/-   ##
==========================================
+ Coverage   78.31%   78.37%   +0.05%     
==========================================
  Files         326      328       +2     
  Lines       15387    15438      +51     
  Branches     3536     3559      +23     
==========================================
+ Hits        12051    12099      +48     
- Misses       3336     3339       +3     

Flag	Coverage Δ
backend	78.37% <92.72%> (+0.05%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Steve-Mcl]

Added tests:

  Instance Alerts emails
    Alerts
      ✔ Crashed, via api (no logs) (79ms)
      ✔ Crashed, generic, via api (including logs)
      ✔ Crashed, out-of-memory, via api (including logs) (51ms)
      ✔ Crashed, uncaught-exception, via api (including logs) (42ms)

[knolleary]

Approved - although have proposed changing the email content Logs... to Logs: as more of a header (and more consistent with the other section headers)