Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NanoMQ] - NanoMQ dies on boot when hostapd is present #79

Open
jr1221 opened this issue Feb 1, 2024 · 2 comments
Open

[NanoMQ] - NanoMQ dies on boot when hostapd is present #79

jr1221 opened this issue Feb 1, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@jr1221
Copy link
Contributor

jr1221 commented Feb 1, 2024

Observed Behavior

Nanomq dies on bootup. It never dies when running in any scenario, except when run on the bootup.

Runs fine when

  • run manually via S75
  • run via rcS after rcK
  • run with cli
  • run with start-stop-daemon
  • run after rming all logs, confs

Dies when:

  • run via S75 by bootup rcS
  • run via cli in another bootup's script

Doesn't seem related to calypso, even when disabled it fails.

Expected Behavior

Should just boot correctly when it is run.

May need to raise with upstream, borderline impossible to reproduce outside of the init script so...

Is this related to #78 ?

Steps to Reproduce

  1. boot nero or tpu
  2. obverse failure

Screenshots (as needed)

Logs

nng-nanomq[274]: broker nng_listen

broker nng_listen entry not found or smthing like that forgot to copy this part down.

@jr1221 jr1221 added the bug Something isn't working label Feb 1, 2024
@jr1221
Copy link
Contributor Author

jr1221 commented Feb 1, 2024

Both this and #78 still occur without:

  • quic
  • plugin enablement and symbol export
  • sqlite

Broken in both ways back to at least 0.20.0, dont have time to test older versions.

@jr1221
Copy link
Contributor Author

jr1221 commented Feb 1, 2024

This bug only has been fixed via the wierdest thing I have ever seen on linux. Short story to myself is the hostapd commit in #77 will fix this bug.

Long story:
nanomq and start-stop-daemon spawn PIDs in the bootup sequence in /var/run. The rcS script of the system calls the scripts which spawns nanomq, hostapd, etc. With the hostapd bug the script I wrote for it was breaking start-stop-daemon, starting hostapd and alotting TWO PIDs for it. Basically start-stop-daemon was being forked from in its fork management mode. Somehow either the action of ending this instance of start-stop-daemon prematurely and/or the action of alotting the dead PID allowed for a NNG internal mechanism to have trouble. I have two theories on what trouble NanoNNG ran into.

  1. start-stop-daemon triggered a mode internally via an undiscovered bug that prevented forking of nanomq from the main process. This killed the nng_listener, propogating up and killing nanomq.
  2. hostapd running in the background entered an undefined state in which the PID for its own port management was somehow left dangling, meaning the syscalls the nng_listen socket mechanism ran could not alot 1883 to the process, killing the listener.

These are total guesses but either way any hanging processes in the busybox init phase should be examined as their issues could propogate to other processes.

@jr1221 jr1221 closed this as completed Feb 1, 2024
jr1221 added a commit that referenced this issue Feb 1, 2024
jr1221 added a commit that referenced this issue Feb 1, 2024
* builds cleanly

* calypso working

see nanomq issue! probably not from calypso

* initial plugin support

* basic plugin and support

still cant find symbols via buildroot, plugin tested on debian.

* remove upstream header

* add nanomq plugin ability

actually works on buildroot so happy

* add nanomq and calypso to nero

* vital fixups

* important: switch to pid forking of nanomq

* bump calypso version

* fixes nanomq bug (#79) by some miracle

* bump nanomq version

* add comments to the timestamp plugin
@jr1221 jr1221 reopened this Feb 8, 2024
@jr1221 jr1221 changed the title [NanoMQ] - NanoMQ dies on boot [NanoMQ] - NanoMQ dies on boot when hostapd is present Feb 8, 2024
jr1221 added a commit that referenced this issue Mar 2, 2024
* builds cleanly

* calypso working

see nanomq issue! probably not from calypso

* initial plugin support

* basic plugin and support

still cant find symbols via buildroot, plugin tested on debian.

* remove upstream header

* add nanomq plugin ability

actually works on buildroot so happy

* add nanomq and calypso to nero

* vital fixups

* important: switch to pid forking of nanomq

* bump calypso version

* fixes nanomq bug (#79) by some miracle

* bump nanomq version

* add comments to the timestamp plugin
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant