Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix onie discover stopped while console XOFF is triggered #386

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

audihsu-qci
Copy link
Contributor

This patch print logs to files, which was printed to /dev/console directly
- tee_log_file to /var/log/onie-tee.log
- _log_console_msg to /var/log/onie-console.log
- /etc/rcS.d/S30console-logger.sh will tail these logs to /dev/console

Cause:
While XOFF is trigger with console port somehow, onie discover will stop
searching installer after a while.

Reproduce:
1. In console terminal, type CTRL+S to send XOFF
2. telnet/ssh to DUT, and tail -f /var/log/onie.log
3. The discover loop will stop searching after a while,
since linux kernel console buffer is full.

This patch print logs to files, which was printed to /dev/console directly
    - tee_log_file to /var/log/onie-tee.log
    - _log_console_msg to /var/log/onie-console.log
    - /etc/rcS.d/S30console-logger.sh will tail these logs to /dev/console

Cause:
    While XOFF is trigger with console port somehow, onie discover will stop
searching installer after a while.

Reproduce:
    1. In console terminal, type CTRL+S to send XOFF
    2. telnet/ssh to DUT, and tail -f /var/log/onie.log
    3. The discover loop will stop searching after a while,
       since linux kernel console buffer is full.
@cbrune
Copy link
Contributor

cbrune commented Jun 23, 2016

Interesting. Good find.


. /lib/onie/functions

touch $console_log_file && /usr/bin/tail -f $console_log_file > /dev/console &
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking for this whole patch, it seems we can simplify it a little bit.

Do we really need both $console_log_file and $tee_log_file?

It seems we only need $tee_log_file.

Copy link
Contributor Author

@audihsu-qci audihsu-qci Jun 24, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since there are _log_console_msg and other different log message levels, I shall keep the console_log_file and tee_log_file.
Unless the logging message definition is changed.

@cbrune
Copy link
Contributor

cbrune commented Jun 24, 2016

Thinking about this a bit more.... Are we sure this is really a problem worth solving?

Why would someone type ctrl-S on the serial console in the first place? This might be a problem for a developer who is working on ONIE, even that would rare. I don't see how this would ever impact an end user.

My big concern with the new implementation is that we are creating two new log files that will grow without bound. We already have /var/log/onie.log that captures all the output Imagine this scenario:

  1. a new switch is plugged in a data center on a Friday afternoon
  2. the operator has problems configuring DHCP or whatever and goes home for the weekend
  3. the switch keeps running, going through the image discovery waterfall
  4. the two new files keep growing, along with syslog file /var/log/onie.log
  5. all of this is in a ramdisk
  6. by Monday morning when the operator comes back the system has run out of memory
  7. the operator fixes the DHCP problem, but switch will not install an image because it is out of memory

That sounds bad to me.

I think this is a "won't fix". What we should do, however, is make a note of the problem. Instead of a pull request with a fix, I will create an "issue" to track this problem.

@cbrune
Copy link
Contributor

cbrune commented Jun 24, 2016

I created #387 to track this issue. I do not think we will fix this problem, however.

@audihsu-qci
Copy link
Contributor Author

The XOFF may not be sent by human. With console port, the XOFF is just some signals. That means, if someone plug ethernet port into console port, which is also RJ45, the console may receive the XOFF.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants