Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ESP8266 Enhance Performance #10

Open
ghost opened this issue Feb 28, 2017 · 22 comments
Open

ESP8266 Enhance Performance #10

ghost opened this issue Feb 28, 2017 · 22 comments

Comments

@ghost
Copy link

ghost commented Feb 28, 2017

Splitting off from #2 - to avoid the discussion drowning in that one

  1. I have a rather urgent need to work on this for a non-LW application too
  2. This task falls under my new dayjob at OpenBuilds - which means I will have about a week to work on this uninterrupted. Yay
  3. If I work on this I may be able to take some of the stress of the other team members

So... Off to start thinking about this. Now that we have comms in the backend for ESP I am considering replacing websocket with a telnet implementation instead. That has proved to work well in a benchtest earlier this week.

Open to thoughts though @cprezzi @DarklyLabs

Pitfalls

I am aware of the conns that we cant then just host LW somewhere and connect to an ESP on the LAN.

But as time has gone on, I honestly believe LW will be only an Electron app. Thats definitely the way to go for distribution, update, code signing, and all the other things we have to think about (speaking of code signing - https://mspoweruser.com/microsoft-just-added-the-best-way-of-preventing-installation-of-bloatware-in-windows-10/ is a particularly worrying thing. We may need to start signing with Authenticode and just get into the windows store if thats where its heading in the future). It also solves some of the cross platform issues. Its not like one can run a laser shop without at least one PC in the shop.... So I dont think ending up with a solution that requires the application to be installed somewhere, is a big issue. server.js can even be run on a SBC (Pi) without the GUI if thats needed
@ghost ghost mentioned this issue Feb 28, 2017
Closed
@cprezzi
Copy link
Member

cprezzi commented Feb 28, 2017 •
edited
Loading

I'm not sure yet, if websocket realy is the problem, or just the additional delay the ESP is generating, but you will find out. At least a lower level network protocoll could reduce the performance need on the WLAN<>Serial gateway.

I was already thinking if it's possible to crunch the Websocket<>Serial part of lw.comm-server into an ESP32 directly or at least a C.H.I.P or the new RPi Zero W, so we wouldn't need an additional server for such installations.

You are right, it's enough to publish one electron installer (per OS) that has the app and the server in it. The integrated server doesn't bother much if not used. But I wouldn't drop the option to install the server (without GUI) separately on a machine like an RPi or a spare workshop PC, even if this will probably not be used very often.

By the way: This is like Repetier Host works. All workstations also have a server installed, even if not used (but can be used for "cloud computing").

@ghost
Copy link
Author

ghost commented Feb 28, 2017 via email

@cprezzi
Copy link
Member

cprezzi commented Feb 28, 2017

I would still integrate a mini Webpage for mobile / tablet control in the server only version, if possible.

Would be cool if you could organize some ESP32s for me. Sent you a PM.

@ghost
Copy link
Author

ghost commented Feb 28, 2017

@openhardwarecoza Can you explain this a little more?
We have machines out now which use the ESP setup you helped develop for us. It is working as we would like with our machines appearing on the network and allowing customers to connect directly to them.
Why change this now?
What will it improve?
Will customers with multiple machines be supported?
How will we instigate this update to the ESP firmware?

@ghost
Copy link
Author

ghost commented Feb 28, 2017 via email

@ghost
Copy link
Author

ghost commented Feb 28, 2017

@openhardwarecoza Ok, taking a very deep breath!
Can you point me to where we can see how to perform an OTA update to the ESP module?

@ghost
Copy link
Author

ghost commented Mar 1, 2017 via email

@cprezzi
Copy link
Member

cprezzi commented Mar 1, 2017

Should I activate Telnet in dev_comms?
It's already implemented, but never tested, so I disabled it for the moment.

@ghost
Copy link
Author

ghost commented Mar 1, 2017 via email

@cprezzi
Copy link
Member

cprezzi commented Mar 1, 2017

Ok, I'll do.

@ghost
Copy link
Author

ghost commented Mar 1, 2017 •
edited by ghost
Loading

Ok, so I created two very simple nodejs scripts (extracted sections from lw.comms-server) to time the delay between sending a gcode, and receiving back the OK:

telnet

var serialport = require("serialport");
var SerialPort = serialport;
var net = require('net');

var buffer, date, olddate, telnetBuffer;

var ws = net.connect(23, '10.2.0.203');
// ws.binaryType = "arraybuffer";


ws.on('connect', function (e) {
  console.log(e);
  console.log("telnet Connected");
  setTimeout(function (argument) {
    ws.write('G1 X1\n');
  }, 1000);
  setTimeout(function (argument) {
    ws.write('G1 X1\n');
  }, 5000);
});

ws.on('data', function (response) {
    // console.log('...')
                  //var bytes = new Uint8Array(data);
                  for (var i = 0; i < response.length; i++) {
                      if (response[i] != 0x0d) {
                          telnetBuffer += String.fromCharCode(response[i]);
                      }
                  }
                  var responseArray;
                  if (telnetBuffer.substr(-1) === '\n') {
                      responseArray = telnetBuffer.split('\n');
                      telnetBuffer = responseArray.pop();
                  } else {
                      responseArray = telnetBuffer.split('\n');
                      telnetBuffer = '';
                  }
                  var data = '';
                  while (responseArray.length > 0) {
                      data = responseArray.shift();
                      // console.log('Telnet: ' + data, 3);
                      if (data.indexOf('ok') === 0) { // Got an OK so we are clear to send
                        var end = new Date() - date;
                        console.log("time: %dms", end);
                        date = new Date();
                        ws.write('G1 X1\n');
                        console.log(data);
                      }
                  }
              });

websocket

var serialport = require("serialport");
var SerialPort = serialport;
const WebSocket = require('ws');
var buffer, date, olddate;

var ws = new WebSocket('ws://10.1.0.240/');
ws.binaryType = "arraybuffer";


ws.onopen = function(e) {
  console.log(e);
  console.log("ESP8266 Connected");
  setTimeout(function (argument) {
    ws.send('G1 X1\n');
  }, 1000);
  setTimeout(function (argument) {
    ws.send('G1 X1\n');
  }, 2000);
};

ws.onmessage = function(e){

  // console.log(e.data)
  var data = "";
  if(e.data instanceof ArrayBuffer){
    var bytes = new Uint8Array(e.data);
    for (var i = 0; i < bytes.length; i++) {
      data += String.fromCharCode(bytes[i]);
    }
  } else {
    data = e.data;
  }
    buffer += data
    var split = buffer.split("\n");
    buffer = split.pop(); //last not fin data back to buffer
    // console.log(split)
    for (i=0; i< split.length; i++) {
      var response = split[i];
      // console.log(response)
      // trigger line handling event here
      if(response.indexOf("ok") != -1 || response == "start\r" || response.indexOf('<') == 0){
        if (response.indexOf("ok") == 0) { // Got an OK so we are clear to send
          var end = new Date() - date;
          console.log("time: %dms", end);
          date = new Date();
          ws.send('G1 X1\n');
          console.log(response);
          // uploadLine()
        } else if (response.indexOf('<') != -1) {
          console.log(response);
        } else {
          console.log(response)
        }
        blocked = false;
      }
  }
};

And the results are in!

Websocket: 52-57ms
Telnet: 5-6ms !!!!

@ghost
Copy link
Author

ghost commented Mar 1, 2017

telnet

telnettime

websocket

websocket

@ghost
Copy link
Author

ghost commented Mar 1, 2017

Added bonus, this drops a whole extra protocol from our stack, since we can use the same protocol for smoothieware ethernet, Smoothieware wifi, grbl wifi and theoretically also tinyg wifi

@cprezzi
Copy link
Member

cprezzi commented Mar 2, 2017

Cool. That sounds very promissing!

@ghost
Copy link
Author

ghost commented Mar 2, 2017

I was positively surprised too! Didnt realize that the extra headers used to wrap the packet make such a huge time difference

@cprezzi
Copy link
Member

cprezzi commented Mar 2, 2017

When I implemented the Telnet part, I tried a Telnet library first, but I realised that it is just overhead and decided to use the low level net lib.

@cprezzi
Copy link
Member

cprezzi commented Mar 2, 2017 •
edited
Loading

This means I have to extend the Telnet part in lw.comm-server to support multiple firmwares (like with USB) and not only smoothie, but that's not a big deal ;)

@ghost
Copy link
Author

ghost commented Mar 2, 2017 via email

@ghost
Copy link
Author

ghost commented Mar 2, 2017 via email

@cprezzi
Copy link
Member

cprezzi commented Mar 2, 2017

Yep. I like low level anyway. Less dependency to libraries that I can't control ;)
That's a bit of a concern with LW4. We just have too many dependencies.

@ghost
Copy link
Author

ghost commented Mar 2, 2017 via email

@ghost
Copy link
Author

ghost commented Apr 1, 2017

Just checking in to see if there has been any progress made with the Telnet approach or improving the current server performance over Wifi?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@cprezzi