Server node 404; devices appearing and disappearing in the hub in a loop

[DLavigneGH]

Hi,
I have a Linux Ubuntu VM as a HUB (it doesn't have any devices connected, no plan to anyway) and a physical Windows machine as a server-node with a bunch of Android devices connected to it. I have this issue where I am seeing my devices in the hub web page but they keep disappearing and re-appearing.

The server-node terminal is showing:

[HTTP]
[ADB] Getting connected devices
[ADB] Connected devices: [{"udid":"2871e519553f7ece","state":"device"},{"udid":"R5CW70NAE1V","state":"device"}]
[device-farm] Android Device details for 2871e519553f7ece already available
[device-farm] Android Device details for R5CW70NAE1V already available
[device-farm] Updating remote android devices http://10.0.0.37:31337/device-farm/api/register
[device-farm] Pushed devices to hub [{"adbPort":5037,"systemPort":49874,"sdk":"9","realDevice":true,"name":"crownqltesq","busy":false,"state":"device","udid":"2871e519553f7ece","platform":"android","deviceType":"real","host":"http://10.0.9.38:4723","totalUtilizationTimeMilliSec":1529417,"sessionStartTime":0,"userBlocked":false,"offline":false,"meta":{"revision":0,"created":1707756392950,"version":0},"$loki":1},{"adbPort":5037,"systemPort":49885,"sdk":"14","realDevice":true,"name":"a54xcs","busy":false,"state":"device","udid":"R5CW70NAE1V","platform":"android","deviceType":"real","host":"http://10.0.9.38:4723","totalUtilizationTimeMilliSec":325442,"sessionStartTime":0,"userBlocked":false,"offline":false,"meta":{"revision":0,"created":1707756392950,"version":0},"$loki":2}]
[device-farm] Device "2871e519553f7ece" already exists in database
[device-farm] Device "R5CW70NAE1V" already exists in database
[HTTP] --> GET /device-farm/api/status
[HTTP] {}
[HTTP] --> GET /device-farm/api/status
[HTTP] {}
[HTTP] No route found for /device-farm/api/status
[HTTP] <-- GET /device-farm/api/status 404 3 ms - 211
[HTTP]
[HTTP] No route found for /device-farm/api/status
[HTTP] <-- GET /device-farm/api/status 404 3 ms - 211

At the start we see the devices, but then when the 404 occurs the devices are gone.
Another test; I used Postman to GET request to http://10.0.0.37:31337/device-farm/api/status and always received 200, no 404.

{
    "status": "ok",
    "version": "unknown (not running from npm package)"
}

Here is it's node-config.json:

{
    "server": {
      "port": 4723,
      "plugin": {
        "device-farm": {
          "platform": "android",
          "hub": "http://10.0.0.37:31337" 
        }
      }
    }
  }

I did a test net connection with powershell to the hub

Test-Netconnection 10.0.0.37 -port 31337
ComputerName     : 10.0.0.37
RemoteAddress    : 10.0.0.37
RemotePort       : 31337
InterfaceAlias   : Ethernet                                                                         
SourceAddress    : 10.0.9.38                                                                        
TcpTestSucceeded : True

The hub-config.json

{
  "server": {
    "port": 31337,
    "plugin": {
      "device-farm": {
        "platform": "none"
      }
    }
  }
}

("platform": "none" here because it doesn't have any devices and will not have any in the future, putting anything else just fails to launch server)
It receives the device info from the windows server:

[device-farm-main] Received non-webdriver request with url /device-farm/api/register. So, not proxying it to downstream.
[debug] [device-farm-main] Added 2 new devices to local database
[device-farm-main] Added new devices: [{"userBlocked":false,"offline":false,"adbPort":5037,"systemPort":49874,"sdk":"9","realDevice":true,"name":"crownqltesq","busy":false,"state":"device","udid":"2871e519553f7ece","platform":"android","deviceType":"real","host":"http://10.0.9.38:4723","totalUtilizationTimeMilliSec":1529417,"sessionStartTime":0},{"userBlocked":false,"offline":false,"adbPort":5037,"systemPort":49885,"sdk":"14","realDevice":true,"name":"a54xcs","busy":false,"state":"device","udid":"R5CW70NAE1V","platform":"android","deviceType":"real","host":"http://10.0.9.38:4723","totalUtilizationTimeMilliSec":325442,"sessionStartTime":0}]
[HTTP] <-- POST /device-farm/api/register?type=add 200 1 ms - 16

But then a moment later

[device-farm-main] Device Farm is not running at http://10.0.9.38:4723. Error: AxiosError: Request failed with status code 404
[device-farm-main] Device Farm is not running at http://10.0.9.38:4723. Error: AxiosError: Request failed with status code 404
[device-farm-main] Removing device 2871e519553f7ece from host http://10.0.9.38:4723 from device list.
[device-farm-main] Removing device R5CW70NAE1V from host http://10.0.9.38:4723 from device list.
[debug] [device-farm-main] Removing device with udid(s): 2871e519553f7ece, R5CW70NAE1V because the node is not alive

I can test net connection to that server:

Test-NetConnection  10.0.9.38 -port 4723
ComputerName     : 10.0.9.38
RemoteAddress    : 10.0.9.38
RemotePort       : 4723
InterfaceAlias   : Ethernet
SourceAddress    : 10.0.22.14
TcpTestSucceeded : True

or directly from the VM

telnet 10.0.9.38 4723
Trying 10.0.9.38...
Connected to 10.0.9.38.

Again with Postman I did GET request to http://10.0.9.38:4723/device-farm and also get 200.

What am I missing? I see the 404 error but I am not sure what or where I did something wrong.
Server and hub are launched using:
appium server -ka 800 --use-plugins=device-farm --config ./hub-config.json -pa /wd/hub
appium server -ka 800 --use-plugins=device-farm --config ./node-config.json -pa /wd/hub

Both using Appium 2.2.1
Windows: - [email protected] [installed (npm)]
Ubuntu VM: - [email protected] [installed (npm)]

We do have some network rules here but I have been told that the vlan traffic between Linux VM <> Windows machines should be going through okay both sides.

[DLavigneGH]

I worked on this a full day :P now I noticed the device-farm plugin on my windows machine doesn't match the version from my VM. I am updating it and testing I'll keep you posted.

[DLavigneGH]

That was it, the devices stopped disappearing after I updated the plugin on windows to have its version match with the vm.