LockFile Error (Device Client is already running.) #418

nishida1023 · 2023-06-19T11:23:44Z

Describe the bug
LockFile Error

To Reproduce
Start aws-iot-device-client before fleet provisioning is completed.

Expected behavior

aws-iot-device-client does not stop.

Actual behavior

aws-iot-device-client stopped with an error.

Logs

2023-06-19T11:10:22.203Z [DEBUG] {SharedCrtResourceManager.cpp}: Attempting to disconnect MQTT connection
2023-06-19T11:10:22.216Z [INFO]  {SharedCrtResourceManager.cpp}: MQTT Connection is now disconnected
2023-06-19T11:10:22.216Z [DEBUG] {LockFile.cpp}: creating lockfile
2023-06-19T11:10:22.216Z [ERROR] {LockFile.cpp}: Pid 42199 associated with active process ./aws-iot-device-client in lockfile: /run/lock/devicecl.lock
2023-06-19T11:10:22.219Z [ERROR] {Main.cpp}: *** AWS IOT DEVICE CLIENT FATAL ERROR: Error obtaining lockfile: Device Client is already running.
2023-06-19T11:10:22.219Z [ERROR] {Main.cpp}: *** AWS IOT DEVICE CLIENT FATAL ERROR: An instance of Device Client is already running.
2023-06-19T11:10:22.219Z [DEBUG] {SharedCrtResourceManager.cpp}: Attempting to disconnect MQTT connection
2023-06-19T11:10:22.219Z [ERROR] {SharedCrtResourceManager.cpp}: MQTT Connection failed to disconnect
AWS IoT Device Client must abort execution, reason: An instance of Device Client is already running.
Please check the AWS IoT Device Client logs for more information

Environment (please complete the following information):
OS: Ubuntu
Version 22.04.1
Architecture: x86_64
Device Client commit: 239d10d

Additional context
If fleet provisioning has not been completed, the reason for the error is that init() is called twice.
https://github.com/awslabs/aws-iot-device-client/pull/391/files

The text was updated successfully, but these errors were encountered:

HarshGandhi-AWS · 2023-06-20T17:36:02Z

Hello @nishida1023 , can you share complete logs with config passed? Just by looking at the logs you shared, it is difficult to understand what is the issue. Also what do you mean by " Start aws-iot-device-client before fleet provisioning is completed."? Can you share exact steps to reproduce the issue?

HarshGandhi-AWS · 2023-06-20T17:50:27Z

Also it looks like there is a bug in our code. With Fleet Provisioning enabled, we are calling init() twice which will cause DC abort.
https://github.com/awslabs/aws-iot-device-client/blob/main/source/main.cpp#L354-L358
https://github.com/awslabs/aws-iot-device-client/blob/main/source/main.cpp#L402-L405

nishida1023 · 2023-06-21T08:45:51Z

@HarshGandhi-AWS
Sorry my English is not good.

Also what do you mean by " Start aws-iot-device-client before fleet provisioning is completed."?

The bug occurs only if the aws-iot-device-client-runtime.conf doesn't exist.

Also it looks like there is a bug in our code. With Fleet Provisioning enabled, we are calling init() twice which will cause DC abort.

I noticed this bug and created an ISSUE. This bug is included in the latest main branch.

can you share complete logs with config passed?

ubuntu@ubuntu:~/work/aws-iot-device-client/build$ ./aws-iot-device-client 
2023-06-21T10:58:32.548Z [INFO]  {Config.cpp}: Successfully fetched JSON config file: 
    {
      "endpoint":	"xxxx.iot.ap-northeast-1.amazonaws.com",
      "cert":	"~/certs/testconn/xxxx.pem.crt",
      "key":	"~/certs/testconn/xxxx.pem.key",
      "root-ca":	"~/certs/AmazonRootCA1.pem",
      "thing-name":	"xxxx",
      "logging":	{
        "level":	"DEBUG",
        "type":	"stdout",
        "file": "./aws-iot-device-client.log",
        "enable-sdk-logging":	true,
        "sdk-log-level":	"DEBUG",
        "sdk-log-file": "./sdk.log"
      },
      "jobs":	{
        "enabled":	true,
        "handler-directory": "/home/ubuntu/.aws-iot-device-client/jobs"
      },
      "tunneling":	{
        "enabled":	true
      },
      "device-defender":	{
        "enabled":	true,
        "interval": 300
      },
      "fleet-provisioning":	{
        "enabled":	true,
        "template-name": "xxxx",
        "template-parameters": "{\"SerialNumber\": \"xxxx\"}",
        "csr-file": "~/certs/testconn/device.csr",
        "device-key": "~/certs/testconn/device-key.pem.key"
      },
      "samples": {
        "pub-sub": {
          "enabled": true,
          "publish-topic": "test/dc/pubtopic",
          "publish-file": "/home/ubuntu/.aws-iot-device-client/pubsub/publish-file.txt",
          "subscribe-topic": "test/dc/subtopic",
          "subscribe-file": "/home/ubuntu/.aws-iot-device-client/pubsub/subscribe-file.txt",
          "publish-on-change": true
        }
      },
      "config-shadow":	{
        "enabled":	false
      },
      "sample-shadow": {
        "enabled": false,
        "shadow-name": "test_shadow",
        "shadow-input-file": "/tmp/shadow_in/shadow-input-file",
        "shadow-output-file": "/tmp/shadow_out/shadow-output-file"
      }
    }

2023-06-21T10:58:32.548Z [INFO]  {FileUtils.cpp}: Successfully create directory /home/ubuntu/.aws-iot-device-client/sample-shadow/ with required permissions 700
2023-06-21T10:58:32.548Z [INFO]  {Config.cpp}: ~/.aws-iot-device-client/sample-shadow/default-sample-shadow-document
2023-06-21T10:58:32.549Z [INFO]  {Config.cpp}: Succesfully create default file: /home/ubuntu/.aws-iot-device-client/sample-shadow/default-sample-shadow-document required for storage of shadow document
2023-06-21T10:58:32.549Z [DEBUG] {Config.cpp}: Did not find a runtime configuration file, assuming Fleet Provisioning has not run for this device
2023-06-21T10:58:32.549Z [DEBUG] {Config.cpp}: Did not find a http proxy config file /home/ubuntu/.aws-iot-device-client/http-proxy.conf, assuming HTTP proxy is disabled on this device
2023-06-21T10:58:32.549Z [DEBUG] {EnvUtils.cpp}: Updated PATH environment variable to: /home/ubuntu/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin:/home/ubuntu/.aws-iot-device-client:/home/ubuntu/.aws-iot-device-client/jobs:/home/ubuntu/work/aws-iot-device-client/build:/home/ubuntu/work/aws-iot-device-client/build/jobs
2023-06-21T10:58:32.549Z [INFO]  {Main.cpp}: Now running AWS IoT Device Client version v1.8.26-239d10d
2023-06-21T10:58:32.550Z [INFO]  {SharedCrtResourceManager.cpp}: SDK logging is enabled. Check ./sdk.log for SDK logs.
2023-06-21T10:58:32.550Z [DEBUG] {LockFile.cpp}: creating lockfile
2023-06-21T10:58:32.550Z [DEBUG] {Retry.cpp}: Retryable function starting, it will retry until success
2023-06-21T10:58:32.555Z [INFO]  {SharedCrtResourceManager.cpp}: Establishing MQTT connection with client id xxxx...
2023-06-21T10:58:32.914Z [INFO]  {SharedCrtResourceManager.cpp}: MQTT connection established with return code: 0
2023-06-21T10:58:32.915Z [INFO]  {SharedCrtResourceManager.cpp}: Shared MQTT connection is ready!
2023-06-21T10:58:32.915Z [INFO]  {FleetProvisioning.cpp}: Fleet Provisioning Feature has been started.
2023-06-21T10:58:32.915Z [INFO]  {FileUtils.cpp}: Successfully create directory /home/ubuntu/.aws-iot-device-client/keys/ with required permissions 700
2023-06-21T10:58:32.915Z [INFO]  {FileUtils.cpp}: Successfully create directory /home/ubuntu/.aws-iot-device-client/ with required permissions 745
2023-06-21T10:58:32.916Z [INFO]  {FleetProvisioning.cpp}: Successfully fetched CSR file '/home/ubuntu/certs/testconn/device.csr' and stored its content.
2023-06-21T10:58:32.916Z [INFO]  {FleetProvisioning.cpp}: Provisioning new device certificate using CreateCertificateFromCsr API
2023-06-21T10:58:32.916Z [INFO]  {FleetProvisioning.cpp}: Subscribing to CreateCertificateFromCsr Accepted and Rejected topics
2023-06-21T10:58:33.015Z [INFO]  {FleetProvisioning.cpp}: Publishing to CreateCertificateFromCsr topic
2023-06-21T10:58:33.249Z [INFO]  {FleetProvisioning.cpp}: CreateCertificateFromCsrResponse certificateId: xxxx. ***
2023-06-21T10:58:33.250Z [INFO]  {FleetProvisioning.cpp}: Stored certificate in /home/ubuntu/.aws-iot-device-client/keys/xxxx.crt file
2023-06-21T10:58:33.250Z [INFO]  {FleetProvisioning.cpp}: Attempting to set permissions for certificate...
2023-06-21T10:58:33.250Z [INFO]  {FleetProvisioning.cpp}: Successfully set permissions on provisioned public certificate
2023-06-21T10:58:33.250Z [INFO]  {FleetProvisioning.cpp}: Subscribing to RegisterThing Accepted and Rejected topics
2023-06-21T10:58:33.396Z [INFO]  {FleetProvisioning.cpp}: Publishing to RegisterThing topic
2023-06-21T10:58:33.735Z [INFO]  {FleetProvisioning.cpp}: RegisterThingResponse ThingName: xxxx.
2023-06-21T10:58:33.735Z [DEBUG] {FleetProvisioning.cpp}: {
"runtime-config": {
    "completed-fp": true,
    "cert": "/home/ubuntu/.aws-iot-device-client/keys/xxxx.pem.crt",
    "key": "/home/ubuntu/certs/testconn/device-key.pem.key",
    "thing-name": "xxxx",
    "device-config": {
        
        }
    }
}
2023-06-21T10:58:33.735Z [INFO]  {FleetProvisioning.cpp}: Exported runtime configurations to: ~/.aws-iot-device-client/aws-iot-device-client-runtime.conf
2023-06-21T10:58:33.735Z [INFO]  {FleetProvisioning.cpp}: Successfully provisioned thing: xxxx
2023-06-21T10:58:33.736Z [INFO]  {Config.cpp}: Successfully fetched JSON config file: {
"runtime-config": {
    "completed-fp": true,
    "cert": "/home/ubuntu/.aws-iot-device-client/keys/xxxx.pem.crt",
    "key": "/home/ubuntu/certs/testconn/device-key.pem.key",
    "thing-name": "xxxx",
    "device-config": {
        
        }
    }
}
2023-06-21T10:58:33.736Z [DEBUG] {SharedCrtResourceManager.cpp}: Attempting to disconnect MQTT connection
2023-06-21T10:58:33.740Z [INFO]  {SharedCrtResourceManager.cpp}: MQTT Connection is now disconnected
2023-06-21T10:58:33.741Z [DEBUG] {LockFile.cpp}: creating lockfile
2023-06-21T10:58:33.741Z [ERROR] {LockFile.cpp}: Pid 49039 associated with active process ./aws-iot-device-client in lockfile: /run/lock/devicecl.lock
2023-06-21T10:58:33.741Z [ERROR] {Main.cpp}: *** AWS IOT DEVICE CLIENT FATAL ERROR: Error obtaining lockfile: Device Client is already running.
2023-06-21T10:58:33.741Z [ERROR] {Main.cpp}: *** AWS IOT DEVICE CLIENT FATAL ERROR: An instance of Device Client is already running.
2023-06-21T10:58:33.741Z [DEBUG] {SharedCrtResourceManager.cpp}: Attempting to disconnect MQTT connection
2023-06-21T10:58:33.741Z [ERROR] {SharedCrtResourceManager.cpp}: MQTT Connection failed to disconnect
AWS IoT Device Client must abort execution, reason: An instance of Device Client is already running.
Please check the AWS IoT Device Client logs for more information

HarshGandhi-AWS · 2023-06-29T16:41:52Z

Hello @nishida1023 , thank you for sharing details.
As I suspected, this issue happens when Fleet Provisioning feature is enabled. I will work on resolving this issue today.

To unblock yourself from using device client, I would suggest disabling Fleet Provisioning feature via config and starting the device client again. If you want to use Fleet Provisioning, then unfortunately there is no workaround and you will have to wait until the bug is fixed.

HarshGandhi-AWS · 2023-06-29T23:52:12Z

Hello @nishida1023 , this issue is been resolved in the latest commit.

I will close this issue now, feel free to reopen this issue or create a new one if you still face any difficulty using Device Client. Thank you.

nishida1023 · 2023-06-30T05:03:21Z

@HarshGandhi-AWS
Thank you for resolving the issue!

nishida1023 added the bug Something isn't working label Jun 19, 2023

HarshGandhi-AWS mentioned this issue Jun 29, 2023

Removed DC initialization code which creates multiple lock files. #419

Merged

HarshGandhi-AWS closed this as completed Jun 29, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

LockFile Error (Device Client is already running.) #418

LockFile Error (Device Client is already running.) #418

nishida1023 commented Jun 19, 2023 •

edited

Loading

HarshGandhi-AWS commented Jun 20, 2023

HarshGandhi-AWS commented Jun 20, 2023 •

edited

Loading

nishida1023 commented Jun 21, 2023 •

edited

Loading

HarshGandhi-AWS commented Jun 29, 2023

HarshGandhi-AWS commented Jun 29, 2023

nishida1023 commented Jun 30, 2023

LockFile Error (Device Client is already running.) #418

LockFile Error (Device Client is already running.) #418

Comments

nishida1023 commented Jun 19, 2023 • edited Loading

HarshGandhi-AWS commented Jun 20, 2023

HarshGandhi-AWS commented Jun 20, 2023 • edited Loading

nishida1023 commented Jun 21, 2023 • edited Loading

HarshGandhi-AWS commented Jun 29, 2023

HarshGandhi-AWS commented Jun 29, 2023

nishida1023 commented Jun 30, 2023

nishida1023 commented Jun 19, 2023 •

edited

Loading

HarshGandhi-AWS commented Jun 20, 2023 •

edited

Loading

nishida1023 commented Jun 21, 2023 •

edited

Loading