Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File-Locking for ~/.jfrog/jfrog-cli.conf.v5 is unreliable #350

Open
HWoidt opened this issue Mar 7, 2022 · 0 comments
Open

File-Locking for ~/.jfrog/jfrog-cli.conf.v5 is unreliable #350

HWoidt opened this issue Mar 7, 2022 · 0 comments
Labels
bug Something isn't working

Comments

@HWoidt
Copy link

HWoidt commented Mar 7, 2022

Describe the bug
When concurrently executing configuration commands from jfrog cli, the config file in ~/.jfrog/jfrog-cli.conf.v5 may become corrupted. This causes all jfrog cli commands that are executed later to fail with the error message "[Error] invalid character 't' after top-level value" (See corrupted json below under 'Screenshots')

To Reproduce
The relevant operations are:

  1. lock.currentTime = time.Now().UnixNano()
  2. file, err := os.OpenFile(lock.fileName, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0666)
  3. filesList, err := fileutils.ListFiles(filepath.Dir(lock.fileName), false)

The files in the lockfile directory are sorted according to the timestamp embedded in the filename. The oldest file containing a PID that is still running "wins" the lock and may go ahead to change the config file.
A possible cause for the observed behavior is the following sequence of events where two processes P1 and P2 want to access the config file concurrently:

  • P1: time.now()
  • P2: time.now()
  • P2: os.OpenFile()
  • P2: fileutils.ListFiles() -> yay I'm the oldest one -> goes ahead to change the config file
  • P1: os.OpenFile()
  • P1: fileutils.ListFiles() -> yay I'm the oldes... ☠️

Expected behavior
The lockfile mechanism works reliable.

Screenshots

$ cat ~/.jfrog/jfrog-cli.conf.v5
{
  "servers": [],
  "version": "5"
}toryUrl": "*redacted*",
      "user": "*redacted*",
      "password": "*redacted*",
      "serverId": "*redacted*",
      "isDefault": true
    }
  ],
  "version": "5"
}

Versions

  • JFrog CLI core version: master
  • JFrog CLI version (if applicable): 2.12.1
  • Artifactory version: 7.33.12

Additional context
We have multiple Azure-Pipeline Agents running on the same self-hosted Ubuntu 20.04.3 machine, executing tasks from the Jfrog-Azure-Devops Plugin in parallel. The plugin uses jfrog-cli internally.

Possible Solution
Use file-creation or file-modification time from the filesystem as the sorting key. Since this might have lower resolution, a collision of identical timestamps has a higher probability. When a process recognizes, that two files have the same timestamp, the process can remove their own file, wait a random amount of time (This is important to prevent two processes going back-and-forth forever), and retry the operation.

@HWoidt HWoidt added the bug Something isn't working label Mar 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant