Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Running multiple qiimes concurrently before refreshing the cache causes fireworks #186

Open
BenKaehler opened this issue Apr 26, 2018 · 6 comments

Comments

@BenKaehler
Copy link
Member

BenKaehler commented Apr 26, 2018

Bug Description
I tried to run 10 qiimes concurrently before I had refreshed the cache.

My log follows. Manually refreshing the cache before starting fixed the problem.

Screenshots

Click to expand!
2018-04-26 22:34:40,847:compute-5-2.local:34110:INFO:{'bash_command': ('qiime', 'feature-classifier', 'fit-classifier-naive-bayes', '--i-reference-reads', '../ref/99_otus_v4.qza', '--i-reference-taxonomy', '../ref/99_otu_taxonomy.qza', '--o-classifier', 'classifiers/99_otus_v4-{}.qza', '--i-class-weight', 'weights/{}.qza'), 'log_level': 'INFO', 'log_path': 'log'}
2018-04-26 22:34:40,865:compute-5-2.local:34110:INFO:{'mpi_logging': '0.1.1-dev', 'dispatcher': '0.1.1-dev', 'mpimap': '0.1.1-dev'}
2018-04-26 22:34:47,648:compute-5-4.local:6665:WARNING:Got return code 1 for
qiime feature-classifier fit-classifier-naive-bayes --i-reference-reads ../ref/99_otus_v4.qza --i-reference-taxonomy ../ref/99_otu_taxonomy.qza --o-classifier classifiers/99_otus_v4-weights-normalise-False-unobserved-weight-1e-06-fold-4.qza --i-class-weight weights/weights-normalise-False-unobserved-weight-1e-06-fold-4.qza:
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
Traceback (most recent call last):
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 118, in _get_cached_state
    return json.load(fh)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 268, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 319, in loads
    return _default_decoder.decode(s)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 355, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 24585 (char 24584)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/bin/qiime", line 11, in <module>
    sys.exit(qiime())
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 1061, in invoke
    cmd_name, cmd, args = self.resolve_command(ctx, args)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 1100, in resolve_command
    cmd = self.get_command(ctx, cmd_name)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/commands.py", line 99, in get_command
    plugin = self._plugin_lookup[name]
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/commands.py", line 75, in _plugin_lookup
    import q2cli.cache
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 302, in <module>
    CACHE = DeploymentCache()
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 61, in __init__
    self._state = self._get_cached_state(refresh=refresh)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 123, in _get_cached_state
    return json.load(fh)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 268, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 319, in loads
    return _default_decoder.decode(s)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 355, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 24585 (char 24584)

2018-04-26 22:34:47,724:compute-5-4.local:6664:WARNING:Got return code 1 for
qiime feature-classifier fit-classifier-naive-bayes --i-reference-reads ../ref/99_otus_v4.qza --i-reference-taxonomy ../ref/99_otu_taxonomy.qza --o-classifier classifiers/99_otus_v4-weights-normalise-False-unobserved-weight-1e-06-fold-2.qza --i-class-weight weights/weights-normalise-False-unobserved-weight-1e-06-fold-2.qza:
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
Traceback (most recent call last):
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 118, in _get_cached_state
    return json.load(fh)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 268, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 319, in loads
    return _default_decoder.decode(s)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 355, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 24585 (char 24584)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/bin/qiime", line 11, in <module>
    sys.exit(qiime())
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 1061, in invoke
    cmd_name, cmd, args = self.resolve_command(ctx, args)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 1100, in resolve_command
    cmd = self.get_command(ctx, cmd_name)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/commands.py", line 99, in get_command
    plugin = self._plugin_lookup[name]
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/commands.py", line 75, in _plugin_lookup
    import q2cli.cache
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 302, in <module>
    CACHE = DeploymentCache()
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 61, in __init__
    self._state = self._get_cached_state(refresh=refresh)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 123, in _get_cached_state
    return json.load(fh)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 268, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 319, in loads
    return _default_decoder.decode(s)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 355, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 24585 (char 24584)

2018-04-26 22:34:47,755:compute-5-4.local:6663:WARNING:Got return code 1 for
qiime feature-classifier fit-classifier-naive-bayes --i-reference-reads ../ref/99_otus_v4.qza --i-reference-taxonomy ../ref/99_otu_taxonomy.qza --o-classifier classifiers/99_otus_v4-weights-normalise-False-unobserved-weight-1e-06-fold-7.qza --i-class-weight weights/weights-normalise-False-unobserved-weight-1e-06-fold-7.qza:
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
Traceback (most recent call last):
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 118, in _get_cached_state
    return json.load(fh)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 268, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 319, in loads
    return _default_decoder.decode(s)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 355, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 24585 (char 24584)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/bin/qiime", line 11, in <module>
    sys.exit(qiime())
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 1061, in invoke
    cmd_name, cmd, args = self.resolve_command(ctx, args)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 1100, in resolve_command
    cmd = self.get_command(ctx, cmd_name)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/commands.py", line 99, in get_command
    plugin = self._plugin_lookup[name]
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/commands.py", line 75, in _plugin_lookup
    import q2cli.cache
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 302, in <module>
    CACHE = DeploymentCache()
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 61, in __init__
    self._state = self._get_cached_state(refresh=refresh)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 123, in _get_cached_state
    return json.load(fh)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 268, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 319, in loads
    return _default_decoder.decode(s)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 355, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 24585 (char 24584)

2018-04-26 22:34:49,521:compute-5-0.local:39148:WARNING:Got return code 1 for
qiime feature-classifier fit-classifier-naive-bayes --i-reference-reads ../ref/99_otus_v4.qza --i-reference-taxonomy ../ref/99_otu_taxonomy.qza --o-classifier classifiers/99_otus_v4-uniform.qza --i-class-weight weights/uniform.qza:
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
Traceback (most recent call last):
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 118, in _get_cached_state
    return json.load(fh)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 268, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 319, in loads
    return _default_decoder.decode(s)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 49229 (char 49228)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/bin/qiime", line 11, in <module>
    sys.exit(qiime())
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 1061, in invoke
    cmd_name, cmd, args = self.resolve_command(ctx, args)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 1100, in resolve_command
    cmd = self.get_command(ctx, cmd_name)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/commands.py", line 99, in get_command
    plugin = self._plugin_lookup[name]
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/commands.py", line 75, in _plugin_lookup
    import q2cli.cache
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 302, in <module>
    CACHE = DeploymentCache()
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 61, in __init__
    self._state = self._get_cached_state(refresh=refresh)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 123, in _get_cached_state
    return json.load(fh)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 268, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 319, in loads
    return _default_decoder.decode(s)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 355, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ':' delimiter: line 1 column 223221 (char 223220)

2018-04-26 22:34:49,675:compute-5-0.local:39147:WARNING:Got return code 1 for
qiime feature-classifier fit-classifier-naive-bayes --i-reference-reads ../ref/99_otus_v4.qza --i-reference-taxonomy ../ref/99_otu_taxonomy.qza --o-classifier classifiers/99_otus_v4-weights-normalise-False-unobserved-weight-1e-06-fold-3.qza --i-class-weight weights/weights-normalise-False-unobserved-weight-1e-06-fold-3.qza:
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
Traceback (most recent call last):
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 118, in _get_cached_state
    return json.load(fh)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 268, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 319, in loads
    return _default_decoder.decode(s)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 355, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ':' delimiter: line 1 column 49415 (char 49414)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/bin/qiime", line 11, in <module>
    sys.exit(qiime())
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 1061, in invoke
    cmd_name, cmd, args = self.resolve_command(ctx, args)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 1100, in resolve_command
    cmd = self.get_command(ctx, cmd_name)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/commands.py", line 99, in get_command
    plugin = self._plugin_lookup[name]
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/commands.py", line 75, in _plugin_lookup
    import q2cli.cache
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 302, in <module>
    CACHE = DeploymentCache()
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 61, in __init__
    self._state = self._get_cached_state(refresh=refresh)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 123, in _get_cached_state
    return json.load(fh)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 268, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 319, in loads
    return _default_decoder.decode(s)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 355, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 99399 (char 99398)

2018-04-26 22:35:26,448:compute-5-2.local:34111:WARNING:Got return code 1 for
qiime feature-classifier fit-classifier-naive-bayes --i-reference-reads ../ref/99_otus_v4.qza --i-reference-taxonomy ../ref/99_otu_taxonomy.qza --o-classifier classifiers/99_otus_v4-weights-normalise-False-unobserved-weight-1e-06-fold-5.qza --i-class-weight weights/weights-normalise-False-unobserved-weight-1e-06-fold-5.qza:
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
Traceback (most recent call last):
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 118, in _get_cached_state
    return json.load(fh)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 268, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 319, in loads
    return _default_decoder.decode(s)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 355, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 24577 (char 24576)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/bin/qiime", line 11, in <module>
    sys.exit(qiime())
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 1061, in invoke
    cmd_name, cmd, args = self.resolve_command(ctx, args)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 1100, in resolve_command
    cmd = self.get_command(ctx, cmd_name)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/commands.py", line 99, in get_command
    plugin = self._plugin_lookup[name]
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/commands.py", line 75, in _plugin_lookup
    import q2cli.cache
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 302, in <module>
    CACHE = DeploymentCache()
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 61, in __init__
    self._state = self._get_cached_state(refresh=refresh)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 123, in _get_cached_state
    return json.load(fh)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 268, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 319, in loads
    return _default_decoder.decode(s)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 355, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 73866 (char 73865)

2018-04-26 22:35:26,623:compute-5-2.local:34112:WARNING:Got return code 1 for
qiime feature-classifier fit-classifier-naive-bayes --i-reference-reads ../ref/99_otus_v4.qza --i-reference-taxonomy ../ref/99_otu_taxonomy.qza --o-classifier classifiers/99_otus_v4-weights-normalise-False-unobserved-weight-1e-06-fold-9.qza --i-class-weight weights/weights-normalise-False-unobserved-weight-1e-06-fold-9.qza:
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
Traceback (most recent call last):
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 118, in _get_cached_state
    return json.load(fh)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 268, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 319, in loads
    return _default_decoder.decode(s)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 355, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 24577 (char 24576)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/bin/qiime", line 11, in <module>
    sys.exit(qiime())
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 1061, in invoke
    cmd_name, cmd, args = self.resolve_command(ctx, args)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 1100, in resolve_command
    cmd = self.get_command(ctx, cmd_name)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/commands.py", line 99, in get_command
    plugin = self._plugin_lookup[name]
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/commands.py", line 75, in _plugin_lookup
    import q2cli.cache
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 302, in <module>
    CACHE = DeploymentCache()
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 61, in __init__
    self._state = self._get_cached_state(refresh=refresh)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 123, in _get_cached_state
    return json.load(fh)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 268, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 319, in loads
    return _default_decoder.decode(s)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 355, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 73866 (char 73865)

2018-04-26 22:35:26,825:compute-5-1.local:42065:WARNING:Got return code 1 for
qiime feature-classifier fit-classifier-naive-bayes --i-reference-reads ../ref/99_otus_v4.qza --i-reference-taxonomy ../ref/99_otu_taxonomy.qza --o-classifier classifiers/99_otus_v4-weights-normalise-False-unobserved-weight-1e-06-fold-6.qza --i-class-weight weights/weights-normalise-False-unobserved-weight-1e-06-fold-6.qza:
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
Traceback (most recent call last):
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 118, in _get_cached_state
    return json.load(fh)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 268, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 319, in loads
    return _default_decoder.decode(s)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 355, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 24577 (char 24576)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/bin/qiime", line 11, in <module>
    sys.exit(qiime())
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 1061, in invoke
    cmd_name, cmd, args = self.resolve_command(ctx, args)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 1100, in resolve_command
    cmd = self.get_command(ctx, cmd_name)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/commands.py", line 99, in get_command
    plugin = self._plugin_lookup[name]
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/commands.py", line 75, in _plugin_lookup
    import q2cli.cache
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 302, in <module>
    CACHE = DeploymentCache()
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 61, in __init__
    self._state = self._get_cached_state(refresh=refresh)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 123, in _get_cached_state
    return json.load(fh)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 268, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 319, in loads
    return _default_decoder.decode(s)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 355, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 73866 (char 73865)

2018-04-26 22:35:27,012:compute-5-1.local:42064:WARNING:Got return code 1 for
qiime feature-classifier fit-classifier-naive-bayes --i-reference-reads ../ref/99_otus_v4.qza --i-reference-taxonomy ../ref/99_otu_taxonomy.qza --o-classifier classifiers/99_otus_v4-weights-normalise-False-unobserved-weight-1e-06-fold-1.qza --i-class-weight weights/weights-normalise-False-unobserved-weight-1e-06-fold-1.qza:
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
Traceback (most recent call last):
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 118, in _get_cached_state
    return json.load(fh)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 268, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 319, in loads
    return _default_decoder.decode(s)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 355, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 24577 (char 24576)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/bin/qiime", line 11, in <module>
    sys.exit(qiime())
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 1061, in invoke
    cmd_name, cmd, args = self.resolve_command(ctx, args)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/click/core.py", line 1100, in resolve_command
    cmd = self.get_command(ctx, cmd_name)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/commands.py", line 99, in get_command
    plugin = self._plugin_lookup[name]
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/commands.py", line 75, in _plugin_lookup
    import q2cli.cache
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 302, in <module>
    CACHE = DeploymentCache()
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 61, in __init__
    self._state = self._get_cached_state(refresh=refresh)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/site-packages/q2cli/cache.py", line 123, in _get_cached_state
    return json.load(fh)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 268, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/__init__.py", line 319, in loads
    return _default_decoder.decode(s)
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/kaehler/miniconda2/envs/qiime2-2018.2/lib/python3.5/json/decoder.py", line 355, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 73866 (char 73865)
@ebolyen
Copy link
Member

ebolyen commented Apr 26, 2018

We should probably write the config to a temp location and move it after it is fully written, then we won't have to mess with locking or other things like that.

@danteese
Copy link
Contributor

It would be nice if you can provide some code to reproduce the error above. I tried the command qiime dev refresh-cache & qiime dev refresh-cache & qiime dev refresh-cache and I didn't detect something different from the normal execution...

Captura de Pantalla 2019-09-16 a la(s) 20 16 33

@ebolyen
Copy link
Member

ebolyen commented Sep 17, 2019

Wow this is an older issue!
I'm still able to reproduce with something like this:

for i in `seq 1 20`; do qiime dev refresh-cache &  done
[1]   Done                    qiime dev refresh-cache
[2]   Done                    qiime dev refresh-cache
[3]   Done                    qiime dev refresh-cache
[4]   Done                    qiime dev refresh-cache
[5]   Done                    qiime dev refresh-cache
[6]   Done                    qiime dev refresh-cache
[7]   Done                    qiime dev refresh-cache
[8]   Exit 1                  qiime dev refresh-cache
[9]   Done                    qiime dev refresh-cache
[10]   Done                    qiime dev refresh-cache
[11]   Done                    qiime dev refresh-cache
[12]   Done                    qiime dev refresh-cache
[13]   Done                    qiime dev refresh-cache
[14]   Exit 1                  qiime dev refresh-cache
[15]   Done                    qiime dev refresh-cache
[16]   Done                    qiime dev refresh-cache
[17]   Done                    qiime dev refresh-cache
[18]   Done                    qiime dev refresh-cache
[19]-  Exit 1                  qiime dev refresh-cache
[20]+  Done                    qiime dev refresh-cache

Same JSONDecodeError as it tramples over itself.

Is this something you'd be interested in helping with @danteese?

@danteese
Copy link
Contributor

danteese commented Sep 17, 2019

For sure, old issue!
And it's quite interesting because this is your execution on my computer:

(qiime2-dev) > $ for i in `seq 1 20`; do qiime dev refresh-cache &  done
[2] 79152
[3] 79153
[4] 79154
[5] 79155
[6] 79156
[7] 79157
[8] 79158
[9] 79159
[10] 79160
[11] 79161
[12] 79162
[13] 79163
[14] 79164
[15] 79165
[16] 79166
[17] 79167
[18] 79168
[19] 79169
[20] 79170
[21] 79171

dantebazaldua@x86_64-apple-darwin13 ~                                        [17:45:17]
(qiime2-dev) > $ QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.

[17]    79167 done       qiime dev refresh-cache
[4]    79154 done       qiime dev refresh-cache
(qiime2-dev) > $
[19]    79169 done       qiime dev refresh-cache
(qiime2-dev) > $
[15]    79165 done       qiime dev refresh-cache
(qiime2-dev) > $
[12]    79162 done       qiime dev refresh-cache
(qiime2-dev) > $
[11]    79161 done       qiime dev refresh-cache
(qiime2-dev) > $
[10]    79160 done       qiime dev refresh-cache
(qiime2-dev) > $
[9]    79159 done       qiime dev refresh-cache
(qiime2-dev) > $
[7]    79157 done       qiime dev refresh-cache
(qiime2-dev) > $
[6]    79156 done       qiime dev refresh-cache
(qiime2-dev) > $
[5]    79155 done       qiime dev refresh-cache
(qiime2-dev) > $
[3]    79153 done       qiime dev refresh-cache
(qiime2-dev) > $
[21]  + 79171 done       qiime dev refresh-cache
(qiime2-dev) > $
[20]  + 79170 done       qiime dev refresh-cache
(qiime2-dev) > $
[18]  + 79168 done       qiime dev refresh-cache
(qiime2-dev) > $
[16]  + 79166 done       qiime dev refresh-cache
(qiime2-dev) > $
[14]  + 79164 done       qiime dev refresh-cache
(qiime2-dev) > $
[13]  + 79163 done       qiime dev refresh-cache
(qiime2-dev) > $
[8]  + 79158 done       qiime dev refresh-cache
(qiime2-dev) > $
[2]  + 79152 done       qiime dev refresh-cache

As you can see I'm not able to reproduce the exit 1 with the JSONDecodeError. My first thought would be the version of qiime2-dev but I'm not sure ...

@ElDeveloper
Copy link
Member

ElDeveloper commented Sep 18, 2019 via email

@ebolyen
Copy link
Member

ebolyen commented Sep 18, 2019

You both must have nicer computers than me. What about 100 iterations? Really we just need it to read as the file is being written.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants