Metrics are not updated anymore

[kelson42]

since many weeks (looks like) on metrics.kiwix.org

[rgaudin]

since many weeks (looks like) on metrics.kiwix.org

No, it's not been like this for weeks. I suppose those graph are recreated periodically. Logs included a ton of references to some index being readonly in elasticsearch.

I've restarted with a fresh eleasticsearch folder and it looks better but some graphs are missing. I want to believe those will be generated in a periodic task so I'll check tomorrow.

For the record, there are some errors in the logs… The git ones I've seen them often, for a while. The other ones might be new…

Starting Elasticsearch
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Starting Elasticsearch Server:.
Starting MariaDB
Starting MariaDB database server: mysqld.
Starting Cron
Starting periodic command scheduler: cron.
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  387  100   351  100    36  20647   2117 --:--:-- --:--:-- --:--:-- 22764
Waiting for Elasticsearch  to start...{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":".kibana","index_uuid":"_na_","index":".kibana"}],"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":".kibana","index_uuid":"_na_","index":".kibana"},"status":404}Starting Kibiter
Waiting for Kibiter to start....
Settings Kibana
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
   14    0     0  100    14      0     11  0:00:01  0:00:01 --:--:--    11
   14    0     0  100    14      0      6  0:00:02  0:00:02 --:--:--     6
  104  100    90  100    14     40      6  0:00:02  0:00:02 --:--:--    47
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  146  100   131  100    15    129     14  0:00:01  0:00:01 --:--:--   144
  146  100   131  100    15    129     14  0:00:01  0:00:01 --:--:--   144
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  247  100   199  100    48    195     47  0:00:01  0:00:01 --:--:--   242
  247  100   199  100    48    195     47  0:00:01  0:00:01 --:--:--   242
{"settings":{"buildNum":{"userValue":16428},"indexPattern:placeholder":{"userValue":"*"}}}{"settings":{"buildNum":{"userValue":16428},"indexPattern:placeholder":{"userValue":"*"},"histogram:barTarget":{"userValue":"20"}}}{"settings":{"buildNum":{"userValue":16428},"indexPattern:placeholder":{"userValue":"*"},"histogram:barTarget":{"userValue":"20"},"defaultIndex":{"userValue":"26d36150-0f7c-11ea-ae8c-d9e77f11fa16"}}}
Import dashboard
2022-07-11 13:48:17,888 Dashboard Overview imported
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
   56  100    21  100    35   1105   1842 --:--:-- --:--:-- --:--:--  2947
{"acknowledged":true}Starting SirMordred to build a GrimoireLab dashboard
This will usually take a while...
Exception in thread Global tasks:
Traceback (most recent call last):
  File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.7/dist-packages/sirmordred/task_manager.py", line 83, in run
    task = tc(self.config)
  File "/usr/local/lib/python3.7/dist-packages/sirmordred/task_panels.py", line 285, in __init__
    for ds in self.panels_menu:
TypeError: 'NoneType' object is not iterable

2022-07-11 13:56:27,854 - grimoire_elk.elk - ERROR - [git] Problem executing study enrich_demography:git, 'after_key'
2022-07-11 13:56:27,854 - sirmordred.task_manager - ERROR - [git] Exception in Task Manager 'after_key'
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/sirmordred/task_manager.py", line 99, in run
    task.execute()
  File "/usr/local/lib/python3.7/dist-packages/sirmordred/task_enrich.py", line 447, in execute
    raise e
  File "/usr/local/lib/python3.7/dist-packages/sirmordred/task_enrich.py", line 437, in execute
    self.__studies(retention_time)
  File "/usr/local/lib/python3.7/dist-packages/sirmordred/task_enrich.py", line 350, in __studies
    do_studies(ocean_backend, enrich_backend, studies_args, retention_time=retention_time)
  File "/usr/local/lib/python3.7/dist-packages/grimoire_elk/elk.py", line 459, in do_studies
    raise e
  File "/usr/local/lib/python3.7/dist-packages/grimoire_elk/elk.py", line 456, in do_studies
    study(ocean_backend, enrich_backend, **params)
  File "/usr/local/lib/python3.7/dist-packages/grimoire_elk/enriched/git.py", line 580, in enrich_demography
    super().enrich_demography(ocean_backend, enrich_backend, date_field, author_field=author_field)
  File "/usr/local/lib/python3.7/dist-packages/grimoire_elk/enriched/enrich.py", line 1854, in enrich_demography
    Enrich.run_demography(self, date_field, author_field, log_prefix)
  File "/usr/local/lib/python3.7/dist-packages/grimoire_elk/enriched/enrich.py", line 1885, in run_demography
    for author in authors_min_max_data:
  File "/usr/local/lib/python3.7/dist-packages/grimoire_elk/enriched/enrich.py", line 1948, in fetch_authors_min_max_dates
    after = aggregations_author['after_key'][author_field]
KeyError: 'after_key'
Exception in thread git:
Traceback (most recent call last):
  File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.7/dist-packages/sirmordred/task_manager.py", line 99, in run
    task.execute()
  File "/usr/local/lib/python3.7/dist-packages/sirmordred/task_enrich.py", line 447, in execute
    raise e
  File "/usr/local/lib/python3.7/dist-packages/sirmordred/task_enrich.py", line 437, in execute
    self.__studies(retention_time)
  File "/usr/local/lib/python3.7/dist-packages/sirmordred/task_enrich.py", line 350, in __studies
    do_studies(ocean_backend, enrich_backend, studies_args, retention_time=retention_time)
  File "/usr/local/lib/python3.7/dist-packages/grimoire_elk/elk.py", line 459, in do_studies
    raise e
  File "/usr/local/lib/python3.7/dist-packages/grimoire_elk/elk.py", line 456, in do_studies
    study(ocean_backend, enrich_backend, **params)
  File "/usr/local/lib/python3.7/dist-packages/grimoire_elk/enriched/git.py", line 580, in enrich_demography
    super().enrich_demography(ocean_backend, enrich_backend, date_field, author_field=author_field)
  File "/usr/local/lib/python3.7/dist-packages/grimoire_elk/enriched/enrich.py", line 1854, in enrich_demography
    Enrich.run_demography(self, date_field, author_field, log_prefix)
  File "/usr/local/lib/python3.7/dist-packages/grimoire_elk/enriched/enrich.py", line 1885, in run_demography
    for author in authors_min_max_data:
  File "/usr/local/lib/python3.7/dist-packages/grimoire_elk/enriched/enrich.py", line 1948, in fetch_authors_min_max_dates
    after = aggregations_author['after_key'][author_field]
KeyError: 'after_key'

2022-07-11 14:08:56,266 - grimoire_elk.elk - ERROR - Error feeding raw from cocom (https://github.com/kiwix/ipfs-portal): git command - 'git <command> [<revision>...] -- [<file>...]'
 (return code: 128)
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/grimoire_elk/elk.py", line 194, in feed_backend
    ocean_backend.feed(**params)
  File "/usr/local/lib/python3.7/dist-packages/grimoire_elk/raw/elastic.py", line 231, in feed
    self.feed_items(items)
  File "/usr/local/lib/python3.7/dist-packages/grimoire_elk/raw/elastic.py", line 247, in feed_items
    for item in items:
  File "/usr/local/lib/python3.7/dist-packages/perceval/backend.py", line 316, in fetch
    for item in self.fetch_items(category, **kwargs):
  File "/usr/local/lib/python3.7/dist-packages/graal/graal.py", line 184, in fetch_items
    for commit in commits:
  File "/usr/local/lib/python3.7/dist-packages/perceval/backends/core/git.py", line 164, in fetch_items
    for commit in commits:
  File "/usr/local/lib/python3.7/dist-packages/perceval/backends/core/git.py", line 259, in parse_git_log_from_iter
    for commit in parser.parse():
  File "/usr/local/lib/python3.7/dist-packages/perceval/backends/core/git.py", line 542, in parse
    for line in self.stream:
  File "/usr/local/lib/python3.7/dist-packages/perceval/backends/core/git.py", line 1057, in log
    for line in self._exec_nb(cmd_log, cwd=self.dirpath, env=self.gitenv):
  File "/usr/local/lib/python3.7/dist-packages/perceval/backends/core/git.py", line 1287, in _exec_nb
    raise RepositoryError(cause=cause)
perceval.errors.RepositoryError: git command - 'git <command> [<revision>...] -- [<file>...]'
 (return code: 128)
2022-07-11 14:08:57,238 - grimoire_elk.elk - ERROR - Error feeding raw from cocom (https://github.com/kiwix/java-libkiwix): git command - 'git <command> [<revision>...] -- [<file>...]'
 (return code: 128)
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/grimoire_elk/elk.py", line 194, in feed_backend
    ocean_backend.feed(**params)
  File "/usr/local/lib/python3.7/dist-packages/grimoire_elk/raw/elastic.py", line 231, in feed
    self.feed_items(items)
  File "/usr/local/lib/python3.7/dist-packages/grimoire_elk/raw/elastic.py", line 247, in feed_items
    for item in items:
  File "/usr/local/lib/python3.7/dist-packages/perceval/backend.py", line 316, in fetch
    for item in self.fetch_items(category, **kwargs):
  File "/usr/local/lib/python3.7/dist-packages/graal/graal.py", line 184, in fetch_items
    for commit in commits:
  File "/usr/local/lib/python3.7/dist-packages/perceval/backends/core/git.py", line 164, in fetch_items
    for commit in commits:
  File "/usr/local/lib/python3.7/dist-packages/perceval/backends/core/git.py", line 259, in parse_git_log_from_iter
    for commit in parser.parse():
  File "/usr/local/lib/python3.7/dist-packages/perceval/backends/core/git.py", line 542, in parse
    for line in self.stream:
  File "/usr/local/lib/python3.7/dist-packages/perceval/backends/core/git.py", line 1057, in log
    for line in self._exec_nb(cmd_log, cwd=self.dirpath, env=self.gitenv):
  File "/usr/local/lib/python3.7/dist-packages/perceval/backends/core/git.py", line 1287, in _exec_nb
    raise RepositoryError(cause=cause)
perceval.errors.RepositoryError: git command - 'git <command> [<revision>...] -- [<file>...]'
 (return code: 128)
2022-07-11 14:08:58,140 - grimoire_elk.elk - ERROR - Error feeding raw from cocom (https://github.com/kiwix/k8s): git command - 'git <command> [<revision>...] -- [<file>...]'
 (return code: 128)
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/grimoire_elk/elk.py", line 194, in feed_backend
    ocean_backend.feed(**params)
  File "/usr/local/lib/python3.7/dist-packages/grimoire_elk/raw/elastic.py", line 231, in feed
    self.feed_items(items)
  File "/usr/local/lib/python3.7/dist-packages/grimoire_elk/raw/elastic.py", line 247, in feed_items
    for item in items:
  File "/usr/local/lib/python3.7/dist-packages/perceval/backend.py", line 316, in fetch
    for item in self.fetch_items(category, **kwargs):
  File "/usr/local/lib/python3.7/dist-packages/graal/graal.py", line 184, in fetch_items
    for commit in commits:
  File "/usr/local/lib/python3.7/dist-packages/perceval/backends/core/git.py", line 164, in fetch_items
    for commit in commits:
  File "/usr/local/lib/python3.7/dist-packages/perceval/backends/core/git.py", line 259, in parse_git_log_from_iter
    for commit in parser.parse():
  File "/usr/local/lib/python3.7/dist-packages/perceval/backends/core/git.py", line 542, in parse
    for line in self.stream:
  File "/usr/local/lib/python3.7/dist-packages/perceval/backends/core/git.py", line 1057, in log
    for line in self._exec_nb(cmd_log, cwd=self.dirpath, env=self.gitenv):
  File "/usr/local/lib/python3.7/dist-packages/perceval/backends/core/git.py", line 1287, in _exec_nb
    raise RepositoryError(cause=cause)
perceval.errors.RepositoryError: git command - 'git <command> [<revision>...] -- [<file>...]'
 (return code: 128)

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.