sirmordred fails panels task on opendistro 1.11.0 #40
Comments
|
I'm working on it |
|
Hi @marcofranssen, Sorry for the late reply, kidash is available to use for I'm still working on it and I'll keep you updated about it. Best, |
|
Thanks for the update, looking forward to the fix. Happy to test a
prerelease of the mordred image as well.
…On Thu, Nov 12, 2020 at 4:38 PM Quan Zhou ***@***.***> wrote:
Hi @marcofranssen <https://github.com/marcofranssen>,
Sorry for the late reply, kidash is available to use for ES 6.x but OpenDistro
1.11.0 uses ES 7.x due to that the structure of the index patterns and
the panels have changed and we have to adapt kidash
<https://github.com/chaoss/grimoirelab-kidash> and the panels in sigils
<https://github.com/chaoss/grimoirelab-sigils> accordingly.
I'm still working on it and I'll keep you updated about it.
Best,
Quan
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#40 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAFJTTPUNAF3SH7OCTBCZQ3SPP6RDANCNFSM4TLJYDCQ>
.
|
|
Hi @zhquan, Just catching up with you on the progress of supporting OpenDistro. Would be great if you have a A long time ago I filed an issue on the kibiter project that told me to use opendistro instead. Also found this blogpost. https://blog.bitergia.com/2020/06/08/kibiter-new-release-now-supporting-open-distro-for-elasticsearch/ What can I do to make it work? |
|
Hi @marcofranssen, The SirMordred release is not ready yet, but you can do this:
i.e for the dashboard I will let you know when SirMordred is ready. I hope it helps you. Best, |
|
@zhquan I have build following docker container to import dashboards FROM python:3-alpine
RUN apk --no-cache add git
RUN git clone -b opendistro https://github.com/zhquan/grimoirelab-kidash.git /kidash
WORKDIR /kidash
RUN python3 setup.py install
RUN git clone -b opendistro https://github.com/zhquan/grimoirelab-sigils.git sigils
ENTRYPOINT [ "bin/kidash" ]Now I run this inside the container $ kidash -e http://admin:admin@elasticsearch --import sigils/json/git-index-pattern.json
2021-01-22 15:46:49,190 Index pattern git from ['git'] imported
$ kidash -e http://admin:admin@elasticsearch --import sigils/json/git.json
2021-01-22 15:46:53,752 Dashboard Git importedBut where in opendistro can I now find the menu-item with the dashboard? |
|
@marcofranssen - The |
|
@zhquan so how do I navigate to the dashboard in opendistro? I do see the index patterns, but struggle to find the dashboard itself. |
|
@marcofranssen - You will see the dashboard on the |
|
@zhquan Although kidash reports the index pattern and dashboard are created, I can't see them in Kibana. When I click Dashboards I'm redirected to the page to create a index pattern, so it seems the index pattern created from kidash isn't compatible with opendistro or isn't created accordingly. Could you try with my setup? docker-compose.yml version: '3.7'
services:
gateway:
image: traefik:v2.3
command:
- --api.insecure=true
- --entrypoints.http.address=:80
- --entrypoints.https.address=:443
- --providers.docker=true
- --providers.docker.exposedByDefault=false
- --serversTransport.insecureSkipVerify=true
- --log.level=DEBUG
labels:
traefik.enable: "true"
traefik.http.routers.http-elasticsearch.rule: Host(`localhost`) && PathPrefix(`/es`) || Host(`elasticsearch`)
traefik.http.routers.http-elasticsearch.entrypoints: http
traefik.http.routers.http-elasticsearch.middlewares: redirect-to-https
traefik.http.middlewares.redirect-to-https.redirectscheme.scheme: https
ports:
- "80:80"
- "443:443"
- "8080:8080"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
es01:
image: amazon/opendistro-for-elasticsearch:${OPENDISTRO_VERSION}
environment:
cluster.name: grimoire-cluster
node.name: es01
discovery.seed_hosts: es01,es02
cluster.initial_master_nodes: es01,es02
bootstrap.memory_lock: "true"
ES_JAVA_OPTS: -Xms${ES_HEAP_SIZE}m -Xmx${ES_HEAP_SIZE}m
ports:
- 9200:9200
- 9600:9600 # required for Performance Analyzer
volumes:
- "./es-data/es01:/usr/share/elasticsearch/data:delegated"
ulimits:
memlock:
soft: -1
hard: -1
labels:
traefik.enable: "true"
traefik.http.routers.elasticsearch.entrypoints: https
traefik.http.routers.elasticsearch.rule: (Host(`localhost`) || Host(`${TRAEFIK_HOST}`)) && PathPrefix(`/es`) || Host(`elasticsearch`)
traefik.http.routers.elasticsearch.middlewares: es-stripprefix
traefik.http.routers.elasticsearch.tls:
traefik.http.middlewares.es-stripprefix.stripprefix.prefixes: /es
traefik.http.services.elasticsearch.loadbalancer.server.port: 9200
traefik.http.services.elasticsearch.loadbalancer.server.scheme: https
es02:
image: amazon/opendistro-for-elasticsearch:${OPENDISTRO_VERSION}
environment:
cluster.name: grimoire-cluster
node.name: es02
discovery.seed_hosts: es01,es02
cluster.initial_master_nodes: es01,es02
bootstrap.memory_lock: "true"
ES_JAVA_OPTS: -Xms${ES_HEAP_SIZE}m -Xmx${ES_HEAP_SIZE}m
volumes:
- "./es-data/es02:/usr/share/elasticsearch/data:delegated"
ulimits:
memlock:
soft: -1
hard: -1
labels:
traefik.enable: "true"
traefik.http.routers.elasticsearch.entrypoints: https
traefik.http.routers.elasticsearch.rule: (Host(`localhost`) || Host(`${TRAEFIK_HOST}`)) && PathPrefix(`/es`) || Host(`elasticsearch`)
traefik.http.routers.elasticsearch.middlewares: es-stripprefix
traefik.http.routers.elasticsearch.tls:
traefik.http.middlewares.es-stripprefix.stripprefix.prefixes: /es
traefik.http.services.elasticsearch.loadbalancer.server.port: 9200
traefik.http.services.elasticsearch.loadbalancer.server.scheme: https
mariadb:
image: "mariadb:${MARIADB_VERSION}"
restart: always
env_file:
- .env
command: --wait_timeout=2592000 --interactive_timeout=2592000 --max_connections=300
ports:
- "3306:3306"
volumes:
- "./mariadb-data:/var/lib/mysql:delegated"
kibana:
image: amazon/opendistro-for-elasticsearch-kibana:${OPENDISTRO_VERSION}
environment:
ELASTICSEARCH_URL: https://elasticsearch
ELASTICSEARCH_HOSTS: https://elasticsearch
links:
- "gateway:elasticsearch"
depends_on:
- gateway
labels:
traefik.enable: "true"
traefik.http.routers.dashboard.entrypoints: http
traefik.http.routers.dashboard.rule: Host(`localhost`) || Host(`${TRAEFIK_HOST}`)
traefik.http.services.dashboard.loadbalancer.server.port: 5601
kidash:
image: kidash
build: opendistro-dashboards
entrypoint: /usr/bin/env sh
command: |-
-c '
kidash --debug -e http://admin:admin@elasticsearch --import sigils/json/git-index-pattern.json
kidash --debug -e http://admin:admin@elasticsearch --import sigils/json/git.json
'
depends_on:
- kibana
- gateway
links:
- "gateway:elasticsearch"
hatstall:
image: grimoirelab/hatstall:latest
environment:
- DATABASE_DIR=/db/
- ADMIN_USER=admin
- ADMIN_PASS=admin
volumes:
- ./conf/grimoirelab/apache-hatstall.conf:/home/grimoirelab/apache-hatstall.conf
- ./conf/grimoirelab/shdb.cfg:/home/grimoirelab/shdb.cfg
links:
- mariadb
depends_on:
- gateway
labels:
traefik.enable: "true"
traefik.http.routers.hatstall.entrypoints: http
traefik.http.routers.hatstall.rule: (Host(`localhost`) || Host(`${TRAEFIK_HOST}`)) && (PathPrefix(`/identities/hatstall`) || PathPrefix(`/static/hatstall`) || PathPrefix(`/accounts/login`))
traefik.http.services.hatstall.loadbalancer.server.port: 80
mordred:
restart: on-failure:5
image: bitergia/mordred:grimoirelab-${GRIMOIRELAB_VERSION}
volumes:
- ./conf/grimoirelab/setup.cfg:/home/bitergia/conf/setup.cfg
- ./conf/grimoirelab/credentials.cfg:/home/bitergia/conf/credentials.cfg
- ./conf/grimoirelab/aliases.json:/home/bitergia/conf/aliases.json
- ./conf/grimoirelab/projects.json:/home/bitergia/conf/projects.json
- ./conf/grimoirelab/orgs.json:/home/bitergia/conf/organizations.json
- ./conf/grimoirelab/identities.yml:/home/bitergia/conf/identities.yml
- ./logs:/home/bitergia/logs:delegated
- ./.perceval:/home/bitergia/.perceval:delegated
- ./mordred-stage.sh:/home/bitergia/stage
links:
- "gateway:elasticsearch"
depends_on:
- mariadb
- gatewayopendistro-dashboards/Dockerfile FROM python:3-alpine
RUN apk --no-cache add git
RUN git clone -b opendistro https://github.com/zhquan/grimoirelab-kidash.git /kidash
WORKDIR /kidash
RUN python3 setup.py install
RUN git clone -b opendistro https://github.com/zhquan/grimoirelab-sigils.git sigils
ENTRYPOINT [ "bin/kidash" ]Howto run Put in place the grimorelab configs as defined in the volumes in the docker-compose setup. Left that out here as you probably know better what to put there. docker-compose up -d
# wait for kibana to be up at http://localhost
docker-compose up kidash
Also see here the debug log from kidash kidash_1 | 2021-01-26 10:51:00,394 Debug mode activated
kidash_1 | 2021-01-26 10:51:00,394 Reading panels JSON file: sigils/json/git-index-pattern.json
kidash_1 | 2021-01-26 10:51:00,394 Reading panel from directory: sigils/json/git-index-pattern.json
kidash_1 | 2021-01-26 10:51:00,394 Index-Pattern detected.
kidash_1 | 2021-01-26 10:51:00,451 Removing `release_date` from item index-pattern:git since not allowed, and adding it to Sigils index
kidash_1 | 2021-01-26 10:51:05,527 Release info added to Sigils index for index-pattern:git
kidash_1 | 2021-01-26 10:51:05,594 Reading panel from directory: sigils/json/git-index-pattern.json
kidash_1 | 2021-01-26 10:51:05,594 Index pattern git from ['git'] imported
kidash_1 | 2021-01-26 10:51:05,921 Debug mode activated
kidash_1 | 2021-01-26 10:51:05,921 Reading panels JSON file: sigils/json/git.json
kidash_1 | 2021-01-26 10:51:05,921 Reading panel from directory: sigils/json/git.json
kidash_1 | 2021-01-26 10:51:05,922 Panel detected.
kidash_1 | 2021-01-26 10:51:05,986 Cleaning dashboard from studies vis
kidash_1 | 2021-01-26 10:51:05,987 Not fixing height in Kibana >= 6 versions.
kidash_1 | 2021-01-26 10:51:05,987 Removing `release_date` from item dashboard:Git since not allowed, and adding it to Sigils index
kidash_1 | 2021-01-26 10:51:10,586 Release info added to Sigils index for dashboard:Git
kidash_1 | 2021-01-26 10:51:10,830 Reading panel from directory: sigils/json/git.json
kidash_1 | 2021-01-26 10:51:10,831 Dashboard Git imported |
|
@marcofranssen - you have to tell |
|
@zhquan thanks that did the trick. Looking forward to have this natively in mordred. |
|
@zhquan @marcofranssen thanks for participating in the discussion. Do you think we can close this issue? |
Elasticsearch and Kibana versions I'm using from opendistro which was suggested in chaoss/grimoirelab-sirmordred#491
I'm running a docker-compose setup with following images.
amazon/opendistro-for-elasticsearch:1.11.0amazon/opendistro-for-elasticsearch-kibana:1.11.0bitergia/mordred:grimoirelab-0.2.50Making a curl request shows the following
$ curl https://admin:admin@elasticsearch/.kibana/doc/dashboard:Overview -k {"_index":".kibana_1","_type":"doc","_id":"dashboard:Overview","found":false}Originally posted by @marcofranssen in chaoss/grimoirelab-sirmordred#492 (comment)
The text was updated successfully, but these errors were encountered: