Skip to content
This repository has been archived by the owner on Dec 14, 2023. It is now read-only.

Add monitoring to the rabbitmq service #840

Open
wants to merge 16 commits into
base: master
Choose a base branch
from
Open

Add monitoring to the rabbitmq service #840

wants to merge 16 commits into from

Conversation

thepsalmist
Copy link

@thepsalmist thepsalmist commented Oct 26, 2022 •
edited
Loading

Add monitoring to the rabbitmq service by:

  1. Leveraging on rabbitmq's (version > 3.8) inbuilt rabbitmq_prometheus plugin
  2. Expose port 15692 on the rabbitmq service to access the scrapped rabbitmq services from prometheus.
  3. Add prometheus alertmanager, and an email receiver

philbudne and others added 16 commits March 6, 2022 17:37
@philbudne
Make no_dedup_sentences the default for extract-and-vector
815042a 
	(can be overridden via MC_NO_DEDUP_SENTENCES env var)

apps/common/src/python/mediawords/util/config/__init__.py: add env_bool function
apps/extract-and-vector/bin/extract_and_vector_worker.py: honor MC_NO_DEDUP_SENTENCES
@philbudne
common/Dockerfile: skip jieba.cache creation; makes empty root owned …
c77dd18 
…file
@philbudne
solr-base/Dockerfile: try cloning mediacloud config as mediacloud64
f37b7f9
@philbudne
common/Dockerfile: reenable jieba.cache creation & chown it
e1be9a4
@philbudne
apps/common/src/python/mediawords/solr/request.py: add/use SOLR_COLLE…
1868296 
…CTION

Set to "mediacloud2" (solr alias!)
@philbudne
apps/import-solr-data/src/perl/MediaWords/Solr/Dump.pm: speedups for …
4ab3729 
…processing backlog

up FETCH_BLOCK_SIZE from 100 to 200:
ammortizes citus connection startup time by processing fetching larger blocks of text.

Honor skip_update_snapshot option (defaults to true):
skip setting snapshots.searchable=true
@philbudne
add apps/solr-base/src/solr/aliases.json with "mediacloud2" solr alias
791248a
@philbudne
apps/common/src/requirements.txt: force MarkupSafe==2.0.1
c4fb3d6 
(Jinja2 2.11.3 can't cope with the new MarkupSafe 2.1.1)
@philbudne
solr-zookeeper: preload aliases.json into zookeeper
5845a61
@philbudne
apps/postgresql-server/bin/apply_migrations.sh: increase PGCTL_START_…
40391ed 
…TIMEOUT to 3hrs
@philbudne
postgresql-pgbouncer/conf/pgbounder.init:
f395116 
PG server running on postgresql EC2 server w/o docker
@philbudne
pgbouncer.ini: use postgresql server ip
76be844
@philbudne
apps/webapp-api/src/perl/MediaWords/Controller/Api/V2/Timespans.pm
bf74554 
Removed extra AND to try to fix mediacloud#833 as suggested by Rahul.
@thepsalmist
update rabbitmq to include prometheus plugin and update docker compos…
b75c6ca 
…e file
@thepsalmist
add prometheus alertmanager
1e9f0c3
@thepsalmist
alertmanager email config variabes
f8bf9cf
@thepsalmist thepsalmist marked this pull request as ready for review November 2, 2022 05:19
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@thepsalmist @philbudne