Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add multihost support for PostgreSQL collector #646

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Add multihost support for PostgreSQL collector #646

wants to merge 4 commits into from

Conversation

wstranger
Copy link

@wstranger wstranger commented May 9, 2017

Add multihost support for PostgreSQL collector

for instance in self.config['instances'].keys():
# Get list of databases
#import pdb
#pdb.set_trace()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably can remove this

@coveralls
Copy link

coveralls commented May 10, 2017

Coverage Status

Changes Unknown when pulling 1ebc62f on wstranger:master into ** on python-diamond:master**.

@coveralls
Copy link

coveralls commented May 10, 2017

Coverage Status

Changes Unknown when pulling e6d03ba on wstranger:master into ** on python-diamond:master**.

config.update({
'path': 'postgres',
'host': 'localhost',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently this PR is a breaking change. However, if you keep this config option, add it to the instance list in the collect method if it isn't already in it, and then exclude host from falling back to the default in _get_config, then it would not be a breaking change.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I already added it in my last commit in the collect method. Same way as pgbouncer collector.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see the host config option being used in e6d03ba

@coveralls
Copy link

coveralls commented May 11, 2017

Coverage Status

Changes Unknown when pulling b6eb57d on wstranger:master into ** on python-diamond:master**.

@kormoc
Copy link
Contributor

kormoc commented May 12, 2017

Can you quickly give a run down on why the multiple collector core code doesn't work for this? Thanks!

@wstranger
Copy link
Author

I didn't find any suitable implementation in core and use pgbouncer collector code as reference.

@kormoc
Copy link
Contributor

kormoc commented May 12, 2017

Hrm. I can't find it documented in the docs :(

Basically, core allows you to spin up multiple instances of any collector via multiple config files of the pattern
/etc/diamond/collectors/ExampleCollector - foobar.conf. Every individual config file will be a entirely independent collector process and can be configured as such.

@wstranger
Copy link
Author

wstranger commented May 12, 2017

Ok. At the doc http://diamond.readthedocs.io/en/latest/Getting-Started/Configuration/ - we can see next -

The configuration file name should match the name of the diamond collector class.

Hrm. Nothing about wildcards.
Slony, Redis, Elastic, pgbouncer collectors use same way as this one.
We are planning to use diamond in docker environment and I'm afraid it is not convenient to manage over 100 configuration files.
Thank you.

@shortdudey123
Copy link
Member

PR LGTM now
However, i will defer to @kormoc on merging or not


# HACK: setting default with subcategory messes up merging of configs,
# so we only set the default if one wasn't provided.
if not instances:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't this code never run since you defined a default for instances that is not None?

# HACK: setting default with subcategory messes up merging of configs,
# so we only set the default if one wasn't provided.
if not instances:
instances = {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to set to self.instances since you use it in _get_fig()

@@ -14,43 +14,60 @@ class TestPostgresqlCollector(CollectorTestCase):
def setUp(self, allowed_names=None):
if not allowed_names:
allowed_names = []

default_config = get_collector_config('PostgresqlCollector', {})
self.default_collector = PostgresqlCollector(default_config, None)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't look like you ever used this. Can you add tests using it to test the default config behavior?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants