Pymongo3 + some fixes #19
Conversation
Fix the broken get live logs Improve the MongoDB collection indexes
| self.db = self.conn[self.database] | ||
| self.db[self.collection].ensure_index([('host_name', pymongo.ASCENDING), ('time', pymongo.ASCENDING), ('lineno', pymongo.ASCENDING)], name='logs_idx') | ||
| self.db[self.collection].ensure_index([('time', pymongo.ASCENDING), ('lineno', pymongo.ASCENDING)], name='time_1_lineno_1') |
There was a problem hiding this comment.
Hi @mohierf !
If you remove this index, you will get an error like this:
OperationFailure: Executor error during find command: OperationFailed: Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit.
This is due the following code, that use exactly that kind of sort by timestamp and lineno:
File "shinken/modules/logstore-mongodb/module.py", line 312, in get_live_data_log
dbresult = [Logline([(c,) for c in columns], [x[col] for col in columns]) for x in self.db[self.collection].find(filter_element).sort([(u'time', pymongo.ASCENDING), (u'lineno', pymongo.ASCENDING)])]
There was a problem hiding this comment.
Ohhh, I've just realized that you have changed those lines too :)
There was a problem hiding this comment.
Hi David, I just wanted to give a try :) It looks okay despite the tests are not yet ok.
mohierf
force-pushed
the
pymongo3
branch
2 times, most recently
from
a62c679
to
699afd3
Compare
Scheduler broks is an array not a dict
Codecov Report
@@ Coverage Diff @@
## master #19 +/- ##
=========================================
Coverage ? 78.49%
=========================================
Files ? 1
Lines ? 265
Branches ? 0
=========================================
Hits ? 208
Misses ? 57
Partials ? 0
Continue to review full report at Codecov.
|
Get the modifications from @Azef1 and fix the remaining problems