-
-
Notifications
You must be signed in to change notification settings - Fork 33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Metrics with default jvm.json not working #53
Comments
@myllynen could you send me a copy of /var/lib/pcp/tmp/mmv/KeyboardReader? Thanks. |
I doublechecked everything else and I can't see anything else obviously wrong; this is with: java-1.8.0-openjdk-headless-1.8.0.161-0.b14.el7_4.x86_64 and in addition to the above steps I've made /var/lib/pcp/tmp/mmv accessible for the user, otherwise no Java / PCP configuration changes locally, SELinux permissive, under /etc/parfait only the jvm.json is present and it is identical to current git master version. |
I've now tested this on a few more systems and it more and more looks like this is not a local issue, it can be observed also on freshly installed Fedora 27 VM (either with PCP 4.0 or git master), the only system where this works is RHEL 7.4 with: java-1.8.0-openjdk-headless-1.8.0.161-0.b14.el7_4.x86_64 |
@myllynen I think this may be a pmrep/python/fetchgroup/pmdammv issue related to handling instance domains in a slightly unusual state. pminfo -df mmv shows correct output using the KeyboardReader.gz MMV file. I suspect metrics like java.memorypool.tenured, java.memorypool.survivor etc have no values on this JVM version. For reference, you can examine the generated KeyboardReader file directly (taking java/parfait totally out of the picture), using /var/lib/pcp/pmdas/mmv/mmvdump - this shows a valid MMV files with the usual set of default Java metrics. You can use qa/src/mmv_poke from the pcp repo to modify the file (esp. the PID) such that it becomes active from pmdammv(1) perspective, and the metrics can then be inspected directly using pminfo and other client tools (again, this is just to isolate things from the java/Parfait aspects, which don't appear to be at issue here). |
Hmm, ok so it seems like there are two unrelated issues here:
And with all versions since then (this is from Fedora 27 with PCP 4.0):
IOW, looks like things are consistent in recent releases how pmval/pmrep/etc handle those metrics.
Thanks. |
The root cause of the underlying indom issue here is now understood. It's a logic error in pmdammv, fix will be in pcp-4.1.0 and later versions of PCP. |
Several people (marko, lzap, tallpaul) have reported this one, finally got to the bottom of it. The symptoms are "Unknown or illegal instance domain identifier" errors on indom lookups, sometimes. Root cause was a logic error in pmdammv indom setup code incorrectly overwriting count and offset local variables while parsing mappings. To exercise the fix I've modernized qa/src/indom.c and used it in new test qa/1422 to tickle the problem using a canned MMV mapping which is known to expose it. The original bug report (against Parfait) is this one: performancecopilot/parfait#53
With Java 8 on RHEL 7 and Parfait agent compiled from git master:
Only a very few metrics (e.g., mmv.KeyboardReader.java.jvm.classloader.loaded.*) are actually working.
The text was updated successfully, but these errors were encountered: