-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathpcpcheat.tex
279 lines (236 loc) · 11.6 KB
/
pcpcheat.tex
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% PCP Cheat Sheet,
% Christian Horn <[email protected]>
%
% based on:
% https://ctan.math.illinois.edu/macros/latex/contrib/tcolorbox/tcolorbox-tutorial-poster.pdf
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\documentclass[12pt]{article}
\usepackage[a3paper,landscape]{geometry}
\usepackage{lipsum}
\usepackage{lmodern}
\usepackage{enumerate}
\usepackage{hyperref}
\usepackage{verbatim}
\usepackage[poster]{tcolorbox}
\tcbuselibrary{listings}
\pagestyle{empty}
\begin{document}
\begin{tcbposter}[
coverage = {
spread,
interior style={top color=yellow,bottom color=yellow!50!red},
watermark text={PCP cheat sheet},
watermark color=yellow,
},
poster = {showframe=false,columns=4,rows=5},
boxes = {
enhanced standard jigsaw,sharp corners=downhill,arc=3mm,boxrule=1mm,
colback=white,opacityback=0.75,colframe=blue,
title style={left color=black,right color=cyan},
fonttitle=\bfseries\Large\scshape
}
]
% Here, we insert the poster content later
\posterbox[blankest,interior engine=path,height=3cm,
halign=center,valign=center,fontupper=\bfseries\large,colupper=red!25!black
]{name=title,column=1,span=3,below=top}{
\resizebox{18cm}{!}{\bfseries\Huge Performance Co-Pilot cheat sheet}\\[3mm]
\href{https://github.com/christianhorn/pcp-cheat-sheet}{https://github.com/christianhorn/pcp-cheat-sheet}
% Christian Horn \textless [email protected]\textgreater
}
%----------------------------------------------------------------------------------
% Basics
%----------------------------------------------------------------------------------
\posterbox[adjusted title=PCP basics]{name=basics,below=title}{
% sequence=1 between title and bottom}{
\textbf{Installation}\\
Package \textbf{zero-conf} pulls in dependencies, starts daemons, starts
archiving of a default set of metrics. On RPM based distros (RHEL,
Fedora, CentOS etc.): \\
% \textit{ \hspace*{0.5cm}\# dnf -y install pcp-zeroconf \\ } \\
\hspace*{0.5cm}\# dnf -y install pcp-zeroconf \\
Installation via Ansible playbook: \\
\hspace*{0.5cm}\# linux-system-roles.metrics \\
Verify pcp installation: \\
\hspace*{0.5cm}\# pcp \\
\hspace*{0.5cm}\# systemctl status pmcd pmlogger \\
\textbf{Important tools}\\
Tools from package 'pcp-system-tools' can be used with either
the running pmcd, or PCP archive files:
\begin{itemize}
\item pcp atop
\item pcp free
\item pcp iostat
\item pcp dstat
\end{itemize}
\textbf{Important Paths}\\
\hspace*{0.5cm}/var/lib/pcp/config/ \\
\hspace*{0.5cm}/var/log/pcp/ \\
\hspace*{0.5cm}/etc/pcp/ \\
\textbf{Working with metrics}\\
Which metrics are offered by the running pmcd? \\
\hspace*{0.5cm}\# pminfo \\
Which metrics related to cpu are available? \\
\hspace*{0.5cm}\# pminfo $\vert$ grep cpu \\
\vspace{0.0em} % When there are two boxes, some whitespace may need to be added if the one on the right has more content
}
%----------------------------------------------------------------------------------
% Links
%----------------------------------------------------------------------------------
\posterbox[adjusted title=Links]{name=links,column=2,span=2,above=bottom}{
\begin{enumerate}[{[1]}]
% \item\label{litA} Important Authors, \textit{Important Title}
\item \href{https://access.redhat.com/articles/1145953}{kbase: Index of (PCP) articles, solutions, tutorials, white papers}
\item Ansible:\href{https://github.com/linux-system-roles/metrics}{https://github.com/linux-system-roles/metrics} \href{https://github.com/performancecopilot/ansible-pcp}{https://github.com/performancecopilot/ansible-pcp}
\item \href{https://pcp.io}{Performance Co-Pilot site}
\item Articles: \href{https://www.redhat.com/en/blog/how-solve-performance-mysteries-performance-co-pilot-red-hat-enterprise-linux}{Solve performance mysteries with PCP} / \href{https://www.redhat.com/en/blog/examining-container-performance-rhel-8-pcp-and-pdma-podman}{PCP and podman} / \href{https://www.redhat.com/en/blog/implementing-dstat-performance-co-pilot}{PCP and dstat}
\end{enumerate}
}
%----------------------------------------------------------------------------------
% Remote collection
%----------------------------------------------------------------------------------
\posterbox[adjusted title=Remote collection]
{name=remotecollection,column=3,below=title}{
\textbf{Install PCP on clients} \\
Setup client systems to offer metrics via pmcd: install pcp, open packet
based firewall, enable remote access in pmcd: \\
\footnotesize{
\hspace*{0.5cm}\# yum -y install pcp \\
\hspace*{0.5cm}\# firewall-cmd -\/-permanent -\/-zone=public \textbackslash \\
\hspace*{1.1cm} -\/-add-port=44321/tcp \\
\hspace*{0.5cm}\# firewall-cmd -\/-reload \\
\hspace*{0.5cm}\# CONFIG=/etc/sysconfig/pmcd \\
\hspace*{0.5cm}\# if grep -q PMCD\_LOCAL \$CONFIG; then \textbackslash \\
\hspace*{1.1cm} sed -ie 's,PMCD\_LOCAL.*,PMCD\_LOCAL=0,' \textbackslash \\
\hspace*{1.1cm} /etc/sysconfig/pmcd \textbackslash \\
\hspace*{0.9cm} else \\
\hspace*{1.1cm} echo 'PMCD\_LOCAL=0' >>/etc/sysconfig/pmcd \\
\hspace*{0.9cm} fi \\
\hspace*{0.5cm}\# grep PMCD\_LOCAL /etc/sysconfig/pmcd \\
\hspace*{0.5cm}\# service pmcd restart \\
\hspace*{0.5cm}\# chkconfig pmcd on \\
}
\textbf{Install PCP on collector system} \\
On the collector, we install pcp-zeroconf which also sets up logging to archive files.
We then set variable CLIENT to the clients name, create a config- and controlfile,
and notify pmlogger of the changes.
\footnotesize{
\hspace*{0.5cm}\# yum -y install pcp-zeroconf \\
\hspace*{0.5cm}\# CLIENT=rhel7u8a \\
\hspace*{0.5cm}\# /usr/libexec/pcp/bin/pmlogconf \textbackslash \\
\hspace*{1.1cm} /var/lib/pcp/config/pmlogger/config.\$CLIENT \\
\hspace*{0.5cm}\# echo "\$CLIENT.local n n " \textbackslash \\
\hspace*{1.1cm} PCP\_LOG\_DIR/pmlogger/\$CLIENT.local" \textbackslash \\
\hspace*{1.1cm} " -r -T30d -c config.\$CLIENT" \textbackslash \\
\hspace*{1.1cm} \textgreater/etc/pcp/pmlogger/control.d/\$CLIENT \\
\hspace*{0.5cm}\# /usr/libexec/pcp/bin/pmlogger\_check
}
% \center{
% \includegraphics[width=20em]{pics/s05_browsers_pcp_servers_overview_componentsb.png}
% }
% pmlogconf - configures configfile, including the pmdas you run on the system
}
%----------------------------------------------------------------------------------
% Live metrics
%----------------------------------------------------------------------------------
\posterbox[adjusted title=Live metrics]
{name=livemetrics,column=3,between=remotecollection and links}{
Accessing kernel.all.load metric of local system: \\
\hspace*{0.5cm}\# pmrep -p kernel.all.load \\
Get live metrics from 2 remote systems: \\
\hspace*{0.5cm}\# pmstat -h local: -h remotesystem1
}
%----------------------------------------------------------------------------------
% Archive files
%----------------------------------------------------------------------------------
\posterbox[adjusted title=Archive files]
{name=archivefiles,column=4,below=top}{
% {name=archivefiles,sequence=3 between title and links then
% 4 between top and bottom}{
\textbf{Basics} \\
Which archive is pmlogger logging into? \\
\hspace*{0.5cm}\# pcp \\
Set a variable to current archive, and evaluate how many metrics
are logged in the archive: \\
\hspace*{0.5cm}\# cd /var/log/pcp/pmlogger/\textless hostname\textgreater \\
\hspace*{0.5cm}\# pminfo -a \textless archivename\textgreater $\vert$ wc -l \\
% TODO: more verbose syntax here
Have pmdiff point out 'significant peaks' in archives: \\
\hspace*{0.5cm}\# pmdiff -a \textless archivename\textgreater \\
Merge all archive files ending in .0 into a single archive: \\
\hspace*{0.5cm}\# pmlogextract *.0 single-archive-name \\
\textbf{Accessing metrics} \\
Which time span is covered by the archive file, time displayed
for the timezone you are running while analyzing? \\
\hspace*{0.5cm}\# pmdumplog -L \textless archive\textgreater \\
Which time span is covered by the archive file, time displayed \\
with the timezone of the system which recorded the archive file? \\
\hspace*{0.5cm}\# pmdumplog -L \textless archive\textgreater -z \\
Most basic access to metrics: \\
\hspace*{0.5cm}\# pmstat -t 1 \\
\hspace*{0.5cm}\# pmstat -a \textless archivename\textgreater \textless metric\textgreater \\
\hspace*{0.5cm}\# pmrep -a \textless archivename\textgreater \textless metric\textgreater \\
\hspace*{0.5cm}\# pcp -a 20180831.11.31 -\/-origin @1pm \textbackslash \\
\hspace*{1.0cm} dstat -\/-time -\/-disk -\/-mem 60sec 10 \\
\hspace*{0.5cm}\# pmrep -a 20220128 kernel.cpu -p -t 300 \textbackslash \\
\hspace*{1.0cm} -\/-hostzone -S @19:30 \\
Show I/O stats for sda, starting at Jul 19th \\
\hspace*{0.5cm}\# pmiostat -z -a \textless archive\textgreater -x t \textbackslash \\
\hspace*{1.0cm} -t 10m -S'@Jul 19 19:55:00' \textbackslash \\
\hspace*{1.0cm} -s 10 -P0 -R 'sda\$' \\
\textbf{Graphical access} \\
\hspace*{0.5cm}\# pmchart \\
\hspace*{0.5cm}\# dnf -y install pcp2pdf; \textbackslash \\
\hspace*{1.0cm} pcp2pdf -a \textless archivename\textgreater \\
}
%----------------------------------------------------------------------------------
% pmdas
%----------------------------------------------------------------------------------
\posterbox[adjusted title=pmdas]
{name=pmdas,column=2,below=title}{
\textbf{PMDA installation} \\
PMDA's are code pieces capable of reading metrics from their area
like sensors, database, and so on.
PMDA's can be searched as packages and installed, for example on yum4/dnf
distros: \\
\hspace*{0.5cm}\# dnf search pcp-pmda \\
\hspace*{0.5cm}\# dnf install -y pcp-pmda-lmsensors \\
\hspace*{0.5cm}\# cd /var/lib/pcp/pmdas/lmsensors \\
\hspace*{0.5cm}\# less README \\
\hspace*{0.5cm}\# ./Install
}
%----------------------------------------------------------------------------------
% pmie
%----------------------------------------------------------------------------------
\posterbox[adjusted title=pmie]
{name=pmie,column=1,below=basics}{
pmie, performance metrics interference engine, can react on defined
metric states: send email on high load, and so on. \\
\hspace*{0.5cm}\# pmie -\/-verbose -\/-timestamp -\/-interval 1 \\
\hspace*{0.5cm}\# /etc/pcp/pmie/config.default \\
\hspace*{0.5cm}\# pmie -\/-archive 20200512 -\/-config \textless rules\textgreater
}
%----------------------------------------------------------------------------------
% pmdiff
%----------------------------------------------------------------------------------
\posterbox[adjusted title=Anomaly search in archives]
{name=diff,column=2,between=pmdas and links}{
Which metrics are remarkably different in a certain timeframe?
Example: we had I/O problems from 2am to 3am: \\
\hspace*{0.5cm}\# cat cull \\
\hspace*{1.2cm} rsyslog. \\
\hspace*{0.5cm}\# pmdiff -z -X ./cull -q 10 \textbackslash \\
\hspace*{1.2cm} -\/-start @10:00 -\/-finish @10:30 \textbackslash \\
\hspace*{1.2cm} -\/-begin @12:00 -\/-end @12:30 \textbackslash \\
\hspace*{1.2cm} ./archives/20120512 $\vert$ less \\
What are the top 5 cpu and memory hogs? \\
\hspace*{0.5cm}\# pmrep proc.hog.cpu, proc.memory.rss \textbackslash \\
\hspace*{1.2cm} -J 5 -1 -g -b MB \\
What are the current PIDs, and how much rss uses process with pid 75? \\
\hspace*{0.5cm}\# pmrep proc.smaps.rss -g $\vert$ less \\
\hspace*{0.5cm}\# pmrep proc.smaps.rss -g -i '.*75.*'
}
\end{tcbposter}
\end{document}