Get Expert Help

-

If you need assistance, visit the community forum for comprehensive and free database knowledge.

+

If you need assistance, visit the community forum for comprehensive and free database knowledge.

Contact our Percona Database Experts for professional support and services.

{# Add the last updated timestamp from git commits using gitstamp extension #} diff --git a/docs/percona-toolkit.pod b/docs/percona-toolkit.pod index 0f0db520f..21f6177e5 100644 --- a/docs/percona-toolkit.pod +++ b/docs/percona-toolkit.pod @@ -406,7 +406,7 @@ databases while running. =item F Defaults file for the MySQL client library (the C client library used by -DBD::mysql, I). All tools all read the +DBD::mysql, I). All tools read the C<[client]> section within the defaults file. If you omit this, the standard defaults files will be read in the usual order. "Standard" varies from system to system, because the filenames to read are compiled into the client library. @@ -441,7 +441,7 @@ lead to weird situations, like the server allowing LOCAL INFILE, but the client throwing exceptions if it's used. However, as long as the server allows LOAD DATA, clients can easily -re-enable it; see L +re-enable it; see L and L. This option does exactly that. @@ -539,7 +539,7 @@ Include the following information in your bug report: =item * Tool C<--version> -=item * MySQL version of all servers involved +=item * MySQL, MongoDB, or PostgreSQL version of all servers involved =item * Output from the tool including STDERR @@ -603,6 +603,6 @@ Place, Suite 330, Boston, MA 02111-1307 USA. =head1 VERSION -Percona Toolkit v3.6.0 released 2024-05-25 +Percona Toolkit v3.6.0 released 2024-06-12 =cut diff --git a/docs/pt-galera-log-explainer.rst b/docs/pt-galera-log-explainer.rst deleted file mode 100644 index 3b3cda00f..000000000 --- a/docs/pt-galera-log-explainer.rst +++ /dev/null @@ -1,246 +0,0 @@ -.. _pt-galera-log-explainer: - -================================== -:program:`pt-galera-log-explainer` -================================== - -Filter, aggregate and summarize multiple galera logs together. -This is a toolbox to help navigating Galera logs. - -Usage -===== - -.. code-block:: bash - - pt-galera-log-explainer [--since=] [--until=] [-vv] [--merge-by-directory] [--pxc-operator] - - -Commands available -================== - -list -~~~~ - -.. code-block:: bash - - pt-galera-log-explainer [flags] list { --all | [--states] [--views] [--events] [--sst] [--applicative] } - -List key events in chronological order from any number of nodes (sst, view changes, general errors, maintenance operations) -It will aggregates logs together by identifying them using node names, IPs and internal Galera identifiers. - - - -It can be from a single node: - -.. code-block:: bash - - pt-galera-log-explainer list --all --since 2023-01-05T03:24:26.000000Z /var/log/mysql/*.log - -or from multiple nodes. - -.. code-block:: bash - - pt-galera-log-explainer list --all *.log - -You can filter by type of events - -.. code-block:: bash - - pt-galera-log-explainer list --sst --views *.log - -whois -~~~~~ -Find out information about nodes, using any type of information - -.. code-block:: bash - - pt-galera-log-explainer [flags] whois [--json] [--type { nodename | ip | uuid | auto }] - - -.. code-block:: bash - - pt-galera-log-explainer whois '218469b2' mysql.log - pt-galera-log-explainer whois '172.17.0.3' mysql.log - pt-galera-log-explainer whois 'galera-node2' mysql.log - - -conflicts -~~~~~~~~~ - -List every replication failure votes (Galera 4) - -.. code-block:: bash - - pt-galera-log-explainer conflicts [--json|--yaml] *.log - -ctx -~~~ - -Get the tool crafted context for a single log. -It will contain everything the tool extracted from the log file: version, sst information, known uuid-ip-nodename mappings, ... - -.. code-block:: bash - - pt-galera-log-explainer ctx mysql.log - -regex-list -~~~~~~~~~~ - -Will print every implemented regexes: -* regex: the regex that will be used against the log files -* internalRegex: the golang regex that will be used to extract piece of information -* type: the regex group it belong to -* verbosity: the required level of verbosity to which it will be printed - -.. code-block:: bash - - pt-galera-log-explainer regex-list - -Available flags -~~~~~~~~~~~~~~~ - -``-h``, ``--help`` - Show help and exit. - -``--no-color`` - Remove every color special characters - -``--since`` - Only list events after this date. It will affect the regex applied to the logs. - Format: 2023-01-23T03:53:40Z (RFC3339) - -``--until`` - Only list events before this date. This is only implemented in the tool loop, it does not alter regexes. - Format: 2023-01-23T03:53:40Z (RFC3339) - -``--merge-by-directory`` - Instead of relying on extracted information, logs will be merged by their base directory - It is useful when logs are very sparse and already organized by nodes. - -``--skip-merge`` - Disable the ability to merge log files together. Can be used when every nodes have the same ``wsrep_node_name`` - -``-v``, ``--verbosity`` - ``-v``: display in the timeline every mysql info the tool used - ``-vv``: internal tool debug - -``--pxc-operator`` - Analyze logs from Percona PXC operator. Operator logs should be automatically detected (see ``--skip-operator-detection``). - It will prevent logs from being merged together, add operator specific regexes, and fine-tune regexes for logs taken from ``pt-k8s-debug-collector``. - Off by default because it negatively impacts performance for non-k8s setups. - -``--skip-operator-detection`` - Disable automatic detection of PXC operator logs. When detected, a message will be shown. - Detection is done using a prefix regex. - -``--exclude-regexes`` - Remove regexes from analysis. Use ``pt-galera-log-explainer regex-list | jq .`` to have the list - -``--grep-cmd`` - grep v3 binary command path. For Darwin systems, it could need to be set to ``ggrep`` - Default: ``grep`` - -``--version`` - Show version and exit. - -``--custom-regexes`` - Add custom regexes, printed in magenta. Format: (golang regex string)=[optional static message to display]. - If the static message is left empty, the captured string will be printed instead. Custom regexes are separated using semi-colon. - Example: ``--custom-regexes="Page cleaner took [0-9]*ms to flush [0-9]* pages=;doesn't recommend.*pxc_strict_mode=unsafe query used"`` - - -Example outputs -=============== - -.. code-block:: bash - - $ pt-galera-log-explainer list --all --no-color --since=2023-03-12T19:41:28.493046Z --until=2023-03-12T19:44:59.855491Z tests/logs/upgrade/* - identifier 172.17.0.2 node2 tests/logs/upgrade/node3.log - current path tests/logs/upgrade/node1.log tests/logs/upgrade/node2.log tests/logs/upgrade/node3.log - last known ip 172.17.0.2 - last known name node2 - mysql version 8.0.28 - - 2023-03-12T19:41:28.493046Z starting(8.0.28) | | - 2023-03-12T19:41:28.500789Z started(cluster) | | - 2023-03-12T19:43:17.630191Z | node3 joined | - 2023-03-12T19:43:17.630208Z node3 joined | | - 2023-03-12T19:43:17.630221Z node2 joined | | - 2023-03-12T19:43:17.630243Z | node1 joined | - 2023-03-12T19:43:17.634138Z | | node2 joined - 2023-03-12T19:43:17.634229Z | | node1 joined - 2023-03-12T19:43:17.643210Z | PRIMARY(n=3) | - 2023-03-12T19:43:17.648163Z | | PRIMARY(n=3) - 2023-03-12T19:43:18.130088Z CLOSED -> OPEN | | - 2023-03-12T19:43:18.130230Z PRIMARY(n=3) | | - 2023-03-12T19:43:18.130916Z OPEN -> PRIMARY | | - 2023-03-12T19:43:18.904410Z will receive IST(seqno:178226792) | | - 2023-03-12T19:43:18.913328Z | | node1 cannot find donor - 2023-03-12T19:43:18.913429Z node1 cannot find donor | | - 2023-03-12T19:43:18.913565Z | node1 cannot find donor | - 2023-03-12T19:43:19.914122Z | | node1 cannot find donor - 2023-03-12T19:43:19.914259Z node1 cannot find donor | | - 2023-03-12T19:43:19.914362Z | node1 cannot find donor | - 2023-03-12T19:43:20.914957Z | | (repeated x97)node1 cannot find donor - 2023-03-12T19:43:20.915143Z (repeated x97)node1 cannot find donor | | - 2023-03-12T19:43:20.915262Z | (repeated x97)node1 cannot find donor | - 2023-03-12T19:44:58.999603Z | | node1 cannot find donor - 2023-03-12T19:44:58.999791Z node1 cannot find donor | | - 2023-03-12T19:44:58.999891Z | node1 cannot find donor | - 2023-03-12T19:44:59.817822Z timeout from donor in gtid/keyring stage | | - 2023-03-12T19:44:59.839692Z SST error | | - 2023-03-12T19:44:59.840669Z | | node2 joined - 2023-03-12T19:44:59.840745Z | | node1 left - 2023-03-12T19:44:59.840933Z | node3 joined | - 2023-03-12T19:44:59.841034Z | node1 left | - 2023-03-12T19:44:59.841189Z NON-PRIMARY(n=1) | | - 2023-03-12T19:44:59.841292Z PRIMARY -> OPEN | | - 2023-03-12T19:44:59.841352Z OPEN -> CLOSED | | - 2023-03-12T19:44:59.841515Z terminated | | - 2023-03-12T19:44:59.841529Z former SST cancelled | | - 2023-03-12T19:44:59.848349Z | | node1 left - 2023-03-12T19:44:59.848409Z | | PRIMARY(n=2) - 2023-03-12T19:44:59.855443Z | node1 left | - 2023-03-12T19:44:59.855491Z | PRIMARY(n=2) | - - $ pt-galera-log-explainer whois 172.17.0.2 --no-color tests/logs/upgrade/* - ip: - └── 172.17.0.2 - ├── nodename: - │ └── node1 (2023-03-12 19:35:07.644683 +0000 UTC) - │ - └── uuid: - ├── 1d3ea8f5 (2023-03-12 07:24:13.789261 +0000 UTC) - ├── 54ab931e (2023-03-12 07:43:08.563339 +0000 UTC) - ├── fecde235 (2023-03-12 08:46:48.963504 +0000 UTC) - ├── a07872e1 (2023-03-12 08:49:41.206124 +0000 UTC) - ├── 60da0bf9-aa9c (2023-03-12 12:29:48.873397 +0000 UTC) - ├── 35b62086-902c (2023-03-12 13:04:23.979636 +0000 UTC) - ├── ca2c2a5f-a82a (2023-03-12 19:35:05.878879 +0000 UTC) - └── eefb9c8a-b69a (2023-03-12 19:43:17.133756 +0000 UTC) - - - -Requirements -============ - -grep, version 3 -On Darwin based OS, grep is only version 2 due to license limitations. --grep-cmd can be used to point the correct grep binary, usually ggrep - - -Compatibility -============= - -* Percona XtraDB Cluster: 5.5 to 8.0 -* MariaDB Galera Cluster: 10.0 to 10.6 -* logs from PXC operator pods (error.log, recovery.log, post.processing.log) - -Known issues -============ - -* Nodes sharing the same ip, or nodes with identical names are not supported -* Sparse files identification can be missed, resulting in many columns displayed. ``--merge-by-directory`` can be used, but files need to be organized already in separate directories - This is mainly when the log file does not contain enough information. -* Some information will seems missed. Depending on the case, it may be simply unimplemented yet, or it was disabled later because it was found to be unreliable (node index numbers are not reliable for example) -* Columns width are sometimes too large to be easily readable. This usually happens when printing SST events with long node names -* When some display corner-cases seems broken (events not deduplicated, ...), it is because of extra hidden internal events. diff --git a/docs/pt-galera-log-explainer.rst b/docs/pt-galera-log-explainer.rst new file mode 120000 index 000000000..cd10655bc --- /dev/null +++ b/docs/pt-galera-log-explainer.rst @@ -0,0 +1 @@ +../src/go/pt-galera-log-explainer/README.rst \ No newline at end of file diff --git a/docs/pt-k8s-debug-collector.rst b/docs/pt-k8s-debug-collector.rst deleted file mode 100644 index 6b72ad16f..000000000 --- a/docs/pt-k8s-debug-collector.rst +++ /dev/null @@ -1,191 +0,0 @@ -.. _pt-k8s-debug-collector: - -================================== -:program:`pt-k8s-debug-collector` -================================== - -Collects debug data (logs, resource statuses etc.) from a k8s/OpenShift cluster. Data is packed into the ``cluster-dump.tar.gz`` archive in the current working directory. - -Data that will be collected -=========================== - -.. code-block:: bash - - "pods", - "replicasets", - "deployments", - "statefulsets", - "replicationcontrollers", - "events", - "configmaps", - "cronjobs", - "jobs", - "podsecuritypolicies", - "poddisruptionbudgets", - "perconaxtradbbackups", - "perconaxtradbclusterbackups", - "perconaxtradbclusterrestores", - "perconaxtradbclusters", - "clusterrolebindings", - "clusterroles", - "rolebindings", - "roles", - "storageclasses", - "persistentvolumeclaims", - "persistentvolumes", - "modes", - "your-custom-resource" (depends on 'resource' flag) - -Data, collected for PXC -~~~~~~~~~~~~~~~~~~~~~~~ - -.. code-block:: bash - - "perconaxtradbbackups", - "perconaxtradbclusterbackups", - "perconaxtradbclusterrestores", - "perconaxtradbclusters" - -Summary, collected for PXC (available in file summary.txt) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. code-block:: bash - - "pt-mysql-summary" - -Individual files, collected for PXC -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. code-block:: bash - - "var/lib/mysql/mysqld-error.log", - "var/lib/mysql/innobackup.backup.log", - "var/lib/mysql/innobackup.move.log", - "var/lib/mysql/innobackup.prepare.log", - "var/lib/mysql/grastate.dat", - "var/lib/mysql/gvwstate.dat", - "var/lib/mysql/mysqld.post.processing.log", - "var/lib/mysql/auto.cnf" - -Data, collected for MySQL -~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. code-block:: bash - - "perconaservermysqlbackups", - "perconaservermysqlrestores", - "perconaservermysqls" - -Summary, collected for MySQL (available in file summary.txt) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. code-block:: bash - - "pt-mysql-summary" - -Data, collected for MongoDB -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. code-block:: bash - - "perconaservermongodbbackups", - "perconaservermongodbrestores", - "perconaservermongodbs" - -Summary, collected for MongoDB (available in file summary.txt) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. code-block:: bash - - "pt-mongodb-summary" - -Data, collected for PostgreSQL -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. code-block:: bash - - "perconapgclusters", - "pgclusters", - "pgpolicies", - "pgreplicas", - "pgtasks" - -Summary, collected for PostgreSQL (available in file summary.txt) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. code-block:: bash - - "pg_gather" - -Usage -===== - -``pt-k8s-debug-collector `` - -Supported Flags -================ - -``--resource`` - -Targeted custom resource name. Supported values: - -* ``pxc`` - PXC - -* ``psmdb`` - MongoDB - -* ``pg`` - PostgreSQL Operator v1 (deprecated) - -* ``pgv2`` - PostgreSQL Operator v2 - -* ``ps`` - MySQL - -* ``none`` - Collect only general Kubernetes data, do not collect anything specific to the particular operator). - -* ``auto`` - Auto-detect custom resource - -Default: ``auto`` - -``--namespace`` - -Targeted namespace. By default data will be collected from all namespaces - -``--cluster`` - -Targeted cluster. By default data from all available clusters to be collected - -``--kubeconfig`` - -Path to kubeconfig. Default configuration be used if none specified - -``--forwardport`` - -Port to use when collecting database-specific summaries. By default, 3306 will be used for PXC and MySQL, 27017 for MongoDB, and 5432 for PostgreSQL - -``--version`` - -Print version info - -Requirements -============ - -- Installed, configured, and available in PATH ``kubectl`` -- Installed, configured, and available in PATH ``pt-mysql-summary`` for PXC and MySQL -- Installed, configured, and available in PATH ``mysql`` for PXC and MySQL -- Installed, configured, and available in PATH ``pt-mongodb-summary`` for MongoDB -- Installed, configured, and available in PATH ``psql`` for PostgreSQL - -Known Issues -============ - -On Kubernetes 1.21 - 1.24 warning is printed: - -.. code-block:: bash - - 2022/12/15 17:43:16 Error: get resource podsecuritypolicies in namespace default: error: , stderr: Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+ - , stdout: apiVersion: v1 - items: [] - kind: List - metadata: - resourceVersion: "" - -This warning is harmless and does not affect data collection. We will remove podsecuritypolicies once everyone upgrade to Kubernetes 1.25 or newer. Before that we advise to ignore this warning. diff --git a/docs/pt-k8s-debug-collector.rst b/docs/pt-k8s-debug-collector.rst new file mode 120000 index 000000000..23edde705 --- /dev/null +++ b/docs/pt-k8s-debug-collector.rst @@ -0,0 +1 @@ +../src/go/pt-k8s-debug-collector/README.rst \ No newline at end of file diff --git a/docs/pt-mongodb-index-check.rst b/docs/pt-mongodb-index-check.rst deleted file mode 100644 index 3e31f0802..000000000 --- a/docs/pt-mongodb-index-check.rst +++ /dev/null @@ -1,72 +0,0 @@ -.. _pt-mongodb-index-check: - -================================= -:program:`pt-mongodb-index-check` -================================= - -Performs checks on MongoDB indexes. - -Checks available -================ - -Duplicated indexes -~~~~~~~~~~~~~~~~~~ - -Check for indexes that are the prefix of other indexes. For example if we have these 2 indexes - -.. code-block:: javascript - - db.getSiblingDB("testdb").test_col.createIndex({"f1": 1, "f2": -1, "f3": 1, "f4": 1}, {"name": "idx_01"}); - db.getSiblingDB("testdb").test_col.createIndex({"f1": 1, "f2": -1, "f3": 1}, {"name": "idx_02"}); - - -The index ``idx_02`` is the prefix of ``idx_01`` because it has the same -keys in the same order so, ``idx_02`` can be dropped. - -Unused indexes. -~~~~~~~~~~~~~~~ - -This check gets the ``$indexstats`` for all indexes and reports those -having ``accesses.ops`` = 0. - -Usage -===== - -Run the program as ``pt-mongodb-index-check [flags]`` - -Available commands -~~~~~~~~~~~~~~~~~~ - -================ ================================== -Command Description -================ ================================== -check-duplicated Run checks for duplicated indexes. -check-unused Run check for unused indexes. -check-all Run all checks -================ ================================== - -Available flags -~~~~~~~~~~~~~~~ - -+----------------------------+----------------------------------------+ -| Flag | Description | -+============================+========================================+ -| –all-databases | Check in all databases excluding | -| | system dbs. | -+----------------------------+----------------------------------------+ -| –databases=DATABASES,… | Comma separated list of databases to | -| | check. | -+----------------------------+----------------------------------------+ -| –all-collections | Check in all collections in the | -| | selected databases. | -+----------------------------+----------------------------------------+ -| –collections=COLLECTIONS,… | Comma separated list of collections to | -| | check. | -+----------------------------+----------------------------------------+ -| –mongodb.uri= | Connection URI | -+----------------------------+----------------------------------------+ -| –json | Show output as JSON | -+----------------------------+----------------------------------------+ -| –version | Show version information | -+----------------------------+----------------------------------------+ - diff --git a/docs/pt-mongodb-index-check.rst b/docs/pt-mongodb-index-check.rst new file mode 120000 index 000000000..7eef47ec3 --- /dev/null +++ b/docs/pt-mongodb-index-check.rst @@ -0,0 +1 @@ +../src/go/pt-mongodb-index-check/README.rst \ No newline at end of file diff --git a/docs/pt-mongodb-query-digest.rst b/docs/pt-mongodb-query-digest.rst deleted file mode 100644 index 33edaa633..000000000 --- a/docs/pt-mongodb-query-digest.rst +++ /dev/null @@ -1,101 +0,0 @@ -.. _pt-mongodb-query-digest: - -================================== -:program:`pt-mongodb-query-digest` -================================== - -``pt-mongodb-query-digest`` reports query usage statistics -by aggregating queries from MongoDB query profiler. - -Usage -===== - -.. code-block:: bash - - pt-mongodb-query-digest [OPTIONS] - -It runs the following command:: - - db.getSiblingDB("samples").system.profile.find({"op":{"$nin":["getmore", "delete"]}}); - -Then the results are grouped by fingerprint and namespace -(database.collection). -The fingerprint is calculated as a sorted list of keys in the document -with a maximum depth level of 10. -By default, the results are sorted by ascending query count. - -.. note:: ``pt-mongodb-query-digest`` cannot collect statistics - from MongoDB instances that require connection via SSL. - Support for SSL will be added in the future. - -Options -------- - -``-?``, ``--help`` - Show help and exit - -``-a``, ``--authenticationDatabase`` - Specifies the database used to establish credentials and privileges - with a MongoDB server. - By default, the ``admin`` database is used. - -``-c``, ``--no-version-check`` - Don't check for updates - -``-d``, ``--database`` - Specifies which database to profile - -``-f``, ``--output-format`` - Specifies the report output format. Valid options are: ``text``, ``json``. - The default value is ``text``. - -``-l``, ``--log-level`` - Specifies the log level: - ``panic``, ``fatal``, ``error``, ``warn``, ``info``, ``debug error`` - -``-n``, ``--limit`` - Limits the number of queries to show - -``-o``, ``--order-by`` - Specifies the sorting order using fields: - ``count``, ``ratio``, ``query-time``, ``docs-scanned``, ``docs-returned``. - - Adding a hyphen (``-``) in front of a field denotes reverse order. - For example: ``--order-by="count,-ratio"``. - -``-p``, ``--password`` - Specifies the password to use when connecting to a server - with authentication enabled. - - Do not add a space between the option and its value: ``-p``. - - If you specify the option without any value, - you will be prompted for the password. - -``-u``, ``--user`` - Specifies the user name for connecting to a server - with authentication enabled. - -``-v``, ``--version`` - Show version and exit - -Output Example -============== - -.. code-block:: none - - # Query 2: 0.00 QPS, ID 1a6443c2db9661f3aad8edb6b877e45d - # Ratio 1.00 (docs scanned/returned) - # Time range: 2017-01-11 12:58:26.519 -0300 ART to 2017-01-11 12:58:26.686 -0300 ART - # Attribute pct total min max avg 95% stddev median - # ================== === ======== ======== ======== ======== ======== ======= ======== - # Count (docs) 36 - # Exec Time ms 0 0 0 0 0 0 0 0 - # Docs Scanned 0 148.00 0.00 74.00 4.11 74.00 16.95 0.00 - # Docs Returned 2 148.00 0.00 74.00 4.11 74.00 16.95 0.00 - # Bytes recv 0 2.11M 215.00 1.05M 58.48K 1.05M 240.22K 215.00 - # String: - # Namespaces samples.col1 - # Fingerprint $gte,$lt,$meta,$sortKey,filter,find,projection,shardVersion,sort,user_id,user_id - - diff --git a/docs/pt-mongodb-query-digest.rst b/docs/pt-mongodb-query-digest.rst new file mode 120000 index 000000000..d78ea6998 --- /dev/null +++ b/docs/pt-mongodb-query-digest.rst @@ -0,0 +1 @@ +../src/go/pt-mongodb-query-digest/README.rst \ No newline at end of file diff --git a/docs/pt-mongodb-summary.rst b/docs/pt-mongodb-summary.rst deleted file mode 100644 index 61c772ddf..000000000 --- a/docs/pt-mongodb-summary.rst +++ /dev/null @@ -1,168 +0,0 @@ -.. pt-mongodb-summary: - -============================= -:program:`pt-mongodb-summary` -============================= - -``pt-mongodb-summary`` collects information about a MongoDB cluster. -It collects information from several sources -to provide an overview of the cluster. - -Usage -===== - -.. code-block:: bash - - pt-mongodb-summary [OPTIONS] [HOST[:PORT]] - -By default, if you run ``pt-mongodb-summary`` without any parameters, -it will try to connect to ``localhost`` on port ``27017``. -The program collects information about MongoDB instances -by running administration commands and formatting the output. - -.. note:: ``pt-mongodb-summary`` requires to be run by user - with the ``clusterAdmin`` or ``root`` built-in roles. - -.. note:: ``pt-mongodb-summary`` cannot collect statistics - from MongoDB instances that require connection via SSL. - Support for SSL will be added in the future. - -Options -------- - -``-a``, ``--auth-db`` - Specifies the database used to establish credentials and privileges - with a MongoDB server. - By default, the ``admin`` database is used. - -``-f``, ``--output-format`` - Specifies the report output format. Valid options are: ``text``, ``json``. - The default value is ``text``. - -``-p``, ``--password`` - Specifies the password to use when connecting to a server - with authentication enabled. - - Do not add a space between the option and its value: ``-p``. - - If you specify the option without any value, - ``pt-mongodb-summary`` will ask for password interactively. - -``-u``, ``--user`` - Specifies the user name for connecting to a server - with authentication enabled. - -Output Example -============== - -.. code-block:: none - - # Instances #################################################################################### - ID Host Type ReplSet - 0 localhost:17001 PRIMARY r1 - 1 localhost:17002 SECONDARY r1 - 2 localhost:17003 SECONDARY r1 - 0 localhost:18001 PRIMARY r2 - 1 localhost:18002 SECONDARY r2 - 2 localhost:18003 SECONDARY r2 - - # This host - # Mongo Executable ############################################################################# - Path to executable | /home/karl/tmp/MongoDB32Labs/3.0/bin/mongos - # Report On 0 ######################################## - User | karl - PID Owner | mongos - Time | 2016-10-30 00:18:49 -0300 ART - Hostname | karl-HP-ENVY - Version | 3.0.11 - Built On | Linux x86_64 - Started | 2016-10-30 00:18:49 -0300 ART - Datadir | /data/db - Process Type | mongos - - # Running Ops ################################################################################## - - Type Min Max Avg - Insert 0 0 0/5s - Query 0 0 0/5s - Update 0 0 0/5s - Delete 0 0 0/5s - GetMore 0 0 0/5s - Command 0 22 16/5s - - # Security ##################################################################################### - Users 0 - Roles 0 - Auth disabled - SSL disabled - - - # Oplog ######################################################################################## - Oplog Size 18660 Mb - Oplog Used 55 Mb - Oplog Length 0.91 hours - Last Election 2016-10-30 00:18:44 -0300 ART - - - # Cluster wide ################################################################################# - Databases: 3 - Collections: 17 - Sharded Collections: 1 - Unsharded Collections: 16 - Sharded Data Size: 68 GB - Unsharded Data Size: 0 KB - # Balancer (per day) - Success: 6 - Failed: 0 - Splits: 0 - Drops: 0 - -Sections --------- - -Output is separated into the following sections: - -* **Instances** - - This section lists all hosts connected to the current MongoDB instance. - For this, ``pt-mongodb-summary`` runs the ``listShards`` command - and then the ``replSetGetStatus`` on every instance - to collect its ID, type, and replica set. - -* **This host** - - This section provides an overview of the current MongoDB instance - and the underlying OS. - For this, ``pt-mongodb-summary`` groups information - collected from ``hostInfo``, ``getCmdLineOpts``, ``serverStatus``, - and the OS process (by process ID). - -* **Running Ops** - - This section provides minimum, maximum, and average operation counters - for ``insert``, ``query``, ``update``, ``delete``, ``getMore``, - and ``command`` operations. - For this, ``pt-mongodb-summary`` runs the ``serverStatus`` command - 5 times at regular intervals (every second). - -* **Security** - - This section provides information about the security settings. - For this, ``pt-mongodb-summary``, parses ``getCmdLineOpts`` output - and queries the ``admin.system.users`` - and ``admin.system.roles`` collections. - -* **Oplog** - - This section contains details about the MongoDB operations log (oplog). - For this, ``pt-mongodb-summary`` collects statistics - from the oplog on every host in the cluster, - and returns those with the smallest ``TimeDiffHours`` value. - -* **Cluster wide** - - This section provides information about the number of sharded and - unsharded databases, collections, and their size. - For this, ``pt-mongodb-summary`` runs the ``listDatabases`` command - and then runs ``collStats`` for every collection in every database. - diff --git a/docs/pt-mongodb-summary.rst b/docs/pt-mongodb-summary.rst new file mode 120000 index 000000000..135488139 --- /dev/null +++ b/docs/pt-mongodb-summary.rst @@ -0,0 +1 @@ +../src/go/pt-mongodb-summary/README.rst \ No newline at end of file diff --git a/docs/pt-pg-summary.rst b/docs/pt-pg-summary.rst deleted file mode 100644 index 814291361..000000000 --- a/docs/pt-pg-summary.rst +++ /dev/null @@ -1,475 +0,0 @@ -======================== -:program:`pt-pg-summary` -======================== - -**pt-pg-summary** collects information about a PostgreSQL cluster. - -Usage ------ - -``pt-pg-summary [options] [host:[port]]`` - -Binaries --------- -Please check the `releases `_ tab to download the binaries. - -Parameters -^^^^^^^^^^ - -+--------+---------------------------------+---------+----------------------------------------------------------------------------+ -| Short | Long | Default | Description | -+========+=================================+=========+============================================================================+ -| | ``--help`` | | Show context-sensitive help (also try ``--help-long`` and ``--help-man``). | -+--------+---------------------------------+---------+----------------------------------------------------------------------------+ -| | ``--version`` | | Show application version. | -+--------+---------------------------------+---------+----------------------------------------------------------------------------+ -| | ``--databases=DATABASES`` | | Summarize this comma-separated list of databases. All if not specified. | -+--------+---------------------------------+---------+----------------------------------------------------------------------------+ -| ``-h`` | ``--host=HOST`` | | Host to connect to or local Unix socket for connection | -+--------+---------------------------------+---------+----------------------------------------------------------------------------+ -| ``-W`` | ``--password=PASSWORD`` | | Password to use when connecting. | -+--------+---------------------------------+---------+----------------------------------------------------------------------------+ -| ``-p`` | ``--port=PORT`` | | Port number to use for connection. | -+--------+---------------------------------+---------+----------------------------------------------------------------------------+ -| | ``--sleep=SLEEP`` | 10 | Seconds to sleep when gathering status counters. | -+--------+---------------------------------+---------+----------------------------------------------------------------------------+ -| ``-U`` | ``--username=USERNAME`` | | User for login if not current user. | -+--------+---------------------------------+---------+----------------------------------------------------------------------------+ -| | ``--disable-ssl`` | true | Disable SSL for the connection. | -+--------+---------------------------------+---------+----------------------------------------------------------------------------+ -| | ``--verbose`` | false | Show verbose log. | -+--------+---------------------------------+---------+----------------------------------------------------------------------------+ -| | ``--debug`` | false | Show debug information in the logs. | -+--------+---------------------------------+---------+----------------------------------------------------------------------------+ - - -.. Currently hidden -.. --list-encrypted-tables Include a list of the encrypted tables in all databases -.. --ask-pass Prompt for a password when connecting to PostgreSQL -.. --config Config file -.. --defaults-file Only read PostgreSQL options from the given file -.. --read-samples Create a report from the files found in this directory -.. --save-samples Save the data files used to generate the summary in this directory - - -Output -^^^^^^ - -The output is grouped into these categories: - -AllDatabases - Selects ``datname`` from ``pg_database`` where ``datistemplate`` is false. - -ClusterInfo - Selects cluster information from ``pg_stat_activity``. - -ConnectedClients - Counts the connected clients by selecting from ``pg_stat_activity``. - -Connections - Selects ``state`` from ``pg_stat_activity`` and counts them. - -Counters - Selects various counter values from ``pg_stat_database``. - -DatabaseWaitEvents - Shows database wait events from ``pg_locks``, ``pg_stat_database``, ``pg_class``, and ``pg_stat_activity``. - -Databases - Shows the name and size of databases from ``pg_stat_database``. - -GlobalWaitEvents - Shows global wait evens from ``pg_stat_activity``. - -IndexCacheHitRatio - Shows index hit ratios from ``pg_statio_user_indexes``. - -PortAndDatadir - Shows port and data directory name from ``pg_settings``. - -ServerVersion - Shows the value of ``server_version_num``. - -Setting - Selects ``name`` and ``setting`` from ``pg_settings``. - -SlaveHosts10 - Selects information for PostgreSQL version 10. - -SlaveHosts96 - Selects information for PostgreSQL version 9.6. - -TableAccess - Shows table access information by selecting from ``pg_locks``, ``pg_stat_database`` and ``pg_class``. - -TableCacheHitRatio - Shows table cache hit ratio information from ``pg_statio_user_tables``. - -Tablespaces - Show owner and location from ``pg_catalog.pg_tablespace``. - - -Output example -"""""""""""""" - -.. code-block:: html - - ##### --- Database Port and Data_Directory --- #### - +----------------------+----------------------------------------------------+ - | Name | Setting | - +----------------------+----------------------------------------------------+ - | data_directory | /var/lib/postgresql/data | - +----------------------+----------------------------------------------------+ - - ##### --- List of Tablespaces ---- ###### - +----------------------+----------------------+----------------------------------------------------+ - | Name | Owner | Location | - +----------------------+----------------------+----------------------------------------------------+ - | pg_default | postgres | | - | pg_global | postgres | | - +----------------------+----------------------+----------------------------------------------------+ - - - ##### --- Cluster Information --- #### - +------------------------------------------------------------------------------------------------------+ - Usename : postgres - Time : 2020-04-21 13:38:22.770077 +0000 UTC - Client Address : 172.19.0.1 - Client Hostname: - Version : PostgreSQL 9.6.17 on x86_64-pc-linux-gnu (Debian 9.6.17-2.pgdg90+1), compiled by - Started : 2020-04-21 13:36:59.909175 +0000 UTC - Is Slave : false - +------------------------------------------------------------------------------------------------------+ - - ##### --- Databases --- #### - +----------------------+------------+ - | Dat Name | Size | - +----------------------+------------+ - | postgres | 7071 kB | - | template1 | 6961 kB | - | template0 | 6961 kB | - +----------------------+------------+ - - ##### --- Index Cache Hit Ratios --- #### - - Database: postgres - +----------------------+------------+ - | Index Name | Ratio | - +----------------------+------------+ - | index hit rate | 0.00 | - +----------------------+------------+ - - ##### --- Table Cache Hit Ratios --- #### - Database: postgres - +----------------------+------------+ - | Index Name | Ratio | - +----------------------+------------+ - | cache hit rate | 0.00 | - +----------------------+------------+ - - ##### --- List of Wait_events for the entire Cluster - all-databases --- #### - No stats available - - ##### --- List of users and client_addr or client_hostname connected to --all-databases --- #### - +----------------------+------------+---------+----------------------+---------+ - | Wait Event Type | Client | State | Count | - +----------------------+------------+---------+----------------------+---------+ - | postgres | 172.19.0.1/32 | active | 1 | - +----------------------+------------+---------+----------------------+---------+ - - ##### --- Counters diff after 10 seconds --- #### - - +----------------------+-------------+------------+--------------+-------------+------------+-------------+------------+-------------+------------+------------+-----------+-----------+-----------+------------+ - | Database | Numbackends | XactCommit | XactRollback | BlksRead | BlksHit | TupReturned | TupFetched | TupInserted | TupUpdated | TupDeleted | Conflicts | TempFiles | TempBytes | Deadlocks | - +----------------------+-------------+------------+--------------+-------------+------------+-------------+------------+-------------+------------+------------+-----------+-----------+-----------+------------+ - | postgres | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | - | template0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | - | template1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | - +----------------------+-------------+------------+--------------+-------------+------------+-------------+------------+-------------+------------+------------+-----------+-----------+-----------+------------+ - - ##### --- Table access per database --- #### - Database: postgres - +----------------------------------------------------+------+--------------------------------+---------+ - | Relname | Kind | Datname | Count | - +----------------------------------------------------+------+--------------------------------+---------+ - | pg_class | r | postgres | 1 | - | pg_stat_database | v | postgres | 1 | - | pg_locks | v | postgres | 1 | - | pg_class_tblspc_relfilenode_index | i | postgres | 1 | - | pg_class_relname_nsp_index | i | postgres | 1 | - | pg_class_oid_index | i | postgres | 1 | - +----------------------------------------------------+------+--------------------------------+---------+ - - ##### --- Instance settings --- #### - Setting Value - allow_system_table_mods : off - application_name : - archive_command : (disabled) - archive_mode : off - archive_timeout : 0 - array_nulls : on - authentication_timeout : 60 - autovacuum : on - autovacuum_analyze_scale_factor : 0.1 - autovacuum_analyze_threshold : 50 - autovacuum_freeze_max_age : 200000000 - autovacuum_max_workers : 3 - autovacuum_multixact_freeze_max_age : 400000000 - autovacuum_naptime : 60 - autovacuum_vacuum_cost_delay : 20 - autovacuum_vacuum_cost_limit : -1 - autovacuum_vacuum_scale_factor : 0.2 - autovacuum_vacuum_threshold : 50 - autovacuum_work_mem : -1 - backend_flush_after : 0 - backslash_quote : safe_encoding - bgwriter_delay : 200 - bgwriter_flush_after : 64 - bgwriter_lru_maxpages : 100 - bgwriter_lru_multiplier : 2 - block_size : 8192 - bonjour : off - bonjour_name : - bytea_output : hex - check_function_bodies : on - checkpoint_completion_target : 0.5 - checkpoint_flush_after : 32 - checkpoint_timeout : 300 - checkpoint_warning : 30 - client_encoding : UTF8 - client_min_messages : notice - cluster_name : - commit_delay : 0 - commit_siblings : 5 - config_file : /var/lib/postgresql/data/postgresql.conf - constraint_exclusion : partition - cpu_index_tuple_cost : 0.005 - cpu_operator_cost : 0.0025 - cpu_tuple_cost : 0.01 - cursor_tuple_fraction : 0.1 - data_checksums : off - data_directory : /var/lib/postgresql/data - data_sync_retry : off - DateStyle : ISO, MDY - db_user_namespace : off - deadlock_timeout : 1000 - debug_assertions : off - debug_pretty_print : on - debug_print_parse : off - debug_print_plan : off - debug_print_rewritten : off - default_statistics_target : 100 - default_tablespace : - default_text_search_config : pg_catalog.english - default_transaction_deferrable : off - default_transaction_isolation : read committed - default_transaction_read_only : off - default_with_oids : off - dynamic_library_path : $libdir - dynamic_shared_memory_type : posix - effective_cache_size : 524288 - effective_io_concurrency : 1 - enable_bitmapscan : on - enable_hashagg : on - enable_hashjoin : on - enable_indexonlyscan : on - enable_indexscan : on - enable_material : on - enable_mergejoin : on - enable_nestloop : on - enable_seqscan : on - enable_sort : on - enable_tidscan : on - escape_string_warning : on - event_source : PostgreSQL - exit_on_error : off - external_pid_file : - extra_float_digits : 2 - force_parallel_mode : off - from_collapse_limit : 8 - fsync : on - full_page_writes : on - geqo : on - geqo_effort : 5 - geqo_generations : 0 - geqo_pool_size : 0 - geqo_seed : 0 - geqo_selection_bias : 2 - geqo_threshold : 12 - gin_fuzzy_search_limit : 0 - gin_pending_list_limit : 4096 - hba_file : /var/lib/postgresql/data/pg_hba.conf - hot_standby : off - hot_standby_feedback : off - huge_pages : try - ident_file : /var/lib/postgresql/data/pg_ident.conf - idle_in_transaction_session_timeout : 0 - ignore_checksum_failure : off - ignore_system_indexes : off - integer_datetimes : on - IntervalStyle : postgres - join_collapse_limit : 8 - krb_caseins_users : off - krb_server_keyfile : FILE:/etc/postgresql-common/krb5.keytab - lc_collate : en_US.utf8 - lc_ctype : en_US.utf8 - lc_messages : en_US.utf8 - lc_monetary : en_US.utf8 - lc_numeric : en_US.utf8 - lc_time : en_US.utf8 - listen_addresses : * - lo_compat_privileges : off - local_preload_libraries : - lock_timeout : 0 - log_autovacuum_min_duration : -1 - log_checkpoints : off - log_connections : off - log_destination : stderr - log_directory : pg_log - log_disconnections : off - log_duration : off - log_error_verbosity : default - log_executor_stats : off - log_file_mode : 0600 - log_filename : postgresql-%Y-%m-%d_%H%M%S.log - log_hostname : off - log_line_prefix : - log_lock_waits : off - log_min_duration_statement : -1 - log_min_error_statement : error - log_min_messages : warning - log_parser_stats : off - log_planner_stats : off - log_replication_commands : off - log_rotation_age : 1440 - log_rotation_size : 10240 - log_statement : none - log_statement_stats : off - log_temp_files : -1 - log_timezone : Etc/UTC - log_truncate_on_rotation : off - logging_collector : off - maintenance_work_mem : 65536 - max_connections : 100 - max_files_per_process : 1000 - max_function_args : 100 - max_identifier_length : 63 - max_index_keys : 32 - max_locks_per_transaction : 64 - max_parallel_workers_per_gather : 0 - max_pred_locks_per_transaction : 64 - max_prepared_transactions : 0 - max_replication_slots : 0 - max_stack_depth : 2048 - max_standby_archive_delay : 30000 - max_standby_streaming_delay : 30000 - max_wal_senders : 0 - max_wal_size : 64 - max_worker_processes : 8 - min_parallel_relation_size : 1024 - min_wal_size : 5 - old_snapshot_threshold : -1 - operator_precedence_warning : off - parallel_setup_cost : 1000 - parallel_tuple_cost : 0.1 - password_encryption : on - port : 5432 - post_auth_delay : 0 - pre_auth_delay : 0 - quote_all_identifiers : off - random_page_cost : 4 - replacement_sort_tuples : 150000 - restart_after_crash : on - row_security : on - search_path : "$user", public - segment_size : 131072 - seq_page_cost : 1 - server_encoding : UTF8 - server_version : 9.6.17 - server_version_num : 90617 - session_preload_libraries : - session_replication_role : origin - shared_buffers : 16384 - shared_preload_libraries : - sql_inheritance : on - ssl : off - ssl_ca_file : - ssl_cert_file : server.crt - ssl_ciphers : HIGH:MEDIUM:+3DES:!aNULL - ssl_crl_file : - ssl_ecdh_curve : prime256v1 - ssl_key_file : server.key - ssl_prefer_server_ciphers : on - standard_conforming_strings : on - statement_timeout : 0 - stats_temp_directory : pg_stat_tmp - superuser_reserved_connections : 3 - synchronize_seqscans : on - synchronous_commit : on - synchronous_standby_names : - syslog_facility : local0 - syslog_ident : postgres - syslog_sequence_numbers : on - syslog_split_messages : on - tcp_keepalives_count : 9 - tcp_keepalives_idle : 7200 - tcp_keepalives_interval : 75 - temp_buffers : 1024 - temp_file_limit : -1 - temp_tablespaces : - TimeZone : Etc/UTC - timezone_abbreviations : Default - trace_notify : off - trace_recovery_messages : log - trace_sort : off - track_activities : on - track_activity_query_size : 1024 - track_commit_timestamp : off - track_counts : on - track_functions : none - track_io_timing : off - transaction_deferrable : off - transaction_isolation : read committed - transaction_read_only : off - transform_null_equals : off - unix_socket_directories : /var/run/postgresql - unix_socket_group : - unix_socket_permissions : 0777 - update_process_title : on - vacuum_cost_delay : 0 - vacuum_cost_limit : 200 - vacuum_cost_page_dirty : 20 - vacuum_cost_page_hit : 1 - vacuum_cost_page_miss : 10 - vacuum_defer_cleanup_age : 0 - vacuum_freeze_min_age : 50000000 - vacuum_freeze_table_age : 150000000 - vacuum_multixact_freeze_min_age : 5000000 - vacuum_multixact_freeze_table_age : 150000000 - wal_block_size : 8192 - wal_buffers : 512 - wal_compression : off - wal_keep_segments : 0 - wal_level : minimal - wal_log_hints : off - wal_receiver_status_interval : 10 - wal_receiver_timeout : 60000 - wal_retrieve_retry_interval : 5000 - wal_segment_size : 2048 - wal_sender_timeout : 60000 - wal_sync_method : fdatasync - wal_writer_delay : 200 - wal_writer_flush_after : 128 - work_mem : 4096 - xmlbinary : base64 - xmloption : content - zero_damaged_pages : off - - ##### --- Processes start up command --- #### - No postgres process found - -Minimum auth role -^^^^^^^^^^^^^^^^^ - -This program needs to run some commands like ``getShardMap`` and to be able to run those commands -it needs to run under a user with the ``clusterAdmin`` or ``root`` built-in roles. diff --git a/docs/pt-pg-summary.rst b/docs/pt-pg-summary.rst new file mode 120000 index 000000000..7661eb1b3 --- /dev/null +++ b/docs/pt-pg-summary.rst @@ -0,0 +1 @@ +../src/go/pt-pg-summary/README.rst \ No newline at end of file diff --git a/docs/pt-secure-collect.rst b/docs/pt-secure-collect.rst deleted file mode 100644 index 64748fb93..000000000 --- a/docs/pt-secure-collect.rst +++ /dev/null @@ -1,173 +0,0 @@ -============================ -:program:`pt-secure-collect` -============================ - -NAME -==== - -:program:`pt-secure-collect` - collect, sanitize, pack and encrypt data. - -SYNOPSIS -======== - -Usage ------ - -:: - - pt-secure-data [] [ ...] - -By default, :program:`pt-secure-collect` will collect the output of: - -- ``pt-stalk`` -- ``pt-summary`` -- ``pt-mysql-summary`` - -Global flags ------------- - -.. option:: --help - - Show context-sensitive help (also try --help-long and --help-man). - -.. option:: --debug - - Enable debug log level. - -COMMANDS -======== - -* **Help command** - - Show help - -* **Collect command** - - Collect, sanitize, pack and encrypt data from pt-tools. Usage: - - :: - - pt-secure-collect collect - - .. option:: --bin-dir - - Directory having the Percona Toolkit binaries (if they are not in PATH). - - .. option:: --temp-dir - - Temporary directory used for the data collection. - Default: ``${HOME}/data_collection_{timestamp}`` - - .. option:: --include-dir - - Include this dir into the sanitized tar file. - - .. option:: --config-file - - Path to the config file. Default: ``~/.my.cnf`` - - .. option:: --mysql-host - - MySQL host. Default: ``127.0.0.1`` - - .. option:: --mysql-port - - MySQL port. Default: ``3306`` - - .. option:: --mysql-user - - MySQL user name. - - .. option:: --mysql-password - - MySQL password. - - .. option:: --ask-mysql-pass - - password. - - .. option:: --extra-cmd - - Also run this command as part of the data collection. This parameter can - be used more than once. - - .. option:: --encrypt-password - - Encrypt the output file using this password. If omitted, it will be asked - in the command line. - - .. option:: --no-collect - - Do not collect data - - .. option:: --no-sanitize - - Do not sanitize data - - .. option:: --no-encrypt - - Do not encrypt the output file. - - .. option:: --no-sanitize-hostnames - - Do not sanitize hostnames. - - .. option:: --no-sanitize-queries - - Do not replace queries by their fingerprints. - - .. option:: --no-remove-temp-files - - Do not remove temporary files. - -* **Decrypt command** - - Decrypt an encrypted file. The password will be requested from the - terminal. Usage: - - :: - - pt-secure-collect decrypt [flags] - - .. option:: --outfile - - Write the output to this file. If omitted, the output file - name will be the same as the input file, adding the ``.aes`` extension. - -* **Encrypt command** - - Encrypt a file. The password will be requested from the terminal. Usage: - - :: - - pt-secure-collect encrypt [flags] - - .. option:: --outfile - - Write the output to this file. If omitted, the output file - name will be the same as the input file, without the ``.aes`` extension. - -* **Sanitize command** - - Replace queries in a file by their fingerprints and obfuscate hostnames. - Usage: - - :: - - pt-secure-collect sanitize [flags] - - .. option:: --input-file - - Input file. If not specified, the input will be Stdin. - - .. option:: --output-file - - Output file. If not specified, the input will be Stdout. - - .. option:: --no-sanitize-hostnames - - Do not sanitize host names. - - .. option:: --no-sanitize-queries - - Do not replace queries by their fingerprints. diff --git a/docs/pt-secure-collect.rst b/docs/pt-secure-collect.rst new file mode 120000 index 000000000..115df1091 --- /dev/null +++ b/docs/pt-secure-collect.rst @@ -0,0 +1 @@ +../src/go/pt-secure-collect/README.rst \ No newline at end of file diff --git a/docs/release_notes.rst b/docs/release_notes.rst index d8eb3669c..c29b0bb8d 100644 --- a/docs/release_notes.rst +++ b/docs/release_notes.rst @@ -1,4 +1,4 @@ -Percona Toolkit +Release Notes *************** v3.6.0 released 2024-06-12 diff --git a/src/go/pt-galera-log-explainer/README.rst b/src/go/pt-galera-log-explainer/README.rst index 3b3cda00f..6b4f2a365 100644 --- a/src/go/pt-galera-log-explainer/README.rst +++ b/src/go/pt-galera-log-explainer/README.rst @@ -244,3 +244,8 @@ Known issues * Some information will seems missed. Depending on the case, it may be simply unimplemented yet, or it was disabled later because it was found to be unreliable (node index numbers are not reliable for example) * Columns width are sometimes too large to be easily readable. This usually happens when printing SST events with long node names * When some display corner-cases seems broken (events not deduplicated, ...), it is because of extra hidden internal events. + +Authors +======= + +Yoann La Cancellera diff --git a/src/go/pt-k8s-debug-collector/README.rst b/src/go/pt-k8s-debug-collector/README.rst index 6b72ad16f..e2e6012e1 100644 --- a/src/go/pt-k8s-debug-collector/README.rst +++ b/src/go/pt-k8s-debug-collector/README.rst @@ -189,3 +189,8 @@ On Kubernetes 1.21 - 1.24 warning is printed: resourceVersion: "" This warning is harmless and does not affect data collection. We will remove podsecuritypolicies once everyone upgrade to Kubernetes 1.25 or newer. Before that we advise to ignore this warning. + +Authors +======= + +Max Dudin, Andrii Dema, Carlos Salguero, Sveta Smirnova diff --git a/src/go/pt-mongodb-index-check/README.rst b/src/go/pt-mongodb-index-check/README.rst index 3e31f0802..6969f8274 100644 --- a/src/go/pt-mongodb-index-check/README.rst +++ b/src/go/pt-mongodb-index-check/README.rst @@ -70,3 +70,7 @@ Available flags | –version | Show version information | +----------------------------+----------------------------------------+ +Authors +======= + +Carlos Salguero diff --git a/src/go/pt-mongodb-query-digest/README.md b/src/go/pt-mongodb-query-digest/README.md deleted file mode 100644 index 4faf30b8e..000000000 --- a/src/go/pt-mongodb-query-digest/README.md +++ /dev/null @@ -1,49 +0,0 @@ -#pt-mongodb-query-digest - -This program reports query usage statistics by aggregating queries from MongoDB query profiler. -The queries are the result of running: -```javascript -db.getSiblingDB("samples").system.profile.find({"op":{"$nin":["getmore", "delete"]}}); -``` -and then, the results are grouped by fingerprint and namespace (database.collection). - -The fingerprint is calculated as the **sorted list** of the keys in the document. The max depth level is 10. -The last step is sorting the results. The default sort order is by ascending query count. - -##Sample output -``` -# Query 3: 0.06 QPS, ID 0b906bd86148def663d11b402f3e41fa -# Ratio 1.00 (docs scanned/returned) -# Time range: 2017-02-03 16:01:37.484 -0300 ART to 2017-02-03 16:02:08.43 -0300 ART -# Attribute pct total min max avg 95% stddev median -# ================== === ======== ======== ======== ======== ======== ======= ======== -# Count (docs) 100 -# Exec Time ms 2 3 0 1 0 0 0 0 -# Docs Scanned 5 7.50K 75.00 75.00 75.00 75.00 0.00 75.00 -# Docs Returned 92 7.50K 75.00 75.00 75.00 75.00 0.00 75.00 -# Bytes recv 1 106.12M 1.06M 1.06M 1.06M 1.06M 0.00 1.06M -# String: -# Namespaces samples.col1 -# Operation query -# Fingerprint find,shardVersion -# Query {"find":"col1","shardVersion":[0,"000000000000000000000000"]} - -``` - -##Command line parameters - -|Short|Long|Help| -|-----|----|----| -|-?|--help|Show help| -|-a|--authenticationDatabase|database used to establish credentials and privileges with a MongoDB server admin| -|-c|--no-version-check|Don't check for updates| -|-d|--database|database to profile| -|-f|--output-format|report output format. Valid values are text, json. Default: text| -|-l|--log-level|Log level:, panic, fatal, error, warn, info, debug error| -|-n|--limit|show the first n queries| -|-o|--order-by|comma separated list of order by fields (max values): `count`, `ratio`, `query-time`, `docs-scanned`, `docs-returned`.
A `-` in front of the field name denotes reverse order.
Example:`--order-by="count,-ratio"`).| -|-p|--password[=password]|Password (optional). If it is not specified it will be asked| -|-s|--skip-collections|Comma separated list of collections to skip. Default: `system.profile`. It is possible to use an empty list by setting `--skip-collections=""`| -|-u|--user|Username| -|-v|--version|Show version & exit| - diff --git a/src/go/pt-mongodb-query-digest/README.rst b/src/go/pt-mongodb-query-digest/README.rst new file mode 100644 index 000000000..03ae5d3e3 --- /dev/null +++ b/src/go/pt-mongodb-query-digest/README.rst @@ -0,0 +1,115 @@ +.. _pt-mongodb-query-digest: + +================================== +:program:`pt-mongodb-query-digest` +================================== + +``pt-mongodb-query-digest`` reports query usage statistics +by aggregating queries from MongoDB query profiler. + +Usage +===== + +.. code-block:: bash + + pt-mongodb-query-digest [OPTIONS] + +It runs the following command:: + + db.getSiblingDB("samples").system.profile.find({"op":{"$nin":["getmore", "delete"]}}); + +Then the results are grouped by fingerprint and namespace +(database.collection). +The fingerprint is calculated as a sorted list of keys in the document +with a maximum depth level of 10. +By default, the results are sorted by ascending query count. + +Options +------- + +``-?``, ``--help`` + Show help and exit + +``-a``, ``--authenticationDatabase`` + Specifies the database used to establish credentials and privileges + with a MongoDB server. + By default, the ``admin`` database is used. + +``-c``, ``--no-version-check`` + Don't check for updates + +``-d``, ``--database`` + Specifies which database to profile + +``-f``, ``--output-format`` + Specifies the report output format. Valid options are: ``text``, ``json``. + The default value is ``text``. + +``-l``, ``--log-level`` + Specifies the log level: + ``panic``, ``fatal``, ``error``, ``warn``, ``info``, ``debug error`` + +``-n``, ``--limit`` + Limits the number of queries to show + +``-o``, ``--order-by`` + Specifies the sorting order using fields: + ``count``, ``ratio``, ``query-time``, ``docs-scanned``, ``docs-returned``. + + Adding a hyphen (``-``) in front of a field denotes reverse order. + For example: ``--order-by="count,-ratio"``. + +``-p``, ``--password`` + Specifies the password to use when connecting to a server + with authentication enabled. + + Do not add a space between the option and its value: ``-p``. + + If you specify the option without any value, + you will be prompted for the password. + +``-s``, ``--skip-collections`` + Comma separated list of collections to skip. + + Collection ``system.profile`` is skipped by default. + + It is possible to use an empty list by setting ``--skip-collections=""``. + +``--sslCAFile`` + Specifies SSL CA cert file used for authentication. + +``--sslPEMKeyFile`` + Specifies SSL client PEM file used for authentication. + +``-u``, ``--user`` + Specifies the user name for connecting to a server + with authentication enabled. + +``-v``, ``--version`` + Show version and exit + +Output Example +============== + +.. code-block:: none + + # Query 3: 0.06 QPS, ID 0b906bd86148def663d11b402f3e41fa + # Ratio 1.00 (docs scanned/returned) + # Time range: 2017-02-03 16:01:37.484 -0300 ART to 2017-02-03 16:02:08.43 -0300 ART + # Attribute pct total min max avg 95% stddev median + # ================== === ======== ======== ======== ======== ======== ======= ======== + # Count (docs) 100 + # Exec Time ms 2 3 0 1 0 0 0 0 + # Docs Scanned 5 7.50K 75.00 75.00 75.00 75.00 0.00 75.00 + # Docs Returned 92 7.50K 75.00 75.00 75.00 75.00 0.00 75.00 + # Bytes recv 1 106.12M 1.06M 1.06M 1.06M 1.06M 0.00 1.06M + # String: + # Namespaces samples.col1 + # Operation query + # Fingerprint find,shardVersion + # Query {"find":"col1","shardVersion":[0,"000000000000000000000000"]} + +Authors +======= + +Carlos Salguero diff --git a/src/go/pt-mongodb-summary/README.rst b/src/go/pt-mongodb-summary/README.rst index a6b3bb769..7e1729c2f 100644 --- a/src/go/pt-mongodb-summary/README.rst +++ b/src/go/pt-mongodb-summary/README.rst @@ -1,36 +1,63 @@ -pt-mongodb-summary -================== -**pt-mongodb-summary** collects information about a MongoDB cluster. +.. pt-mongodb-summary: + +============================= +:program:`pt-mongodb-summary` +============================= + +``pt-mongodb-summary`` collects information about a MongoDB cluster. +It collects information from several sources +to provide an overview of the cluster. Usage ------ -pt-mongodb-summary [options] [host:[port]] +===== + +.. code-block:: bash + + pt-mongodb-summary [OPTIONS] [HOST:[PORT]] + +By default, if you run ``pt-mongodb-summary`` without any parameters, +it will try to connect to ``localhost`` on port ``27017``. +The program collects information about MongoDB instances +by running administration commands and formatting the output. + +.. note:: ``pt-mongodb-summary`` requires to be run by user + with the ``clusterAdmin`` or ``root`` built-in roles. + +.. note:: ``pt-mongodb-summary`` cannot collect statistics + from MongoDB instances that require connection via SSL. + Support for SSL will be added in the future. -Default host:port is `localhost:27017`. For better results, host must be a **mongos** server. -Binaries --------- -Please check the `releases `_ tab to download the binaries. +Options +------- + +``-a``, ``--auth-db`` + Specifies the database used to establish credentials and privileges + with a MongoDB server. + By default, the ``admin`` database is used. + +``-f``, ``--output-format`` + Specifies the report output format. Valid options are: ``text``, ``json``. + The default value is ``text``. -Parameters -^^^^^^^^^ -|Short|Long|Default|Description| -|-----|----|-------|-----------| -|-a|--auth-db|admin|database used to establish credentials and privileges with a MongoDB server| -|-f|--output-format|report output format|Valid values are text, json. Default: text| -|-f|--output-format|text|output format: text, json. Default: text| -|-p|--password|empty|password to use when connecting if DB auth is enabled| -|-u|--user|empty|user name to use when connecting if DB auth is enabled| +``-p``, ``--password`` + Specifies the password to use when connecting to a server + with authentication enabled. + Do not add a space between the option and its value: ``-p``. -``-p`` is an optional parameter. If it is used it shouldn't have a blank between the parameter and its value: `-p` -It can be also used as `-p` without specifying a password; in that case, the program will ask the password to avoid using a password in the command line. + If you specify the option without any value, + ``pt-mongodb-summary`` will ask for password interactively. +``-u``, ``--user`` + Specifies the user name for connecting to a server + with authentication enabled. Output example -"""""""""""""" -.. code-block:: html +============== + +.. code-block:: none # Instances #################################################################################### ID Host Type ReplSet @@ -90,9 +117,56 @@ Output example Splits: 0 Drops: 0 -Minimum auth role -^^^^^^^^^^^^^^^^^ +Sections +-------- + +Output is separated into the following sections: + +* **Instances** + + This section lists all hosts connected to the current MongoDB instance. + For this, ``pt-mongodb-summary`` runs the ``listShards`` command + and then the ``replSetGetStatus`` on every instance + to collect its ID, type, and replica set. + +* **This host** + + This section provides an overview of the current MongoDB instance + and the underlying OS. + For this, ``pt-mongodb-summary`` groups information + collected from ``hostInfo``, ``getCmdLineOpts``, ``serverStatus``, + and the OS process (by process ID). + +* **Running Ops** + + This section provides minimum, maximum, and average operation counters + for ``insert``, ``query``, ``update``, ``delete``, ``getMore``, + and ``command`` operations. + For this, ``pt-mongodb-summary`` runs the ``serverStatus`` command + 5 times at regular intervals (every second). + +* **Security** + + This section provides information about the security settings. + For this, ``pt-mongodb-summary``, parses ``getCmdLineOpts`` output + and queries the ``admin.system.users`` + and ``admin.system.roles`` collections. + +* **Oplog** + + This section contains details about the MongoDB operations log (oplog). + For this, ``pt-mongodb-summary`` collects statistics + from the oplog on every host in the cluster, + and returns those with the smallest ``TimeDiffHours`` value. + +* **Cluster wide** + + This section provides information about the number of sharded and + unsharded databases, collections, and their size. + For this, ``pt-mongodb-summary`` runs the ``listDatabases`` command + and then runs ``collStats`` for every collection in every database. -This program needs to run some commands like ``getShardMap`` and to be able to run those commands -it needs to run under a user with the ``clusterAdmin`` or ``root`` built-in roles. +Authors +======= +Carlos Salguero diff --git a/src/go/pt-pg-summary/README.rst b/src/go/pt-pg-summary/README.rst index 754b03e73..240244a06 100644 --- a/src/go/pt-pg-summary/README.rst +++ b/src/go/pt-pg-summary/README.rst @@ -1,116 +1,86 @@ -pt-pg-summary -============= -**pt-pg-summary** collects information about a PostgreSQL cluster. +.. pt-pg-summary: -Usage ------ - -``pt-pg-summary [options] [host:[port]]`` - -Binaries --------- -Please check the `releases `_ tab to download the binaries. +======================== +:program:`pt-pg-summary` +======================== -Parameters -^^^^^^^^^^ +``pt-pg-summary`` collects information about a PostgreSQL cluster. -+--------+---------------------------------+---------+----------------------------------------------------------------------------+ -| Short | Long | Default | Description | -+========+=================================+=========+============================================================================+ -| | ``--help`` | | Show context-sensitive help (also try ``--help-long`` and ``--help-man``). | -+--------+---------------------------------+---------+----------------------------------------------------------------------------+ -| | ``--version`` | | Show application version. | -+--------+---------------------------------+---------+----------------------------------------------------------------------------+ -| | ``--databases=DATABASES`` | | Summarize this comma-separated list of databases. All if not specified. | -+--------+---------------------------------+---------+----------------------------------------------------------------------------+ -| ``-h`` | ``--host=HOST`` | | Host to connect to. | -+--------+---------------------------------+---------+----------------------------------------------------------------------------+ -| ``-W`` | ``--password=PASSWORD`` | | Password to use when connecting. | -+--------+---------------------------------+---------+----------------------------------------------------------------------------+ -| ``-p`` | ``--port=PORT`` | | Port number to use for connection. | -+--------+---------------------------------+---------+----------------------------------------------------------------------------+ -| | ``--sleep=SLEEP`` | 10 | Seconds to sleep when gathering status counters. | -+--------+---------------------------------+---------+----------------------------------------------------------------------------+ -| ``-U`` | ``--username=USERNAME`` | | User for login if not current user. | -+--------+---------------------------------+---------+----------------------------------------------------------------------------+ -| | ``--disable-ssl`` | true | Disable SSL for the connection. | -+--------+---------------------------------+---------+----------------------------------------------------------------------------+ -| | ``--verbose`` | false | Show verbose log. | -+--------+---------------------------------+---------+----------------------------------------------------------------------------+ -| | ``--debug`` | false | Show debug information in the logs. | -+--------+---------------------------------+---------+----------------------------------------------------------------------------+ +Usage +===== +.. code-block:: bash -.. Currently hidden -.. --list-encrypted-tables Include a list of the encrypted tables in all databases -.. --ask-pass Prompt for a password when connecting to PostgreSQL -.. --config Config file -.. --defaults-file Only read PostgreSQL options from the given file -.. --read-samples Create a report from the files found in this directory -.. --save-samples Save the data files used to generate the summary in this directory + pt-pg-summary [OPTIONS] [HOST:[PORT]] +Options +------- -Output -^^^^^^ +``--help``, ``--help-long``, ``--help-man`` + Shows context-sensitive help. ``--help-long`` and ``--help-man`` provide more verbose output. -The output is grouped into these categories: +``--version`` + Show application version and exit. | -AllDatabases - Selects ``datname`` from ``pg_database`` where ``datistemplate`` is false. +``--databases`` + Summarizes this comma-separated list of databases. + + All if not specified. -ClusterInfo - Selects cluster information from ``pg_stat_activity``. +``-h``, ``--host`` + Host or local Unix socket for connection. -ConnectedClients - Counts the connected clients by selecting from ``pg_stat_activity``. +``-W``, ``--password`` + Password to use when connecting. | -Connections - Selects ``state`` from ``pg_stat_activity`` and counts them. +``-p``, ``--port`` + Port number to use for connection. | -Counters - Selects various counter values from ``pg_stat_database``. +``--sleep`` + Seconds to sleep when gathering status counters. + + Sleeps 10 seconds if not provided. -DatabaseWaitEvents - Shows database wait events from ``pg_locks``, ``pg_stat_database``, ``pg_class``, and ``pg_stat_activity``. +``-U``, ``--username`` + User for login if not current user. -Databases - Shows the name and size of databases from ``pg_stat_database``. +``--disable-ssl`` + Disable SSL for the connection. -GlobalWaitEvents - Shows global wait evens from ``pg_stat_activity``. + Enabled by default. -IndexCacheHitRatio - Shows index hit ratios from ``pg_statio_user_indexes``. +``--verbose`` + Show verbose log. -PortAndDatadir - Shows port and data directory name from ``pg_settings``. +``--debug`` + Show debug information in the logs. -ServerVersion - Shows the value of ``server_version_num``. -Setting - Selects ``name`` and ``setting`` from ``pg_settings``. +Experimental Options +-------------------- -SlaveHosts10 - Selects information for PostgreSQL version 10. +``--list-encrypted-tables`` + Include a list of the encrypted tables in all databases. -SlaveHosts96 - Selects information for PostgreSQL version 9.6. +``--ask-pass`` + Prompt for a password when connecting to PostgreSQL. -TableAccess - Shows table access information by selecting from ``pg_locks``, ``pg_stat_database`` and ``pg_class``. +``--config`` + Configuration file. -TableCacheHitRatio - Shows table cache hit ratio information from ``pg_statio_user_tables``. +``--defaults-file`` + Only read PostgreSQL options from the given file. -Tablespaces - Show owner and location from ``pg_catalog.pg_tablespace``. +``--read-samples`` + Create a report from the files found in this directory. +``--save-samples`` + Save the data files used to generate the summary in this directory. Output example -"""""""""""""" +============== -.. code-block:: html +.. code-block:: none ##### --- Database Port and Data_Directory --- #### +----------------------+----------------------------------------------------+ @@ -466,8 +436,80 @@ Output example ##### --- Processes start up command --- #### No postgres process found -Minimum auth role -^^^^^^^^^^^^^^^^^ +Sections +-------- + +Output is separated into the following sections: + +* **AllDatabases** + + Selects ``datname`` from ``pg_database`` where ``datistemplate`` is false. + +* **ClusterInfo** + + Selects cluster information from ``pg_stat_activity``. + +* **ConnectedClients** + + Counts the connected clients by selecting from ``pg_stat_activity``. + +* **Connections** + + Selects ``state`` from ``pg_stat_activity`` and counts them. + +* **Counters** + + Selects various counter values from ``pg_stat_database``. + +* **DatabaseWaitEvents** + + Shows database wait events from ``pg_locks``, ``pg_stat_database``, ``pg_class``, and ``pg_stat_activity``. + +* **Databases** + + Shows the name and size of databases from ``pg_stat_database``. + +* **GlobalWaitEvents** + + Shows global wait evens from ``pg_stat_activity``. + +* **IndexCacheHitRatio** + + Shows index hit ratios from ``pg_statio_user_indexes``. + +* **PortAndDatadir** + + Shows port and data directory name from ``pg_settings``. + +* **ServerVersion** + + Shows the value of ``server_version_num``. + +* **Setting** + + Selects ``name`` and ``setting`` from ``pg_settings``. + +* **SlaveHosts10** + + Selects information for PostgreSQL version 10. + +* **SlaveHosts96** + + Selects information for PostgreSQL version 9.6. + +* **TableAccess** + + Shows table access information by selecting from ``pg_locks``, ``pg_stat_database`` and ``pg_class``. + +* **TableCacheHitRatio** + + Shows table cache hit ratio information from ``pg_statio_user_tables``. + +* **Tablespaces** + + Show owner and location from ``pg_catalog.pg_tablespace``. + +Authors +======= -This program needs to run some commands like ``getShardMap`` and to be able to run those commands -it needs to run under a user with the ``clusterAdmin`` or ``root`` built-in roles. +Carlos Salguero diff --git a/src/go/pt-secure-collect/README.md b/src/go/pt-secure-collect/README.md deleted file mode 100644 index 8a93e7286..000000000 --- a/src/go/pt-secure-collect/README.md +++ /dev/null @@ -1,88 +0,0 @@ -# pt-secure-collect -Collect, sanitize, pack and encrypt data. By default, this program will collect the output of: - -- `pt-stalk --no-stalk --iterations=2 --sleep=30 --host=$mysql-host --dest=$temp-dir --port=$mysql-port --user=$mysql-user --password=$mysql-pass` -- `pt-summary` -- `pt-mysql-summary --host=$mysql-host --port=$mysql-port --user=$mysql-user --password=$mysql-pass` - -Internal variables placeholders will be replaced with the corresponding flag values. For example, `$mysql-host` will be replaced with the values specified in the `--mysql-host` flag. - -Usage: -``` -pt-secure-collect [] [ ...] -``` - - -### Global flags -|Flag|Description| -|-----|-----| -|--help|Show context-sensitive help (also try --help-long and --help-man).| -|--debug|Enable debug log level.| - -### **Commands** -#### **Help command** -Show help - -#### **Collect command** -Collect, sanitize, pack and encrypt data from pt-tools. -Usage: -``` -pt-secure-collect collect -``` - -|Flag|Description| -|-----|-----| -|--bin-dir|Directory having the Percona Toolkit binaries (if they are not in PATH).| -|--temp-dir|Temporary directory used for the data collection. Default: ${HOME}/data_collection\_{timestamp}| -|--include-dir|Include this dir into the sanitized tar file| -|--config-file|Path to the config file. Default: `~/.my.cnf`| -|--mysql-host|MySQL host. Default: `127.0.0.1`| -|--mysql-port|MySQL port. Default: `3306`| -|--mysql-user|MySQL user name.| -|--mysql-password|MySQL password.| -|--ask-mysql-pass|Ask MySQL password.| -|--extra-cmd|Also run this command as part of the data collection. This parameter can be used more than once.| -|--encrypt-password|Encrypt the output file using this password.
If omitted, it will be asked in the command line.| -|--no-collect|Do not collect data| -|--no-sanitize|Do not sanitize data| -|--no-encrypt|Do not encrypt the output file.| -|--no-sanitize-hostnames|Do not sanitize host names.| -|--no-sanitize-queries|Do not replace queries by their fingerprints.| -|--no-remove-temp-files|Do not remove temporary files.| - -#### **Decrypt command** -Decrypt an encrypted file. The password will be requested from the terminal. -Usage: -``` -pt-secure-collect decrypt [flags] -``` -|Flag|Description| -|-----|------| -|--outfile|Write the output to this file.
If omitted, the output file name will be the same as the input file, adding the `.aes` extension| - - -#### **Encrypt command** -Encrypt a file. The password will be requested from the terminal. -Usage: -``` -pt-secure-collect encrypt [flags] -``` -|Flag|Description| -|-----|------| -|--outfile|Write the output to this file.
If omitted, the output file name will be the same as the input file, without the `.aes` extension| - - -#### **Sanitize command** -Replace queries in a file by their fingerprints and obfuscate hostnames. -Usage: -``` -pt-secure-collect sanitize [flags] -``` - -|Flag|Description| -|-----|-----| -|--input-file| Input file. If not specified, the input will be Stdin.| -|--output-file|Output file. If not specified, the input will be Stdout.| -|--no-sanitize-hostnames|Do not sanitize host names.| -|--no-sanitize-queries|Do not replace queries by their fingerprints.| - diff --git a/src/go/pt-secure-collect/README.rst b/src/go/pt-secure-collect/README.rst new file mode 100644 index 000000000..36bf289af --- /dev/null +++ b/src/go/pt-secure-collect/README.rst @@ -0,0 +1,178 @@ +============================ +:program:`pt-secure-collect` +============================ + +NAME +==== + +:program:`pt-secure-collect` - collect, sanitize, pack and encrypt data. + +SYNOPSIS +======== + +Usage +----- + +:: + + pt-secure-data [] [ ...] + +By default, :program:`pt-secure-collect` will collect the output of: + +- ``pt-stalk --no-stalk --iterations=2 --sleep=30`` +- ``pt-summary`` +- ``pt-mysql-summary`` + +Global flags +------------ + +.. option:: --help + + Show context-sensitive help (also try --help-long and --help-man). + +.. option:: --debug + + Enable debug log level. + +COMMANDS +======== + +* **Help command** + + Show help + +* **Collect command** + + Collect, sanitize, pack and encrypt data from pt-tools. Usage: + + :: + + pt-secure-collect collect + + .. option:: --bin-dir + + Directory having the Percona Toolkit binaries (if they are not in PATH). + + .. option:: --temp-dir + + Temporary directory used for the data collection. + Default: ``${HOME}/data_collection_{timestamp}`` + + .. option:: --include-dir + + Include this dir into the sanitized tar file. + + .. option:: --config-file + + Path to the config file. Default: ``~/.my.cnf`` + + .. option:: --mysql-host + + MySQL host. Default: ``127.0.0.1`` + + .. option:: --mysql-port + + MySQL port. Default: ``3306`` + + .. option:: --mysql-user + + MySQL user name. + + .. option:: --mysql-password + + MySQL password. + + .. option:: --ask-mysql-pass + + Ask MySQL password. + + .. option:: --extra-cmd + + Also run this command as part of the data collection. This parameter can + be used more than once. + + .. option:: --encrypt-password + + Encrypt the output file using this password. If omitted, it will be asked + in the command line. + + .. option:: --no-collect + + Do not collect data + + .. option:: --no-sanitize + + Do not sanitize data + + .. option:: --no-encrypt + + Do not encrypt the output file. + + .. option:: --no-sanitize-hostnames + + Do not sanitize hostnames. + + .. option:: --no-sanitize-queries + + Do not replace queries by their fingerprints. + + .. option:: --no-remove-temp-files + + Do not remove temporary files. + +* **Decrypt command** + + Decrypt an encrypted file. The password will be requested from the + terminal. Usage: + + :: + + pt-secure-collect decrypt [flags] + + .. option:: --outfile + + Write the output to this file. If omitted, the output file + name will be the same as the input file, adding the ``.aes`` extension. + +* **Encrypt command** + + Encrypt a file. The password will be requested from the terminal. Usage: + + :: + + pt-secure-collect encrypt [flags] + + .. option:: --outfile + + Write the output to this file. If omitted, the output file + name will be the same as the input file, without the ``.aes`` extension. + +* **Sanitize command** + + Replace queries in a file by their fingerprints and obfuscate hostnames. + Usage: + + :: + + pt-secure-collect sanitize [flags] + + .. option:: --input-file + + Input file. If not specified, the input will be Stdin. + + .. option:: --output-file + + Output file. If not specified, the input will be Stdout. + + .. option:: --no-sanitize-hostnames + + Do not sanitize host names. + + .. option:: --no-sanitize-queries + + Do not replace queries by their fingerprints. + +Authors +======= + +Carlos Salguero