[WordPress] maraidb crashed and wont start

[AchillesAlama]

Platform

AWS

bndiagnostic ID know more about bndiagnostic ID

0a3efab1-f5ff-0641-b56f-27af43f64041

bndiagnostic output

===== Begin of bndiagnostic tool output =====

? Apache: Found possible issues
? Mariadb: Found possible issues
✓ Connectivity: No issues found
✓ Wordpress: No issues found
? Resources: Found possible issues
✓ Processes: No issues found
✓ Php: No issues found

[Apache]

Found recent error or warning messages in the Apache error log.

[Mon Sep 04 12:46:48.727121 2023] [proxy_fcgi:error] [pid 19781:tid 
140349775267584] [client **ip_address**:53004] AH01071: Got error 'PHP message: 
WordPress database error Can't create table 
`bitnami_wordpress`.`wp_jtl_connector_link_order` (errno: 165 "Table is read 
only") for query 
Press [Enter] to continue:
 CREATE TABLE IF NOT EXISTS `wp_jtl_connector_link_order` (
 `endpoint_id` BIGINT(20) unsigned NOT NULL,
 `host_id` INT(10) unsigned NOT NULL,
 PRIMARY K [Mon Sep 04 12:47:07.288972 2023] [proxy_fcgi:error] [pid 19769:tid 
140350504974080] [client **ip_address**:16092] AH01071: Got error 'PHP message: 
WordPress database error Can't create table 
`bitnami_wordpress`.`wp_jtl_connector_link_order` (errno: 165 "Table is read 
only") for query 
 CREATE TABLE IF NOT EXISTS `wp_jtl_connector_link_order` (
 `endpoint_id` BIGINT(20) unsigned NOT NULL,
 `host_id` INT(10) unsigned NOT NULL,
 PRIMARY KE [Mon Sep 04 12:47:07.483482 2023] [proxy_fcgi:error] [pid 19769:tid 
140350521759488] [client **ip_address**:38424] AH01071: Got error 'PHP message: 
WordPress database error Can't create table 
`bitnami_wordpress`.`wp_jtl_connector_link_order` (errno: 165 "Table is read 
only") for query 
 CREATE TABLE IF NOT EXISTS `wp_jtl_connector_link_order` (
 `endpoint_id` BIGINT(20) unsigned NOT NULL,
 `host_id` INT(10) unsigned NOT NULL,
 PRIMARY K

Please check the following guide to troubleshoot server issues:

Press [Enter] to continue:
https://docs.bitnami.com/general/apps/wordpress/troubleshooting/debug-erro
rs-apache/

[Mariadb]

Found recent error messages in the MariaDB error log:

2023-09-04 12:44:45 384 [ERROR] InnoDB: Failed to find tablespace for table 
`bitnami_wordpress`.`wp_wpforms_payments` in the cache. Attempting to load the 
tablespace with space id 124

Please check the following guide to troubleshoot MariaDB issues:

https://docs.bitnami.com/aws/apps/wordpress/troubleshooting/debug-errors-m
ariadb/

[Resources]

Your instance has little available RAM memory.

 total used free shared buff/cache available Mem: 3946 2883 166 183 896 602 
Swap: 0 0 0
Press [Enter] to continue:

You could try to increase your instance's memory. Please check your cloud
provider's documentation for more information.

You can also enable swap memory to improve performance.

https://docs.bitnami.com/installer/faq/linux-faq/administration/increase-m
emory-linux/

===== End of bndiagnostic tool output =====

bndiagnostic was not useful. Could you please tell us why?

it was

Describe your issue as much as you can

Hallo everyone,

i have a problem with MariaDB, out of no where the Wordpress site that i manage went down, went in to check it out found mariadb was down, looked into the logs and found this

2023-09-04 12:08:55 0 [Note] InnoDB: Number of pools: 1
2023-09-04 12:08:55 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
2023-09-04 12:08:55 0 [Note] InnoDB: Using Linux native AIO
2023-09-04 12:08:55 0 [Note] InnoDB: Initializing buffer pool, total size = 268435456, chunk size = 134217728
2023-09-04 12:08:55 0 [Note] InnoDB: Completed initialization of buffer pool
2023-09-04 12:08:55 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=18020391,24612461
2023-09-04 12:08:55 0 [ERROR] InnoDB: Missing FILE_CREATE, FILE_DELETE or FILE_MODIFY before FILE_CHECKPOINT for tablespace 133
2023-09-04 12:08:55 0 [ERROR] InnoDB: Plugin initialization aborted with error Data structure corruption
2023-09-04 12:08:55 0 [Note] InnoDB: Starting shutdown...
2023-09-04 12:08:55 0 [ERROR] Plugin 'InnoDB' init function returned error.
2023-09-04 12:08:55 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2023-09-04 12:08:55 0 [Note] Plugin 'FEEDBACK' is disabled.
2023-09-04 12:08:55 0 [ERROR] Unknown/unsupported storage engine: InnoDB
2023-09-04 12:08:55 0 [ERROR] Aborting

so i set innodb_force_recovery = 6 (1 to 5 did not work) to get a backup of the data. and now mariadb only opens when this flag is on, what can i do here ?

[jotamartos]

Hi @AchillesAlama,

Please take a look at previous similar tickets to try to solve this issue.

https://github.com/bitnami/vms/issues?q=is%3Aissue+innodb_force_recovery

The MariaDB's documentation will also help

https://mariadb.com/kb/en/innodb-recovery-modes/

If you continue running into issues, you can try migrating the data to a new server

https://docs.bitnami.com/general/how-to/migrate-wordpress/

[AchillesAlama]

Hi @AchillesAlama,

Please take a look at previous similar tickets to try to solve this issue.

https://github.com/bitnami/vms/issues?q=is%3Aissue+innodb_force_recovery

The MariaDB's documentation will also help

https://mariadb.com/kb/en/innodb-recovery-modes/

If you continue running into issues, you can try migrating the data to a new server

https://docs.bitnami.com/general/how-to/migrate-wordpress/

wouldn't it be easier to uninstall mariadb and reinstall it then re-import the db from backup?
i can't seem to uninstall mariadb tho, using command sudo apt-get purge mariadb-* but it doesn't find it, dose it have another location or?

Reading package lists... Done
Building dependency tree       
Reading state information... Done
Note, selecting 'mariadb-rocksdb-engine-10.2' for glob 'mariadb-*'
Note, selecting 'mariadb-rocksdb-engine-10.3' for glob 'mariadb-*'
Note, selecting 'mariadb-test' for glob 'mariadb-*'
Note, selecting 'mariadb-test-data-10.0' for glob 'mariadb-*'
Note, selecting 'mariadb-test-data' for glob 'mariadb-*'
Note, selecting 'mariadb-plugin-cracklib-password-check' for glob 'mariadb-*'
Note, selecting 'mariadb-test-5.5' for glob 'mariadb-*'
Note, selecting 'mariadb-common' for glob 'mariadb-*'
Note, selecting 'mariadb-plugin-connect' for glob 'mariadb-*'
Note, selecting 'mariadb-server-core-5.5' for glob 'mariadb-*'
Note, selecting 'mariadb-server-5.1' for glob 'mariadb-*'
Note, selecting 'mariadb-server-5.2' for glob 'mariadb-*'
Note, selecting 'mariadb-server-5.3' for glob 'mariadb-*'
Note, selecting 'mariadb-server-5.5' for glob 'mariadb-*'
Note, selecting 'mariadb-tokudb-engine-5.5' for glob 'mariadb-*'
Note, selecting 'mariadb-server-core-10.0' for glob 'mariadb-*'
Note, selecting 'mariadb-server-core-10.1' for glob 'mariadb-*'
Note, selecting 'mariadb-server-core-10.2' for glob 'mariadb-*'
Note, selecting 'mariadb-server-core-10.3' for glob 'mariadb-*'
Note, selecting 'mariadb-connect-engine-10.0' for glob 'mariadb-*'
Note, selecting 'mariadb-connect-engine-10.1' for glob 'mariadb-*'
Note, selecting 'mariadb-galera-server' for glob 'mariadb-*'
Note, selecting 'mariadb-backup' for glob 'mariadb-*'
Note, selecting 'mariadb-client-5.5' for glob 'mariadb-*'
Note, selecting 'mariadb-server-10.0' for glob 'mariadb-*'
Note, selecting 'mariadb-server-10.1' for glob 'mariadb-*'
Note, selecting 'mariadb-server-10.2' for glob 'mariadb-*'
Note, selecting 'mariadb-server-10.3' for glob 'mariadb-*'
Note, selecting 'mariadb-plugin-oqgraph' for glob 'mariadb-*'
Note, selecting 'mariadb-plugin-gssapi-client' for glob 'mariadb-*'
Note, selecting 'mariadb-plugin-spider' for glob 'mariadb-*'
Note, selecting 'mariadb-galera-server-10.0' for glob 'mariadb-*'
Note, selecting 'mariadb-plugin-rocksdb' for glob 'mariadb-*'
Note, selecting 'mariadb-plugin-gssapi-server' for glob 'mariadb-*'
Note, selecting 'mariadb-plugin-tokudb' for glob 'mariadb-*'
Note, selecting 'mariadb-client-10.0' for glob 'mariadb-*'
Note, selecting 'mariadb-client-10.1' for glob 'mariadb-*'
Note, selecting 'mariadb-client-10.2' for glob 'mariadb-*'
Note, selecting 'mariadb-client-10.3' for glob 'mariadb-*'
Note, selecting 'mariadb-server' for glob 'mariadb-*'
Note, selecting 'mariadb-tokudb-engine-10.0' for glob 'mariadb-*'
Note, selecting 'mariadb-tokudb-engine-10.1' for glob 'mariadb-*'
Note, selecting 'mariadb-client-core-10.0' for glob 'mariadb-*'
Note, selecting 'mariadb-client-core-10.1' for glob 'mariadb-*'
Note, selecting 'mariadb-client-core-10.2' for glob 'mariadb-*'
Note, selecting 'mariadb-client-core-10.3' for glob 'mariadb-*'
Note, selecting 'mariadb-client' for glob 'mariadb-*'
Note, selecting 'mariadb-test-10.0' for glob 'mariadb-*'
Note, selecting 'mariadb-test-10.1' for glob 'mariadb-*'
Note, selecting 'mariadb-oqgraph-engine-10.0' for glob 'mariadb-*'
Note, selecting 'mariadb-oqgraph-engine-10.1' for glob 'mariadb-*'
Note, selecting 'mariadb-galera-server-5.5' for glob 'mariadb-*'
Note, selecting 'mariadb-client-core-5.5' for glob 'mariadb-*'
Note, selecting 'mariadb-gssapi-server-10.1' for glob 'mariadb-*'
Note, selecting 'mariadb-gssapi-server-10.2' for glob 'mariadb-*'
Note, selecting 'mariadb-gssapi-client-10.1' for glob 'mariadb-*'
Note, selecting 'mariadb-gssapi-client-10.2' for glob 'mariadb-*'
Note, selecting 'mariadb-plugin-mroonga' for glob 'mariadb-*'
Note, selecting 'mariadb-backup-10.1' for glob 'mariadb-*'
Note, selecting 'mariadb-backup-10.2' for glob 'mariadb-*'
Package 'mariadb-galera-server-10.0' is not installed, so not removed
Package 'mariadb-galera-server-5.5' is not installed, so not removed
Package 'mariadb-server-10.0' is not installed, so not removed
Package 'mariadb-server-5.1' is not installed, so not removed
Package 'mariadb-server-5.2' is not installed, so not removed
Package 'mariadb-server-5.3' is not installed, so not removed
Package 'mariadb-server-5.5' is not installed, so not removed
Package 'mariadb-backup-10.1' is not installed, so not removed
Package 'mariadb-backup-10.2' is not installed, so not removed
Package 'mariadb-client-10.1' is not installed, so not removed
Package 'mariadb-client-10.0' is not installed, so not removed
Package 'mariadb-client-10.2' is not installed, so not removed
Package 'mariadb-client-5.5' is not installed, so not removed
Package 'mariadb-client-core-10.0' is not installed, so not removed
Package 'mariadb-client-core-10.1' is not installed, so not removed
Package 'mariadb-client-core-10.2' is not installed, so not removed
Package 'mariadb-client-core-5.5' is not installed, so not removed
Package 'mariadb-connect-engine-10.0' is not installed, so not removed
Package 'mariadb-connect-engine-10.1' is not installed, so not removed
Package 'mariadb-server-10.1' is not installed, so not removed
Package 'mariadb-gssapi-client-10.1' is not installed, so not removed
Package 'mariadb-gssapi-client-10.2' is not installed, so not removed
Package 'mariadb-gssapi-server-10.1' is not installed, so not removed
Package 'mariadb-gssapi-server-10.2' is not installed, so not removed
Package 'mariadb-oqgraph-engine-10.0' is not installed, so not removed
Package 'mariadb-oqgraph-engine-10.1' is not installed, so not removed
Package 'mariadb-rocksdb-engine-10.2' is not installed, so not removed
Package 'mariadb-rocksdb-engine-10.3' is not installed, so not removed
Package 'mariadb-tokudb-engine-10.0' is not installed, so not removed
Package 'mariadb-tokudb-engine-10.1' is not installed, so not removed
Package 'mariadb-tokudb-engine-5.5' is not installed, so not removed
Package 'mariadb-galera-server' is not installed, so not removed
Package 'mariadb-server-10.2' is not installed, so not removed
Package 'mariadb-server-core-10.0' is not installed, so not removed
Package 'mariadb-server-core-10.1' is not installed, so not removed
Package 'mariadb-server-core-10.2' is not installed, so not removed
Package 'mariadb-server-core-5.5' is not installed, so not removed
Package 'mariadb-test-10.0' is not installed, so not removed
Package 'mariadb-test-10.1' is not installed, so not removed
Package 'mariadb-test-5.5' is not installed, so not removed
Package 'mariadb-test-data-10.0' is not installed, so not removed
Package 'mariadb-backup' is not installed, so not removed
Package 'mariadb-client' is not installed, so not removed
Package 'mariadb-client-10.3' is not installed, so not removed
Package 'mariadb-client-core-10.3' is not installed, so not removed
Package 'mariadb-common' is not installed, so not removed
Package 'mariadb-plugin-connect' is not installed, so not removed
Package 'mariadb-plugin-cracklib-password-check' is not installed, so not removed
Package 'mariadb-plugin-gssapi-client' is not installed, so not removed
Package 'mariadb-plugin-gssapi-server' is not installed, so not removed
Package 'mariadb-plugin-mroonga' is not installed, so not removed
Package 'mariadb-plugin-oqgraph' is not installed, so not removed
Package 'mariadb-plugin-rocksdb' is not installed, so not removed
Package 'mariadb-plugin-spider' is not installed, so not removed
Package 'mariadb-plugin-tokudb' is not installed, so not removed
Package 'mariadb-server' is not installed, so not removed
Package 'mariadb-server-10.3' is not installed, so not removed
Package 'mariadb-server-core-10.3' is not installed, so not removed
Package 'mariadb-test' is not installed, so not removed
Package 'mariadb-test-data' is not installed, so not removed
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

[jotamartos]

Hi @AchillesAlama,

We do not use the system's package manager to install/configure the components. We build them from scratch and configure them using our own scripts.

As I mentioned above, you can try to recover the database by following the MariaDB's official documentation or you can migrate the data to a new instance.

[AchillesAlama]

i have migrated to a new lightsail instance and everything works expect for one thing, i have php 8.2 installed and i cannot go above php 8.0 since not all plugins i use are supported, can't i make an instance with php8.0 instead of php8.2, i don't see an option for this?

[AchillesAlama]

never-mind, not using bitnami anymore (switching php version should be as easy as selecting from a drop-down and restarting the server), i'm installing my own packages.