cyrus-sasl broken in 2023Q4, 2022Q4, 2021Q4

[xmerlin]

After the 2020Q4 release, it is impossible to use the SQL / auxprop plugin of Cyrus-SASL. The bug affects all daemons using Cyrus-SASL for authentication. Specifically, attached are the steps to reproduce the problem on a minimal configuration of Postfix.

install packages postfix postfix-mysql cyrus-sasl cy2-login cy2-plain cy2-sql

CREATE DATABASE

CREATE DATABASE postfix_db;
GRANT SELECT ON postfix_db.* TO 'postfix_user'@'localhost' IDENTIFIED BY 'password';
FLUSH PRIVILEGES;
USE postfix_db;

CREATE TABLE virtual_users (
id int(11) NOT NULL auto_increment,
email varchar(100) NOT NULL,
password varchar(100) NOT NULL,
PRIMARY KEY (id),
UNIQUE KEY email (email)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

INSERT INTO virtual_users (email, password) VALUES ('[email protected]', 'password');

configure postfix

/opt/local/etc/postfix/main.cf

smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
smtpd_sasl_local_domain = $myhostname
smtpd_recipient_restrictions = permit_sasl_authenticated,permit_mynetworks,reject_unauth_destination
smtpd_sasl_type = cyrus
smtpd_sasl_path = smtpd

broken_sasl_auth_clients = yes

virtual_mailbox_maps = proxy:mysql:/opt/local/etc/postfix/mysql_virtual_mailbox_maps.cf

/opt/local/etc/postfix/mysql_virtual_mailbox_maps.cf

user = postfix_user
password = password
hosts = localhost
dbname = postfix_db
query = SELECT 1 FROM virtual_users WHERE email='%s'

/opt/local/etc/postfix/smtpd.conf

pwcheck_method: auxprop
auxprop_plugin: sql
mech_list: plain login

sql_engine: mysql
sql_hostnames: localhost
sql_user: postfix_user
sql_passwd: password
sql_database: postfix_db
sql_select: SELECT password FROM virtual_users WHERE email = '%u@%r' ;

sql_usessl: 0
sql_verbose: yes
sql_log_level: 7

[jperkin]

Some initial notes:

• The smtpd.conf location needs to be /opt/local/etc/sasl2/smtpd.conf.
• With that in place, the smtpd process crashes.

The stack is as follows:

core 'core.smtpd.8477' of 8477:	smtpd -n smtp -t inet -u -s 2
 fffffc7fbc7ea343 mysql_cset_escape_slashes (0, 4b6770, 5b0db8, 4) + 83
 fffffc7fbc764125 _mysql_escape_str (4b6770, 5b0db8) + 25
 fffffc7fbc763882 sql_auxprop_lookup (599b50, 48e950, 0, 5c0db1, 10) + 1a2
 fffffc7fbcc9ab33 _sasl_auxprop_lookup (48e950, 0, 5c0db1, 10) + 1a3
 fffffc7fbcc9bacc _sasl_canon_user_lookup (5c0010, 5ac910, 10, 7, 5c0880) + 10c
 fffffc7fbc782676 login_server_mech_step (5b1940, 48e950, 5b2de0, 8, fffffc7fffdff060, fffffc7fffdff05c, 5c0880) + 1a6
 fffffc7fbcca7a89 sasl_server_step (5c0010, 5b2de0, 8, fffffc7fffdff060, fffffc7fffdff05c) + a9
 0000000000429b80 xsasl_cyrus_server_next (5a7b10, 4f67c0, 599500) + a0
 0000000000426bb4 smtpd_sasl_authenticate (fffffc7fffdff1e0, 5b1b30, 0) + 54
 0000000000413320 smtpd_sasl_auth_cmd_wrapper (fffffc7fffdff1e0, 2, 492660) + 50
 00000000004190e2 smtpd_proto (fffffc7fffdff1e0) + 992
 00000000004197e3 smtpd_service (598ca0, fffffc7fffdfff59, fffffc7fffdffda8) + 163
 fffffc7fbc85492c single_server_wakeup (17, 0) + cc
 fffffc7fbcaaaf98 event_loop (ffffffff) + 298
 fffffc7fbc855af8 single_server_main (8, fffffc7fffdffd68, 419680) + d98
 000000000041a617 ???????? ()
 00000000004114d7 _start_crt () + 87
 0000000000411438 _start () + 18

Looking at the source for mysql_cset_escape_slashes():

size_t mysql_cset_escape_slashes(const MARIADB_CHARSET_INFO * cset, char *newstr,
                     const char * escapestr, size_t escapestr_len )
{
  const char   *newstr_s = newstr;
  const char   *newstr_e = newstr + 2 * escapestr_len;
  const char   *end = escapestr + escapestr_len;
  my_bool  escape_overflow = FALSE;

  for (;escapestr < end; escapestr++) {
    char esc = '\0';
    unsigned int len = 0;

    /* check unicode characters */
    if (cset->char_maxlen > 1 && (len = cset->mb_valid(escapestr, end))) {
...

we can see why it's crashing, the function is being called with cset set to NULL, but there is no check in the function for it being valid and it immediately tries to deference it with cset->char_maxlen.

It's likely setting a valid charset will resolve the problem. An open question is why this has changed, and maybe we're no longer specifying a default or something.

[jperkin]

Ok, I've found the problem. We have an historical setting in our builds of:

include/pkgoptions.mk:MYSQL_CHARSET=                    utf8

However, this is no longer a valid character set:

MariaDB [(none)]> show character set where charset like '%utf8%';
+---------+---------------+--------------------+--------+
| Charset | Description   | Default collation  | Maxlen |
+---------+---------------+--------------------+--------+
| utf8mb3 | UTF-8 Unicode | utf8mb3_general_ci |      3 |
| utf8mb4 | UTF-8 Unicode | utf8mb4_general_ci |      4 |
+---------+---------------+--------------------+--------+
2 rows in set (0.001 sec)

I'll get that setting removed so that we use the default, which is set as:

databases/mariadb106-client/Makefile.common:MARIADB_CHARSET?=   ${MYSQL_CHARSET:Uutf8mb4}
databases/mariadb106-client/Makefile.common:CMAKE_ARGS+=                -DDEFAULT_CHARSET=${MARIADB_CHARSET}

[xmerlin]

utf8mb4 is related to MySQL 8.x. I have tested everything using MySQL 5.7 and found that the bug lies elsewhere. The SQL plugin was not loaded in Q4 for the years 2023, 2022, and 2021, and no queries were executed on the database (I have already debugged every query in the database).

[xmerlin]

I've tried also with /opt/local/etc/sasl2/smtpd.conf same results

[jperkin]

With the change to the default charset, and using all of the configuration files you've provided above, it now works for me:

220 135d7c87-8fb4-4f4e-8e42-dd45f2661dc9.localdomain ESMTP
ehlo localhost
250-135d7c87-8fb4-4f4e-8e42-dd45f2661dc9.localdomain
250-PIPELINING
250-SIZE 51200000
250-VRFY
250-ETRN
250-AUTH PLAIN LOGIN
250-AUTH=PLAIN LOGIN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250-DSN
250 CHUNKING
auth login
334 VXNlcm5hbWU6
dGVzdEBleGFtcGxlLmNvbQ==
334 UGFzc3dvcmQ6
cGFzc3dvcmQ=
235 2.7.0 Authentication successful

One issue is that the cy2-sql package is not multi-mysql aware, so it will always be built against whatever the default MySQL implementation is for the branch, so in the case of 2023Q4 that is mariadb 10.6. I'll look at fixing this, so that I can also test against MySQL 5.7.

In the meantime please provide details of how the build against MySQL 5.7 is failing for you. Thanks.

[jperkin]

FWIW I just built custom versions of cy2-sql and postfix-mysql that explicitly link against mysql-client-5.7, as well as rebuilding mysql 5.7 with the charset fix, and that is able to authenticate successfully too.

[xmerlin]

I confirm that the charset fix works. It would probably be advisable to modify cy2-sql to make it multi-MySQL aware, so that people can install postfix-mysql/cy2-sql and Percona MySQL on the same machine. As soon as I find five minutes, I will investigate whether the fix also resolves the MySQL client crash in the Percona cluster.

thank you

[xmerlin]

I just tested MySQL Percona 8.0.36-28 and can confirm that after changing the encoding, the MySQL client no longer crashes.