Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Statement leaks #158

Open
pimeys opened this issue May 31, 2021 · 3 comments
Open

Statement leaks #158

pimeys opened this issue May 31, 2021 · 3 comments

Comments

@pimeys
Copy link
Contributor

pimeys commented May 31, 2021

Hi. For some reason we haven't been getting this error until now, when user reported crashes due to reaching max_prepared_stmt_count. We typically set the LRU to 1000 statements for everybody, and up until now, nobody except one user have been able to trigger the bug.

I'm able to get the same error, but only when disabling the cache completely:

use mysql_async::{self as my, prelude::Queryable as _};

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let config = my::OptsBuilder::default()
        .stmt_cache_size(Some(0))
        .ip_or_hostname("localhost")
        .tcp_port(3308)
        .user(Some("root"))
        .pass(Some("prisma"))
        .db_name(Some("prisma"));

    let mut conn = my::Conn::new(config).await?;

    loop {
        let stmt = conn.prep("SELECT 1").await?;
        let _: Vec<my::Row> = conn.exec(&stmt, my::Params::Empty).await?;
    }
}

This code will increment the Prepared_stmt_count in show global status like '%stmt%'; very fast, and eventually crash the program when reaching the limit. If setting the statement cache size to 1, the bug doesn't happen.

But. There is a very probable chance that even when enabling the cache, we still leak statements in certain scenarios. In our user's case, the value is 1000 and there are probably thousands of different statements, we again reach the maximum and crash. I cannot reproduce the second case.

Here's more information in a separate issue: prisma/prisma#6872

@pimeys
Copy link
Contributor Author

pimeys commented May 31, 2021

This fixes the case where we disable the cache, but still use statements:

#159

The other leak I haven't yet found.

@pimeys
Copy link
Contributor Author

pimeys commented May 31, 2021

The user reports how these problems started after our January 5th release. I can read from our commits some changes after that how we use mysql_async:

  • Upgrade from 0.24.2 to 0.26.0 (and later 0.27.1)
  • Upgrade to Tokio 1.0
  • We stopped using the internal pool of this crate in favor of mobc

I'm sorry this is a bit hard to reproduce...

@pimeys pimeys closed this as completed Jun 1, 2021
@pimeys pimeys reopened this Jun 1, 2021
@pimeys
Copy link
Contributor Author

pimeys commented Jun 1, 2021

Ok, reopening this if there's a bug in the mysql_async LRU. I'm definitely just utilizing the LRU we use for tokio_postgres, so that's a good way for me to deal with this now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant