Why is There No MySQL Write Support?

[gengteng]

Hello,

I am developing a data synchronization tool that uses datafusion, sqlx, and clickhouse-rs to support read and write operations for MySQL, PostgreSQL, and ClickHouse. I found the datafusion-table-providers crate very promising and would like to consider replacing my current implementation with it.

However, I noticed that while it supports reading from MySQL, there is no implementation for writing (e.g., INSERT or UPDATE). Additionally, I couldn't find support for ClickHouse.

Can you explain why MySQL write support is missing? Is there any plan to add support for ClickHouse as well?

Thanks.

[phillipleblanc]

In general, the reason a feature or functionality or specific provider in this crate is missing is because no one has needed it yet. I would happily review a PR to add write functionality to MySQL.

For clickhouse see #9

[gengteng]

Hi @phillipleblanc ! 😄 Thank you for your response! I’ve decided to revisit this issue and implement the MySQL write support based on the PostgreSQL implementation. I plan to submit a PR soon and would love to get your feedback. Looking forward to your guidance and collaboration!

[phillipleblanc]

Awesome! I'll review the PR once you have submitted it.

[gengteng]

Hi @phillipleblanc 😄,

I’ve now completed the development and testing for the MySQL write support (#134). The implementation includes handling varchar and json types, which are mapped to DataFusion's LargeUtf8 type, and try_cast_to conversion is updated to handle LargeString.

I’ve also added test cases, and using my local version of datafusion-federation (with datafusion-contrib/datafusion-federation#71), the integration tests pass successfully.

Please take a look at the PR and let me know if there are any adjustments or improvements needed.

Thanks for your time, and looking forward to your review!