Docker Image Tag 4.1.1 is missing support for MySQL

[MVanDerlofske]

Bug description

When using the Docker Image tagged 4.1.1 it appears to be missing the ability to support MySQL as a datasource (among others).

Reproduce (performed on both my local system and a newly setup test system):

1. export TAG=4.1.1
2. docker compose -f docker-compose-image-tag.yml up
3. resolve dependency issues that arise (reported in other bug tickets)

Open the Superset interface and add a new database connection. You will note that the MySQL and Presto options are missing.

Screenshots/recordings

Screenshot of Add Database window on 4.1.1 tag:

Screenshot of editing an existing MySQL connection from an upgraded system:

Screenshot of expected add database window (taken for latest-dev 2024-12-14):

Screenshot of edit database connection window (taken for latest-dev 2024-12-14):

Superset version

4.1.1

Python version

I don't know

Node version

I don't know

Browser

Chrome

Additional context

Error callstack:
2024-12-14 16:11:45,537:warning:superset.views.error_handling:SupersetErrorsException
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1484, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1469, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File "/usr/local/lib/python3.10/site-packages/flask_appbuilder/security/decorators.py", line 95, in wraps
return f(self, *args, **kwargs)
File "/app/superset/views/base_api.py", line 119, in wraps
duration, response = time_function(f, self, *args, **kwargs)
File "/app/superset/utils/core.py", line 1364, in time_function
response = func(*args, **kwargs)
File "/app/superset/views/base_api.py", line 91, in wraps
return f(self, *args, **kwargs)
File "/app/superset/utils/log.py", line 303, in wrapper
value = f(*args, **kwargs)
File "/app/superset/sqllab/api.py", line 409, in execute_sql_query
command_result: CommandResult = command.run()
File "/app/superset/utils/decorators.py", line 266, in wrapped
return on_error(ex)
File "/app/superset/utils/decorators.py", line 236, in on_error
raise ex
File "/app/superset/utils/decorators.py", line 259, in wrapped
result = func(*args, **kwargs)
File "/app/superset/commands/sql_lab/execute.py", line 105, in run
status = self._run_sql_json_exec_from_scratch()
File "/app/superset/commands/sql_lab/execute.py", line 158, in _run_sql_json_exec_from_scratch
return self._sql_json_executor.execute(
File "/app/superset/sqllab/sql_json_executer.py", line 111, in execute
raise SupersetErrorsException(
superset.exceptions.SupersetErrorsException: [SupersetError(message="No module named 'MySQLdb'", error_type=<SupersetErrorType.GENERIC_DB_ENGINE_ERROR: 'GENERIC_DB_ENGINE_ERROR'>, level=<ErrorLevel.ERROR: 'error'>, extra={'engine_name': 'MySQL', 'issue_codes': [{'code': 1002, 'message': 'Issue 1002 - The database returned an unexpected error.'}]})]

Checklist

• I have searched Superset docs and Slack and didn't find a solution to my problem.
• I have searched the GitHub issue tracker and didn't find a similar bug report.
• I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

[dosubot]

Hello, @MVanDerlofske! I'm here to help you with any bugs, questions, or contributions you may have. Let's get this sorted out together!

The issue with the Docker Image tagged 4.1.1 for Superset not supporting MySQL as a datasource is due to the absence of the mysqlclient package, which is necessary for MySQL database connections. Starting from version 4.1.0, the Docker images do not include database drivers like those for MySQL or Postgres by default.

To resolve this, you need to manually install the mysqlclient package. Here's how you can do it:

1. Create a requirements-local.txt file in the docker directory of your project.
2. Add mysqlclient to this file.
3. Rebuild and restart your Docker containers using the following command:

   docker compose -f docker-compose-non-dev.yml up

If you're using a customized Docker image, you should rebuild your local image with the new driver included by running:

docker compose build --force-rm

Then, relaunch Superset.

For more detailed instructions, you can refer to the Superset documentation on installing database drivers in Docker images ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[MVanDerlofske]

attempts to install mysql manually failed, I tried using both pymysql and mysqlclient.
pymysql - installed successfully however the same issue presented itself both in the UI and in the error log.
mysqlclient failed to install due to missing dependencies which were not installable via the requirements file.

[sfirke]

Thank you for reporting. I agree with Dosu bot, I am hopeful that if you can get mysqlclient installed that will explain what's going on -- and I think it's bundled in the -dev releases so it makes sense that you don't see this on the latest dev image.

So far the best way I've seen to install packages with 4.1.1 is here, #30553 (comment) -- though I note they say this didn't work for mysqlclient. I think if we can get that debugged, that's the best way forward here. There is some discussion in other issues, like this one, about the issue of not bundling database drivers in the base image anymore.

[sfirke]

I'll mark this needs-more-info b/c I'd like to know that installing mysqlclient fixes this -- and ideally we link to documentation of how to do that successfully in a Dockerfile in 4.1.1. Then we can close it as a duplicate of other issues related to no longer including mysqlclient and psycopg2 in the base Superset image, which isn't really a bug per se.

[bjornjorgensen]

have the same issue when I use the helm chart

kubectl logs my-superset-bd8897d4c-rbql2 -n default

Defaulted container "superset" out of: superset, wait-for-postgres (init)
[2024-12-23 23:33:28 +0000] [8] [INFO] Starting gunicorn 22.0.0
[2024-12-23 23:33:28 +0000] [8] [INFO] Listening at: http://0.0.0.0:8088 (8)
[2024-12-23 23:33:28 +0000] [8] [INFO] Using worker: gthread
[2024-12-23 23:33:28 +0000] [9] [INFO] Booting worker with pid: 9
Loaded your LOCAL configuration at [/app/pythonpath/superset_config.py]
2024-12-23 23:33:30,178:ERROR:superset.app:Failed to create app

(...)
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/create.py", line 544, in create_engine
dbapi = dialect_cls.dbapi(**dbapi_args)
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 811, in dbapi
import psycopg2
ModuleNotFoundError: No module named 'psycopg2'
[2024-12-23 23:33:30 +0000] [9] [INFO] Worker exiting (pid: 9)
[2024-12-23 23:33:30 +0000] [8] [ERROR] Worker (pid:9) exited with code 3
[2024-12-23 23:33:30 +0000] [8] [ERROR] Shutting down: Master
[2024-12-23 23:33:30 +0000] [8] [ERROR] Reason: Worker failed to boot.