-
Notifications
You must be signed in to change notification settings - Fork 14.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(chart): download CSV chart contents from S3 for Athena charts #31485
base: master
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@william-fundrecs I assume this PR was mistakenly opened against upstream Superset, not your fork? |
@villebro No, this was functionality I thought would be useful to donate back to superset, we already make use of this in our presto Athena DB backed Superset instance. |
Can we do it without adding two feature flags, perhaps leaning on config? And do we need to single out S3? Could there be numerous other options for where people get/store their CSV? |
Potentially yes, it made sense to me to add the feature flag to enable the S3 feature but perhaps not as sensible to use a feature flag to hide the default CSV/XLSX right click options, if there is a better way to hide these I can change to it. This is coloured by our own needs, we wanted to hide the default options in favour of using the S3 download as it's a lot faster, our users will only be using full export and retrieving the file handle from S3 is multitudes faster than creating the dataframe and processing the file. S3 was specified as this is existing Athena functionality, it can be configured to automatically persist query results to a CSV inside a configured bucket. Potentially there are other use cases for other DB engines but we only use Athena Presto and PostGreSQL in-house. If someone wants to use this PR as a foundation for expanding file output locations that is fine with me. |
SUMMARY
Add a right click context menu option if the chart is an Athena table to download full chart contents directly from S3 bucket instead of through Superset
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
TESTING INSTRUCTIONS
Set required env vars
SUPERSET_REGION=<my_aws_region>
SUPERSET_WORKGROUP=<my_superset-workgroup>
SUPERSET_ATHENA_DB=<my_superset_db>
Enable feature flags
'DOWNLOAD_CSV_FROM_S3'
'SHOW_DEFAULT_CSV_OPTIONS'
Ensure Athena is set to automatically persist query results to S3 bucket in CSV format
Test download of chart through right click option.
ADDITIONAL INFORMATION
No DB changes
resolves #31482