SNOW-902662: DF.to_pandas_batches() batch size parameter #1027
Assignees
Labels
feature
New feature or request
status-triage_done
Initial triage done, will be further handled by the driver team
Comments
ghost
added
the
github-actions
bot
changed the title
|
Hi @MarcoFreitas0 , I would like to have a look on this issue! |
|
I am also interested in this feature |
sfc-gh-ashahi
added
the
status-triage_done
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Current behaviour
When using
DataFrame.to_pandas_batches()it returns a Pandas Dataframe Iterator that generates Pandas Dataframes with a "random" number of rows.Desired behaviour
I would like to insert a parameter in the
to_pandas_batches()method, where I fix the number of rows for each Pandas Dataframe generated.How would this improve
snowflake-snowpark-python?This would be beneficial since the user would be able to control the chunk sizes to process and be sure that his processes don't get overloaded by the snowflake back-end's calculation of the number of rows.
The text was updated successfully, but these errors were encountered: