You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
If we do aggregations on data frames, we should avoid using aggregators that require an RDD (df.rdd.aggregate(...)). If possible, we should replace all these patterns by UDAFs, which will simplify the code a lot because we don't have to work with Row objects anymore.
Prerequisites
Spark version used supports UDAFs
The text was updated successfully, but these errors were encountered:
Description
If we do aggregations on data frames, we should avoid using aggregators that require an RDD (
df.rdd.aggregate(...)
). If possible, we should replace all these patterns by UDAFs, which will simplify the code a lot because we don't have to work with Row objects anymore.Prerequisites
The text was updated successfully, but these errors were encountered: