General ticket for Array/List data type

[izveigor]

Is your feature request related to a problem or challenge?

This is a common ticket for all other tickets.
The ticket was created for the next purposes:

• Track the progress of implementing array functions
• Group issues by category
• Helps developers better understand problem selection

Describe the solution you'd like

Tickets:

Array creating:

• Сustomize behavior of NULLS in array functions #6808
• Supports NULL in arrays #6556
• Support empty array #6561
• make_array does not properly support nulls  #6887
• make_array can not form columns with lists #6993

Array functions:

• Unable to determine return type for function ArrayFill  #6779
• Support FixedSizeList in array methods #6560
• Implement unnest function #6555
• Column support for array functions #6804
• New array method array_contains #6557
• New functions for working with arrays #6119
• Add array expressions #6075
• Support array concat with different dimensions #6849
• array_contains returning unexpected values with column parameter #6972
• array_concat with arrays with different dimensions does not work properly #6992

Array operators:

• Array comparision #6806
• New concatenation operator for working with arrays #6603
• Array containment operator #6884

Cast:

• Cast between array elements #6558

Concept:

• [Epic] General ticket for the concept of the practical implementation of ARRAY #6980
• Implement array_has_all function #6973
• Implement array_element function #6974
• Implement array_slice function #6975
• Implement array_has_any function #6976
• Implement array_has function #6977
• Implement array_intersect function #6978
• Implement array_except function #6979
• Implement array_union function #6981

Other:

• Array functions should treat an array as an element #6985
• Implement AnyOp and AllOp operators #6602
• Write Signature method for list datatypes. #6559
• Improve performance of unnest even more #6961
• Improve GetIndexedField in protobuf #7146
• Regression: list index does not support with columns #7147

Discussions:

• The concept of practical implementation of the array #6855

Describe alternatives you've considered

Additional context

• [Epic] General ticket for the concept of the practical implementation of ARRAY #6980
• [EPIC] Improved support for nested / structured types (Struct , List, ListArray, and other Composite types) #2326
• Blog post with DataFusion Jun - Sep 2023 #6780

[ahirner]

Any idea whether aggregations on array elements are covered in this epic? I had no success so far:

> select max(unnest([1,3,5]));
type_coercion
caused by
Internal error: Unnest should be rewritten to LogicalPlan::Unnest before type coercion.
This was likely caused by a bug in DataFusion's code and we would welcome that you file an bug report in our issue tracker

[jayzhan211]

Yes! But since we are working on strengthening existing function, so not yet works on these.

…

On Fri, May 3, 2024, 11:50 AM Alexander Hirner ***@***.***> wrote: Any idea whether aggregations on array elements are covered in this epic? I had no success so far: > select max(unnest([1,3,5])); type_coercion caused by Internal error: Unnest should be rewritten to LogicalPlan::Unnest before type coercion. This was likely caused by a bug in DataFusion's code and we would welcome that you file an bug report in our issue tracker — Reply to this email directly, view it on GitHub <#6863 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADZCLR4DOSM7XTQZL2F5CDLZAMCQ7AVCNFSM6AAAAAA2AGNMUKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJSGA3TGOBQGY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>