Skip to content

Commit

Permalink
fix: test failure in test_fsl_packed_struct (#3227)
Browse files Browse the repository at this point in the history
The current main has test failure in `test_fsl_packed_struct` because I
introduced statistics gathering for `StructDataBlock`, but not
statistics gathering in `FixedSizeListDataBlock`.

To fix this test, I can add statistics gathering(in this case, only
`Stat::MaxLength` is needed) for `FixedSizeListDataBlock`.

I think the variants of `FixedSizeList's child datablock` can only be
either `fixed width datablock` or another `fixed size list`?

This PR however only disables the packed struct encoding test for `fixed
size list`.

----------

after reading
https://docs.rs/arrow-array/53.3.0/src/arrow_array/array/fixed_size_list_array.rs.html#133,
it looks like that the child of a `fixed size list` can be any array
type, which means to support `MaxLength` statistics for `fixed size list
data block`, we need to have `MaxLength` statistics for all other
`datablock` types.
  • Loading branch information
broccoliSpicy authored Dec 10, 2024
1 parent c4cb87a commit faf776d
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion rust/lance-encoding/src/encodings/physical/packed_struct.rs
Original file line number Diff line number Diff line change
Expand Up @@ -350,10 +350,12 @@ pub mod tests {
}

// the current Lance V2.1 `packed-struct encoding` doesn't support `fixed size list`.
// the current Lance V2.0 test is disabled for now as we don't have statistics for `FixedSizeList`
#[rstest]
#[test_log::test(tokio::test)]
async fn test_fsl_packed_struct(
#[values(LanceFileVersion::V2_0, /*LanceFileVersion::V2_1)*/)] version: LanceFileVersion,
#[values(/*LanceFileVersion::V2_0,*/ /*LanceFileVersion::V2_1)*/)]
version: LanceFileVersion,
) {
let int_array = Arc::new(Int32Array::from(vec![12, 13, 14, 15]));

Expand Down

0 comments on commit faf776d

Please sign in to comment.