2113 using compact incomplete on a library with dynamic schema with a named index can result in an unreadable index #2116
Conversation
G-D-Petrov
requested review from
alexowens90,
willdealtry and
poodlewars
as code owners
| auto new_df_field_index_count = new_df_descriptor.index().type() == IndexDescriptor::Type::EMPTY ? 0 : new_df_descriptor.index().field_count(); | ||
|
|
||
| // If either index is empty, we consider them to match | ||
| if (df_in_store_index_field_count == 0 || new_df_field_index_count == 0) { |
There was a problem hiding this comment.
This check is to accommodate the existing behavior around empty DFs and Series, both of which have essentially empty indexes, even though for series the types is RowCount, I think
|
The benchmarks are reporting ~30% performance degradation on the FinalizeStagedData benchmarks:
I think that this is due to the new check over all of the segments to make sure that the index names are the same, which was not done before. IGNORE THIS: The latest commit fixes this - 300ae92 |
| @@ -86,6 +86,31 @@ inline void check_normalization_index_match( | |||
| } | |||
| } | |||
|
|
|||
| inline bool index_names_match( | |||
There was a problem hiding this comment.
Can we add schema_checks.cpp and add this there?
|
I think it's worth adding a test for |
Reference Issues/PRs
Fixes #2113
What does this implement or fix?
Any other comments?
Checklist
Checklist for code changes...