Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow testing records with sibling whitespace in SLT tests and add more string tests #13197

Merged

Conversation

@findepi findepi marked this pull request as draft October 31, 2024 08:47
@github-actions github-actions bot added the sqllogictest SQL Logic Tests (.slt) label Oct 31, 2024
@findepi

This comment was marked as resolved.

@findepi findepi force-pushed the findepi/add-empty-string-to-string-test-data-5c1dbd branch 2 times, most recently from 7d41bee to e64d6d0 Compare October 31, 2024 14:43
@findepi findepi changed the title Add empty string to string test data Allow testing records with sibling whitespace in SLT tests and add more LIKE predicate tests Oct 31, 2024
@findepi findepi changed the title Allow testing records with sibling whitespace in SLT tests and add more LIKE predicate tests Allow testing records with sibling whitespace in SLT tests and add more string tests Oct 31, 2024
@findepi findepi marked this pull request as ready for review October 31, 2024 14:45
@findepi findepi force-pushed the findepi/add-empty-string-to-string-test-data-5c1dbd branch 2 times, most recently from d4ea9ef to 1a3ac85 Compare October 31, 2024 14:51
@findepi
Copy link
Member Author

findepi commented Oct 31, 2024

@alamb @comphead ready for review

@findepi findepi force-pushed the findepi/add-empty-string-to-string-test-data-5c1dbd branch from 1a3ac85 to fa96eb3 Compare October 31, 2024 15:35
Copy link
Contributor

@comphead comphead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @findepi why tests have different results, excluding formatting

.collect::<Vec<_>>();
let actual = actual
.iter()
.map(|strs| strs.iter().join(" "))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we prob can do this in a sinlge map?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not quite sure what you are suggesting here

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm suggesting to have a single collection iteration instead of two

.map(|strs| strs.iter().join(" ").trim_end().to_owned())

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, we can do this in single map.
i placed this into two map() steps so that the common part of actual and expected processing looks the same and so that i can write a nice comment

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @findepi and @comphead - the code looks good to me. I have a suggestion about improving the docs but we could do that as a follow on PR too

.collect::<Vec<_>>();
let actual = actual
.iter()
.map(|strs| strs.iter().join(" "))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not quite sure what you are suggesting here

let expected = expected
.iter()
// Trailing whitespace from lines in SLT will typically be removed, but do not fail if it is not
// If particular test wants to cover trailing whitespace on a value,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is probably a good thing to add in the user documentation (aka https://github.com/apache/datafusion/blob/main/datafusion/sqllogictest/README.md)

I think it would be hard to find here for someone writing sqllogictests

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it feels so edge-case'y. but yes, we can add

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will make a follow on PR

Copy link
Contributor

@alamb alamb Nov 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@findepi
Copy link
Member Author

findepi commented Oct 31, 2024

Thanks @findepi why tests have different results, excluding formatting

@comphead please see https://github.com/apache/datafusion/pull/13197/commits
i separated commits to make it more visible. Some changes (formatting) are because of the first change, which adds support for trailing whitespace.
The other changes (new rows) are bcause i added more data to string test cases.
i could separate into two PRs, but then it would not be clear why i do the first change in the first place. it's just supportive for the second one.

@alamb
Copy link
Contributor

alamb commented Nov 1, 2024

Thanks @findepi and @comphead

@alamb alamb merged commit 5e53b63 into apache:main Nov 1, 2024
24 checks passed
@findepi findepi deleted the findepi/add-empty-string-to-string-test-data-5c1dbd branch November 1, 2024 19:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sqllogictest SQL Logic Tests (.slt)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants