Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

merge 1.6.1 into master #99

Merged
merged 7 commits into from
Jan 12, 2024
Merged

merge 1.6.1 into master #99

merged 7 commits into from
Jan 12, 2024

Conversation

bxparks
Copy link
Owner

@bxparks bxparks commented Jan 12, 2024

  • 1.6.1 (2024-01-12)
    • Bug Fix: Prevent amnesia that causes multiple type mismatches warnings
      • If a data set contains multiple records with a column which do not
        match each other, then the old code would remove the corresponding
        internal schema_entry for that column, and print a warning message.
      • This means that subsequent records would recreate the schema_entry,
        and a subsequent mismatch would print another warning message.
      • This also meant that if there was a second record after the most
        recent mismatch, the script would output a schema entry for the
        mismatching column, corresponding to the type of the last record which
        was not marked as a mismatch.
      • The fix is to use a tombstone entry for the offending column, instead
        of deleting the schema_entry completely. Only a single warning
        message is printed, and the column is ignored for all subsequent
        records in the input data set.
      • See
        [Issue#98](https://github.com/bxparks/bigquery-schema-generator/issues/98]
        which identified this problem which seems to have existed from the
        very beginning.

@bxparks bxparks merged commit 6405d35 into master Jan 12, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant