-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support Avro Doc Comments materialize_sink_kafka
#373
Conversation
This is still WIP, so let me know if this is too deep too soon, but I think we should track the SQL syntax more closely here. I think you want something more like: format {
avro {
doc_on_type {
database_name = "..."
schema_name = "..."
type_name = "..."
doc = "..."
# At least one of key or value must be true.
key = {true | false}
value = {true | false}
}
doc_on_column {
database_name = "..."
schema_name = "..."
item_name = "..."
column_name = "..."
doc = "..."
# At least one of key or value must be true.
key = {true | false}
value = {true | false}
}
}
} |
materialize_sink_kafka
c.WriteString("VALUE ") | ||
} | ||
c.WriteString(fmt.Sprintf("DOC ON TYPE %[1]s = %[2]s", | ||
b.format.Avro.DocType.Object.QualifiedName(), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Based on docs, unsure if we need to also support aliases for the object reference. Or if it should just always make the connection.
@@ -285,6 +285,66 @@ func SinkFormatSpecSchema(elem string, description string, required bool) *schem | |||
Optional: true, | |||
ForceNew: true, | |||
}, | |||
"avro_doc_type": { | |||
Description: "**Private Preview** Add top level documentation comment to the generated Avro schemas.", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Open to better descriptions for all these.
The acceptance test failing is within Kafka Sources, not sinks. Must be due to some of the shared structs within |
Co-authored-by: Bobby Iliev <[email protected]>
Fixes #359
Add two new attributes to the
materialize_sink_kafka
,avro_doc_type
andavro_doc_column
to support Avro comments. Adds the following syntax to the SQL