Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(avro-schema): logical type support #19607

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions bin/configs/avro-schema.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,6 @@ generatorName: avro-schema
outputDir: samples/openapi3/schema/petstore/avro-schema
inputSpec: modules/openapi-generator/src/test/resources/3_0/avro-schema/petstore.yaml
templateDir: modules/openapi-generator/src/main/resources/avro-schema
additionalProperties:
useLogicalTypes: true
logicalTypeTimeQuantifier: nanos
2 changes: 2 additions & 0 deletions docs/generators/avro-schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,12 @@ These options may be applied as additional-properties (cli) or configOptions (pl
|ensureUniqueParams|Whether to ensure parameter names are unique in an operation (rename parameters that are not).| |true|
|enumUnknownDefaultCase|If the server adds new enum cases, that are unknown by an old spec/client, the client will fail to parse the network response.With this option enabled, each enum will have a new case, 'unknown_default_open_api', so that when the server sends an enum case that is not known by the client/spec, they can safely fallback to this case.|<dl><dt>**false**</dt><dd>No changes to the enum's are made, this is the default option.</dd><dt>**true**</dt><dd>With this option enabled, each enum will have a new case, 'unknown_default_open_api', so that when the enum case sent by the server is not known by the client/spec, can safely be decoded to this case.</dd></dl>|false|
|legacyDiscriminatorBehavior|Set to false for generators with better support for discriminators. (Python, Java, Go, PowerShell, C# have this enabled by default).|<dl><dt>**true**</dt><dd>The mapping in the discriminator includes descendent schemas that allOf inherit from self and the discriminator mapping schemas in the OAS document.</dd><dt>**false**</dt><dd>The mapping in the discriminator includes any descendent schemas that allOf inherit from self, any oneOf schemas, any anyOf schemas, any x-discriminator-values, and the discriminator mapping schemas in the OAS document AND Codegen validates that oneOf and anyOf schemas contain the required discriminator and throws an error if the discriminator is missing.</dd></dl>|true|
|logicalTypeTimeQuantifier|The quantifier for time-related logical types (`timestamp` and `local-timestamp`).|<dl><dt>**nanos**</dt><dd>nanoseconds</dd><dt>**micros**</dt><dd>microseconds</dd><dt>**millis**</dt><dd>milliseconds</dd></dl>|millis|
|packageName|package for generated classes (where supported)| |null|
|prependFormOrBodyParameters|Add form or body parameters to the beginning of the parameter list.| |false|
|sortModelPropertiesByRequiredFlag|Sort model properties to place required parameters before optional parameters.| |true|
|sortParamsByRequiredFlag|Sort method arguments to place required parameters before optional parameters.| |true|
|useLogicalTypes|Use logical types for fields, when matching OpenAPI types. Currently supported: `date-time`, `date`.| |false|

## IMPORT MAPPING

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
import java.util.Arrays;
import java.util.Collections;
import java.util.EnumSet;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
Expand All @@ -37,11 +38,32 @@
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import lombok.Getter;
import lombok.Setter;

public class AvroSchemaCodegen extends DefaultCodegen implements CodegenConfig {
private final Logger LOGGER = LoggerFactory.getLogger(AvroSchemaCodegen.class);
private static final String AVRO = "avro-schema";

/**
* See https://avro.apache.org/docs/++version++/specification/#logical-types
*/
public static final String USE_LOGICAL_TYPES = "useLogicalTypes";
public static final String USE_LOGICAL_TYPES_DESC = "Use logical types for fields, when matching OpenAPI types. Currently supported: `date-time`, `date`.";
/**
* See https://avro.apache.org/docs/++version++/specification/#timestamps
*/
public static final String LOGICAL_TYPES_TIME_QUANTIFIER = "logicalTypeTimeQuantifier";
public static final String LOGICAL_TYPES_TIME_QUANTIFIER_DESC = "The quantifier for time-related logical types (`timestamp` and `local-timestamp`).";

protected String packageName = "model";

@Getter @Setter
protected boolean useLogicalTypes = false; // this defaults to false for backwards compatibility

@Getter @Setter
protected String logicalTypeTimeQuantifier = "millis";

public AvroSchemaCodegen() {
super();

Expand Down Expand Up @@ -96,6 +118,15 @@ public AvroSchemaCodegen() {
typeMapping.put("BigDecimal", "string");

cliOptions.add(new CliOption(CodegenConstants.PACKAGE_NAME, CodegenConstants.PACKAGE_NAME_DESC));
cliOptions.add(CliOption.newBoolean(USE_LOGICAL_TYPES, USE_LOGICAL_TYPES_DESC).defaultValue(Boolean.FALSE.toString()));

CliOption logicalTimeQuantifier = new CliOption(LOGICAL_TYPES_TIME_QUANTIFIER, LOGICAL_TYPES_TIME_QUANTIFIER_DESC).defaultValue(this.getLogicalTypeTimeQuantifier());
Map<String, String> timeQuantifierOptions = new HashMap<>();
timeQuantifierOptions.put("nanos", "nanoseconds");
timeQuantifierOptions.put("micros", "microseconds");
timeQuantifierOptions.put("millis", "milliseconds");
logicalTimeQuantifier.setEnum(timeQuantifierOptions);
cliOptions.add(logicalTimeQuantifier);
}

@Override
Expand All @@ -109,6 +140,16 @@ public void processOpts() {
}

additionalProperties.put("packageName", packageName);

if (!convertPropertyToBooleanAndWriteBack(USE_LOGICAL_TYPES, this::setUseLogicalTypes)) {
// This sets the default if the option was not specified.
additionalProperties.put(USE_LOGICAL_TYPES, useLogicalTypes);
}

if (convertPropertyToStringAndWriteBack(LOGICAL_TYPES_TIME_QUANTIFIER, this::setLogicalTypeTimeQuantifier) == null) {
// This sets the default if the option was not specified.
additionalProperties.put(LOGICAL_TYPES_TIME_QUANTIFIER, logicalTypeTimeQuantifier);
}
}

@Override
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{{#useLogicalTypes}}{{#isDate}}{ "type": "int", "logicalType": "date" }{{/isDate}}{{#isDateTime}}{ "type": "long", "logicalType": "timestamp-{{{logicalTypeTimeQuantifier}}}" }{{/isDateTime}}{{^isDate}}{{^isDateTime}}"{{{dataType}}}"{{/isDateTime}}{{/isDate}}{{/useLogicalTypes}}{{^useLogicalTypes}}"{{{dataType}}}"{{/useLogicalTypes}}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would be great to format this better; if there is a way to use whitespace in the template and then make it collapse in the result, please let me know.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, that's changing the template operator, right? I basically wish I could change the string in this template to:

Suggested change
{{#useLogicalTypes}}{{#isDate}}{ "type": "int", "logicalType": "date" }{{/isDate}}{{#isDateTime}}{ "type": "long", "logicalType": "timestamp-{{{logicalTypeTimeQuantifier}}}" }{{/isDateTime}}{{^isDate}}{{^isDateTime}}"{{{dataType}}}"{{/isDateTime}}{{/isDate}}{{/useLogicalTypes}}{{^useLogicalTypes}}"{{{dataType}}}"{{/useLogicalTypes}}
{{#useLogicalTypes}}
{{#isDate}}
{
"type": "int",
"logicalType": "date"
}
{{/isDate}}
{{#isDateTime}}
{
"type": "long",
"logicalType": "timestamp-{{{logicalTypeTimeQuantifier}}}"
}
{{/isDateTime}}
{{^isDate}}
{{^isDateTime}}
"{{{dataType}}}"
{{/isDateTime}}
{{/isDate}}
{{/useLogicalTypes}}
{{^useLogicalTypes}}
"{{{dataType}}}"
{{/useLogicalTypes}}

and still have well-spaced output in the resulting schema. Something like a {{#collapseWhitespace}}...{{/collapseWhitespace}} operator that takes anything in its body and reduces all \w+ to

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nope. i don't think mustache comes with anything to do that

looks like avsc file are just JSON file so users can use a linter to auto format the auto-generated output as they do with other generators

Original file line number Diff line number Diff line change
@@ -1 +1 @@
{{^isEnum}}{{^isContainer}}{{#isPrimitiveType}}"{{dataType}}"{{/isPrimitiveType}}{{^isPrimitiveType}}"{{package}}.{{dataType}}"{{/isPrimitiveType}}{{/isContainer}}{{#isContainer}}{{>typeArray}}{{/isContainer}}{{/isEnum}}{{#isEnum}}{{>typeEnum}}{{/isEnum}}
{{^isEnum}}{{^isContainer}}{{#isPrimitiveType}}{{>dataType}}{{/isPrimitiveType}}{{^isPrimitiveType}}"{{package}}.{{dataType}}"{{/isPrimitiveType}}{{/isContainer}}{{#isContainer}}{{>typeArray}}{{/isContainer}}{{/isEnum}}{{#isEnum}}{{>typeEnum}}{{/isEnum}}
2 changes: 1 addition & 1 deletion samples/openapi3/schema/petstore/avro-schema/Order.avsc
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
},
{
"name": "shipDate",
"type": ["null", "string"],
"type": ["null", { "type": "long", "logicalType": "timestamp-nanos" }],
"doc": "",
"default": null
},
Expand Down
Loading