Add boolean setting to include string size in schema discovery

[haleemur]

Description

Include a tap setting discover_string_field_size (happy to pick a different name for this field, this is the first thing that came to mind). Setting this to true would discovery string fields' types as

"<field_name>": {"type": ["string", "null"], "maxLength": X}"

where X, a positive integer, is the output of this query

select character_maximum_length 
from information_schema.columns 
where table_name = <table_name> 
  and table_schema = <table_schema>
  and column_name = <field_name>

Use Case

May targets map string types to a size-independent type (e.g. postgres: string, snowflake: string, bigquery: string). These targets can ignore the maxLength attribute.

Some targets have to map strings to varchar and specify a size. In these cases, it is necessary to know the maxLength

Default Behaviour & Default Value

The setting should be false by default to preserve existing behaviour. Users will have to specify this setting when configuring meltano.

[edgarrmondragon]

The SDK v0.41.0 will make this significantly easier fwiw: https://sdk.meltano.com/en/latest/guides/sql-tap.html#custom-type-mapping

Nevermind, it will actually make it the default 😅

meltano/sdk@4ce312f

[haleemur]

In that case, I'll close this issue.