Skip to content

Commit

Permalink
Create google-cloud-sql-sqlserver.md
Browse files Browse the repository at this point in the history
  • Loading branch information
@dyaffe
dyaffe authored Oct 13, 2023
1 parent 7fe8a62 commit 2e23bca
Showing 1 changed file with 209 additions and 0 deletions.
209 changes: 209 additions & 0 deletions ...s/reference/Connectors/materialization-connectors/google-cloud-sql-sqlserver.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,209 @@
# Google Cloud SQL for SQLServer

This connector materializes Flow collections into tables in a Microsoft SQLServer database.

It is available for use in the Flow web application. For local development or
open-source workflows,
[`ghcr.io/estuary/materialize-sqlserver:dev`](https://ghcr.io/estuary/materialize-sqlserver:dev) provides the latest version of the connector as a Docker image. You can also follow the link in your browser to see past image versions.

## Prerequisites

To use this connector, you'll need:

* A SQLServer database to which to materialize, and user credentials.
* SQLServer 2017 and later are supported
* The connector will create new tables in the database per your specification,
so user credentials must have access to create new tables.
* At least one Flow collection

## Configuration

To use this connector, begin with data in one or more Flow collections.
Use the below properties to configure a SQLServer materialization, which will direct one or more of your Flow collections to your desired tables, or views, in the database.

### Properties

#### Endpoint

| Property | Title | Description | Type | Required/Default |
|-----------------------------|------------------------|--------------------------------------------------------------------------------------------|--------|------------------|
| **`/database`** | Database | Name of the logical database to materialize to. | string | Required |
| **`/address`** | Address | Host and port of the database. If only the host is specified, port will default to `3306`. | string | Required |
| **`/password`** | Password | Password for the specified database user. | string | Required |
| **`/user`** | User | Database user to connect as. | string | Required |

#### Bindings

| Property | Title | Description | Type | Required/Default |
|---|---|---|---|---|
| `/additional_table_create_sql` | Additional Table Create SQL | Additional SQL statement(s) to be run in the same transaction that creates the table. | string | |
| `/delta_updates` | Delta Update | Should updates to this table be done via delta updates. | boolean | `false` |
| **`/table`** | Table | Table name to materialize to. It will be created by the connector, unless the connector has previously created it. | string | Required |

### Sample

```yaml
materializations:
${PREFIX}/${mat_name}:
endpoint:
connector:
image: ghcr.io/estuary/materialize-sqlserver:dev
config:
database: flow
address: localhost:5432
password: flow
user: flow
bindings:
- resource:
table: ${TABLE_NAME}
source: ${PREFIX}/${COLLECTION_NAME}
```
## Connecting to SQLServer
1. Allow connections to the database from the Estuary Flow IP address.
1. [Enable public IP on your database](https://cloud.google.com/sql/docs/sqlserver/configure-ip#add) and add
`34.121.207.128` as an authorized IP address. See the instructions below to use SSH Tunneling instead of enabling public access.

:::info
Alternatively, you can allow secure connections via SSH tunneling as described in the setup steps for
[self-hosted databases](#setup-self-hosted-sql-server).
:::

2. In your SQL client, connect to your instance as the default `sqlserver` user and issue the following commands.

```sql
USE <database>;
-- Create user and password for use with the connector.
CREATE LOGIN flow_materialize WITH PASSWORD = 'secret';
CREATE USER flow_materialize FOR LOGIN flow_materialize;
-- Grant control on the database to flow_materialize
GRANT CONTROL ON DATABASE::<database> TO flow_materialize;
```

3. In the Cloud Console, note the instance's host under Public IP Address. Its port will always be `1433`.
Together, you'll use the host:port as the `address` property when you configure the connector.

## Delta updates

This connector supports both standard (merge) and [delta updates](../../../concepts/materialization.md#delta-updates).
The default is to use standard updates.

## Reserved words

SQLServer has a list of reserved words that must be quoted in order to be used as an identifier.
Flow considers all the reserved words in the official [SQLServer documentation](https://learn.microsoft.com/en-us/sql/t-sql/language-elements/reserved-keywords-transact-sql?view=sql-server-2017).

These reserved words are listed in the table below. Flow automatically quotes fields that are in this list.

|Reserved words| | | | |
|---|---|---|---|---|
|absolute|connect|else|intersect|on|
|action|connection|end|intersection|only|
|ada|constraint|end-exec|interval|open|
|add|constraints|equals|into|opendatasourc|
|admin|constructor|errlvl|is|openquery|
|after|contains|escape|isolation|openrowset|
|aggregate|containstable|every|iterate|openxml|
|alias|continue|except|join|operation|
|all|convert|exception|key|option|
|allocate|corr|exec|kill|or|
|alter|corresponding|execute|language|order|
|and|count|exists|large|ordinality|
|any|covar_pop|exit|last|out|
|are|covar_samp|external|lateral|outer|
|array|create|extract|leading|output|
|as|cross|false|left|over|
|asc|cube|fetch|less|overlaps|
|asensitive|cume_dist|file|level|overlay|
|assertion|current|fillfactor|like|pad|
|asymmetric|current_catal|filter|like_regex|parameter|
|at|current_date|first|limit|parameters|
|atomic|current_defau|float|lineno|partial|
|authorization|current_path|for|ln|partition|
|avg|current_role|foreign|load|pascal|
|backup|current_schem|fortran|local|path|
|before|current_time|found|localtime|percent|
|begin|current_times|free|localtimestam|percent_rank|
|between|current_trans|freetext|locator|percentile_co|
|binary|current_user|freetexttable|lower|percentile_di|
|bit|cursor|from|map|pivot|
|bit_length|cycle|full|match|plan|
|blob|data|fulltexttable|max|position|
|boolean|database|function|member|position_rege|
|both|date|fusion|merge|postfix|
|breadth|day|general|method|precision|
|break|dbcc|get|min|prefix|
|browse|deallocate|global|minute|preorder|
|bulk|dec|go|mod|prepare|
|by|decimal|goto|modifies|preserve|
|call|declare|grant|modify|primary|
|called|default|group|module|print|
|cardinality|deferrable|grouping|month|prior|
|cascade|deferred|having|multiset|privileges|
|cascaded|delete|hold|names|proc|
|case|deny|holdlock|national|procedure|
|cast|depth|host|natural|public|
|catalog|deref|hour|nchar|raiserror|
|char|desc|identity|nclob|range|
|char_length|describe|identity_inse|new|read|
|character|descriptor|identitycol|next|reads|
|character_len|destroy|if|no|readtext|
|check|destructor|ignore|nocheck|real|
|checkpoint|deterministic|immediate|nonclustered|reconfigure|
|class|diagnostics|in|none|recursive|
|clob|dictionary|include|normalize|ref|
|close|disconnect|index|not|references|
|clustered|disk|indicator|null|referencing|
|coalesce|distinct|initialize|nullif|regr_avgx|
|collate|distributed|initially|numeric|regr_avgy|
|collation|domain|inner|object|regr_count|
|collect|double|inout|occurrences_r|regr_intercep|
|column|drop|input|octet_length|regr_r2|
|commit|dump|insensitive|of|regr_slope|
|completion|dynamic|insert|off|regr_sxx|
|compute|each|int|offsets|regr_sxy|
|condition|element|integer|old|regr_syy|
|relative|semanticsimil|structure|truncate|window|
|release|semanticsimil|submultiset|try_convert|with|
|replication|sensitive|substring|tsequal|within|group|
|restore|sequence|substring_reg|uescape|within|
|restrict|session|sum|under|without|
|result|session_user|symmetric|union|work|
|return|set|system|unique|write|
|returns|sets|system_user|unknown|writetext|
|revert|setuser|table|unnest|xmlagg|
|revoke|shutdown|tablesample|unpivot|xmlattributes|
|right|similar|temporary|update|xmlbinary|
|role|size|terminate|updatetext|xmlcast|
|rollback|smallint|textsize|upper|xmlcomment|
|rollup|some|than|usage|xmlconcat|
|routine|space|then|use|xmldocument|
|row|specific|time|user|xmlelement|
|rowcount|specifictype|timestamp|using|xmlexists|
|rowguidcol|sql|timezone_hour|value|xmlforest|
|rows|sqlca|timezone_minu|values|xmliterate|
|rule|sqlcode|to|var_pop|xmlnamespaces|
|save|sqlerror|top|var_samp|xmlparse|
|savepoint|sqlexception|trailing|varchar|xmlpi|
|schema|sqlstate|tran|variable|xmlquery|
|scope|sqlwarning|transaction|varying|xmlserialize|
|scroll|start|translate|view|xmltable|
|search|state|translate_reg|waitfor|xmltext|
|second|statement|translation|when|xmlvalidate|
|section|static|treat|whenever|year|
|securityaudit|statistics|trigger|where|zone|
|select|stddev_pop|trim|while|
|semantickeyph|stddev_samp|true|width_bucket|

## Changelog

The changelog includes a list of breaking changes made to this connector. Backwards-compatible changes are not listed.

**Proceed with caution when editing materializations created with previous versions of this connector;
editing always upgrades your materialization to the latest connector version.**

#### V1: 2023-09-01

- First version

0 comments on commit 2e23bca

Please sign in to comment.