Skip to content

Commit

Permalink
Docs: Dashboard updates; typo/format fixes (#1846)
Browse files Browse the repository at this point in the history
* update web app docs

* misc typo/format fixes
  • Loading branch information
aeluce authored Jan 3, 2025
1 parent 0594363 commit a618709
Show file tree
Hide file tree
Showing 16 changed files with 121 additions and 101 deletions.
4 changes: 2 additions & 2 deletions site/docs/concepts/advanced/evolutions.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,12 +53,12 @@ When you attempt to publish a breaking change to a collection in the Flow web ap

Click the **Apply** button to trigger an evolution and update all necessary specification to keep your Data Flow functioning. Then, review and publish your draft.

If you enabled [AutoDiscover](../captures.md#autodiscover) on a capture, any breaking changes that it introduces will trigger an automatic schema evolution, so long as you selected the **Breaking change re-versions collections** option(`evolveIncompatibleCollections`).
If you enabled [AutoDiscover](../captures.md#autodiscover) on a capture, any breaking changes that it introduces will trigger an automatic schema evolution, so long as you selected the **Breaking change re-versions collections** option (`evolveIncompatibleCollections`).

## What do schema evolutions do?

The schema evolution feature is available in the Flow web app when you're editing pre-existing Flow entities.
It notices when one of your edit would cause other components of the Data Flow to fail, alerts you, and gives you the option to automatically update the specs of these components to prevent failure.
It notices when one of your edits would cause other components of the Data Flow to fail, alerts you, and gives you the option to automatically update the specs of these components to prevent failure.

In other words, evolutions happen in the *draft* state. Whenever you edit, you create a draft.
Evolutions add to the draft so that when it is published and updates the active data flow, operations can continue seamlessly.
Expand Down
2 changes: 1 addition & 1 deletion site/docs/concepts/collections.md
Original file line number Diff line number Diff line change
Expand Up @@ -332,7 +332,7 @@ If desired, a derivation could re-key the collection
on `[/userId, /name]` to materialize the various `/name`s seen for a `/userId`.

This property makes keys less lossy than they might otherwise appear,
and it is generally good practice to chose a key that reflects how
and it is generally good practice to choose a key that reflects how
you wish to _query_ a collection, rather than an exhaustive key
that's certain to be unique for every document.

Expand Down
2 changes: 1 addition & 1 deletion site/docs/concepts/connectors.md
Original file line number Diff line number Diff line change
Expand Up @@ -219,7 +219,7 @@ sops:
```
You then use this `config.yaml` within your Flow specification.
The Flow runtime knows that this document is protected by `sops`
The Flow runtime knows that this document is protected by `sops`,
will continue to store it in its protected form,
and will attempt a decryption only when invoking a connector on your behalf.

Expand Down
7 changes: 4 additions & 3 deletions site/docs/concepts/derivations.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,8 +218,8 @@ into JSON arrays or objects and embeds them into the mapped document:
`{"greeting": "hello", "items": [1, "two", 3]}`.
If parsing fails, the raw string is used instead.

If you would like to select all columns of the input collection,
rather than `select *`, use `select JSON($flow_document)`, e.g.
If you would like to select all columns of the input collection,
rather than `select *`, use `select JSON($flow_document)`, e.g.
`select JSON($flow_document where $status = open;`.

As a special case if your query selects a _single_ column
Expand Down Expand Up @@ -608,6 +608,7 @@ Flow read delays are very efficient and scale better
than managing very large numbers of fine-grain timers.

[See Grouped Windows of Transfers for an example using a read delay](#grouped-windows-of-transfers)

[Learn more from the Citi Bike "idle bikes" example](https://github.com/estuary/flow/blob/master/examples/citi-bike/idle-bikes.flow.yaml)

### Read priority
Expand Down Expand Up @@ -639,7 +640,7 @@ For SQLite derivations,
the entire SQLite database is the internal state of the task.
TypeScript derivations can use in-memory states with a
recovery and checkpoint mechanism.
Estuary intends to offer an additional mechanisms for
Estuary intends to offer additional mechanisms for
automatic internal state snapshot and recovery in the future.

The exact nature of internal task states vary,
Expand Down
2 changes: 1 addition & 1 deletion site/docs/concepts/import.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ sidebar_position: 7
---
# Imports

When you work on a draft Data Flow [using `flowctl draft`](../concepts/flowctl.md#working-with-drafts),
When you work on a draft Data Flow [using `flowctl draft`](../guides/flowctl/edit-draft-from-webapp.md),
your Flow specifications may be spread across multiple files.
For example, you may have multiple **materializations** that read from collections defined in separate files,
or you could store a **derivation** separately from its **tests**.
Expand Down
31 changes: 17 additions & 14 deletions site/docs/concepts/materialization.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ You define and configure materializations in **Flow specifications**.
Materializations use real-time [connectors](./connectors.md) to connect to many endpoint types.

When you use a materialization connector in the Flow web app,
flow helps you configure it through the **discovery** workflow.
Flow helps you configure it through the **discovery** workflow.

To begin discovery, you tell Flow the connector you'd like to use, basic information about the endpoint,
and the collection(s) you'd like to materialize there.
Expand Down Expand Up @@ -67,7 +67,7 @@ materializations:
# Name of the collection to be read.
# Required.
name: acmeCo/example/collection
# Lower bound date-time for documents which should be processed.
# Lower bound date-time for documents which should be processed.
# Source collection documents published before this date-time are filtered.
# `notBefore` is *only* a filter. Updating its value will not cause Flow
# to re-process documents that have already been read.
Expand All @@ -93,11 +93,11 @@ materializations:
# Priority applied to documents processed by this binding.
# When all bindings are of equal priority, documents are processed
# in order of their associated publishing time.
#
#
# However, when one binding has a higher priority than others,
# then *all* ready documents are processed through the binding
# before *any* documents of other bindings are processed.
#
#
# Optional. Default: 0, integer >= 0
priority: 0

Expand Down Expand Up @@ -362,24 +362,27 @@ field implemented. Consult the individual connector documentation for details.
### How It Works

1. **Source Capture Level:**
- If the source capture provides a schema or namespace, it will be used as the default schema for all bindings in
- the materialization.

If the source capture provides a schema or namespace, it will be used as the default schema for all bindings in the materialization.

2. **Manual Overrides:**
- You can still manually configure schema names for each binding, overriding the default schema if needed.

You can still manually configure schema names for each binding, overriding the default schema if needed.

3. **Materialization-Level Configuration:**
- The default schema name can be set at the materialization level, ensuring that all new captures within that
- materialization automatically inherit the default schema name.

The default schema name can be set at the materialization level, ensuring that all new captures within that materialization automatically inherit the default schema name.

### Configuration Steps

1. **Set Default Schema at Source Capture Level:**
- When defining your source capture, specify the schema or namespace. If no schema is provided, Estuary Flow will
- automatically assign a default schema.

When defining your source capture, specify the schema or namespace. If no schema is provided, Estuary Flow will automatically assign a default schema.

2. **Override Schema at Binding Level:**
- For any binding, you can manually override the default schema by specifying a different schema name.

For any binding, you can manually override the default schema by specifying a different schema name.

3. **Set Default Schema at Materialization Level:**
- During the materialization configuration, set a default schema name for all captures within the materialization.

During the materialization configuration, set a default schema name for all captures within the materialization.
4 changes: 2 additions & 2 deletions site/docs/concepts/schemas.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ Flow can usually generate suitable JSON schemas on your behalf.

For systems like relational databases, Flow will typically generate a complete JSON schema by introspecting the table definition.

For systems that store unstructured data, Flow will typically generate a very minimal schema, and will rely on schema inferrence to fill in the details. See [continuous schema inferenece](#continuous-schema-inference) for more information.
For systems that store unstructured data, Flow will typically generate a very minimal schema, and will rely on schema inference to fill in the details. See [continuous schema inference](#continuous-schema-inference) for more information.

### Translations

Expand All @@ -72,7 +72,7 @@ Schema inference is also used to provide translations into other schema flavors:
### Annotations

The JSON Schema standard introduces the concept of
[annotations](http://json-schema.org/understanding-json-schema/reference/generic.html#annotations),
[annotations](https://json-schema.org/understanding-json-schema/reference/annotations),
which are keywords that attach metadata to a location within a validated JSON document.
For example, `title` and `description` can be used to annotate a schema with its meaning:

Expand Down
2 changes: 1 addition & 1 deletion site/docs/concepts/storage-mappings.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Flow tasks — captures, derivations, and materializations — use recovery logs
Recovery logs are an opaque binary log, but may contain user data.

The recovery logs of a task are always prefixed by `recovery/`,
so a task named `acmeCo/produce-TNT` would have a recovery log called `recovery/acmeCo/roduce-TNT`
so a task named `acmeCo/produce-TNT` would have a recovery log called `recovery/acmeCo/produce-TNT`

Flow prunes data from recovery logs once it is no longer required.

Expand Down
Loading

0 comments on commit a618709

Please sign in to comment.