Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docs: update flagd provider specs #1432

Merged
merged 19 commits into from
Oct 30, 2024
Merged

docs: update flagd provider specs #1432

Changes from 1 commit
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
40b3f58
docs: update flagd provider specs
toddbaert Oct 28, 2024
db9d89d
fixup: lifecycle summary
toddbaert Oct 28, 2024
2e228f0
fixup: md lint
toddbaert Oct 28, 2024
2840d38
Update docs/reference/specifications/providers.md
toddbaert Oct 29, 2024
ef81f4a
Update docs/reference/specifications/providers.md
toddbaert Oct 29, 2024
7095d0f
Update docs/reference/specifications/providers.md
toddbaert Oct 29, 2024
a522c28
Update docs/reference/specifications/providers.md
toddbaert Oct 29, 2024
d416de1
Update docs/reference/specifications/providers.md
toddbaert Oct 29, 2024
4683011
Update docs/reference/specifications/providers.md
toddbaert Oct 29, 2024
460be57
Update docs/reference/specifications/providers.md
toddbaert Oct 29, 2024
3978df7
Update docs/reference/specifications/providers.md
toddbaert Oct 29, 2024
ba39b62
Update docs/reference/specifications/providers.md
toddbaert Oct 29, 2024
8366519
fixup: pr feedback
toddbaert Oct 29, 2024
17f79d5
fixup: FLAGD_TARGET_URI
toddbaert Oct 29, 2024
fe9ad0b
fixup: links
toddbaert Oct 29, 2024
60f6231
fixup: feedback from mike
toddbaert Oct 30, 2024
3d5ae9d
fixup: flags changed
toddbaert Oct 30, 2024
bb1da79
Merge branch 'main' into docs/flagd-spec-updates
beeme1mr Oct 30, 2024
075efb7
fixup: flags changed++
toddbaert Oct 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
122 changes: 71 additions & 51 deletions docs/reference/specifications/providers.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,54 +13,16 @@ flagd providers are as essential as the flagd daemon itself, acting as the "brid
In fact, flagd providers may be the most crucial part of the flagd framework, as they can be used without an active flagd instance.
This document outlines their behavior and configuration.

## Naming

Consistent with our [naming conventions](../naming.md), the flagd provider name (no matter what language or runtime is in use) is `flagd`.

## In-Process vs RPC Evaluation

There are two modes of operation (`resolvers`) for flagd providers; _in-process_ and _RPC_.
Both modes have their advantages and disadvantages.
For more information on the architectural implications of these different modes, see the [RPC vs In-Process Evaluation](/architecture/#rpc-vs-in-process-evaluation) page.

## Configuration

### Configuration options

Most options can be defined in the constructor, or as environment variables, with constructor options having the highest
precedence.

Below are the supported configuration parameters (note that not all apply to both resolver modes):

| Option name | Environment variable name | Explanation | Type & Values | Default | Compatible resolver |
| ------------------ | -------------------------- | ------------------------------------------------------------------------------- | ---------------------------- | ----------------------------- | ------------------- |
| resolver | FLAGD_RESOLVER | mode of operation | String - `rpc`, `in-process` | rpc | rpc & in-process |
| host | FLAGD_HOST | remote host | String | localhost | rpc & in-process |
| port | FLAGD_PORT | remote port | int | 8013 (rpc), 8015 (in-process) | rpc & in-process |
| targetUri | FLAGD_TARGET_URI | alternative to host/port, supporting custom name resolution | string | null | rpc & in-process |
| tls | FLAGD_TLS | connection encryption | boolean | false | rpc & in-process |
| socketPath | FLAGD_SOCKET_PATH | alternative to host port, unix socket | String | null | rpc & in-process |
| certPath | FLAGD_SERVER_CERT_PATH | tls cert path | String | null | rpc & in-process |
| deadlineMs | FLAGD_DEADLINE_MS | deadline for unary calls, and timeout for initialization | int | 500 | rpc & in-process |
| streamDeadlineMs | FLAGD_STREAM_DEADLINE_MS | deadline for streaming calls, useful as an application-layer keepalive | int | 600000 | rpc & in-process |
| retryBackoffMs | FLAGD_RETRY_BACKOFF_MS | initial backoff for stream retry | int | 1000 | rpc & in-process |
| retryBackoffMaxMs | FLAGD_RETRY_BACKOFF_MAX_MS | maximum backoff for stream retry | int | 120000 | rpc & in-process |
| retryGraceAttempts | FLAGD_RETRY_GRACE_ATTEMPTS | amount of stream retry attempts before provider moves from STALE to ERROR state | int | 5 | rpc & in-process |
| keepAliveTime | FLAGD_KEEP_ALIVE_TIME_MS | http 2 keepalive | long | 0 | rpc & in-process |
| cache | FLAGD_CACHE | enable cache of static flags | String - `lru`, `disabled` | lru | rpc |
| maxCacheSize | FLAGD_MAX_CACHE_SIZE | max size of static flag cache | int | 1000 | rpc |
| selector | FLAGD_SOURCE_SELECTOR | selects a single sync source to retrieve flags from only that source | string | null | in-process |
| contextEnricher | - | sync-metadata to evaluation context mapping function | function | identity function | in-process |

### Custom Name Resolution

Some implementations support [gRPC custom name resolution](https://grpc.io/docs/guides/custom-name-resolution/), and abstractions to introduce additional resolvers.
Specifically, a custom resolver for `envoy` has been implemented in some providers, which overrides the authority header with the authority specified in the envoy URL scheme.
Below is an example of a custom target string which will use envoy sidecar proxy for name resolution:

```text
envoy://localhost:9211/flagd-sync.service
```

The custom name resolver provider in this case will use the endpoint name i.e. `flagd-sync.service` as [authority](https://github.com/grpc/grpc-java/blob/master/examples/src/main/java/io/grpc/examples/nameresolve/ExampleNameResolver.java#L55-L61)
and connect to `localhost:9211`.

## flagd Provider Lifecycle

flagd providers are built to adhere to the [provider lifecycle](https://openfeature.dev/specification/sections/flag-evaluation/#17-provider-lifecycle-management) defined in the OpenFeature specification.
Expand All @@ -73,18 +35,19 @@ The lifecycle is summarized below:
- if stream connection fails or exceeds the time specified by `deadline`, abort initialization (SDK will emit `PROVIDER_ERROR`), and attempt to [reconnect](#stream-reconnection)
- while connected:
- flags are resolved according to resolver mode; either by calling evaluation RPCs, or by evaluating the stored `flag set` rules
- for RPC providers, flags resolved with `reason=STATIC` are [cached](#flag-evaluation-caching)
- for RPC providers, flags resolved with `reason=STATIC` are [cached](#flag-evaluation-caching)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is under "while connected" so I think it's fine as is.

- if flags change the associated stream (event or sync) indicates flags have changed, flush cache, or update `flag set` rules respectively and emit `PROVIDER_CONFIGURATION_CHANGED`
- if stream disconnects:
- [reconnect](#stream-reconnection) with exponential backoff
- if reconnect attempt > `retryGraceAttempts`
- emit `PROVIDER_ERROR`
- RPC mode resolves `STALE` from cache where possible
- in-process mode resolves `STALE` from stored `flag set` rules
- if reconnect attempt <= `retryGraceAttempts`
- emit `PROVIDER_STALE`
- RPC mode resolves `STALE` from cache where possible
- in-process mode resolves `STALE` from stored `flag set` rules
- if reconnect attempt > `retryGraceAttempts`
- emit `PROVIDER_ERROR`
- RPC mode evaluation cache is purged
- RPC mode resolves `STALE` from cache where possible
- in-process mode resolves `STALE` from stored `flag set` rules
- on stream reconnection:
- emit `PROVIDER_READY` and `PROVIDER_CONFIGURATION_CHANGED`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should it emit PROVIDER_CONFIGURATION_CHANGED if we reconnect and the config did not change in the meantime?

Copy link
Member Author

@toddbaert toddbaert Oct 29, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ya, we don't know if the config has changed, since we could have missed change events, so we fire a change regardless to make sure any change handlers run; change handlers are only a hook to cause additional evaluations, so if no changes have actually happened, it's not problem (flag values will just be the same).

This is how the Java provider currently works. We could just only run READY in this case, but IMO that's risky since it is possible that a missed change event would never be detected and handlers which re-evaluate flags never run, so I consumer stays out of sync.

- in-process providers store the latest `flag set` rules
Expand Down Expand Up @@ -120,7 +83,7 @@ stateDiagram-v2

note right of ERROR
stream disconnected, attempting to reconnect,
evaluation cache invalidated*,
evaluation cache purged*,
ERROR emitted
end note

Expand Down Expand Up @@ -206,7 +169,9 @@ This pattern is consistent with OpenFeature's [static context paradigm](https://

flagd supports the OFREP protocol, meaning client-side OFREP providers can also be used for client-side use-cases.

### RPC Mode Provider Metadata
<!-- markdownlint-disable MD024 -->
### Provider Metadata
<!-- markdownlint-enable MD024 -->

The provider metadata includes properties returned from the [provider_ready event payload](./protos.md#eventstreamresponse) data.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any update mechanism?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need one, because right now we are counting on this being immutable for the lifetime of the remote flagd - so for it to change the remote would have to restart, meaning we would get a reconnect and we'd run the connect handler again which would update this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @beeme1mr on this.


Expand Down Expand Up @@ -248,6 +213,61 @@ In-process flagd providers also inject any properties returned by the [sync-meta
This allows for static properties defined in flagd to be added to in-process evaluations.
If only a subset of the sync-metadata response is desired to be injected into the evaluation context, you can define a mapping function with the `contextEnricher` option.

### In-Process Mode Provider Metadata
<!-- markdownlint-disable MD024 -->
### Provider Metadata
<!-- markdownlint-enable MD024 -->

The provider metadata includes the top-level metadata properties in the [flag definition](../flag-definitions.md).

### Offline (File) Mode

The in-process resolver mode can also use a file based [flag definition](../flag-definitions.md).
This does not connect to a flagd instance or gRPC sync implementation, and instead polls a flag definition from a file.

!!! note

This mode does not support [context enrichment via sync-metadata](#sync-metadata-properties-in-the-evaluation-context).

## Configuration

### Configuration options

Most options can be defined in the constructor, or as environment variables, with constructor options having the highest
precedence.

Below are the supported configuration parameters (note that not all apply to both resolver modes):

| Option name | Environment variable name | Explanation | Type & Values | Default | Compatible resolver |
| --------------------- | ------------------------------ | ------------------------------------------------------------------------------- | ---------------------------- | ----------------------------- | ------------------- |
| resolver | FLAGD_RESOLVER | mode of operation | String - `rpc`, `in-process` | rpc | rpc & in-process |
| host | FLAGD_HOST | remote host | String | localhost | rpc & in-process |
| port | FLAGD_PORT | remote port | int | 8013 (rpc), 8015 (in-process) | rpc & in-process |
| targetUri | FLAGD_TARGET_URI | alternative to host/port, supporting custom name resolution | string | null | rpc & in-process |
| tls | FLAGD_TLS | connection encryption | boolean | false | rpc & in-process |
| socketPath | FLAGD_SOCKET_PATH | alternative to host port, unix socket | String | null | rpc & in-process |
| certPath | FLAGD_SERVER_CERT_PATH | tls cert path | String | null | rpc & in-process |
| deadlineMs | FLAGD_DEADLINE_MS | deadline for unary calls, and timeout for initialization | int | 500 | rpc & in-process |
| streamDeadlineMs | FLAGD_STREAM_DEADLINE_MS | deadline for streaming calls, useful as an application-layer keepalive | int | 600000 | rpc & in-process |
| retryBackoffMs | FLAGD_RETRY_BACKOFF_MS | initial backoff for stream retry | int | 1000 | rpc & in-process |
| retryBackoffMaxMs | FLAGD_RETRY_BACKOFF_MAX_MS | maximum backoff for stream retry | int | 120000 | rpc & in-process |
| retryGraceAttempts | FLAGD_RETRY_GRACE_ATTEMPTS | amount of stream retry attempts before provider moves from STALE to ERROR state | int | 5 | rpc & in-process |
| keepAliveTime | FLAGD_KEEP_ALIVE_TIME_MS | http 2 keepalive | long | 0 | rpc & in-process |
| cache | FLAGD_CACHE | enable cache of static flags | String - `lru`, `disabled` | lru | rpc |
| maxCacheSize | FLAGD_MAX_CACHE_SIZE | max size of static flag cache | int | 1000 | rpc |
| selector | FLAGD_SOURCE_SELECTOR | selects a single sync source to retrieve flags from only that source | string | null | in-process |
| offlineFlagSourcePath | FLAGD_OFFLINE_FLAG_SOURCE_PATH | offline, file-based flag definitions, overrides host/port/targetUri | string | null | in-process |
| offlinePollIntervalMs | FLAGD_OFFLINE_POLL_MS | poll interval for reading offlineFlagSourcePath | int | 5000 | in-process |
| contextEnricher | - | sync-metadata to evaluation context mapping function | function | identity function | in-process |

### Custom Name Resolution

The provider metadata includes the top-level metadata property in the [flag definition](../flag-definitions.md).
Some implementations support [gRPC custom name resolution](https://grpc.io/docs/guides/custom-name-resolution/), and abstractions to introduce additional resolvers.
Specifically, a custom resolver for `envoy` has been implemented in some providers, which overrides the authority header with the authority specified in the envoy URL scheme.
Below is an example of a custom target string which will use envoy sidecar proxy for name resolution:

```text
envoy://localhost:9211/flagd-sync.service
```

The custom name resolver provider in this case will use the endpoint name i.e. `flagd-sync.service` as [authority](https://github.com/grpc/grpc-java/blob/master/examples/src/main/java/io/grpc/examples/nameresolve/ExampleNameResolver.java#L55-L61)
and connect to `localhost:9211`.
Loading