Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Internal] Parallel Hedging: Adds Docs #4082

Closed
Closed
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 101 additions & 0 deletions docs/Parallel Hedging Preview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# Paralell Hedging Preview

## Paralell Hedging

When Building a new `CosmosClient` there will be an option to include paralell hedging in that client.

```csharp
CosmosClient client = new CosmosClientBuilder("connection string")
.WithSpeculativeProcessing(type: SpeculationType.Threshold, speculativeThreshold: TimeSpan.FromMilliseconds(500))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove Speculative/Speculation term everywhere

This has burned us already with Walmart - for customers "speculation" sounds too random. In Java we have used the term ThresholdBasedAvailabilityStartegy or something along those lines.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like here
WithAvailabilityStrategy(AvailabilityStartegyType.Threshold) or such

.Build();
```

or

```csharp
CosmosClientOptions options = new CosmosClientOptions()
{
SpeculationOptions = new SpeculationOptions(type: SpeculationType.Threshold, speculativeThreshold: TimeSpan.FromMilliseconds(500))
};

CosmosClient client = new CosmosClient(
accountEndpoint: "account endpoint",
authKeyOrResourceToken: "auth key or resource token",
clientOptions: options);
```

The example above will create a `CosmosClient` instance with speculative processing enabled with at 500ms threhshold. This means that if a request takes longer than 500ms the SDK will send a new request to the backend in order of the Preferred Regions List. This process will repeat until a request comes back. The SDK will then return the first response that comes back from the backend. The threshold parameter is a required parameter can can be set to any value greater than 0. There are also methods that will allow a user to disable/enabled speculative processing after the client has been created. If you include speculative processing when creating the client it will be enabled by default.

```csharp
client.DisableSpeculativeProcessing(); // Disables speculative processing
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request level granular is also one ASK.
Assuming if we have that then are these APIs at Client scope still necessary?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 requestOptions level override definitely is needed - especially for queries (but not just)

client.EnableSpeculativeProcessing(); // Enables speculative processing
```

## Dynamic Preferred Regions

If you want to change the preferred regions for a `CosmosClient` you can do so by callinga new method.

```csharp
client.UpdatePreferredRegions(new List<string>() { "West US", "East US" });
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making this mutable is risky - in Java we have instead allowed specifying excluded regions (and made that mutable) as well as configurable per request options. Achieves the same - but hasn't the same implications of making preferred regions mutable everywhere

```

This will update the preferred regions for the client. The SDK will then use the new preferred regions for all new requests.

## Region Choice at Request Level

To choose what region a request will be routed to at a per request level there is a new option on the `RequestOptions` object.

```csharp
RequestOptions requestOptions = new RequestOptions()
{
LocationToRouteTo = "West US"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will it be a single Region or Equivalent to PReferredRegions?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again - why do the opposite of Java (we used excluded regions here) - region to route to as an override owuld allow routing to a region which is not even in preferred regions?

I think it just opens a can of worms

};

ItemResponse<MyItem> response = await container.ReadItemAsync<MyItem>("id", partitionKey, requestOptions);
```

```csharp
QueryRequestOptions requestOptions = new QueryRequestOptions()
{
LocationToRouteTo = "West US"
};

FeedIterator<MyItem> iterator = container.GetItemQueryIterator<MyItem>("SELECT * FROM c", requestOptions);
```

This will override any logic the SDK does to determine where to route requests and will route the request to the region specified in the `LocationToRouteTo` property.

## Region Specific Conistency Levels
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would be really curious why we would want to add this to public API ?


To set a region specific consistency level there is a option when creating a `CosmosClient` in `CosmosClientOptions`.

```csharp
CosmosClient client = new CosmosClientBuilder()
.WithPerRegionConsistencyLevels(
new ObservableCollection<string, ConsistencyLevel>()
{
{ "West US", ConsistencyLevel.Eventual },
{ "East US", ConsistencyLevel.Session }
})
.Build();
```

or

```csharp
CosmosClientOptions options = new CosmosClientOptions()
{
RegionConsistencyLevel = new ObservableCollection<string, ConsistencyLevel>()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow - why is this needed?

{
{ "West US", ConsistencyLevel.Eventual },
{ "East US", ConsistencyLevel.Session }
}
};

CosmosClient client = new CosmosClient(
accountEndpoint: "account endpoint",
authKeyOrResourceToken: "auth key or resource token",
clientOptions: options);
```

This will set the consistency level for the regions specified in the dictionary. If a region is not specified in the dictionary the SDK will use the default consistency level for the account.