Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ISSUE] Does Workspace client use route optimized endpoints #951

Open
moritzmeister opened this issue Jun 19, 2024 · 0 comments
Open

[ISSUE] Does Workspace client use route optimized endpoints #951

moritzmeister opened this issue Jun 19, 2024 · 0 comments

Comments

@moritzmeister
Copy link

Description
We have a bunch of route optimized model serving endpoints that we query using the go sdk.
Route optimized endpints have essentially two hosts, the regular workspace host and the host for the optimized routing.

When setting up the workspace client, we use the regular workspace host. So now we are wondering if the query method then actually uses the route optimized endpoint?

We saw that even for a route optimized endpoint it's possible to call the non-optimized host/endpoint.

Reproduction

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"io"
	"os"

	"github.com/databricks/databricks-sdk-go"
	"github.com/databricks/databricks-sdk-go/service/serving"
)

func main() {
	w := databricks.Must(databricks.NewWorkspaceClient(&databricks.Config{
		Host:         "https://host.cloud.databricks.com/",
		ClientSecret: "",
		ClientID:     "",
	}))

	// Open the JSON file
	file, err := os.Open("data.json")
	if err != nil {
		fmt.Println("Error opening file:", err)
		return
	}
	defer file.Close()

	// Read the file's content
	byteValue, err := io.ReadAll(file)
	if err != nil {
		fmt.Println("Error reading file:", err)
		return
	}

	// Unmarshal the JSON content into the struct
	var dataframe serving.DataframeSplitInput
	err = json.Unmarshal(byteValue, &dataframe)
	if err != nil {
		fmt.Println("Error unmarshalling JSON:", err)
		return
	}

	// Print the struct to verify the content
	fmt.Printf("%+v\n", dataframe)

	request := serving.QueryEndpointInput{
		Name:           "model_name",
		DataframeSplit: &dataframe,
	}

	response, err := w.ServingEndpoints.Query(context.Background(), request)
	if err != nil {
		panic(err)
	}
	fmt.Println(response.Predictions)
}

Expected behavior
Use optimized endpoint host when the model serving is route optimized.

Other Information

  • Version: 0.42.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant