Skip to content

Commit

Permalink
feat(evals): async evals + cookbook using deployments
Browse files Browse the repository at this point in the history
  • Loading branch information
jalexanderII committed Mar 13, 2024
1 parent f54b3ba commit 3345a9d
Showing 1 changed file with 13 additions and 0 deletions.
13 changes: 13 additions & 0 deletions src/cookbook/tracing_with_fetched_deployed_prompt.ts
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,19 @@ async function llmJudgeEval(log: Log): Promise<number> {
}

const _ragTemplate = async (deployment_id: string, llm_inputs: Record<string, any>): Promise<string> => {
// The Deployed Prompt is:
// Use the following pieces of context to answer the question. Do not make up an answer if no context is provided to help answer it.
//
// Context:
// ---------
// {{context}}
//
// ---------
// Question: {{question}}
// ---------
//
// Answer:

const deployedPrompt: UseDeployedPromptResponse = await getPrompt(deployment_id, llm_inputs);
const response = await callOpenAI(
deployedPrompt.prompt?.messages,
Expand Down

0 comments on commit 3345a9d

Please sign in to comment.