LLM_COMPONENT_HANDLER.md

Writing LLM Validation & Customization Handlers

Table of contents

• Introduction
• LLM Validation & Customization Handler Structure
  • Using Javascript
  • Using Typescript
• Writing LLM Validation & Customization Handler Functions
• Supported Events

Introduction

A Large Language Model Validation & Customization Handler (LLMVCH) enables you to perform custom validations on the request and the response. In addition, it allows you to customize any bot message created by the invokeLLM component before it is sent to the user. Finally, you can implement a handler method that fires when the user selects the standard Submit action in a bot response generated by the invokeLLM component.

The LLMVCH is deployed as part of a component service, and is configured against an LLM service under skill settings in the configuration tab.

LLM Validation & Customization Handler Structure

Using Javascript

The transformation handler exports two objects: the metadata object that provides the name of the component and the eventHandlerType (which should be set to LlmComponent), and the handlers object that contains the event handler functions.

module.exports = {
  metadata: {
    name: 'myLlmComponentHandler',
    eventHandlerType: 'LlmComponent'
  },
  handlers: { 

    /**
    * Handler to validate the request payload
    * @param {ValidateRequestEvent} event
    * @param {LlmComponentContext} context - LLM context, see https://oracle.github.io/bots-node-sdk/LlmComponentContext.html
    * @returns {boolean} returns true when payload is valid
    */
    validateRequestPayload: async (event, context) => { 
      if (context.getCurrentTurn() === 1 && context.isJsonValidationEnabled()) {
        context.addJSONSchemaFormattingInstruction();
      }
      return true;
    },

    /**
    * Handler to validate the response payload
    * @param {ValidateResponseEvent} event 
    * @param {LlmComponentContext} context - LLM context, see https://oracle.github.io/bots-node-sdk/LlmComponentContext.html
    * @returns {boolean} returns true when payload is valid
    */
    validateResponsePayload: async (event, context) => { 
      let errors = event.allValidationErrors || [];
      if (errors.length > 0) {
        return context.handleInvalidResponse(errors);
      }
      return true;
    },

    /**
    * Handler to change the candidate bot messages that will be sent to the user
    * @param {ChangeBotMessagesLlmEvent} event 
    * @param {LlmComponentContext} context - LLM context, see https://oracle.github.io/bots-node-sdk/LlmComponentContext.html
    * @returns {NonRawMessage[]} returns list of bot messages
    */
    changeBotMessages: async (event, context) => { 
      return event.messages;
    },

    /**
    * Handler that fires when the Submit action is executed by the user.
    * Use this handler to add your custom logic to process the LLM response.
    * @param {SubmitEvent} event 
    * @param {LlmComponentContext} context - LLM context, see https://oracle.github.io/bots-node-sdk/LlmComponentContext.html
    */
    submit: async (event, context) => { 
    }
  }
}; 

If needed, you can define the metadata and handlers members as functions rather than as an objects.

Using TypeScript

In TypeScript, the event handler class implements the LlmComponentHandler interface. This interface requires both of the following methods:

• The metadata method that returns an object of type LlmComponentHandlerMetadata.
• The handlers method that returns an object of type LlmComponentHandlers.

import { LlmComponentContext, LlmComponentHandler, LlmComponentHandlers, LlmComponentHandlerMetadata, ValidateRequestEvent, ValidateResponseEvent
  , ChangeBotMessagesLlmEvent, SubmitEvent, NonRawMessage } from '@oracle/bots-node-sdk/typings/lib2';

export class MyComponentHandler implements LlmComponentHandler {

  public metadata(): LlmComponentHandlerMetadata {
    return { 
      name: 'myLlmComponentHandler',    
      eventHandlerType: 'LlmComponent'
      };
  }

  public handlers(): LlmComponentHandlers {
    return {

      /**
        * Handler to validate the request payload
        * @param {ValidateRequestEvent} event 
        * @param {LlmComponentContext} context - LLM context, see https://oracle.github.io/bots-node-sdk/LlmComponentContext.html
        * @returns {boolean} returns true when payload is valid
        */
      validateRequestPayload: async (event: ValidateRequestEvent, context: LlmComponentContext): Promise<boolean> => { 
        if (context.getCurrentTurn() === 1 && context.isJsonValidationEnabled()) {
          context.addJSONSchemaFormattingInstruction();
        }
        return true;
      },

      /**
      * Handler to validate the response payload
      * @param {ValidateResponseEvent} event 
      * @param {LlmComponentContext} context - LLM context, see https://oracle.github.io/bots-node-sdk/LlmComponentContext.html
      * @returns {boolean} returns true when payload is valid
      */
      validateResponsePayload: async (event: ValidateResponseEvent, context: LlmComponentContext): Promise<boolean> => { 
        let errors = event.allValidationErrors || [];
        if (errors.length > 0) {
          return context.handleInvalidResponse(errors);
        }
        return true;
      },

      /**
       * Handler to change the candidate bot messages that will be sent to the user
       * @param {ChangeBotMessagesLlmComponent} event 
       * @param {LlmComponentContext} context - LLM context, see https://oracle.github.io/bots-node-sdk/LlmComponentContext.html
       * @returns {NonRawMessage[]} returns list of bot messages
       */
      changeBotMessages: async (event: ChangeBotMessagesLlmEvent, context: LlmComponentContext): Promise<NonRawMessage[]> => { 
        return event.messages;
      },

     /**
      * Handler that fires when the Submit action is executed by the user.
      * Use this handler to add your custom logic to process the LLM response.
      * @param {SubmitEvent} event 
      * @param {LlmComponentContext} context - LLM context, see https://oracle.github.io/bots-node-sdk/LlmComponentContext.html
      */
      submit: async (event: SubmitEvent, context: LlmComponentContext): Promise<void> => { 
      }
     
    };
  }  
} 

Writing LLM Validation & Customization Handler Functions

The first argument of each event method is the event object. The properties available in this object depend on the type of event. See the list of supported entity events for information on which properties are available with which event.

The second argument of each event method is the context object. This object references the LlmComponentContext that provides access to convenience methods you can use to create your event handler logic.

TIP: if you are using a JavaScript IDE like Visual Studio Code, you can get code insight and code completion support by defining the types used in the event handlers as follows in your JavaScript handler file:

const { LlmComponentContext, ValidateRequestEvent, ValidateResponseEvent , ChangeBotMessagesLlmEvent, SubmitEvent
, NonRawMessage } = require ('@oracle/bots-node-sdk/typings/lib2');

When using TypeScript, you will automatically get code completion support if your IDE supports it.

Supported Events

The table below lists the event methods that can be implemented:

Event	Description	Event Properties
validateRequestPayload	A handler that can be used to validate the request body.	payload: The request body object.
validateResponsePayload	A handler that can be used to validate the response body.	payload: The response body object. validationEntities: List of entity names that is specified in the Validation Entities property of the associated LLM state in the visual flow diagrammer. entityMatches: key-value pairs with the entityName as key and a list of matches as value. This property is only set when the Validation Entities property is set in the LLM state. entityValidationErrors: key-value pairs with the entityName or composite bag item as key and an error message as value. This property is only set when the Validation Entities property is set, and there are missing entity matches or missing composite bag item matches when the entity is a composite bag entity. jsonValidationErrors: if the jsonValidation property is true, and the response is not a valid JSON object, this property contains a single entry with the error message stating the response is not a valid JSON object. If the response is valid JSON and the JSON Schema property is set in the LLM state, this property contains key-value pairs with the schema path as key and the schema validation error message as value, when the response doesn't comply with the schema. allValidationErrors: List of all entity validation errors and JSON validation errors.
changeBotMessages	A handler that can be used to customize the bot response generated by the LLM state.	messages: list of candidate bot messages. messageType: The type of bot message, the type can be one of the following: fullResponse: bot message sent when full LLM response has been received. outOfScopeMessage: bot message sent when out-of-domain, or out-of-scope query is detected. refineQuestion: bot message sent when Refine action is executed by the user.
submit	Handler that fires when the Submit standard action is executed by the user.	None