Can I invoke the service handler of model 2 from the service handler of model 1 ?

[jesudasvf]

DualModels_possibility.pdf
Is the question asked in the second slide in the pdf possible in bentoml ?

If yes, please guide us, if not please let us know otherwise.

thanks, Jesudas

[parano]

Hi @jesudasvf - yes certainly you can do that.

There is one example with multiple models in the doc that shows you can have multiple models bundled with one BentoService: https://docs.bentoml.org/en/latest/concepts.html#packaging-model-artifacts

import bentoml
from bentoml.adapters import DataframeInput
from bentoml.artifact import SklearnModelArtifact, XgboostModelArtifact

@bentoml.env(infer_pip_packages=True)
@artifacts([
    SklearnModelArtifact("model_a"),
    XgboostModelArtifact("model_b")
])
class MyPredictionService(bentoml.BentoService):

    @bentoml.api(input=DataframeInput(), batch=True)
    def predict(self, df):
        # assume the output of model_a will be the input of model_b in this example:
        df = self.artifacts.model_a.predict(df)

        return self.artifacts.model_b.predict(df)

Packaging models with it:

svc = MyPredictionService()
svc.pack('model_a', my_model_a)
svc.pack('model_b', my_model_b)
svc.save()

Here's another example from the gallery: https://github.com/bentoml/gallery/blob/master/keras/text-classification/keras-text-classification.ipynb

import pandas as pd
import numpy as np
from tensorflow import keras
from tensorflow.keras.preprocessing import sequence, text
from bentoml import api, env, BentoService, artifacts
from bentoml.artifact import KerasModelArtifact, PickleArtifact
from bentoml.adapters import JsonInput


max_features = 1000

@artifacts([
    KerasModelArtifact('model'),
    PickleArtifact('word_index')
])
@env(pip_packages=['tensorflow==1.14.0', 'numpy', 'pandas'])
class KerasTextClassificationService(BentoService):
   
    def word_to_index(self, word):
        if word in self.artifacts.word_index and self.artifacts.word_index[word] <= max_features:
            return self.artifacts.word_index[word]
        else:
            return self.artifacts.word_index["<UNK>"]
    
    def preprocessing(self, text_str):
        sequence = text.text_to_word_sequence(text_str)
        return list(map(self.word_to_index, sequence))
    
    @api(input=JsonInput(), batch=False)
    def predict(self, parsed_json):
        if type(parsed_json) == list:
            input_data = list(map(self.preprocessing, parsed_json))
        else: # expecting type(parsed_json) == dict:
            input_data = [self.preprocessing(parsed_json['text'])]

        input_data = sequence.pad_sequences(input_data,
                                            value=self.artifacts.word_index["<PAD>"],
                                            padding='post',
                                            maxlen=80)

        return self.artifacts.model.predict_classes(input_data)

Looking at your slides, I think you don't really need to invoke the endpoint(service handler) of the other model, but you can simply invoke the model from the same API endpoint. Here's the pseudo code that how this could work:

@artifacts([
   PyTorchModelArtifact('face_recognizer'),
   PyTorchModelArtifact('face_detector'),
])
class FaceRecognizerAndDetector(BentoService):

  @api(input=ImageInput(), batch=True):
  def recognize_face(self, image_list):
         recognized_faces = self.artifacts.face_recognizer.predict(image_list)
         return recognized_faces

  @api(input=ImageInput(), bath=True):
  def detect_face(self, image_list):
         recognized_faces = self.artifacts.face_recognizer.predict(image_list)
         detected_faces = self.artifacts.face_detector.predict(recognized_faces)
         return detected_faces

Above is definitely the recommended approach to this. Another alternative if you want to re-use the recognize_face method, is that you can invoke the other API callback function directly:

@artifacts([
   PyTorchModelArtifact('face_recognizer'),
   PyTorchModelArtifact('face_detector'),
])
class FaceRecognizerAndDetector(BentoService):

  @api(input=ImageInput(), batch=True):
  def recognize_face(self, image_list):
         recognized_faces = self.artifacts.face_recognizer.predict(image_list)
         return recognized_faces

  @api(input=ImageInput(), batch=True):
  def detect_face(self, image_list):
         recognized_faces = self.recognize_face(image_list)  # calls the API function defined above
         detected_faces = self.artifacts.face_detector.predict(recognized_faces)
         return detected_faces

And lastly if you only concern about online serving, and you may end up deploying the same service horizontally across many nodes, it might make sense to invoke the endpoint by actually sending an HTTP request to your deployed endpoint:

@artifacts([
   PyTorchModelArtifact('face_recognizer'),
   PyTorchModelArtifact('face_detector'),
])
class FaceRecognizerAndDetector(BentoService):

  @api(input=ImageInput(), batch=True):
  def recognize_face(self, image_list):
         recognized_faces = self.artifacts.face_recognizer.predict(image_list)
         return recognized_faces

  @api(input=ImageInput(), batch=True):
  def detect_face(self, image_list):
         recognized_faces = requests.post(url='localhost/recognize_face', data=image_list) # change URL to your actual deployed endpoint
         detected_faces = self.artifacts.face_detector.predict(recognized_faces)
         return detected_faces

[jesudasvf]

Thanks @parano for the detailed response, Will work on the same as guided and shall keep you updated. Thanks once again.