validate numpy string dtype in bentoml's NumpyNdarray

[oogou11]

The validation of the string type in bentoml._internal.io_descriptors.numpy within the provided _to_proto_impl function fails. The mapping relationship provided by FIELDPB_TO_NPDTYPE_NAME_MAP is "string_value" -> "<U", but doesn't the dtype in numpy include character width? Therefore, in this piece of code:

fieldpb = npdtype_to_fieldpb_map()[obj.dtype]
dtypepb = npdtype_to_dtypepb_map(version=version)[obj.dtype]

an exception is always thrown. My usage is as follows:

import bentoml
from bentoml.io import NumpyNdarray, JSON
import numpy as np 

svc = bentoml.Service("example_bento_service")

@svc.api(input=JSON(), output=NumpyNdarray())
def predict(input_array: dict[int, str]) -> np.ndarray:
    data = [["zhang", "san"], ["li", "si"]]
    res = np.array(data, dtype="<U") 
    return res

the version is bentoml==1.3.3 and my test starting server command:

bentoml serve-grpc service.py:svc --port=6100  --api-workers=1

[TEHREEMZUBAIR]

The issue you're encountering relates to how BentoML handles numpy string arrays and their dtypes, particularly when using Unicode strings (dtype "<U"). In BentoML 1.3.3, the mapping between field types in the Protobuf definition (used by BentoML) and Numpy dtypes is limited, leading to problems when handling variable-length Unicode strings like those represented by "<U".

Numpy string types ("<U") indeed include a character width, which indicates how many characters can be stored in each element of the array. The issue you're encountering arises because BentoML’s internal logic fails to map variable-length Unicode dtypes to a compatible Protobuf field type.

Solution Steps:

1. Use Fixed Length for Unicode Strings (<Ulength): Instead of using a variable-length Unicode string dtype ("<U"), you can specify a fixed character width, like "<U5", where the number specifies the number of Unicode characters allowed in each string. This way, you can avoid the issue caused by variable-length strings.

2. Custom Serialization: If the above approach still doesn't solve the problem, another option is to convert your Numpy array to a list of strings (or a simpler format) and serialize it as JSON before returning it as output.

Example Fix:

Here’s how you can modify your predict function to specify the character width for the Unicode strings in the Numpy array:

import bentoml
from bentoml.io import NumpyNdarray, JSON
import numpy as np 

svc = bentoml.Service("example_bento_service")

@svc.api(input=JSON(), output=NumpyNdarray())
def predict(input_array: dict[int, str]) -> np.ndarray:
    # Set a fixed width for the Unicode strings
    data = [["zhang", "san"], ["li", "si"]]
    res = np.array(data, dtype="<U5")  # Specify character width of 5
    return res

Alternatively, if you want to stick to variable-length strings, you can serialize the Numpy array to a simpler format like JSON:

@svc.api(input=JSON(), output=JSON())
def predict(input_array: dict[int, str]) -> dict:
    # Use a list of lists to return data as JSON
    data = [["zhang", "san"], ["li", "si"]]
    return {"result": data}

In this case, you would change the output descriptor from NumpyNdarray to JSON.

Additional Note:

Ensure your input/output formats match between the API definition (@svc.api(input=JSON(), output=NumpyNdarray())) and the return type. If BentoML still throws an error, try checking the BentoML logs for more detailed information on the exception and ensure that both input and output formats are consistent.

[oogou11]

thank you for your support. I am currently using the custom serialization approach.