Vietnamese string

[ductt-1167]

Hi all and thanks in advance!
I have a project need to use BentoML, It is very good to use, but when I try to use a Vietnamese string, for example:
@api(input=ImageInput(), batch=False)
def predict(self, image):

predictions = self.artifacts.model(image)

chars = """aAàÀảẢãÃáÁạẠăĂằẰẳẲẵẴắẮặẶâÂầẦẩẨẫẪấẤậẬbBcCdDđĐðÐeEèÈẻẺẽẼéÉẹẸêÊềỀểỂễỄếẾệỆfFgGhHiIìÌỉỈĩĨíÍịỊjJkKlLmMnNoOòÒỏỎõÕóÓọỌôÔồỒổỔỗỖốỐộỘơƠờỜởỞỡỠớỚợỢpPqQrRsStTuUùÙủỦũŨúÚụỤưƯừỪửỬữỮứỨựỰvVwWxXyYỳỲỷỶỹỸýÝỵỴzZ0123456789!"#$%&'()*+,-./:;<=>?@[\]^_`{|}~–°−“”—’ """
vocab = []
for i in chars:
    vocab.append(i)

text_list = []
predict_indices = np.argmax(predictions, axis=2)
for i in range(predict_indices.shape[0]):
    ans = ""
    merged_list = [k for k, _ in groupby(predict_indices[i])]
    for p in merged_list:
        if p != len(vocab):
            ans += str(int(p))+' '

    text_list.append(ans)

return text_list

And when I call
service = OCRModel()
service.pack('model', prediction_model)
service.save()

Error that:
Traceback (most recent call last):
File "E:/20203/Sun/Task eKYC/CRNN_OCR_BentoML/bento_packer.py", line 13, in
service.save()
File "E:\20203\Sun\Task eKYC\CRNN_OCR_BentoML\venv\lib\site-packages\bentoml\service_init_.py", line 749, in save
return yc.repository.upload(self, version, labels)
File "E:\20203\Sun\Task eKYC\CRNN_OCR_BentoML\venv\lib\site-packages\bentoml\yatai\client\bento_repository_api.py", line 196, in upload
save_to_dir(bento_service, tmpdir, version, silent=True)
File "E:\20203\Sun\Task eKYC\CRNN_OCR_BentoML\venv\lib\site-packages\bentoml\saved_bundle\bundler.py", line 246, in save_to_dir
_write_bento_content_to_dir(bento_service, path)
File "E:\20203\Sun\Task eKYC\CRNN_OCR_BentoML\venv\lib\site-packages\bentoml\saved_bundle\bundler.py", line 105, in _write_bento_content_to_dir
bento_service.class.module, os.path.join(path, bento_service.name)
File "E:\20203\Sun\Task eKYC\CRNN_OCR_BentoML\venv\lib\site-packages\bentoml\saved_bundle\local_py_modules.py", line 147, in copy_local_py_modules
finder.run_script(target_module_file)
File "F:\Python\lib\modulefinder.py", line 109, in run_script
self.load_module('main', fp, pathname, stuff)
File "F:\Python\lib\modulefinder.py", line 287, in load_module
co = compile(fp.read()+'\n', pathname, 'exec')
File "F:\Python\lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 1260: character maps to

when I try replace chars = 'abcdefghijklm...' (not unicode), not have erorr
Thanks!

[aarnphm]

hi @ductt-1167, I think this is a not a bentoml problem, rather than how to decode inputs with different unicode

[ductt-1167]

at what line did the error was thrown?

when I call
service = OCRModel()
service.pack('model', prediction_model)
service.save()

Error that:
Traceback (most recent call last):
File "E:/20203/Sun/Task eKYC/CRNN_OCR_BentoML/bento_packer.py", line 13, in
service.save()
File "E:\20203\Sun\Task eKYC\CRNN_OCR_BentoML\venv\lib\site-packages\bentoml\service_init_.py", line 749, in save
return yc.repository.upload(self, version, labels)
File "E:\20203\Sun\Task eKYC\CRNN_OCR_BentoML\venv\lib\site-packages\bentoml\yatai\client\bento_repository_api.py", line 196, in upload
save_to_dir(bento_service, tmpdir, version, silent=True)
File "E:\20203\Sun\Task eKYC\CRNN_OCR_BentoML\venv\lib\site-packages\bentoml\saved_bundle\bundler.py", line 246, in save_to_dir
_write_bento_content_to_dir(bento_service, path)
File "E:\20203\Sun\Task eKYC\CRNN_OCR_BentoML\venv\lib\site-packages\bentoml\saved_bundle\bundler.py", line 105, in _write_bento_content_to_dir
bento_service.class.module, os.path.join(path, bento_service.name)
File "E:\20203\Sun\Task eKYC\CRNN_OCR_BentoML\venv\lib\site-packages\bentoml\saved_bundle\local_py_modules.py", line 147, in copy_local_py_modules
finder.run_script(target_module_file)
File "F:\Python\lib\modulefinder.py", line 109, in run_script
self.load_module('main', fp, pathname, stuff)
File "F:\Python\lib\modulefinder.py", line 287, in load_module
co = compile(fp.read()+'\n', pathname, 'exec')
File "F:\Python\lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 1260: character maps to

when I try replace chars = 'abcdefghijklm...' (not unicode), not have erorr

[aarnphm]

if p != len(vocab):
                ans += str(int(p))+' '

I'm pretty sure you need to decode your unicode string here

[ductt-1167]

Oh I tried, but has same error, I try only declare chars = '"aAàÀảẢãÃáÁạẠăĂằẰ..' and not using it, has same error, so I think error in this line
chars = """aAàÀảẢãÃáÁạẠăĂằẰẳẲẵẴắẮặẶâÂầẦẩẨẫẪấẤậẬbBcCdDđĐðÐeEèÈẻẺẽẼéÉẹẸêÊềỀểỂễỄếẾệỆfFgGhHiIìÌỉỈĩĨíÍịỊjJkKlLmMnNoOòÒỏỎõÕóÓọỌôÔồỒổỔỗỖốỐộỘơƠờỜởỞỡỠớỚợỢpPqQrRsStTuUùÙủỦũŨúÚụỤưƯừỪửỬữỮứỨựỰvVwWxXyYỳỲỷỶỹỸýÝỵỴzZ0123456789!"#$%&'()*+,-./:;<=>?@[]^_`{|}~–°−“”—’ """

[aarnphm]

Also I think save_to_dir in older version have some limitatipn. We are currently working on a new release soon and I will try to address this issue. Thank tou so much for bringing it up.

[ductt-1167]

Also I think save_to_dir in older version have some limitatipn. We are currently working on a new release soon and I will try to address this issue. Thank tou so much for bringing it up.

I use window has this error, but my friend build in ubuntu not have error! ^^