DMatrix/Booster has not been intialized or has already been disposed

[lubushyn]

Good time of the day community. I have an issue where I need your help.
I have a model:

import bentoml
from bentoml.frameworks.sklearn import SklearnModelArtifact
from bentoml.adapters import DataframeInput
import pandas as pd
import xgboost as xgb
from xgboost.core import DMatrix


@bentoml.env(infer_pip_packages=True)
@bentoml.artifacts([SklearnModelArtifact('model')])
class RentalPredictionXgBoost(bentoml.BentoService):
    @bentoml.api(input=DataframeInput(), batch=True)
    def predict(self, df: pd.DataFrame):
        datamatrix = DMatrix(data=df)
        return self.artifacts.model.predict(datamatrix)

And packer:

from prediction_service import RentalPredictionXgBoost 

bento_service = RentalPredictionXgBoost()
bento_service.pack('model', clf._Booster)
saved_path = bento_service.save()
saved_path

When I call /predict endpoint I have next error:

Traceback (most recent call last):
  File "/Users/oliubushyn/opt/anaconda3/envs/AI/lib/python3.9/site-packages/bentoml/service/inference_api.py", line 176, in wrapped_func
    return self._user_func(*args, **kwargs)
  File "/Users/oliubushyn/bentoml/repository/RentalPredictionXgBoost/20211005185837_EE3C19/RentalPredictionXgBoost/prediction_service.py", line 16, in predict
    return self.artifacts.model.predict(datamatrix)
  File "/Users/oliubushyn/opt/anaconda3/envs/AI/lib/python3.9/site-packages/xgboost/core.py", line 1485, in predict
    self._validate_features(data)
  File "/Users/oliubushyn/opt/anaconda3/envs/AI/lib/python3.9/site-packages/xgboost/core.py", line 2046, in _validate_features
    if self.feature_names != data.feature_names:
  File "/Users/oliubushyn/opt/anaconda3/envs/AI/lib/python3.9/site-packages/xgboost/core.py", line 796, in feature_names
    _check_call(_LIB.XGDMatrixGetStrFeatureInfo(self.handle,
  File "/Users/oliubushyn/opt/anaconda3/envs/AI/lib/python3.9/site-packages/xgboost/core.py", line 189, in _check_call
    raise XGBoostError(py_str(_LIB.XGBGetLastError()))
xgboost.core.XGBoostError: [18:58:54] /opt/concourse/worker/volumes/live/7a2b9f41-3287-451b-6691-43e9a6c0910f/volume/xgboost-split_1619728204606/work/src/c_api/c_api.cc:301: DMatrix/Booster has not been intialized or has already been disposed.
Stack trace:
  [bt] (0) 1   libxgboost.dylib                    0x0000000134df20fe dmlc::LogMessageFatal::~LogMessageFatal() + 110
  [bt] (1) 2   libxgboost.dylib                    0x0000000134de59cf XGDMatrixGetStrFeatureInfo + 111
  [bt] (2) 3   libffi.7.dylib                      0x000000010f2bbead ffi_call_unix64 + 85
  [bt] (3) 4   ???                                 0x0000000309612b70 0x0 + 13042264944

Could you please assist me what's going on?

[aarnphm]

Hi @lubushyn, I don't think that is the correct way of accessing the Booster from a clf. I assume you are currently using Sklearn classifier from XgBoost right? Also can you provide an example of your serving code?

[lubushyn]

@aarnphm I even did

from prediction_service import RentalPredictionXgBoost 

bento_service = RentalPredictionXgBoost()
bento_service.pack('model', clf._Booster)
saved_path = bento_service.save()
saved_path

I am serving with bentoml like

bentoml serve RentalPredictionXgBoost:latest

It is Sklearn regressor from XgBoost.

import xgboost as xgb
clf = xgb.XGBRegressor(
                    max_depth=50,
                    learning_rate=0.02,
                    gamma=0.02,
                    min_child_weight=1.0,
                    max_delta_step=0.0,
                    subsample=1.0,
                    colsample_bytree=1.0,
                    colsample_bylevel=1.0,
                    reg_alpha=0.0,
                    reg_lambda=1.0,
                    n_estimators=500,
                    verbosity=1,
                    n_jobs=4,
                    scale_pos_weight=1.0,
                    base_score=0.5,
                    random_state=1337,
                    missing=None,
                    objective='reg:squarederror',
                    
                  )

[aarnphm]

Currently we don't have support for regressor type for XgBoost. With the upcoming 1.0 releases, we are doing major redesign of the codebase, and was considering to support more features for different frameworks. An ideal example is to support Dask and Sklearn model from XgBoost. Thus once the releases is out it would be great if you can contribute on supporting these features from XgBoost.

have you tried just to pack the regressor ask the model inside the service and see if that works?

[lubushyn]

I did not try yet. Let me try. Seems should not work.

[lubushyn]

@aarnphm Can you please point me out in documentation where this limitation is described? As I understood correctly no way to serve XGBoost Regressor with BentoML for now.

[aarnphm]

BentoML/bentoml/frameworks/xgboost.py

Line 86 in b19def0

bst = xgb.Booster()

for our current implementation of XgBoost. You can also check out our bentoml-1.0 branch as well.

https://docs.bentoml.org/en/latest/frameworks.html#bentoml.frameworks.xgboost.XgboostModelArtifact only shows example from booster that is acquired from xgboost.train(), which returns a Booster instance.