Add token metrics: model_input_tokens and model_output_tokens

[luohua13]

No description provided.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[RobertSamoilescu]

Hi @luohua13,

Thank you so much for opening this PR. It is indeed a useful metric to expose when working with LLMs. What I would like to suggest to abstract away the use of Prometheus and use the custom metrics as presented in the docs here. The only issue with the custom metrics approach is that MLServer currently supports only histogram type of metrics as seen here. If you think that a histogram might not be appropriate, it would be great if you consider adding support for Counters as well.

Other things to consider, although those might be already addressed, are:

• ensure that this PR does not affect or generalizes across other pipeline tasks (e.g., classification, qa, etc)
• padding - when performing batch inference, some inputs might be padded (how do we count the tokens in this case)

Please also add a brief description of what this PR is trying to solve, what your approach is, etc.

Thank you again for your contribution!