TGI Server should be installable via pip

[cdoern]

Feature request

pip install text-generation-inference pip install text-generation-launcher. Having these as published packages will allow projects to use the serving and client functionality.

I am hitting a wall here, and currently am trying to clone TGI and use the serving functionality. This is proving to be pretty hard because the way the project is built and more focused on being run in a containerized application.

Motivation

Having a local LLM server that can handle batched requests would be amazing. llama-cpp-python is the only comparable embedded llm serving tool and it isn't built to really handle concurrent request like TGI is.

We are looking to use this in instructlab as a serving backend option if published to pypi or even locally installable via pip.

Your contribution

I could add the pyproject.toml and setup.py with some help around what parts of the project we'd want built into a python package!