Flask/Gunicorn server api

[ghost]

Hi,

Hope you are all well !

I would like to setup a flask server with a query endpoint for processing a message.
Is it possible to push an example of gpt2bot with a flask server or gunicorn ?

In fact, I am writing a multi bot project (written in Golang) and I d like to aggregate response of several type of chatbots with an agent. The projet reference is there: https://github.com/paper2code/telegram-multibot

As I am much more a gopher than a pythonista, I am requesting your kind help on that. ^^

Can you help me on that ?

Cheers,
X

[polakowo]

You need to adapt interactive_bot.py by creating a route around the following lines

gpt2bot/gpt2bot/interactive_bot.py

Lines 35 to 66 in a091b7e

	# A single turn is a group of user messages and bot responses right after
	turn = {
	'user_messages': [],
	'bot_messages': []
	}
	turns.append(turn)
	turn['user_messages'].append(prompt)
	# Merge turns into a single history (don't forget EOS token)
	history = ""
	from_index = max(len(turns)-max_turns_history-1, 0) if max_turns_history >= 0 else 0
	for turn in turns[from_index:]:
	# Each turn begings with user messages
	for message in turn['user_messages']:
	history += message + tokenizer.eos_token
	for message in turn['bot_messages']:
	history += message + tokenizer.eos_token
	# Generate bot messages
	bot_messages = generate_response(
	model,
	tokenizer,
	history,
	config,
	mmi_model=mmi_model,
	mmi_tokenizer=mmi_tokenizer
	)
	if num_samples == 1:
	bot_message = bot_messages[0]
	else:
	# TODO: Select a message that is the most appropriate given the context
	# This way you can avoid loops
	bot_message = random.choice(bot_messages)

Having query prompt it will generate a bot message bot_message that you can send back to the user.

To create a Flask app for text generation you can look at my other project textai, for example this app. I'm currently preparing a new release of gpt2bot, so maybe after I finished that I will consider creating a Flask version.

[ghost]

Awesome mate :-)

When do you think you gonna release it ?

For the flask server, we managed to create one so no hurries ^^
https://github.com/paper2code/gpt2bot/blob/master/server.py

Cheers,
X

[polakowo]

Looks good. You would also need some sort of session management and ideally a route for updating the parameters of the generator such as temperature, no need to restart the server to update the config.

I will release it the next week I guess, I'm trying to decouple it from dialogpt and generalize it to accept any gpt2 model.

[ghost]

Hi @polakowo

I tried to create a new flask server from your latest changes with no luck...

Do you mind to create it ? That would be so awesome and kind of you :-)

Cheers,
X