AssertionErrors after starting the SDG

[acsankar]

getting assertion error when starting the synthetic data generation

INFO 2024-11-10 08:00:58,565 instructlab.model.backends.llama_cpp:232: Starting server process, press CTRL+C to shutdown server...
INFO 2024-11-10 08:00:58,565 instructlab.model.backends.llama_cpp:233: After application startup complete see http://127.0.0.1:8000/docs for API.
Exception:
Traceback (most recent call last):
File "/Users/sankar/Desktop/Code/granite/test1/granite1/lib/python3.11/site-packages/llama_cpp/server/errors.py", line 171, in custom_route_handler
response = await original_route_handler(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/sankar/Desktop/Code/granite/test1/granite1/lib/python3.11/site-packages/fastapi/routing.py", line 301, in app
raw_response = await run_endpoint_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/sankar/Desktop/Code/granite/test1/granite1/lib/python3.11/site-packages/fastapi/routing.py", line 212, in run_endpoint_function
return await dependant.call(**values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/sankar/Desktop/Code/granite/test1/granite1/lib/python3.11/site-packages/llama_cpp/server/app.py", line 264, in create_completion
assert len(body.prompt) <= 1
^^^^^^^^^^^^^^^^^^^^^
AssertionError
Exception:
Traceback (most recent call last):
File "/Users/sankar/Desktop/Code/granite/test1/granite1/lib/python3.11/site-packages/llama_cpp/server/errors.py", line 171, in custom_route_handler
response = await original_route_handler(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/sankar/Desktop/Code/granite/test1/granite1/lib/python3.11/site-packages/fastapi/routing.py", line 301, in app
raw_response = await run_endpoint_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/sankar/Desktop/Code/granite/test1/granite1/lib/python3.11/site-packages/fastapi/routing.py", line 212, in run_endpoint_function
return await dependant.call(**values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/sankar/Desktop/Code/granite/test1/granite1/lib/python3.11/site-packages/llama_cpp/server/app.py", line 264, in create_completion
assert len(body.prompt) <= 1
^^^^^^^^^^^^^^^^^^^^^
AssertionError
Exception:
Traceback (most recent call last):

[bbrowning]

This will be fixed by #358 once it merges. Note that these error messages, while scary looking, should be harmless. We get these in all of our CI tests as well but the data generation still completes.