Handle many plates

[nlarusstone]

Problem

If there are many plates in a single file, we struggle. We do fall back to a longer context model:

llm = ChatOpenAI(temperature=0, model="gpt-4").with_fallbacks(
    [ChatOpenAI(temperature=0, model="gpt-3.5-turbo-16k")]
)

but GPT-3.5 is worse at this task and it still has a token limit. In addition, performance seems to degrade if there are a lot of plates.

See examples/MultiPlateMetadata.xlsx for an example of a file (with 13 96-well plates) where we would like to be able to call something like the following:

res = parse_plates(df, num_plates=13, num_rows=8, num_cols=12)

Suggested avenues of exploration

• Write a custom text splitter (see an example here: https://python.langchain.com/docs/modules/data_connection/document_transformers/text_splitters/recursive_text_splitter)