Improve table data extraction from Confluence

[cocquerel]

Checked

• I searched existing ideas and did not find a similar one
• I added a very descriptive title
• I've clearly described the feature request and motivation for it

Feature request

Extract table data in a format that would be meaningful by LLM. One idea would be to format table data in JSON taking column headers as attribute names.

Motivation

I'm using the LangChain Confluence loader to extract data. I'm facing issue when the Confluence page is containing data in table. Indeed, the data is extracted line by line and we are missing the relationship between cells and column headers. Due to that, LLMs are not able to understand data correctly.

Proposal (If applicable)

No response

[MonoMarkor]

hello, did you find a way to solve this issue? In my expereince using llama 3.1 8b, it understands the table but the problem is the formatting of a single cell within a table when it contains text in new lines.
There is no new line or space between the text that are in the same cell but on different lines. There is nothing seperating the two lines of text. as soon as the string character of the first line ends the string of the second line starts

[cocquerel]

The issue is not with the LLM but with the LangChain Confluence loader that removes table formatting and so relationship between cell content and column header is removed before sending the extracted data to the LLM.
We are currently using a workaround that is a bit heavy process. It consist of adding some non-visible data using markdown in the Confluence page so that data extraction allows to generate meaningful sentences like shown in screenshot.

[aliyarly]

langchain expert, is there any plan for this use case?