From e73acf9eb7bf91aab1a5ed6991571166e5a222c3 Mon Sep 17 00:00:00 2001 From: Stuart Page <38261603+stuartjohnpage@users.noreply.github.com> Date: Mon, 26 Feb 2024 11:46:03 -0600 Subject: [PATCH] tweaked readme again --- README.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 3424109..5f1ef57 100644 --- a/README.md +++ b/README.md @@ -1,11 +1,18 @@ # Chunker: Flexible Text Chunking for Elixir -![tests](https://github.com/revelrylabs/text_chunker_ex/actions/workflows/test.yml/badge.svg) - +## About Chunker is an Elixir library for segmenting large text documents, optimizing them for efficient embedding and storage within vector databases for use in resource augmented generation (RAG) applications. It prioritizes context preservation and adaptability, and is therefore ideal for analytical, NLP, and other applications where understanding the relationship between text segments is crucial. +## Motivation + +Fill the gap in the Elixir ecosystem for a good semantic text chunker, and give access to langchain-style chunking. + +## Build Status + +![tests](https://github.com/revelrylabs/text_chunker_ex/actions/workflows/test.yml/badge.svg) + ## Key Features - Semantic Chunking: Prioritizes chunking text into meaningful blocks based on separators relevant to the specified format (e.g., headings, paragraphs in Markdown).