Skip to content

Commit

Permalink
tweaked readme again
Browse files Browse the repository at this point in the history
  • Loading branch information
stuartjohnpage committed Feb 26, 2024
1 parent 4ec888a commit e73acf9
Showing 1 changed file with 9 additions and 2 deletions.
11 changes: 9 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,18 @@
# Chunker: Flexible Text Chunking for Elixir

![tests](https://github.com/revelrylabs/text_chunker_ex/actions/workflows/test.yml/badge.svg)

## About
Chunker is an Elixir library for segmenting large text documents, optimizing them for efficient embedding and storage within vector databases for use in resource augmented generation (RAG) applications.

It prioritizes context preservation and adaptability, and is therefore ideal for analytical, NLP, and other applications where understanding the relationship between text segments is crucial.

## Motivation

Fill the gap in the Elixir ecosystem for a good semantic text chunker, and give access to langchain-style chunking.

## Build Status

![tests](https://github.com/revelrylabs/text_chunker_ex/actions/workflows/test.yml/badge.svg)

## Key Features

- Semantic Chunking: Prioritizes chunking text into meaningful blocks based on separators relevant to the specified format (e.g., headings, paragraphs in Markdown).
Expand Down

0 comments on commit e73acf9

Please sign in to comment.