Skip to content

Commit

Permalink
Merge branch 'llmware-ai:main' into main
Browse files Browse the repository at this point in the history
  • Loading branch information
MacOS authored Mar 1, 2024
2 parents 5b8fd1a + 14104dd commit 04bfd8a
Show file tree
Hide file tree
Showing 26 changed files with 1,746 additions and 513 deletions.
10 changes: 5 additions & 5 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,25 +1,25 @@
FROM python:3.11-slim-bookworm
FROM python:3.11-bookworm

ARG USERNAME=llmware
ARG USER_UID=1000
ARG USER_GID=$USER_UID
ENV PYTHONPATH=/llmware


RUN apt-get update \
&& apt-get install -y --no-install-recommends git bash \
&& apt-get purge -y --auto-remove
&& apt-get install -y git bash postgresql musl-dev \
&& pip install --no-cache-dir psycopg \
&& apk del --no-cache .build-deps

RUN git clone https://github.com/llmware-ai/llmware.git
RUN /llmware/scripts/dev/load_native_libraries.sh
RUN cd llmware/llmware && pip install -r requirements.txt


# Create the user
RUN groupadd --gid $USER_GID $USERNAME \
&& useradd --uid $USER_UID --gid $USER_GID -m $USERNAME \
&& chown -R $USERNAME:$USER_GID /llmware
ENV PYTHONPATH=/llmware

WORKDIR /llmware

CMD /bin/bash
19 changes: 14 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,15 @@
![PyPI - Version](https://img.shields.io/pypi/v/llmware?color=blue)
[![discord](https://img.shields.io/badge/Chat%20on-Discord-blue?logo=discord&logoColor=white)](https://discord.gg/MhZn5Nc39h)

`llmware` is a unified framework for developing LLM-based application patterns including Retrieval Augmented Generation (RAG). This project provides an integrated set of tools that anyone can use - from a beginner to the most sophisticated AI developer - to rapidly build industrial-grade, knowledge-based enterprise LLM applications with specific focus on making it easy to integrate open source small specialized models and connecting enterprise knowledge safely and securely.
`llmware` is a unified framework for developing LLM-based application patterns including Retrieval Augmented Generation (RAG).

This project provides an integrated set of tools that anyone can use - from a beginner to the most sophisticated AI developer - to rapidly build industrial-grade, knowledge-based enterprise LLM applications.

Our specific focus is on making it easy to integrate open source small specialized models and connecting enterprise knowledge safely and securely.

[Join us on Discord](https://discord.gg/MhZn5Nc39h) | [Watch Youtube Tutorials](https://www.youtube.com/@llmware) | [Explore our Model Families on Huggingface](https://www.huggingface.co/llmware)

New to RAG? [Check out the Fast Start video series](https://www.youtube.com/playlist?list=PL1-dn33KwsmD7SB9iSO6vx4ZLRAWea1DB)

🔥🔥🔥 [**Multi-Model Agents with SLIM Models**](examples/SLIM-Agents/) - [**Intro-Video**](https://www.youtube.com/watch?v=cQfdaTcmBpY) 🔥🔥🔥
Can't wait? Get SLIMs right away:
Expand Down Expand Up @@ -700,17 +705,21 @@ Interested in contributing to llmware? Information on ways to participate can be

Questions and discussions are welcome in our [github discussions](https://github.com/llmware-ai/llmware/discussions).

## 📣 Release notes and Change Log
## 📣 Release notes and Change Log

**Friday, February 16 - v0.2.3 WIP Update**
**Wednesday, Februray 28 - v0.2.4 Update**
- Major upgrade of GGUF Generative Model class - support for Stable-LM-3B, CUDA build options, and better control over sampling strategies.
- Note: new GGUF llama.cpp built libs packaged with build starting in v0.2.4.
- Improved GPU support for HF Embedding Models.

**Friday, February 16 - v0.2.3 Update**
- Added 10+ embedding models to ModelCatalog - nomic, jina, bge, gte, ember and uae-large.
- Updated OpenAI support >=1.0 and new text-3 embedding models.
- SLIM model keys and output_values now accessible in ModelCatalog.
- Updating encodings to 'utf-8-sig' to better handle txt/csv files with bom.

**Reported notable issues on priority resolution path**
- stablelm-based models using gguf
- older linux versions with GLIBC < 2.34
- older linux versions with GLIBC < 2.34 (especially Ubuntu20)
- 3.12 python support - waiting on one last dependency (coming soon)

**Supported Operating Systems**: MacOS (Metal and x86), Linux (x86 and aarch64), Windows
Expand Down
45 changes: 30 additions & 15 deletions devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -1,33 +1,48 @@
{

"name": "LLMWARE Dev",
//"build": { "dockerfile": "../Dockerfile" },
"image": "provocoai/llmware:dev-01", //
"RemoteUser": "${localEnv:USER}",

// This is the image to use if doing development on only the llmware code base
//"image": "provocoai/llmware:dev-01", /
// using the dockerComposeFile option allows us to setup
// the full env around the llm code base include milvus and mongodb.
"dockerComposeFile": "docker-compose.yaml",

"service": "llmware", // we define the container to connect too

// update based on individual perference and mount point defined below
"workspaceFolder": "/llmware",

// "remoteUser": "llmware", //used with docker instead of podman
// "containerUser": "llmware", // used with docker instead of podman

// when we close the devcontainer env make sure we clean up after ourselfs
"shutdownAction": "stopCompose",

// needed for podman to keep the id's the same
"runArgs": [
"--name",
"${localWorkspaceFolderBasename}", // Container name
"-it",
"-l",
"com.docker.compose.project=devcontainers" // Container group name
"--userns=keep-id"
],
// you can setup your local directory in the devcontainer here. The mount line is an example and mounts your home directory into the /code directory

// Define the mountpoints into the llmware container.
// I use $HOMEDIR/code/provoco as my default codebase and
// then set the target to /code so I have access to all my code in /code
"mounts" : [
//"source=${localEnv:HOME},target=/code,type=bind,consistency=cached"
"source=${localEnv:HOME}/code/provoco,target=/code,Z,type=bind,consistency=cached"
// "source=${localENV:HOME}/.vscode-server,target=/root/.vscode-server,Z,type=bind,consistency=cached"
],

// setup the github cli feature so we can commit within the UI
"features": {
"ghcr.io/devcontainers/features/docker-outside-of-docker:1": {
"dockerDashComposeVersion": "v2"
},
"ghcr.io/devcontainers/features/github-cli:1": {}
"ghcr.io/devcontainers/features/github-cli:1": {}
},

"customizations": {
"vscode": {
"extensions": [
"esbenp.prettier-vscode", // prettify the code extension
"ms-python.python", //python code extensions
"ms-python.vscode-pylance" / vscode python extension
"ms-python.vscode-pylance" // vscode python extension
]
}
}
Expand Down
53 changes: 53 additions & 0 deletions devcontainer/devcontainer.json-single.bk
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
{
"name": "LLMWARE Dev",
//"build": { "dockerfile": "../Dockerfile" },
"containerUser": "llmware",
"updateRemoteUserUID": true,
"image": "provocoai/llmware:dev-01",
"remoteUser": "llmware",
"runArgs": [
"--userns=keep-id:uid=1000,gid=1000"
],


// "runArgs": [
// "--name",
// "${localWorkspaceFolderBasename}", // Container name
// "-it",
// "-l",
// "com.docker.compose.project=devcontainers" // Container group name
// ],
// you can setup your local directory in the devcontainer here. The mount line is an example and mounts your home directory into the /code directory
"mounts" : [
"source=/home/harrison/code/provoco,target=/code,type=bind,consistency=cached,Z",
"source=/home/harrison/.vscode,target=/root/.vscode,type=bind,consistency=cached,Z"
],
"customizations": {
// Configure properties specific to VS Code.
"vscode": {
// Add the IDs of extensions you want installed when the container is created.
"extensions": [
//"mongodb.mongodb-vscode",
"ms-python.python"
//"ms-azuretools.vscode-docker"
]
}
},
"postCreateCommand": "apt-get install ",
"features": {}
// "features": {
// "ghcr.io/devcontainers/features/docker-outside-of-docker:1": {
// "dockerDashComposeVersion": "v2"
// },
// "ghcr.io/devcontainers/features/github-cli:1": {}
// },
// "customizations": {
// "vscode": {
// "extensions": [
// "esbenp.prettier-vscode", // prettify the code extension
// "ms-python.python", //python code extensions
// "ms-python.vscode-pylance" // vscode python extension
// ]
// }
// }
}
108 changes: 108 additions & 0 deletions devcontainer/docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
version: "3.5"

services:
llmware:
container_name: llmware
image: provocoai/llmware:dev-01
depends_on:
- "milvus"
- "mongodb"
command: sleep infinity
network_mode: host

mongodb:
container_name: mongodb
image: mongo:5.0.10
# To secure MongoDB, uncomment and set the following values
# environment:
# - MONGO_INITDB_DATABASE=admin
# - MONGO_INITDB_ROOT_USERNAME=admin
# - MONGO_INITDB_ROOT_PASSWORD=changeme
volumes:
- llmware-mongodb:/data/db:Z
ports:
- '27017:27017'

etcd:
container_name: milvus-etcd
image: quay.io/coreos/etcd:v3.5.5
environment:
- ETCD_AUTO_COMPACTION_MODE=revision
- ETCD_AUTO_COMPACTION_RETENTION=1000
- ETCD_QUOTA_BACKEND_BYTES=4294967296
- ETCD_SNAPSHOT_COUNT=50000
volumes:
- llmware-etcd:/etcd:Z
command: etcd -advertise-client-urls=http://127.0.0.1:2379 -listen-client-urls http://0.0.0.0:2379 --data-dir /etcd
healthcheck:
test: ["CMD", "etcdctl", "endpoint", "health"]
interval: 30s
timeout: 20s
retries: 3

minio:
container_name: milvus-minio
image: minio/minio:RELEASE.2023-03-20T20-16-18Z
environment:
MINIO_ACCESS_KEY: minioadmin
MINIO_SECRET_KEY: minioadmin
ports:
- "9001:9001"
- "9000:9000"
volumes:
- llmware-minio:/minio_data:Z
command: minio server /minio_data --console-address ":9001"
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
interval: 30s
timeout: 20s
retries: 3

milvus:
container_name: milvus
image: milvusdb/milvus:v2.3.0
command: ["milvus", "run", "standalone"]
environment:
ETCD_ENDPOINTS: etcd:2379
MINIO_ADDRESS: minio:9000
volumes:
- llmware-milvus:/var/lib/milvus:Z
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9091/healthz"]
interval: 30s
start_period: 90s
timeout: 20s
retries: 3
ports:
- "19530:19530"
- "9091:9091"
depends_on:
- "etcd"
- "minio"


dev-neo4j:
container_name: devneo4j
hostname: neo4j
image: neo4j:5.15.0-community
ports:
- 7474:7474
- 7687:7687
restart: always

volumes:
- $HOME/neo4j/data:/data:Z
- $HOME/neo4j/logs:/logs:Z

environment:
- NEO4J_AUTH=none

volumes:
llmware-mongodb:
driver: local
llmware-etcd:
driver: local
llmware-minio:
driver: local
llmware-milvus:
driver: local
39 changes: 35 additions & 4 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,19 @@
version: "3.5"

services:
llmware:
container_name: llmware
image: provocoai/llmware:dev-01
volumes:
- $HOME/code/provoco:/code:Z
- $HOME/.vscode:/root/.vscode:Z
network_mode: service:mongodb
# - service:milvus
# - service:etcd
# - service:minio
# - service:devneo4j
command: sleep infinity

mongodb:
container_name: mongodb
image: mongo:5.0.10
Expand All @@ -10,7 +23,7 @@ services:
# - MONGO_INITDB_ROOT_USERNAME=admin
# - MONGO_INITDB_ROOT_PASSWORD=changeme
volumes:
- llmware-mongodb:/data/db
- llmware-mongodb:/data/db:Z
ports:
- '27017:27017'

Expand All @@ -23,7 +36,7 @@ services:
- ETCD_QUOTA_BACKEND_BYTES=4294967296
- ETCD_SNAPSHOT_COUNT=50000
volumes:
- llmware-etcd:/etcd
- llmware-etcd:/etcd:Z
command: etcd -advertise-client-urls=http://127.0.0.1:2379 -listen-client-urls http://0.0.0.0:2379 --data-dir /etcd
healthcheck:
test: ["CMD", "etcdctl", "endpoint", "health"]
Expand All @@ -41,7 +54,7 @@ services:
- "9001:9001"
- "9000:9000"
volumes:
- llmware-minio:/minio_data
- llmware-minio:/minio_data:Z
command: minio server /minio_data --console-address ":9001"
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
Expand All @@ -57,7 +70,7 @@ services:
ETCD_ENDPOINTS: etcd:2379
MINIO_ADDRESS: minio:9000
volumes:
- llmware-milvus:/var/lib/milvus
- llmware-milvus:/var/lib/milvus:Z
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9091/healthz"]
interval: 30s
Expand All @@ -71,6 +84,23 @@ services:
- "etcd"
- "minio"


dev-neo4j:
container_name: devneo4j
hostname: neo4j
image: neo4j:5.15.0-community
ports:
- 7474:7474
- 7687:7687
restart: always

volumes:
- $HOME/neo4j/data:/data:Z
- $HOME/neo4j/logs:/logs:Z

environment:
- NEO4J_AUTH=none

volumes:
llmware-mongodb:
driver: local
Expand All @@ -80,4 +110,5 @@ volumes:
driver: local
llmware-milvus:
driver: local


Loading

0 comments on commit 04bfd8a

Please sign in to comment.