v0.6.6.post1

What's Changed

• Docs SEO Improvements by @Dominastorm in #586
• Add dashboard to docs by @shrjain1312 in #585
• Bump Pydantic to V2 by @Dominastorm in #582

Full Changelog: v0.6.6...v0.6.6.post1

v0.6.6

What's Changed

• Zeno Integration by @shrjain1312 in #576
• Add Valid Question Operator by @ashish-1600 in #577
• Supported LLMs doc update by @shrjain1312 in #578
• Rca open source by @ashish-1600 in #574
• Fix variable name by @anas-rabhi in #580
• Update llama_index.py by @devanshi00 in #564
• Add support for llama-index v0.10+ by @Dominastorm in #581

New Contributors

• @anas-rabhi made their first contribution in #580
• @devanshi00 made their first contribution in #564

Full Changelog: v0.6.5.post2...v0.6.6

v0.6.5.post2

What's Changed

• Fix checkset run for managed user by @ashish-1600 in #572
• Fix Custom Prompt Eval by @ashish-1600 in #573

Full Changelog: v0.6.5.post1...v0.6.5.post2

v0.6.5.post1

What's Changed

• Fix response matching operator by @sourabhagr in #570

Full Changelog: v0.6.5...v0.6.5.post1

v0.6.5

Updated dependencies

v0.6.4

All New User Interface

• Dashboard
  • The new No-Code dashboard allows you to run evaluations without writing any code
  • You can visualize these results locally by running the dashboard on your system

New Operators

Introducing three new operators that fit right into your RAG pipelines.

Query Enhancement Operators

• Sub Query Completeness
  • See how well the sub-queries cover the main question

Contextual Operators

• Context Reranking
  • Check whether the reranking improved the order of your documents
• Context Conciseness
  • Check if a part of your context is as good as the original context to answer your query

Integrations

API Integrations

• Mistral API Integration

  • Use the up-and-coming Mistral LLMs to perform your evaluations.

• Langfuse Integration

  • View traces, analyse use cases and user segments while evaluating your LLMs

v0.6.3

Update pyproject.toml (#551)

* Update pyproject.toml

* Update pyproject.toml

v0.6.2

Update pyproject.toml (#550)

v0.6.1

Updated dependencies

UpTrain v0.6

We are thrilled to announce a significant array of enhancements aimed at improving user experience, ease of use, and overall functionality in UpTrain v0.6!

New Features:

1. Local Evaluation Capability ✨

   • Users can now run evaluations locally on their systems, providing more flexibility and control over the evaluation process.

2. Custom Prompt Evaluation 🎛️

   • Introducing the ability to create custom evaluations tailored to specific user needs, empowering users with more control over the evaluation process.

3. Scenario Description Parameter for Operators 📝

   • Operators can now specify additional context to the Language Model (LLM) using the scenario description parameter, enhancing the quality of evaluations.

4. Modular Prompt Templates 🧩

   • Release of customizable prompt templates featuring customizable instructions, few-shot examples, scenario descriptions, and output formats, providing users with versatile tools for prompt creation.

5. New Integrations 🚀

   • Vector DBs Integration 🔍
     • Integration with vector databases such as Qdrant, ChromaDB, and FAISS for RAG operations, query responses, and evaluation using UpTrain.
   • Framework Integration 🛠️
     • Integration with LLamaindex framework for streamlined operations.
   • LLM Providers Integration 💡
     • Integration with LLMs like Mistral and Llama from platforms such as Anyscale and Together AI for evaluation purposes.
   • LLM Embeddings Integration 🧠
     • Integration with Jina for generating embeddings to enhance RAG operations and evaluations.

6. Research Integration 📚

   • UpTrain now incorporates the state-of-the-art Spade framework for auto-generating assertions to identify poor LLM outputs, facilitating seamless evaluation on user datasets.

7. Root Cause Analysis 🕵️

   • UpTrain facilitates root cause analysis for failure issues in RAG pipelines, aiding in the identification and resolution of problems.

8. Vector Search Integration 🔍

   • Enhanced vector search capability allows for comparing different embedding models, enabling users to derive more relevant context from vector databases.

New Evaluations:

1. Jailbreak Detection 🚨

   • Identify attempts to perform illegal activities or misuse of the LLM. Users can specify a model purpose to ensure adherence to intended usage.

2. Code Hallucination 💻

   • Determine the grounding of code generated by the LLM based on provided documents/context, ensuring coherence and relevance.