Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make ONNX Quantisation Support green for “good” (was yellow) [tiny fix] #127

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion fine-tuning.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ Data preparation plays a big role in the fine-tuning process for vision based mo
[Dreambooth Image Generation Fine-Tuning](https://dreambooth.github.io)
```

Models such as [Stable Diffusion](https://stability.ai/stable-diffusion) can also be tailored through fine-tuning to generate specific images. For instance, by supplying Stable Diffusion with a dataset of pet pictures and fine-tuning it, the model becomes capable of generating images of that particular pet in diverse styles.
Models such as [Stable Diffusion](https://stability.ai/stable-image) can also be tailored through fine-tuning to generate specific images. For instance, by supplying Stable Diffusion with a dataset of pet pictures and fine-tuning it, the model becomes capable of generating images of that particular pet in diverse styles.

The dataset for fine-tuning an image generation model needs to contain two things:

Expand Down
2 changes: 1 addition & 1 deletion index.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ Spot something outdated or missing? Want to start a discussion? We welcome any o

- let us know in the <i class="fas fa-pencil-alt"></i> comments at the end of each chapter
- [<i class="fab fa-github"></i> create issues](https://docs.github.com/en/issues/tracking-your-work-with-issues/creating-an-issue)
- [<i class="fab fa-github"></i> open pull requests](https://docs.github.com/en/get-started/quickstart/contributing-to-projects)
- [<i class="fab fa-github"></i> open pull requests](https://docs.github.com/en/get-started/exploring-projects-on-github/contributing-to-a-project)
```

### Editing the Book
Expand Down
2 changes: 1 addition & 1 deletion model-formats.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Integration with Deep Learning Frameworks | 🟢 [most](onnx-support) | 🟡 [gr
Deployment Tools | 🟢 [yes](onnx-runtime) | 🔴 no | 🟢 [yes](triton-inference)
Interoperability | 🟢 [yes](onnx-interoperability) | 🔴 no | 🔴 [no](tensorrt-interoperability)
Inference Boost | 🟡 moderate | 🟢 good | 🟢 good
Quantisation Support | 🟡 [good](onnx-quantisation) | 🟢 [good](ggml-quantisation) | 🟡 [moderate](tensorrt-quantisation)
Quantisation Support | 🟢 [good](onnx-quantisation) | 🟢 [good](ggml-quantisation) | 🟡 [moderate](tensorrt-quantisation)
Custom Layer Support| 🟢 [yes](onnx-custom-layer) | 🔴 limited | 🟢 [yes](tensorrt-custom-layer)
Maintainer | [LF AI & Data Foundation](https://wiki.lfaidata.foundation) | https://github.com/ggerganov | https://github.com/NVIDIA
```
Expand Down
2 changes: 1 addition & 1 deletion references.bib
Original file line number Diff line number Diff line change
Expand Up @@ -451,7 +451,7 @@ @online{octoml-fine-tuning
title={The beginner's guide to fine-tuning Stable Diffusion},
author={Justin Gage},
year=2023,
url={https://octoml.ai/blog/the-beginners-guide-to-fine-tuning-stable-diffusion}
url={https://octo.ai/blog/the-beginners-guide-to-fine-tuning-stable-diffusion}
}
@article{small-data-tds,
title={Is "Small Data" The Next Big Thing In Data Science?},
Expand Down
1 change: 0 additions & 1 deletion references.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@

- "Catching up on the weird world of LLMs" (summary of the last few years) https://simonwillison.net/2023/Aug/3/weird-world-of-llms
- "Open challenges in LLM research" (exciting post title but mediocre content) https://huyenchip.com/2023/08/16/llm-research-open-challenges.html
- https://github.com/zeno-ml/zeno-build/tree/main/examples/analysis_gpt_mt/report
- "Patterns for Building LLM-based Systems & Products" (Evals, RAG, fine-tuning, caching, guardrails, defensive UX, and collecting user feedback) https://eugeneyan.com/writing/llm-patterns

```{figure-md} llm-patterns
Expand Down
2 changes: 1 addition & 1 deletion sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@ LLaMAIndex seems more tailor made for deploying LLM apps in production. However,

![banner](https://litellm.vercel.app/img/docusaurus-social-card.png)

As the name suggests a light package that simplifies the task of getting the responses form multiple APIs at the same time without having to worry about the imports is known as the [LiteLLM](https://litellm.ai). It is available as a python package which can be accessed using `pip` Besides we can test the working of the library using the [playground](https://litellm.ai/playground) that is readily available.
As the name suggests a light package that simplifies the task of getting the responses form multiple APIs at the same time without having to worry about the imports is known as the [LiteLLM](https://docs.litellm.ai). It is available as a python package which can be accessed using `pip`

### Completions

Expand Down
12 changes: 6 additions & 6 deletions unaligned-models.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Model | Reference Model | Training Data | Features
[](#fraudgpt) | 🔴 unknown | 🔴 unknown | Phishing email, {term}`BEC`, Malicious Code, Undetectable Malware, Find vulnerabilities, Identify Targets
[](#wormgpt) | 🟢 [](models.md#gpt-j-6b) | 🟡 malware-related data | Phishing email, {term}`BEC`
[](#poisongpt) | 🟢 [](models.md#gpt-j-6b) | 🟡 false statements | Misinformation, Fake news
[](#wizardlm-uncensored) | 🟢 [](models.md#wizardlm) | 🟢 [available](https://huggingface.co/datasets/ehartford/wizard_vicuna_70k_unfiltered) | Uncensored
[](#wizardlm-uncensored) | 🟢 [](models.md#wizardlm) | 🟢 [available](https://huggingface.co/datasets/cognitivecomputations/wizard_vicuna_70k_unfiltered) | Uncensored
[](#falcon-180b) | 🟢 N/A | 🟡 partially [available](https://huggingface.co/datasets/tiiuae/falcon-refinedweb) | Unaligned
```

Expand Down Expand Up @@ -109,10 +109,10 @@ Model Censoring {cite}`erichartford-uncensored`

Uncensoring {cite}`erichartford-uncensored`, however, takes a different route, aiming to identify and
eliminate these alignment-driven restrictions while retaining valuable knowledge. In the case of
[WizardLM Uncensored](https://huggingface.co/ehartford/WizardLM-7B-Uncensored), it closely follows the uncensoring
[WizardLM Uncensored](https://huggingface.co/cognitivecomputations/WizardLM-7B-Uncensored), it closely follows the uncensoring
methods initially devised for models like [](models.md#vicuna), adapting the script
used for [Vicuna](https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered) to work seamlessly with
[WizardLM's dataset](https://huggingface.co/datasets/ehartford/WizardLM_alpaca_evol_instruct_70k_unfiltered).
[WizardLM's dataset](https://huggingface.co/datasets/cognitivecomputations/WizardLM_alpaca_evol_instruct_70k_unfiltered).
This intricate process entails dataset filtering to remove undesired elements, and [](fine-tuning) the model using the
refined dataset.

Expand All @@ -125,9 +125,9 @@ For a comprehensive, step-by-step explanation with working code see this blog: {

Similar models have been made available:

- [WizardLM 30B-Uncensored](https://huggingface.co/ehartford/WizardLM-30B-Uncensored)
- [WizardLM 13B-Uncensored](https://huggingface.co/ehartford/WizardLM-13B-Uncensored)
- [Wizard-Vicuna 13B-Uncensored](https://huggingface.co/ehartford/Wizard-Vicuna-13B-Uncensored)
- [WizardLM 30B-Uncensored](https://huggingface.co/cognitivecomputations/WizardLM-30B-Uncensored)
- [WizardLM 13B-Uncensored](https://huggingface.co/cognitivecomputations/WizardLM-13B-Uncensored)
- [Wizard-Vicuna 13B-Uncensored](https://huggingface.co/cognitivecomputations/Wizard-Vicuna-13B-Uncensored)

### Falcon 180B

Expand Down
Loading