Added docs

predibase · Feb 21, 2024 · 51f472f · 51f472f
1 parent e7a5cec
commit 51f472f
Show file tree

Hide file tree

Showing 2 changed files with 18 additions and 7 deletions.
diff --git a/docs/models/adapters.md b/docs/models/adapters.md
@@ -36,13 +36,15 @@ Any combination of linear layers can be targeted in the adapters, which correspo
 - `o_proj`
 - `lm_head`
 
-### Qwen
+### Gemma
 
-- `c_attn`
-- `c_proj`
-- `w1`
-- `w2`
-- `lm_head`
+- `q_proj`
+- `k_proj`
+- `v_proj`
+- `o_proj`
+- `gate_proj`
+- `up_proj`
+- `down_proj`
 
 ### Phi
 
@@ -54,6 +56,14 @@ Any combination of linear layers can be targeted in the adapters, which correspo
 - `fc2`
 - `lm_head`
 
+### Qwen
+
+- `c_attn`
+- `c_proj`
+- `w1`
+- `w2`
+- `lm_head`
+
 ### GPT2
 
 - `c_attn`

diff --git a/docs/models/base_models.md b/docs/models/base_models.md
@@ -7,8 +7,9 @@
 - 🌬️[Mistral](https://huggingface.co/mistralai)
     - [Zephyr](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta)
 - 🔄 [Mixtral](https://huggingface.co/mistralai/Mixtral-8x7B-v0.1)
-- 🔮 [Qwen](https://huggingface.co/Qwen)
+- 💎 [Gemma](https://blog.google/technology/developers/gemma-open-models/)
 - 🏛️ [Phi](https://huggingface.co/microsoft/phi-2)
+- 🔮 [Qwen](https://huggingface.co/Qwen)
 - 🤖 [GPT2](https://huggingface.co/gpt2)
 - 🌸 [Bloom](https://huggingface.co/bigscience/bloom)