Skip to content

Commit

Permalink
add todo
Browse files Browse the repository at this point in the history
  • Loading branch information
ByronHsu committed Aug 13, 2024
1 parent e85611c commit 04b9545
Showing 1 changed file with 7 additions and 0 deletions.
7 changes: 7 additions & 0 deletions Readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,10 @@ apply_liger_kernel_to_llama()
model = transformers.AutoModelForCausalLM.from_pretrained("<some llama model>")
```

[TODO] huggingface trainer
[TODO] lightning trainer


### 2. 🧩 Compose Your Own Model

```python
Expand All @@ -97,6 +101,9 @@ loss = loss_fn(model.weight, input, target)
loss.backward()
```

[TODO] multi-head llama3 model
[TODO] large vocab model

## ⚙️ Note on ML Compiler

### 1. ⚡ Torch Compile
Expand Down

0 comments on commit 04b9545

Please sign in to comment.