Introduce Knowledge Distillation Base#417

Closed

austin362667 wants to merge 12 commits intolinkedin:mainfrom austin362667:feat/distill_base

+461-101

Commits on Dec 7, 2024

Add liger and naive distill base

austin362667
authored and
shivam15s
committed
Format

austin362667
authored and
shivam15s
committed
Refactor beta

austin362667
authored and
shivam15s
committed
Remove imports

austin362667
authored and
shivam15s
committed
Fix distill base chunk_size scaling

austin362667
authored and
shivam15s
committed
Fix chunk division

austin362667
authored and
shivam15s
committed
Remove chunk arg

austin362667
authored and
shivam15s
committed
Fix distillation_loss arg typo

austin362667
authored and
shivam15s
committed
use torch no grad and change normalization term
shivam15s
committed
rearrange fns for readability
shivam15s
committed
add no grad in tests
shivam15s
committed
Merge branch 'main' into feat/distill_base
shivam15s
authored