Inference Optimization Chapter

[Anindyadeep]

Type

new chapter

Chapter/Page

Something else

Description

Doing training or inference models are fairly easy, when we have smaller number of parameters. But when the scale of parameters and data increases, it becomes increasingly difficult to optimize that (interms of compute and performance). So a dedicated chapter on inference optimization and arithmetics on resource calculations becomes very useful.

Some reference links:

• Mastering LLM Inference by NVIDIA
• Transformer Math 101
• Transformer Inference Arithmetic
• A twitter thread on some optimizations

[filopedraz]

https://twitter.com/amanrsanger/status/1728502445184274710?s=46