Add an option to do the final step on a CPU #98
Conversation
I needed to do this to be able to run the model on a 4070 (with 8 GM RAM).
|
@vbuterin a better option is to run the the model in half precision (16). using the --model-half parameter I can run the model on a 3070ti ( 8GB), but model takes less than 4 GB looking at nvidia-smi and is faster than CPU decoding. |
|
@ksasso1028 --model-half seems to break inpainting though last I checked. |
|
@cherrerajobs whats happening is the input audio condition (for inpainting or audio init) is not being converted to half precision before running the VAE. |
|
I can put in a PR for this, will refer back in this comment |
|
The power determine its me. IP address SDK Stack Over all Country Jesus Its Live In etherscan TRUST ME ONLY 1 STEP. See Me Later In Kuala Lumpur , Malaysia NAME : MOHD SHAH RIZAL BIN AZMAN |
|
Jesus Its Live In etherscan PEOPLE SAID TO JESUS MAYBE I HAVE DIGITAL HANA BANK OF UNITED STATES WITH DEFFRENCE BANKING SYSTEM VALUE I HAVE DIGITAL HANA BANK OF MALAYSIA IF YOU TRUST ME ANY BANKING SYSTEM NO DEFFRENCE TRUST ME ONLY 1 STEP. See Me Later In Kuala Lumpur , Malaysia NAME : MOHD SHAH RIZAL BIN AZMAN |
|
"Hello Girl Canada" HANA BANK OF CANADA FOR MORE PROJECT I WANT BUILDING CONTRUCTIONS OIL PALM INDUSTRY FOR SUPPORT SCIENCE & TECHNOLOGY USA ECONOMICS FORUM PARTNER MALAYSIA AND ECONOMICS FORUM PARTNER MALAYSIA SO THE GOV USA PROFIT EVERYSINGLE KILOGRAM $1.00 USD UNLIMETED , SO FOR MORE SAFETY PEOPLE OF UNITED STATES OIL PALM BUSINESS INDUSTRY PLASTICS & BOX ( 1 ) NON HAHAL OIL PALM - NON MUSLIMS NEXT YOU GOV WLL BRING YOU CHANCE A LIFE , AND BUILDING MORE INDUSTRY FOR YOUR BUSINESS MORE DEFFRENCE BANKING. GOV USA SELL BACK IN MARKETING ONLY USA Jesus Its Live In etherscan TRUST ME ONLY 1 STEP. See Me Later In Kuala Lumpur , Malaysia NAME : MOHD SHAH RIZAL BIN AZMAN |
I needed to do this to be able to run the model on a 4070 (with 8 GB VRAM).