You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
One of SGDP's main points is the use of momentum, currently the default settings shown paired with the visualization have it disabled. This has a pretty large impact on its speed, I did a little test seeing how long it'd take to go from randn noise to a goal image and it handled it in 3 steps reaching a squared sum difference of 7.0072e-10 with a learning rate of 0.1, momentum set to 0.9888544, and nesterov disabled. With 0 momentum, the poor thing takes 27 steps to reach 0.7700. Plus, the project page for AdamP / SGDP shows the same test as the visualizations used here, but they use momentum with various settings and it definitely reaches the goal. The github page for AdamP also gives default parameters of "SGDP(params, lr=0.1, weight_decay=1e-5, momentum=0.9, nesterov=True)", and this seems closer to the demonstrations on their project page, even though their own code defaults momentum to 0. Thus, I believe the values on their page showing an example of importing and using SGDP should be used for the default values on this page.
The text was updated successfully, but these errors were encountered:
One of SGDP's main points is the use of momentum, currently the default settings shown paired with the visualization have it disabled. This has a pretty large impact on its speed, I did a little test seeing how long it'd take to go from randn noise to a goal image and it handled it in 3 steps reaching a squared sum difference of 7.0072e-10 with a learning rate of 0.1, momentum set to 0.9888544, and nesterov disabled. With 0 momentum, the poor thing takes 27 steps to reach 0.7700. Plus, the project page for AdamP / SGDP shows the same test as the visualizations used here, but they use momentum with various settings and it definitely reaches the goal. The github page for AdamP also gives default parameters of "SGDP(params, lr=0.1, weight_decay=1e-5, momentum=0.9, nesterov=True)", and this seems closer to the demonstrations on their project page, even though their own code defaults momentum to 0. Thus, I believe the values on their page showing an example of importing and using SGDP should be used for the default values on this page.
The text was updated successfully, but these errors were encountered: