-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mixed precision #37
Mixed precision #37
Conversation
Due to lbfgs optim it is required to change the logic of closure.
import time | ||
|
||
os.environ['KMP_DUPLICATE_LIB_OK'] = 'TRUE' | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
тут видимо нужно добавить
sys.path.append('../')
sys.path.pop()
sys.path.append(os.path.abspath(os.path.join(os.path.dirname( file ), '..')))
чтобы скрипт запускался.
@@ -125,9 +125,9 @@ | |||
|
|||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
тут если поставить параметры use_cache=True, save_always=True, mixed_precision=True, то при втором запуске (уже с кэшем) выскакивает ошибка.
examples/example_wave_physics.py
Outdated
@@ -182,27 +182,27 @@ def wave_experiment(grid_res, CACHE): | |||
|
|||
start = time.time() | |||
|
|||
equation = Equation(grid, wave_eq, bconds).set_strategy('NN') | |||
equation = Equation(grid, wave_eq, bconds).set_strategy('autograd') | |||
|
|||
img_dir = os.path.join(os.path.dirname(__file__), 'wave_example_physics_img') | |||
|
|||
if not (os.path.isdir(img_dir)): | |||
os.mkdir(img_dir) | |||
|
|||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
тут ошибка аналогичная как и в скрипте example_wave_periodic.py. При втором запуске с параметрами use_cache=True, save_always=True, mixed_precision=True.
tedeous/solver.py
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Две функции closure и closure_cuda в рамках отдного метода избыточны. Но тут в любом случае нужно разгружать метод solve. Это уже дело рефакторинга самого класса Solver.
* Now cache is working with AMP * Slight refactor cache module
Removed scaler state dict in cache module.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added PyTorch automatic mixed precision for both CPU/GPU.
This is a good opportunity to change the precision of calculations. However, further improvements are required to close the issue #27.
Added PyTorch automatic mixed precision for both CPU/GPU + example. LBFGS is not currently supported due to PyTorch limitations.
This mode allows you to quickly obtain a solution while sacrificing accuracy. An additional method can be tensor decomposition of the model parameters. Currently, experiments are being conducted with tensor decomposition (tensor-train).
PR closes the issue #27.