Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Setup tape for pullback for ForwardDiff over ReverseDiff #383

Closed

Conversation

Vaibhavdixit02
Copy link
Contributor

Initial draft to sketch out what I think it should look like. Fixes #321

@codecov-commenter
Copy link

codecov-commenter commented Jul 25, 2024

Codecov Report

Attention: Patch coverage is 0% with 15 lines in your changes missing coverage. Please review.

Project coverage is 93.05%. Comparing base (bec25c6) to head (c4b4202).

Files Patch % Lines
...t/DifferentiationInterfaceReverseDiffExt/onearg.jl 0.00% 15 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #383      +/-   ##
==========================================
- Coverage   96.64%   93.05%   -3.59%     
==========================================
  Files         100       98       -2     
  Lines        4915     4910       -5     
==========================================
- Hits         4750     4569     -181     
- Misses        165      341     +176     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@gdalle
Copy link
Member

gdalle commented Jul 25, 2024

Wait I don't understand, pullback is a first order operator, what does it have to do with ForwardDiff over ReverseDiff?
What we need are two things:

  • an HVP with ForwardDiff over ReverseDiff
  • a pullback with ReverseDiff that uses the existing tape mechanism

Which are you trying to tackle?

@Vaibhavdixit02
Copy link
Contributor Author

a pullback with ReverseDiff that uses the existing tape mechanism

As I understand the current pullback needs to compile the tape everytime and cannot use the preparation in the presence of the Duals from ForwardDiff, so this let's you avoid that by adding a preparation and pullback that can work with Duals.

The hvp gets called fine even now other than the recompilation so I am not sure what you mean needs to be done there.

@Vaibhavdixit02 Vaibhavdixit02 deleted the reversepullbackwithtape branch July 26, 2024 18:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Tape-able pullback with ReverseDiff
3 participants