-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Save Predictions + Add prediction logging #335
Conversation
open( | ||
embedding_path.joinpath(f"{test_fold_str}.predicted-labels.pkl"), "wb" | ||
), | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
a) We can't do any interesting downstream analysis unless predictions AND targets are pickled. This MUST be fixed.
b) Is it difficult to pickle these scores where the test stuff is actually computed, rather than relying upon nebulous global state that may change? (For b) maybe make an issue and move on.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
By global state, I mean split_grid_points[i].predictor.test_predicted_labels. Maybe this is okay. but we still must fix a)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Targets were already saved from before by the embedding module. @jorshi Thanks for adding it here as well along with the prediction logits. Will be in one place.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For b) #340
No description provided.