Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Task] Support pre-trained embeddings for ranking and session based models via the new dataloader functionality #1043

Closed
6 tasks
EvenOldridge opened this issue Mar 29, 2023 · 1 comment
Assignees
Milestone

Comments

@EvenOldridge
Copy link
Member

EvenOldridge commented Mar 29, 2023

  • Update the input block to support the pre-trained embeddings:

  • Add a module to extract pre-trained embeddings

  • Show an example of how to transform pre-trained embeddings before aggregating them with the other trainable embeddings in the model. Such as Layer Normalization or MLP layer.

  • Update DLRM, DeepFM, etc to include an optional projection layer for combining pretrained embeddings with the other embeddings in the model.

  • Implement and evaluate different aggregation methods to combine pre-trained embeddings with the other trainable embeddings.

  • Add an example demonstrating support for pretrained embeddings for DLRM or another non session based model. Artificial example? KDD cup?

Constraints

  • Dataloader should support fixed-size 3D tensors.
  • Dataloader needs to add a tag to the pre-trained embeddings so that the T4R input module can differentiate between trainable features and pre-trained embeddings.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants