-
Notifications
You must be signed in to change notification settings - Fork 128
Description
Problem:
The session-based blocks defined in #451 are implemented as standalone tasks and we need to link them together to provide known session-based models.
Goal:
- Implement the high-level blocks (inputs block, next item prediction task) needed for defining sequential models.
Constraints:
- [Task] First support of session-based recommendation with RNNs block #451 should be finalized before starting this work
- Update examples and unit tests with the new definition of InputBlock, Masking and Next Item Prediction task.
Starting Point:
-
Simplify the InputBlock
[Task] Decouple InputBlock from Embedding creation allowing for user defined embeddings #404
Notes: -
Simplify the definition of input block by exposing the embedding argument to the user and remove the current nested structure to support 2-D and 3-D inputs.
[Task] Decouple InputBlock from Embedding creation allowing for user defined embeddings #404 -
Create a
SequenceBasedInputBlockthat accepts context and sparse schemas. Dropped as the new InputBlockV2 accepts multiple branches and also automatically broadcast context features to 3-D scores. -
Add
to-2Dandto-3Dtransformations as apostblock in the InputBlockV2.
-AddBroadcastToSequenceLayer to broadcast context features to match sequence shapes. models#737 -
Support advanced masking tasks: CLM, MLM (PLM is excluded based on our research paper)
Introduces Masked Language Modeling for Transformers models#780 -
Implementation of Transformer block based in the configs defined in HuggingFace - transformers-based library.
Note: This is an example of how Transformer-based architectures are implemented in HF (as keras layers).
Transformer Block (for HuggingFace support) models#771 -
Refactor base model to include a new design of masking/target generation for loss calculation and metrics.
Introduces Masked Language Modeling for Transformers models#780 -
Update weight-tying strategy based on the new refactor of the InputBlock (embeddings are independent tables and model context would be removed)
Introducing Outputs-V2 models#715 -
Support setting the masking task for
fitandevaluate
Introduces Masked Language Modeling for Transformers models#780 -
Example of transformer session-based recommendation for training and deployment
MM transformer training and serving example models#1045