Add final L2 normalization option and update README for Embedding Gemma-300M#931
Open
kamalkraj wants to merge 1 commit intogoogle-ai-edge:mainfrom
Open
Add final L2 normalization option and update README for Embedding Gemma-300M#931kamalkraj wants to merge 1 commit intogoogle-ai-edge:mainfrom
kamalkraj wants to merge 1 commit intogoogle-ai-edge:mainfrom
Conversation
This commit introduces an optional final_l2_norm parameter to the EmbeddingGemma model, allowing users to enable or disable the final L2 normalization step. This enhancement is particularly useful for advanced use cases such as Matryoshka embeddings. Changes: - Model Update: Modified EmbeddingGemma in embedding_gemma.py to accept a final_l2_norm argument in __init__ and build_model. The forward method now conditionally applies L2 normalization. - Conversion Script: Updated convert_to_tflite.py to include a --final_l2_norm flag (defaulting to True), passing it to the model builder. - Verification: Updated verify.py and verify_util.py to support the final_l2_norm flag, ensuring the verification process matches the model configuration. - Documentation: Updated README.md to document the new flag, mention Matryoshka embeddings, and include litert-torch in the requirements. The default behavior remains unchanged (final_l2_norm=True), ensuring backward compatibility.
kamalkraj
force-pushed
the
embedding_gemma
branch
from
8580e12 to
5f82baf
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduce an option for final L2 normalization in the Embedding Gemma-300M model. Update the README to reflect this new feature and provide clearer instructions for model conversion and verification. Adjust formatting in relevant files for consistency.