1- # KerasNLP: Modular NLP Workflows for Keras
1+ # KerasNLP: Multi-framework NLP Models
22[ ![ ] ( https://github.com/keras-team/keras-nlp/workflows/Tests/badge.svg?branch=master )] ( https://github.com/keras-team/keras-nlp/actions?query=workflow%3ATests+branch%3Amaster )
33![ Python] ( https://img.shields.io/badge/python-v3.9.0+-success.svg )
44[ ![ contributions welcome] ( https://img.shields.io/badge/contributions-welcome-brightgreen.svg?style=flat )] ( https://github.com/keras-team/keras-nlp/issues )
55
66KerasNLP is a natural language processing library that works natively
7- with TensorFlow, JAX, or PyTorch. Built on Keras 3, these models, layers,
8- metrics, and tokenizers can be trained and serialized in any framework and
9- re-used in another without costly migrations.
10-
11- KerasNLP supports users through their entire development cycle. Our workflows
12- are built from modular components that have state-of-the-art preset weights when
13- used out-of-the-box and are easily customizable when more control is needed.
7+ with TensorFlow, JAX, or PyTorch. KerasNLP provides a repository of pre-trained
8+ models and a collection of lower-level building blocks for language modeling.
9+ Built on Keras 3, models can be trained and serialized in any framework
10+ and re-used in another without costly migrations.
1411
1512This library is an extension of the core Keras API; all high-level modules are
16- [ ` Layers ` ] ( https://keras.io/api/layers/ ) or
17- [ ` Models ` ] ( https://keras.io/api/models/ ) that receive that same level of polish
18- as core Keras. If you are familiar with Keras, congratulations! You already
19- understand most of KerasNLP.
20-
21- See our [ Getting Started guide] ( https://keras.io/guides/keras_nlp/getting_started )
22- to start learning our API. We welcome [ contributions] ( CONTRIBUTING.md ) .
13+ Layers and Models that receive that same level of polish as core Keras.
14+ If you are familiar with Keras, congratulations! You already understand most of
15+ KerasNLP.
16+
17+ All models support JAX, TensorFlow, and PyTorch from a single model
18+ definition and can be fine-tuned on GPUs and TPUs out of the box. Models can
19+ be trained on individual accelerators with built-in PEFT techniques, or
20+ fine-tuned at scale with model and data parallel training. See our
21+ [ Getting Started guide] ( https://keras.io/guides/keras_nlp/getting_started )
22+ to start learning our API. Browse our models on
23+ [ Kaggle] ( https://www.kaggle.com/organizations/keras/models ) .
24+ We welcome contributions.
2325
2426## Quick Links
2527
@@ -28,7 +30,7 @@ to start learning our API. We welcome [contributions](CONTRIBUTING.md).
2830- [ Home Page] ( https://keras.io/keras_nlp )
2931- [ Developer Guides] ( https://keras.io/guides/keras_nlp )
3032- [ API Reference] ( https://keras.io/api/keras_nlp )
31- - [ Getting Started guide ] ( https://keras.io/guides/keras_nlp/getting_started )
33+ - [ Pre-trained Models ] ( https://www.kaggle.com/organizations/keras/models )
3234
3335### For contributors
3436
@@ -38,53 +40,13 @@ to start learning our API. We welcome [contributions](CONTRIBUTING.md).
3840- [ API Design Guide] ( API_DESIGN_GUIDE.md )
3941- [ Call for Contributions] ( https://github.com/keras-team/keras-nlp/issues?q=is%3Aissue+is%3Aopen+label%3A%22contributions+welcome%22 )
4042
41- ## Installation
42-
43- KerasNLP supports both Keras 2 and Keras 3. We recommend Keras 3 for all new
44- users, as it enables using KerasNLP models and layers with JAX, TensorFlow and
45- PyTorch.
46-
47- ### Keras 2 Installation
48-
49- To install the latest KerasNLP release with Keras 2, simply run:
50-
51- ```
52- pip install --upgrade keras-nlp
53- ```
54-
55- ### Keras 3 Installation
56-
57- There are currently two ways to install Keras 3 with KerasNLP. To install the
58- stable versions of KerasNLP and Keras 3, you should install Keras 3 ** after**
59- installing KerasNLP. This is a temporary step while TensorFlow is pinned to
60- Keras 2, and will no longer be necessary after TensorFlow 2.16.
61-
62- ```
63- pip install --upgrade keras-nlp
64- pip install --upgrade keras>=3
65- ```
66-
67- To install the latest nightly changes for both KerasNLP and Keras, you can use
68- our nightly package.
69-
70- ```
71- pip install --upgrade keras-nlp-nightly
72- ```
73-
74- > [ !IMPORTANT]
75- > Keras 3 will not function with TensorFlow 2.14 or earlier.
76-
77- Read [ Getting started with Keras] ( https://keras.io/getting_started/ ) for more information
78- on installing Keras 3 and compatibility with different frameworks.
79-
8043## Quickstart
8144
82- Fine-tune BERT on a small sentiment analysis task using the
83- [ ` keras_nlp.models ` ] ( https://keras.io/api/keras_nlp/models/ ) API:
45+ Fine-tune BERT on IMDb movie reviews:
8446
8547``` python
8648import os
87- os.environ[" KERAS_BACKEND" ] = " tensorflow " # Or "jax " or "torch"!
49+ os.environ[" KERAS_BACKEND" ] = " jax " # Or "tensorflow " or "torch"!
8850
8951import keras_nlp
9052import tensorflow_datasets as tfds
@@ -96,8 +58,8 @@ imdb_train, imdb_test = tfds.load(
9658 batch_size = 16 ,
9759)
9860# Load a BERT model.
99- classifier = keras_nlp.models.BertClassifier .from_preset(
100- " bert_base_en_uncased " ,
61+ classifier = keras_nlp.models.Classifier .from_preset(
62+ " bert_base_en " ,
10163 num_classes = 2 ,
10264 activation = " softmax" ,
10365)
@@ -107,7 +69,35 @@ classifier.fit(imdb_train, validation_data=imdb_test)
10769classifier.predict([" What an amazing movie!" , " A total waste of my time." ])
10870```
10971
110- For more in depth guides and examples, visit https://keras.io/keras_nlp/ .
72+ Try it out [ in a colab] ( https://colab.research.google.com/gist/mattdangerw/e457e42d5ea827110c8d5cb4eb9d9a07/kerasnlp-quickstart.ipynb ) .
73+ For more in depth guides and examples, visit
74+ [ keras.io/keras_nlp] ( https://keras.io/keras_nlp/ ) .
75+
76+ ## Installation
77+
78+ To install the latest KerasNLP release with Keras 3, simply run:
79+
80+ ```
81+ pip install --upgrade keras-nlp
82+ ```
83+
84+ To install the latest nightly changes for both KerasNLP and Keras, you can use
85+ our nightly package.
86+
87+ ```
88+ pip install --upgrade keras-nlp-nightly
89+ ```
90+
91+ Note that currently, installing KerasNLP will always pull in TensorFlow for use
92+ of the ` tf.data ` API for preprocessing. Even when pre-processing with ` tf.data ` ,
93+ training can still happen on any backend.
94+
95+ Read [ Getting started with Keras] ( https://keras.io/getting_started/ ) for more
96+ information on installing Keras 3 and compatibility with different frameworks.
97+
98+ > [ !IMPORTANT]
99+ > We recommend using KerasNLP with TensorFlow 2.16 or later, as TF 2.16 packages
100+ > Keras 3 by default.
111101
112102## Configuring your backend
113103
@@ -145,7 +135,8 @@ KerasNLP provides access to pre-trained models via the `keras_nlp.models` API.
145135These pre-trained models are provided on an "as is" basis, without warranties
146136or conditions of any kind. The following underlying models are provided by third
147137parties, and subject to separate licenses:
148- BART, DeBERTa, DistilBERT, GPT-2, OPT, RoBERTa, Whisper, and XLM-RoBERTa.
138+ BART, BLOOM, DeBERTa, DistilBERT, GPT-2, Llama, Mistral, OPT, RoBERTa, Whisper,
139+ and XLM-RoBERTa.
149140
150141## Citing KerasNLP
151142
0 commit comments