v0.6.4

Summary

This point release simplifies our support for Keras 3 and Keras 2.

• If Keras 2 is installed, KerasNLP will use Keras 2 and TensorFlow.
• If Keras 3 is installed, KerasNLP will use Keras 3 and run on any backend.

If you have any issue installing KerasNLP, please open an issue.

What's Changed

• 0.6.4 cherry picks by @mattdangerw in #1350
• Version bump for 0.6.4.dev0 pre-release by @mattdangerw in #1351
• Version bump for 0.6.4 release by @mattdangerw in #1356

Full Changelog: v0.6.3...v0.6.4

v0.6.3

Summary

This release adds support for running KerasNLP against Keras 3. You can try this today by installing tf-nightly and tensorflow-text-nightly.

pip install keras-nlp
pip uninstall -y tensorflow-text tensorflow keras
pip install tensorflow-text-nightly tf-nightly

Otherwise, this release should be a no-op for all users. No new features, no change in default behavior.

Upcoming changes

After the release of Keras 3, we will drop support for running KerasNLP against the Keras Core package (no more import keras_core as keras), in favor of Keras 3. Keras 3 is the long-term replacement for Keras Core.

What's Changed

• Cherry picks for 0.6.3 by @mattdangerw in #1297
• Version bump 0.6.3 by @mattdangerw in #1298
• Bump the version to 0.6.3.dev1 by @mattdangerw in #1301
• Version bump to 0.6.3 by @mattdangerw in #1302

Full Changelog: v0.6.2...v0.6.3

v0.6.2

Summary

• Support mixed precision on keras-core on all of jax, torch and tensorflow.
• Add keras_nlp.layers.RotaryEmbedding for rotary embeddings.
• Add keras_nlp.layers.ReversibleEmbedding to better support tied or untied weights for logit projections.
• Many bug fixes and improvements.

What's Changed

• Generic RotaryEmbedding Layer by @shivance in #1180
• Raise ValueError when number of dims evaluate to zero by @sampathweb in #1198
• Add XLNetBackbone by @susnato in #1084
• Switch from tf.nest to dm-tree by @mattdangerw in #1199
• Fix CI for keras-core 0.1.4 by @mattdangerw in #1202
• Fix ModuleNotFoundError keras_nlp.models.xlnet by @shivance in #1204
• Add support for "untied" embedding weights in language models by @mattdangerw in #1201
• Add start_index argument to all position embedding layers by @mattdangerw in #1209
• Remove windows line endings by @mattdangerw in #1210
• Fix Autograph error with perplexity metric by @shivance in #1211
• [JAX backend]: Fix errors with perplexity by @shivance in #1213
• Improve layer naming consistency by @mattdangerw in #1219
• Stop asserting key order in bart preprocessor by @mattdangerw in #1221
• Remove file level docstrings by @mattdangerw in #1222
• Fix typos by @mattdangerw in #1220
• Typo fix by @mattdangerw in #1223
• Fix RotaryEmbedding import by @shivance in #1217
• Update transformer_decoder for the proper naming of the sublayers. by @qlzh727 in #1230
• Replace tf with numpy by @mattdangerw in #1232
• Update to always using ops.shape by @mattdangerw in #1231
• Add a test harness based on keras-core's run_layer_test by @mattdangerw in #1238
• fixed token_to_id doc + error msg by @jackd in #1240
• Changed default TokenAndPositionEmbedding initializer to 'uniform' by @jackd in #1237
• Add compat shims for the upcoming keras-core release by @mattdangerw in #1244
• Depend on latest keras-core by @mattdangerw in #1246
• Removed the undefined self.sequence_length by @sahusiddharth in #1245
• Bump devcontainer to 3.9 by @mattdangerw in #1249
• Add a mixed precision test and fix mixed precision errors for layers by @mattdangerw in #1242
• Quick fix for 0.1.7 keras-core release by @mattdangerw in #1251
• Small docstring fixes for the upcoming release by @mattdangerw in #1253

New Contributors

• @qlzh727 made their first contribution in #1230
• @jackd made their first contribution in #1240
• @sahusiddharth made their first contribution in #1245

Full Changelog: v0.6.1...v0.6.2

v0.6.1

With the 0.6.1. release, all remaining models, metrics and samplers have been ported to keras-core. The full KerasNLP API is now available on TensorFlow, PyTorch and Jax (instructions).

Summary

• FNet and DeBERTa are now multi-backend.
  • All keras_nlp.models.FNetXX and keras_nlp.models.DebertaV3XX symbols work on all backends.
• keras_nlp.samplers.BeamSampler and keras_nlp.samplers.ContrastiveSampler work on all backends.
• All keras_nlp.metrics classes work on all backends.
  • For Jax and PyTroch, pass python strings to metrics (as tensor strings are strictly tensorflow).
• Restored the mask_positions named argument to MaskedLMHead.

What's Changed

• Update README for Keras Core by @jbischof in #1135
• Ignore errors in UTF-8 decoding by @abheesht17 in #1150
• Ports GPTNeoX to KerasCore by @shivance in #1137
• Small fix for mixed precision generation on tf by @mattdangerw in #1153
• Port DeBERTa to multi-backend by @abheesht17 in #1155
• Change all tensors passed to tf.data.Dataset to numpy by @mattdangerw in #1161
• Fix broken tests by @mattdangerw in #1163
• Pin keras-core to 0.1.0 while investigating failures by @mattdangerw in #1168
• Run GPU tests on Jax + Torch by @ianstenbit in #1160
• Fix flakes in masked lm testing by removing any indeterminism by @mattdangerw in #1171
• Always install the correct version with pip_build by @mattdangerw in #1174
• Remove tests for preprocessing inside a functional model by @mattdangerw in #1175
• Extend the timeout for large tests by @mattdangerw in #1103
• Add GPTNeoXCausalLM by @shivance in #1110
• Bump tensorflow to latest stable by @mattdangerw in #1170
• Add compute_output_shape to tokenizer by @shivance in #1166
• Stop pinning keras-core by @mattdangerw in #1178
• Port FNet by @abheesht17 in #1164
• Automate the update image flow by @mattdangerw in #1179
• Restore mask_position argument name by @mattdangerw in #1185
• Port contrastive sampler to multi-backend by @mattdangerw in #1187
• Port BeamSampler to core by @shivance in #1181
• Port metrics to multi-backend by @mattdangerw in #1186

New Contributors

• @ianstenbit made their first contribution in #1160

Full Changelog: v0.6.0...v0.6.1

v0.6.0

KerasNLP is adding experimental support for Jax and PyTorch backends on top of the Keras Core library. Read the anouncement, and browse the full library documentation, including how to specify the backend when running your code.

Support for both Jax and PyTorch is still experimental, expect some rough edges and please give us feedback!

Summary

• This release should be equivalent to 0.5.2 with the addition of multi-backend support.
• The following API symbols are currently restricted to the tensorflow backend:
  • keras_nlp.models.DebertaV3*
  • keras_nlp.models.FNet*
  • keras_nlp.metrics
  • keras_nlp.samplers.BeamSampler
  • keras_nlp.samplers.ContrastiveSampler
• Note that there are two ways you can run on top of Tensorflow.
  • If you run your scripts/colab without any changes, KerasNLP will use tf.keras for all layer and modeling implementations. This should be a no-op from previous releases of the library.
  • If you run your scripts/colab with KERAS_BACKEND={jax, torch, tensorflow}, you will be trying the new Keras Core library, using the specified backend. This is a great way to test out the future of the library!
  • Full details on runtime specification is available along with the Keras Core documentation.

What's Changed

• small updates to the release doc by @chenmoneygithub in #1031
• Sampler docstring edit by @abuelnasr0 in #1033
• Fix program crash for id_to_token() method in SentencePieceTokenizer by @abuelnasr0 in #1040
• Update our release process to preview docs before release by @mattdangerw in #1043
• Add Whisper Tokenizer and Audio Feature Extractor by @abheesht17 in #847
• Also strip padding token for opt by @mattdangerw in #1028
• Add regex dep by @mattdangerw in #1044
• Add BartSeq2SeqLM and conditional text generation with BART by @abheesht17 in #974
• Support list/tuple inputs for special tokens in StartEndPacker layer by @abheesht17 in #1045
• Support list/tuple inputs for special tokens in MultiSegmentPacker layer by @abheesht17 in #1046
• Fix a misleading part of our cached MHA docs by @mattdangerw in #1048
• Always pass weight name by kwarg by @mattdangerw in #1053
• Always pass metrics in a list or dict by @mattdangerw in #1054
• Move Defaults to to end of arg docstring and standardise values by @SamuelMarks in #1057
• Fix beam search for BART by @abheesht17 in #1058
• Replace tf.dtype with "dtype" by @mattdangerw in #1059
• Test shapes directly by @mattdangerw in #1064
• Clean up metrics tests by @mattdangerw in #1063
• Remove metrics merge tests by @mattdangerw in #1065
• Fix whisper feature inputs by @mattdangerw in #1069
• Always specify shape when creating variables by @mattdangerw in #1067
• Remove ragged support from position embeddings by @mattdangerw in #1068
• Clean up dtype handling for preprocessing layers by @mattdangerw in #1066
• Add BART finetuned on CNN+DM for summarisation by @abheesht17 in #1060
• Fix saving bug by @mattdangerw in #1073
• Fix t5 forward pass by @mattdangerw in #1082
• Feat/make transformer decoder callable without causal mask by @ferraric in #1083
• Adding GPTNeoXBackbone by @shivance in #1056
• Add a common test case by @mattdangerw in #1095
• Update register_keras_serializable to use saving module by @mattdangerw in #1094
• Don't test tf format by @mattdangerw in #1104
• Add GPTNeoXPreprocessor by @shivance in #1093
• Split layers into layers/modeling & layers/preprocessing by @mattdangerw in #1102
• Fix merge conflict from #1102 by @mattdangerw in #1105
• Add a common base class for generative models by @mattdangerw in #1096
• Add GPTNeoXCausalLMPreprocessor by @shivance in #1106
• Add Whisper Presets by @abheesht17 in #1089
• Refactor RotaryEmbedding and GPTNeoXAttention by @shivance in #1101
• Remove all the secret keys for ci by @mattdangerw in #1126
• Fix publish to pypi action by @mattdangerw in #1127
• Unexport models that are not in the 0.6 release by @mattdangerw in #1125
• Bump the version to 0.6.0 by @mattdangerw in #1128

New Contributors

• @SamuelMarks made their first contribution in #1057
• @ferraric made their first contribution in #1083

Full Changelog: v0.5.2...v0.6.0

v0.5.2

What's Changed

• Fix unclosed fenced docstrings by @mattdangerw in #1025
• Fix a bug with computing the output mask after generate by @mattdangerw in #1029

Full Changelog: v0.5.1...v0.5.2

v0.5.1

What's Changed

• Fix typos in export by @chenmoneygithub in #1024

Full Changelog: v0.5.0...v0.5.1

v0.5.0

In this 0.5 release, we are bringing generative AI to KerasNLP!

Summary

• Added text generation task model keras_nlp.models.GPT2CausalLM and keras_nlp.models.OPTCausalLM along with corresponding preprocessors. Both task models exposed a public generate() method for text generation.
• Refactored text generation utils into sampler APIs in keras_nlp.samplers for better UX and scalability.
• Added MaskedLM task models keras_nlp.models.XXXMaskedLM, e.g., keras_nlp.models.BertMaskedLM.

What's Changed

• Update python version in readme to 3.8 by @haifeng-jin in #618
• Modify our pip install line so we upgrade tf by @mattdangerw in #616
• Use Adam optimizer for quick start by @mattdangerw in #620
• Clean up class name and self in calls to super() by @mbrukman in #628
• Update word_piece_tokenizer.py by @ADITYADAS1999 in #617
• Add DeBERTaV3 Conversion Script by @abheesht17 in #633
• Add AlbertTokenizer and AlbertPreprocessor by @abheesht17 in #627
• Create Backbone base class by @jbischof in #621
• Add TPU testing by @chenmoneygithub in #591
• Add Base Preprocessor Class by @abheesht17 in #638
• Add keras_nlp.samplers by @chenmoneygithub in #563
• Add ALBERT Backbone by @abheesht17 in #622
• Add a small script to count parameters in our presets by @mattdangerw in #610
• Clean up examples/ directory by @ADITYADAS1999 in #637
• Fix Small BERT Typo by @abheesht17 in #651
• Rename examples/bert -> examples/bert_pretraining by @mattdangerw in #647
• Add FNet Preprocessor by @abheesht17 in #646
• Add FNet Backbone by @abheesht17 in #643
• Small DeBERTa Docstring Fixes by @abheesht17 in #666
• Add Fenced Docstring Testing by @abheesht17 in #640
• Corrected the epsilon value by @soma2000-lang in #665
• Consolidate docstring formatting weirdness in Backbone and Preprocessor base classes by @mattdangerw in #654
• Fix value_dim in TransformerDecoder's cross-attn layer by @abheesht17 in #667
• Add ALBERT Presets by @abheesht17 in #655
• Add Base Task Class by @abheesht17 in #671
• Implement TopP, TopK and Beam samplers by @chenmoneygithub in #652
• Add FNet Presets by @abheesht17 in #659
• Bump the year to 2023 by @mattdangerw in #679
• Add BART Backbone by @abheesht17 in #661
• Handle trainable and name in the backbone base class by @mattdangerw in #680
• Ignore Task Docstring for Testing by @abheesht17 in #683
• Light-weight benchmarking script by @NusretOzates in #664
• Conditionally import tf_text everywhere by @mattdangerw in #684
• Expose token_embedding as a Backbone Property by @abheesht17 in #676
• Move from_preset to base tokenizer classes by @shivance in #673
• add f_net_classifier and f_net_classifier_test by @ADITYADAS1999 in #670
• import rouge_scorer directly from rouge_score package by @sampathweb in #691
• Fix typo in requirements file juypter -> jupyter by @mattdangerw in #693
• Temporary fix to get nightly green again by @mattdangerw in #696
• GPT2 Text Generation APIs by @chenmoneygithub in #592
• Run keras saving tests on nightly and fix RobertaClassifier test by @mattdangerw in #692
• Speed up pip install keras-nlp; simplify deps by @mattdangerw in #697
• Add AlbertClassifier by @shivance in #668
• Make tokenizer, backbone, preprocessor properties settable on base class by @mattdangerw in #700
• Update to latest black by @mattdangerw in #708
• RobertaMaskedLM task and preprocessor by @mattdangerw in #653
• Default compilation for BERT/RoBERTa classifiers by @jbischof in #695
• Add start/end token padding to GPT2Preprocessor by @chenmoneygithub in #704
• Don't install tf stable when building our nightly image by @mattdangerw in #711
• Add OPT Backbone and Tokenizer by @mattdangerw in #699
• Small OPT Doc-string Edits by @abheesht17 in #716
• Default compilation other classifiers by @Plutone11011 in #714
• Add BartTokenizer and BART Presets by @abheesht17 in #685
• Add an add_prefix_space Arg in BytePairTokenizer by @shivance in #715
• Opt presets by @mattdangerw in #707
• fix import of tensorflow_text in tf_utils by @sampathweb in #723
• Check for masked token in roberta tokenizer by @mattdangerw in #742
• Improve test coverage for special tokens in model tokenizers by @mattdangerw in #743
• Fix the sampler truncation strategy by @chenmoneygithub in #713
• Add ALBERT Conversion Script by @abheesht17 in #736
• Add FNet Conversion Script by @abheesht17 in #737
• Add BART Conversion Script by @abheesht17 in #739
• Pass Correct LayerNorm Epsilon value to TransformerEncoder in Backbones by @TheAthleticCoder in #731
• Improving the layer Description. by @Neeshamraghav012 in #734
• Adding ragged support to SinePositionEncoding by @apupneja in #751
• Fix trailing space by @mattdangerw in #755
• Adding an AlbertMaskedLM task + Fix Projection layer dimension in MaskedLMHead by @shivance in #725
• New docstring example for TokenAndPosition Embedding layer. by @Neeshamraghav012 in #760
• Add a note for TPU issues for deberta_v3 by @mattdangerw in #758
• Add missing exports to models API by @mattdangerw in #763
• Autogenerate preset table by @Cyber-Machine in #690
• Version bump to 0.5.0 by @mattdangerw in #767
• Adding a FNetMaskedLM task model and preprocessor by @apupneja in #740
• Add a DistilBertMaskedLM task model by @ADITYADAS1999 in #724
• Add cache support to decoding journey by @chenmoneygithub in #745
• Handle [MASK] token in DebertaV3Tokenizer by @abheesht17 in #759
• Update README for 2.4.1 release by @mattdangerw in #757
• Fix typo in test docstring by @jbischof in #791
• Fixed Incorrect Links for FNet and DeBERTaV3 models by @Cyber-Machine in #793
• Patch 1 - doc-string spell fix by @atharvapurdue in #781
• Don't rely on core keras initializer config details by @mattdangerw in #802
• Simplify the cache decoding graph by @mattdangerw in #780
• Fix Fenced Doc-String #782 by @atharvapurdue in #785
• Solve #721 Deberta masklm model by @Plutone11011 in #732
• Add from_config to sampler by @mattdangerw in #803
• BertMaskedLM Task Model and Preprocessor by @Cyber-Machine in #774
• Stop generation once end_token_id is seen by @chenmoneygithub in https://github.com/k...

v0.4.1

The 0.4.1 release is a minor release with new model architectures and compilation defaults for task models. If you encounter any problems or have questions, please open an issue!

Summary

• Added compilation defaults for all task models (e.g. keras_nlp.models.BertClassifier). No existing functionality is changed, but users of task models can now skip calling .compile() and use default learning rates and optimization strategies provided by the library.
• Added keras_nlp.models.AlbertBackbone, keras_nlp.models.AlbertClassifier, preprocessor, and tokenizer layers for pre-trained ALBERT models.
• Added keras_nlp.models.FNetBackbone, keras_nlp.models.FNetClassifier, preprocessor, and tokenizer layers for pre-trained FNet models.
• Added keras_nlp.models.DebertaV3Backbone, keras_nlp.models.DebertaV3Classifier, preprocessor, and tokenizer layers for pre-trained DeBERTaV3 models.

What's Changed

• Update python version in readme to 3.8 by @haifeng-jin in #618
• Modify our pip install line so we upgrade tf by @mattdangerw in #616
• Use Adam optimizer for quick start by @mattdangerw in #620
• Clean up class name and self in calls to super() by @mbrukman in #628
• Update word_piece_tokenizer.py by @ADITYADAS1999 in #617
• Add DeBERTaV3 Conversion Script by @abheesht17 in #633
• Add AlbertTokenizer and AlbertPreprocessor by @abheesht17 in #627
• Create Backbone base class by @jbischof in #621
• Add TPU testing by @chenmoneygithub in #591
• Add Base Preprocessor Class by @abheesht17 in #638
• Add keras_nlp.samplers by @chenmoneygithub in #563
• Add ALBERT Backbone by @abheesht17 in #622
• Add a small script to count parameters in our presets by @mattdangerw in #610
• Clean up examples/ directory by @ADITYADAS1999 in #637
• Fix Small BERT Typo by @abheesht17 in #651
• Rename examples/bert -> examples/bert_pretraining by @mattdangerw in #647
• Add FNet Preprocessor by @abheesht17 in #646
• Add FNet Backbone by @abheesht17 in #643
• Small DeBERTa Docstring Fixes by @abheesht17 in #666
• Add Fenced Docstring Testing by @abheesht17 in #640
• Corrected the epsilon value by @soma2000-lang in #665
• Consolidate docstring formatting weirdness in Backbone and Preprocessor base classes by @mattdangerw in #654
• Fix value_dim in TransformerDecoder's cross-attn layer by @abheesht17 in #667
• Add ALBERT Presets by @abheesht17 in #655
• Add Base Task Class by @abheesht17 in #671
• Implement TopP, TopK and Beam samplers by @chenmoneygithub in #652
• Add FNet Presets by @abheesht17 in #659
• Bump the year to 2023 by @mattdangerw in #679
• Add BART Backbone by @abheesht17 in #661
• Handle trainable and name in the backbone base class by @mattdangerw in #680
• Ignore Task Docstring for Testing by @abheesht17 in #683
• Light-weight benchmarking script by @NusretOzates in #664
• Conditionally import tf_text everywhere by @mattdangerw in #684
• Expose token_embedding as a Backbone Property by @abheesht17 in #676
• Move from_preset to base tokenizer classes by @shivance in #673
• add f_net_classifier and f_net_classifier_test by @ADITYADAS1999 in #670
• import rouge_scorer directly from rouge_score package by @sampathweb in #691
• Fix typo in requirements file juypter -> jupyter by @mattdangerw in #693
• Temporary fix to get nightly green again by @mattdangerw in #696
• GPT2 Text Generation APIs by @chenmoneygithub in #592
• Run keras saving tests on nightly and fix RobertaClassifier test by @mattdangerw in #692
• Speed up pip install keras-nlp; simplify deps by @mattdangerw in #697
• Add AlbertClassifier by @shivance in #668
• Make tokenizer, backbone, preprocessor properties settable on base class by @mattdangerw in #700
• Update to latest black by @mattdangerw in #708
• RobertaMaskedLM task and preprocessor by @mattdangerw in #653
• Default compilation for BERT/RoBERTa classifiers by @jbischof in #695
• Add start/end token padding to GPT2Preprocessor by @chenmoneygithub in #704
• Don't install tf stable when building our nightly image by @mattdangerw in #711
• Add OPT Backbone and Tokenizer by @mattdangerw in #699
• Small OPT Doc-string Edits by @abheesht17 in #716
• Default compilation other classifiers by @Plutone11011 in #714
• Add BartTokenizer and BART Presets by @abheesht17 in #685
• Add an add_prefix_space Arg in BytePairTokenizer by @shivance in #715
• Opt presets by @mattdangerw in #707
• fix import of tensorflow_text in tf_utils by @sampathweb in #723
• Check for masked token in roberta tokenizer by @mattdangerw in #742
• Improve test coverage for special tokens in model tokenizers by @mattdangerw in #743
• Fix the sampler truncation strategy by @chenmoneygithub in #713
• Add ALBERT Conversion Script by @abheesht17 in #736
• Add FNet Conversion Script by @abheesht17 in #737
• Add BART Conversion Script by @abheesht17 in #739
• Pass Correct LayerNorm Epsilon value to TransformerEncoder in Backbones by @TheAthleticCoder in #731
• Improving the layer Description. by @Neeshamraghav012 in #734
• Adding ragged support to SinePositionEncoding by @apupneja in #751
• Fix trailing space by @mattdangerw in #755
• Adding an AlbertMaskedLM task model and preprocessor by @shivance in #725
• New docstring example for TokenAndPosition Embedding layer. by @Neeshamraghav012 in #760
• Add a note for TPU issues for deberta_v3 by @mattdangerw in #758
• Add missing exports to models API by @mattdangerw in #763
• Autogenerate preset table by @Cyber-Machine in #690
• Remove work in progress API for 0.4.1 release by @mattdangerw in #765
• Version bump to 0.4.1.dev0 by @mattdangerw in #766
• Version bump to 0.4.1 by @mattdangerw in #768

New Contributors

• @haifeng-jin made their first contribution in #618
• @mbrukman made their first contribution in #628
• @soma2000-lang made their first contribution in #665
• @NusretOzates made their first contribution in #664
• @shivance made their first contribution in #673
• @Plutone11011 made their first contribution in #714
• @TheAthleticCoder made their first contribution in #731
• @Neeshamraghav012 made their first contribution in #734
• @Cyber-Machine made their first contribution in https://github.com/keras-team/keras-nlp/pull/...

r0.4.1.dev0

Summary

• Dev release to test out the upcoming 0.4.1.

What's Changed

• Update python version in readme to 3.8 by @haifeng-jin in #618
• Modify our pip install line so we upgrade tf by @mattdangerw in #616
• Use Adam optimizer for quick start by @mattdangerw in #620
• Clean up class name and self in calls to super() by @mbrukman in #628
• Update word_piece_tokenizer.py by @ADITYADAS1999 in #617
• Add DeBERTaV3 Conversion Script by @abheesht17 in #633
• Add AlbertTokenizer and AlbertPreprocessor by @abheesht17 in #627
• Create Backbone base class by @jbischof in #621
• Add TPU testing by @chenmoneygithub in #591
• Add Base Preprocessor Class by @abheesht17 in #638
• Add keras_nlp.samplers by @chenmoneygithub in #563
• Add ALBERT Backbone by @abheesht17 in #622
• Add a small script to count parameters in our presets by @mattdangerw in #610
• Clean up examples/ directory by @ADITYADAS1999 in #637
• Fix Small BERT Typo by @abheesht17 in #651
• Rename examples/bert -> examples/bert_pretraining by @mattdangerw in #647
• Add FNet Preprocessor by @abheesht17 in #646
• Add FNet Backbone by @abheesht17 in #643
• Small DeBERTa Docstring Fixes by @abheesht17 in #666
• Add Fenced Docstring Testing by @abheesht17 in #640
• Corrected the epsilon value by @soma2000-lang in #665
• Consolidate docstring formatting weirdness in Backbone and Preprocessor base classes by @mattdangerw in #654
• Fix value_dim in TransformerDecoder's cross-attn layer by @abheesht17 in #667
• Add ALBERT Presets by @abheesht17 in #655
• Add Base Task Class by @abheesht17 in #671
• Implement TopP, TopK and Beam samplers by @chenmoneygithub in #652
• Add FNet Presets by @abheesht17 in #659
• Bump the year to 2023 by @mattdangerw in #679
• Add BART Backbone by @abheesht17 in #661
• Handle trainable and name in the backbone base class by @mattdangerw in #680
• Ignore Task Docstring for Testing by @abheesht17 in #683
• Light-weight benchmarking script by @NusretOzates in #664
• Conditionally import tf_text everywhere by @mattdangerw in #684
• Expose token_embedding as a Backbone Property by @abheesht17 in #676
• Move from_preset to base tokenizer classes by @shivance in #673
• add f_net_classifier and f_net_classifier_test by @ADITYADAS1999 in #670
• import rouge_scorer directly from rouge_score package by @sampathweb in #691
• Fix typo in requirements file juypter -> jupyter by @mattdangerw in #693
• Temporary fix to get nightly green again by @mattdangerw in #696
• GPT2 Text Generation APIs by @chenmoneygithub in #592
• Run keras saving tests on nightly and fix RobertaClassifier test by @mattdangerw in #692
• Speed up pip install keras-nlp; simplify deps by @mattdangerw in #697
• Add AlbertClassifier by @shivance in #668
• Make tokenizer, backbone, preprocessor properties settable on base class by @mattdangerw in #700
• Update to latest black by @mattdangerw in #708
• RobertaMaskedLM task and preprocessor by @mattdangerw in #653
• Default compilation for BERT/RoBERTa classifiers by @jbischof in #695
• Add start/end token padding to GPT2Preprocessor by @chenmoneygithub in #704
• Don't install tf stable when building our nightly image by @mattdangerw in #711
• Add OPT Backbone and Tokenizer by @mattdangerw in #699
• Small OPT Doc-string Edits by @abheesht17 in #716
• Default compilation other classifiers by @Plutone11011 in #714
• Add BartTokenizer and BART Presets by @abheesht17 in #685
• Add an add_prefix_space Arg in BytePairTokenizer by @shivance in #715
• Opt presets by @mattdangerw in #707
• fix import of tensorflow_text in tf_utils by @sampathweb in #723
• Check for masked token in roberta tokenizer by @mattdangerw in #742
• Improve test coverage for special tokens in model tokenizers by @mattdangerw in #743
• Fix the sampler truncation strategy by @chenmoneygithub in #713
• Add ALBERT Conversion Script by @abheesht17 in #736
• Add FNet Conversion Script by @abheesht17 in #737
• Add BART Conversion Script by @abheesht17 in #739
• Pass Correct LayerNorm Epsilon value to TransformerEncoder in Backbones by @TheAthleticCoder in #731
• Improving the layer Description. by @Neeshamraghav012 in #734
• Adding ragged support to SinePositionEncoding by @apupneja in #751
• Fix trailing space by @mattdangerw in #755
• Adding an AlbertMaskedLM task model and preprocessor by @shivance in #725
• New docstring example for TokenAndPosition Embedding layer. by @Neeshamraghav012 in #760
• Add a note for TPU issues for deberta_v3 by @mattdangerw in #758
• Add missing exports to models API by @mattdangerw in #763
• Autogenerate preset table by @Cyber-Machine in #690
• Version bump to 0.5.0 by @mattdangerw in #767

New Contributors

• @haifeng-jin made their first contribution in #618
• @mbrukman made their first contribution in #628
• @soma2000-lang made their first contribution in #665
• @NusretOzates made their first contribution in #664
• @shivance made their first contribution in #673
• @Plutone11011 made their first contribution in #714
• @TheAthleticCoder made their first contribution in #731
• @Neeshamraghav012 made their first contribution in #734
• @apupneja made their first contribution in #751
• @Cyber-Machine made their first contribution in #690

Full Changelog: v0.4.0...v0.4.1.dev0