v0.4.1

The 0.4.1 release is a minor release with new model architectures and compilation defaults for task models. If you encounter any problems or have questions, please open an issue!

Summary

• Added compilation defaults for all task models (e.g. keras_nlp.models.BertClassifier). No existing functionality is changed, but users of task models can now skip calling .compile() and use default learning rates and optimization strategies provided by the library.
• Added keras_nlp.models.AlbertBackbone, keras_nlp.models.AlbertClassifier, preprocessor, and tokenizer layers for pre-trained ALBERT models.
• Added keras_nlp.models.FNetBackbone, keras_nlp.models.FNetClassifier, preprocessor, and tokenizer layers for pre-trained FNet models.
• Added keras_nlp.models.DebertaV3Backbone, keras_nlp.models.DebertaV3Classifier, preprocessor, and tokenizer layers for pre-trained DeBERTaV3 models.

What's Changed

• Update python version in readme to 3.8 by @haifeng-jin in #618
• Modify our pip install line so we upgrade tf by @mattdangerw in #616
• Use Adam optimizer for quick start by @mattdangerw in #620
• Clean up class name and self in calls to super() by @mbrukman in #628
• Update word_piece_tokenizer.py by @ADITYADAS1999 in #617
• Add DeBERTaV3 Conversion Script by @abheesht17 in #633
• Add AlbertTokenizer and AlbertPreprocessor by @abheesht17 in #627
• Create Backbone base class by @jbischof in #621
• Add TPU testing by @chenmoneygithub in #591
• Add Base Preprocessor Class by @abheesht17 in #638
• Add keras_nlp.samplers by @chenmoneygithub in #563
• Add ALBERT Backbone by @abheesht17 in #622
• Add a small script to count parameters in our presets by @mattdangerw in #610
• Clean up examples/ directory by @ADITYADAS1999 in #637
• Fix Small BERT Typo by @abheesht17 in #651
• Rename examples/bert -> examples/bert_pretraining by @mattdangerw in #647
• Add FNet Preprocessor by @abheesht17 in #646
• Add FNet Backbone by @abheesht17 in #643
• Small DeBERTa Docstring Fixes by @abheesht17 in #666
• Add Fenced Docstring Testing by @abheesht17 in #640
• Corrected the epsilon value by @soma2000-lang in #665
• Consolidate docstring formatting weirdness in Backbone and Preprocessor base classes by @mattdangerw in #654
• Fix value_dim in TransformerDecoder's cross-attn layer by @abheesht17 in #667
• Add ALBERT Presets by @abheesht17 in #655
• Add Base Task Class by @abheesht17 in #671
• Implement TopP, TopK and Beam samplers by @chenmoneygithub in #652
• Add FNet Presets by @abheesht17 in #659
• Bump the year to 2023 by @mattdangerw in #679
• Add BART Backbone by @abheesht17 in #661
• Handle trainable and name in the backbone base class by @mattdangerw in #680
• Ignore Task Docstring for Testing by @abheesht17 in #683
• Light-weight benchmarking script by @NusretOzates in #664
• Conditionally import tf_text everywhere by @mattdangerw in #684
• Expose token_embedding as a Backbone Property by @abheesht17 in #676
• Move from_preset to base tokenizer classes by @shivance in #673
• add f_net_classifier and f_net_classifier_test by @ADITYADAS1999 in #670
• import rouge_scorer directly from rouge_score package by @sampathweb in #691
• Fix typo in requirements file juypter -> jupyter by @mattdangerw in #693
• Temporary fix to get nightly green again by @mattdangerw in #696
• GPT2 Text Generation APIs by @chenmoneygithub in #592
• Run keras saving tests on nightly and fix RobertaClassifier test by @mattdangerw in #692
• Speed up pip install keras-nlp; simplify deps by @mattdangerw in #697
• Add AlbertClassifier by @shivance in #668
• Make tokenizer, backbone, preprocessor properties settable on base class by @mattdangerw in #700
• Update to latest black by @mattdangerw in #708
• RobertaMaskedLM task and preprocessor by @mattdangerw in #653
• Default compilation for BERT/RoBERTa classifiers by @jbischof in #695
• Add start/end token padding to GPT2Preprocessor by @chenmoneygithub in #704
• Don't install tf stable when building our nightly image by @mattdangerw in #711
• Add OPT Backbone and Tokenizer by @mattdangerw in #699
• Small OPT Doc-string Edits by @abheesht17 in #716
• Default compilation other classifiers by @Plutone11011 in #714
• Add BartTokenizer and BART Presets by @abheesht17 in #685
• Add an add_prefix_space Arg in BytePairTokenizer by @shivance in #715
• Opt presets by @mattdangerw in #707
• fix import of tensorflow_text in tf_utils by @sampathweb in #723
• Check for masked token in roberta tokenizer by @mattdangerw in #742
• Improve test coverage for special tokens in model tokenizers by @mattdangerw in #743
• Fix the sampler truncation strategy by @chenmoneygithub in #713
• Add ALBERT Conversion Script by @abheesht17 in #736
• Add FNet Conversion Script by @abheesht17 in #737
• Add BART Conversion Script by @abheesht17 in #739
• Pass Correct LayerNorm Epsilon value to TransformerEncoder in Backbones by @TheAthleticCoder in #731
• Improving the layer Description. by @Neeshamraghav012 in #734
• Adding ragged support to SinePositionEncoding by @apupneja in #751
• Fix trailing space by @mattdangerw in #755
• Adding an AlbertMaskedLM task model and preprocessor by @shivance in #725
• New docstring example for TokenAndPosition Embedding layer. by @Neeshamraghav012 in #760
• Add a note for TPU issues for deberta_v3 by @mattdangerw in #758
• Add missing exports to models API by @mattdangerw in #763
• Autogenerate preset table by @Cyber-Machine in #690
• Remove work in progress API for 0.4.1 release by @mattdangerw in #765
• Version bump to 0.4.1.dev0 by @mattdangerw in #766
• Version bump to 0.4.1 by @mattdangerw in #768

New Contributors

• @haifeng-jin made their first contribution in #618
• @mbrukman made their first contribution in #628
• @soma2000-lang made their first contribution in #665
• @NusretOzates made their first contribution in #664
• @shivance made their first contribution in #673
• @Plutone11011 made their first contribution in #714
• @TheAthleticCoder made their first contribution in #731
• @Neeshamraghav012 made their first contribution in #734
• @Cyber-Machine made their first contribution in #690

Full Changelog: v0.4.0...v0.4.1