Can BERT be finetuned for text generation?

[jefke24]

Maybe someone can clarify this for me or point me in the right direction.
I've been reading up and trying to find more information online, but for me it's still not clear.

So I'm aware that you can finetune the pretrained GPT2 model to generate text for a more specialized case, I'm clear on that.
But GPT2 is a monster of a model, even the base-version, as most GPT-models are.

I've seen a lot of examples where BERT is being finetuned, but I'm still wondering: can BERT be finetuned for text generation?
I've seen examples of finetuning for classification, for sentiment analysis etc.
But it seems BERT can not be finetuned for text generation, is this correct or am I wrong in this?

Or is there some other more lightweight pretrained model which could be finetuned for text generation, except BERT or GPT2?
I'm realistic, I'm not expecting the same type of results with such a model like with GPT2, obviously.
But is there anything else out there, more lightweight?

Preferably a transformer-style model, I'm also aware of using LSTM or RNN models (with GRU) to do text generation.

[abuelnasr0]

you can't use an encoder model for text generation. and bert is an encoder model. only decoder or encoder-decoder models can be used for text generation.
there is another decoder model which is OPT but it's even larger that GPT.

[abuelnasr0]

#1030 (comment)
use this solution with training GPT from scratch example it worked.