Researchers publicly released the Flan-T5 checkpoints, which provide high performance in a few shots compared to the much larger PaLM 62B model.
Fine-tuning language models for a group of data sets formulated as instructions has been instrumental in improving model generalization and performance for unseen tasks. In an effort to move forward, Google AI has released a new open-source language model, Flan-T5 , which is capable of over 1,800 different tasks. The paper primarily explores fine-tuning instructions in areas such as scaling the number of tasks and model size, as well as thought-chain data. The paper states, “We found that fine-tuning instructions with the above aspects significantly improves performance on various model classes ( PaLM , T5 , U-PaLM), cue settings (zero-shot, multiple-shot, CoT) and evaluation tests. (MMLU, BBH, TyDiQA, MGSM, open generation)”.The team publicly released the Flan-T5 checkpoints, which provide high performance in a few shots .
compared to the much larger PaLM 62B model. In addition, fine-tuning instructions is a common method used to improve performance and usability of pre-trained language models. Researchers claim that with Flan-T5, the new model will lead to improved prompting abilities and multi-step reasoning.