SimpleText Best of Labs in CLEF-2023: Scientific Text Simplification Using Multi-prompt Minimum Bayes Risk Decoding

We investigate the use of large language models (LLMs) for scientific text simplification in the context of SimpleText in CLEF-2023 Shared Task 3. Our methodology integrates fine-tuning of the Alpaca LoRA 7B model, collecting candidate simplifications from different prompt designs and multistep prompts using data from Task 2, which includes complex terms and their definitions. In this way, we generate a variety of simplification candidates, including candidates that provide definitions of complex terms in the text. The multi-prompt candidates are then re-ranked using Minimum Bayes Risk Decoding with LENS as the utility function, resulting in a number of interesting source-prompt distributions and better SARI. An additional perturbation ablation study is performed, which shows that the efficient reference-less metric Simplicity Level Estimate (SLE) doesn’t rate ungrammatical simplifications lower, revealing its inadequacy as a selection criterion. Finally, we observe an ablation between the simplifications of the domain-adapted Alpaca LoRA and the newer LLama3 Instruct, indicating the adequacy of older models to compete with newer stronger models through in-domain instruction tuning and Minimum Bayes Risk Decoding.