LEMON: Lossless model expansion
Published in ICLR 2024, 2023
Yite Wang, Jiahao Su, Hanlin Lu, Cong Xie, Tianyi Liu, Jianbo Yuan, Haibin Lin, Ruoyu Sun, and Hongxia Yang.
LEMON proposes a lossless model expansion approach that scales model size while preserving previous functionality.