Researchers have developed a new technique to enhance the performance of large language models without requiring additional computational power for fine-tuning. This method, known as WeGeFT (pronounced wee-gift), is reported to outperform previous techniques in various tasks such as commonsense reasoning, arithmetic reasoning, instruction following, code generation, and visual recognition.
Large language models are AI systems pretrained on extensive datasets to predict word sequences in response to user queries. However, their general nature leaves room for improvement when addressing specific topics like math questions or computer code writing.
“In order to improve a model’s ability to perform more specific tasks, you need to fine-tune the model,” says Tianfu Wu, co-corresponding author of a paper on the work and an associate professor of computer engineering at North Carolina State University. “However, these models are so large that it is not feasible to re-train the entire model. Instead, you want to determine the smallest number of changes necessary to improve the model’s performance. We’ve developed a technique, called WeGeFT (pronounced wee-gift), that represents a significant advance for fine-tuning these large models.”
The breakthrough in fine-tuning these large models was LoRA (Low-Rank Adaptation), introduced in 2022. LoRA identifies key parameters likely to enhance a model’s task-specific performance using mathematical tools. Attempts to improve upon LoRA often required more computational power or failed to boost performance with existing resources.
“WeGeFT builds on LoRA but incorporates additional mathematical tools that allow us to determine which of the key parameters the model is already familiar with and which parameters the model would need to ‘learn,’” says Wu. “By placing more weight on the truly novel parameters, we are able to improve model performance compared to LoRA without incorporating significant new computational demands.”
Proof-of-concept testing showed WeGeFT performed as well as or better than LoRA and its variants across various downstream tasks: commonsense reasoning, arithmetic reasoning, instruction following, code generation, and visual recognition.
“We think this is a valuable step forward,” Wu says. “We are now exploring ways that WeGeFT could also be used to identify elements of the model that are responsible for harmful outputs with the goal of improving AI alignment and ‘surgery’ to improve model safety and outputs. We expect that work to be forthcoming.”
The research paper titled “WeGeFT: Weight-Generative Fine-Tuning for Multi-Faceted Efficient Adaptation of Large Models” will be presented at the International Conference on Machine Learning from July 13-19 in Vancouver, Canada. The paper was co-authored by Chinmay Savadikar from NC State and independent researcher Xi Song.
This research received support from grants provided by the National Science Foundation under numbers 1909644, 2024688, and 2013451; along with funding from the Army Research Office under grants W911NF1810295 and W911NF2210010.
