Recent artificial intelligence advances have largely focused on text, but AI increasingly shows promise in other contexts, including manufacturing and the service industry. In these sectors, targeted AI improvements can improve product quality and worker safety, according to a new study co-authored by an interdisciplinary team of experts from the University of Notre Dame.

The study, published in Information Fusion, explores how a class of AI tools capable of processing multiple types of inputs and reasoning can affect the future of work. These tools, which include ChatGPT, are known as multimodal large language models. And while most studies on AI and work have focused on office work, this new research examined production work settings, where the benefits of AI may seem less apparent.

Notre Dame researchers collaborated with Indiana welding experts at the Elkhart Area Career Center, Plymouth High School, Career Academy South Bend, Plumbers & Pipefitters Local Union 172 and Ivy Tech Community College to gather images for the study, leveraging relationships cultivated through the work of the University’s iNDustry Labs. Northern Indiana has one of the highest concentrations of manufacturing jobs in the United States and iNDustry Labs has collaborated with more than 80 companies in the region on more than 200 projects.

Research focused on welding across several industries: RV and marine, aeronautical and farming. The study examined how accurately large language models assessed weld images to determine whether the welds shown would work for different products. Researchers found that while these AI tools showed promise in assessing weld quality, they performed significantly better analyzing curated online images compared to actual welds.

“This discrepancy underscores the need to incorporate real-world welding data when training these AI models, and to use more advanced knowledge distillation strategies when interacting with AI,” said co-author Nitesh Chawla, the Frank M. Freimann Professor of Computer Science and Engineering at the University of Notre Dame and the founding director of the University’s Lucy Family Institute for Data and Society. “That will help AI systems ensure that welds work as they should. Ultimately, this will help improve worker safety, product quality and economic opportunity.”

Researchers discovered that context-specific prompts may enhance the performance of AI models in some cases, and noted that the size or complexity of the models did not necessarily lead to better performance. Ultimately, the study’s co-authors recommended that future studies focus on improving models’ ability to reason in unfamiliar domains.

“Our study shows the need to fine-tune AI to be more effective in manufacturing and to provide more robust reasoning and responses in industrial applications,” said Grigorii Khvatski, a doctoral student in Notre Dame’s Department of Computer Science and Engineering and a Lucy Family Institute Scholar.

Yong Suk Lee, associate professor of technology, economy and global affairs in Notre Dame’s Keough School of Global Affairs and program chair for technology ethics at Notre Dame's Institute for Ethics and the Common Good, said the study’s findings have important implications for the future of work.

“As AI adoption in industrial contexts grows, practitioners will need to balance the trade-offs between using complex, expensive general-purpose models and opting for fine-tuned models that better meet industry needs,” Lee said. “Integrating explainable AI into these decision-making frameworks will be critical to ensuring that AI systems are not only effective but also transparent and accountable.”

The study received funding from the U.S. National Science Foundation Future of Work program and is one of the federally funded research projects at the University of Notre Dame.

In addition to Chawla, Khvatski and Lee, study co-authors include Corey Angst, the Jack and Joan McGraw Family Collegiate Professor of IT, Analytics and Operations in the University’s Mendoza College of Business; Maria Gibbs, senior director of Notre Dame’s iNDustry Labs; and Robert Landers, advanced manufacturing collegiate professor in Notre Dame’s College of Engineering.

Originally published by Josh Stowe at keough.nd.edu on May 5.

Contact: Tracy DeStazio, associate director of media relations, 574-631-9958 or tdestazi@nd.edu