This article provides a summary of the papers presented at the AI4Dev and LLM4HPC workshops, held recently at major high-performance computing (HPC) conferences (ISC'25, ICPP'25), which highlight recent advancements in leveraging state-of-the-art AI and LLMs to enhance scientific software development.
Introduction
The release of ChatGPT in late 2022 was a revolutionary step for human-computer interactions. Since then, the AI landscape has evolved rapidly, opening new avenues for enhancing software development processes via state-of-the-art large language models (LLMs) and agentic AI. Recognizing this potential, one of the authors organized one of the first workshops tackling the intersection of AI and scientific software development, AI4Dev (SC'23 Laguna et al.). This vision has paid off, as AI is now at the forefront of HPC research and development, shifting paradigms in how scientific software is created, optimized, and maintained. Following this initial success, we have continued creating venues in top HPC conferences where researchers and practitioners can share their latest advances in software development aided by AI. This article summarizes the key insights and advances related to better scientific software in the recent workshops hosted at ISC'25 and ICPP'25.
LLM4HPC at ISC'25
The ISC'25 1st LLM4HPC workshop had two paper presentations on the topic of leveraging LLMs for HPC software development, gathering approximately 30 attendees for the first-time workshop.
Rin Tanaka (University of Electro-Communications) presented "Analysis of MPI Parallel Code Generated by GPT-4o" showing a comprehensive evaluation of GPT-4o's ability to generate parallel code using the standard message passing interface (MPI). Unsurprisingly, they conclude that MPI codes were generated incorrectly, even though many passed unit tests. The work was recognized with a best paper distinction by the workshop committee.
Pedro Valero-Lara (Oak Ridge National Laboratory) presented "Leveraging AI for Productive and Trustworthy HPC Software: Challenges and Research Directions" showcasing several research directions to leverage AI for the HPC software development lifecycle. The paper captures the goals and priorities of the U.S. Department of Energy (DOE)-funded Durban and Ellora projects as part of the Advanced Scientific Computing Research (ASCR) "Advancements in Artificial Intelligence for Science" program.
AI4Dev at ICPP'25
The ICPP'25 AI4Dev workshop featured two paper presentations and a panel discussion on the intersection of AI and scientific software development. The half-day workshop gathered nearly 30 attendees.
- A keynote presentation by Abhinav Bhatele (University of Maryland, College Park) titled “Lost in Translation: LLMs and Whole-Repository Porting of HPC Codes,” showcased his group’s research efforts on the capabilities of current LLMs, the current state-of-the-art, and limitations in this important topic.
Papers presented:
Bowen Cui (George Mason University) presented the first accepted paper titled “Comprehensive Evaluation of LLMs in HPC Code Performance Optimization,” which identified advantages and gaps in LLMs for HPC motif for real applications.
The second paper, “AI Assistants to Enhance and Exploit the PETSc Knowledge Base,” was presented by Junchao Zhang (Argonne National Laboratory). The work showcased how industry-driven AI technologies can enhance the development process of well-established libraries, with focus on the Portable, Extensible Toolkit for Scientific Computation (PETSc), widely used in HPC. The paper highlights the fact that, as a mature library, PETSc has a great deal of valuable, but unstructured, information available, so there is a need to adapt this knowledge, along with the niche nature of HPC, to make LLMs more effective for their developer and user community.
Panel discussion:
The workshop ended with a panel of experts who are actively working on the use of AI for Development:
Sunita Chandrasekaran (University of Delaware) provided a comprehensive overview of her group's efforts on LLM4VV, an ecosystem to leverage LLMs for Validation and Verification for OpenMP and OpenACC test suites.
Ali Jannesari (Iowa State University) presented "AI for HPC Code: From Representation to Generation and Performance Analysis," a recount of the efforts of his Software Analytics and Pervasive Parallelism Lab, covering a broad spectrum of the HPC code development process, including their state-of-the-art CodeRosetta work.
Jeffrey Carver (University of Alabama) provided an overview at the intersection of HPC and software engineering research, including insights on where teams spend their time developing software, positive peer-review experiences, and testing challenges, aspects that can be leveraged with the proper use of AI.
Daniel Nichols (Lawrence Livermore National Laboratory) presented his research efforts in the last few years on the LLM's capabilities to write parallel code, which includes his work on the HPC-Coder tool and the ParEval benchmark suite.
Overall, the panel highlighted the importance of collaboration between AI researchers and HPC software developers to address the unique challenges in this domain.
Future directions and events
AI is here to stay as the largest technological revolution of our time, and HPC software development can be significantly improved by leveraging AI technologies. Through these workshops, we have explored some of the findings and insights from the community experts at the intersection of AI and HPC.
For those interested in community activities or submitting papers on this topic, the following upcoming workshops are planned for 2026:
Acknowledgments
This work was supported by the U.S. Department of Energy (DOE) Office of Science, Advanced Scientific Computing Research (ASCR) under the "Advancements in Artificial Intelligence for Science" program and the "Next Generation Scientific Software Technologies" program. We thank the above speakers (many BSSw fellows and honorable mentions) and co-authors for their contributions to the workshops and to this article.
Author bios
William F. Godoy is a Senior Computer Scientist in the Computer Science and Mathematics Division at Oak Ridge National Laboratory (ORNL). His interests are in the areas of HPC, AI for scientific software, programming models, and workflows. He is a 2022 BSSw Fellowship honorable mention, an IEEE Senior member, an ACM member, and a US-RSE member.
Ignacio Laguna is the Group Leader of the Parallel Systems Group (PSG) at the Center for Applied Scientific Computing (CASC) at the Lawrence Livermore National Laboratory (LLNL), California. His main area of research in HPC is programming models and systems, software correctness, program analysis, debugging, compilers, testing, and fault tolerance. He is a 2019 BSSw Fellow and an IEEE Senior member.
Pedro Valero-Lara is a Senior Computer Scientist at Oak Ridge National Laboratory (ORNL). His research interests include parallel programming models, math libraries, applications, and AI for the scientific software ecosystem. He is a 2020 IEEE TCHPC Early Award recipient and an IEEE member.
