Birds of a Feather Archive

Reproducibility of Computational Research: Building a Community of Practice for the AI Era


Authors: Hakizumwami Runesha (University of Chicago), Benjamin Lynch (University of Minnesota), Sandra Gesing (San Diego Supercomputing Center (SDSC))

Abstract: Reproducibility remains a fundamental challenge for computational science and AI-enabled research. Building on successful BoFs at SC22, SC24, and PEARC22-PEARC25, we established the Reproducibility of Computational Research Community of Practice, which now includes an active mailing list, public website, webinar series, and emerging working groups. This interactive BoF will engage the SC26 community in shaping the next phase of this effort by sharing community progress, launching working groups, and exploring how AI can improve the capture of reproducible computational artifacts while addressing new challenges. The session welcomes researchers, research computing professionals, software developers, librarians, and cyberinfrastructure practitioners.

Long Description: Scientific discovery depends on research that is transparent, trustworthy, and reproducible. While funding agencies increasingly require data management, software sharing, and open science practices, computational reproducibility remains difficult to achieve. Reproducing published computational results often requires much more than access to datasets or source code. Software environments, workflows, provenance, computational infrastructure, and domain-specific knowledge are frequently missing, making published results difficult or impossible to reproduce.

Although the importance of reproducibility is widely recognized, reproducibility practices are still not routinely integrated into the daily research workflow. Researchers often assemble reproducibility artifacts only after completing a project, creating additional work and reducing the likelihood that complete, reusable artifacts are ever produced. At the same time, the rapid growth of AI-assisted research introduces both new opportunities and new challenges. Large language models, agentic AI systems, and emerging provenance tools have the potential to automate documentation, workflow capture, and metadata generation, while also raising important questions about transparency, validation, and long-term stewardship.

Over the past four years, discussions at PEARC22, SC22, PEARC23, SC24, PEARC24, and PEARC25 have demonstrated sustained community interest in developing practical approaches to reproducibility. These discussions have evolved beyond conference sessions and resulted in the creation of the Reproducibility of Computational Research Community of Practice, which now includes a public website, mailing list, webinar series, and growing community of practitioners. The Community of Practice provides a forum for sharing experiences, highlighting tools, identifying best practices, and fostering collaborations across institutions.

Since its launch, the Community of Practice has organized multiple webinars and is establishing recurring activities and community working groups. Initial working groups focus on: • Webinars, workshops, and training • Tools, resources, and best practices • Repositories and reproducible artifacts

Additional information, webinar recordings, and community resources are available at https://reproducibility.rcc.uchicago.edu/.

The SC26 BOF will serve as the annual gathering of this growing community and has three primary objectives: 1. Report progress on the Community of Practice, share accomplishments, and gather community feedback to help shape future priorities. 2. Launch and expand community working groups by recruiting volunteers and identifying collaborative activities that can be sustained beyond SC26. 3. Explore the role of AI in computational reproducibility, including opportunities for automated provenance capture, workflow documentation, metadata generation, and reproducibility support.

Rather than focusing solely on existing tools, this BOF emphasizes building a sustainable community ecosystem that lowers barriers to reproducibility through shared services, institutional practices, education, and collaboration. We seek input from researchers, research computing professionals, software developers, librarians, cyberinfrastructure providers, and anyone interested in advancing reproducible computational science. The outcomes of the BOF, including discussion summaries, recommendations, and community action items, will be openly shared through the Community of Practice website, webinars, and mailing list to encourage continued engagement beyond SC26.

Website: https://reproducibility.rcc.uchicago.edu/



Back to Birds of a Feather Archive Listing