• Information For
    • Computational Molecular Sciences Community
    • Environmental System Science Community
    • Exascale Computing Community
    • Scientific Libraries Community
    • Software Engineering Community
    • Community of Supercomputer Facilities and their Users
  • Contribute to BSSw
  • Receive Our Email Digest
  • Contact BSSw
  • Information For
    Computational Molecular Sciences Community Environmental System Science Community Exascale Computing Community Scientific Libraries Community Software Engineering Community Community of Supercomputer Facilities and their Users
  • Contribute to BSSw
  • Receive Our Email Digest
  • Resources

    Better

    Planning

    • Software Process Improvement
    • Software Engineering
    • Requirements
    • Design
    • Software Interoperability

    Better

    Development

    • Documentation
    • Configuration and Builds
    • Revision Control
    • Release and Deployment
    • Issue Tracking
    • Programming Languages
    • Development Tools
    • Refactoring

    Better

    Performance

    • High-Performance Computing (HPC)
    • Performance at Leadership Computing Facilities
    • Performance Portability

    Better

    Reliability

    • Testing
    • Continuous Integration Testing
    • Reproducibility
    • Debugging

    Better

    Collaboration

    • Projects and Organizations
    • Strategies for More Effective Teams
    • Funding Sources and Programs
    • Software Publishing and Citation
    • Licensing
    • Discussion and Question Sites
    • Conferences and Workshops

    Better

    Skills

    • Online Learning
    • Personal Productivity and Sustainability

    View All Resources

    • Better Planning
    • Better Development
    • Better Performance
    • Better Reliability
    • Better Collaboration
    • Better Skills
    • View All Resources
  • Blog
  • Events
  • About
    • Team
    • Policies and Code of Conduct
    • BSSw Fellowship Program
  • Home
  • Blog
  • Building Community through Software...

Building Community through Software Policies

Share on LinkedIn Share on Facebook Tweet Copied! Permalink

PublishedAug 12, 2019
Authors Piotr Luszczek and Ulrike Meier Yang
TOPICS
Better Collaboration
Projects and Organizations

It takes a whole community to build a sustainable ecosystem consisting of various independently developed software products that work well together. Providing a set of guidelines is an essential ingredient. This article discusses community policies for long-term development of a sustainable software ecosystem for scalable high-performance libraries and applications.

Building a software community

The design of the Extreme-scale Scientific Software Development Kit (xSDK, see https://xsdk.info) has been driven by the vision to provide a collection of related but highly diverse software products that enable and support the rapid development of large, well-performing application codes. The realization that software development is a community process was the initial thought that guided the xSDK project. This approach may be partially attributed to the interdisciplinary nature of computational science in general and to the xSDK's founding team in particular. Furthermore, the race toward extreme-scale computing became a true catalyst of the xSDK effort, with its focus on achieving a productive and sustainable software environment. We contrast this endeavor with moonshot-like projects that achieve a narrowly defined goal, where any benefits beyond the specific target are purely coincidental. By bringing together community-building qualities and software-development practices, the xSDK is working to achieve longevity by following a cohesive set of principled guidelines that now benefit the broad efforts to make exascale computing a reality. The founding packages that conformed to the initial set of community policies were few. Over time, however, the number has grown and is now approaching 20, with new packages joining every year (see the figure above).

Basic design of community policies

At the onset of the xSDK project, it became clear that to achieve the seamless coordination required to build a variety of software libraries---that are actually developed independently at different institutions---required a set of guidelines. To this end, the founding representatives of the xSDK member packages put together an initial draft of xSDK community policies. This draft was a result of lengthy discussions among software experts who all shared the goal of overcoming incompatibilities among their respective software packages that hindered building and using the xSDK libraries in combination with each other. Other benefits of these consistent policies include better software quality, productivity, and sustainability.

The xSDK community guidelines (https://xsdk.info/policies) consist of two sets of policies: (1) mandatory policies that affect software quality and ensure compatibility among xSDK libraries and (2) recommended policies that affect additional software quality and sustainability topics. The recommended policies are not necessarily less important than the mandatory policies; instead, recommended policies could be initially categorized as "recommended" to provide existing members with time to achieve compatibility, and these policies could become mandatory at a later time. An xSDK-compatible package is defined as a package that satisfies all of the mandatory policies and can use or be used by at least one other xSDK library.

Allowing for gradual change

To keep the policies up to date and in line with new software developments, the xSDK policy design process allows for policy evolution. The community at large can propose amendments to existing policies as well as suggest new policies. Proposals are then considered and voted on by the community at large and members with existing xSDK packages. The procedure inherently provides a balance between preserving compatibility and gradual obsolescence, as well as between software engineering ideals and what is actually feasible. The latter leads to recommended policies, which---again---are not necessarily less important than mandatory policies but provide some members more time to achieve compatibility before becoming required as policy.

Community of communities

Any software effort will at some point involve interactions with users, and the interactions will lead to a number of known outcomes, including various bugs that range in severity as well as issues with the software features themselves, where they can be too advanced or insufficient for the intended purpose. Similar effects may be observed when the community-defining policies are introduced to a wider audience. Hence, feedback from the participating teams becomes an important aspect for maintaining the relevance of the policies.

For that reason, the xSDK core development team reaches out to other groups to seek additional input and commentary. An example of such outreach is the involvement of application developers and staff members from leadership-class computer facilities. Their feedback sometimes falls outside the prevailing views of the library developers and has a potential for contributing new ideas and out-of-the-box thinking. After discussion, some of this new feedback finds its way into the new community policies' release, leading to an overall improved set of policies.

The xSDK team always welcomes feedback, which can be provided at https://github.com/xsdk-project/xsdk-community-policies.

Author bios

Ulrike Meier Yang leads the Mathematical Algorithms & Computing Group in the Center for Applied Scientific Computing of Lawrence Livermore National Laboratory. She leads the xSDK project in DOE's Exascale Computing Project and the Linear Solvers topical area in the SciDAC FASTMath Institute; she is a developer of the software library hypre. She earned her Ph.D. in computer science from the University of Illinois at Urbana-Champaign. Her research interests are numerical algorithms, parallel computing, and scientific software design.

Piotr Luszczek is a research assistant professor at the Innovative Computing Laboratory in the University of Tennessee. Piotr earned his Ph.D. in computer science from the University of Tennessee, Knoxville. His research interests include benchmarking, numerical linear algebra for high-performance computing, automatic performance tuning, and stochastic performance models. He has over a decade of experience developing HPC numerical software for large-scale, distributed-memory multicore systems with hardware accelerators. Piotr serves as a co-PI on the ECP xSDK project that aims to improve access to world-class software on exascale machines.

Comment

More on Projects and Organizations

NSF-Sponsored 2021 BSSw Fellows

Published Mar 16, 2021

By Hai Ah Nam

Better Scientific Software: 2020 Highlights

Published Jan 07, 2021

By Rinku Gupta

Introducing the 2021 BSSw Fellows

Published Dec 16, 2020

By Hai Ah Nam

The RSE-HPC-2020 Workshop: Creating Community, Building Careers, Addressing Challenges

Published Oct 26, 2020

By Charles Ferenbaugh, Sandra Gesing, Simon Hettrick, and Daniel S. Katz

Applications Open for the 2021 BSSw Fellowship Program

Published Aug 19, 2020

By Hai Ah Nam

2019 BSSw Fellows Guide Developers through Each Stage of the Scientific Software Lifecycle

Published Jul 17, 2020

By Hai Ah Nam, Rene Gassmoeller, Ignacio Laguna, Tanu Malik, and Kyle Niemeyer

Working Remotely: The Spack Team

Published May 16, 2020

By Todd Gamblin and Elaine M. Raybourn

Research Software Engineer Stories

Published Apr 24, 2020

By Vanessasaurus

Scientific Software Projects and Their Communities

Published Mar 23, 2020

By Rene Gassmoeller

Spreading Ideas about Better Scientific Software

Published Feb 28, 2020

By David E. Bernholdt

US Research Software Engineer (US-RSE) Association

Published Jan 15, 2020

By Ian Cosden, Chris Hill, Sandra Gesing, and Charles Ferenbaugh

Better Scientific Software: 2019 Highlights

Published Jan 03, 2020

By Rinku Gupta

Introducing the 2020 BSSw Fellows

Published Dec 13, 2019

By Hai Ah Nam

Software Sustainability in the Molecular Sciences

Published Nov 14, 2019

By Theresa Windus and T. Daniel Crawford

Research Software Science: A Scientific Approach to Understanding and Improving How We Develop and Use Software for Research

Published Sep 25, 2019

By Mike Heroux

Data-driven Software Sustainability

Published Sep 18, 2019

By Daniel S. Katz

Applications Open for 2020 BSSw Fellowship Program ... Q&A Webinar on Sept 20, 2019

Published Sep 04, 2019

By Hai Ah Nam

2018 BSSw Fellows Tackle Scientific Productivity Challenges

Published Mar 28, 2019

By Hai Ah Nam

Accelerating Scientific Discovery with Reusable Software: Special issue of IEEE CiSE

Published Mar 19, 2019

By Scott Lathrop

Better Scientific Software: 2018 Highlights

Published Jan 04, 2019

By Lois Curfman McInnes, David E. Bernholdt, and Mike Heroux

Introducing the 2019 BSSw Fellows

Published Dec 11, 2018

By David E. Bernholdt, Mike Heroux, and Lois Curfman McInnes

SC18: Does That Stand for “Software Conference”?

Published Nov 08, 2018

By David E. Bernholdt

Building Connections and Community within an Institution

Published Oct 26, 2018

By Greg Watson and Elsa Gonsiorowski, PhD

Applications Open for 2019 BSSw Fellowship Program ... Q&A Webinar on Sept 21, 2018

Published Sep 10, 2018

By David E. Bernholdt, Mike Heroux, and Lois Curfman McInnes

URSSI: Conceptualizing a US Research Software Sustainability Institute

Published Jul 30, 2018

By Daniel S. Katz, Jeff Carver, Sandra Gesing, Karthik Ram, and Nic

Research Software Engineer: A New Career Track?

Published Jun 14, 2018

By Chris Richardson

Scaling Small Teams to a Team of Teams: Shared Consciousness

Published Apr 17, 2018

By Elaine M. Raybourn and David Moulton

BSSw Fellowship Activity: Promoting Software Citation

Published Mar 13, 2018

By Daniel S. Katz

Call for Papers ... Accelerating Scientific Discovery with Reusable Software

Published Feb 26, 2018

By Scott Lathrop

Introducing the 2018 BSSw Fellows

Published Feb 05, 2018

By David E. Bernholdt, Mike Heroux, and Lois Curfman McInnes

New FAQ List for BSSw Fellowship Program ... Applications Due by Jan 5, 2018

Published Dec 13, 2017

By Mike Heroux and Lois Curfman McInnes

Applications Open for New BSSw Fellowship Program ... Q&A Webinar on Dec 12, 2017

Published Dec 01, 2017

By Mike Heroux and Lois Curfman McInnes

BSSw Site Launch at SC17 ... Contribute to Better Scientific Software!

Published Nov 13, 2017

By David E. Bernholdt, Mike Heroux, and Lois Curfman McInnes

Pending BSSw Site Launch at SC17

Published Sep 27, 2017

By Lois Curfman McInnes

logo Better Scientific Software
  • BSSw Fellowship Program
  • Policies
  • Site Contributors
  • Contact BSSw
  • Receive Our Email Digest
  • Follow Our RSS Feed

Copyright © 2021 Better Scientific Software under MIT License

United States Department of Energy Office of Science National Nuclear Security Administration Exascale Computing Project