Software Sustainability Matrix

The Software Sustainability Matrix (SSM) is a joint effort of the Scientific Computing and Imaging (SCI) Institute and Kitware to improve the sustainability of the open-source package ecosystem making up the foundational software portfolio for the Center for Integrative Biomedical Computing (CIBC) led by the SCI for the National Institutes of Health (NIH). ^1,2 The goal of the SSM effort is to create numerical scores for various aspects of software sustainability for a software package as well a combined sustainability score for the package. This SSM process is being used to guide efforts to improve the sustainability of the CIBC software package ecosystem, with the hope to transition the continuing support of this software to the open-source community.¹ However, the SSM approach may be applicable to other package ecosystems and therefore may be of more general interest.

The SSM score is composed from component scores in four areas: Impact, Risks, Community, and Technology, as shown in Figure 1, and each area is broken down into several subareas.

Scores in the range [0-100] in these four areas (Impact, Risks, Community, and Technology) are then combined in a weighted sum:

    SSM Score = F^I * I + F^R * R + F^C * C + F^T * T

to produce a single SSM Score [0-100], where F^I + F^R + F^C + F^T = 1. The SCI & Kitware process³ typically uses the weighting factors F^I = F^C = 1/3 and F^R = F^T = 1/6. It is not specified how the SCI & Kitware program computes the component scores in the four areas 'I', 'R', 'C', and 'T' for the software packages of interest, and the authors state that computing these individual component scores is ultimately a subjective matter.

It is stated by the authors that the primary benefit of estimating the SSM score by a software package team is not in computing the final score, but instead in highlighting areas of possible improvement. It is claimed that many teams make significant improvements in the sustainability for their packages while working through the process of estimating the SSM score by adopting better processes in different areas. In this respect, the SSM impact is similar to the impact of the OpenSSF Best Practices Badge Program.⁴ That is, by having a team score each suggested practice, the team will often implement many suggested best practices, especially for practices that are not too difficult to adopt and support.

In summary, the Software Sustainability Matrix (SSM) is a promising approach to help quantify the sustainability of individual software packages (and collectively entire software package ecosystems) and subsequently to drive efforts to make targeted improvements for sustainability.

References

• ¹The Scientific Computing and Imaging (SCI) Institute and Kitware: Putting Software Sustainability into Practice.
  Will Schroeder, Jess Tate, Jean-Christophe Fillion-Robin, Christopher Johnson, Dženan Zukić, Matt McCormick, Lee A. Newberg, Daniel White, Alexandra Warner, Ross Whitaker and Rob MacLeod. Kitware Blog, January 22, 2021
• ²SCI Institute and Kitware: Putting Software Sustainability into Practice.
  Will Schroeder, Jess Tate, Jean-Christophe Fillion-Robin, Christopher Johnson, Dženan Zukić, Matt McCormick, Lee A. Newberg, Daniel White, Alexandra Warner, Ross Whitaker and Rob MacLeod. Center for Integrative Biomedical Computing Blog
• ³Scoring Software Sustainability.
  Will Schroeder, Matt McCormick and Jean-Christophe Fillion-Robin. Kitware Blog, May 8, 2021
• ⁴OpenSSF Best Practices Badge Program