• Information For
    • Computational Molecular Sciences Community
    • Environmental System Science Community
    • Exascale Computing Community
    • Scientific Libraries Community
    • Software Engineering Community
    • Supercomputer Facilities and their Users
  • Contribute to BSSw
  • Receive Our Email Digest
  • Contact BSSw
  • Information For
    Computational Molecular Sciences Community Environmental System Science Community Exascale Computing Community Scientific Libraries Community Software Engineering Community Supercomputer Facilities and their Users
  • Contribute to BSSw
  • Receive Our Email Digest
  • Resources

    Better

    Planning

    • Software Process Improvement
    • Software Engineering
    • Requirements
    • Design
    • Software Interoperability
    • Software Sustainability

    Better

    Development

    • Documentation
    • Configuration and Builds
    • Revision Control
    • Release and Deployment
    • Issue Tracking
    • Programming Languages
    • Development Tools
    • Refactoring

    Better

    Performance

    • High-Performance Computing (HPC)
    • Performance at Leadership Computing Facilities
    • Performance Portability
    • Cloud Computing
    • Big Data

    Better

    Reliability

    • Peer Code Review
    • Testing
    • Continuous Integration Testing
    • Reproducibility
    • Debugging

    Better

    Collaboration

    • Projects and Organizations
    • Strategies for More Effective Teams
    • Inclusivity
    • Funding Sources and Programs
    • Software Publishing and Citation
    • Licensing
    • Discussion and Question Sites
    • Conferences and Workshops

    Better

    Skills

    • Online Learning
    • In-Person Learning
    • Personal Productivity and Sustainability

    View All Resources

    • Better Planning
    • Better Development
    • Better Performance
    • Better Reliability
    • Better Collaboration
    • Better Skills
    • View All Resources
  • Blog
  • Events
  • About
    • Site Overview
    • Team
    • Policies and Code of Conduct
    • BSSw Fellowship Program
  • Home
  • Blog
  • Improving Team Practices with RateYourProject.org

Improving Team Practices with RateYourProject.org

Share on LinkedIn Share on Facebook Tweet Copied! Permalink

PublishedJul 15, 2021
Author Gregory Watson
TOPICS
Better Planning
Software Engineering

RateYourProject.org is a tool designed to help teams begin the process of Productivity and Sustainability Improvement Planning (PSIP), a new technique for improving team practices. It was developed by the PSIP team, which is part of the IDEAS-ECP project.

PSIP is a lightweight methodology for improving practices used by scientific software teams. Originally envisioned for helping teams improve software engineering practices such as coding standards, testing, documenting, etc., the PSIP approach has proved general enough for other non-technical types of practices as well, such as on-boarding, communications, etc.



The core of the PSIP process comprises six steps of an iterative process. The first two steps form the self-assessment phase, where teams identify their current practices and areas that would benefit from improvement, then set goals for how to improve those practices. The next two steps form the planning phase, where teams create progress tracking cards (PTCs) to record and monitor progress towards their goals, record the current state, and create a plan for how to improve the practices. The final two steps form the execution phase, an iterative process where the plans are put into action, and progress is monitored by recording it in the PTCs as part of the team's normal task tracking activities.

Getting started

Like any new idea, convincing people that it is practical and worthwhile is one thing, but actually getting them to try it out is a whole different ball game. This is particularly an issue when there are time and cost implications in trying out the idea, and if the benefits of the activity are not immediately apparent. The diagram below (courtesy of Scott Weigle) indicates these challenges.



For PSIP, the main barrier to getting started is assessing the current project practices and identifying areas that would benefit from improvement. RateYourProject.org provides a simple and fast way to get teams on the road to improving their practices.

How it works

RateYourProject.org is a self assessment tool that works by guiding the user through a series of common development practices and allows the user to choose a response that most closely matches their current practice level using a scale of "None", "Basic", "Intermediate", and "Advanced". The scale is designed to make it easy for teams to match their practices with the practice description. Practices are grouped under the categories Better Development, Better Planning, Better Performance, Better Reliability, and Better Collaboration. This enables an overall assessment to be presented as a spider chart like this one, which clearly shows that Better Planning and Better Performance are categories that could benefit most from improvement, although there is room for improvement in all categories.


Overall assessment is presented with a spider chart, which shows there is room for improvement in all categories and makes clear the categories that would benefit most from improvement.


Once the overall assessment has been established, teams are able to examine the individual categories to determine what practices they could focus on for improvement. In this example, selecting the Better Planning category will then provide a more detailed assessment of the practice areas. In this case, the project could clearly benefit from creating onboarding and offboarding processes to manage staff turnover, and could also benefit from looking at how to improve contribution management, requirements analysis, and software design practices.


The tool guides users through a series of common development practices, allowing them to choose a response that most closely matches their current practice level.


Continuous improvement

PSIP is a process of continuous improvement: practices needing improvement are identified, a plan is made, and then the plan is executed and the process starts again. The RateYourProject.org tool allows the current assessment to be saved, then reviewed later in order to check progress or to identify another area that could be improved. The site provides a link on the main assessment page that can be used to return to the current assessment. This provides a simple and safe mechanism for recording progress towards achieving improvement goals.

Feedback and future work

RateYourProject.org has been used to self assess project practices by a number of groups, including The HDF Group, Sandia National Laboratories, and Oak Ridge National Laboratory, as well as in tutorials that the PSIP team has presented. Feedback from users has been positive, pointing to its ease of use and the clear benefits of having a focal point for multiple team members to quickly provide their perspective when collecting input on how to improve project practices.

The PSIP team is working on a variety of improvements and enhancements to the tool. These include helping teams create progress tracking cards once they have identified practice areas for improvement, and then adding these cards to the existing task tracking systems used by the teams. We're also looking at how the tool could be integrated directly with version control hosting services such as GitHub and GitLab, more detailed assessments, and alternate presentation styles. Additional information about the tool and other PSIP related information can be found on the PSIP web page or using the PSIP Practice Guides.

Further reading

M.A. Heroux et al. (2020) Lightweight Software Process Improvement Using Productivity and Sustainability Improvement Planning (PSIP). In: Juckeland G., Chandrasekaran S. (eds.) Tools and Techniques for High Performance Computing. HUST 2019, SE-HER 2019, WIHPC 2019. Communications in Computer and Information Science, vol 1190. Springer, Cham. https://doi.org/10.1007/978-3-030-44728-1_6.

E. Pourmal, R. Milewicz and E. Gonsiorowski, Using the PSIP Toolkit to Achieve Your Goals – A Case Study at The HDF Group, webinar, June 2021, (slides, recording).

Author bio

Gregory Watson is Group Leader of the Application Engineering Group at Oak Ridge National Laboratory. He completed his Ph.D. in Computer Science in 2000 from Monash University. Greg's research interests include software engineering practices, development environments, programming tools, and modeling and simulation tools for high performance and scientific computing. He is founder of the Eclipse Parallel Tools Platform, and project leader of the Eclipse Science Top Level Project.

Comment

More on Software Engineering

Recap: Encouraging and Enabling Mutual Ownership in a RSE Community of Practice

Published Dec 09, 2022

By Miranda Mundt, Jonathan Bisila, Jon Guyer, Daniel Howard, Daniel S. Katz, Reed Milewicz, Henry Schreiner, Joshua Teves, and Chris Wiswell

My Reading List for Research Software Engineers

Published Oct 26, 2022

By Chase Million

Rethinking Software Variants

Published Sep 29, 2022

By Axel Huebl

Trusted Cyberinfrastructure Evaluation, Guidance, and Programs for Assurance of Scientific Software

Published May 13, 2022

By Elisa Heymann Pignolo, Barton Miller, and Sean Peisert

Reflecting on Our Community: The SC21 BoF on Software Engineering and Reuse in Modeling, Simulation, and Data Analytics for Science and Engineering

Published Dec 21, 2021

By David E. Bernholdt, Michael Bader, Michelle Barker, Ben Brown, Anshu Dubey, Nasir Eisty, Sandra Gesing, Patricia Grubel, Rinku Gupta, Michael A. Heroux, Saswata Hier-Majumder, Axel Huebl, Mozhgan Kabiri Chimeh, Daniel S. Katz, Tomislav Maric, Lois Curfman McInnes, Bill Miller, Manish Parashar, Ulf D. Schiller, Jean Sexton, Peter Vaillancourt, Marion Weinzierl, and Yo Yehudi

logo Better Scientific Software
  • BSSw Fellowship Program
  • Policies
  • Site Contributors
  • Contact BSSw
  • Receive Our Email Digest
  • Follow Our RSS Feed

Copyright © 2023 Better Scientific Software under MIT License

United States Department of Energy Office of Science National Nuclear Security Administration Exascale Computing Project