• Information For
    • Computational Molecular Sciences Community
    • Environmental System Science Community
    • Exascale Computing Community
    • Scientific Libraries Community
    • Software Engineering Community
    • Supercomputer Facilities and their Users
  • Contribute to BSSw
  • Receive Our Email Digest
  • Contact BSSw
  • Information For
    Computational Molecular Sciences Community Environmental System Science Community Exascale Computing Community Scientific Libraries Community Software Engineering Community Supercomputer Facilities and their Users
  • Contribute to BSSw
  • Receive Our Email Digest
  • Resources

    Better

    Planning

    • Software Process Improvement
    • Software Engineering
    • Requirements
    • Design
    • Software Interoperability
    • Software Sustainability

    Better

    Development

    • Documentation
    • Configuration And Builds
    • Revision Control
    • Release And Deployment
    • Issue Tracking
    • Programming Languages
    • Development Tools
    • Refactoring

    Better

    Performance

    • High Performance Computing (Hpc)
    • Performance At Leadership Computing Facilities
    • Performance Portability
    • Cloud Computing
    • Big Data

    Better

    Reliability

    • Peer Code Review
    • Testing
    • Continuous Integration Testing
    • Reproducibility
    • Debugging

    Better

    Collaboration

    • Projects And Organizations
    • Strategies For More Effective Teams
    • Inclusivity
    • Funding Sources And Programs
    • Software Publishing And Citation
    • Licensing
    • Discussion And Question Sites
    • Conferences And Workshops
    • Research Software Engineers

    Better

    Skills

    • Online Learning
    • In Person Learning
    • Personal Productivity And Sustainability

    View All Resources

    • Better Planning
    • Better Development
    • Better Performance
    • Better Reliability
    • Better Collaboration
    • Better Skills
    • View All Resources
  • Blog
  • Events
  • About
    • Site Overview
    • Team
    • Policies and Code of Conduct
    • BSSw Fellowship Program
  • Home
  • Blog
  • Improve User Confidence in Your Software...

Improve User Confidence in Your Software Updates

Share on LinkedIn Share on Facebook Tweet Copied! Permalink

Puppy at work

PublishedJul 14, 2017
Author Mike Heroux
TOPICS
Better Reliability
Testing
Reproducibility
Better Development
Documentation
Track Deep Dive

Informing users about what has changed is key to instilling confidence.

When customers update to a new version of your software, the changes they will encounter are not just about new features: often (perhaps mostly) such changes include improvements to existing capabilities.

Customers who are integrating your latest version are looking for changes in behavior. Such changes include timing differences and changes in input requirements and output data. In HPC software, changes in output can be common, especially with floating-point computations where difference in the order of operations can produce correct but different results.

In these situations, customers don’t necessarily mind that results have changed, but they want to know that the change is expected, not the result of a regression.

You can improve customer confidence in your updates by considering the following:

  • Create an issue in your database (e.g., a GitHub or JIRA issue) for the feature, and give it a label indicating that the feature may change software behavior from the user’s perspective.
  • Notify known users of the change prior to release.
  • Document any changes that result in different behavior from your software.
  • Describe in release notes what kind of behavior change can be expected.
  • Provide users with an option to restore previous behavior (e.g., via a runtime or compile time parameter).
  • Include performance differences, even if the changes are improvements.

Sources for behavior change may include the following:

  • Performance optimizations for vectorization: Vectorization represents one of the current commodity performance improvement curves. As a resource for concurrency, we continue to increase the number of simultaneous operations a process can perform (as either SIMD or SIMT). Introducing vector operations into your code, directly or through compiler transformations, will lead to differences in floating-point results, including differences from one architecture to the next.
  • Reordering of irregular (gather/scatter) computations for better performance: Changes in the order of irregular computations can improve cache utilization and reduce memory bandwidth requirements, leading to better performance. These changes also lead to differences in floating-point results.
  • Changes in heuristics for automatic parameter settings: Many algorithms are tunable, able to exploit problem details to improve robustness, reliability, or performance. Automatic parameter setting can improve software usability by reducing how many details the user needs to explicitly manage. Improved heuristics, often derived from customer use, can lead to changes in behavior, even though the change is an improvement.

Comment

More on Testing, Reproducibility, and Documentation

Julia's Value Proposition for Better Scientific Software

Deep Dive

Published Apr 14, 2023

By William F Godoy

ACM REP '23: ACM Conference on Reproducibility and Replicability, Santa Cruz, CA, USA, June 27-29, 2023

Community

Published Jun 05, 2023

By Carlos Maltzahn and Philippe Bonnet

Experiences Replacing Master/Slave Terminology in ALE3D and Sierra

Experience

Published Mar 27, 2022

By Mark C. Miller

Rethinking Software Variants

Deep Dive

Published Sep 29, 2022

By Axel Huebl

Celebrating the Fifth Anniversary of the Correctness Workshop: Looking Back and Looking Forward

Community

Published Feb 22, 2022

By Ignacio Laguna and Cindy Rubio-González

logo Better Scientific Software
  • BSSw Fellowship Program
  • Policies
  • Site Contributors
  • Contact BSSw
  • Receive Our Email Digest
  • Follow Our RSS Feed

Copyright © 2023 Better Scientific Software under MIT License

United States Department of Energy Office of Science National Nuclear Security Administration Exascale Computing Project